Before Transformer models, the attention mechanism was proposed as a helper for improving conventional DL models such as RNNs.