Simplifying Transformers: On the Power of ‘Attention’ in Natural Language Processing- AN INTUITION

In this article, we explore the mechanics of attention, encoder-decoder dynamics, and positional encodings, unraveling the essence of how these elements unite to elevate the capabilities of language models.

Rahul S
5 min readAug 17


Transformers break the text into parts, use attention to focus on important information, and generate meaningful output using the encoder and decoder.

Imagine you’re part of a study group, and you need to work together to understand a complex article. Instead of discussing it out loud, you decide to write down your thoughts on a shared document. This way, everyone can contribute their ideas and see what others have written.

Transformers work in a similar way. They are powerful because
1) they can handle long and complex text,
2) capture subtle meanings, and
3) generate coherent responses.

They can be used for various tasks like translation, answering questions, summarization, and more.

They have two main parts: the encoder and the decoder. The encoder helps the model understand the input text, and the decoder generates the output or response based on that understanding.

ENCODER: The encoder is responsible for reading the input text, understanding the words, and capturing their meaning. It does this by dividing the input text into smaller parts called “tokens.” Tokens can be words, phrases, or even individual characters, depending on how the model is designed. Each token is represented by a unique numerical value, called a “vector,” which the computer can work with.

ATTENTION: Just like passing around a shared document, transformers use something called “attention” to pay attention to different parts of the text. It’s like when your study group members focus on different ideas in the shared document.

Attention helps the transformer understand the relationships between words…



Attention in Transformers

2 min read

Nov 27

Deep Learning: Importance of Data Normalization

3 min read

Oct 8

Deep Learning: What Makes Transformers So Effective?

2 min read

Oct 6

Deep Learning: Guidelines for model optimization and tuning

10 min read

Dec 1, 2022

Deep learning: A non-mathematical intuition of how a neural network learns

5 min read

Nov 30, 2022

Deep Learning: GELU (Gaussian Error Linear Unit) Activation Function

2 min read

Aug 24

Unlocking Artistic Magic: Decoding Neural Style Transfer (NST)

4 min read

Aug 11

Deep Learning: Internal Covariate Shift & Batch Normalization

3 min read

Aug 23

Deep Learning: Activation Functions — 10 Tricky questions

4 min read

Aug 17

Deep Learning: Impact of Gradient Descent Optimization Algorithm during Training

1 min read

Apr 20

Rahul S

I learn as I write | LLM, NLP, Statistics, ML

Recommended from Medium


See more recommendations