Research note / 2026
AI / TRANSFORMERS
Attention is
all you need.
My visual notes on the paper that introduced the Transformer: a model that learns which parts of information matter most to one another.
The big idea
Look at every word, then decide what matters.
When reading a sentence, a Transformer does not have to move through it one word at a time. Self-attention lets each word compare itself with every other word and assign more weight to the useful connections.
For example, in “The animal didn’t cross the street because it was too tired,” attention can connect “it” strongly to “animal.”
01 / SELF-ATTENTION
Every token can look
at every other token.
Each connection receives a score. Stronger scores mean the model pays more attention there.
Queries, keys & values
Three ways to describe a word.
For each token, the model creates three learned vectors: a query asks what it is looking for, a key says what it offers, and a value carries its information. Comparing queries with keys creates the attention scores.
02 / Q · K · V
Match, weigh,
then combine.
03 / MULTI-HEAD ATTENTION
More than one
way to look.
Different attention heads can learn different patterns at the same time: grammar, position, meaning, or something we have not named yet.
grammar
meaning
position
context
My takeaway