Visual notes / 2026
COMPUTER SCIENCE / TRANSFORMERS
Attention is
all you need.
I made these notes to understand the Transformer by taking it apart visually, one idea at a time.
The big idea
Compare every word, then keep the useful connections.
A Transformer does not need to read a sentence only one word at a time. Self-attention lets each token compare itself with the others and put more weight on the connections that help.
In “The animal didn’t cross the street because it was too tired,” attention can connect “it” strongly to “animal.” That visible link is the idea I wanted to understand.
01 / SELF-ATTENTION
Every token can look
at every other token.
Each connection receives a score. Stronger scores mean the model pays more attention there.
Queries, keys & values
Three jobs for the same token.
For each token, the model creates three learned vectors. A query asks what it needs, a key describes what it can match, and a value carries the information that may be used. Comparing queries with keys produces the attention scores.
02 / Q · K · V
Match. Weigh.
Combine.
03 / MULTI-HEAD ATTENTION
Look for several
patterns at once.
Different attention heads can learn different patterns at the same time: grammar, position, meaning, or relationships we have not named yet.
grammar
meaning
position
context
My takeaway