Chapter 23 — Attention explained with a highlighter

YOUR QUESTION      -->  the QUERY      "what am I looking for?"
EACH PARAGRAPH'S   -->  the KEY        "what is this paragraph about?"
  TOPIC LABEL
THE PARAGRAPH'S    -->  the VALUE      "the actual content I'd take away"
  CONTENT

Process:
  1. compare your QUERY to every KEY        -> a relevance score per paragraph
  2. turn the scores into percentages       -> "how much highlighter on each"
  3. take a weighted blend of the VALUES    -> your answer

The database analogy (if you prefer)

A Python dictionary is a hard lookup:

table = {"cat": "a small furry animal", "dog": "a loyal pet"}
table["cat"]        # exact match, get exactly one value

Attention is the soft version:

soft_lookup(query) = sum over all entries of:
                        (how well query matches this key) x (this entry's value)

Instead of picking one entry, you get a weighted blend of all of them, where the weights come from similarity. And because it's a smooth blend rather than a hard pick, it is differentiable — which means it can be learned by gradient descent.

In a Transformer, every word does this simultaneously

The crucial move: it's self-attention. Each word in the sentence produces its own query, and looks at the keys and values of every word (including itself).

Sentence:  "the  animal  didn't  cross  ...  because  it  was  tired"
                                                      ^^
                                          this word's query is roughly
                                          "I'm a pronoun; what noun am I?"

Its attention weights might come out as:
   the: 0.02   animal: 0.51   didn't: 0.03   cross: 0.05   ...   tired: 0.21

So the new representation of "it" becomes mostly "animal" plus a bit of "tired".
The word "it" has been ENRICHED with what it refers to.

Do that in every layer, 32 times, and representations become deeply context-aware.

Where do queries, keys, and values come from?

Each word starts as an embedding vector x. Three learned matrices project it into three different roles:

   q = x @ W_Q       "what I'm looking for"
   k = x @ W_K       "what I offer to others"
   v = x @ W_V       "what I'll contribute if attended to"

W_Q, W_K, W_V are ordinary weight matrices — knobs, learned by gradient descent like everything else. Nobody tells the model what queries should mean. It discovers, through training, that it's useful for pronouns to query for nouns.