LLMs in Plain English

SIMPLE PHYSICS ONE ANSWER

Text
  "Explain transformer in simple words"

  ↓ tokenizer

Tokens
  ["Explain", "transformer", "in simple", "words"]

  ↓ lookup in the embeddings table

Vectors
  e1, e2, e3, e4

  ↓ + position signals

Transformer blocks × N
  attention → MLP → attention → MLP → ...

  ↓ take the state of the last position

LM head
  logits over the entire dictionary

  ↓ softmax/sampling

Next token
  "."

  ↓ this token is added to the input

The cycle repeats

text → tokens → ids → vectors → contextual vectors → next-token scores → chosen token

Search the database:
query → index → found document → show to user

LLM:
prompt → forward pass network → logits by dictionary → token

Important:
the model doesn't know the "correct paragraph"
the model knows "which continuation is similar to the truth"

WHAT DO WEIGHTS STORE?

Not so:
  weight_9281 = "Paris is the capital of France"

More like this:
  many scales together strengthen patterns
  "after 'the capital of France' often comes 'Paris'"
  "in SQL, after SELECT there is often a list of columns"
  "the pronoun 'she' is often associated with the nearest female subject"

Knowledge is distributed, and does not lie piecemeal

Search the database:
query → index → found document → show to user

LLM:
prompt → forward pass network → logits by dictionary → token

Important:
the model doesn't know the "correct paragraph"
the model knows "which continuation is similar to the truth"

WHAT DO WEIGHTS STORE?

Not so:
  weight_9281 = "Paris is the capital of France"

More like this:
  many scales together strengthen patterns
  "after 'the capital of France' often comes 'Paris'"
  "in SQL, after SELECT there is often a list of columns"
  "the pronoun 'she' is often associated with the nearest female subject"

Knowledge is distributed, and does not lie piecemeal

EXAMPLE OF TOKENIZATION

Text:
  "Neural networks explain text"

Possible parsing:
  ["Neuro", "networks", "explain", "nyayut", "text"]

Other text:
  "transformers are useful"

Possible parsing:
  ["transform", "ers", "are", "useful"]

The same symbolic world is cut into pieces of a dictionary
There are no letters inside the model, there are ID tokens

"Neural networks explain text"
→ tokenizer
→ [4811, 932, 18452, 77, 901]
→ then the network works only with numbers

vocab_size = 128000
d_model = 4096

Embedding matrix E ∈ R^[128000 × 4096]

token_id = 4811
vector = E[4811]

This is not a calculation "by meaning"
This is a lookup: we took a ready-made table row

WHY DO YOU NEED A POSITION

Phrase A:
  "The dog bit the man"

Phrase B:
  "a man was bitten by a dog"

Same words, but different meaning.

Without position:
  {dog, bitten, human} = almost the same thing

With position:
  token_vector + position_signal
  → the model knows who is earlier and who is later

ONE WORD, TWO CONTEXTS

Before the context:
  "bank" → e_bank

After blocks in a sentence:
  "I put the money in the bank"
  → h_bank_money

After blocks in another sentence:
  "the boat sailed to the river bank and the bank"
  → h_bank_river

The meaning changes not in the token dictionary, but inside hidden states

PROBLEM WITHOUT ATTENTION

"Marina came home because she was tired"

To understand "she", it is useful to see "Marina".

Weak scheme:
  token → token → token → token → ... → "she"
  long-distance communication is blurred

With attention:
  "she" ───────────────▶ "Marina"
  the necessary connection is made direct

for token i:
q_i = W_Q x_i
k_i = W_K x_i
v_i = W_V x_i

score(i, j) = q_i · k_j
weights(i, :) = softmax(scores)
new_x_i = Σ_j weights(i, j) v_j

That is, token i collects a weighted mixture of value from other tokens

INTUITION BY EXAMPLE

Phrase:
  "Marina came home because she was tired"

The "she" token builds the query:
  What kind of subject do you mean?

Other tokens offer keys:
  "Marina" → fits strongly
  "home" → fits weakly
  “tired” → helps in meaning, but not as a subject

After softmax:
  weight("Marina") = 0.62
  weight("home") = 0.05
  weight("tired") = 0.18
  ...

New vector "she" receives a lot of information from "Marina"

attention:
  “she” realized that she was connected with “Marina”

MLP:
  reworked this context
  enhanced beneficial symptoms
  reduced the noise
  prepared a presentation for the next block

Attention = gather connections
MLP = transform collected

ANATOMY OF ONE BLOCK

x
│
├─ self-attention(x) = a
│
├─ x + a ← residual
│
├─layernorm(...)
│
├─MLP(...)
│
├─ + previous path ← residual again
│
└─ layernorm(...) → block output

The block does not write everything from scratch, but improves the current state

ENCODER-ONLY

Login:
  [token1] [token2] [token3] [token4]

Attention:
  everyone sees everyone

Output:
  h1 h2 h3 h4

Next:
  or take one common vector
  or read the label for each token

Strength: Understanding the entire input

DECODER-ONLY

Position 1 sees: [1]
Position 2 sees: [1,2]
Position 3 sees: [1,2,3]
Position 4 sees: [1,2,3,4]

You can't:
  token 2 look at token 4

You can:
  at each step predict token 5

GPT, Llama, Claude-like models are decoder-only

ENCODER-DECODER

Source text
  "Translate: The cat is sleeping"
        │
        ▼
  ENCODER reads the whole thing
        │
        ▼
  h1 h2 h3 h4 ... hN
        │
        ▼
  DECODER writes:
  "Cat"
  "The cat is sleeping"
  ...

At each step, the decoder has two sources:
  1. already written part of the answer
  2. the entire input through cross-attention

Simple: reader + writer in one system

EXAMPLE OF TEACHING WITH A PHRASE

Text:
  "The cat is sitting on the carpet"

The model is taught in many steps at once:
  "Cat" → predict "sits"
  "The cat is sitting" → predict "on"
  "The cat is sitting on" → predict "carpet"

During training:
  the model sees the correct prefix
  not your own past answer

From millions of such chains “language knowledge” is born

for each position:
  logits → softmax → P(next token)
  compare with the correct token
  calculate loss
  adjust the weights a little

repeat this on billions of examples

SIMPLE MLM EXAMPLE

Original:
  "The cat is sitting on a warm carpet"

Broken input:
  "Cat [MASK] on warm [MASK]"

Task:
  guess "sits" and "carpet"

To fill in the blank, you need to understand the entire context around

1. the model predicted the probabilities
2. saw the correct token
3. got the error value
4. calculated which weights increased the error
5. moved them a little in the opposite direction

Training = billions of micro-weight shifts

prompt = system + history + user_text

tokenize(prompt)
embed(prompt_tokens)
pass all tokens through all blocks
get hidden states for all positions

This is where the understanding of the context before the first word of the answer comes from

AUTO REGRESSION

Context:
  "Explain attention"

Step 1:
  the model selects "simple"

Step 2:
  context = "Explain attention in simple terms"
  the model chooses "in words"

Step 3:
  context = "Explain attention in simple words"
  the model selects "."

The answer is built from left to right, and not a whole paragraph at once

NUMBERS IN THE LAST STEP

Last hidden state
        │
        ▼
LM head
        │
        ▼
logits:
  "yes" = 8.2
  "no" = 6.9
  "maybe" = 6.1
  "," = 4.4

softmax →
  P("yes") = 0.63
  P("no") = 0.17
  P("possible") = 0.08
  P(",") = 0.01

temperature ↓
  T = 0.2 → distribution is sharper, almost always “yes”
  T = 1.0 → balance between certainty and variability
  T = 1.5 → more rare variants

little context
or
there are too many similar patterns
or
required fact rare / fresh / controversial

→ the model selects a statistically plausible continuation
→ not a verified fact

WEIGHTS vs CONTEXT

Weights:
  long-term model parameters
  change only by training/fine-tuning

Prompt:
  temporary input of the current request
  affects only through attention in this launch

Result:
  prompt = directs
  fine-tuning = rebuilds the model's habits