LLMs in Plain English

If you take away all the marketing, there is a repetitive computational chain going on inside the LLM. The text is divided into tokens, the tokens are turned into vectors, the stack of transformer blocks recalculates these vectors many times, after which the model evaluates the probabilities from the dictionary and selects the next token.

The model does not open internal Wikipedia and pull out a paragraph from it. Each time, it re-calculates which token now looks the most plausible, given all the current context and trained weights.

The weight of a neural network is not a “cell with knowledge”, but a small numerical coefficient. The meaning is distributed among billions of such coefficients: somewhere the tendency to associate a pronoun with a noun is stored, somewhere a code pattern, somewhere statistics of facts and style.

The model does not see “text” like a human. It receives tokens as input: sometimes they are whole words, sometimes parts of words, sometimes punctuation marks or spaces. Such a compromise is needed so that the dictionary is not endless and does not break up into too long chains of letters.

The token with ID `4811` does not mean anything by itself. Therefore, its index is used to pull out a row from the large embeddings table. The result is a vector of hundreds or thousands of numbers, which the network can already process with linear algebra.

If you simply add up a set of tokens, phrases with different word orders will look the same. Therefore, position information is added to the model: which token is first, which is second, who is closer, who is further.

The starting embedding for a word is usually the same. But after several transformer blocks, the representation of the word changes under the influence of neighboring tokens. This is why the same word can mean different things in different sentences.

To understand a token, it's helpful to take a quick look at other important tokens. In a long phrase, the word at the end may depend on the word at the beginning. Attention provides a direct mechanism: not to drag the whole meaning through a narrow neck, but to immediately choose what to look at now.

Self-attention is not magic, but a way to mix information between positions. Each token creates three versions of itself: query, key and value. Query says “what am I looking for”, key says “how can I help”, value is “what information I convey if I am selected”.

Attention answers the question “from whom to get information.” But after this, the model must still transform the already collected meaning. This is what MLP does: a small two-layer block that is applied separately to each position.

Transformer is not just attention and MLP. Residual connections and normalization are also important. Residual allows each block not to break an already useful representation, but to neatly add an improvement on top of the old one. LayerNorm keeps numbers in a stable range.

The encoder model sees the entire input. Each token can look both left and right. Therefore, encoder is especially good where you need to understand the text, rather than continue it: classification, search, reranking, NER, feature extraction.

The decoder-only model also uses transformer blocks, but with a causal mask. The token at position `i` sees only what came before it. It is this limitation that makes generation possible: the model honestly predicts the next token without knowing the future answer.

In the encoder-decoder architecture, the two roles are separated. Encoder reads all the source text and builds a representation of it. The decoder generates a response one token at a time, but at each step it can look at the encoder outputs through cross-attention. This is useful for translating, summarizing and converting text from format A to format B.

The main task of the decoder-only model is simple: to guess the next token using the left context. During training, she is shown the correct text in its entirety and is forced to predict the next token at each position. This is called next-token prediction.

Encoder models are usually trained differently. They corrupt the input: they hide some of the tokens or distort the text, and then ask them to restore what they missed. To cope, the model is forced to look at the left and right context simultaneously and learn to "understand" well.

After each prediction, the model is compared with the correct answer. If the probability of the correct token was too small, you need to slightly adjust the weights that led to this. Backprop is a way to neatly propagate an error back throughout the network.

When a user sends a request, the model first runs the entire prompt through itself. At this step, it builds context states for all existing tokens. This is a preparatory pass before the first new token is generated.

After the first new token, the model adds it to the context and repeats the calculation. Then another one. And one more thing. Hence the feeling of flow when streaming. On the outside it looks like printing text, but on the inside it's a cycle of "evaluate probabilities → select token → add to context."

At the output of the generation block, the model does not output a word directly. It produces a set of numbers throughout the dictionary. These numbers are called logits. After softmax they turn into probabilities. Temperature changes the “sharpness” of this distribution: low makes the choice more conservative, high adds variety.

The model was trained to continue the text plausibly, not to verify the truth of each statement. Therefore, if the context is weak, the question is ambiguous, or similar patterns are mixed in the weights, it can confidently give a polished but incorrect answer.

Prompt works as a temporary context. It can direct the model's attention, set the role, give examples and push towards the desired style. But it does not change the weights themselves. Once the request completes, this temporary context disappears unless you save them in the next context.

If you remove the noise of terms, the choice of architecture depends on the type of task. Need to understand the input? More often encoder. Do you need to freely continue the text? Decoder-only LLM. Do you need to convert one text to another with a strict binding to the input? Often encoder-decoder.