NEW RUNTIME

Knowledge field manual

Learn

A browsable field manual for LLM and machine-learning engineering: 34 guides, 246 sections, and 1,055 focused knowledge cards.

34 guides1,055 cards7 learning tracks
/learn/#all-guidesBrowse every guide
A practical starting route

Build the mechanism first. Add systems second.

Start with how a model produces tokens, move through retrieval, then study agent loops and production evaluation.

  1. 01LLM Foundations
  2. 02Retrieval & RAG
  3. 03Agent Systems
  4. 04Evaluation & Improvement

Seven entry points

Choose the problem you need to understand.

Each track is ordered, but no route is mandatory. Enter at a guide, leave at a card, and return through the related links.

  1. 01

    LLM Foundations

    Build a mechanical model of tokens, neural networks, generation, prompts, and inference.

    Guides
    5
    Cards
    132
  2. 02

    Prompting & Adaptation

    Move from hand-written instructions to context systems, fine-tuning, and programmatic prompt optimization.

    Guides
    3
    Cards
    103
  3. 03

    Retrieval & RAG

    Understand vector search, enrichment, runtime branching, and production RAG implementation.

    Guides
    4
    Cards
    115
  4. 04

    Agent Systems

    Study pipeline patterns, tool loops, protocols, orchestration frameworks, and operational agent design.

    Guides
    6
    Cards
    210
  5. 05

    Application Engineering

    Turn LLM behavior into maintainable Python services, graph runtimes, APIs, and deployable systems.

    Guides
    6
    Cards
    206
  6. 06

    Evaluation & Improvement

    Measure quality, observe production behavior, validate judges, and turn failures into a data flywheel.

    Guides
    6
    Cards
    179
  7. 07

    ML & Decision Systems

    Connect language-model work to statistics, encoders, classical machine learning, forecasting, causality, and bandits.

    Guides
    4
    Cards
    110

Complete library

All 34 guides.

Search by concept, implementation concern, framework, metric, or card topic.

34 guides

  1. Vol. 01 · Agent Systems

    LLM Pipeline Patterns

    A comprehensive catalog of steps and tricks for building production systems based on language models. Each pattern is atomically parsed.

    Sections
    10
    Cards
    47
  2. Vol. 02 · Evaluation & Improvement

    LLM Observability & Evaluations

    How to understand what is happening inside the pipeline - and what exactly is broken. Tracing, metrics, automatic evaluations, testing prompts and secure deployment of changes.

    Sections
    8
    Cards
    35
  3. Vol. 03 · Application Engineering

    The LLM Framework Ecosystem

    Key abstractions, architectural patterns, pitfalls and selection criteria for the main frameworks of the LLM ecosystem. Each one is disassembled atomically.

    Sections
    8
    Cards
    28
  4. Vol. 04 · Prompting & Adaptation

    Fine-Tuning Strategies

    When to use fine-tuning, and when is RAG or prompting enough? LoRA, QLoRA, SFT, DPO, RLHF, distillation - broken down into focused cards. How to prepare data, how to evaluate the result, how not to waste the GPU.

    Sections
    8
    Cards
    27
  5. Vol. 05 · LLM Foundations

    LLM Internals: From Token to Logit to Answer

    The full path from the input request to the next token: tokenization, embeddings, positional encoding, self-attention, MLP, normalization, KV-cache, logits, softmax and sampling parameters - parsed atomically.

    Sections
    7
    Cards
    28
  6. Vol. 06 · Agent Systems

    LLM Scaffolding: Tool Calling, Structured Output & Agent Loops

    Full analysis of the orchestration layer over LLM: function calling, JSON schema, constrained decoding, ReAct/CoT patterns, agent cycle, context memory management, error handling and comparison of API providers.

    Sections
    8
    Cards
    27
  7. Vol. 07 · LLM Foundations

    From a Neuron to an LLM

    The route from linear regression to transformer: classical ML models and their limit, biological neuron and perceptron, deep networks and representations, recurrence and its problems, attention as a breakthrough, transformer as architecture and LLM as a result of scale.

    Sections
    8
    Cards
    23
  8. Vol. 08 · ML & Decision Systems

    Statistics for Evaluating LLM Systems

    Basic concepts of statistics as applied to LLM experiments: quality metrics, variance and confidence intervals, hypothesis testing, A/B tests, LLM-as-judge, evaluation pitfalls and a practical checklist for a valid experiment.

    Sections
    7
    Cards
    24
  9. Vol. 09 · Retrieval & RAG

    RAG & Vector Search

    Full route: why words cannot be compared directly → Bag of Words → TF-IDF → Word2Vec → contextual embeddings → cosine distance → how exactly the chunk “matches” with the query → HNSW → hybrid search → the full RAG pipeline parsed down to the byte.

    Sections
    7
    Cards
    24
  10. Vol. 10 · ML & Decision Systems

    BERT, Encoders & Non-Generative Models

    Taxonomy of transformer architectures: encoder-only, decoder-only, encoder-decoder. How BERT works and why bidirectional is better to understand. BERT family. Fine-tuning patterns. T5 and seq2seq. When a small specialized model beats GPT-4. A practical map for choosing an architecture for a task.

    Sections
    7
    Cards
    23
  11. Vol. 11 · Agent Systems

    MCP, A2A & Agent Protocols

    Model Context Protocol (Anthropic, 2024) - architecture, creation of servers in TypeScript and Python, all transports, primitives, security. Agent-to-Agent (Google, 2025) - task model, AgentCard, streaming. Comparison of ecosystem protocols.

    Sections
    7
    Cards
    34
  12. Vol. 12 · Evaluation & Improvement

    Evaluation Process & LLM-as-Judge

    How to build the process of evaluating LLM systems from scratch: minimum viable eval, guardrails vs evaluators, why a binary assessment is better than a scale of 1–5, negative scenarios, LLM-as-judge with correlation to people, detection of degradation after release, synthetic data - when it helps and when it harms.

    Sections
    7
    Cards
    23
  13. Vol. 13 · Retrieval & RAG

    Advanced RAG: Context & Enrichment

    Deep analysis of the context enrichment layer in RAG: RAG vs Fine-tuning - when to choose what, advanced chunking strategies, query augmentation and decomposition, four patterns of context degradation, compression and hygiene, memory layers, incremental index updating, citations & grounding, prompt injection through documents.

    Sections
    7
    Cards
    24
  14. Vol. 14 · LLM Foundations

    LLM Internals: KV Cache & Generation

    An in-depth analysis of the mechanics of inference: how exactly KV-cache works and why O(n²) without it, prefill vs decode phases, GPU memory arithmetic, GQA/MQA/PagedAttention, prompt caching at the API level (Anthropic + OpenAI), sampling from logits to token, speculative decoding, and the main case - “mother washed the frame” through LLM step by step.

    Sections
    8
    Cards
    24
  15. Vol. 15 · LLM Foundations

    LLMs in Plain English

    Not the history of architectures or GPU micro-optimizations, but practical mechanics without academic fog: how text is cut into tokens, how transformer blocks recalculate the meaning, how the encoder and decoder differ, how the model learns and why it responds one token at a time.

    Sections
    7
    Cards
    23
  16. Vol. 16 · ML & Decision Systems

    Classical Machine Learning in Plain English

    A cheat sheet for talking with ML specialists in one language: what are data, features, target, loss, generalization and prediction; what problems are solved by linear models, trees, SVM, kNN, ensembles, clustering and PCA; and why the transition to neural networks did not cancel the basic mechanics of errors and optimization, but scaled it.

    Sections
    7
    Cards
    31
  17. Vol. 17 · ML & Decision Systems

    Forecasting, Causal Inference & Bandits

    The next layer after classic ML. Not just “predict the label”, but understand three different questions: what will happen next, what will happen if we intervene, and what is better to show right now with incomplete information. These are three different formulations of the problem, with different metrics, leakage risks and different language of communication with researchers and product teams.

    Sections
    7
    Cards
    32
  18. Vol. 18 · Retrieval & RAG

    RAG Runtime: From Cosine Similarity to the Final Answer

    Not “what is RAG” in general terms, but what actually happens at the code and object level: what data structures are created during indexing, what exactly the vector store returns, who reacts to `score`, what if/else branches are triggered after retrieval, how rerank, packing, refusal, citations and debugging are done. This volume is about RAG as an executable program.

    Sections
    7
    Cards
    33
  19. Vol. 19 · Retrieval & RAG

    Production RAG in Python

    A practical map of a minimal, but “adult” RAG service in Python. Not an abstract pipeline, but a repository structure, Pydantic models, FastAPI dependencies, retrieval service, vector store adapter, prompt/answer layer, reindex worker, observability and tests. The idea is simple: after this volume it should be clear what files the production RAG consists of and how the request moves through the code.

    Sections
    7
    Cards
    34
  20. Vol. 20 · Application Engineering

    The Python Utility Stack for LLM Pipelines

    LLM applications live not only on models. Around them there is almost always templating, structure validation, string sanitization, JSON parsing, regex, file paths, batching, caching, async I/O and little standard library functions that save hours. This volume is not about “AI”, but about service Python, which makes the LLM pipeline stable and readable.

    Sections
    7
    Cards
    37
  21. Vol. 21 · Agent Systems

    LangChain & LangGraph Under the Hood

    This volume is not about “which framework is more fashionable,” but about the modern architecture of the LangChain Inc. stack. In the current version, `LangChain` is a high level with models, messages, tools, middleware and fast `create_agent`, and `LangGraph` is a low-level runtime for stateful agents: `State`, `Nodes`, `Edges`, `Command`, `thread_id`, `checkpoints`, `interrupts`, replay and long-running execution. Below is how this is really connected and what exactly happens during startup.

    Sections
    7
    Cards
    32
  22. Vol. 22 · Application Engineering

    Production LangChain + LangGraph: Code Structure & Runtime

    This volume answers a more applied question: how does a real Python project on LangChain + LangGraph spread out into files? Not “which framework is better”, but where exactly “state”, “context”, “tools”, “middleware”, “nodes”, “graph.compile(...)”, “checkpointer”, “store”, HTTP-endpoints, “thread id”, streaming and “resume” live. This is a codebase map, not a concept map.

    Sections
    7
    Cards
    34
  23. Vol. 23 · LLM Foundations

    What a Prompt Is and Why a Question Becomes an Answer

    The main paradox of LLM is this: if the model is just predicting the next token, why does it answer the question meaningfully, and not just continue our text? This volume analyzes the prompt as the model's actual input, chat mode as a special data format, instruction tuning as a shift in behavior towards the “assistant”, and the connection of prompt with the model weights.

    Sections
    7
    Cards
    34
  24. Vol. 24 · Application Engineering

    Mini-Repository: FastAPI + LangGraph + LangChain as a Service

    This volume does not answer the question “how to write another agent”, but the question “how to make a live external application”. At the center is one solid Python mini-repo: ‘FastAPI’ as a public HTTP/SSE layer, ‘LangGraph’ as runtime, ‘LangChain’ as a layer of models, tools and middleware, external persistence for threads, scaling, auth, health/readiness and a separate place for ‘MCP’ as a tool/data plane.

    Sections
    7
    Cards
    36
  25. Vol. 25 · Application Engineering

    Starter Kit: FastAPI + LangGraph + LangChain by File

    It's almost a project template. Not an abstract architecture, but a small, but production-focused starter kit: what files to create, what to put in them, how to assemble a ‘FastAPI’ service, where ‘LangGraph’ and ‘LangChain’ live, how to add durability, streaming, ‘resume’, tests, Docker and an optional ‘MCP’ layer.

    Sections
    7
    Cards
    35
  26. Vol. 26 · Application Engineering

    Deployment & DevOps for an LLM Service

    This volume is not about the code inside the agent, but about the operation outside: ingress, TLS, forwarded headers, “root path”, probes, secrets, queues, tracing, metrics, token/cost budgets, autoscaling, rollout/rollback and the place of “MCP” in the production circuit. This is a map of what it takes for an LLM service to not just run, but live and experience real traffic.

    Sections
    7
    Cards
    36
  27. Vol. 27 · Evaluation & Improvement

    The Production Observability & Evaluation Loop

    This volume does not repeat the general review of evals. It shows the operational cycle of the team: how incident cases, datasets, offline evals, human review, canary gates and rollout/rollback decisions are born from production traces and user signals. It’s not “what evals are,” it’s “how the team lives with evals and observability every day.”

    Sections
    7
    Cards
    35
  28. Vol. 28 · Prompting & Adaptation

    Context Engineering & Prompt Caching

    This volume is not about the “magical prompt”, but about the engineering layer around it: from what blocks the context is collected, how the state lives between moves, what exactly is cached, when compaction is needed, how not to drown in tool outputs and what to measure in production. The idea is simple: the quality of an LLM system is often determined not by the model per se, but by the context you give it and how you manage its life cycle.

    Sections
    7
    Cards
    35
  29. Vol. 29 · Evaluation & Improvement

    The Data Flywheel: From Production Cases to Release Gates

    This volume is about the main engine of a high-quality LLM team: how production traces and user pain turn into curated cases, label taxonomy, human review queue, regression set, canary set and decisions about what to fix prompt, what to retriever, and what to train. So it's not "where to get the dataset," but how the team grows it out of their own work and turns the data into a system improvement cycle.

    Sections
    7
    Cards
    35
  30. Vol. 30 · Agent Systems

    Reasoning Models & Tool Loops

    This volume is about a new layer of LLM engineering: models that know how to spend more inference-time compute on reasoning, and applications that make them alternate with tools. It is important to understand two things at once: what is the reasoning runtime itself and how is the loop model -> tool -> model, where state, budgets, stop conditions and engineering invariants live.

    Sections
    7
    Cards
    35
  31. Vol. 31 · Agent Systems

    The OpenClaw Phenomenon and What to Learn from It

    The project started as Clawdbot, was renamed Moltbot on January 27, 2026, and then OpenClaw on January 30, 2026. But the essence of the phenomenon was not in the title. This was a rare case of an open-source LLM product hitting the right intersection of UX, architecture, distribution, and memetics. This volume looks at what decisions it really made, why it went viral, and what lessons to learn from it without copying its mistakes.

    Sections
    7
    Cards
    35
  32. Vol. 32 · Evaluation & Improvement

    LLM-as-Judge Meta-Evaluation

    A separate volume is not about eval in general, but about the validation of the judge itself: agreement with people, metrics for binary and ordinal verdicts, resistance to repeated runs, confidence calibration, bias audits, pairwise ranking and minimum reporting standard. Agreement → classifier view → stability → calibration → bias → reporting.

    Sections
    7
    Cards
    25
  33. Vol. 33 · Evaluation & Improvement

    Core Metrics for LLM Pipelines

    One volume about the most useful metrics for LLM systems: from average, variance and standard deviation to accuracy, precision, recall, F1, MRR, MAP, nDCG@k, calibration, pass@k, latency and cost per success. Each card answers four questions: what it measures, how to calculate it, how to read, and where to apply it in the pipeline.

    Sections
    5
    Cards
    26
  34. Vol. 34 · Prompting & Adaptation

    DSPy & GEPA: Implementation Cookbook

    A big volume about how to really build systems on DSPy: all the key abstractions from official documentation, a built-in module map, data design and metrics, the choice of an optimizer, deep dive by GEPA and step-by-step rollout plans from the first baseline to production.

    Sections
    8
    Cards
    41