{
  "type": "new-runtime-knowledge-guide",
  "version": 1,
  "generated_at": "2026-07-23",
  "volume": 19,
  "slug": "production-rag-python-reference",
  "title": "Production RAG in Python",
  "description": "A practical map of a minimal, but “adult” RAG service in Python. Not an abstract pipeline, but a repository structure, Pydantic models, FastAPI dependencies, retrieval service, vector store adapter, prompt/answer layer, reindex worker, observability and tests. The idea is simple: after this volume it should be clear what files the production RAG consists of and how the request moves through the code.",
  "track": {
    "slug": "retrieval-and-rag",
    "title": "Retrieval & RAG",
    "route": "https://newruntime.com/learn/tracks/retrieval-and-rag/",
    "position": 4
  },
  "counts": {
    "sections": 7,
    "cards": 34
  },
  "routes": {
    "html": "https://newruntime.com/learn/production-rag-python-reference/",
    "json": "https://newruntime.com/learn/production-rag-python-reference.json"
  },
  "sections": [
    {
      "number": "01",
      "title": "Project And File Structure",
      "intro": "",
      "cards": [
        {
          "number": "01",
          "title": "Minimal production RAG repo structure",
          "tag": "Layout",
          "description": "The main task of the project structure is to separate the transport layer, domain models, adapters to external services and orchestration services. Then the retrieval logic is not spread across endpoints, and the indexer does not depend on HTTP and UI.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "EXAMPLE OF FILE TREE\n\napp/\n  main.py\n  api/\n    ask.py\n    ingest.py\n  core/\n    config.py\n    logging.py\n    deps.py\n  domain/\n    models.py\n    protocols.py\n  adapters/\n    embeddings_openai.py\n    vector_store_qdrant.py\n    llm_openai.py\n    reranker_api.py\n  services/\n    chunking_service.py\n    indexing_service.py\n    retrieval_service.py\n    prompt_service.py\n    answer_service.py\n  workers/\n    reindex_worker.py\n  tests/\n    test_chunking.py\n    test_retrieval.py\n    test_answer_flow.py"
            }
          ]
        },
        {
          "number": "02",
          "title": "`main.py`: startup and wiring only",
          "tag": "Layout",
          "description": "`main.py` must not contain retrieval logic. His job is to create a FastAPI app, connect routers, configure lifespan hooks and, if necessary, warm up shared clients. The thinner `main.py`, the lower the risk of turning the project into one giant file.",
          "blocks": [
            {
              "kind": "code",
              "label": "skeleton",
              "language": "python",
              "value": "app = FastAPI(...)\n\napp.include_router(ask_router)\napp.include_router(ingest_router)\n\n@asynccontextmanager\nasync def lifespan(app):\n  init_shared_clients()\n  yield\n  close_shared_clients()"
            }
          ]
        },
        {
          "number": "03",
          "title": "`api/` layer should be thin",
          "tag": "Layout",
          "description": "Endpoint should not build embeddings itself, go to Qdrant and render the prompt. It accepts a request, receives the necessary services through dependency injection and delegates to orchestration. Otherwise, the code will quickly become unsupportable.",
          "blocks": [
            {
              "kind": "code",
              "label": "bad and good approach",
              "language": "text",
              "value": "bad:\n  endpoint = all business logic\n\ngood:\n  endpoint -> service.answer(request)\n  endpoint -> return response_model"
            }
          ]
        },
        {
          "number": "04",
          "title": "FastAPI dependencies as a way to distribute shared services",
          "tag": "Layout",
          "description": "Dependency injection in FastAPI is convenient to use not only for auth, but also for delivering settings, clients and services to endpoints. Then the handler does not create Qdrant/OpenAI clients for each request and does not know the wiring details.",
          "blocks": [
            {
              "kind": "code",
              "label": "dependency example",
              "language": "python",
              "value": "def get_retrieval_service() -> RetrievalService:\n  return RetrievalService(\n    vector_store=get_vector_store(),\n    embedder=get_embedder(),\n    reranker=get_reranker(),\n  )\n\n@router.post(\"/ask\")\nasync def ask(req: AskRequest, svc: RetrievalService = Depends(get_retrieval_service)):\n  ..."
            }
          ]
        },
        {
          "number": "05",
          "title": "`config.py`: all thresholds and names should live in one place",
          "tag": "Infra",
          "description": "If `0.72`, `20`, `3000` and `text-embedding-3-large` are scattered across different files, tuning and debugging will become a pain. Configuration should be centralized and typed, usually through Pydantic settings or a similar layer.",
          "blocks": [
            {
              "kind": "code",
              "label": "typical settings fields",
              "language": "text",
              "value": "EMBED_MODEL\nCHAT_MODEL\nQDRANT_URL\nCOLLECTION_NAME\nRETRIEVAL_TOP_K\nRETRIEVAL_MIN_SCORE\nRERANK_TOP_K\nMAX_CONTEXT_TOKENS"
            }
          ]
        }
      ]
    },
    {
      "number": "02",
      "title": "Pydantic Models And Domain Contracts",
      "intro": "",
      "cards": [
        {
          "number": "06",
          "title": "Which Pydantic models are needed from the very beginning?",
          "tag": "Models",
          "description": "Even if the project is small, it’s worth starting a model right away. The minimum set is: request/response for API, chunk model for indexing, search hit model for retrieval, packed context item and trace model for observability. This makes the code predictable and reduces the number of “formless vocabularies”.",
          "blocks": [
            {
              "kind": "code",
              "label": "basic set",
              "language": "text",
              "value": "AskRequest\nAskResponse\nChunkRecord\nSearchHit\nRerankHit\nContextItem\nRAGTrace"
            }
          ]
        },
        {
          "number": "07",
          "title": "`AskRequest` and `AskResponse` should be boringly strict",
          "tag": "Models",
          "description": "The API layer should be extremely clear: what exactly the client can send and what exactly it will receive in response. It’s better to immediately provide `filters`, `history`, `tenant_id`, `debug` and fields for citations than to break the contract later in battle.",
          "blocks": [
            {
              "kind": "code",
              "label": "example models",
              "language": "python",
              "value": "class AskRequest(BaseModel):\n    query: str\n    history: list[str] = []\n    tenant_id: str\n    filters: dict[str, str] | None = None\n    debug: bool = False\n\nclass AskResponse(BaseModel):\n    answer: str\n    citations: list[dict]\n    trace_id: str | None = None"
            }
          ]
        },
        {
          "number": "08",
          "title": "`SearchHit` is the central runtime entity",
          "tag": "Models",
          "description": "Once the search has returned candidates, all further logic revolves around `SearchHit`. This is where `score`, `payload`, `text`, `doc_id`, sometimes `rerank_score`, sometimes `source_label` live. It is better to explicitly store retrieval fields than to re-extract them from a raw Qdrant/SQL response each time.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "class SearchHit(BaseModel):\n    chunk_id: str\n    doc_id: str\n    text: str\n    retrieval_score: float\n    rerank_score: float | None = None\n    metadata: dict = Field(default_factory=dict)"
            }
          ]
        },
        {
          "number": "09",
          "title": "`ChunkRecord` is better treated as a domain object rather than raw JSON",
          "tag": "Models",
          "description": "Indexing pipeline becomes easier if the chunk after parser/chunker already has an explicit form. Then the embedder, vector store adapter and worker do not talk through “something similar to a dict”, but through a normal object with ID, text and metadata.",
          "blocks": [
            {
              "kind": "code",
              "label": "chunk model example",
              "language": "python",
              "value": "class ChunkRecord(BaseModel):\n    chunk_id: str\n    doc_id: str\n    text: str\n    section: str | None = None\n    token_count: int\n    metadata: dict = Field(default_factory=dict)"
            }
          ]
        },
        {
          "number": "10",
          "title": "Protocols/interfaces separate services from libraries",
          "tag": "Models",
          "description": "If RetrievalService directly depends on a specific `QdrantClient` and a specific `OpenAI` SDK, the project becomes tightly coupled. It is much cleaner to describe small protocol interfaces: `Embedder`, `VectorStore`, `LLMClient`, `Reranker`. Then the library changes, but the orchestration code remains.",
          "blocks": [
            {
              "kind": "code",
              "label": "minimum contracts",
              "language": "python",
              "value": "class Embedder(Protocol):\n    def embed_query(self, text: str) -> list[float]: ...\n\nclass VectorStore(Protocol):\n    def search(self, vector: list[float], limit: int, filters: dict | None) -> list[SearchHit]: ...\n\nclass LLMClient(Protocol):\n    def generate(self, prompt: str) -> str: ..."
            }
          ]
        }
      ]
    },
    {
      "number": "03",
      "title": "Indexing Layer: From Document to Upsert",
      "intro": "",
      "cards": [
        {
          "number": "11",
          "title": "`ChunkingService`: separate service, not a helper function",
          "tag": "Indexing",
          "description": "Chunking is one of the most sensitive layers of RAG, so it is best to isolate it. Then you can separately test the cutting rules for Markdown, PDF, code blocks and overlap. And most importantly, the chunking logic version becomes explicit, and not hidden in a random utility function.",
          "blocks": [
            {
              "kind": "code",
              "label": "API example",
              "language": "python",
              "value": "class ChunkingService:\n    def split(self, doc: ParsedDocument) -> list[ChunkRecord]:\n        ...\n\nchunker = ChunkingService(\n    max_tokens=300,\n    overlap_tokens=40,\n)"
            }
          ]
        },
        {
          "number": "12",
          "title": "`IndexingService`: orchestrates parse → chunk → embed → upsert",
          "tag": "Indexing",
          "description": "IndexingService is where parser, chunker, embedder and vector store come together. It receives a raw document, builds chunk records, batches embedding requests and makes an upsert into the index. This is a separate use-case orchestration, and not a job in one endpoint.",
          "blocks": [
            {
              "kind": "code",
              "label": "skeleton indexing service",
              "language": "python",
              "value": "class IndexingService:\n    def ingest_document(self, doc: RawDocument) -> int:\n        parsed = self.parser.parse(doc)\n        chunks = self.chunker.split(parsed)\n        vectors = self.embedder.embed_documents([c.text for c in chunks])\n        self.vector_store.upsert(chunks, vectors)\n        return len(chunks)"
            }
          ]
        },
        {
          "number": "13",
          "title": "Qdrant adapter should hide SDK details",
          "tag": "Infra",
          "description": "Services generally do not need to know whether `query_points`, `upsert` or SQL is being called via pgvector. Therefore, the adapter must convert internal domain objects into the required SDK format and back. Then migration from one vector DB to another does not break the upper layers.",
          "blocks": [
            {
              "kind": "code",
              "label": "upsert adapter example",
              "language": "python",
              "value": "class QdrantVectorStore(VectorStore):\n    def upsert(self, chunks: list[ChunkRecord], vectors: list[list[float]]) -> None:\n        points = [\n            models.PointStruct(\n                id=chunk.chunk_id,\n                vector=vec,\n                payload={\"doc_id\": chunk.doc_id, \"text\": chunk.text, **chunk.metadata},\n            )\n            for chunk, vec in zip(chunks, vectors)\n        ]\n        self.client.upsert(collection_name=self.collection, points=points)"
            }
          ]
        },
        {
          "number": "14",
          "title": "It's better to move the Reindex worker out of the request path",
          "tag": "Indexing",
          "description": "Indexing documents can be time-consuming and expensive. Therefore, production systems often move ingestion to a separate worker: Celery/RQ/Dramatiq/Arq/cron-job/queue consumer. The HTTP endpoint only queues the task and returns `202 Accepted`.",
          "blocks": [
            {
              "kind": "code",
              "label": "pattern",
              "language": "python",
              "value": "@router.post(\"/ingest\")\nasync def ingest(req: IngestRequest):\n    job_id = enqueue_reindex(req.doc_id)\n    return {\"job_id\": job_id, \"status\": \"queued\"}"
            }
          ]
        },
        {
          "number": "15",
          "title": "Index versioning and idempotent upsert save nerves",
          "tag": "Ops",
          "description": "When changing the embedding model, chunking logic or metadata schema, old and new points cannot be quietly mixed. It is practically useful to store `embedding_model_version`, `chunker_version` and be able to safely reindex a document or an entire collection without double garbage.",
          "blocks": [
            {
              "kind": "list",
              "label": "what to store",
              "items": [
                "embedding_version",
                "chunker_version",
                "doc_updated_at",
                "do not mix points of different embedding spaces"
              ]
            }
          ]
        }
      ]
    },
    {
      "number": "04",
      "title": "Query-Time Retrieval Service",
      "intro": "",
      "cards": [
        {
          "number": "16",
          "title": "`RetrievalService` - main orchestrator before LLM",
          "tag": "Retrieval",
          "description": "It is RetrievalService that most often answers the user’s question “what happens after cosine similarity?” It builds query embedding, calls the vector store, filters the results, decides about rerank, does dedup and prepares context candidates for the next layer.",
          "blocks": [
            {
              "kind": "code",
              "label": "service core",
              "language": "python",
              "value": "class RetrievalService:\n    def retrieve(self, query: str, filters: dict | None) -> list[SearchHit]:\n        q_emb = self.embedder.embed_query(query)\n        hits = self.vector_store.search(q_emb, limit=self.top_k, filters=filters)\n        hits = self.apply_thresholds(hits)\n        hits = self.dedup(hits)\n        if self.reranker:\n            hits = self.rerank(query, hits)\n        return hits"
            }
          ]
        },
        {
          "number": "17",
          "title": "What it does `vector_store.search()` do in the adapter",
          "tag": "Retrieval",
          "description": "Inside the SDK adapter, the Qdrant/pgvector/FAISS response is turned into normalized `SearchHit` objects. It is important that a single format emerges. Then RetrievalService does not know whether the score came as `distance`, `similarity` or the `result.score` field.",
          "blocks": [
            {
              "kind": "code",
              "label": "search adapter example",
              "language": "python",
              "value": "def search(self, vector, limit, filters):\n    result = self.client.query_points(\n        collection_name=self.collection,\n        query=vector,\n        limit=limit,\n        query_filter=to_qdrant_filter(filters),\n        with_payload=True,\n    )\n    return [self._to_hit(point) for point in result.points]"
            }
          ]
        },
        {
          "number": "18",
          "title": "The gate after score is just a function in Python",
          "tag": "Retrieval",
          "description": "Once the vector store has returned `score`, the next step almost always looks like pure Python logic. This is where the decision is made to “leave a hit”, “ask for clarification”, “return the FAQ directly”, “run a reranker” or “go to LLM”.",
          "blocks": [
            {
              "kind": "code",
              "label": "real branching layer",
              "language": "python",
              "value": "def apply_thresholds(self, hits: list[SearchHit]) -> list[SearchHit]:\n    if not hits:\n        return []\n\n    if hits[0].retrieval_score < self.min_score:\n        return []\n\n    return [h for h in hits if h.retrieval_score >= self.per_hit_score]"
            }
          ]
        },
        {
          "number": "19",
          "title": "Dedup and expand_neighbors are also service methods",
          "tag": "Retrieval",
          "description": "If retrieval returns three almost identical adjacent pieces, the service must correct this before the prompt layer. Therefore, in real code there are almost always `dedup_hits`, `group_by_doc`, `expand_neighbors`, `fetch_parent_blocks` or similar methods.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "def dedup(self, hits):\n    seen = set()\n    out = []\n    for hit in hits:\n        key = (hit.doc_id, hit.chunk_id)\n        if key in seen:\n            continue\n        seen.add(key)\n        out.append(hit)\n    return out"
            }
          ]
        },
        {
          "number": "20",
          "title": "Reranker integration is better kept as a separate adapter + service method",
          "tag": "Retrieval",
          "description": "Reranker should not be “another line inside the endpoint”. It’s logical to move it into the adapter and call it from RetrievalService so that you can easily turn off rerank, change the provider, and log the original retrieval scores separately from the rerank scores.",
          "blocks": [
            {
              "kind": "code",
              "label": "example rerank path",
              "language": "python",
              "value": "def rerank(self, query: str, hits: list[SearchHit]) -> list[SearchHit]:\n    scored = self.reranker.score(query, [h.text for h in hits])\n    for hit, rerank_score in zip(hits, scored):\n        hit.rerank_score = rerank_score\n    return sorted(hits, key=lambda h: h.rerank_score or 0, reverse=True)"
            }
          ]
        },
        {
          "number": "21",
          "title": "Hybrid retrieval is better assembled as a composition, not an `if`-noodle",
          "tag": "Retrieval",
          "description": "If you want dense + BM25 + metadata filters, it is better to build it as two separate retrieval strategies and then combine them via fusion. This way the code remains testable: `dense_retriever`, `sparse_retriever`, `fusion_service`, and not one 200-line function with a bunch of flags.",
          "blocks": [
            {
              "kind": "code",
              "label": "skeleton",
              "language": "python",
              "value": "dense_hits = self.dense_retriever.retrieve(query, filters)\nsparse_hits = self.sparse_retriever.retrieve(query, filters)\nhits = self.fusion_service.rrf(dense_hits, sparse_hits)\nreturn hits[:self.top_k]"
            }
          ]
        }
      ]
    },
    {
      "number": "05",
      "title": "Prompt Layer and Generation Layer",
      "intro": "",
      "cards": [
        {
          "number": "22",
          "title": "`PromptService` should not deal with retrieval",
          "tag": "Generation",
          "description": "The Prompt layer receives a ready-made `SearchHit` or `ContextItem` list and only renders the context, system instructions, refusal policy and citation labels. If the prompt builder suddenly went into the database for additional pieces, the boundaries of the layers begin to flow.",
          "blocks": [
            {
              "kind": "code",
              "label": "correct signature",
              "language": "python",
              "value": "class PromptService:\n    def build(self, query: str, context_items: list[ContextItem]) -> str:\n        ..."
            }
          ]
        },
        {
          "number": "23",
          "title": "`pack_context()` - critical function between retrieval and prompt",
          "tag": "Generation",
          "description": "Packing is almost always needed as a separate function or service method. It counts tokens, limits length, assigns source labels, and turns `SearchHit` into `ContextItem`. This is where the real cut-off of the context often occurs, and not at the vector DB level.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "def pack_context(hits: list[SearchHit], max_tokens: int) -> list[ContextItem]:\n    items = []\n    budget = max_tokens\n    for i, hit in enumerate(hits, start=1):\n        n = estimate_tokens(hit.text)\n        if n > budget:\n            continue\n        items.append(ContextItem(\n            label=f\"SOURCE {i}\",\n            chunk_id=hit.chunk_id,\n            text=hit.text,\n            token_count=n,\n            metadata=hit.metadata,\n        ))\n        budget -= n\n    return items"
            }
          ]
        },
        {
          "number": "24",
          "title": "`AnswerService` collects retrieval + prompt + llm + postprocess",
          "tag": "Generation",
          "description": "If the RetrievalService is responsible for the search, then the AnswerService is usually responsible for the complete happy path of the request. It calls retrieval, decides to refuse, packs the context, builds a prompt, calls LLM and collects the final response object.",
          "blocks": [
            {
              "kind": "code",
              "label": "service skeleton",
              "language": "python",
              "value": "class AnswerService:\n    def answer(self, req: AskRequest) -> AskResponse:\n        hits = self.retrieval.retrieve(req.query, req.filters)\n        if not hits:\n            return self.refusal(\"Insufficient data in sources\")\n        context = self.pack_context(hits)\n        prompt = self.prompt_service.build(req.query, context)\n        raw = self.llm.generate(prompt)\n        return self.postprocess(raw, context)"
            }
          ]
        },
        {
          "number": "25",
          "title": "`postprocess()` needs to know about citations and schema",
          "tag": "Generation",
          "description": "The final text of the response rarely goes to the client “as is”. Post-processing can pull out citation labels, substitute real URLs, check JSON schema, remove junk markdown, and even fall back to a safer response if the model produced something outside the contract.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "def postprocess(self, raw: str, context: list[ContextItem]) -> AskResponse:\n    citations = extract_citations(raw)\n    citations = map_labels_to_sources(citations, context)\n    answer = cleanup_markdown(raw)\n    return AskResponse(answer=answer, citations=citations)"
            }
          ]
        },
        {
          "number": "26",
          "title": "Refusal is a separate method, not a random `return`",
          "tag": "Generation",
          "description": "If the failure is scattered across the design in different lines, the behavior of the system quickly becomes inconsistent. It is more practical to have one function `refusal_response(reason=...)` so that the UI, analytics, and trace understand the failure in the same way.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "def refusal(self, reason: str) -> AskResponse:\n    return AskResponse(\n        answer=\"I don’t see sufficient evidence in the available sources.\",\n        citations=[],\n        trace_id=None,\n    )"
            }
          ]
        }
      ]
    },
    {
      "number": "06",
      "title": "Infra, Observability And Tests",
      "intro": "",
      "cards": [
        {
          "number": "27",
          "title": "`RAGTrace` is a required object for debugging",
          "tag": "Ops",
          "description": "If trace is collected into a separate model, it can be logged, stored in the database, shown in the debug UI and used in golden tests. This is much better than just printing random lines about score into the log. The Trace should reflect the entire path of the request through the system.",
          "blocks": [
            {
              "kind": "code",
              "label": "example trace model",
              "language": "python",
              "value": "class RAGTrace(BaseModel):\n    trace_id: str\n    query: str\n    filters: dict\n    retrieval_hits: list[SearchHit]\n    packed_chunk_ids: list[str]\n    prompt_tokens: int\n    answer_preview: str\n    latency_ms: dict[str, int]"
            }
          ]
        },
        {
          "number": "28",
          "title": "Metrics by layer, not just `request_count`",
          "tag": "Ops",
          "description": "A good service has not only HTTP latency, but also metrics of retrieval and generation layers: average top-1 score, percentage of empty retrieval, rerank latency, packed token count, refusal rate, cache hit rate. These numbers help you understand what is broken even before reading the logs.",
          "blocks": [
            {
              "kind": "list",
              "label": "minimum set",
              "items": [
                "retrieval_latency_ms",
                "top1_score",
                "llm_input_tokens",
                "refusal_rate",
                "rerank_latency_ms"
              ]
            }
          ]
        },
        {
          "number": "29",
          "title": "Caching: Where is it useful in Python RAG",
          "tag": "Infra",
          "description": "In production code, the cache is usually useful in three places: embeddings of identical chunks during ingestion, retrieval-only shortcut / semantic cache for similar questions, and prompt/result cache for very expensive LLM calls. But the cache must be strictly tied to the index version, otherwise you will return the outdated truth.",
          "blocks": [
            {
              "kind": "code",
              "label": "typical keys",
              "language": "text",
              "value": "embedding_cache_key =\n  sha256(embedding_model + chunk_text)\n\nsemantic_cache_key =\n  tenant_id + nearest_query_cluster + index_version\n\nanswer_cache_key =\n  prompt_hash + model_name"
            }
          ]
        },
        {
          "number": "30",
          "title": "You need to test not only the endpoint, but also every stage",
          "tag": "Ops",
          "description": "It is better to cover Production RAG in layers: unit tests for chunking and thresholds, integration tests for retrieval adapter, golden tests for end-to-end answer flow, and separate regression cases for known failures. Then the bug is localized to a specific function faster.",
          "blocks": [
            {
              "kind": "code",
              "label": "test pyramid",
              "language": "text",
              "value": "unit:\n  test_chunker_splits_headers()\n  test_apply_thresholds_refuses_low_scores()\n\nintegration:\n  test_qdrant_adapter_maps_hits()\n\ngolden:\n  test_answer_flow_for_known_query()"
            }
          ]
        },
        {
          "number": "31",
          "title": "Failure injection is more useful than it seems",
          "tag": "Ops",
          "description": "It is very useful to be able to locally simulate failures: empty retrieval, slow reranker, broken LLM output, timeouts vector store, outdated index version. Then you can see how the service really degrades: it honestly refuses, crashes 500 times, or begins to hallucinate.",
          "blocks": [
            {
              "kind": "list",
              "label": "what to simulate",
              "items": [
                "no hits",
                "top score below threshold",
                "reranker timeout",
                "malformed model output"
              ]
            }
          ]
        }
      ]
    },
    {
      "number": "07",
      "title": "Full Python Flow And Main Thought",
      "intro": "",
      "cards": [
        {
          "number": "32",
          "title": "Full happy path of the request in terms of files and classes",
          "tag": "Bridge",
          "description": "When everything is put together correctly, the request flows through the project almost like a dependency diagram. This is the mental model that is useful to keep in mind when reading or writing production RAG code.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "END-TO-END FLOW\n\napp/api/ask.py\n  ask(req, answer_service)\n        ↓\napp/services/answer_service.py\n  answer(req)\n        ↓\napp/services/retrieval_service.py\n  retrieve(query, filters)\n        ↓\napp/adapters/embeddings_openai.py\n  embed_query(query)\n        ↓\napp/adapters/vector_store_qdrant.py\n  search(q_emb, top_k, filters)\n        ↓\napp/services/retrieval_service.py\n  thresholds → dedup → rerank\n        ↓\napp/services/prompt_service.py\n  build(query, context)\n        ↓\napp/adapters/llm_openai.py\n  generate(prompt)\n        ↓\napp/services/answer_service.py\n  postprocess → AskResponse"
            }
          ]
        },
        {
          "number": "33",
          "title": "What will you change most often in a real project?",
          "tag": "Bridge",
          "description": "In production RAG, it is not the framework or the endpoint that most often change. The following changes: chunking rules, metadata filters, thresholds, fusion weights, rerank policy, prompt template, refusal rules and a set of logged trace fields. Therefore, the project should be built so that these pieces are small and isolated.",
          "blocks": [
            {
              "kind": "list",
              "label": "frequent change points",
              "items": [
                "chunk size / overlap",
                "top-k / min-score",
                "prompt template",
                "reranker on/off",
                "citation rendering"
              ]
            }
          ]
        },
        {
          "number": "34",
          "title": "The main engineering thought of the volume",
          "tag": "Bridge",
          "description": "“Production RAG in Python” is not “connected FastAPI and Qdrant”. This is neatly laid out code: APIs, contracts, adapters, orchestration services, workers, traces and tests. The sooner the project takes this form, the easier it is to explain later why, after `cosine similarity = 0.81`, the system went to `rerank`, and not immediately to LLM.",
          "blocks": [
            {
              "kind": "list",
              "label": "short reminder",
              "items": [
                "endpoint thin",
                "services orchestration",
                "adapters hide SDK",
                "trace is required",
                "do not mix runtime layers"
              ]
            }
          ]
        }
      ]
    }
  ]
}