{ "type": "new-runtime-knowledge-guide", "version": 1, "generated_at": "2026-07-23", "volume": 22, "slug": "langchain-langgraph-codebase-reference", "title": "Production LangChain + LangGraph: Code Structure & Runtime", "description": "This volume answers a more applied question: how does a real Python project on LangChain + LangGraph spread out into files? Not “which framework is better”, but where exactly “state”, “context”, “tools”, “middleware”, “nodes”, “graph.compile(...)”, “checkpointer”, “store”, HTTP-endpoints, “thread id”, streaming and “resume” live. This is a codebase map, not a concept map.", "track": { "slug": "application-engineering", "title": "Application Engineering", "route": "https://newruntime.com/learn/tracks/application-engineering/", "position": 3 }, "counts": { "sections": 7, "cards": 34 }, "routes": { "html": "https://newruntime.com/learn/langchain-langgraph-codebase-reference/", "json": "https://newruntime.com/learn/langchain-langgraph-codebase-reference.json" }, "sections": [ { "number": "01", "title": "Production-Repository Map", "intro": "", "cards": [ { "number": "01", "title": "The production agent on LangGraph is not one graph.py, but several layers.", "tag": "Framing", "description": "A good project usually divides code into four layers: contracts, executables, orchestration graph, and serving/runtime. Then LangChain remains a layer of standard building blocks, and LangGraph becomes the place where these blocks are combined into a meaningful lifecycle.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "THE RIGHT PLAYING\n\ncontracts\n → state / context / schemas / settings\n\ncomponents\n → model / tools / prompts / middleware / clients\n\ngraph\n → nodes / routes / subgraphs / compile\n\nruntime\n → api / invoke / stream / resume / persistence / tests" } ] }, { "number": "02", "title": "The most important boundary in the code is “what the knot does” vs. “where the graph goes next.”", "tag": "Framing", "description": "Noda is responsible for the action: call the model, make a tool lookup, check approval. Routing is responsible for controlling flow: whether to go to ‘tools’, ‘review’, ‘END’ or another subgraph. If you mix it in one place, the graph quickly becomes unreadable.", "blocks": [ { "kind": "list", "label": "practice", "items": [ "node = work", "route = transition", "Runtime in a single function" ] } ] }, { "number": "03", "title": "One of the normal file trees for this project", "tag": "Framing", "description": "This is not the only correct structure, but it shows how to share responsibility. The main idea is that ‘contracts’ should not depend on the API layer, ‘tools’ should not know about HTTP, and ‘graph’ should not create database clients directly inside the nodes.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "SUGGESTED TREE\n\napp/\n agent/\n contracts/\n state.py\n context.py\n schemas.py\n settings.py\n components/\n models.py\n prompts.py\n tools.py\n middleware.py\n clients.py\n graph/\n nodes.py\n routes.py\n subgraphs.py\n build.py\n runtime/\n service.py\n stream.py\n persistence.py\n api/\n chat.py\n resume.py\n tests/\n test_tools.py\n test_nodes.py\n test_graph.py" } ] }, { "number": "04", "title": "Request lifecycle should be considered before writing nodes", "tag": "Framing", "description": "It is very useful to decide in advance what the service does at the input of the query: whether thread creates, how it forms “config”, where it takes “context”, how it causes “graph.invoke” or “graph.astream”, what it returns to the outside, and how it processes “ interrupt ”.", "blocks": [ { "kind": "code", "label": "routine", "language": "text", "value": "HTTP request\n→ build input state\n→ build config {\"configurable\": {\"thread_id\": ...}}\n→ build runtime context\n→ invoke / astream\n→ inspect final state or __interrupt__\n→ return HTTP response" } ] }, { "number": "05", "title": "One graph package per business flow is usually better.", "tag": "Framing", "description": "If the project has support-agent, sales-agent and document-review workflow, do not try to cram everything into one megagraph. Individual graphs are easier to test, compile, version and maintain.", "blocks": [ { "kind": "list", "label": "chip", "items": [ "different state schema", "Different tools and middleware", "common components can be shuffled below" ] } ] } ] }, { "number": "02", "title": "Contracts and basic schemes", "intro": "", "cards": [ { "number": "06", "title": "‘state.py’ is the main contract of the entire count", "tag": "Contracts", "description": "In LangGraph, the state determines what data can live inside the execution. It is better to keep it in a separate file and discuss as a public contract: which keys are required, which transient, which accumulative, which write middleware, which write tools, which node and read.", "blocks": [ { "kind": "code", "label": "stately", "language": "sql", "value": "from langchain.messages import AnyMessage\nfrom langgraph.graph.message import add_messages\nfrom typing_extensions import TypedDict, Annotated\n\nclass AgentState(TypedDict):\n messages: Annotated[list[AnyMessage], add_messages]\n user_query: str\n retrieved_docs: list[str]\n structured_answer: dict | None\n approved: bool | None" } ] }, { "number": "07", "title": "Context.py for immutable runtime dependencies", "tag": "Contracts", "description": "Context is a dependency injection for a specific start. There live well \"user id\", feature flags, tenancy, database handles, environment mode. It is not part of the evolving state or part of the default LLM prompt.", "blocks": [ { "kind": "code", "label": "example", "language": "sql", "value": "from dataclasses import dataclass\n\n@dataclass\nclass Context:\n user_id: str\n tenant_id: str\n can_send_email: bool" } ] }, { "number": "08", "title": "`schemas.py` - external borders are better to type rigidly", "tag": "Contracts", "description": "Structured output, HTTP payloads, tool input/output contracts, and final API responses are usually best handled through ‘Pydantic’. Then the internal graph state can remain light, and the outer boundaries become validated and stable.", "blocks": [ { "kind": "code", "label": "example", "language": "sql", "value": "from pydantic import BaseModel\n\nclass AnswerSchema(BaseModel):\n answer: str\n citations: list[str]\n needs_human_review: bool" } ] }, { "number": "09", "title": "*settings.py should configure the project, not manage the graph", "tag": "Contracts", "description": "Settings are useful to make separately: model name, timeout, max tokens, DSN for checkpointer-a, streaming mode, feature flags. But \"settings.py\" should not create a graph by itself. Its job is to deliver configuration, not mix wiring with orchestration logic.", "blocks": [ { "kind": "list", "label": "What to keep in settings", "items": [ "MODEL_NAME", "DB_DSN", "MAX_TOKENS", "random node side effects" ] } ] }, { "number": "10", "title": "Good rule: state for running, store for memory between threads", "tag": "Contracts", "description": "This distinction is better fixed at the level of contracts. State is responsible for what happens inside the current run/thread. The store is responsible for long-lived user or application data through different threads and sessions.", "blocks": [ { "kind": "table", "headers": [ "Essence", "For what?" ], "rows": [ [ "State", "messages, intermediate results, current approval" ], [ "Store", "preferences, profile, cross-thread memory" ] ] } ] } ] }, { "number": "03", "title": "Components of LangChain-Layer", "intro": "", "cards": [ { "number": "11", "title": "'models.py' is a separate location for 'init chat model(...)' and model policy", "tag": "Components", "description": "The model is best initialized in one place. Then it is easier to change the provider, temperature, timeouts, streaming behavior and runtime-configurable parameters. If the model is smeared on the nodes, you quickly lose control of the call policy.", "blocks": [ { "kind": "code", "label": "example", "language": "sql", "value": "from langchain.chat_models import init_chat_model\n\ndef build_model():\n return init_chat_model(\n \"provider:model\",\n temperature=0,\n timeout=30,\n )" } ] }, { "number": "12", "title": "'prompts.py' or 'prompts/' - system prompt is better not to hide in node", "tag": "Components", "description": "Even if the prompt is short, keeping it separate is useful: it is easier to review, test, version and change without shoveling the orchestration code. This is especially important if middleware then makes dynamic prompt generation.", "blocks": [ { "kind": "list", "label": "store", "items": [ "system prompts", "response instructions", "tool-use policy hints" ] } ] }, { "number": "13", "title": "Tools.py: Tools should use ToolRuntime, not global variables", "tag": "Components", "description": "The official approach now is access to the state, context, store, config, stream writer and `tool call id' via `ToolRuntime'. This makes the tools testable and doesn’t make you drag hidden singletons or global request contexts.", "blocks": [ { "kind": "code", "label": "example", "language": "sql", "value": "from langchain.tools import tool, ToolRuntime\n\n@tool\ndef fetch_preferences(runtime: ToolRuntime[Context]) -> str:\n user_id = runtime.context.user_id\n if runtime.store:\n memory = runtime.store.get((\"users\",), user_id)\n if memory:\n return memory.value[\"preferences\"]\n return \"No preferences\"" } ] }, { "number": "14", "title": "“middleware.py” is a place for cross-cutting logic, not “another prompt”", "tag": "Components", "description": "Through middleware, it is convenient to do dynamic prompt, dynamic model selection, trimming, tool error handling, permission-based tool filtering and audit/logging. These are cross-cutting concerns that should not be smeared on each node with your hands.", "blocks": [ { "kind": "code", "label": "frequent hooks", "language": "text", "value": "@before_model\n@after_model\n@wrap_model_call\n@wrap_tool_call\n@dynamic_prompt\n\nNode-style hooks are suitable for sequential logic, wrap-style – for control around call-a" } ] }, { "number": "15", "title": "`clients.py' or `repositories.py' protects tools from direct SDK dirt", "tag": "Components", "description": "It is better to depend on a small client wrapper than to directly collect SQL, HTTP-requests and deserialization in place. The tool remains orchestration-aware, but does not turn into 200 lines of integration code.", "blocks": [ { "kind": "list", "label": "pattern", "items": [ "tool → repository", "tool → raw httpx/sqlalchemy everywhere" ] } ] } ] }, { "number": "04", "title": "Assembling Count by Files", "intro": "", "cards": [ { "number": "16", "title": "Nodes.py – Nodes should return updates, not mutate state", "tag": "Graph", "description": "The official examples of LangGraph constantly repeat one idea: the node reads the state and returns the changes. This makes execution predictable, helps reducers work correctly and makes debugging easier through checkpoints/state history.", "blocks": [ { "kind": "code", "label": "fine", "language": "python", "value": "def retrieve_node(state: AgentState):\n docs = retriever.search(state[\"user_query\"])\n return {\"retrieved_docs\": docs}" }, { "kind": "code", "label": "badly", "language": "python", "value": "def retrieve_node(state: AgentState):\n state[\"retrieved_docs\"] = retriever.search(...)\n return state" } ] }, { "number": "17", "title": "routes.py is a separate location for conditional edges and command(...)", "tag": "Graph", "description": "When the logic of transitions lives apart, it becomes clear why the graph goes to ‘review’, ‘tools’ or ‘END’. This is more important than it seems: routing logic is quickly becoming the main place of business rules, and it should be read separately from node actions themselves.", "blocks": [ { "kind": "code", "label": "example", "language": "sql", "value": "from langgraph.graph import END\n\ndef route_after_model(state: AgentState):\n if state.get(\"approved\") is False:\n return END\n if needs_human_review(state):\n return \"review\"\n if has_tool_calls(state[\"messages\"][-1]):\n return \"tools\"\n return END" } ] }, { "number": "18", "title": "'build.py' or 'graph.py' should only do wiring and 'compile'", "tag": "Graph", "description": "This is the place where state schema is associated with node, ribs, checkpointer, store and sometimes static interrupts. A good build graph() contains almost no business logic, it only collects the execution object from the already finished pieces.", "blocks": [ { "kind": "code", "label": "example", "language": "sql", "value": "from langgraph.graph import StateGraph, START\n\ndef build_graph(checkpointer, store):\n builder = StateGraph(AgentState)\n builder.add_node(\"retrieve\", retrieve_node)\n builder.add_node(\"answer\", answer_node)\n builder.add_edge(START, \"retrieve\")\n builder.add_edge(\"retrieve\", \"answer\")\n return builder.compile(checkpointer=checkpointer, store=store)" } ] }, { "number": "19", "title": "Subgraphs are useful when you need a separate state boundary.", "tag": "Graph", "description": "Subgraphs are not only useful for multi-agent. It’s also a way to split commands, allocate a separate approval-flow or keep a private internal state at a specific module without clogging the parent graph.", "blocks": [ { "kind": "list", "label": "scenario", "items": [ "document review subgraph", "specialized planner subgraph", "private agent memory inside subgraph" ] } ] }, { "number": "20", "title": "Approval flow is better to make a separate node with 'interrupt()'", "tag": "Graph", "description": "If human confirmation is part of the business flow, don’t encode it through the strange external ifs around the graph. It is much cleaner to make explicit approval node, which pauses execution and then continues the graph through Command(resume=..)", "blocks": [ { "kind": "code", "label": "example", "language": "sql", "value": "from langgraph.types import interrupt\n\ndef review_node(state: AgentState):\n approved = interrupt({\n \"question\": \"Approve answer?\",\n \"draft\": state[\"structured_answer\"],\n })\n return {\"approved\": bool(approved)}" } ] } ] }, { "number": "05", "title": "Serving Layer and Runtime Action", "intro": "", "cards": [ { "number": "21", "title": "service.py typically builds input, config, and context before calling a graph", "tag": "Runtime", "description": "HTTP handlers are better made thin. The basic orchestration around calling a graph can live in the service: form a state update, collect a 'thread id', if necessary 'checkpoint id', transmit a runtime context, and select 'invoke' versus 'astream'.", "blocks": [ { "kind": "code", "label": "typical", "language": "text", "value": "config = {\"configurable\": {\"thread_id\": thread_id}}\ncontext = Context(user_id=user_id, tenant_id=tenant_id, can_send_email=False)\n\nresult = graph.invoke(\n {\"messages\": [{\"role\": \"user\", \"content\": text}]},\n config=config,\n context=context,\n)" } ] }, { "number": "22", "title": "One must have a clear path for ‘resume’, not just ‘invoke’.", "tag": "Runtime", "description": "If a project has interrupts, you need a separate endpoint or service method that accepts ‘thread id’ and ‘resume’ payload. It’s not “the same invoke” because it’s important to continue a particular thread rather than start a new run.", "blocks": [ { "kind": "code", "label": "resume path", "language": "text", "value": "config = {\"configurable\": {\"thread_id\": thread_id}}\n\nresult = graph.invoke(\n Command(resume=resume_payload),\n config=config,\n)" } ] }, { "number": "23", "title": "Streaming endpoint should be able to listen not only tokens, but also updates", "tag": "Runtime", "description": "For production UI, it is usually useful to stream both message chunks and state updates. Then you can show the answer for tokens, and understand which node the graph is now on, and catch \" interrupt \" in time.", "blocks": [ { "kind": "code", "label": "example", "language": "text", "value": "async for mode, chunk in graph.astream(\n inputs,\n config=config,\n stream_mode=[\"messages\", \"updates\", \"custom\"],\n):\n ...\n\nIn interactive flows, subgraphs are often useful." } ] }, { "number": "24", "title": "‘thread id’ and ‘checkpoint id’ are part of the runtime API.", "tag": "Runtime", "description": "If the service stores execution history or can replay/debug, the user or internal tooling may need not only “thread id”, but also “checkpoint id”. With them, you can get a specific snapshot or replay execution after a given point.", "blocks": [ { "kind": "code", "label": "debug / replay", "language": "text", "value": "state = graph.get_state({\n \"configurable\": {\"thread_id\": thread_id}\n})\n\nhistory = list(graph.get_state_history({\n \"configurable\": {\"thread_id\": thread_id}\n}))\n\nCheckpoint id becomes useful when the tooling or API can address a particular snapshot." } ] }, { "number": "25", "title": "Sometimes the service calls ready-made 'create agent', sometimes compiled graph - and that's okay.", "tag": "Runtime", "description": "If the task is simple, the service can keep the agent object from create agent. If the orchestration is custom, the service holds compiled 'StateGraph'. In production code, both forms are often found in different modules of the same project.", "blocks": [ { "kind": "table", "headers": [ "Form", "When appropriate." ], "rows": [ [ "`create_agent`", "typical agent loop" ], [ "compiled graph", "Custom branching / interrupts / subgraphs" ] ] } ] } ] }, { "number": "06", "title": "Memory, Persistence and Operator", "intro": "", "cards": [ { "number": "26", "title": "`persistence.py' must explicitly select a checkpointer under the medium", "tag": "Ops", "description": "For local development, “InMemorySaver” is enough. For production, official materials recommend DB-backed persistence. The most useful thing here is to centralize the creation of a checkpointer, rather than scattering it across different build functions.", "blocks": [ { "kind": "list", "label": "matrix", "items": [ "local dev → InMemorySaver", "prod → database-backed checkpointer", "thread id is mandatory for persistence" ] } ] }, { "number": "27", "title": "Store is a separate resource for long-term memory, not a replacement for checkpointer.", "tag": "Ops", "description": "Checkpointer keeps the graph state on super-steps and threads. A store is needed when information needs to survive conversations and be accessible through different threads. Usually, both resources are initialized side by side, but solve different tasks.", "blocks": [ { "kind": "code", "label": "compile", "language": "text", "value": "graph = builder.compile(\n checkpointer=checkpointer,\n store=store,\n)" } ] }, { "number": "28", "title": "Memory should be limited: trim or summarize, otherwise the thread inflates.", "tag": "Ops", "description": "In both LangChain agents and LangGraph workflows, short memory lives in state/messages. With the growth of the thread, you either need to trimm the messages or summarize the story. Cost, latency and noise increase in the prompt context.", "blocks": [ { "kind": "code", "label": "options", "language": "text", "value": "1. trim_messages utility\n2. before model middleware that cuts history\n3. summarize old messages and store summary\n4. custom filtering strategy" } ] }, { "number": "29", "title": "DB-backed saver/store usually requires a 'setup()' or separate migration step.", "tag": "Ops", "description": "The official materials separately remind: for database-backed persistence, you need to prepare a database scheme. It’s better to frame it as an explicit deploy/startup step, rather than hoping that “it will appear” during the first query.", "blocks": [ { "kind": "list", "label": "rule", "items": [ "Setup/migrations separately", "Lazy schema creation in the middle of the user request" ] } ] }, { "number": "30", "title": "Code before 'interrupt()' must be idempotent", "tag": "Ops", "description": "Interrupt resumes node from the beginning, not from a specific line. Therefore, everything that has been done before \"interrupt()\" can be done again. If there was a non-idempotent side effect, you will get duplicates, re-entries, or inconsistency.", "blocks": [ { "kind": "list", "label": "safer", "items": [ "upsert before interrupting", "side effects after interruption", "create row before interrupt" ] } ] } ] }, { "number": "07", "title": "Tests, Skeleton and Anti-Patterns", "intro": "", "cards": [ { "number": "31", "title": "Test not only the final answer, but also the nodes, transitions and resume path.", "tag": "Testing", "description": "A good set of tests here is layered: unit tests tools and middleware, unit tests node functions, integration tests compiled graph, separate tests for interrupt/resume and, if necessary, snapshot-state tests. Otherwise, the error in routing will easily hide behind “the answer seems to have turned out.”", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "TEST PYRAMID\n\ntool tests\n ↓\nnode tests\n ↓\nroute / middleware tests\n ↓\ncompiled graph integration tests\n ↓\ninterrupt / resume / replay tests" } ] }, { "number": "32", "title": "Minimal production skeleton looks like this.", "tag": "Bridge", "description": "If you compress everything to the most useful minimum, there are a few mandatory files. This is already a good starter for a real service, not for a laptop demo.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "MINIMAL SKELETON\n\ncontracts/state.py\ncontracts/context.py\ncontracts/schemas.py\n\ncomponents/models.py\ncomponents/tools.py\ncomponents/middleware.py\n\ngraph/nodes.py\ngraph/routes.py\ngraph/build.py\n\nruntime/service.py\nruntime/persistence.py\napi/chat.py\n\ntests/test_graph.py" } ] }, { "number": "33", "title": "Frequent anti-patterns in this codebase", "tag": "Testing", "description": "It is not the library that usually breaks down, but the architecture around it. The most common problems are: giant graph.py, business logic inside route functions without tests, tools with direct access to globals, an HTTP layer that itself builds half the state, and the lack of an explicit resume path.", "blocks": [ { "kind": "list", "label": "red flags", "items": [ "whole project in one module", "tool = 200 lines of raw SDK code", "No interrupt/resume test", "No ownership of the state schema" ] } ] }, { "number": "34", "title": "The main idea of the volume", "tag": "Bridge", "description": "Production 'LangChain' + 'LangGraph' project is more conveniently understood as a regular backend with clear layers: contracts, components, orchestration, runtime. When ‘state’ and ‘context’ are defined explicitly, ‘tools’ use ‘ToolRuntime’ and the graph is assembled in a separate ‘build.py’, things become much less ‘magical’ and much more engineering.", "blocks": [ { "kind": "list", "label": "takeaway", "items": [ "State - Main Contract", "tools/middleware – layer of components", "graph/build — wiring", "service/api — invoke, stream, resume" ] } ] } ] } ] }