{
  "type": "new-runtime-knowledge-guide",
  "version": 1,
  "generated_at": "2026-07-23",
  "volume": 25,
  "slug": "fastapi-langgraph-langchain-starter-kit",
  "title": "Starter Kit: FastAPI + LangGraph + LangChain by File",
  "description": "It's almost a project template. Not an abstract architecture, but a small, but production-focused starter kit: what files to create, what to put in them, how to assemble a ‘FastAPI’ service, where ‘LangGraph’ and ‘LangChain’ live, how to add durability, streaming, ‘resume’, tests, Docker and an optional ‘MCP’ layer.",
  "track": {
    "slug": "application-engineering",
    "title": "Application Engineering",
    "route": "https://newruntime.com/learn/tracks/application-engineering/",
    "position": 5
  },
  "counts": {
    "sections": 7,
    "cards": 35
  },
  "routes": {
    "html": "https://newruntime.com/learn/fastapi-langgraph-langchain-starter-kit/",
    "json": "https://newruntime.com/learn/fastapi-langgraph-langchain-starter-kit.json"
  },
  "sections": [
    {
      "number": "01",
      "title": "What We Build",
      "intro": "",
      "cards": [
        {
          "number": "01",
          "title": "Starter kit is a small service that already looks like a live product.",
          "tag": "Frame.",
          "description": "The idea is not to show 500 files, but to assemble a minimum set that is already fairly similar to production: there are ‘FastAPI’, service layer, compiled graph, persistence, streaming, ‘resume’, health endpoints, Docker and tests. After that, you can already increase the domain logic.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "START COMPLEX\n\npublic HTTP API\n  ↓\nservice layer\n  ↓\nLangGraph runtime\n  ↓\nLangChain components\n  ↓\nexternal persistence + external tools"
            }
          ]
        },
        {
          "number": "02",
          "title": "The starter kit has three modes: “chat”, “stream”, “resume”.",
          "tag": "Frame.",
          "description": "If you make only one sync endpoint, the project will look like a demo. As soon as streaming and durable human-in-the-loop paths through ‘resume’ appear, code behaves like a real service rather than a playground.",
          "blocks": [
            {
              "kind": "list",
              "label": "minimum",
              "items": [
                "sync chat",
                "streaming",
                "interrupt / resume"
              ]
            }
          ]
        },
        {
          "number": "03",
          "title": "Startup project should be stateless from above and durable from below",
          "tag": "Frame.",
          "description": "This is one of the main engineering ideas. An HTTP process doesn’t have to be the only place where a conversation lives. It is better to keep the service as stateless as possible, and the thread state and memory are transferred to external persistence, so that you can experience restarts and scaling.",
          "blocks": [
            {
              "kind": "code",
              "label": "briefly",
              "language": "text",
              "value": "web process:\n  preferably disposable and replaceable\n\nconversation state:\n  live outside the process"
            }
          ]
        },
        {
          "number": "04",
          "title": "Starter kit is more important than perfect business logic.",
          "tag": "Frame.",
          "description": "At the start, it is more important for you to learn how to properly separate layers: input API, config, state/context, tools, graph build, service orchestration, persistence, tests. If these boundaries are correct, then you can easily change prompts, tools and even entire graph flow without destroying the base.",
          "blocks": [
            {
              "kind": "list",
              "label": "layered",
              "items": [
                "api",
                "agent",
                "infra",
                "tests"
              ]
            }
          ]
        },
        {
          "number": "05",
          "title": "One good starter kit is better than five laptop examples.",
          "tag": "Frame.",
          "description": "In production, most of the time is eaten not by models, but by glue code and lifecycle: service start, config, resources, streaming, errors, resumes, health checks and tracing. Starter kit is valuable because it shows these boring but real things at once.",
          "blocks": [
            {
              "kind": "list",
              "label": "That's why it's here",
              "items": [
                "lifespan",
                "app.state",
                "Docker",
                "tests"
              ]
            }
          ]
        }
      ]
    },
    {
      "number": "02",
      "title": "File Starter Kit",
      "intro": "",
      "cards": [
        {
          "number": "06",
          "title": "One of the most normal minimum repo trees",
          "tag": "Starter",
          "description": "This structure is already sufficient for a live microservice and is not overloaded. There's a clear input, a separate agent layer, an infrastructure layer and tests.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "REPO TREE\n\napp/\n  main.py\n  core/\n    settings.py\n    logging.py\n  api/\n    deps.py\n    routers/\n      chat.py\n      health.py\n  agent/\n    contracts/\n      state.py\n      context.py\n      schemas.py\n    components/\n      models.py\n      prompts.py\n      tools.py\n      middleware.py\n    graph/\n      nodes.py\n      routes.py\n      build.py\n    services/\n      chat_service.py\n      stream_service.py\n  infra/\n    persistence.py\n    mcp_clients.py\ntests/\nDockerfile\npyproject.toml"
            }
          ]
        },
        {
          "number": "07",
          "title": "pyproject.toml - fix the stack immediately",
          "tag": "Starter",
          "description": "In the starting skeleton, it is useful to immediately fix the dependencies so that the project is reproducible. This service usually requires fastapi, uvicorn, langchain, langgraph, pydantic, pydantic-settings and test stack.",
          "blocks": [
            {
              "kind": "code",
              "label": "minimum",
              "language": "text",
              "value": "dependencies = [\n  \"fastapi\",\n  \"uvicorn[standard]\",\n  \"langchain\",\n  \"langgraph\",\n  \"pydantic\",\n  \"pydantic-settings\",\n]\n\ndev = [\n  \"pytest\",\n  \"httpx\",\n]"
            }
          ]
        },
        {
          "number": "08",
          "title": "`app/` - application, `tests/` - separate external view",
          "tag": "Starter",
          "description": "It seems trivial, but separation is very helpful. `tests/` shall not be embedded in graph modules. They look at the system from the outside and check HTTP, service wiring and graph behavior as separate layers.",
          "blocks": [
            {
              "kind": "list",
              "label": "Why is it useful?",
              "items": [
                "less connectivity",
                "Tests closer to actual use"
              ]
            }
          ]
        },
        {
          "number": "09",
          "title": "This is where the naming convention comes in.",
          "tag": "Starter",
          "description": "If you don't immediately decide where you have 'thread id', where 'run id', as routers and service methods are called, then things get confusing. Starter kit is also useful because it sets the project language: “chat”, “stream”, “resume”, “health”, “build graph”, “RequestContext”, “AgentState”.",
          "blocks": [
            {
              "kind": "code",
              "label": "nameplate",
              "language": "text",
              "value": "HTTP:\n  /v1/chat\n  /v1/chat/stream\n  /v1/threads/{thread_id}/resume\n\nPython:\n  ChatRequest\n  ChatResponse\n  ChatService\n  build_graph\n  RequestContext\n  AgentState"
            }
          ]
        },
        {
          "number": "10",
          "title": "Starter kit is better done as a single repository, not as a zoo microservices",
          "tag": "Starter",
          "description": "At the start, it is more useful to assemble one quality service with all layers than it is premature to break everything down into “api-service”, “graph-service”, “tool-service”, “mcp-service”. You will always have time to divide into several services later, when real operational pain appears.",
          "blocks": [
            {
              "kind": "list",
              "label": "better off",
              "items": [
                "repo",
                "One deployable service",
                "MCP can be added as an option later."
              ]
            }
          ]
        }
      ]
    },
    {
      "number": "03",
      "title": "Core Files and Start of Service",
      "intro": "",
      "cards": [
        {
          "number": "11",
          "title": "‘core/settings.py’ – Config must come from env, not code",
          "tag": "Core",
          "description": "Almost any living system quickly begins to live in different environments: local, staging, prod. Therefore, the settings of the model, DSN, timeouts and flags are better put in the settings model, rather than smeared into modules.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "class Settings(BaseSettings):\n    app_name: str = \"starter-agent\"\n    model_name: str = \"provider:model\"\n    api_prefix: str = \"/v1\"\n    log_level: str = \"INFO\"\n    request_timeout_s: int = 60\n    database_url: str\n\ndef get_settings() -> Settings:\n    return Settings()"
            }
          ]
        },
        {
          "number": "12",
          "title": "'main.py' must collect resources through 'lifespan'",
          "tag": "Core",
          "description": "This is one of the main production patterns of FastAPI. At the start of the application, you once create settings, persistence, compiled graph and service objects, fold them into “app.state”, and the routes then only get the finished objects.",
          "blocks": [
            {
              "kind": "code",
              "label": "skeleton",
              "language": "sql",
              "value": "from contextlib import asynccontextmanager\n\n@asynccontextmanager\nasync def lifespan(app: FastAPI):\n    settings = get_settings()\n    checkpointer, store = await build_persistence(settings)\n    graph = build_graph(settings=settings, checkpointer=checkpointer, store=store)\n    app.state.chat_service = ChatService(graph=graph, settings=settings)\n    yield\n\napp = FastAPI(lifespan=lifespan)\napp.include_router(chat_router)\napp.include_router(health_router)"
            }
          ]
        },
        {
          "number": "13",
          "title": "`api/deps.py' - a place for service injection and request context",
          "tag": "Core",
          "description": "Dependencies in FastAPI is convenient to use not only for auth, but also for access to objects from \"app.state\". It is also good to build a request-scoped context, which then falls into the graph runtime.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "def get_chat_service(request: Request) -> ChatService:\n    return request.app.state.chat_service\n\ndef get_request_context() -> RequestContext:\n    return RequestContext(\n        user_id=\"demo-user\",\n        tenant_id=\"demo-tenant\",\n    )"
            }
          ]
        },
        {
          "number": "14",
          "title": "“core/logging.py” should be added immediately, even if it is small.",
          "tag": "Core",
          "description": "If the logging is not centralized at the beginning, then it spreads through the “print()” and random “basicConfig”. For a starter kit, one place is enough to configure the format, level, and correlation fields like ‘request id’ and ‘thread id’.",
          "blocks": [
            {
              "kind": "list",
              "label": "logs are useful",
              "items": [
                "request_id",
                "thread_id",
                "run_id"
              ]
            }
          ]
        },
        {
          "number": "15",
          "title": "In the starter kit, it is already useful to distinguish “RequestContext” separately from “AgentState”.",
          "tag": "Core",
          "description": "This distinction is very disciplined. “RequestContext” contains immutable launch data: who is the user, which tenant, what rights. AgentState is an evolving state of execution. If you mix them, the count will begin to live with too many duties at once.",
          "blocks": [
            {
              "kind": "code",
              "label": "formula",
              "language": "tex",
              "value": "context = who started the run and what conditions\nWhat happened inside the run?"
            }
          ]
        }
      ]
    },
    {
      "number": "04",
      "title": "Contracts, Components and Graph",
      "intro": "",
      "cards": [
        {
          "number": "16",
          "title": "*contracts/state.py - the main contract inside runtime",
          "tag": "Graph",
          "description": "The state of the graph should be short, understandable and suitable for serialization. For a starter kit, you almost always need “messages”, a pair of domain fields and a flag for human review.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "sql",
              "value": "from typing_extensions import TypedDict, Annotated\nfrom langgraph.graph.message import add_messages\n\nclass AgentState(TypedDict):\n    messages: Annotated[list, add_messages]\n    answer: dict | None\n    review_required: bool | None\n    metadata: dict"
            }
          ]
        },
        {
          "number": "17",
          "title": "`components/models.py`, `tools.py`, `middleware.py` is a layer of LangChain building blocks",
          "tag": "Graph",
          "description": "This is where the model, tools and middleware live. Even if you have orchestration by hand on 'LangGraph', this layer is conveniently assembled according to the canons of LangChain: 'init chat model', '@tool', 'ToolRuntime', '@wrap tool call', '@dynamic prompt' and structured output contracts.",
          "blocks": [
            {
              "kind": "code",
              "label": "tool",
              "language": "python",
              "value": "@tool\ndef get_profile(runtime: ToolRuntime[RequestContext]) -> str:\n    user_id = runtime.context.user_id\n    return f\"profile for {user_id}\""
            }
          ]
        },
        {
          "number": "18",
          "title": "`graph/nodes.py` — node should return updates, not mutate state",
          "tag": "Graph",
          "description": "Even in the starter kit, it is useful to write nodes “adult” right away: read the state, called the internal component, returned the update. This facilitates replay, persistence and testing.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "async def agent_node(state: AgentState, runtime):\n    result = await inner_agent.ainvoke(\n        {\"messages\": state[\"messages\"]},\n        context=runtime.context,\n    )\n    return {\n        \"messages\": result[\"messages\"],\n        \"answer\": result.get(\"structured_response\"),\n    }"
            }
          ]
        },
        {
          "number": "19",
          "title": "`graph/routes.py` - routing is best taken separately",
          "tag": "Graph",
          "description": "Starter kit routing logic should also be isolated. If you need a review, go to the review; if not, finish. This small function gives a lot of order when the graph starts to grow.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "sql",
              "value": "from langgraph.graph import END\n\ndef route_after_agent(state: AgentState):\n    if state.get(\"review_required\"):\n        return \"review\"\n    return END"
            }
          ]
        },
        {
          "number": "20",
          "title": "'graph/build.py' should only do wiring and 'compile'",
          "tag": "Graph",
          "description": "This is where you collect state schema, nodes, routes and persistence. The cleaner the file, the easier it is to replace models, tools and nodes without breaking the architecture. In starter kit, it’s helpful to discipline yourself with this rule.",
          "blocks": [
            {
              "kind": "code",
              "label": "skeleton",
              "language": "python",
              "value": "def build_graph(settings, checkpointer, store):\n    builder = StateGraph(AgentState)\n    builder.add_node(\"agent\", agent_node)\n    builder.add_node(\"review\", review_node)\n    builder.add_edge(START, \"agent\")\n    builder.add_conditional_edges(\"agent\", route_after_agent)\n    builder.add_edge(\"review\", END)\n    return builder.compile(checkpointer=checkpointer, store=store)"
            }
          ]
        }
      ]
    },
    {
      "number": "05",
      "title": "Service Layer and Public API",
      "intro": "",
      "cards": [
        {
          "number": "21",
          "title": "‘services/chat service.py’ is where HTTP becomes graph invocation",
          "tag": "API",
          "description": "The service layer is designed to keep routers unaware of the 'RunnableConfig', 'thread id', 'Command(resume=...)' and shape of the final answer. This location links external request/response schemas and internal runtime.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "class ChatService:\n    def __init__(self, graph, settings):\n        self.graph = graph\n        self.settings = settings\n\n    async def chat(self, req: ChatRequest, ctx: RequestContext):\n        config = {\"configurable\": {\"thread_id\": req.thread_id or new_thread_id()}}\n        result = await self.graph.ainvoke(\n            {\"messages\": [{\"role\": \"user\", \"content\": req.message}]},\n            config=config,\n            context=ctx,\n        )\n        return to_chat_response(result, config)"
            }
          ]
        },
        {
          "number": "22",
          "title": "`stream service.py` is a separate layer for `astream` and `StreamingResponse`",
          "tag": "API",
          "description": "Streaming is better not to hide inside the usual chat service. A separate service helps neatly form a generator, collect several ‘stream mode’ and decide how to code out tokens, updates and interrupts.",
          "blocks": [
            {
              "kind": "code",
              "label": "idea",
              "language": "python",
              "value": "async for item in graph.astream(\n    inputs,\n    config=config,\n    context=ctx,\n    stream_mode=[\"messages\", \"updates\"],\n):\n    yield encode_sse(item)"
            }
          ]
        },
        {
          "number": "23",
          "title": "'api/routers/chat.py' - three endpoints for starter kit",
          "tag": "API",
          "description": "The minimum set of public APIs for this service is very specific: \"POST /v1/chat\", \"POST /v1/chat/stream\", \"POST /v1/threads/{thread id}/resume\". That’s enough for a frontend, another service, or CLI to live with.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "API SURFACE\n\nPOST /v1/chat\nPOST /v1/chat/stream\nPOST /v1/threads/{thread_id}/resume\nGET  /healthz\nGET  /readyz"
            }
          ]
        },
        {
          "number": "24",
          "title": "*contracts/schemas.py should describe both requests and interrupt payload",
          "tag": "API",
          "description": "A common mistake is to type only the usual answer. But if you have a human review, you also have an interrupt contract. It is also better to formalize a separate scheme: pause type, message, action schema, resume payload.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "python",
              "value": "class ChatRequest(BaseModel):\n    thread_id: str | None = None\n    message: str\n\nclass ResumeRequest(BaseModel):\n    payload: dict\n\nclass InterruptPayload(BaseModel):\n    type: str\n    message: str\n    data: dict"
            }
          ]
        },
        {
          "number": "25",
          "title": "'health.py' is useful to do right away and 'readyz' to check real readiness",
          "tag": "API",
          "description": "‘healthz’ can simply say ‘the process is alive’. “readyz” is more useful to do a little smarter: check that the service object is collected, persistence is available and basic dependencies have risen. For starter kit, a simple but honest readiness check is enough.",
          "blocks": [
            {
              "kind": "list",
              "label": "difference",
              "items": [
                "healthz = process alive",
                "readyz = can serve traffic"
              ]
            }
          ]
        }
      ]
    },
    {
      "number": "06",
      "title": "Persistence, Docker and Life Outside",
      "intro": "",
      "cards": [
        {
          "number": "26",
          "title": "infra/persistence.py – one builder for checkpointer and store",
          "tag": "Deploy",
          "description": "Even if you use in-memory mode in local dev, it is useful to start a single module in the starter kit that can collect persistence resources. Then the path to durable execution in staging/prod remains direct, not chaotic.",
          "blocks": [
            {
              "kind": "code",
              "label": "idea",
              "language": "python",
              "value": "async def build_persistence(settings):\n    if settings.environment == \"local\":\n        return in_memory_checkpointer(), in_memory_store()\n    return durable_checkpointer(settings), durable_store(settings)"
            }
          ]
        },
        {
          "number": "27",
          "title": "If you have containers, it is often better to have one process per container.",
          "tag": "Deploy",
          "description": "FastAPI docs are a distinct reminder: in orchestrators like Kubernetes, one Uvicorn process per container and replica scaling is often preferred over a complex multiprocess inside. This simplifies memory model and lifecycle applications.",
          "blocks": [
            {
              "kind": "list",
              "label": "simple practice",
              "items": [
                "1 process / container",
                "scale via replicas",
                "More workers are needed outside of container orchestration"
              ]
            }
          ]
        },
        {
          "number": "28",
          "title": "Dockerfile for starter kit should be boring and predictable",
          "tag": "Deploy",
          "description": "You don't need platform art. More important is a fast, clear image that puts dependencies, copies the code and runs Uvicorn. Difficulty can always be added, but the chaos in the container from the beginning only gets in the way.",
          "blocks": [
            {
              "kind": "code",
              "label": "example",
              "language": "sql",
              "value": "FROM python:3.12-slim\nWORKDIR /app\nCOPY pyproject.toml .\nRUN pip install .\nCOPY app ./app\nCMD [\"uvicorn\", \"app.main:app\", \"--host\", \"0.0.0.0\", \"--port\", \"8000\"]"
            }
          ]
        },
        {
          "number": "29",
          "title": "Starter kit should already experience 'thread id' across restarts",
          "tag": "Deploy",
          "description": "If the old 'thread id' doesn't resolve after the restart, you don't have a service yet, you have a demo. Even a minimal production-oriented build should be built so that the conversation state can continue after the process is upgraded or dropped.",
          "blocks": [
            {
              "kind": "list",
              "label": "ripeness",
              "items": [
                "resume",
                "shared persistence",
                "state only in RAM"
              ]
            }
          ]
        },
        {
          "number": "30",
          "title": "Starter kit: HTTP, service, graph, interrupt",
          "tag": "Deploy",
          "description": "The minimum working set of tests should cover several layers. API tests verify contracts. Service tests check the conversion of HTTP payload to runtime call. Graph tests test routing and persistent behavior. A separate test is required for “interrupt/resume”.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "TEST SET\n\ntest_chat_route.py\ntest_resume_route.py\ntest_chat_service.py\ntest_graph_flow.py\ntest_interrupt_resume.py"
            }
          ]
        }
      ]
    },
    {
      "number": "07",
      "title": "MCP and Where to Grow Next",
      "intro": "",
      "cards": [
        {
          "number": "31",
          "title": "*infra/mcp clients.py is a normal extension point for tools",
          "tag": "MCP",
          "description": "If later you want to go to the MCP server instead of direct SDKs, it is more convenient to provide a separate module-adapter in advance. Then LangChain tools continue to look the same, and transport and protocol logic does not follow in graph code.",
          "blocks": [
            {
              "kind": "code",
              "label": "idea",
              "language": "text",
              "value": "tool\n→ repository / adapter\n→ mcp_client\n→ remote MCP server"
            }
          ]
        },
        {
          "number": "32",
          "title": "www.mcp server.py add when your service becomes a tool/data plane",
          "tag": "MCP",
          "description": "You can add an MCP server if you want an IDE, external hosts, or other agents to use your capabilities. But this is the second interface of the product, not a mandatory part of the basic HTTP service.",
          "blocks": [
            {
              "kind": "code",
              "label": "Minimal logic",
              "language": "python",
              "value": "mcp = FastMCP(\"starter-service\")\n\n@mcp.tool\ndef search_kb(query: str) -> str:\n    ...\n\n@mcp.resource(\"thread://{thread_id}\")\ndef get_thread(thread_id: str) -> str:\n    ..."
            }
          ]
        },
        {
          "number": "33",
          "title": "The evolution of starter kit usually looks like this.",
          "tag": "Outcome",
          "description": "First, you need to make a minimum durable service. Then you add better auth, observability, queue for long jobs, MCP adapters, richer tools, more complex graph, evals and rate limiting. But the foundation remains the same.",
          "blocks": [
            {
              "kind": "code",
              "label": "growth-track",
              "language": "text",
              "value": "1. basic chat / stream / resume\n2. durable persistence\n3. auth + readiness + structured logs\n4. background jobs + queue\n5. MCP client/server if needed\n6. evals, guardrails, tracing, budgets"
            }
          ]
        },
        {
          "number": "34",
          "title": "Frequent anti-patterns starter kit",
          "tag": "Outcome",
          "description": "The most common problems are predictable: giant `main.py`, graph compilation for each request, state only in RAM, no resume endpoint, routes with business logic, tools with global singletons and zero tests on the path.",
          "blocks": [
            {
              "kind": "list",
              "label": "red flags",
              "items": [
                "all in one file",
                "No lifespan.",
                "There is no durable persistence",
                "No 'resume' API"
              ]
            }
          ]
        },
        {
          "number": "35",
          "title": "The main idea of the volume",
          "tag": "Outcome",
          "description": "A good starter kit for ‘FastAPI + LangGraph + LangChain’ is not a set of fun libraries, but a small backend with the right boundaries. If you have a ‘lifespan’, ‘app.state’, typed contracts, compiled graph, service layer, durable persistence and an honest ‘chat/stream/resume’ API, then you no longer have a demo, but the basis of a live external service.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "Summary formula\n\nFastAPI\n  + clean repo layout\n  + LangChain components\n  + LangGraph runtime\n  + external persistence\n  + chat/stream/resume contract\n  =\nStarter kit that can be developed in production service"
            }
          ]
        }
      ]
    }
  ]
}