{
  "type": "new-runtime-knowledge-guide",
  "version": 1,
  "generated_at": "2026-07-23",
  "volume": 3,
  "slug": "llm-frameworks",
  "title": "The LLM Framework Ecosystem",
  "description": "Key abstractions, architectural patterns, pitfalls and selection criteria for the main frameworks of the LLM ecosystem. Each one is disassembled atomically.",
  "track": {
    "slug": "application-engineering",
    "title": "Application Engineering",
    "route": "https://newruntime.com/learn/tracks/application-engineering/",
    "position": 1
  },
  "counts": {
    "sections": 8,
    "cards": 28
  },
  "routes": {
    "html": "https://newruntime.com/learn/llm-frameworks/",
    "json": "https://newruntime.com/learn/llm-frameworks.json"
  },
  "sections": [
    {
      "number": "01",
      "title": "LangChain — Orchestration Foundation",
      "intro": "",
      "cards": [
        {
          "number": "01",
          "title": "LangChain - key abstractions",
          "tag": "orchestration",
          "description": "LangChain is an abstraction layer on top of the LLM API. His philosophy: everything is a component, components are connected through pipes into chains (LCEL). The main value is not in the “magic”, but in uniform interfaces for 100+ providers and tools.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "Key primitives:\n\nPromptTemplate → prompt template with variables\nChatModel → wrapper for LLM API (OpenAI, Anthropic, ...)\nOutputParser → output parsing: str/JSON/Pydantic\nRetriever → search abstraction (vector, BM25, hybrid)\nTool → called tool with description for LLM\nMemory → conversation context storage\nChain → an outdated way to connect components\nLCEL pipe → prompt | model | parser ← modern way"
            },
            {
              "kind": "list",
              "label": "When to buy LangChain",
              "items": [
                "rapid prototype RAG",
                "need integration with 50+ providers",
                "the team already knows LangChain",
                "complex stateful agent → LangGraph",
                "production without overhead → pure SDK"
              ]
            }
          ]
        },
        {
          "number": "02",
          "title": "LCEL — LangChain Expression Language",
          "tag": "orchestration",
          "description": "A modern way to assemble chains is through the | operator. Provides streaming, batching, tracing and async out of the box for any chain. Replaces the old Chain classes.",
          "blocks": [
            {
              "kind": "code",
              "label": "LCEL Syntax",
              "language": "typescript",
              "value": "# Simple RAG chain\nchain = (\n    {\"context\": retriever, \"question\": RunnablePassthrough()}\n    | prompt_template\n    | chat_model\n    | StrOutputParser()\n)\n\n# Everything works automatically:\nchain.invoke(query) # synchronously\nchain.stream(query) # token streaming\nchain.batch([q1, q2, q3]) # in parallel\nawait chain.ainvoke(query) #async"
            }
          ]
        },
        {
          "number": "03",
          "title": "LangChain - pitfalls",
          "tag": "pitfalls",
          "description": "LangChain is often criticized for its abstractions that hide what's going on under the hood - and this is what causes most problems in production.",
          "blocks": [
            {
              "kind": "code",
              "label": "Common problems",
              "language": "text",
              "value": "✗ Magic prompts\n  Default prompts inside Chain classes\n  → unpredictable, impossible to control\n\n✗ Hidden LLM challenges\n  RetrievalQA does extra. calls unnoticed\n  → unexpected costs, hidden latency\n\n✗ Versioning\n  Frequent breaking changes between minor versions\n  → pin version, test updates carefully\n\n✓ Solution: LCEL + explicit prompts + tracing everything"
            }
          ]
        }
      ]
    },
    {
      "number": "02",
      "title": "LangGraph — Stateful Agent Graphs",
      "intro": "",
      "cards": [
        {
          "number": "04",
          "title": "LangGraph - philosophy and abstractions",
          "tag": "graph / state",
          "description": "LangGraph is a way to describe agents as state machines with explicit state. Not chains, but a graph: nodes are actions/LLM calls, edges are transitions between states. This gives something that LangChain does not: loops, branches, human-in-the-loop, persistent state.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "Agent graph: ReAct with human-in-the-loop\n\n      [START]\n         ↓\n      [agent] ← LLM decides: call the tool or give a response\n       ↙ ↘\n[tools] [END]\n   ↓\n[agent] ← tool result → next solution\n\nKey idea: cycles are possible - the agent can spin\nas much as needed until it solves the problem or hits the limit"
            },
            {
              "kind": "list",
              "label": "When to take LangGraph instead of LangChain",
              "items": [
                "agent with loops and branching",
                "need human-in-the-loop (pause, confirmation)",
                "persistent state between sessions",
                "multi-agent with clear orchestration"
              ]
            }
          ]
        },
        {
          "number": "05",
          "title": "State + Reducer is the heart of LangGraph",
          "tag": "graph / state",
          "description": "Each node reads and writes to a common State. Reducer determines how the state is updated when there is a conflict (append vs replace). This makes the graph reproducible and debuggable.",
          "blocks": [
            {
              "kind": "code",
              "label": "Example State",
              "language": "sql",
              "value": "from typing import Annotated\nfrom langgraph.graph import add_messages\n\nclass AgentState(TypedDict):\n    messages: Annotated[list, add_messages]\n    # add_messages = reducer: append, not replace\n    retrieved_docs: list[str]\n    iteration_count: int\n    final_answer: str | None\n\n# each node receives state and returns patch:\ndef agent_node(state: AgentState) -> dict:\n    response = llm.invoke(state[\"messages\"])\n    return {\"messages\": [response]}"
            }
          ]
        },
        {
          "number": "06",
          "title": "Conditional Edges - flow control",
          "tag": "graph / state",
          "description": "Edges can be conditional: a function looks at the state and decides which node to go to next. This allows you to build complex logic without imperative code inside nodes.",
          "blocks": [
            {
              "kind": "code",
              "label": "Branching example",
              "language": "python",
              "value": "def should_continue(state) -> str:\n    last_msg = state[\"messages\"][-1]\n    if last_msg.tool_calls:\n        return \"tools\" # → tools node\n    elif state[\"iteration_count\"] > 5:\n        return \"force_end\" # → force stop\n    else:\n        return \"end\" # → final response\n\ngraph.add_conditional_edges(\"agent\", should_continue,\n    {\"tools\": \"tools\", \"end\": END, \"force_end\": END})"
            }
          ]
        },
        {
          "number": "07",
          "title": "Checkpointing - persistence and HITL",
          "tag": "graph / state",
          "description": "LangGraph can save state after each step in the database (Postgres, SQLite, Redis). This gives: continuation after a failure, human-in-the-loop (pause → wait for confirmation → continue), time travel (rollback to the previous state).",
          "blocks": [
            {
              "kind": "code",
              "label": "Human-in-the-loop pattern",
              "language": "python",
              "value": "checkpointer = PostgresSaver(conn)\ngraph = graph.compile(\n    checkpointer=checkpointer,\n    interrupt_before=[\"dangerous_action\"] ← pause here\n)\n\n# 1. Run before interrupt\ngraph.invoke(input, config={\"thread_id\": \"t1\"})\n# 2. Show the person a pending action\nstate = graph.get_state(config)\n# 3. The person approves → continue\ngraph.invoke(None, config) # resume"
            }
          ]
        }
      ]
    },
    {
      "number": "03",
      "title": "LlamaIndex — RAG & Data Layer",
      "intro": "",
      "cards": [
        {
          "number": "08",
          "title": "LlamaIndex - philosophy and abstractions",
          "tag": "rag framework",
          "description": "LlamaIndex is focused on one thing: connecting LLM with your data. While LangChain is a general-purpose orchestration tool, LlamaIndex is a specialized data layer. The main primitive is Index and QueryEngine.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "sql",
              "value": "LlamaIndex abstractions:\n\nDocument → raw document + metadata\nNode → document chunk (atomic index unit)\nIndex → search structure (VectorStore, Summary, KG...)\nRetriever → strategy for retrieving from Index\nNodePostprocessor → reranker, filters, metadata enrichment\nQueryEngine → retriever + LLM + response synthesis\nChatEngine → QueryEngine + dialogue history\nPipeline → IngestionPipeline for indexing"
            },
            {
              "kind": "list",
              "label": "When LlamaIndex beats LangChain",
              "items": [
                "complex RAG logic (parent-doc, multi-hop)",
                "many data sources (PDF, Notion, SQL, ...)",
                "need advanced indexes (KG, Summary)",
                "orchestration is more difficult → LangGraph is better"
              ]
            }
          ]
        },
        {
          "number": "09",
          "title": "Types of indexes - not just vectors",
          "tag": "rag framework",
          "description": "LlamaIndex offers several types of indexes for different tasks. The choice of index is an architectural decision that affects the quality of answers more than the choice of model.",
          "blocks": [
            {
              "kind": "code",
              "label": "Types of Indexes",
              "language": "text",
              "value": "VectorStoreIndex standard RAG for embeddings\nSummaryIndex summarization of the entire corpus sequentially\nKeywordTableIndex keyword extraction → keyword search\nKnowledgeGraph entity graph → search by relationships\nDocumentSummary summary per doc → document selection → details\nTreeIndex hierarchical tree of summations\n\nMulti-Index: different indexes for different types of questions"
            }
          ]
        },
        {
          "number": "10",
          "title": "IngestionPipeline - correct indexing",
          "tag": "rag framework",
          "description": "Document processing pipeline before indexing: transformations, splitters, metadata, deduplication. With caching - re-indexing skips already processed documents.",
          "blocks": [
            {
              "kind": "code",
              "label": "Example",
              "language": "text",
              "value": "pipeline = IngestionPipeline(\n    transformations=[\n        SentenceSplitter(chunk_size=512, overlap=64),\n        TitleExtractor(), # LLM metadata\n        QuestionsAnsweredExtractor(), # hypothetical questions\n        OpenAIEmbedding(),\n    ],\n    vector_store=chroma_store,\n    cache=IngestionCache(), # do not reindex old\n)"
            }
          ]
        }
      ]
    },
    {
      "number": "04",
      "title": "Multi-Agent Frameworks — CrewAI · AutoGen · Agno",
      "intro": "",
      "cards": [
        {
          "number": "11",
          "title": "CrewAI - roles, tasks, teams",
          "tag": "multi-agent",
          "description": "CrewAI models multi-agent systems as a team with roles. Each agent is a role + a set of tools + a goal. Tasks are assigned to agents and executed sequentially or in parallel. Very readable declarative API - good for business processes where roles are obvious.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "CrewAI abstractions:\n\nAgent = role + backstory + goal + tools + llm\nTask = description + expected_output + agent\nCrew = [agents] + [tasks] + process + memory\n\nProcess types:\n  sequential agent 1 → agent 2 → agent 3\n  hierarchical manager LLM → delegates tasks to agents\n                             ↑ better quality control\n\nCrew example for WB seller analytics:\nresearcher → collect data on competitors\nanalyst → build BCG matrix\nwriter → write a report in PDF"
            },
            {
              "kind": "list",
              "label": "When CrewAI",
              "items": [
                "business processes with clear roles",
                "quick start of a multi-agent prototype",
                "complex graph → LangGraph more precise",
                "hidden prompts inside the framework"
              ]
            }
          ]
        },
        {
          "number": "12",
          "title": "AutoGen — conversational multi-agent",
          "tag": "multi-agent",
          "description": "AutoGen (Microsoft) is based on the idea of interlocutor agents: agents exchange messages with each other, like in a chat. GroupChat allows multiple agents to discuss a task until consensus.",
          "blocks": [
            {
              "kind": "code",
              "label": "Key primitives",
              "language": "text",
              "value": "ConversableAgent base agent with LLM + human_input_mode\nAssistantAgent LLM agent without human input\nUserProxyAgent executes code, optionally requests a person\nGroupChat multiple agents + next selection manager\nGroupChatManager LLM decides who speaks next\n\nPattern: Coder + Reviewer + Executor\nCoder writes → Reviewer criticizes → Executor launches"
            }
          ]
        },
        {
          "number": "13",
          "title": "Agno (ex-Phidata) - lightweight agents",
          "tag": "multi-agent",
          "description": "Agno is a minimalistic framework for agents: minimum abstractions, maximum control. Agent = LLM + tools + storage. No magic. Easy to read and easy to debug. Built-in support for memory and knowledge bases.",
          "blocks": [
            {
              "kind": "code",
              "label": "Minimal example",
              "language": "text",
              "value": "agent = Agent(\n    model=Claude(id=\"claude-sonnet-4\"),\n    tools=[DuckDuckGoSearch(), PythonTool()],\n    memory=AgentMemory(db=SqliteMemoryDb()),\n    knowledge=PDFKnowledgeBase(path=\"docs/\"),\n    instructions=[\"Answer in Russian\", \"Use numbers\"],\n    show_tool_calls=True,\n    markdown=True,\n)\nagent.print_response(\"WB 2025 Market Analysis\")"
            },
            {
              "kind": "list",
              "label": "Applicable",
              "items": [
                "transparent simple agents",
                "quick start without learning the framework"
              ]
            }
          ]
        },
        {
          "number": "14",
          "title": "Comparison of multi-agent frameworks",
          "tag": "comparison",
          "description": "The choice between frameworks is determined not by “which is more powerful”, but by “which abstractions coincide with your task.”",
          "blocks": []
        }
      ]
    },
    {
      "number": "05",
      "title": "Eval Frameworks — RAGAS · ARES · DeepEval · Braintrust",
      "intro": "",
      "cards": [
        {
          "number": "15",
          "title": "RAGAS - standard for RAG eval",
          "tag": "evals",
          "description": "RAGAS is a specialized eval framework for RAG systems. Uniqueness: reference-free metrics - you don’t need a dataset with GT answers for most metrics. Assessment via LLM judge with atomic approvals.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "What RAGAS evaluates and how:\n\nFaithfulness\n  answer → atomic statements → NLI vs context\n  = proportion of statements supported by context\n\nAnswer Relevance\n  answer → LLM generates N questions → embed → cos(questions, query)\n  = to what extent the answer falls within the topic of the question\n\nContext Precision\n  Each chunk → LLM : “Is LLM useful for an answer?” → precision@k\n  = share of useful chunks among retrieved\n\nContext Recall [requires GT]\n  GT → atomic statements → NLI vs context\n  = proportion of GT facts present in the context"
            },
            {
              "kind": "list",
              "label": "Applicable",
              "items": [
                "RAG assessment without GT dataset",
                "diagnostics: retrieval vs generation problem",
                "expensive for LLM-judge on large datasets"
              ]
            }
          ]
        },
        {
          "number": "16",
          "title": "DeepEval - pytest for LLM",
          "tag": "evals",
          "description": "DeepEval integrates into pytest and allows you to write eval tests as unit tests. Large library of ready-made metrics + support for custom ones. Fits well into CI/CD.",
          "blocks": [
            {
              "kind": "code",
              "label": "Syntax",
              "language": "python",
              "value": "@pytest.mark.parametrize(\"case\", test_cases)\ndef test_rag_quality(case):\n    actual_output = rag_pipeline(case.input)\n\n    assert_test(\n        test_case=LLMTestCase(\n            input=case.input,\n            actual_output=actual_output,\n            retrieval_context=case.context\n        ),\n        metrics=[\n            FaithfulnessMetric(threshold=0.8),\n            AnswerRelevancyMetric(threshold=0.7),\n            HallucinationMetric(threshold=0.1),\n        ]\n    )"
            }
          ]
        },
        {
          "number": "17",
          "title": "Braintrust - eval as a product",
          "tag": "evals",
          "description": "Braintrust is a platform for eval with UI: launching experiments, comparing prompt versions, history of runs, drill-down on cases. Like Weights & Biases, but for LLM pipelines. The SDK works without a platform.",
          "blocks": [
            {
              "kind": "code",
              "label": "Experiment concept",
              "language": "typescript",
              "value": "Eval(\n    name=\"RAG pipeline v2 vs v3\",\n    data=dataset, # list {input, expected}\n    task=rag_pipeline, # function → output\n    scores=[ # metrics\n        Faithfulness,\n        AnswerRelevancy,\n        NumTokens, # cost tracking\n        Latency\n    ],\n    experiment_name=\"v3-with-rerank\"\n)"
            }
          ]
        },
        {
          "number": "18",
          "title": "ARES - autonomous eval system",
          "tag": "evals",
          "description": "ARES (Stanford) is a system that trains small LM classifiers for evaluation instead of expensive LLM judges. You need a small number of human tags for training - further quickly and cheaply at any volume.",
          "blocks": [
            {
              "kind": "code",
              "label": "Principle",
              "language": "text",
              "value": "# One-time use:\nhuman_labels = 150 labeled examples\nsynthetic_data = LLM.generate_training_data(docs)\nclassifier = finetune(small_lm, human_labels + synthetic)\n\n# Next - cheap and fast:\nscores = classifier.score(10_000_examples)\n# vs LLM-judge: 100x cheaper, comparable quality"
            },
            {
              "kind": "list",
              "label": "Applicable",
              "items": [
                "large volumes of assessment",
                "eval cost reduction"
              ]
            }
          ]
        }
      ]
    },
    {
      "number": "06",
      "title": "Observability — Langfuse · Phoenix Arize · Helicone",
      "intro": "",
      "cards": [
        {
          "number": "19",
          "title": "Langfuse — open-source LLM observability",
          "tag": "observability",
          "description": "Langfuse is the most complete open-source tool: traces, eval, datasets, prompt management, cost tracking - in one place. Self-hosted via Docker. Integration via SDK or OpenTelemetry.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "sql",
              "value": "What Langfuse can do:\n\nTraces & Spans pipeline visualization, waterfall latency\nScores eval-assessments attached to trace/span\nDatasets golden datasets + experiments (A/B eval)\nPrompt Management versioning prompts + fetch at runtime\nCost Tracking tokens and $ by model, user, feature\nUsers & Sessions grouping traces by user\nEvaluators LLM-as-judge directly in the interface\n\nIntegration: 2 lines of code\nfrom langfuse.openai import openai # drop-in replacement"
            },
            {
              "kind": "list",
              "label": "Strengths",
              "items": [
                "self-hosted, GDPR-compliant",
                "prompt management + tracing in one",
                "integration with LangChain/LlamaIndex out of the box",
                "UI is slower than Arize at high volumes"
              ]
            }
          ]
        },
        {
          "number": "20",
          "title": "Phoenix Arize - ML observability for LLM",
          "tag": "observability",
          "description": "Arize Phoenix - from ML observability to LLM. Strengths: visualization of embeddings, drift detection at the vector space level, UMAP query projections. The best choice if you need to analyze retrieval quality visually.",
          "blocks": [
            {
              "kind": "code",
              "label": "Unique features vs Langfuse",
              "language": "sql",
              "value": "Embedding visualization\n  UMAP/TSNE projections: see query clusters\n  → find anomalies and OOD queries visually\n\nRetrieval analysis\n  scatter plot: query embeddings vs doc embeddings\n  → see where retrieval misses\n\nDrift detection\n  comparison of distributions between periods\n  → automatically finds data drift\n\nOpenInference\n  open tracing standard LLM (like OTEL)"
            }
          ]
        },
        {
          "number": "21",
          "title": "Helicone — gateway + observability",
          "tag": "observability",
          "description": "Helicone works as a proxy gateway between your code and the LLM API. Zero-instrumentation: you change one URL, you get all the metrics. Built-in caching, rate limiting, cost alerts.",
          "blocks": [
            {
              "kind": "code",
              "label": "Connection in 1 line",
              "language": "text",
              "value": "client = OpenAI(\n    base_url=\"https://oai.helicone.ai/v1\",\n    # ← instead of api.openai.com\n    default_headers={\n        \"Helicone-Auth\": f\"Bearer {HELICONE_KEY}\",\n        \"Helicone-Cache-Enabled\": \"true\", # cache\n        \"Helicone-User-Id\": user_id, # per-user\n    }\n)"
            },
            {
              "kind": "list",
              "label": "Applicable",
              "items": [
                "no time for instrumentation",
                "need cache + rate limit immediately",
                "less flexibility than Langfuse"
              ]
            }
          ]
        },
        {
          "number": "22",
          "title": "OpenTelemetry for LLM - tracing standard",
          "tag": "observability",
          "description": "OpenInference (Arize) and OpenLLMetry (Traceloop) extend OpenTelemetry to be LLM-specific. The span attributes are standardized: input/output, model, tokens. Allows you to send traces to any OTEL-compatible backend.",
          "blocks": [
            {
              "kind": "code",
              "label": "Standard attributes OTEL LLM span",
              "language": "text",
              "value": "gen_ai.system = \"anthropic\"\ngen_ai.request.model = \"cloud-sonnet-4\"\ngen_ai.usage.input_tokens = 1240\ngen_ai.usage.output_tokens = 312\ngen_ai.prompt = [messages]\ngen_ai.completion = [response]\n# → goes to Langfuse / Jaeger / Grafana Tempo"
            }
          ]
        }
      ]
    },
    {
      "number": "07",
      "title": "DSPy — Prompt Optimization as Code",
      "intro": "",
      "cards": [
        {
          "number": "23",
          "title": "DSPy - philosophy: program, don't rush",
          "tag": "prompt optimization",
          "description": "DSPy reverses the approach: instead of writing prompts manually, you describe the task signature (inputs → outputs) and quality metrics, and the optimizer itself finds the best prompts, few-shot examples and call chains. This is a compiler for LLM programs.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "typescript",
              "value": "Traditional approach:\n  the prompt was written manually → works on GPT-4 → the model was changed → the prompt is broken\n\nDSPy approach:\n  signature = \"question → answer\" # what we do\n  metric = lambda pred, gt: f1(pred, gt) # how to measure\n  optimizer.compile(program, metric, trainset)\n  # DSPy generates prompts itself + few-shot examples\n  # re-optimizes when changing models in minutes\n\nKey primitives:\nSignature InputField + OutputField = declarative specification\nModule dspy.Predict / dspy.ChainOfThought / dspy.ReAct\nOptimizer BootstrapFewShot/MIPRO/BayesianSignatureOptimizer\nMetric function (prediction, ground_truth) → float"
            },
            {
              "kind": "list",
              "label": "When DSPy beats manual prompting",
              "items": [
                "the base model changes frequently",
                "there is a dataset with a metric - the prompt can be optimized",
                "multi-hop tasks with multiple LLM calls",
                "no dataset → DSPy will not help",
                "high entry threshold"
              ]
            }
          ]
        },
        {
          "number": "24",
          "title": "DSPy Modules - building blocks",
          "tag": "prompt optimization",
          "description": "DSPy modules are typed LLM call patterns. Each module is independently optimized and can be combined into programs.",
          "blocks": [
            {
              "kind": "code",
              "label": "Main modules",
              "language": "text",
              "value": "dspy.Predict\n  direct call by signature, without CoT\n\ndspy.ChainOfThought\n  adds a reasoning field → response\n  # prompt with CoT is generated automatically\n\ndspy.ReAct\n  agent with tools based on the ReAct pattern\n\ndspy.Retrieve\n  search in the connected knowledge base\n\ndspy.MultiChainComparison\n  several reasoning paths → the best"
            }
          ]
        },
        {
          "number": "25",
          "title": "TextGrad & OPRO - DSPy alternatives",
          "tag": "prompt optimization",
          "description": "TextGrad (Stanford) - \"gradient descent\" for text: LLM calculates the \"gradient\" (critique) and updates the prompt. OPRO (Google) - LLM as an optimizer: generates prompt candidates, evaluates, improves iteratively.",
          "blocks": [
            {
              "kind": "code",
              "label": "TextGrad idea",
              "language": "text",
              "value": "loss = metric(output, gt) # \"how bad\"\ngradient = LLM( # \"why is it bad\"\n    f\"Output: {output}\nLoss: {loss}\nWhat's wrong with the prompt?\"\n)\nnew_prompt = LLM( # \"fix\"\n    f\"Old prompt: {prompt}\nGradient: {gradient}\nImproved prompt:\"\n)"
            }
          ]
        }
      ]
    },
    {
      "number": "08",
      "title": "Navigating the ecosystem - when to take what",
      "intro": "",
      "cards": [
        {
          "number": "26",
          "title": "Decision Tree - choosing a framework",
          "tag": "navigation",
          "description": "Ask yourself these questions before choosing a framework. A wrong choice at the beginning costs correspondence later.",
          "blocks": [
            {
              "kind": "code",
              "label": "Diagram",
              "language": "text",
              "value": "What are you building?\n ├── RAG with data → LlamaIndex (indices) + LangChain (orchestration)\n ├── Stateful agent with cycles → LangGraph\n ├── Multi-agent business process → CrewAI (fast) / LangGraph (reliable)\n ├── Research/agent dialogue → AutoGen\n ├── Simple agent, control is important → Agno / pure SDK\n └── Prompts to optimize auto → DSPy\n\nWhat are you assessing?\n ├── RAG (without GT) → RAGAS\n ├── LLM tests in pytest → DeepEval\n ├── Experiments / A/B prompts → Braintrust\n └── Small team, all at once → Langfuse (datasets + experiments)\n\nWhat are you monitoring?\n ├── Traces + prompts + cost → Langfuse (self-hosted)\n ├── Embeddings drift + retrieval → Phoenix Arize\n └── Zero instrumentation, tomorrow → Helicone"
            }
          ]
        },
        {
          "number": "27",
          "title": "Antipattern - a framework for the sake of a framework",
          "tag": "pitfalls",
          "description": "The most common mistake: they take LangChain/LlamaIndex “because everyone uses it” - and get an abstraction layer that hides bugs, slows down debugging and creates a dependence on other people’s breaking changes.",
          "blocks": [
            {
              "kind": "code",
              "label": "When a framework is not needed",
              "language": "text",
              "value": "✗ One or two LLM challenges\n  → direct SDK (anthropic, openai) is better\n\n✗ The team does not know the framework\n  → time to study > time to write yourself\n\n✗ Specific logic in every step\n  → abstractions get in the way, you write workarounds\n\n✓ The framework is justified when:\n  → 10+ integrations are needed (providers, vector databases)\n  → complex stateful graph (LangGraph)\n  → the team already knows him"
            }
          ]
        },
        {
          "number": "28",
          "title": "Stack for 2025 - what really works",
          "tag": "recommendations",
          "description": "Community opinion based on the results of production experience: which combinations give the least number of problems with the greatest flexibility.",
          "blocks": [
            {
              "kind": "code",
              "label": "Recommended Stacks",
              "language": "text",
              "value": "Minimalistic (small team):\n  SDK + Instructor + Langfuse\n\nRAG product:\n  LlamaIndex + LangGraph (agent) + RAGAS + Langfuse\n\nMulti-agent enterprise:\n  LangGraph + LangChain tools + Braintrust + Arize\n\nResearch/Experiments:\n  DSPy + Langfuse + DeepEval\n\nMain principle:\n  start with the minimum → add frameworks\n  only when the pain without them is obvious"
            }
          ]
        }
      ]
    }
  ]
}