DSPy & GEPA: Implementation Cookbook

In conventional prompt engineering, task signature, formatting, reasoning strategy, few-shot examples, and model-specific hacks are mixed into one fragile text. DSPy lays this out in layers and allows you to change the layer without rewriting the entire system. That is why it is useful not only for optimization, but also for normal architecture.

DSPy’s official programming overview doesn’t start with optimizers, but with task design and pipeline. The right start: first decide what inputs and outputs the system needs, where retrieval or tools are needed, and then think about how to compile prompt internals.

The DSPy documentation clearly suggests a cyclical process: programme, assemble devset, define metrics, optimize, go back and change the pipeline itself if the metric or architecture is weak.

GEPA does not replace a DSPy program, but sits on top of it as a reflective optimizer. Its task is not to invent the entire architecture for you, but to improve the text components of an existing program using score, feedback and execution traces.

dspy.LM specifies model and generation defaults. Documentation shows work with OpenAI, Anthropic, Gemini, Vertex, Databricks, Ollama, local OpenAI-compatible endpoints and other LiteLLM providers. LM is cached by default, stores call history, usage and cost metadata, and can switch via dspy.configure and dspy.context.

DSPy: Signature is a declarative specification for input/output behavior. It's about describing what to do, not manually dictating models how to ask.

Class-based signatures are needed when you want to clearly define the meaning of the task: docstring, field descriptions, output limitations and complex types. DSPy supports not only 'str' but also 'int', 'bool', 'list[str]', 'dict[...]', 'optional[...]', custom types and special types like 'dspy'. Image' and dspy. History.

The basic data container in DSPy is ‘Example’. It is similar to dict but knows which fields are inputs. 'Prediction' is a special subclass 'Example' that returns modules. Metric, eval and optimization work with the same semantics of objects.

dspy.Module is an analogue of nn.Module for LM programs. Inside, you can store sub-modules, predictors, and your control flow. All serious DSPy systems, from multi-hop RAG to tool agents, end up with their own forward() classes.

Officially, adapters are responsible for translating signature, demos, input data, and DSPy types into LM messages and parsing the response back to Prediction. They also manage history and function calls. By docs, `ChatAdapter' is used by default; `JSONAdapter' is good for native structured output and lower latency.

The documentation says plainly: dspy. Predict is the basic module on top of which the rest are built. It simply displays input signature in outputs through the language model and does not extend the signature with additional reasoning fields.

The API defines ‘ChainOfThought’ as a module that reasoning and step by step before predicting output. Learn docs further emphasize that in many cases, simply replacing 'Predict' with 'ChainOfThought' immediately improves quality.

dspy.ReAct implements the classic pattern "Reasoning and Acting": the model sees the list of tools, reason, decides to call the tool or complete the task. In DSPy, this pattern is generalized to arbitrary signatures, not just “question-> answer” chat agents.

The official learn page summarises `ProgramOfThought` is as follows: LM learns to output code, and the result of execution of this code determines the answer. The API specifies that the module runs Python programs and requires runtime to be interpreted.

This is a family for cases when text reasoning is not enough. CodeAct is inherited from both ReAct and ProgramOfThought: it is essentially a tool agent with code interpreter. RLM is a standalone experimental abstraction for very large contexts, where LM explores data through sandboxed Python REPL and recursive sub-LLM calls instead of directly loading giant context into prompt.

Not all built-in modules are about reasoning style. Some of them manage the compute budget and execution policy. 'BestOfN' repeats the module N times and takes the best one by 'reward fn'. Refine does the same thing, but adds a feedback loop between attempts. MultiChainComparison brings several reasoning attempts into one final answer. Parallel is a utility for multithreaded pair launch (module, example).

Learn docs emphasize three types of values on example: inputs, intermediate labels, and final label. Important practical conclusion: DSPy can often be used without intermediate labels and even without final labels, if the metric is able to measure quality by inputs and output.

The DSPy documentation literally says: framework machine-learning-like, so automatic metric is mandatory for both evaluation and optimization. For simple tasks, metric can be an ‘accuracy’/‘exact match’; for most production tasks, metric must be a small DSPy program or a function that checks several properties of long-form output.

Before running a compile, you need to understand the baseline. Docs offer either a regular Python loop or a built-in dspy.evaluate.Evaluate that helps with multi-thread eval, progress bar and sample predictions table.

The official DSPy heuristics are good to start with: already 20 dev examples are useful, 200 go far; for optimizers, you can get value from 30 examples, but it is better to aim for 300+. For many, prompt optimizers docs recommend an atypical split `20% train / 80% val` because prompt optimization is retrained on a small trainset. For GEPA, the more classic ML approach is recommended: maximize the trainset and keep the val large enough to reflect downstream distribution.

Docs describe this family as an automatic signature extension of optimized examples. LabeledFewShot just picks up labeled demos. BootstrapFewShot uses the teacher program to generate demonstrations and filters them with metrics. 'BootstrapFewShotWithRandomSearch' amplifies search, and 'KNNFewShot' picks up close examples before bootstrap.

Copro makes coordinate-ascent over instructions. MIPROv2 combines instruction generation and few-shot search through Bayesian Optimization. SIMBA seeks complex examples through stochastic minibatches and generates self-reflective rules. GEPA analyzes trajectories and textual feedback. This family is worth taking when the main problem is not in the raw control flow, but in how the subtasks are formulated.

‘BootstrapFinetune’ distill-it prompt-based DSPy program in weight updates: the program remains the same in structure, but the steps begin to be performed by finetuned models. BetterTogether combines prompt optimization and finetuning in a sequence like ‘prompt → weight → prompt’. Ensemble uses DSPy programs as one program transformation layer.

DSPy docs themselves give the starting heuristics. They are not a substitute for experimentation, but work well as the first decision tree.

According to official GEPA docs and paper: the optimizer takes the seed candidate, runs it on the training signal, receives traces and numeric score, reflects what worked and what did not, generates a new instruction and selects candidates through the Pareto-style frontier. This is no longer a blind search for scalar reward, but a search for textual diagnoses.

GEPA docs formalize the metric interface more strictly than a conventional eval loop. It can ask for feedback for a specific predictor. To do this, not only "gold" and "pred" arrive in metric, but also "trace", "pred name", "pred trace". If you have a multi-module system, this allows you to give local feedback to the step that is currently being optimized.

The official GEPA API is not about the number of parameters, but how they change the search regime.

GEPA advanced docs describe custom `instruction proposer` and custom `component selector` separately. That is, you can separately decide which components to update and who will write the new instructions. For most tasks, docs are advised to start with a default proposer, and customization to include only when default is already understood.

GEPA can be used as a batch inference-time search strategy. To do this, transmit “valset=trainset”, include “track stats=True” and “track best outputs=True”, and then use “detailed results” and frontier best outputs.

The initial stage of DSPy should be super-pragmatic. The goal is not to “turn on compile right away,” but to build a system skeleton and understand its failure surface.

Proper DSPy decomposition follows locally measured steps, not “how beautiful it sounds.” Each module must have a clear contract and a clear reason to exist. If a step cannot be separately explained or evaluated, it is usually a poor module boundary.

GEPA should be added not “when you heard about the new paper”, but when the team has already stabilized program boundaries and is able to give useful textual feedback. Prior to this, MIPROv2 or simple few-shot methods usually give more value per engineering hour.

DSPy should be implemented at maturity, not one big leap. If the team is building an LM system for the first time, it’s better to gradually add layers, otherwise complexity will explode before real quality comes along.

DSPy is able to show LM history through 'inspect history', but the docs themselves honestly state the limitations: this log covers only LM calls and poorly reflects the role of retrievers, tools and custom modules. For serious work, you need tracing. Official tutorials systemically recommend MLflow autologging for compiles, evals and traces.

Compiled DSPy programs can be saved as state-only JSON or as a whole program directory via `save program=True' and then downloaded via `dspy.load'. The cache documentation separately emphasizes: LM calls are cached by default, prompt caching can be enabled from supported providers, and if necessary, you can write your own dspy.clients. Cache'.

The official deployment tutorial describes two main paths: FastAPI for lightweight REST-serving and MLflow for more production-grade deployment with version and artifact management. Practically, this means: while the product is young, FastAPI is enough; as soon as you want to manage lifecycle compiled programs, tracing and versioning, move to MLflow-centered workflow.

DSPy covers runtime concerns directly in tutorials. Streamify includes token streaming and intermediate status streaming. Asyncify and async patterns are needed when an I/O-bound system serves many concurrent requests. Usage tracking is available at the prediction level by enabling "track usage=True". This is the final production contour.