What happened
Langfuse's November 13, 2026 changelog says its decision-model evaluators can now run on gemini-decision-1, the model behind Google's Gemini Decisions API. The feature is used inside Langfuse's evaluator workflow: a team selects a Google AI Studio connection, chooses gemini-decision-1 in the evaluator, and configures an evaluation question. Langfuse is not introducing its own model or describing a change to the Gemini Decisions API itself; it is adding that model as an option for its evaluation product.
The question formats named in the changelog are Choice, Score, and Yes/No. For each of them, Langfuse says the evaluator writes a score along with probabilities and confidence. This makes the stated output more than a free-form model explanation. A team can frame an evaluation as a bounded decision and retain a score plus signals about the model's decision. The changelog does not specify the schema, storage location, retention policy, or downstream automation available for those fields, but it identifies the outputs that the evaluator records.
Langfuse gives three example uses. A team can flag production answers that contradict its refund policy, route conversations to the team that should review them, or grade answer helpfulness on a rubric from 0 to 3. In each case, the decision model sits between an observed answer and a later action: a reviewer may receive a routed conversation, or an evaluation workflow may record a rubric score. The announcement does not say that these examples run automatically by default, nor does it claim that a score should replace human review.
The documented access path is a Google AI Studio connection. Langfuse says Vertex AI connections are not supported yet. It also says Google bills only input tokens for decisions. No per-token amount, total cost example, latency figure, throughput limit, or model-quality comparison appears in the announcement. For existing Langfuse users, the immediate product change is therefore a new evaluator-model selection with specified question types and structured decision signals, subject to the Google AI Studio connection requirement.
What the archive adds
New Runtime's earlier coverage described Langfuse's Jev evaluator as a typed-decision path that could broaden production evaluation beyond sampled LLM-judge workflows (Jev-as-a-judge in Langfuse Evaluators, September 24, 2026). This is a new integration fact on that story: Langfuse adds Google's decision model as another evaluator option rather than restating the Jev release.
Trend signal
This reinforces the pattern that model answers are becoming task interfaces. A natural-language answer can be assessed through bounded choices, scores, or yes/no outcomes rather than handled only as free-form text. This integration is a concrete but limited example; it does not establish that structured decisions will replace ordinary generative responses.
Why now
Langfuse dated this feature November 13, 2026, one day after Google's Gemini Decisions API release. The timing makes it an early integration of the decision-model capability into an established evaluation workflow, rather than a retrospective product consolidation.
Connections to other events
An earlier New Runtime note on October 21, 2026 covered Langfuse experiment features including metric charts, baseline statistics, score-threshold filtering, configurable comparison tables, and human annotations (experiments now show which metric moved, and why.). The new evaluator option adds another way to create the scores that evaluation and review workflows may use.
Why it matters
Production AI quality work often begins with an ambiguous question: was this answer useful, policy-compliant, or worth a person's attention? Langfuse's integration offers a way to express some of those questions as bounded decisions and record a score, probabilities, and confidence beside the result. That can make review queues and quality metrics easier to construct than an ad hoc collection of free-form judge prompts.
The value remains dependent on the operating workflow. A refund-policy contradiction check can help prioritize review, but the team must define the policy boundary, decide which score states trigger action, and inspect errors. Probabilities and confidence provide additional signals, not a demonstrated measure of correctness. The feature is best understood as an extra evaluation path for teams already collecting Langfuse traces, not as evidence that production judgments can safely be automated without validation.
What happens next
- Langfuse adds Vertex AI support for decision-model evaluators. (3 months, 55%). Watch for: a changelog entry naming Vertex AI connections.
- Teams begin using decision evaluators for explicit routing and policy checks. (6 months, 65%). Watch for: Langfuse examples or templates that define review thresholds and escalation paths.
Limits and uncertainty
Langfuse's changelog reports no accuracy, calibration, latency, supported-volume, or reliability results for gemini-decision-1 evaluators. It does not compare the integration with Jev or other evaluator models, and it provides no per-token price or end-to-end cost estimate beyond Google's input-token billing basis. Vertex AI is explicitly unsupported. The refund-policy, routing, and helpfulness examples show possible uses, but they do not establish that those uses have been independently validated in production. Teams will need to test model outputs against reviewed examples and their own escalation criteria.
Skill to build now
If you already use Langfuse for trace evaluation, define one narrow decision question this week, such as whether an answer contradicts a specific refund rule. Run it against reviewed examples before routing live conversations. Compare the evaluator's score, probabilities, and confidence with human judgments, then set an escalation threshold only for cases where the error trade-off is acceptable.
Reader takeaway
Langfuse has added gemini-decision-1 as an evaluator option through Google AI Studio. It can turn bounded evaluation questions into scores with probabilities and confidence, but teams still need to validate those signals, set review thresholds, and account for the current lack of Vertex AI support.
Open question
How well do gemini-decision-1 scores and confidence values align with human reviewers across real policy and helpfulness evaluations?