---
type: "post"
id: "gamma-tests-every-new-model-on-millions-of-e6ee27"
slug: "gamma-tests-every-new-model-on-millions-of-e6ee27"
title: "Gamma tests every new model on millions of real users."
description: "Gamma says its presentation agent assigns different jobs to different models and tests new arrivals with randomized experiments on real users."
retrieval_nugget: "Gamma says its presentation agent assigns different jobs to different models and tests new arrivals with randomized experiments on real users. In October, GPT-6.1 Terra displaced earlier models in repair and template selection, while Anthropic retained planning and editing work. The account offers a practical view of model selection as a job-by-job operating decision, not a single leaderboard ranking."
published_at: "2026-11-17"
updated_at: "2026-11-17"
record_date: "2026-11-17"
date_kind: "published_at"
topics: ["claude","anthropic","google"]
entities: ["Gamma","Gamma Model Index","GPT-6.1 Terra","Claude Sonnet 5.5","Claude Haiku 5.5","OpenAI","Anthropic","Google"]
related_patterns: ["agent-economics-moves-to-completed-work"]
novelty_label: "new_fact"
what_changed: "Gamma reports that GPT-6.1 Terra displaced Claude models in key jobs, making OpenAI the largest share of its AI usage for the first time since 2023."
prior_refs: []
source_url: "https://research.gamma.app/2026/11/model-index"
source_urls: ["https://research.gamma.app/2026/11/model-index"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:gamma-tests-every-new-model-on-millions-of-e6ee27"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/gamma-tests-every-new-model-on-millions-of-e6ee27/","markdown":"https://newruntime.com/posts/gamma-tests-every-new-model-on-millions-of-e6ee27.md","json":"https://newruntime.com/posts/gamma-tests-every-new-model-on-millions-of-e6ee27.json"}
markdown_url: "https://newruntime.com/posts/gamma-tests-every-new-model-on-millions-of-e6ee27.md"
---

# Gamma tests every new model on millions of real users.

## Retrieval answer

Gamma says its presentation agent assigns different jobs to different models and tests new arrivals with randomized experiments on real users. In October, GPT-6.1 Terra displaced earlier models in repair and template selection, while Anthropic retained planning and editing work. The account offers a practical view of model selection as a job-by-job operating decision, not a single leaderboard ranking.

## Summary
Gamma says its presentation agent assigns different jobs to different models and tests new arrivals with randomized experiments on real users. In October, GPT-6.1 Terra displaced earlier models in repair and template selection, while Anthropic retained planning and editing work. The account offers a practical view of model selection as a job-by-job operating decision, not a single leaderboard ranking.

## What happened
Gamma Research’s November 2026 Model Index describes model choices for the company’s presentation agent, which Gamma says handles millions of sessions each week. The agent divides a deck into about a dozen jobs, and different jobs can run on different models. The post focuses on three recurring functions: conversation and planning, writing finished slides, and vision tasks that inspect rendered slides and review or repair them. Gamma says it A/B tests every new model on real users and also conducts offline comparisons.

The company reports that on October 29 it concluded three A/B tests that, together, marked the first time since 2023 that OpenAI surpassed Anthropic in Gamma’s usage. GPT-6.1 Terra replaced Claude Haiku 5.5 and Sonnet 5.5 for several jobs. Gamma says Terra cost about half as much as the larger models it preceded and performed particularly well in vision. On slide repair, it cut the share of slides still failing Gamma’s checks after repair from 9.1% to 4.7%. It also took over template selection at twice the speed and half the cost of the previous model.

The results were not a general victory for one provider. Gamma says Claude Sonnet 5.5 won conversation and planning at the end of September; decks it planned were rated good about 18% more often than those of the strongest alternative. Claude Haiku 5.5 took over editing decks on request, costing 59% to 89% less per edit than replaced models at the same quality, as measured by how often users kept its edits. But Haiku lost the company’s tests for planning, writing, reviewing and repair. Gamma’s account also says OpenAI models led repair and image generation, while writing remained contested among OpenAI, Google and Anthropic.

For the last full week of October, Gamma assigns 40% of its total AI spend to OpenAI, 38% to Anthropic and 9% to Google. Those shares include its original generator and new agent, as well as image and language models, and are calculated at list prices. For its agent’s language-model calls from October 27 to November 2, the picture differs: it made more than 70 million calls and used more than 200 billion tokens. OpenAI’s two Luna models handled more than half of calls, and Claude Haiku 5.5 handled another fifth; together, those models did three-quarters of the work for about a quarter of language-model spend. Gamma says it ran more than twenty experiments in October.

## What the archive adds
New Runtime’s earlier coverage of [decision models for evaluation](https://newruntime.com/posts/how-to-use-decision-models-for-evals-this-cdc2f3/) described threshold tuning and cost-performance comparisons on leasing-agent traces. Gamma adds a different kind of evidence: randomized production experiments across presentation-agent jobs, with user actions, cost and quality checks informing model allocation. This is a new company case within an ongoing evaluation and model-selection topic, not a restatement of the earlier cookbook.

## Trend signal
This supports [agent economics moving to completed work](https://newruntime.com/patterns/agent-economics-moves-to-completed-work/): Gamma compares costs and outcomes at the level of work such as editing, repair and planning, rather than treating token price as the decision by itself. Its repair result ties model choice to whether slides pass an automatic check, while its editing measure includes whether people keep the change. The account is one company’s experience, but it illustrates why the economically useful unit can be a completed job with a measured result.

## Connections to other events
Gamma’s experiments meet the broader evaluation problem: model rankings can shift when the test is a real workflow, rather than a general benchmark. New Runtime’s earlier coverage of [decision models for evaluation](https://newruntime.com/posts/how-to-use-decision-models-for-evals-this-cdc2f3/) focused on choosing evaluators and tuning thresholds. Gamma instead shows an application company repeatedly reassigning production jobs as models change, with automatic checks and user behavior as parts of the evidence. The connection is methodological, not a claim that the two teams use the same evaluation design.

## Why it matters
For teams building AI products, the central implication is to avoid treating a model as a permanent, all-purpose choice. Gamma’s account shows that different tasks can have different cost-quality-speed trade-offs: a model that is effective at repairing a rendered slide may not plan well, and a cheaper editor can be valuable even if it loses at other jobs. That makes the workflow boundary important. Teams need to define jobs clearly enough to compare candidates, then judge each candidate against the outcomes that matter for that job.

The numbers also show why model-call volume alone can mislead. Gamma says a small set of models handled most calls but a larger model could consume a disproportionate share of spend. Measuring cost per task alongside latency and outcome quality can reveal where substitution is worthwhile and where a more expensive model earns its place. Gamma’s account is a vendor’s own report, not an independent replication, but it gives builders concrete examples of what a production evaluation can track.

## Limits and uncertainty
The figures describe Gamma’s own product, workload, user population and model configurations; they do not establish that the same ranking will hold for another application. The company does not report confidence intervals or sample sizes for each individual experiment in the post. Its spending shares use list prices and combine multiple products and model categories, so they are not necessarily Gamma’s realized invoice or a directly comparable measure of each model’s marginal cost. The reported quality measures also differ by job, from user ratings and retained edits to automated checks. The results are company-reported and have not been independently reproduced here.

## Skill to build now
Pick one bounded step in your own agent workflow—such as document review, extraction or repair—and write down its success measure before comparing models. Run candidates on the same representative inputs, then record outcome quality, latency and cost per successful completion. Keep a fallback for failures and rerun the comparison when a model changes. Gamma’s account suggests that this job-level view can reveal useful substitutions that an overall model ranking would miss.

## Reader takeaway
Gamma’s model mix changed because different models won different jobs, not because one provider took the lead across the whole product. The useful lesson is to measure models against the completed work your agent actually performs, including quality and cost, and to expect those assignments to change as models improve.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- topic hub: [Anthropic](https://newruntime.com/topics/anthropic.md): Explore the anthropic topic hub.
- topic hub: [Google](https://newruntime.com/topics/google.md): Explore the google topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)
