What happened
Hedwigai’s announcement describes a comparison of Particle-1, DeepSeek V4.1 Flash and GLM-5.3-Flash as the model behind Claude Code. The harness, settings and tools were held constant; only the model changed. Each model ran 12 tasks five times. Tasks included funding rounds, FDA approvals, SEC filings, contract terms, currency conversion, import records, spreadsheet formulas, a 15-company breadth task, a question with no answer, and a follow-up in the same workbook. A run read and edited workbook tabs, used shell, web search and web fetch, and returned a table with sources for each row.
The comparison used high reasoning, top-p 0.85 and a 32,000-token completion cap. Runs had 15 minutes, except one task allowed 30. When context filled, Claude Code compacted it and continued. A grader checked each row against a key and fetched cited pages to test whether they supported the answer. The authors say a second model regraded 35 runs selected in advance and agreed with the first grader on all 300 rows; they caution that most rows were easy. The study treats each run as a statistical unit, reports 95% bootstrap intervals over 60 runs per model, and notes that five repeats per task cannot resolve accuracy gaps smaller than about ten percentage points.
Across tasks, mean row accuracy was 91.2% for Particle-1, 89.4% for DeepSeek V4.1 Flash and 90.8% for GLM-5.3-Flash. Their 95% intervals overlapped: 84.3–96.9%, 82.2–95.8% and 84.2–96.5%, respectively. Citation precision was similarly close at 96.3%, 95.1% and 94.7%. Six tasks were perfect for every run of every model. The open-ended importer task was difficult for all three, at roughly 22–23% row accuracy.
The clearest separation was in speed and output volume. Median run time was 375 seconds for Particle-1, 315 for DeepSeek and 144 for GLM. Median output tokens were 13,988, 13,576 and 7,752; median tool calls were 34, 29 and 19. The report also highlights a task asking for headquarters and current CEOs of 15 companies. Hedwigai says NVIDIA’s board accepted Jensen Huang’s resignation on 11 November, named CFO Colette Kress interim CEO and filed an 8-K on 12 November. Particle-1 returned Kress in all five runs, DeepSeek in three and GLM in none. Runs initially consulted older annual reports naming Huang, making the task a test of looking beyond stale filings.
What the archive adds
New Runtime’s earlier coverage introduced Particle-1 as a coding model post-trained from DeepSeek V4.1 Flash (31 October 2026) and noted its listing on Vercel AI Gateway (4 November 2026). Hedwigai’s results are a new fact on that known story: in this particular harness, Particle-1 did not show lower output-token use or faster completion than its base model.
Trend signal
This comparison supports the importance of evaluating complete agent runs rather than model claims in isolation, consistent with Agent economics moves to completed work. It reports accuracy, elapsed time, tokens and tool calls for the same tasks. The small number of repeats and narrow task mix make this a useful measurement example, not a basis for a general cost-per-completed-task conclusion.
Why now
The CEO task depended on an unusually fresh event: Hedwigai says NVIDIA’s resignation announcement and 8-K came immediately before the requested 12 November snapshot. The example makes stale-source handling visible in the benchmark rather than relying only on static factual questions.
Why it matters
For teams choosing a model behind a coding or research agent, near-equal aggregate accuracy does not imply equal operational value. In Hedwigai’s runs, GLM completed tasks faster and with fewer output tokens, while Particle-1 caught one time-sensitive leadership change more consistently. That may matter for workflows where freshness is critical, but it should not be generalized from five trials of one task.
The comparison also shows why the harness and task mix belong in a model evaluation. A tool-enabled agent can retrieve newer sources than a filing or annual report, but only if its workflow prompts it to search and validate them. Teams should measure success against their own task set, include latency and verification cost, and inspect failures rather than selecting a model from one headline accuracy figure.
Limits and uncertainty
This is a vendor-published benchmark, and Particle AI provided Particle-1 for the study. The 12 workbook tasks, five repetitions per task and one shared Claude Code setup do not establish how the models perform across other harnesses, code changes or production workloads. Hedwigai itself says the sample cannot resolve accuracy differences below about ten points, and overlapping intervals leave the aggregate ranking uncertain. The NVIDIA result concerns one unusually recent leadership change; it does not establish a general advantage at detecting breaking news. The reported latency and token counts are run medians, not independently verified production costs or service-level guarantees.
Reader takeaway
In Hedwigai’s test, Particle-1 was not clearly more accurate than DeepSeek V4.1 Flash or GLM-5.3-Flash, and it was slower and slightly more token-heavy than its base model. Treat the NVIDIA example as a promising edge case, then test your own tasks, freshness requirements and end-to-end costs before choosing.
Open question
Would the same models retain these accuracy and latency trade-offs on a larger set of coding tasks and repeated production-like runs?