What happened
Artificial Analysis says VIALS-AA is its implementation of Handshake AI’s VIALS benchmark, which tests whether language models can interpret scientific images and answer questions about the evidence they contain. The underlying set covers 161 tasks across eight artifact domains and uses 174 source images. Examples include blots, microscopy images, flow cytometry plots, plasmid maps, phylogenetic trees, and molecular structures. The tasks are drawn from professional life-sciences workflows rather than polished figures from publications and textbooks.
For its evaluation, Artificial Analysis gives models a question and image file paths in a prompt. The original image files are available in a sandbox where models can execute Python code, but have no network access. The company uses its Stirrup harness for code execution and also provides an image-viewing tool to models with vision capability. Each model attempts every task three independent times.
Artificial Analysis reports several ways to interpret performance. Its headline pass^3 score is the share of tasks answered correctly on all three repeats. Average of 3 measures average correctness over those runs, while pass@3 counts a task as successful if the model gets it right at least once. A run with no submitted answer or no ANSWER: line receives zero, including a refusal. GPT-6 Luna (medium) grades final answers using the benchmark’s upstream judge prompts; when needed, the judge receives the task images in a second call.
The leaderboard lists 19 models. Gemini 4 Carbon (High) scores 64% on VIALS-AA, followed by GPT-6 Astra (Max) at 63% and Claude Opus 5.5 (Max, Default Fallback) at 61%. For evaluation cost per task, the page lists GPT-6 Luna (Max) at $0.02, GPT-6.1 Sol (Max) at $0.06, and Claude Haiku 5.5 (Max) at $0.07. Those are benchmark-evaluation costs calculated from token usage and model pricing, not a quoted price for using VIALS-AA as a service; the page says no public API is available.
What the archive adds
New Runtime's earlier coverage of Artificial Analysis' Decision Index framed bounded task classes as a more useful basis for comparison than a single overall model ranking (October 17, 2026 note). VIALS-AA is a new evaluation on a different task class, adding scientific-image interpretation and repeated-run performance to that thread rather than restating the Decision Index.
Trend signal
This supports the view that model comparison is becoming more task-specific: VIALS-AA reports results for a defined scientific workflow rather than relying on a general capability label. It fits New Runtime's pattern on task interfaces, where outputs are increasingly evaluated against bounded tasks and requirements (Model answers become task interfaces). The benchmark is evidence about evaluation design, not proof that scientific work can safely be delegated.
Why now
The benchmark’s stated motivation is that scientists routinely use visual artifacts to make research decisions, while fluent descriptions of natural images do not establish competence with scientific evidence. VIALS-AA turns that gap into a repeatable, cost-aware comparison that can be revisited as models change.
Why it matters
For a team building in life sciences, broad vision scores can hide the failure mode that matters: a model may describe an image fluently yet misread a band, plot, map, or structure that carries experimental meaning. VIALS-AA gives teams a concrete task set and distinguishes reliable repeated success from occasional success. The cost-per-task view also makes it possible to ask whether a more capable model’s performance gain justifies its expense for this class of work.
The benchmark should be treated as a screening instrument, not a deployment sign-off. Teams can use its artifact domains to identify where to run their own validation, especially on images and decisions representative of their workflow. Any use that affects research conclusions still needs domain-expert review and checks against the underlying experimental data.
Limits and uncertainty
The leaderboard covers 161 benchmark tasks, not the full range of laboratory work or the variation in image quality and protocols across institutions. Its headline score depends on three repeats and on answer grading by GPT-6 Luna using upstream judge prompts; the page does not establish that this grading matches expert review in every case. Cost figures are calculated from token usage and model pricing. They do not measure the cost of human verification, deployment, or downstream errors. Artificial Analysis also says VIALS-AA has no publicly available API.
Skill to build now
Build a small, domain-reviewed test set from the scientific images your team actually handles. For each item, record the relevant visual evidence, an acceptable answer range, and the expert rationale; then run your candidate model repeatedly and track both consistent correctness and review burden. VIALS-AA’s pass^3 framing is a useful reminder to test repeatability, not just a memorable successful demo.
Reader takeaway
VIALS-AA makes scientific-image interpretation a distinct, measurable model task, with repeatability and evaluation cost visible alongside accuracy. Its current leaders are not a substitute for expert review: use the leaderboard to locate promising models and the benchmark’s task categories to shape validation on your own data.