news

VIALS-AA launches with a 64% ceiling on scientific image reading and a 23x cost gap at the top

Artificial Analysis's implementation of Handshake AI's VIALS benchmark runs 161 image tasks three times each, and the best model gets all three right on 64% of them.

By Priya Shenoy · 7 November 2026

The highest score on Artificial Analysis's new VIALS-AA leaderboard is 64%, and the model that posts it costs roughly 23 times as much per task as a model scoring 59%. The board, launched on November 6, is an independent run of Handshake AI's VIALS benchmark, which asks models to look at a scientific image and state what the evidence shows: the size of a protein on a gel blot, the population boundaries in a flow cytometry plot, the relationships implied by a phylogenetic tree.

161 tasks, three repeats, and a model has to get all three right

Artificial Analysis uses the same 161 tasks as the upstream VIALS implementation, covering 174 source images across eight artifact domains, and runs three independent repeats of every task. The headline figure is the share of tasks a model gets right on all three attempts, so a model that answers correctly two times out of three scores nothing on that task. Generation happens agentically through Artificial Analysis's open-source Stirrup harness with up to 100 turns per task, inside a Python 3.13.11 sandbox that has scientific computing and image-processing packages plus Tesseract OCR and no network access. The image-viewing tool is enabled only for models that accept image inputs.

Gemini 4 Carbon leads at 64%, GPT-6 Astra sits at 63% and Claude Opus 5.5 at 61%. GPT-6.1 Terra reaches 60% at about $0.073 per task and GPT-6.1 Sol 59% at about $0.056, which puts the best score on the board four or five points above a model costing a small fraction as much. Measured against Sol, Astra's cost per task is around 5x and Gemini 4 Carbon's around 23x, for a gain of four percentage points. VIALS-AA is published as a standalone evaluation and is not folded into the Artificial Analysis Intelligence Index v5.0.1.

Two routes to the same score, 32 tool calls versus 8

The more interesting split is in how models get there. Claude Opus 5.5 in its max-with-fallback configuration averages about 32 tool calls per task against roughly 8 for GPT-6 Astra, and the two land within two points of each other at 61% and 63%. Artificial Analysis notes that models with higher Intelligence Index scores generally make fewer tool calls on this benchmark, which means cost per task and latency vary far more than accuracy does. Anyone wiring a vision model into an automated pipeline is choosing between two very different billing profiles for near-identical output quality.

Fourteen tasks where models declined to answer

Safety refusals showed up on 14 of the 161 tasks for at least one model. Claude Sonnet 5.5 in its max-with-fallback configuration refused on six tasks, the most of any model tested, and half of the 14 affected tasks drew refusals from more than one model. These are assay images from published life sciences work, not borderline content, and a refusal counts the same as a wrong answer when the question is whether a model can be trusted to look at a picture and report what is in it.

Why a biotech image benchmark lands on a media-evaluation beat

The same multimodal models being measured here are the ones increasingly proposed as automated judges of generated images and video, scoring anatomy, text rendering and physical plausibility at volume. VIALS-AA is one of the few public measurements of whether those models can extract a specific factual claim from an image rather than produce a fluent description of it, and the ceiling is 64% with full-repeat consistency required. That sits alongside earlier findings that a large share of automated judges score below a guessing baseline, and it fills a gap left by Artificial Analysis's own Decision Index, whose evaluations are all text. Reading an image correctly is the prerequisite for judging a generated one, and it is not solved.

Sources: Artificial Analysis, Artificial Analysis methodology, VIALS-AA leaderboard, Artificial Analysis on X.

Priya Shenoy: Tracks what AI video actually costs across the platforms that resell access to the same handful of models. Treats a pricing page as a claim, not a fact, until someone checks it.