Gemini 4 Argon Tops One Benchmark Index and Ties for Eighth on Another
Google began widening access to Gemini 4 Argon this month, pitching its newest flagship as a model built for the kind of long, document-heavy professional work — finance modeling, legal research, tax prep, enterprise coding — that past frontier launches have mostly ignored. The marketing landed on familiar ground: charts showing Argon ahead of rivals from OpenAI and Anthropic. But the independent evaluators who spent the past week putting Argon through their own test suites have reached strikingly different verdicts, and the split isn't noise — it traces a real fault line in what "frontier" now means.
Vals AI, whose Vals Index blends eight benchmarks across finance, law, tax and coding — weighted by each sector's share of U.S. GDP — ranked Argon first as of October 12, at 68.90%, ahead of Claude Sonnet 5.5 (67.04%) and Claude Opus 5.5 (66.97%). On Vals' narrower Finance Agent v2 test, built around realistic analyst workflows, Argon scored 65.40%, again the best of any model tested. Those are exactly the kind of long, multi-document tasks Google has said Argon was built to handle, and on Vals' numbers, it delivers.
Artificial Analysis tells a different story. Its Intelligence Index, a composite of ten evaluations run across 216 models, scored Argon at 53 — tied with OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1, and trailing Claude Opus 5.5 (58) and Sonnet 5.5 (56). On the firm's Terminal Bench 4 coding test, Argon's 57% lagged Sonnet 5.5 (64%), Opus 5.5 (60%) and even GPT-6 Astra (59%). Argon placed eighth in Artificial Analysis's web-development arena rankings, even as it topped the firm's general text arena at 1,525 points. Two benchmark operators, two very different places on the leaderboard.
A model tuned for spreadsheets, not pull requests
The gap isn't really a disagreement about methodology — it's a function of what each index rewards. Vals weights long professional workflows involving documents and spreadsheets, which is squarely where Google tuned Argon. Artificial Analysis leans toward general reasoning, coding and broad knowledge, categories where Anthropic's and OpenAI's latest models still hold an edge. Argon's one unambiguous win, though, is caution: on Artificial Analysis's AA-Omniscience test, which penalizes wrong answers but not refusals, Argon hallucinated just 15% of the time — the lowest rate of any model scoring 45 or above on the Intelligence Index, against 51% for GPT-6 Astra and 54% for GPT-6.1 Sol. Its raw accuracy, 50% versus GPT-6 Astra's 63%, is lower; it simply declines to guess more often.
Employees versus the highlight reel
The louder story, reported by Bloomberg and amplified across AI trade press this week, comes from inside Google itself. Employees with direct access to Argon told Bloomberg the model struggles with real coding work, particularly front-end design, even as it posts strong standardized scores — and two people used the term "benchmaxxing" to describe a model trained to ace test suites rather than solve messy production problems. Edwin Chen, founder of the data-labeling and evaluation firm Surge AI, has called the broader pattern across the industry "an incredibly pernicious problem." Some Google staff reportedly believe Anthropic's and OpenAI's latest models are improving faster, even as others inside the company insist there's "large consensus" that Argon remains at the frontier.
Tulsee Doshi, who leads Gemini product at Google DeepMind: "We've seen strong performance up close as Googlers have put the model through its paces in recent weeks."
Google disputes that Argon underperforms at coding, pointing to concrete wins like a 2.7x speedup its engineers achieved rewriting the libgav1 video decoder with Argon's help. Koray Kavukcuoglu, who now runs day-to-day operations at Google DeepMind, was unequivocal: "In my mind, it's a certainty that we are always gonna be at the frontier." But Alphabet is still working through the fallout of scrapping a planned Gemini 3.5 Pro release earlier this year — a reversal Bloomberg Intelligence estimates may have wasted up to $400 million in training compute — and losing several senior researchers along the way.
Pricing adds another wrinkle. At Argon's launch discount — $2 per million input tokens and $10 per million output tokens — each Vals Index task costs about $1.99 to run, roughly 60% of GPT-6 Astra's $3.26, largely because Argon is cheap per token even though it uses more than double the output tokens (about 62,000 versus 27,000) to get there. Once the discount expires and prices double, that cost advantage mostly disappears. For a launch sold on both capability and efficiency, the independent numbers suggest Argon delivers neither cleanly — just a sharper picture of which job you're asking it to do.