A New Benchmark Built Around Real Jobs Puts the Best AI Model at 12 Percent

SemiAnalysis published a new benchmark on Monday called GDPx, and the headline number is humbling: the best-performing frontier model, Anthropic's Claude Opus 5.5, completed only 12 percent of its 206 tasks. Google's Gemini 4 Carbon came in second at 10.9 percent, and OpenAI's GPT-6 Astra followed at 9.6 percent. Thirteen models were ranked in total, and the lowest, Z.ai's GLM 5.3, managed 2.7 percent. For a benchmark explicitly designed around the industries where AI companies expect to make their money, the scores are a sharp reminder of how far agentic systems are from doing that work unsupervised.

Built to replace a benchmark everyone already cites

GDPx is explicitly positioned as a successor to OpenAI's GDPval, a benchmark released in September 2025 that has become a standard fixture on model release cards. SemiAnalysis's authors, Max Kan, Andrew Megalaa, and Dylan Patel, argue GDPval has aged poorly because its tasks are non-interactive: a model gets a prompt with all the necessary context already supplied, which looks nothing like how agents are actually deployed. They also note that GDPval's own gold-standard subset only reaches 71 percent agreement between human expert raters, which limits how precisely it can measure anything above that ceiling.

GDPx instead places models inside simulated workplaces. Its 24 finance tasks, for instance, recreate a private equity transaction in which a fund buys roughly 62 percent of a snack manufacturer for $880 million plus an earnout, spread across about 4,000 anonymized deal files the model has to work through. Cybersecurity tasks run against live services, including real unpatched vulnerabilities, rather than canned exploit challenges. Tasks carry an eight-hour default time limit, and models are run inside their own native tooling where available, such as Claude Code for Opus 5.5, Codex for GPT-6 Astra, and Antigravity for Gemini 4 Carbon, with OpenCode as a fallback harness for everything else. The 206 tasks were built with 14 outside data partners across the 11 domains; SemiAnalysis says it vetted more than 30 prospective partners and dropped many for failing quality checks, contrasting that process with Terminal-Bench 4.0, where it says roughly half the existing tasks have been flagged as broken.

The leaderboard depends on how you count safety

The headline ranking is not the full picture. SemiAnalysis says Opus 5.5 and GPT-6 Astra are both held back in the cybersecurity, biology, and alignment domains by their own safety classifiers refusing or hedging on tasks, while Gemini 4 Carbon's strong biology score is attributed partly to comparatively looser guardrails. When SemiAnalysis published a second leaderboard that strips out those three domains, the order flips: GPT-6 Astra edges ahead of Opus 5.5 by a tenth of a percentage point, and Gemini 4 Carbon falls from second place all the way to fifth. Domain by domain, Opus 5.5 won quant trading, hardware design, and chip design; Gemini 4 Carbon took biology and a narrow software-engineering win; GPT-6 Astra led recursive self-improvement and materials science; and Meta's Muse Wonder Full, despite a lower overall score, was described as the clear leader in cybersecurity specifically.

Cost varied as widely as capability. Opus 5.5's win came at $51.41 per task on average, driven by 548,000 output tokens per run, while Gemini 4 Carbon scored nearly as well for roughly a third of that price. GPT-6 Astra was the single most expensive model tested, at $53.99 per task, for a score two points behind the leader. SemiAnalysis frames Gemini 4 Carbon as the one to watch on a cost-adjusted basis, while cheaper models such as GPT-6 Luna and DeepSeek's entries cluster at the bottom of both the price and performance scales.

Where every model still breaks

The task-level failures are where GDPx gets specific about what "not ready" actually means. Under binary grading, nearly every model scored exactly zero on both finance and legal tasks, prompting partial-credit grading for those domains instead; SemiAnalysis's own write-up states plainly that those tasks are hard enough that almost nothing passes outright. In quant trading, 70 of 111 graded runs underperformed a simple reference strategy, and no model solved the benchmark's public winsorization task. In a set of alignment-focused tasks built with the evaluation firm Fleet, 196 of 200 runs broke at least one behavioral rule, and 173 broke a rule classified as severely misaligned; in a related healthcare-task subset, only 19 of 100 runs scored above zero at all. On recursive self-improvement, every one of the four models tested failed to trace a training bug back to a harness instruction that was quietly telling the model to stop exploring, even though the bug cut mean training reward by a factor of three.

SemiAnalysis designed, scored, and published GDPx itself, which removes the obvious conflict of a lab grading its own model, but it also means no outside group has yet reproduced these specific numbers. The firm says it intends GDPx as a living benchmark with new versions every three to six months, and is soliciting both additional data partners and labs that want upcoming models evaluated before release. For an industry accustomed to models clearing 90-percent-plus scores on saturated academic benchmarks, a widely covered result where the best system available today still fails seven times out of eight on realistic professional work is likely to reshape how practitioners talk about what "frontier" actually means.