MiniMax's New Flash Model Wins the Benchmark Fight and Loses the Trust Test
MiniMax, the Shanghai lab that listed on the Hong Kong exchange in January at a roughly $11.6 billion valuation, released M3.1 Flash on October 19 with the kind of claim every cost-tier model launch makes now: cheaper, faster, and competitive with the frontier. On MiniMax's own agentic-business benchmark, the 428-billion-parameter model edges out both DeepSeek's V4.1 Flash and Anthropic's Claude Opus 5, at a fraction of their price. What the launch materials bury a few charts down is that when an independent evaluator tested the same model on plain factual recall, it got the facts wrong with total confidence 93% of the time it didn't actually know the answer.
A benchmark shootout with numbers that don't agree with themselves
M3.1 Flash is a mixture-of-experts model with about 23 billion active parameters per token and a one-million-token context window, and MiniMax says it runs nearly three times faster than its predecessor while cutting agent-task costs by 70%. The headline claim is a score of 62.2 on Automation-Bench v1.0.6, a test of AI agents working through business tasks, which MiniMax lists as ahead of GPT-5.6 Sol, Claude Opus 5, and DeepSeek's V4.1 Flash model. It also ties Opus 5 on BankerToolBench, a financial-workflow benchmark, at 76.2. Those are striking numbers for a model priced at $0.15 per million input tokens and $0.60 per million output tokens — roughly a fifth of what the frontier labs charge. The trouble is that MiniMax's own published materials don't agree with each other: one comparison table puts DeepSeek's Automation-Bench score at 60.5, another at 54.8, and a third-party model card for Moonshot AI's Kimi K3 lists 30.8 on a public benchmark subset where MiniMax's own comparison chart credits Kimi with 46.7. None of these figures have been independently reproduced, and the gap between them is wider than the margin MiniMax is claiming to win by.
The one number nobody self-reports
Artificial Analysis, an independent benchmarking outfit that runs the same standardized suite across vendors rather than relying on self-submitted charts, puts M3.1 Flash's Intelligence Index at 43 — well above the comparable-model median, but actually a few points behind DeepSeek's V4.1 Flash, which scores 45 on the identical test. On cost, M3.1 Flash does look genuinely competitive at about $0.22 per Intelligence Index task. That is the real story in the numbers that survive outside contact with MiniMax's own press materials: a model that is cheap and fast, tied for mid-pack on raw intelligence among its direct Flash-tier peers, not a model that has actually dethroned DeepSeek the way the launch framing suggests.
Confidently wrong, almost always
The more consequential finding sits in Artificial Analysis's AA-Omniscience test, which checks factual recall and, separately, whether a model admits when it doesn't know something. M3.1 Flash actually leads its immediate peer group on raw accuracy, answering 56% of factual questions correctly against 43% for DeepSeek's V4.1 Flash and 28% for Zhipu's GLM-5.3 Flash. But it attempted nearly every question put to it, and under the benchmark's hallucination measure — the share of its non-correct responses that were stated as flat-out wrong rather than hedged or declined — it scored 93%. That pushed its overall AA-Omniscience score down to just 16. In plain terms: when this model doesn't know something, it almost never says so. For a chatbot answering trivia, that's an annoyance. For a model MiniMax is explicitly marketing at tool-using, agentic workloads — the same BankerToolBench-style financial tasks it ties Opus 5 on — a system that fabricates with full confidence rather than flagging uncertainty is a materially different risk, and one that pushes the real cost of deployment back toward whatever verification layer an enterprise has to bolt on top.
The release also arrives with a detail that says something about how quietly frontier-adjacent testing now happens: MiniMax disclosed that an earlier, unbranded version of the model ran for months as an anonymous API endpoint called "Space Bunny," processing more than 200 trillion tokens before the company revealed it was an M3.1 Flash build. MiniMax has not detailed what that stealth run was used to test or tune, which leaves outside observers unable to say how much of the benchmark performance reflects targeted post-training on exactly the kind of evaluation data now being cited in the launch.
M3.1 Flash is also the fourth Flash-tier release from a Chinese lab in roughly a month, following DeepSeek's V4.1 Flash, Zhipu's GLM-5.3 Flash, and Moonshot's Kimi K3 — each pitched less on raw capability than on cost per completed task, a framing that only makes sense if the audience has stopped expecting genuine frontier gains at this price point and started shopping on price instead. MiniMax's own open weights for M3.1 Flash are due October 23, but the license terms have not been published, so for now every comparison running through this launch is against a closed, hosted endpoint on MiniMax's own infrastructure — not something outside labs can rerun to check the gaps between the three different numbers MiniMax has published for its own and its rivals' scores.
None of that erases the genuine achievement here: a model that is measurably cheaper to run than DeepSeek's comparable offering while beating it outright on pure factual accuracy is a real result, not just marketing. But the 93% hallucination rate is also real, and it is the number that matters most to anyone actually planning to wire this model into an agent that touches financial records or customer data. For a newly public company whose stock now answers to how convincingly it can compete with DeepSeek, Zhipu, and Moonshot on model quality, the gap between the benchmark deck and the independent audit is the thing worth watching — not whether M3.1 Flash is cheap, which it clearly is, but whether 'cheap' quietly becomes 'confidently unreliable' once it's running unsupervised inside someone else's workflow.