NVIDIA's Nemotron 3.5 Claims Are Real, But Not in the Way They're Framed
NVIDIA shipped the Nemotron 3.5 family on October 15: a 120-billion-parameter vision-language model called Super VL (120B total, 12B active, in a hybrid Mamba-Transformer mixture-of-experts with a C-RADIO vision encoder and a 1-million-token context window), alongside two smaller Bolt variants built for 24GB and 4GB memory footprints. The headline framing, repeated across NVIDIA's own blog and the coverage that followed, is that a post-trained Super VL beats Claude Opus 5 and GPT-6.1 Sol on specific tasks. The underlying numbers are real. What they show is narrower than the framing suggests.
What the benchmark chart actually compares
NVIDIA ran five specialized evaluations — document intelligence, security attack resilience, financial receipt extraction, scientific experimentation, and medical insurance claims — and reported post-training gains of 9% to 44% over the base model. The two numbers getting cited as wins: medical claims accuracy rising from roughly 28% to 72% after post-training, against Claude Opus 5's roughly 38% on the same evaluation; and security attack resilience rising from roughly 35% to 76%, against GPT-6.1 Sol's roughly 66%. Both are comparisons between a customized, task-specific Super VL checkpoint and general-purpose frontier models answering the same narrow question. NVIDIA has not published the task definitions, sample sizes, or scoring protocol behind any of the five evaluations, so none of these deltas can currently be reproduced or audited by anyone outside the company.
That distinction matters for how enterprises should read the release. The real, verifiable claim is a post-training recipe: roughly 26 trillion tokens of pretraining and continued pretraining, then about 3.8 million post-training samples plus reinforcement learning across roughly 100 open environments in NVIDIA's NeMo Gym, with training recipes and datasets released alongside the weights under the OpenMDW license (commercial use, fine-tuning, and redistribution permitted). That is a legitimate and useful contribution — it tells enterprises that fine-tuning this architecture on a narrow task can produce large accuracy jumps. It does not establish that the base model generally outperforms Opus 5 or GPT-6.1 Sol, and NVIDIA's own materials don't claim otherwise once you read past the chart.
The speed claim doesn't survive contact with independent re-testing
The second claim is a throughput number, and it's a cleaner case of a press figure drifting from independently measured reality. NVIDIA's release says Super VL is up to five times faster than a comparable open model, citing an Artificial Analysis measurement of roughly 625 output tokens per second on four Blackwell B200 GPUs at an Intelligence Index score of 26. Artificial Analysis's own live model page for Super VL — the same benchmark, the same index — currently lists output speed at about 422 tokens per second, with the Intelligence Index score unchanged at 26. That's a 32% gap between the number NVIDIA/press coverage cited and what the benchmark's own public page shows for the identical model and metric at the time of writing. Separately, the Intelligence Index score of 26 is also context-dependent: Artificial Analysis frames it as well above the median for comparably sized open models (8), while a same-size competitor, GLM-5.3-Flash, scores 42 on the identical index — nearly 60% higher than Super VL.
The comparisons are between customized Super VL versions and named models on specific tasks — not a general ranking of the base model.
None of this makes Nemotron 3.5 uninteresting. A 120B/12B-active MoE with a 1M-token context, open weights, open RL environments, and a training recipe enterprises can actually copy is a genuine research artifact, and the post-training results are plausible evidence that narrow fine-tuning on this base is unusually effective — medical-claims and security-eval jumps of 30-plus points are not nothing. But the practitioner takeaway is specific: treat the specialized-task numbers as evidence about what post-training on Super VL can do for a narrowly defined problem with your own data, not as a frontier-model leaderboard result, and treat the speed and Intelligence Index figures as numbers worth re-checking against the live Artificial Analysis page rather than the version that made it into the press cycle. Benchmark charts without published task definitions are becoming a standard feature of model launches; the gap between what gets quoted and what the underlying benchmark site shows a few days later is becoming just as standard.