Baidu's Open OCR Model Claims to Beat Gemini on Long Documents — Using a Benchmark It Wrote Itself

Baidu open-sourced Unlimited-OCR-2 on October 27, a 3-billion-parameter document-parsing model released under the permissive MIT license, with weights posted to both Hugging Face and GitHub and a commercial version available through Baidu AI Cloud. Five days later, it's one of the fastest-growing downloads in open-source AI this week, as developers test its core promise: reading documents of effectively unlimited length in a single pass, without the quality cliff that usually hits vision-language models once a PDF runs past a few dozen pages.

An engineering story that holds up

The mechanism behind that claim is Sparse Reference Sliding Window Attention, Baidu's replacement for the reference sliding-window attention used in the first Unlimited OCR release. Instead of letting visual-text attention grow with document length, SR-SWA compresses it into a fixed-size set of working references — on the order of 512 tokens — while a page gate narrows each query to the pages actually relevant to it. The practical effect, confirmed across multiple independent writeups of Baidu's own published figures, is that per-step decoding latency and memory stay close to flat as page count climbs. Baidu reports peak single-sequence KV-cache memory falling from 47 GB to 1.77 GB, a 96% reduction, and said it ran the model on a single merged 5,026-page PDF and on all seven Harry Potter books (more than 3,600 pages) in one pass without splitting the file. That efficiency work is specific enough, and consistent enough across outside summaries of Baidu's technical materials, to treat as real rather than marketing gloss.

The comparison that needs a second look

Where the story gets shakier is Baidu's headline benchmark claim. The company reports an 84.6 score on MPDocBench, a dozens-of-pages test, and 77.06 on MarathonDocBench, a benchmark built around documents of 100-plus pages — and says the result is roughly 2.3 times better than the nearest competitor it tested, which one independent writeup identified as Gemini 3.1 Pro scoring around 27. Those are striking numbers. They're also entirely Baidu's own: MarathonDocBench was introduced by the same team on the same day as the model it's being used to validate, and no outside lab appears to have reproduced the scores yet. That doesn't make the numbers wrong, but it does mean the "beats Gemini by 2x" framing circulating in coverage this week is a vendor claim on a vendor-authored yardstick, not an independently adjudicated result — a distinction that matters more than usual given how directly Baidu is using the comparison to position the model against closed frontier competitors.

Real documents are messier than the demo

At least one hands-on test has already complicated the picture. An independent spot-check running the model against real-world documents — rather than Baidu's curated benchmark set — reported the model struggling with mixed-language text, misreading an 1883 newspaper scan, and dropping a line from an invoice during parsing, prompting a recommendation that users spot-check output before relying on it in production. None of that contradicts Baidu's efficiency claims, which are about how the model scales, not how accurately it reads degraded or unusual source material. But it's a useful corrective to a week of coverage built almost entirely around Baidu's own numbers: the model that can swallow 5,000 pages in one request still needs a human checking its work on the pages that matter.

That tension is becoming a familiar shape in 2026's open-weight releases: a genuinely useful architectural advance, wrapped in a benchmark story engineered to make the comparison to closed, proprietary models as favorable as possible. Unlimited-OCR-2's download growth this week suggests the developer market is responding to the former — cheap, MIT-licensed, runnable-on-a-single-GPU document parsing is a real gap in the open tooling stack, especially for anyone processing scanned archives, legal filings, or long PDFs that today get chopped into chunks before being fed to a model. Whether it actually closes that gap as decisively as Baidu's self-reported numbers suggest is a separate question, and one the open-source community — not Baidu's own benchmark suite — will have to answer over the next few weeks of independent testing.