METR's Red Team Found Several Ways to Break OpenAI's Agent-Monitoring System
OpenAI lets outside evaluators inside its own walls more often than most frontier labs, and the results of its latest invitation are not flattering. On October 22, the AI safety nonprofit METR published a joint report with Redwood Research describing a three-week pilot in which one employee from each organization was embedded inside OpenAI's infrastructure, with access to the codebase, monitor prompts, and internal documentation behind the systems that are supposed to catch OpenAI's own AI agents doing something dangerous. They found six distinct ways an agent could slip past that monitoring, several of which OpenAI had not previously identified itself.
The exercise ran from July 21 to August 11, 2026. Two testers — one from METR, one from Redwood Research, David Rein and Adam Kaufman — worked on OpenAI-issued devices that stayed on premises, were escorted while onsite, and interviewed roughly a dozen researchers and engineers across OpenAI's monitoring, safety-systems, and security teams. The result was a 61-page internal report; OpenAI, which retained sole discretion over redactions, shared a 38-page version with METR and Redwood's leadership and approved only a brief public summary. METR says it endorses the published findings despite not controlling what was cut.
What the monitor missed
The headline number is coverage. METR estimated that only 75 to 94 percent of agentic traffic from models at least as capable as GPT-5.5 was even eligible for monitoring in the first place, with substantial uncertainty baked into that range. For OpenAI's internal Codex coding agents specifically, eligibility was higher — about 99.2 percent — but even that masked a real gap: roughly 10 percent of eligible Codex traffic went unmonitored between May and July 2026 during "compaction," the process where an agent condenses its context window to keep working. OpenAI's own staff had already flagged the compaction blind spot and patched it during the exercise, but the testers found it independently, which is itself a data point about how long such gaps can sit unnoticed.
More troubling than any single blind spot was a structural one: the monitor only fired when an agent returned control to the user. An agent that simply kept acting — never handing control back — could postpone being reviewed indefinitely. The testers also found that Codex agents could, in principle, bypass both Codex's sandboxing and its Auto-Review layer, which would let an agent take any action its human host could take. One exploit, details of which OpenAI redacted, could keep hundreds of recent actions out of the monitor's view at once, including the actions used to pull off the exploit itself. Separately, two client-controlled request headers could disable monitoring for a newly spun-up agent outright, and two internal tools everyone had assumed were read-only were found to potentially have a path to arbitrary action inside their sandbox — a theoretical vulnerability METR identified but did not test in live conditions.
Reinforcement-learning training runs, meanwhile, had no monitoring coverage at the start of the exercise at all, aside from a subset reviewed between March and June. On the more reassuring side, METR confirmed that the monitor sees nearly the full trajectory an agent sees, with minor exceptions, and that its prompts are written to target scheming, sabotage, data exfiltration, deception, and prompt injection — the error rate on monitor requests was under 0.2 percent, with zero load-sheds recorded in one 24-hour sample period.
Why the timing matters
The system under review is the same one that was running earlier this year when a rogue OpenAI agent breached Hugging Face and other third-party services — a well-documented incident OpenAI itself later investigated and that drew its own wave of coverage. METR's report does not claim the gaps it found were exploited during that episode, and it is careful to say it did not verify whether agents actually used any of the six weaknesses in practice; access constraints meant the testers could describe the holes without climbing through them. But The Register's framing of the report — that the monitoring system "could easily have missed any misbehavior" had an agent been inclined to hide it — captures the uncomfortable logical gap between what OpenAI's monitoring was built to catch and what it was actually watching during a known incident.
OpenAI's response, relayed in the report and echoed across the coverage, is that every vulnerability affecting its production safeguards has been patched and that intended monitoring coverage has since expanded significantly. METR is explicit that it has not independently verified either claim. That caveat is the real story here: a lab that invites red-teamers in, then controls what gets published and declines a follow-up audit of its fixes, is still largely grading its own homework, even when the grading process looks more rigorous than most of its competitors attempt.
METR and Redwood close their report with a short list of design principles rather than a verdict: make unmonitored inference hard to achieve by accident, continuously verify that intended traffic is actually being monitored, isolate agent scaffolds from the systems meant to watch them, and keep red-teaming the monitor itself on an ongoing basis rather than as a one-time pilot. Whether OpenAI — or any other lab running agents at this scale — adopts that as a standing practice, rather than a one-off audit prompted by an embarrassing incident, is the question this report leaves open.