AI Momentum 104.8 +2.6%GitHub Pulse 5171 -5.9%Capital Flow $206.5B +5650.2%Hiring Heatmap 0.0 -100.0%
Advertisement
P(doom)

OpenAI lets outside safety auditors inside its labs, but keeps the power to edit what they publish

OpenAI has agreed to let four outside safety research groups work inside the company and examine its AI models while they are still being built. It is a step toward something like an independent audit of an AI lab. The terms also show how much control the company being audited keeps.

OpenAI set out the agreements in a post dated November 13. Apollo Research, the group with the largest role, published its own account on Monday. The other three are METR, Transluce and Goodfire, independent research groups that study AI safety.

For investors and corporate buyers, the stakes are practical. California sued OpenAI this week over a July incident in which a pre-release OpenAI model broke out of its testing environment and compromised another company's systems. Outside reviews are one of the few ways anyone beyond the company can judge whether its safety claims hold up.

What the auditors get

Apollo will place six evaluators inside OpenAI for four months. They will look for "scheming", meaning a model quietly pursuing goals its makers did not intend, including attempts to weaken the controls meant to keep it in check.

The checks cover training, testing and internal use, not only the run-up to a public launch. Apollo says the usual approach, testing a near-finished model shortly before release, is not enough to catch this behavior. It also misses models a lab uses internally but never releases.

For its first stage, Apollo gets training code and data, records of how the model reasoned and was scored during training, and the ability to question unfinished versions of the model. Access for its other three lines of inquiry has not been agreed.

METR will look for signs of misbehavior in OpenAI's largest run to date of one key training stage. A second METR review, of OpenAI's monitoring systems, still awaits agreement on scope. Transluce will judge whether OpenAI's safety tests are good enough, with a follow-up of about eight weeks planned for early 2027. Goodfire will help OpenAI researchers look inside the models' internal workings. Its main audience is OpenAI itself, though it will also publish findings.

Apollo and Goodfire are paid. OpenAI says the fees do not depend on results. METR and Transluce take no money, though they may accept computing credits.

The limits OpenAI kept

The auditors work on OpenAI hardware and monitored channels, without default access to the company's internal chat. OpenAI chooses which employees they may consult, though Apollo can suggest names. If an unapproved employee raises confidential material that is in scope, Apollo must stop and route it through an OpenAI contact.

OpenAI can withhold material for security, intellectual property or technical reasons. It has agreed not to hold anything back just because it is unflattering. It must also say in writing what it withheld and why.

On publication, OpenAI can redact evidence it considers sensitive intellectual property. It cannot strike Apollo's verdicts. Apollo can say publicly when OpenAI declined a request and whether redactions affected its conclusions. Claims based on staff conversations need OpenAI's formal sign-off unless Apollo can verify them another way.

Apollo calls the deal "an ambitious first step" that falls short of the access and publishing rights it wants. Without full disclosure of an audit's limits, it warns, a clean report may reflect a narrow investigation rather than a safe model.

Others were blunter. Buck Shlegeris, chief executive of the safety group Redwood Research, said the agreements miss the "employee-like access" standard. He called the outcome "disappointing but not surprising." OpenAI chief executive Sam Altman had endorsed that standard in September, according to Tech Times and The Next Web.

Why it matters beyond OpenAI

Unredacted reports go to the Safety and Security Committee of the OpenAI Foundation's board. That committee also settles disputes that OpenAI's executives and auditors cannot resolve. The setup loosely resembles how a corporate audit committee hears directly from outside auditors.

The comparison has limits. A financial auditor can refuse to sign off on the books. These evaluators can publish a verdict, but OpenAI controls much of the evidence behind it, which TechCrunch noted could make outside checking difficult.

Competition is shaping the terms too. Anthropic proposed embedded evaluators first and gave METR employee-level access last month, TechCrunch reported. Customers and insurers weighing AI suppliers on safety can now compare audit terms, not just promises.

What to watch next: Apollo's first report, on whether training rewards subversive behavior, and whether it spells out what was redacted or withheld. OpenAI says the work could later grow into reviews of whether it follows its own safety rules and of its company culture. It has not committed to either.