Anthropic is letting independent evaluators look inside the company, starting with a six-person team from the nonprofit METR. The Anthropic independent evaluators plan gives METR staff company laptops, office space, and internal access close to what Anthropic’s own risk teams have. METR can also publish what it finds, and Anthropic says it cannot veto that.
How the Anthropic Independent Evaluators Plan Works
The announcement follows a promise Anthropic made last month to accept stronger third-party checks on its models and safety work. The company also signed the White House Accord on Super Intelligence, which asks AI companies to work with outside auditors or evaluators.
Anthropic’s argument is simple. It says it cannot share every safety detail with the public, because some of it could help bad actors or is commercially sensitive. So it is giving trusted outsiders deep access instead, and letting them report their conclusions.
The company compares this to bank supervision. Large banks host resident examiners who work inside the bank, because the bank knows more about its own risks than anyone outside does.
What METR Can and Cannot See
Anthropic says its staff are encouraged to share information with the METR team the way they would with a coworker on alignment risk. A few areas stay off limits.
- Data about Anthropic’s customers
- Attorney-client privileged information
- Details about model architectures, which Anthropic sees as a high leak risk with low relevance to risk reviews
Anthropic can require METR to redact information in those narrow categories. However, METR is free to say publicly that something was redacted and whether it thinks the redaction was fair.
Three Tiers of Access for Anthropic Evaluators
Anthropic is building a ladder of access so new partners can earn more trust over time. Its goal is to have several evaluators at the top tier, each publishing its own view of the company’s risk posture.
| Tier | Access | Publishing rights | Current example |
|---|---|---|---|
| First | Narrow, project-based access chosen by Anthropic | Can publish with Anthropic’s approval | SecureBio |
| Second | Broader information access | Can publish without approval, with limited redactions | None named yet |
| Third | Employee-like access to tools and staff | Same as second tier | METR |
SecureBio is currently reviewing the chemical and biological weapons chapter of Anthropic’s August Risk Report. It already published a pilot review of an earlier report.
Other Partners in the Pipeline
Anthropic says it is also working with Accenture and Faculty on red-teaming, alignment checks, and safeguard testing. It is in early talks with other groups, including Hugging Face.
These newer partners get less information and fewer publishing rights for now. Anthropic says it wants to confirm they can protect confidential data before moving them up.
Guardrails Around the METR Team
The METR team works under enforceable agreements and Anthropic’s security standards. Other METR employees, including researchers who work with Anthropic on separate projects, are walled off from that team’s work.
Anthropic stresses that the setup is not a way for anyone to get at its data or its customers’ data. Because evaluators like METR may also review other labs, the company says it stays careful about accidental leaks.
The real test is not the access itself but whether the evaluator publishes something uncomfortable and the company lets it stand.
Why This Matters for AI Safety Oversight
The timing is notable. Anthropic has faced recent scrutiny, including a federal probe into safety incidents covered in our FTC subpoena story and an incident where a Claude agent sent a false police tip.
Anthropic says it is embedding evaluators voluntarily and believes every frontier AI company should do the same. That fits a wider push for outside checks, such as when it asked for binding UK rules alongside OpenAI.
What This Means for You
If you use Claude at work or at home, this change will not alter how the chatbot behaves today. It could change how much you know about the risks behind it.
- Watch for METR’s published reports, which may be the most detailed public look at a frontier lab’s safety practices so far.
- Check whether METR flags any redactions as unjustified, since that will show how real the independence is.
- If your company is choosing an AI vendor, ask whether it allows outside audits with publishing rights.
- Compare this with how rivals handle testing, like the findings in the Gemini safety report for Gemini 4 Argon.
New to the assistant itself? Our Claude AI overview explains the basics. For more daily updates, visit our AI news page.
Sources
This article is based on Anthropic’s own announcement, An update on independent evaluators.
Conclusion
Anthropic has placed six METR staff inside the company with access close to its own risk teams, and METR can publish without an Anthropic veto. SecureBio, Accenture, and Faculty are working at lower levels of access, and Hugging Face is in early talks. The company calls it a step toward real safety oversight, modeled on how bank examiners work. Whether it delivers will depend on what METR actually publishes, so its first reports are worth watching closely.
