Anthropic announced on October 15 that it began giving METR staff insider access similar to that of employees, for risk assessment in its AI models and safety processes. 6 organization employees received equipment, office space and access to internal tools and systems. The arrangement is intended to allow them to see more than the information exposed by regular external audits, and to publish their findings to the public.
Aside from the broadening of access, there remains an essential limitation. Anthropic says it has no veto over publication itself, but may require redaction of information in predefined categories. According to the clarification added on October 16, it is the company that ultimately determines what is included in these categories. METR retains limited rights to describe the scope of the access and information it received. Freedom to publish and control over information are therefore different matters.
What does the METR deal include?
According to Anthropic, the team's access is close in scope to that of its internal risk assessment teams. Company employees are encouraged to share information with the team as they would with a colleague working on model alignment risks. Exceptions include customer information, legally privileged material and information about model architectures. The company describes this as a significant opening of the work environment, but not unrestricted access.
Even within METR the arrangement does not apply to the entire organization. The combined team operates under binding agreements and separately from other METR staff, including researchers involved in other projects with Anthropic. This distinction matters: the arrangement concerns a defined group of evaluators, rather than a general transfer of internal information to every external partner. The arrangement does not grant public permission to inspect customer data.
The company says the team will be able to publish analyses of risk levels, incidents or procedures. These are different fields: assessing a model's ability to perform a dangerous task, examining an event that has already occurred, and examining the way decisions are made. Internal access can be relevant to each of them. Opening the door alone does not establish what evaluators will find when they examine the material.
Why does an external inspection also need internal information?
Testing a model through a user interface can show how it behaves under certain conditions. It does not necessarily reveal how the tests were selected, what previous findings were known to the developer, or how the protections changed over time. Therefore, the outcome seen from the outside and the context held by the company may answer different questions. A thorough assessment must distinguish between the measured ability and the limits of the measurement.
Anthropic presents the arrangement as a way to bridge this information gap without exposing every sensitive detail to the public. This describes the approach the company has chosen; it does not establish that all gaps have been closed. When findings are published, it will be important to examine the result, the scope of access, the excluded areas and the questions the assessment was designed to answer. Such limitations help to understand the findings rather than cancel them out.
For an organization using Claude, an external evaluation document may provide additional material for the supplier selection process. It is not a substitute for checking the local usage scenario. A system suitable for drafting does not automatically receive the same risk profile when connected to systems with operational permissions. The practical question remains what capabilities are tested and in what operating environment they are intended to be used.
Freedom to publish alongside redaction powers
The important distinction in the notice concerns two decisions: whether the evaluator may publish a conclusion, and what details he may include alongside it. Anthropic says METR can publish without prior company approval, subject to restrictions on categories of information that may be redacted. The clarification that the company is decisive in the classification of information limits the meaning of the phrase "without veto". The two instructions should be read together.
This does not mean the company has already blocked an unfavourable finding or that the redactions are unjustified. Customer information or details that could help attackers may warrant protection. On the other hand, when material does not appear in a public document, the reader cannot examine it for himself. Therefore, an explanation of access and publication restrictions is part of the evidence needed to assess the arrangement, not just an administrative detail on the margins.
A business customer can use this distinction when reading any safety report. It is worth asking who defined the task, which systems and versions were tested, and who can restrict publication. There's no need to assume a negative answer to these questions. Asking these questions helps distinguish conclusions based on extensive examination from those that still depend on information held solely by the company being assessed.
3 levels of access, rather than one arrangement for everyone
Anthropic describes 3 levels of engagement with evaluators. At the first level, investigation is limited and publication requires company approval. At the second level, you can publish without permission, subject to restrictions on categories of information. The third level adds an internal access scope similar to that described in the arrangement with METR. This means that the term "external evaluator" alone is not sufficient to understand what can really be tested.
Other collaborations are at different stages. The company describes work with SecureBio, Accenture and Faculty, while noting that early engagements involve more limited access and publication rights. It is not to be inferred that METR's access has already been granted to all of these bodies. Even an initial conversation about cooperation is not equivalent to conducting an assessment or publishing its findings to the public.
How to use the information without drawing excessive conclusions
Security and procurement teams should follow up on the documents that the evaluators publish and link them to the audits they conduct themselves. You can document what actions the agent is allowed to perform, what data he sees, and when human approval is required. This will allow comparison between general safety claims and the actual usage environment. Expanding evaluator access does not itself change permissions in a customer's systems.
The innovation is in the supervision structure that the company opens to an external team. The arrangement may broaden the basis for public assessment, but its value will also depend on the quality of the findings and the explanation of their limitations. For now, the announcement establishes deeper access and more detailed publication rules. The agreement itself provides no basis for treating the models, or every possible use of them, as fully approved for safety.