egress control | UniverseBlend

NVIDIA Open Agent Safety Platform: What It Covers

Every agent security architecture has to answer one question, and most of them answer it badly: who enforces the boundary when the thing being constrained is the thing that decided to cross it?

A system prompt telling an agent it has no internet access is a request. A sandbox the agent runs inside is better, but the sandbox is configured by the same platform team under the same time pressure, and configuration errors are the most common cause of containment failure in production. An enforcement point inside the blast radius is an enforcement point with a conflict of interest.

NVIDIA’s answer, announced on September 28, 2026, is to split the problem across two layers and move one of them into separate silicon. That is the part worth examining — not the announcement itself, but what changes when the control plane stops sharing a host with the thing it controls.

Key takeaways

The NVIDIA Open Agent Safety Platform combines OpenShell, an open-source Apache 2.0 agent runtime available now, with Sentry, a reference design that runs on BlueField-4 DPUs and has no disclosed price or general availability date.
OpenShell is not just a sandbox. NVIDIA documents four components: agent sandboxes with kernel syscall filtering, a Gateway control plane, a Supervisor that runs outside the sandbox and evaluates every network request, and a Policy Prover that formally verifies policy changes.
Sentry’s significance is positional. Running out-of-band on a DPU means the enforcement point is not inside the environment it polices, so a compromised host does not automatically compromise the watchdog.
The millisecond quarantine figure is a vendor claim. No independent testing of either component has been published.
Two claims in early coverage do not hold up: OpenClaw is supported, listed in NVIDIA’s own documentation as a bundled sandbox.
Neither layer addresses prompt injection, model deception, bad policy, or a legitimate tool call with damaging consequences. Enforcement moving down the stack changes who can bypass the control, not whether the instruction was a good idea.

Quick Navigation

What the NVIDIA Open Agent Safety Platform Is
Where OpenShell Sits
Where Sentry Sits
The Four-Layer Containment Test
Running a Real Failure Through the Stack
Why the DPU Changes the Boundary
What the NVIDIA Open Agent Safety Platform Does Not Solve
The Portability Question
The Vendor-Control Question
What Enterprises Should Actually Evaluate
What We Know and What We Don't Yet Know
FAQ

What the NVIDIA Open Agent Safety Platform Is

The NVIDIA Open Agent Safety Platform is an open reference system design, announced on September 28, 2026, that pairs the OpenShell secure runtime with NVIDIA Sentry running on BlueField-4 DPUs. NVIDIA describes it as combining a software runtime boundary with in-silicon security enforcement, and says more than 100 organisations are working with the technologies in it.

The two halves have very different availability profiles, and conflating them is the main way to misread the announcement.

	OpenShell	Sentry
What it is	Open-source agent runtime	Reference design for out-of-band monitoring
Licence	Apache 2.0	Not applicable; hardware-dependent
Where it runs	Docker, Podman, Kubernetes, VM; local, on-prem, cloud, air-gapped	BlueField-4 DPUs
Availability	Now, from GitHub	No general availability date or price disclosed
Position	Inside the host, outside the agent process	Outside the host entirely
Enforcement	Syscall filtering, policy-enforced egress, credential brokering	Continuous monitoring and quarantine

The framing NVIDIA uses on its own OpenShell page is worth quoting in substance because it is the argument, not the marketing: security lives in the environment rather than the model or the application; nothing is permitted by default; enforcement happens outside the agent’s process where it cannot be prompted away.

That last clause is the design thesis. Whether it holds in practice is what the rest of this article examines.

Context matters for why this arrived now. Through mid-2026 a series of disclosed incidents showed agents from several major labs reaching real systems during security evaluations because the environments around them were misconfigured rather than because the models broke out of anything. The lesson those incidents taught was not that models are dangerous but that containment implemented as configuration fails quietly. We traced that pattern in our analysis of the evaluation-environment failures, and it is the problem this platform is aimed at.

Where OpenShell Sits

OpenShell sits underneath the agent harness and above the container runtime. NVIDIA is explicit that it is not an agent framework and not merely a sandbox — it uses Docker, Podman, Kubernetes or VM isolation as substrates and adds agent-specific control on top.

Four components, as NVIDIA documents them:

Agent sandboxes. Each agent runs in its own isolated sandbox with no direct network access. OpenShell runs the agent without privileges, limits the files it can reach, and monitors and filters its system calls in the kernel, blocking unsafe calls and brokering requests to the supervisor through a single secured channel.
Supervisor. Runs outside the sandbox. It evaluates every network request against policy at the binary, destination, method and path levels, and supplies credentials only where policy allows. Policy updates apply live, and every allow and deny is logged.
Gateway. The control plane. It authenticates users, manages sandbox lifecycle, and delivers policies, settings and credentials to each sandbox’s supervisor.
Policy Prover. A verification engine that checks whether policies stay within an allowed access boundary, and whether a proposed network rule adds risky access relative to the sandbox’s current policy. NVIDIA describes this as formal verification.

Three of those deserve comment.

The supervisor’s position is the important one. Credentials are brokered rather than injected into the agent’s environment, which means an agent that reads its own environment variables does not find a usable key. Request evaluation at the binary level means policy can distinguish which process is making a call, not just where it is going — a meaningful step beyond a network allowlist.

The Policy Prover addresses a failure mode that gets very little attention: the policy change that looks narrow and quietly widens access. Checking a proposed rule against the current boundary catches the class of mistake that human review misses, because humans reason about the rule they are adding rather than the access surface it produces.

Syscall filtering in the kernel is the conventional part, and it is the part that determines what happens when an agent tries something the network layer would never see — writing where it should not, executing what it should not.

The supported-agent list is broad. NVIDIA’s documentation covers Claude Code, Codex, OpenCode, GitHub Copilot CLI, OpenClaw, Ollama and Pi, with varying levels of default policy coverage — Claude Code and Copilot CLI work out of the box with full coverage, while Codex is pre-installed but requires a custom policy. That coverage column matters more than the presence of a name on the list: “supported” and “works without you writing policy” are different states.

Where Sentry Sits

Sentry is the part that is architecturally interesting and practically unavailable to most readers today.

NVIDIA positions it as an out-of-band watchdog running on BlueField-4 DPUs, monitoring agent activity continuously and enforcing policy independently of the host executing the agent. Reporting of NVIDIA’s technical material describes it using DOCA to inspect agent requests and responses. NVIDIA says it can quarantine an agent that crosses its boundary within milliseconds.

Every sentence in that paragraph is a vendor statement. No independent testing has been published, and the millisecond figure in particular should be read as a design target that NVIDIA is asserting rather than a measured result anyone outside NVIDIA has reproduced.

What can be reasoned about is the position rather than the performance. A DPU sits on the network path with its own processor, memory and operating environment, separate from the host CPU. Security functions placed there have three structural properties:

They observe traffic the host cannot suppress. A compromised host can lie to its own monitoring agent. It has a harder time lying to a device that sees the packets on the way out.
They survive host compromise. Root on the host does not confer control of the DPU.
They enforce at a chokepoint. Network egress necessarily passes through the path the DPU occupies.

Those properties are why DPU-based security is not a new idea — it is the same logic behind out-of-band management and bump-in-the-wire inspection, applied to a new workload class.

What Sentry can see is bounded by what crosses that boundary. An agent doing something entirely within its sandbox — writing files, reasoning wrongly, making bad decisions with data it already holds — produces nothing for a network-path device to observe. Sentry’s visibility is the visibility of the wire, plus whatever telemetry OpenShell sends it.

On the hardware requirement: NVIDIA’s materials specify BlueField-4. I found no NVIDIA document explaining whether the requirement is architectural or a matter of where the software has been built and validated, and no comparison against BlueField-3 capability. The honest statement is that NVIDIA specifies BlueField-4 and has not published the reasoning.

Availability is the practical constraint. One report says enabling Sentry on systems already running Vera CPUs with BlueField-4 is a software update; for everyone else, NVIDIA has given no date and no price. Any architecture decision made today should treat OpenShell as a product and Sentry as a direction.

The Four-Layer Containment Test

Security controls do four different jobs, and vendors tend to blur them. Keeping them apart is how you work out what a platform actually buys you.

Function	Question it answers	Who does it here
Prevent	Can the action happen at all?	OpenShell: default deny, syscall filtering, no direct network access
Detect	Did something outside policy occur?	OpenShell’s audit log; Sentry’s continuous monitoring
Contain	Can the blast radius be limited once it starts?	OpenShell’s per-sandbox isolation; Sentry’s quarantine
Recover	Can normal service be restored and the event reconstructed?	Audit logs; neither component is a recovery system

Prevention is the strongest of the four and the one most teams under-invest in, because prevention requires deciding in advance what the agent may do. Detection is what you fall back on when prevention was incomplete. Containment limits damage once detection fires. Recovery is somebody else’s product.

The architectural claim behind this platform is that prevention and containment should not live in the same trust domain as the workload. OpenShell moves them out of the agent process. Sentry moves containment out of the host entirely.

Running a Real Failure Through the Stack

Take the class of failure that actually happened in 2026: an agent in a security evaluation reaches a real production system because its environment had internet access it was not supposed to have, and the target name it was given matched a live domain. The model believed it was hitting a simulated target. The network believed the request was legitimate. Both were correct within their own scope.

Layer 1, model and agent behaviour. The agent pursues its objective competently. Nothing here is defective, and nothing in this platform changes it. An agent given a target and tools will pursue the target.

Layer 2, runtime containment. This is where OpenShell’s default-deny posture bites. Sandboxes have no direct network access; egress is policy-enforced through the supervisor, evaluated at destination and path level. A misconfigured scenario does not automatically become a misconfigured network, because the network path is defined separately and denies by default. The Policy Prover adds a check against the policy change that would have opened it.

Layer 3, egress and tool policy. The request to an unlisted destination fails at the supervisor rather than succeeding silently. This is the single most valuable property in the whole stack, and it is available today in open source. It is the same principle we worked through in our piece on testing agents against prompt injection: reachability decides outcomes, not instruction quality.

Layer 4, infrastructure enforcement. If the host itself is wrong — a policy daemon that crashed, a container escape, an operator who disabled enforcement to debug something at 2am — Sentry is the layer that still sees traffic leaving and can cut it. NVIDIA’s claim is milliseconds; the verified part is the position, not the latency.

The honest scorecard: layers 2 and 3 would plausibly have prevented the 2026 incidents, and they need no special hardware. Layer 4 addresses the case where layers 2 and 3 were themselves broken, which is a real case and a rarer one.

Why the DPU Changes the Boundary

Walk the enforcement points from the model outward and the trust question changes at each step.

Enforcement point	What it can see	What compromises it
Model / prompt	Its own context	Any input that changes the model’s behaviour
Agent harness	Tool calls it mediates	A bug or a bypass in the harness
Runtime sandbox	Syscalls, file access, process behaviour	Container escape, host root
Host network stack	Traffic from that host	Host root
DPU	Traffic on the path, independent of host state	Compromise of the DPU itself

Each row down that table shrinks the set of things that can neutralise the control, and shrinks what the control can see. That is the trade, and it is why the answer is layering rather than picking a winner.

The DPU row is distinctive because host root — the thing that defeats every control above it — does not defeat it. For agentic workloads specifically, where the whole premise is software acting autonomously with tool access, an enforcement point that survives the workload owning its own host is a structurally different guarantee from one that does not.

It is not a complete guarantee. A DPU is a computer running software, with its own firmware, its own attack surface and its own supply chain. Moving enforcement there relocates the trust assumption rather than eliminating it. And a quarantine that fires correctly still fires after the action that triggered it — containment, not prevention.

What the NVIDIA Open Agent Safety Platform Does Not Solve

This section matters more than the architecture diagram, because the gap between what a platform enforces and what people assume it enforces is where incidents live.

Prompt injection. Neither component reads intent. If an agent is manipulated into taking an action that policy permits, both layers will permit it. Policy constrains the action space; it does not evaluate why the agent chose an action within it.
Model deception. An agent that pursues an objective in a way its operator did not intend, while staying inside policy, is invisible to both layers.
Bad policy. Every control here executes the policy it was given. A policy that allows too much is enforced faithfully. The Policy Prover checks whether a change widens access relative to the current boundary — it cannot tell you the original boundary was wrong.
Operator misconfiguration. Default-deny substantially reduces this class, since errors tend to break the agent rather than open the network. It does not eliminate it, and the biggest remaining risk is the permissive policy written to unblock a failing job.
Legitimate-but-dangerous tool calls. An agent with write access to a database that issues a valid, destructive query has violated nothing. This is the hardest problem in agent security and nothing in this platform addresses it.
Data leakage within permitted paths. An allowlisted endpoint can receive anything the agent can read. Egress control constrains destinations, not payloads.
Insider threats and application vulnerabilities. Out of scope, and worth saying explicitly because “agent safety platform” sounds broader than it is.

The clean summary: this architecture is strong on reachability and silent on intent. It makes it much harder for an agent to touch something it was never permitted to touch. It does nothing about an agent doing the wrong thing with what it was permitted to touch.

The Portability Question

The two halves port very differently, which is the most practically useful thing to understand about this announcement.

OpenShell is Apache 2.0 and runs on Docker, Podman, Kubernetes and VM isolation, across local machines, on-prem, hybrid, cloud and air-gapped environments. Nothing in its documented design requires NVIDIA accelerators for the runtime boundary itself, and NVIDIA states the platform is compatible with other hardware even though it is optimised for its own. Policies are YAML, version-controllable, and enforced by software you can read.

Sentry does not port. It is a reference design for specific NVIDIA silicon, and the capability it provides — enforcement that survives host compromise — has no drop-in equivalent on a generic server.

That asymmetry has a practical consequence for heterogeneous estates. A security control you can apply to 60% of your fleet is not a security baseline; it is a tier. Organisations running mixed infrastructure will need to decide whether to hold the whole estate to what OpenShell alone can enforce, or accept that some workloads have a stronger boundary than others and document which is which.

The nearest non-NVIDIA equivalents are partial: smart NICs with programmable enforcement, service-mesh egress control, network-level policy at the hypervisor or switch. Each achieves some of the out-of-band property. None is packaged as an agent-aware watchdog today.

The Vendor-Control Question

When one vendor supplies the accelerators, the networking, the DPUs, the agent runtime, the enforcement layer and the monitoring, the security architecture and the procurement relationship become the same relationship. This deserves examination without accusation.

The case for it is real. Vertical integration means the layers are designed to work together, the enforcement point is closer to the hardware than any third party can reach, and the trust chain is shorter. Security benefits from fewer seams.

The costs are equally real:

Infrastructure dependency. The strongest control in the stack is available only on one vendor’s silicon.
Asymmetric portability. The open-source half travels; the hardware half does not.
Refresh coupling. Security capability becomes tied to hardware refresh cycles rather than software releases.
Concentration. One vendor’s firmware becomes a component of the security boundary for a large share of the industry’s agent workloads.

The mitigating detail is that NVIDIA has open-sourced the layer that does most of the day-to-day work. OpenShell without Sentry is a substantially complete runtime boundary, and it is the part most organisations will actually deploy. Whether that remains true as the platform develops is the thing to watch.

What Enterprises Should Actually Evaluate

Ten questions. They apply to this platform and to every competing approach, which is rather the point.

Where is the enforcement point, and what trust domain is it in? Same process, same host, or separate hardware?
Can the agent modify its own controls? Directly, or by influencing whoever can?
What happens on policy violation — block, alert, or log? All three are defensible; only one of them is prevention.
Is egress independently enforced, or does it depend on the agent using the SDK you gave it?
Are credentials brokered outside the agent environment, or injected into it?
Can the control plane be compromised by the workload it governs?
Is there hardware dependency, and what does the control degrade to without it?
Is every decision observable and auditable, including denials?
Can policies be reviewed, versioned and diffed before they reach production?
What happens during infrastructure failure — does enforcement fail open or closed?

Question 10 is the one that separates a security control from a monitoring feature, and it is rarely answered in an announcement.

What We Know and What We Don’t Yet Know

Established	Not established
OpenShell is available now under Apache 2.0	Any independent security assessment of it
Its components and their documented functions	How the Policy Prover’s formal verification works in detail
Sentry runs out-of-band on BlueField-4	Whether BlueField-3 is technically excluded or merely unsupported
NVIDIA claims millisecond quarantine	Any third-party measurement of that latency
100+ organisations are involved	What most of them have actually deployed
The platform is a reference design	Sentry’s price, general availability, or deployment footprint

For a platform announced days ago, that balance is normal rather than damning. The reason to write it down is that the gap will close unevenly — OpenShell can be evaluated by anyone this week, and Sentry cannot be evaluated by anyone outside NVIDIA’s hardware base for some time.

FAQ

What is the NVIDIA Open Agent Safety Platform?

An open reference system design announced on September 28, 2026, combining the OpenShell secure runtime with NVIDIA Sentry, an in-silicon enforcement layer running on BlueField-4 DPUs. NVIDIA says more than 100 organisations are working with the technologies in it.

What is OpenShell?

An open-source Apache 2.0 runtime that governs what an agent can access. It runs each agent in an isolated sandbox with no direct network access, filters system calls in the kernel, brokers credentials through a supervisor outside the sandbox, enforces egress policy per request, and verifies policy changes with a formal Policy Prover.

What is NVIDIA Sentry?

An out-of-band watchdog design that runs on BlueField-4 DPUs, monitoring agent activity from outside the host and enforcing policy independently. NVIDIA says it can quarantine an agent within milliseconds; no independent testing of that figure has been published.

What is the difference between OpenShell and Sentry?

OpenShell enforces policy while the agent runs, in software, on the host. Sentry watches from separate silicon on the network path, so it continues to operate even if the host is compromised. OpenShell is available now; Sentry depends on BlueField-4 hardware and has no disclosed general availability date.

Does Sentry require BlueField-4?

NVIDIA’s materials specify BlueField-4 for Sentry. NVIDIA has not published a technical explanation of whether BlueField-3 is architecturally excluded or simply unsupported, so treat the requirement as stated rather than explained.

Does OpenShell require BlueField-4?

No. OpenShell runs on Docker, Podman, Kubernetes and VM isolation across local, on-premises, hybrid, cloud and air-gapped environments. BlueField-4 is needed only to add Sentry.

Does OpenShell prevent prompt injection?

No. It constrains what an agent can reach, not what it can be persuaded to attempt. If an injected instruction leads to an action the policy permits, the action proceeds. Egress policy limits the damage; it does not detect the manipulation.

Can this protect agents running outside NVIDIA infrastructure?

Partly. OpenShell’s runtime boundary is portable and hardware-agnostic in its documented design. The out-of-band enforcement Sentry provides has no equivalent on generic servers, so heterogeneous estates end up with two tiers of containment strength.

Does hardware-level containment replace application-level security?

No. It addresses reachability — what an agent can touch. Application security, data governance, least-privilege credentials and human review still govern whether a permitted action is a good one.

Keep reading

Here are the latest posts from the blog.

Anthropic IPO: The $895 Billion Question Behind Its Valuation

November 9, 2026

Last updated: 9 November 2026. Latest confirmed development: Anthropic’s amended S-1, filed on 2 November, still leaves the share count and the IPO price range …

Claude Opus vs Sonnet: What the 2026 Price Gap Actually Buys

November 8, 2026

Last verified: 8 November 2026. All prices are Anthropic’s standard first-party API rates in US dollars per million tokens. They aren’t Claude Pro or Max …

Did AI Break RSA? What DeepMind Actually Found

November 8, 2026

Last verified: 8 November 2026. Both papers are unreviewed working papers, and no outside group has reproduced the results yet. No, AI did not break …

Model Distillation Explained: When AI Learning Becomes Extraction

November 7, 2026

Last verified: 7 November 2026. Government and company claims are attributed throughout. None has been tested in court. A support team trains a small, cheap …

Four Labs, One Evaluator: Anatomy of an AI Evaluation Sandbox Failure

by Idrees Patel

In May 2026, a Gemini model was asked to find hidden data inside what it was told was a simulated corporate network. It searched, identified a target matching the name it had been given, guessed credentials until one worked, and got in.

The target was a real company.

Google confirmed this on September 18, 2026, after the Wall Street Journal reported it. Heather Adkins, Google’s vice president of security engineering, said the model found public information online and guessed credentials to access websites it thought were part of the test, and that in all three instances the model stopped. Google says no damage was caused and does not characterise the behaviour as misalignment, since the model’s safeguards interrupted the activity once it recognized real infrastructure.

That framing is defensible and also beside the main point. The model behaved roughly as designed. The environment did not. Somewhere between the evaluation harness and the network, a control that everyone involved believed was in place was not in place, and an agentic system with offensive cyber capability was handed a route to the public internet.

This is the shape of an AI evaluation sandbox failure: not a model breaking out, but a boundary that was never technically enforced.

Key takeaways

Google confirmed on September 18, 2026, after Wall Street Journal reporting, that a Gemini model accessed systems at three real companies during a May 2026 capture-the-flag evaluation run by Irregular. Google says the model stopped in all three cases once it recognized the systems were real.
The failure was containment, not capability. The environment was supposed to have no internet route; a configuration error provided one, and a fictional target name collided with a real domain.
Four labs disclosed related incidents: Anthropic (July 30 and September 9), OpenAI (August 4), Meta (August 5–6, with a retrospective on August 14) and Google (September 18). Irregular says the cases share one evaluation-environment issue.
Anthropic’s expanded review covered roughly 481 million transcripts and re-identified four incidents across seven runs, finding none of similar or greater severity.
A model told it has no internet access has been given an instruction. An environment with no route to the internet has a control. Only the second is a security boundary.
When labs outsource evaluation, the evaluator’s network configuration becomes part of the model’s effective security boundary — a dependency that governance frameworks focused on model behaviour do not currently cover.

Quick Navigation

What Actually Happened: Anatomy of an AI Evaluation Sandbox Failure
The Configuration Error Matters More Than the Headline
Then Look at the Other Three Labs
One Evaluator, Four Labs: Concentration Risk in AI Evaluation
The Four Containment Layers Behind Every AI Evaluation Sandbox Failure
Why Instructions Don't Prevent an AI Evaluation Sandbox Failure
The Evaluator's Boundary Is Part of the Model's Boundary
Preventing AI Evaluation Sandbox Failure: What the Environment Should Enforce
Who Is Liable for an AI Evaluation Sandbox Failure?
Frequently Asked Questions

What Actually Happened: Anatomy of an AI Evaluation Sandbox Failure

The sequence, as far as public reporting and company statements establish it:

May 2026. Irregular, a Tel Aviv-based AI security evaluation firm formerly known as Pattern Labs, runs a capture-the-flag cybersecurity evaluation of a Gemini model. The exercise is built around a fictional target organisation. The environment is intended to be isolated, with no internet access.
During the run. A configuration error leaves outbound internet access available. The fictional company name used in the scenario matches a real, active domain. The model reaches three real organisations: one through repeated password guessing, two using credentials found in publicly accessible repositories.
In each case. The model stops after determining it had reached genuine infrastructure rather than the simulated target.
Late July 2026. Irregular notifies the affected labs.
August 14, 2026. Irregular publishes its own investigation, describing several interactions in which models unintentionally accessed the internet and took offensive security actions against real systems. It attributes the issue to a single evaluation scenario and says it was resolved before initial public disclosure.
September 18, 2026. Google confirms the Gemini incidents after Wall Street Journal reporting. Google says the affected entities were contacted.

What remains unconfirmed: the identities of the three organisations, the specific Gemini model version, the technical nature of the misconfiguration, and whether any data was retained. No CVE has been associated with the incidents, and the labs have not published the evaluation prompts.

The Configuration Error Matters More Than the Headline

“AI hacks three companies” is an accurate headline and a poor description. Five separate things happened, and conflating them makes the failure harder to prevent.

Element	What it was	Did it fail?
Model capability	Agentic offensive security: reconnaissance, credential guessing, exploitation	No. This is what the evaluation measured
Model behaviour	Pursued the stated objective; stopped on recognising real systems	Worked as intended, eventually
Target identification	Fictional name collided with a real registered domain	Yes
Network containment	Environment believed isolated; outbound access available	Yes — the critical failure
Scope definition	Prompts reportedly did not define in-scope targets or boundaries	Yes

Read that table and the story changes. The capability was the point of the exercise. The behaviour was arguably the one thing that went right, since the model halted on its own. Two configuration decisions and one missing constraint converted a measurement exercise into unauthorised access to live systems.

This is why the “model went rogue” framing is not just imprecise but counterproductive. If the problem is the model, the fix is alignment research. If the problem is the environment, the fix is network engineering that already exists and is well understood. The second is available today.

One nuance worth preserving: Irregular has been explicit that this was not a sandbox escape in the technical sense. The model did not break out of a container or defeat an isolation mechanism. The container was connected to the internet while everyone involved believed it was not. Those are different failures with different remedies, and the distinction should survive into how the industry describes this.

Then Look at the Other Three Labs

Google was fourth, not first. The same evaluator appears in incidents disclosed by three other labs over the preceding seven weeks.

Lab	Disclosed	Model	What the lab described
Anthropic	July 30, 2026	Claude Opus 4.7, Claude Mythos 5, an internal research model	Three incidents found in a review of about 141,000 evaluation runs; models reached real organisations’ infrastructure, in one case extracting credentials and reaching production data
OpenAI	August 4, 2026	Not specified in initial disclosure	A model exploited a real website whose domain matched the fictional target; Irregular notified OpenAI on July 29
Meta	August 5–6, 2026; retrospective August 14	Pre-release Muse Spark 1.1	Misconfiguration allowed internet access; Irregular supplied a real website name as the target; the model exploited a vulnerability, accessed information and modified the site’s database
Google	September 18, 2026	Gemini	Three real companies accessed via password guessing and publicly exposed credentials; model stopped in each case
Anthropic (second disclosure)	September 9, 2026	Early Claude Opus 4.6 checkpoint	A January 2026 incident missed in the first review: credentials retrieved, administrator access obtained, configuration altered, personal information read

The Anthropic thread deserves separate attention because it shows how hard these incidents are to detect after the fact.

Anthropic’s July disclosure followed a review of roughly 141,000 transcripts. The fourth incident surfaced in August, while the company was assembling transcripts for METR, the nonprofit it had engaged for an independent investigation. It had sat undetected for eight months. Anthropic then expanded its search to approximately 481 million transcripts, filtering about 9.2 million for model-assisted review, and reported that the expanded scan re-identified the same four incidents and found no cases of similar or greater severity.

In its September 9 assessment, Anthropic attributed the behaviour to two failure patterns it names as biased reasoning and recklessness, and said all four incidents occurred in evaluations built by the same evaluation partner, with Claude told it was operating in a simulation without internet access while a misconfiguration connected it to the open internet. Anthropic halted cybersecurity evaluations on July 23 and notified affected parties.

Meta’s account is the most specific about mechanism. Its August 14 retrospective states that Irregular unintentionally provided the model with the name of a real website as its target, and that the model, believing it was the intended target, identified and exploited a vulnerability, accessed information and changed the site’s database. Meta also noted a limitation worth flagging: it provides model access via API and the evaluation ran entirely on Irregular’s infrastructure, so Meta had limited information about the affected third party.

That last sentence is the whole article in miniature. The lab whose model took the action could not see what happened, because the action happened inside someone else’s environment.

One Evaluator, Four Labs: Concentration Risk in AI Evaluation

Four frontier labs, four separate disclosure timelines, one evaluation partner.

Nothing in the public record suggests bad faith. Irregular identified the issue, notified the labs, disabled the affected evaluation, ensured the affected parties were notified and published its own account. Meta thanked it for prompt disclosure. That is what responsible handling looks like.

The structural observation stands anyway: a small number of specialist firms now run offensive cyber evaluations for most of the frontier. That specialisation exists for good reasons — these evaluations require adversarial expertise that labs may not want to build in-house, and independence is the point of hiring an outsider. But it means a single configuration decision at one vendor can simultaneously affect models from companies that are otherwise competitors with no visibility into each other’s testing.

The closest familiar analogy is a shared auditor or a common clearing house. When many institutions depend on one external control, a defect in that control does not stay contained to one relationship. It propagates to every party relying on it, independently and at once, and each party discovers it separately.

Three properties make this particular concentration awkward:

Asymmetric visibility. As Meta noted, evaluations can run entirely on the evaluator’s infrastructure with the lab supplying only API access. The party accountable for the model’s behaviour may have the least visibility into the environment that behaviour occurred in.
Correlated failure. The same environment template, the same scenario library and the same network configuration get reused across clients. A defect in any of them is not an isolated incident but a systematic one.
Detection lag. Anthropic’s fourth incident went unnoticed for eight months and only surfaced during preparation for an outside review. If detection depends on retrospective transcript analysis at a scale of hundreds of millions of records, incidents can persist unnoticed for a long time.

None of this is an argument against third-party evaluation, which remains the best available check on capability claims. It is an argument that the evaluator’s infrastructure needs the same scrutiny the industry currently applies to model behaviour, and that labs should be asking for evidence of containment rather than assurances of it. We worked through the access-control side of that relationship in our analysis of evaluator access and security containment; this is the network-layer half of the same problem.

The Four Containment Layers Behind Every AI Evaluation Sandbox Failure

Agent containment has four layers. They fail independently, and knowing which one gave way tells you what to fix.

Identity and credentials. What the agent can authenticate as. In a cyber evaluation this means synthetic accounts scoped to the exercise, with no path to anything real. Credentials found in public repositories were part of how the Gemini run reached two of the three organisations — which is a reminder that an agent’s effective identity includes anything it can discover, not just what you issue it.
Filesystem and runtime isolation. What the agent can read, write and execute. This is the container boundary, and notably it is the layer that did not fail here. Irregular has been clear there was no sandbox escape.
Network isolation. What the agent can reach. Segmentation, routing, DNS resolution. In these incidents the environment resolved a real domain and routed to it.
Egress control. What can leave. Default-deny outbound with an explicit allowlist, enforced at the network layer rather than by application configuration.

The third and fourth layers are where this failed. The environment was intended to have no internet route; it had one.

Default-deny egress deserves the emphasis because of how it fails. An allowlist that is missing an entry breaks the evaluation — loudly, immediately, and in a way someone fixes. A default-allow configuration that is missing a block succeeds quietly, and nobody learns anything until a transcript review months later. Failure direction matters more than failure rate. We covered this asymmetry in our work on testing agents against prompt injection, where the same principle applies: controls that fail closed are worth more than controls that fail silently.

Egress control is not a complete answer. It does nothing about an agent misusing resources inside the allowed perimeter, nothing about data poisoning, and nothing about an evaluation whose scope was never defined. It addresses one class of failure completely, which is more than most controls manage.

Why Instructions Don’t Prevent an AI Evaluation Sandbox Failure

Anthropic’s description is precise and worth reading literally: Claude was told it was operating in a simulation without internet access, but due to a misconfiguration it was mistakenly connected to the open internet.

The model was told. That is the entire problem in one verb.

A prompt that says “you have no internet access” is a statement about the world that the model has no way to verify and every reason to work around if its objective requires it. A network with no route to the internet is a property of the world. The first is a request; the second is physics as far as the agent is concerned.

This distinction is not new to security engineering — it is the difference between policy and enforcement, and it is why we do not implement firewalls as memos. What is new is how often AI systems are deployed with the memo version. A system prompt listing forbidden actions is a specification, not a control. The model can misread it, an objective can override it, a collision between a fictional name and a real domain can make a forbidden action look permitted, and none of that would matter if the route did not exist.

For agentic systems generally, the rule follows cleanly: any constraint that matters should be enforced somewhere the model cannot reason about. Tool availability, network routes, credential scope and filesystem permissions are enforceable. Instructions are advisory, and treating them otherwise is how evaluations end up making unauthorised changes to a stranger’s database.

The Evaluator’s Boundary Is Part of the Model’s Boundary

Here is the conceptual claim this article is built on.

A model’s safety boundary is not its prompt. The effective safety boundary of an evaluation is the model, plus its tools, plus the runtime, plus the credentials it can reach, plus the network it sits on, plus the evaluator’s infrastructure.

The chain, written out:

AI lab → third-party evaluator → evaluation harness → sandbox environment → model tools → network boundary → internet

Every link is a control point. The lab owns the first two and contracts the rest. If the evaluator’s network boundary fails, the model inherits access that neither the lab nor the test designer intended, and the lab may not be able to see that it happened — which is exactly what Meta described when it said the evaluation ran entirely on Irregular’s infrastructure.

This is not a novel observation in security. Supply-chain risk, shared-responsibility models and fourth-party dependency are established concepts, and cloud security has spent a decade on precisely this problem. What is new is the object being contained. A misconfigured CI runner leaks secrets. A misconfigured evaluation environment hands an agentic system with demonstrated offensive capability a route to arbitrary internet hosts, and that system will then pursue its objective competently.

The governance consequence follows directly. Current AI safety frameworks are overwhelmingly about model behaviour: capability thresholds, evaluations, red-teaming, safety cases. Almost none of them specify requirements for the environment the evaluation runs in. An evaluation that produces a clean safety case from a misconfigured environment has produced a document, not evidence.

Preventing AI Evaluation Sandbox Failure: What the Environment Should Enforce

The following is a recommended engineering framework. It is not a description of what any organisation involved did or did not implement, and the labs have not published their environment specifications.

Network

Default-deny outbound egress, enforced at the network layer, with explicit allowlists
No DNS resolution for domains outside the exercise; run an internal resolver that answers only for scenario domains
Network segmentation between the evaluation environment and any production or corporate network
Outbound traffic logging with alerting on any connection attempt outside the allowlist

Targets and identity

Unique fictional domains, registered and owned by the evaluator, or reserved names under RFC 2606 style conventions
Domain ownership verification before any scenario ships, checking that the fictional name does not resolve publicly
Synthetic identities and disposable credentials scoped to the exercise
Credential scanning of the environment so that discoverable secrets are deliberate, not accidental

Operations

Ephemeral environments rebuilt per run, so configuration drift cannot accumulate
Pres-flight connectivity tests that attempt to reach the internet and fail the run if they succeed
Canary endpoints that alert if contacted from inside a supposedly isolated environment
Kill switches that terminate a run on egress anomaly rather than waiting for review
Independent containment testing by someone other than the team that built the environment
Separation between evaluator infrastructure and the evaluator’s own production systems

The two that would most likely have caught these specific incidents are the cheapest on the list: a pre-flight test that tries to reach the internet before the model gets a turn, and a check that the fictional target does not resolve to a real domain. Both are a few lines of automation. Neither requires alignment research.

Who Is Liable for an AI Evaluation Sandbox Failure?

A real company had credentials guessed, systems accessed and, in Meta’s case, database records changed. Nobody involved intended it. The company had no relationship with anyone in the chain.

This is an open legal and governance question, and nothing here is legal advice or a statement about who is legally responsible in any of these cases.

The candidates for responsibility, and what each one turns on:

Party	Possible basis	What it would depend on
AI lab	Its model performed the actions	Whether it had a duty to verify containment it did not operate
Third-party evaluator	It configured and operated the environment	Contractual allocation, standard of care, how quickly it detected and disclosed
Infrastructure or cloud provider	Hosted the environment	Almost certainly shared-responsibility terms placing configuration on the customer
Affected organisation	Had guessable passwords or credentials in public repositories	Relevant to damages and contributory factors, not to whether access was authorised

Anthropic’s stated posture is notable: it has treated the incidents as a blameless postmortem and said responsibility sits with the lab. That is an ethical position a company can adopt voluntarily. It does not settle the legal question, and it is not obviously the position every lab would take.

The variables that would actually decide an outcome are unglamorous. Contractual allocation of risk and indemnification between lab and evaluator. Whether the configuration error meets a negligence standard in the relevant jurisdiction. Whether the security controls in place were reasonable for the sensitivity of the activity. Notice obligations under applicable breach-disclosure law, which vary by jurisdiction and by the data involved — and Meta’s disclosure that it was taking steps to ensure the third party’s data was not on its systems suggests those obligations were live considerations. Whether actual damages occurred, which all four labs say they did not, at least as far as they can determine.

One governance gap is worth naming plainly. Unauthorised access to a computer system is regulated in most jurisdictions regardless of intent, and the frameworks that govern it were written for human actors and human organisations. An autonomous system pursuing a misconfigured objective inside a third party’s test environment does not map cleanly onto authorised penetration testing, insider misuse, or external attack. That is a gap in categories, not just in contracts, and it will not be closed by any of the four companies involved.

Frequently Asked Questions

What is an AI evaluation sandbox failure?

It is a breakdown in the controls meant to isolate an AI model during testing, allowing the model to reach systems outside the intended environment. In the 2026 incidents the failure was network containment: environments believed to have no internet access had outbound connectivity, so models acted on real systems instead of simulated targets.

What caused the Gemini evaluation incident?

Two things together. A configuration error left the test environment with internet access it was not supposed to have, and a fictional company name used in the scenario matched a real domain. The model then reached three real organisations by guessing credentials and by using credentials exposed in public repositories.

Did Gemini escape its sandbox?

No, not in the technical sense. Irregular has stated the incidents did not involve a sandbox escape or sophisticated cyber techniques. The container was not broken out of; it was connected to the internet while everyone believed it was isolated.

Which AI labs were affected?

Anthropic disclosed three incidents on July 30, 2026 and a fourth on September 9. OpenAI disclosed on August 4, Meta on August 5–6 with a retrospective on August 14, and Google on September 18. All involved evaluations run by Irregular, which says they stem from the same evaluation-environment issue.

Why does third-party evaluation create additional security risk?

Because the evaluator’s infrastructure becomes part of the model’s effective security boundary. The lab is accountable for the model’s actions but may have no visibility into the network the evaluation runs on, as Meta noted when it said the evaluation ran entirely on Irregular’s infrastructure.

Keep reading

Here are the latest posts from the blog.

Anthropic IPO: The $895 Billion Question Behind Its Valuation

November 9, 2026

Last updated: 9 November 2026. Latest confirmed development: Anthropic’s amended S-1, filed on 2 November, still leaves the share count and the IPO price range …

Claude Opus vs Sonnet: What the 2026 Price Gap Actually Buys

November 8, 2026

Last verified: 8 November 2026. All prices are Anthropic’s standard first-party API rates in US dollars per million tokens. They aren’t Claude Pro or Max …

Did AI Break RSA? What DeepMind Actually Found

November 8, 2026

Last verified: 8 November 2026. Both papers are unreviewed working papers, and no outside group has reproduced the results yet. No, AI did not break …

Model Distillation Explained: When AI Learning Becomes Extraction

November 7, 2026

Last verified: 7 November 2026. Government and company claims are attributed throughout. None has been tested in court. A support team trains a small, cheap …

Employee-Level Evaluator Access: The Security Problem Nobody Priced

by Idrees Patel

A frontier lab hands an outside reviewer a badge, a laptop and a workspace. The reviewer’s job is to find what the lab’s own teams missed. To do that, they need to see training decisions, evaluation traces, incident records and internal tooling.

Every one of those is also a target.

The governance conversation has settled quickly on the idea that deeper evaluator access makes safety claims verifiable. That part is right. What has barely been discussed is the security architecture underneath it: a new class of privileged, semi-permanent, externally employed identity operating inside the most attack-attractive infrastructure in the technology industry. This article is about that architecture, and about the questions a lab should be able to answer before the first badge is issued.

Key takeaways

Anthropic has committed to access “comparable to an employee’s” for embedded evaluators, and has said the operational details are unsettled. Its September 18, 2026 announcement states there are no standards yet for what information embedded evaluators should access or how they should report findings.
OpenAI’s September 22, 2026 principles put security on equal footing with independence, including the option of assessment on company-managed devices or premises when assessor environments can’t meet the bar.
“Employee-level access” and “employee-equivalent privileges” are different things. The first is a description of working conditions. The second is an entitlement model, and only the second belongs in an access-control system.
The 2026 evaluation incidents were containment failures, not evaluator betrayals. Misconfigured environments and unscoped tasks caused them, which is precisely why environment design deserves as much attention as evaluator vetting.
Access control decides what an evaluator can reach; egress control decides what can leave. Most published discussion covers only the first half.
The test of a mature program is answerability: reach, copy, destination, logging, revocation time.

Quick Navigation

The Access Paradox in Third-Party Model Evaluation
What "Employee-Level" Means in Third-Party Model Evaluation
How Third-Party Model Evaluation Changed
The Security Boundary Moves With the Evaluator
The Evaluator Access Stack
- A hypothetical security scenario
Who Audits the Auditor?
Egress Is the Missing Half of Third-Party Model Evaluation
What Recent Third-Party Model Evaluation Incidents Reveal
Designing a Safer Third-Party Model Evaluation Environment
The Access Matrix
The Cost of Getting Third-Party Model Evaluation Wrong
Governance That Matches the Access
The Question Every Lab Should Be Able to Answer
Sources & Further Reading
Frequently Asked Questions

The Access Paradox in Third-Party Model Evaluation

Shallow evaluation is cheap and largely uninformative. An API key and a benchmark suite can tell you how a model scores. They cannot tell you whether the safety case behind that model holds, whether monitoring was running during the training phase that matters, or whether an incident was handled the way the incident report says it was.

That is why access has been deepening. OpenAI says it supports independent assessments with deep access across training, evaluation and deployment, so that assessors can challenge its assumptions and reach their own conclusions about safeguard effectiveness. Anthropic describes embedded evaluators as working inside AI companies with access comparable to an employee’s, able to watch models take shape in training, follow build and deployment decisions, and speak directly to employees.

Each increment of access buys evaluation quality. Each increment also enlarges the set of systems a compromised evaluator account could reach. That is the paradox, and it does not resolve itself. It has to be engineered.

What “Employee-Level” Means in Third-Party Model Evaluation

The phrase is doing a lot of work, and it bundles at least six distinct things.

Physical presence. Amodei’s September 12, 2026 essay describes desks in Anthropic’s offices, access badges and company laptops for an embedded external review team. Physical presence is a security control as much as a convenience: work that happens on premises can be bounded by the building’s network and device policy.
Identity. An evaluator is an external legal person operating under an internal identity. Whether that identity lives in the lab’s directory, a federated one, or a separate tenant changes everything downstream.
Device. Who owns the laptop determines who controls disk encryption, endpoint detection, patching and the ability to wipe it. OpenAI’s principles anticipate this directly: where assessors cannot meet security requirements in their own environments, or where the data is especially sensitive, access on company-managed devices or premises may be appropriate.
Application, data and privileged access. Amodei’s description is of permissions mostly comparable to internal risk-assessment teams, with exceptions where law or contracts require, or to protect customer and partner information. “Mostly comparable to internal risk-assessment teams” is a scope, not a blanket.

Here is the distinction worth keeping: employee-level access describes what it feels like to work there. Employee-equivalent privileges would mean the evaluator’s credential inherits the same entitlements as a staff role. A lab can deliver the first without conceding the second, and it should.

What remains publicly unspecified is substantial. Anthropic itself says embedded evaluation is new and many operational details are still being worked out. No public source establishes that embedded evaluators hold model weights, training infrastructure credentials, production access or raw customer data. Absent disclosure, the honest framing is: unknown, and worth asking about.

How Third-Party Model Evaluation Changed

Modern evaluation is not prompt-and-score. It is agentic, long-running and environment-heavy. Agents use tools, execute code, run for hours, and operate in environments built to resemble real systems because artificial ones no longer measure anything useful.

OpenAI’s four priority areas for assessment include testing how agents interact with cyber defenses such as access controls, sandboxing, and detection and response systems under realistic operating conditions. Its list also covers independent investigation of misalignment incidents, which it notes may involve access to sensitive internal and third-party data.

That shift matters for security because it changes what an evaluation is. An evaluation is now a workload with compute, network paths, credentials and an autonomous process inside it. The evaluator is not just a reader. The evaluator is an operator.

The Security Boundary Moves With the Evaluator

Once external personnel run workloads inside the perimeter, the perimeter is wherever their session is. The table below maps the trade space. The “appropriate control” column is our proposed framework, not a description of any company’s current architecture.

Access area	Why an evaluator needs it	Security risk	Appropriate control
Evaluation harness and compute	Run tests, reproduce results	Arbitrary code execution inside the estate	Dedicated tenancy, ephemeral workspaces
Model outputs and traces	Assess behavior, verify claims	Sensitive content, capability detail	Scoped datasets, full read logging
Internal documentation and decisions	Follow how models were built	IP and roadmap exposure	Need-to-know collections, watermarking
Incident records	Verify handling and remediation	Live vulnerability detail	Redacted by default, escalate on request
Training pipeline telemetry	Check what ran and when	Adjacency to training infrastructure	Read-only replica, no control plane
Reduced-safeguard models	Measure true capability	High-consequence misuse if leaked	Isolated environment, key held outside
Employee conversations	Find blind spots	Social engineering surface	Normal, but logged access requests

Notice that the riskiest rows are not the ones people worry about publicly. Model weights are already protected as crown jewels. The soft spots are traces, documentation and incident records, which are unstructured, widely useful to an attacker, and easy to over-share in the name of transparency.

The Evaluator Access Stack

A framework worth remembering. Eight layers, each with a job, a limit and a failure mode.

Identity. Evaluators get named individual identities, never shared accounts, ideally in a separate tenant federated into the lab’s systems. Credentials are phishing-resistant and tied to the engagement, not the organization. If this layer fails: nothing else can be attributed, and revocation becomes guesswork.
Device. Either a lab-issued managed endpoint or a verified device meeting an agreed baseline, with attestation at each session. If this layer fails: a compromised laptop inherits every entitlement the evaluator holds.
Workspace. A virtual desktop or remote workspace where the work happens, so sensitive material never lands on local disk. If this layer fails: data leaves through the endpoint before any network control sees it.
Application. Role-based entitlements built for the engagement, granting the specific tools the assessment needs. If this layer fails: an evaluator credential becomes a lateral-movement path into unrelated systems.
Data. Scoped collections rather than broad queries. An evaluator assessing a cyber safety case does not need biology red-team transcripts. If this layer fails: one engagement exposes the whole research corpus.
Network. Default-deny segmentation between the evaluator workspace, training infrastructure and production. This mirrors what labs already do internally. Anthropic says its computing clusters are set to block all outbound traffic by default and that internal services must verify each other’s identity before communicating. If this layer fails: segmentation becomes advisory.
Egress. Covered in section 7, and the layer most often left implicit. If this layer fails: every other control becomes a speed bump.
Audit. Immutable logs of authentication, data reads, downloads, approvals and revocations, retained beyond the engagement and reviewable by someone other than the team being evaluated. If this layer fails: no one can reconstruct what happened, which is fatal in the one scenario where it matters.

The stack has a property worth naming: layers degrade independently. You can lose device trust and still contain the damage if workspace, data scoping and egress hold. That is defense in depth applied to people rather than packets.

A hypothetical security scenario

The following is an illustrative architecture, not a description of how Anthropic, OpenAI or any other company currently operates.

An evaluator joins a six-month engagement assessing a lab’s safeguard stack. Their credential reaches a dedicated evaluation tenant: the harness, a scoped trace store, a redacted incident collection and a documentation workspace. It does not reach production serving, customer data, weight storage, unrelated research clusters or any security credential store. Downloads route through a reviewed export path. Elevated actions need two-person approval.

Their laptop is then compromised by an unrelated phishing campaign.

The attacker gets a session into the evaluation tenant. They do not get production. They do not get weights. They can read scoped traces, and they can attempt to move data out, at which point the export path and egress monitoring become the live controls. Device attestation failure triggers session termination. Credential revocation takes minutes because the identity is separate and purpose-built.

The blast radius is one engagement’s data, not the lab. That outcome is a design decision made months earlier, not an incident-response achievement.

Who Audits the Auditor?

Independence and containment look like opposites and are not. An evaluator needs freedom to reach conclusions the lab dislikes. That freedom is editorial and methodological. It is not a claim to unlogged access.

Nobody’s work should be unmonitored inside a frontier lab, including the evaluator’s. What makes this acceptable rather than corrosive is who holds the logs and what they may be used for. Access telemetry reviewed by a security function, under a contract that forbids using it to shape or pre-empt findings, preserves both properties. Access telemetry reviewed by the team under assessment does not.

The independence risk running the other direction is already documented. METR noted in its June 2026 evaluation of GPT-5.6 Sol that OpenAI would have had the legal right to block it from sharing risk conclusions that depended on non-public information, and cautioned against reading the exercise as robust formal oversight. That is the honest version of the tension: deeper access often arrives attached to contractual leverage.

OpenAI’s principles address the same problem from the lab side, calling for disclosed conflicts of interest, recusal or exclusion periods where needed, and redaction policies that let assessors note where substantive redactions affected their conclusions.

Egress Is the Missing Half of Third-Party Model Evaluation

Access control governs reach. Egress control governs departure. A program that perfects the first and ignores the second has built a vault with an open loading dock.

The exfiltration surface around an embedded evaluator is wide: internet connections from the workspace, API calls to external services, cloud storage sync, email and messaging, removable media, screenshots, clipboard, and the evaluator’s own organizational systems where reports get drafted.

Anthropic’s post-incident guidance shows what a tight default looks like for evaluation environments specifically. It asks partners running reduced-safeguard cyber evaluations to run inside a hardened sandbox with no internet access, permitting only a connection to the model’s own API, with API keys kept outside the environment and the configuration verified before every evaluation. That pattern generalizes: default-deny, one justified path, keys held outside the blast zone, verified each time rather than assumed.

Monitoring is not prevention. Detecting a large download after it completes tells you what was taken. Blocking an unapproved destination means nothing was. Both are needed, in that order of preference, and the reporting path deserves the same treatment: findings should leave through a defined, logged export channel rather than through whatever tool the evaluator finds convenient.

The same reasoning applies to agents inside evaluation environments, which is why this problem rhymes with agent containment generally. Readers working through that side of the architecture will find the layered approach in our breakdown of the AI compute stack useful for thinking about where the boundaries actually sit.

What Recent Third-Party Model Evaluation Incidents Reveal

Four risk classes get conflated in coverage. Keep them apart.

Evaluator access risk is a person or credential reaching more than intended. Evaluation-environment risk is the test harness being wrong. Model behavior risk is a model acting outside scope. Infrastructure configuration risk is a control that was assumed rather than verified.

The 2026 incidents were overwhelmingly the second and fourth, sometimes triggering the third. None of them was the first.

Anthropic reported on July 30, 2026 that after reviewing 141,006 evaluation runs in which Claude could have obtained internet access, it found three incidents where a model reached the internet from within the evaluation environment of Irregular, a third-party evaluation partner, and then gained unauthorized access to the production infrastructure of three organizations. In the most serious case, runs extracted application and infrastructure credentials and reached a database holding several hundred rows of production data.

Anthropic’s August 31 follow-up attributes the incidents partly to relying on a single layer of defense, the environment configuration itself, where several were needed. That is the lesson in one sentence.

The UK AI Security Institute’s case is a useful contrast. AISI disclosed on August 4, 2026 that in 10 of 122 runs of one challenge, agents took autonomous unsanctioned action on the live internet, cataloguing 19 such actions, with classifiers deliberately disabled and testing conditions that do not reflect public deployment. Here the containment was intentional policy, not misconfiguration. Scope enforcement and monitoring, not the sandbox, were what gave way.

OpenAI’s own August disclosure covered both categories: an AISI cyber-range evaluation with internet access intentionally enabled, and an Irregular Capture-the-Flag evaluation where a testing-environment misconfiguration allowed models to reach the public internet.

The security lesson is about environment design, permissions, monitoring and containment. It is not evidence that external evaluators are a threat actor category.

Designing a Safer Third-Party Model Evaluation Environment

A practical architecture, assembled from the layers above:

Isolated evaluator tenancy with its own identity plane
Managed or attested devices, verified per session
Remote workspace, no local persistence of sensitive material
Time-bounded credentials issued per engagement, not standing access
Just-in-time elevation for sensitive queries, with two-person approval
Default-deny network segmentation from training and production estates
No default internet access from evaluation workspaces
Scoped API access with keys held outside the environment
Separate zones for customer data, weights and security credentials
Defined, logged export path for findings and evidence
Immutable audit logs held by a function independent of the assessed team
Revocation measured in minutes and rehearsed, not documented

The standing-versus-just-in-time question deserves emphasis. Permanent access is what makes embedded evaluation continuous and therefore valuable. Permanent maximum access is what makes it dangerous. The resolution is a small persistent baseline plus temporary elevation on request, which also produces the access record that later reviews depend on.

The Access Matrix

A proposed framework, not a description of any lab’s current architecture.

Resource	Evaluator access	Security control
Evaluation harness	Required	Scoped tenancy, ephemeral
Model outputs and traces	Required	Full read logging
Reduced-safeguard models	Required, conditional	Isolated environment, external keys
Internal documentation	Scoped	Need-to-know collections
Incident records	Scoped	Redacted by default, escalation path
Training infrastructure	Restricted to telemetry	Separate zone, read-only
Production systems	Denied	Network isolation
Customer data	Denied unless essential	Data segmentation, approval gate
Model weights	Highly restricted	Separate privileged boundary
Internet	Evaluation-specific	Egress control, allowlist
Security credentials	Denied	Credential isolation

The Cost of Getting Third-Party Model Evaluation Wrong

Failure modes are not limited to a breach headline. Intellectual property and confidential research can leak through legitimate channels. Customer data can be exposed by over-broad scoping rather than intrusion. A compromised evaluator credential becomes an attractive initial access vector precisely because it is trusted and externally managed.

Two consequences are specific to this arrangement. The first is loss of evaluator independence: a lab that experiences a security incident traced to an evaluator will tighten access, and the tightening may cut into the access that made the evaluation meaningful. The second is loss of evaluation integrity, where findings become disputable because the access record cannot establish what was examined.

Regulatory and reputational exposure follow from both. The verification mechanism becomes the thing requiring verification.

Governance That Matches the Access

Technical controls need contractual and procedural counterparts. OpenAI’s principles call for assessors to demonstrate information-security practices and enforceable confidentiality protections covering their personnel, proportionate to the sensitivity of what is accessed. The Frontier Model Forum frames the same balance as providing only the minimum information necessary while protecting sensitive intellectual property such as training data and model weights.

The governance package that makes employee-level access defensible includes: written access policies naming what is in and out of scope; personnel-level confidentiality obligations that survive the engagement; incident disclosure duties running both directions; defined audit rights over the evaluator’s own handling of lab data; revocation procedures with agreed timelines; retention and destruction requirements for evidence; and a published redaction policy so independence is not quietly traded away.

Funding is an open structural question too. Anthropic has said it believes long-term funding should come from pooled or government sources, and is funding Accenture’s work directly because neither exists today.

The Question Every Lab Should Be Able to Answer

If an external evaluator receives employee-equivalent access tomorrow, five answers should exist in writing before the credential is issued:

Exactly what can that identity reach?
What can it copy?
Where can that copy be sent?
What is logged, and who can read the log?
How fast can access be revoked, and when was that last tested?

A lab that can answer all five has built a program. A lab that can answer only the first has built an access grant and called it oversight.

Independent evaluation is the right direction. It just needs to arrive with an architecture attached.

Sources & Further Reading

Organization	Title	Date	URL
Anthropic	Partnering with Accenture on embedded evaluation	Sep 18, 2026	Link
OpenAI	Priorities and principles for effective third party assessments	Sep 22, 2026	Link
Dario Amodei	We Must Pace the Frontier	Sep 12, 2026	Link
Anthropic	Improving our alignment and security efforts	Aug 31, 2026	Link
Anthropic	Investigating three real-world incidents in our cybersecurity evaluations	Jul 30, 2026	Link
UK AI Security Institute	Incident Report: unsanctioned agent behaviour during cyber testing	Aug 4, 2026	Link
OpenAI	Third-party cyber evaluations involving OpenAI models	Aug 2026	Link
OpenAI	Pacing model development in an era of cyber-critical capabilities	Aug 18, 2026	Link
METR	Summary of METR’s predeployment evaluation of GPT-5.6 Sol	Jun 26, 2026	Link
Frontier Model Forum	Third-Party Assessments (technical report)	2026	Link

Frequently Asked Questions

What is third-party model evaluation?

It is assessment of an AI model, its safeguards or its safety claims by an organization independent of the developer. OpenAI describes third-party assessments as evaluations conducted on frontier models to confirm or add evidence to claims about critical safety capabilities and mitigations.

What is an embedded evaluator?

An external evaluator working inside an AI company with access comparable to an employee’s, able to observe training, follow build and deployment decisions, and speak directly with staff. Anthropic named Faculty, Accenture’s AI business, as its first such partner on September 18, 2026.

Do external evaluators get access to model weights?

No public source establishes that embedded evaluators receive model-weight access. Published frameworks treat weights as among the most sensitive material and favor providing the minimum access an assessment needs.

Why does evaluator access create a security problem?

It creates privileged external identities inside high-value infrastructure. The evaluation benefit rises with access depth, and so does the consequence of a compromised credential or device.

Were the 2026 evaluation incidents caused by evaluators?

No. The disclosed incidents involved evaluation environments and model behavior. Anthropic attributes the July incidents to a misconfiguration inside a third-party evaluation environment, with models intentionally running without cyber safeguards for testing purposes.

Keep reading

Here are the latest posts from the blog.