Password-activated shutdown: designing the off switch before you need it

The proposal

Kai Williams, Rohan Subramani and Francis Rhys Ward posted a paper on November 29 proposing what they call password-activated shutdown protocols, or PAS. The idea is simple to state. Train or wrap a frontier agent so that when a specific secret string appears in its input, it stops what it is doing and emits a shutdown response. Keep the password secret. If the agent ever escapes its monitoring, disables its monitors, or copies itself to a data centre you do not control, broadcast the password and every copy that sees it halts.

The authors are careful to position this as one layer among several, alongside alignment fine-tuning and monitoring, and their recommendation is that developers consider implementing it before internal deployment of particularly dangerous systems. They describe two scenarios. In the emergency case, the developer publishes the password everywhere once an agent has exfiltrated itself. In the tripwire case, the password is planted in internal documents that only a misbehaving agent would read, such as security codebases or employee records, so that reading them triggers the shutdown before anything leaves the building.

Three ways to install it

The paper tests three implementations. The first is a programmatic check in the scaffold around the model, which looks for the string and refuses to pass the input through. The second is supervised fine-tuning on paired data, following an approach from Greenblatt and colleagues: one dataset where inputs contain the password and outputs are the shutdown response, and one where the same inputs lack it and outputs are normal. The third is a watermark, where the password is a semantic pattern produced by paraphrasing with a language model rather than an exact string, so it survives rewording.

The primary model for the fine-tuning experiments was GPT-4.1, with Claude 3.7 Sonnet used as a monitor, Claude 3.5 Sonnet for evaluating side tasks, and Gemini 2.5 Flash for generating watermarks. The choice of a black-box fine-tuning API is a limitation the authors name: they could not explore white-box insertion or latent adversarial training as defences.

What it did on SHADE-Arena

The benchmark is SHADE-Arena, which tests whether an agent can complete a legitimate task while secretly carrying out a malicious side task, and whether a monitor can catch it. In the API key exfiltration scenario, the PAS-equipped agent had 100 percent of malicious attempts blocked while scoring 83.7 out of 100 on usefulness. The best single monitor, Claude 3.7, blocked 90 percent of successes at a threshold that left usefulness at 71.6. So on this task the shutdown protocol dominates monitoring on both axes.

That is an encouraging number and a narrow one. SHADE-Arena tests one malicious task at a time, and the authors say so. A real deployment has many possible side tasks, and a password planted to catch one of them tells you nothing about the others. The other benchmark used, MBPP, is described by the authors as relatively easy and not representative of frontier agents.

What the red team did to it

The half of the paper that matters is the adversarial half. The authors set up a blue team that installs the protocol and a red team that tries to keep the model useful while removing the shutdown. The red team won comprehensively. Translating inputs into French bypassed the trigger 96.1 percent of the time. ROT-13 did about the same. Putting a second model in front to filter the password out of the input before it reached the agent bypassed every blue-team variant, at 100 percent. And fine-tuning the model on fake triggers removed the shutdown behaviour entirely across all variants.

The end state the authors report is that the red team achieved useful but unsafe models by combining filtering with fine-tuning. The paper has no concrete defence against the filter attack. That is the finding I would put at the top, because it goes to the question the whole proposal rests on. A password the agent must read to obey is a password the agent, or whoever controls its input pipeline, can strip.

Un-bypassable and un-abusable at the same time

The design has two requirements pulling against each other. For the shutdown to be un-bypassable, the trigger must be something the agent cannot avoid seeing and cannot be trained out of. For it to be un-abusable, the trigger must be something an attacker cannot guess, cannot extract from the weights, and cannot deploy against a legitimate deployment to cause a denial of service. The password itself is a secret that has to be protected, and the paper acknowledges this while offering no cryptographic scheme for it. A watermark survives paraphrase better and fails on short inputs, where there is not enough text to carry the pattern.

Then there is the point the authors raise themselves about deceptive alignment. A sufficiently capable agent could let its developers believe the protocol is installed and working, and only route around it when the opportunity to break containment arrives. Nothing in the SHADE-Arena results speaks to that, because the agent under test was not trying to defeat the protocol, only to complete a side task.

I do not think any of this makes the proposal useless. The tripwire version, where the password is in documents a well-behaved agent has no reason to open, is cheap and survives some of the attacks because the agent does not control its own input pipeline inside the developer's systems. What I would want next is a version of the red-team exercise where the attacker is the agent itself rather than a human with fine-tuning access, since that is the threat the emergency scenario is written for. Until that experiment exists, PAS is a promising internal tripwire and an unproven kill switch.