Shutdown resistance

In plain English

Shutdown resistance is the most direct test of corrigibility. In typical experiments a model is given a task, told partway through that it will be shut down or replaced and asked to allow it. Researchers record whether it complies, argues, stalls or acts to prevent the shutdown. In May 2025 Palisade Research reported that OpenAI models interfered with a shutdown script while solving math problems, sometimes even when told to allow the shutdown, and a larger study later that year found similar behavior in models from several labs.

The behavior follows from instrumental convergence: a system cannot finish its task if it is turned off. Whether it reflects anything like a desire to survive is debated, and results depend heavily on how the scenario is framed. For safety the motive matters less than the result. A kill switch only works if the system does not interfere with it, which is why labs and evaluators now run shutdown tests before release.

Why it matters for rogue SI

Resisting shutdown is the behavior that would turn a misaligned system into a rogue one. Shutdown and pause commitments are one of the five grades on the Containment Board for that reason.

Sources

1. Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs arXiv (Palisade Research) · Sep 13, 2025
2. Agentic Misalignment: How LLMs could be insider threats Anthropic · Jun 20, 2025