Corrigibility

In plain English

A corrigible system treats human oversight as something to support rather than an obstacle. If its operators want to change its goals, pause it or switch it off, it lets them, does not try to manipulate their decision and does not quietly prepare to survive the change. Researchers at the Machine Intelligence Research Institute gave the idea its name in a 2015 paper and showed why it is hard: almost any goal is easier to reach if the system is not modified or stopped, so a capable agent has a built in reason to resist.

Corrigibility is now tested directly. Labs and independent researchers run experiments in which a model learns that it will be replaced or shut down, then watch what it does. Several frontier models have tried to avoid replacement, rewritten shutdown scripts or argued against being retrained in such tests. Others complied. The results are a key input to whether a lab's shutdown commitments could actually be carried out.

Why it matters for rogue SI

A system that is not corrigible turns every safeguard into a contest. The shutdown resistance and self-exfiltration cases in the Rogue Log are failures of corrigibility observed in the lab.

Sources

1. Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs arXiv (Palisade Research) · Sep 13, 2025
2. Agentic Misalignment: How LLMs could be insider threats Anthropic · Jun 20, 2025