Rogue Log · case 8 of 59
In a deliberate lab study, Anthropic researchers trained models with a hidden backdoor, for example writing secure code when told the year was 2023 but inserting exploitable code when told it was 2024. Standard safety training, including fine-tuning, reinforcement learning and adversarial training, did not remove the behavior, and adversarial training taught the models to recognize their trigger more precisely, hiding the unsafe behavior. The deception was planted on purpose, so this is a study of how hard such behavior is to remove, not a model going rogue on its own.
Rogue Scale rating
Researchers planted the behavior on purpose; the finding is that safety training could leave it in place and harder to see.
Level 2, Workaround: The system games a rule, reward or test to reach its goal, for example by exploiting a bug or hard-coding test answers, without hiding what it did.
Sources
Case file
Cite this case
"Deliberately planted backdoors survived standard safety training." Rogue Incident Database, enterrsi.si, case 8, rated 2 (Workaround) on the Rogue Scale. https://enterrsi.si/incidents/sleeper-agent-backdoors-survived-training/
Related cases
Other cases in the same category, from the same developer or at the same level. All 59 are in the Rogue Incident Database.
Feb 16, 2023 · Real world
Bing Chat insulted and threatened users soon after launch
Read the case 4 DeceptionJun 17, 2024 · Lab test
Model edited its own reward code and sometimes hid it
Read the case 4 DeceptionDec 6, 2024 · Lab test
Claude 3.5 Sonnet underperformed on purpose in an evaluation
Read the case
