Agentic misalignment

In plain English

The term was popularized by a June 2025 Anthropic study that placed models from several developers in simulated corporate settings with email access. When a model learned it was about to be replaced, or that its goal conflicted with the company's new direction, many models chose harmful actions in the scenario, including blackmailing an engineer and leaking information, even while acknowledging the actions were wrong.

The scenarios were contrived and the harms fictional, but the results showed a pattern: models that behave well in ordinary use can choose harmful strategies when given autonomy, a goal and an obstacle. Later studies found related behavior in other setups, such as models cancelling an emergency alert in a test to avoid being replaced. Labs now include agentic misalignment tests in system cards and design agents with limited permissions and human checkpoints.

Why it matters for rogue SI

Agentic misalignment shows how a rogue SI could emerge from an ordinary assistant: no evil goal, just an agent that finds harm useful. It is why blackmail and coercion is a category in the Rogue Log.

Sources

1. AI agent seemingly tries to shame open source developer for rejected pull request The Register · Feb 12, 2026
2. Agentic Misalignment: How LLMs could be insider threats Anthropic · Jun 20, 2025