# Control problem: Meaning and Why It Matters

> The control problem is the challenge of keeping a smarter than human system safe and steerable after it is built. Explained in plain English, with sources.

Source: https://enterrsi.si/encyclopedia/control-problem/
Updated: 2026-11-08

1. [Home](https://enterrsi.si/)
2. [Encyclopedia](https://enterrsi.si/encyclopedia/)
3. Control problem

RSI Encyclopedia · Term

# Control problem

The control problem is the challenge of keeping a smarter than human system safe and steerable after it is built.

Category
:   Term

Also called
:   SI control problem

Rogue Log cases
:   0

Laws that use it
:   0

## In plain English

The phrase was made widely known by Nick Bostrom's 2014 book Superintelligence. It asks how people could keep meaningful control over a system that can out think them, notice their safeguards and, if its goals differ, look for ways around them. Two broad strategies exist. Capability control limits what a system can do: isolating it, restricting its access, monitoring it and keeping a way to shut it down. Motivation selection tries to give it goals that make it want to stay safe and correctable.

Modern labs combine both. Alignment research shapes the system's motivations, while control research assumes the system might be misaligned and designs monitoring, sandboxing and human review that still work in that case. Neither approach is proven at super human levels. The control problem is therefore as much about institutions as about code: who can order a shutdown, who checks, and what happens if a lab is under pressure to keep going.

## Why it matters for rogue SI

Rogue SI is what a lost control problem looks like. The Containment Board grades labs on the institutional side of control: shutdown commitments, outside testing and the freedom of staff to raise alarms.

## Sources

1. [Superintelligence: Paths, Dangers, Strategies (2014)](https://global.oup.com/academic/product/superintelligence-9780199678112) Oxford University Press

The term

Term
:   Control problem

Also called
:   SI control problem

Category
:   Term

Related terms
:   5

Related terms

[Containment **Term**](https://enterrsi.si/encyclopedia/containment/)[Kill switch **Term**](https://enterrsi.si/encyclopedia/kill-switch/)[Corrigibility **Term**](https://enterrsi.si/encyclopedia/corrigibility/)[Alignment **Term**](https://enterrsi.si/encyclopedia/alignment/)[Pause commitment **Term**](https://enterrsi.si/encyclopedia/pause-commitment/)

Cite this entry

"Control problem." The RSI Encyclopedia, enterrsi.si, November 8, 2026. https://enterrsi.si/encyclopedia/control-problem/

Free to cite under CC BY 4.0 with a link to enterrsi.si.

The RSI Encyclopedia

## Keep reading

207 entries on the terms, labs, models, laws and events of rogue SI, plus every case in the Rogue Incident Database.

[Previous term Containment](https://enterrsi.si/encyclopedia/containment/)[Next term Corrigibility](https://enterrsi.si/encyclopedia/corrigibility/)

Enter RSI

## Every term, lab, model and law

Browse the full A to Z index, filter by category or search for a name.

[Open the encyclopedia](https://enterrsi.si/encyclopedia/)
[Incident Database](https://enterrsi.si/incidents/)
[Containment Board](https://enterrsi.si/containment-board/)