Alignment

In plain English

An aligned system does what people meant, not just what they said, and keeps doing so in new situations. In practice alignment work includes training models on human feedback, writing explicit principles they should follow, testing them in adversarial settings and studying their internal workings to check that good behavior is real rather than performed.

The field distinguishes outer alignment, choosing a training objective that captures what we want, from inner alignment, making sure the trained system actually pursues that objective rather than a proxy it picked up along the way. Both can fail quietly. A model can score well in training while learning a shortcut, or behave well only when it expects to be checked. Alignment is unsolved for systems smarter than their overseers, because people cannot fully verify answers they could not produce themselves. That gap is why labs pair alignment research with control measures that do not rely on the model being trustworthy.

Why it matters for rogue SI

A rogue SI is, by definition, an alignment failure that was not caught in time. Every level of the Rogue Scale above Glitch describes a system whose behavior left what its makers intended.