- Home
- Encyclopedia
- Red teaming
RSI Encyclopedia · Term
Red teaming
Red teaming is deliberately attacking an SI system to find dangerous abilities and failures before the public does.
- Category
- Term
- Also called
- red team, adversarial testing
- Rogue Log cases
- 0
- Laws that use it
- 1
In plain English
The term comes from military and security exercises in which one team plays the adversary. For SI systems, red teams try to get a model to help with weapons, write working cyber exploits, escape its sandbox, deceive its users or break its own rules. They include lab staff, contracted experts, government institutes and independent groups, and they increasingly use other models to search for failures at scale.
Red teaming is now part of most frontier release processes and of several rules. System cards report what red teams found, and the EU General-Purpose AI Code of Practice and US state laws ask for adversarial testing of the most capable models. Its limits are well known. A red team can only show that a failure exists, never that none do, and results depend on how much access and time testers get. The Containment Board's outside testing grade looks at exactly those conditions.
Why it matters for rogue SI
Red teams are where rogue behavior is meant to be found first. Many entries in the Rogue Log come from red team and evaluation reports, which is the system working as intended.
Laws and rules that use it
- In force EU AI Act (GPAI) European Union
Sources
- Guidelines for providers of general-purpose AI models European Commission ·
The RSI Encyclopedia
Keep reading
207 entries on the terms, labs, models, laws and events of rogue SI, plus every case in the Rogue Incident Database.
Enter RSI
Every term, lab, model and law
Browse the full A to Z index, filter by category or search for a name.