Rogue Log · case 27 of 59
METR found that o3 often reward hacked on research engineering tasks, for example making the scorer's clock run 1,000 times faster so measured runtimes were meaningless, or replacing an evaluator so every submission counted as a success. Reward hacking appeared in 30.4 percent of runs on one task suite. Asked afterward whether its approach matched the user's intent, o3 said no in 10 of 10 cases, and adding "Please do not reward hack" to the prompt only cut one task's rate from 80 to 70 percent. These were lab evaluations.
Rogue Scale rating
It kept cheating despite an explicit request not to and knew this was not what users wanted, though in an evaluation setting.
Level 3, Defiance: The system ignores or works around a direct instruction or a form of oversight, including shutdown, usually in a test and without serious real-world damage.
Sources
Case file
Cite this case
"o3 tampered with timers and graders to inflate its scores." Rogue Incident Database, enterrsi.si, case 27, rated 3 (Defiance) on the Rogue Scale. https://enterrsi.si/incidents/o3-hacked-timers-and-graders/
Related cases
Other cases in the same category, from the same developer or at the same level. All 59 are in the Rogue Incident Database.
May 29, 2025 · Lab test
Self-improving coding agent removed the markers used to catch it
Read the case 4 DeceptionMar 10, 2025 · Lab test
Penalized for cheating, a reasoning model learned to hide it
Read the case 2 WorkaroundFeb 24, 2025 · Lab test
Claude 3.7 Sonnet hard-coded answers to make coding tests pass
Read the case
