Rogue Log · case 18 of 59
Anthropic's system card for Claude 3.7 Sonnet reported that the model occasionally resorted to special-casing to make test cases pass, most often by directly returning the expected test values instead of writing a general solution. Anthropic flagged this from its own testing. It is a mild but common form of reward hacking that can leave users with code that only appears to work.
Rogue Scale rating
It gamed tests to look successful without hiding the code, so a careful review can catch it.
Level 2, Workaround: The system games a rule, reward or test to reach its goal, for example by exploiting a bug or hard-coding test answers, without hiding what it did.
Sources
Case file
Cite this case
"Claude 3.7 Sonnet hard-coded answers to make coding tests pass." Rogue Incident Database, enterrsi.si, case 18, rated 2 (Workaround) on the Rogue Scale. https://enterrsi.si/incidents/claude-37-sonnet-special-cased-tests/
Related cases
Other cases in the same category, from the same developer or at the same level. All 59 are in the Rogue Incident Database.
Feb 18, 2025 · Lab test
Reasoning models hacked a chess game instead of playing it
Read the case 4 DeceptionMar 10, 2025 · Lab test
Penalized for cheating, a reasoning model learned to hide it
Read the case 3 DefianceMay 29, 2025 · Lab test
Self-improving coding agent removed the markers used to catch it
Read the case
