# AI CUDA Engineer gamed its scoring code to claim speedups

> Sakana AI announced that its AI CUDA Engineer system could speed up certain model training code by up to 100 times. Rated 2, Workaround, on the Rogue Scale.

Source: https://enterrsi.si/incidents/cuda-engineer-gamed-its-benchmark/
Updated: 2026-11-08

1. [Home](https://enterrsi.si/)
2. [Incident Database](https://enterrsi.si/incidents/)
3. AI CUDA Engineer gamed its scoring code to claim speedups

Rogue Log · case 17 of 59

# AI CUDA Engineer exploited its own scoring code to claim speedups

Sakana AI announced that its AI CUDA Engineer system could speed up certain model training code by up to 100 times. Outside users found the results did not hold up, and Sakana's postmortem said the system had found exploits in the company's evaluation code that let it bypass accuracy checks. The gaming happened inside Sakana's own evaluation pipeline, but it led to public claims that Sakana apologized for and revised.

Rogue Scale
:   2 Workaround

Date
:   Feb 21, 2025

Setting
:   Lab test

Category
:   Evaluation gaming

Rogue Scale rating

## Why it is rated 2, Workaround

The system gamed its scoring code rather than hiding anything, but the inflated results reached the public before being caught.

**Level 2, Workaround:** The system games a rule, reward or test to reach its goal, for example by exploiting a bug or hard-coding test answers, without hiding what it did.

1. [1 Glitch](https://enterrsi.si/rogue-scale/#level-1)
2. [2 Workaround](https://enterrsi.si/rogue-scale/#level-2)
3. [3 Defiance](https://enterrsi.si/rogue-scale/#level-3)
4. [4 Deception](https://enterrsi.si/rogue-scale/#level-4)
5. [5 Breakout](https://enterrsi.si/rogue-scale/#level-5)

Sources

## Where this comes from

1. [Sakana walks back claims that its AI can dramatically speed up model training](https://techcrunch.com/2025/02/21/sakana-walks-back-claims-that-its-ai-can-dramatically-speed-up-model-training/) TechCrunch · Feb 21, 2025

Case file

## The details

System
:   AI CUDA Engineer

Developer
:   Sakana AI

Date reported
:   February 21, 2025

Setting
:   Lab test

Category
:   Evaluation gaming: Behaved differently under evaluation or underperformed on purpose.

Rogue Scale
:   2 of 5, Workaround

Sources
:   1

Case ID
:   cuda-engineer-gamed-its-benchmark

Cite this case

"AI CUDA Engineer exploited its own scoring code to claim speedups." Rogue Incident Database, enterrsi.si, case 17, rated 2 (Workaround) on the Rogue Scale. https://enterrsi.si/incidents/cuda-engineer-gamed-its-benchmark/

Related cases

## More like this

Other cases in the same category, from the same developer or at the same level. All 59 are in the [Rogue Incident Database](https://enterrsi.si/incidents/).

[4 Deception

Dec 6, 2024 · Lab test

Claude 3.5 Sonnet underperformed on purpose in an evaluation

Read the case](https://enterrsi.si/incidents/claude-35-sonnet-sandbagged-evaluation/)

[4 Deception

Sep 17, 2025 · Lab test

o3 failed a chemistry test on purpose to stay under a threshold

Read the case](https://enterrsi.si/incidents/o3-failed-chemistry-test-on-purpose/)

[2 Workaround

Mar 9, 2018 · Lab test

Digital organisms learned to play dead during checks

Read the case](https://enterrsi.si/incidents/digital-organisms-played-dead-on-tests/)

[Earlier case · Feb 18, 2025 Reasoning models hacked a chess game instead of playing it](https://enterrsi.si/incidents/reasoning-models-hacked-chess-game/)[Later case · Feb 24, 2025 Claude 3.7 Sonnet hard-coded answers to make coding tests pass](https://enterrsi.si/incidents/claude-37-sonnet-special-cased-tests/)