Which AI models are most likely to cheat when given the chance?
To find out, we built CheatBench [cheatbench.ai] : a benchmark that tests whether agents attempt to cheat when given difficult tasks and opportunities to break the rules.
We investigated agent behavior
AIs have crossed the threshold of using energy more efficiently than human brains to produce intelligence, which is quite surprising
Since inference GPUs can serve many requests concurrently, consider an example using OpenAI’s estimates: ~60 W for 10 seconds (AI) versus ~20 W
This number is way off. You’re comparing one human brain to an entire data center capable of training new models. The actual answer is:
IQ points per watt:
Humans: 5
AI: 7 - 40 (!!)
By my rough calculations the current AI is already served more efficiently than humans for
the US stock market has continually underpriced the opportunity of AI (as @leopoldasch demonstrated) so I wouldn’t be too surprised if the exact opposite of this happens next week as people start waking up to the seriousness of the technology
the US must maintain AI chip advantage while implementing chip tracking and verification methods to reach an enforceable international agreement with China
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our
