Max Kaufmann

@Max_A_Kaufmann

Maxxing @AnthropicAI. On leave from ML PhD @UofT/@VectorInst | prev. founding team @AISecurityInst, VSR 🎓 @GoogleDeepMind

maxkaufmann.com

Joined May 2017

630 Following

596 Followers

Replies Reposts Media

Max Kaufmann
@Max_A_Kaufmann
Apr 1
Is training against the CoT always bad? RL training can lead to obfuscated CoT making it difficult to 'read an LLMs thoughts'. How can we predict when obfuscation occurs?🤔 Our new @GoogleDeepMind paper introduces a framework to predict this before training starts!
6
Max Kaufmann
@Max_A_Kaufmann
Dec 12, 2025
It would be nice to have cleaner tests as to whether the behaviour was importantly dependant on doing synthetic document finetuning, rather than reward hacking generalising to broad misalignment. I think you could do the same experiments without SDF, by doing a bit of off-policy
Anthropic
@AnthropicAI
Nov 21, 2025
New Anthropic research: Natural emergent misalignment from reward hacking in production RL. “Reward hacking” is where models learn to cheat on tasks they’re given during training. Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious.
1
Max Kaufmann
@Max_A_Kaufmann
Nov 26, 2025
I'm trying to think about where I should donate money. I feel confused about the social value of foreign cash transfers to poor households (like @GiveDirectly ). Naively, I feel like in aggregate the cash transfers might not improve the overall situation in the country, and
3
Max Kaufmann
@Max_A_Kaufmann
Dec 10, 2024
Am at NeurIPS 2024! Drop me a message if you want to chat :))
Max Kaufmann
@Max_A_Kaufmann
May 10, 2024
Inspect is an absolute pleasure to work with, spearheaded by the truly impressive @fly_upside_down
Ian Hogarth
@soundboy
May 10, 2024
1/ Today the UK's AI Safety Institute is open sourcing our safety evaluations platform. We call it "Inspect": gov.uk/government/new…

Max Kaufmann

Max Kaufmann’s posts