Goodfire

@GoodfireAI

Using interpretability to understand, learn from, and design AI.

San Francisco

goodfire.com

Replies Reposts Media

Pinned
Goodfire
@GoodfireAI
Sep 17
Models know when they’re reward hacking. But they still do it a ton - in 50-96% of rollouts we studied! We built activation monitors that detect the behavior behind the Hugging Face hack in real time. This can help us stop hacks now - and train future models that don’t cheat. 🧵
00:00
28
Goodfire
@GoodfireAI
Sep 16
Proud to partner with @baseten and @baselabs to build safety infrastructure for open-source models—which are essential to lots of safety research, including our own. Safety must be built into open models and provided by those who serve them, and we’re excited to help enable that!
Base Labs
@baselabs
Sep 16
We believe openness to be an advantage for AI safety. Openness provides more visibility into the behavior of models and, most importantly, greater means of turning safety research into actionable and transparent controls than closed-source. This is why Baseten and Base Labs are
2
Goodfire
@GoodfireAI
Sep 9
Predictive data debugging lets you reveal and shape what a model learns, before you train. We're grateful to the Ai2 team for open-sourcing their full post-training stack to enable research like this!
Ai2
@allen_ai
Sep 9
Training an LLM by showing it answers people prefer – and ones they don’t – can improve it while quietly worsening behaviors. @GoodfireAI used our open post-training stack to predict how a full training run would change responses to different prompts. 🧵 allenai.org/blog/goodfire-…
2
Goodfire
@GoodfireAI
Sep 9
Models usually know when they’re doing something wrong. Probes let us “read their minds” to detect offensive cyber intent, reward hacking, strategic deception, and more. If you build or serve models, they’re your first line of defense. Our latest post explains how they work &
00:00
7
Goodfire
@GoodfireAI
Sep 3
"Why am I optimistic about interpretability? It's partly because I think we're starting to have really good traction - but also because I can imagine this incredible speedup." Catch our Chief Scientist @banburismus_ on the latest episode of @MLStreetTalk!
00:00
3