Lisa Dunlap

@lisabdunlap

messing around with model auditing | prev PhD @berkeley_ai @BerkeleySky and @arena

lisabdunlap.com

Joined October 2021

Replies Reposts Media

Lisa Dunlap
@lisabdunlap
Feb 16
Always great to see more work in model diffing!
Elias Kempf @ ICML 🇰🇷
@elkmf
Feb 16
New model release? Great. But did the LLM’s behavior change in ways the changelog doesn't mention? We built and evaluated a pipeline to find out! We noticed: different model diffing methods often find the same behavior, but may describe it at very different abstraction levels 🧵
1
Lisa Dunlap
@lisabdunlap
Jan 26
With the rise of agents comes the need to better evaluate their true visual capabilities, VisGym takes a step in this direction and analyzes what representations are best for models and how they fail. It was such an honor to be a part of such an incredible team!
Zirui "Colin" Wang
@zwcolin
Jan 26
🎮 We release VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (w/ @junyi42 @aomaru_21490) 🌐 With 17 environments across multiple domains, we show systematically the brittleness of VLMs in visual interaction, and what training leads to. 🧵[1/8]
00:00
Lisa Dunlap
@lisabdunlap
Jan 14
🌟NEW PAPER🌟 Do you know that changing a visual marker from red to blue can completely reorder VLM leaderboards? In our most recent work, we explore the fragility of visually prompted benchmarks. lisadunlap.github.io/vpbench/
6
Lisa Dunlap
@lisabdunlap
Jan 4
If you ever wondered which woman is the most AI famous: its Marie Curie
1
Lisa Dunlap
@lisabdunlap
Dec 29, 2025
New benchmark idea: two LLMs walk into a bar. I simulated a ton of long, drunk party conversations between LLMs, and boy are the conversations entertaining and slightly concerning... Exhibit A: kimi k2 chatting with Kimi k2
1