Pinned
It’s been great working on DiG-bench with @jcrwhittington and everyone else: a new text-based benchmark for in-context discovery (Discovery in Games).
DiG-bench is within-domain for language models, but contains new text-based concepts and mechanisms that have to be discovered
We’re excited to announce DiG-bench, a new benchmark for discovery!
Over the last few weeks we’ve been testing frontier AI models on our novel discovery games and seeing how they score. Each game is a text-based environment, so they probe discovery capabilities in the natural
