Research to Advance AI
Scale Labs advances AI through research. Our research focuses on agents, post-training, reasoning, safety, evaluation, and alignment, and the science of data.
[LEADERBOARDS]
Benchmarks for frontier, agentic, and safety capabilities
[PAPERS]
Research papers and publications covering agents, post-training, reasoning, safety, evaluation, and alignment, and the science of data.
DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language Models
[BLOG]
Insights, analysis, and updates from Scale Labs
FineART: Scale AI’s First Open-Source Robotics Dataset and Model
We’re releasing FineART-VLA, an open model co-developed with Hugging Face's LeRobot team. It builds upon FineART, Scale’s first open-source robotics dataset, which includes 40,000 plus robot demonstrations.
DistressBench: A clinician-authored benchmark for how AI models handle users in distress
People increasingly turn to chatbots for emotional support, and some of those conversations involve suicide or self-harm. DistressBench measures whether models give the support those conversations need, using rubrics written by licensed clinicians: 718 clinician-authored conversations across 23 suicide and self-harm subcategories.
Introducing Humanity's Sixth Sense: Measuring Intuitive Visual Reasoning
Today we're introducing Humanity's Sixth Sense (HSS), in partnership with Elorian, a benchmark for intuitive visual reasoning with 522 open-ended tasks across 288 images and 234 video clips (17.6 hours of video in total).
Introducing AgentEnv: An Open-Source Framework for Building RL Environments
An agent is only as good as the world it practices in, and building worlds that are realistic, accurate, and scalable all at once is the hard part of scaling RL. Every RL environment Scale builds runs on AgentEnv. Today we're open-sourcing it.
View allAll posts