Scale Labs

@ScaleAILabs

welcome to the lab. from the researchers at @scale_AI

labs.scale.com

Joined October 2025

Replies Reposts Media

Scale Labs
@ScaleAILabs
Sep 23
We're releasing HLE-Diamond: a refined version of Humanity's Last Exam, built with @CAIS. A year of review and community feedback went into refining this subset to make it more reliable for measuring frontier models. Top model tested 60.6% overall. We expect HLE-Diamond to
4
Scale Labs
@ScaleAILabs
Sep 22
Over a year ago, we released SWE-Bench Pro. We refreshed the benchmark today to improve its quality based on community feedback. Our update is meant to ensure the benchmark is clean and accurately measures model capabilities on SWE-Bench Pro tasks. In our update, we observe a
Scale Labs
@ScaleAILabs
Sep 22
SWE-Bench Pro V2 is live. What’s new: 🧵
3
Scale Labs
@ScaleAILabs
Sep 22
SWE-Bench Pro V2 is live. What’s new: 🧵
19
Scale Labs
@ScaleAILabs
Sep 21
600+ proposals in 🤯 We're reaching out to top contributors to start building in our public repo. We’ve also welcomed @SchmidhuberAI, a pioneer of RSI, as a senior advisor to RSI Bench. New blog on our setup and verification pipeline: rsi-benchmark.com/blog/verificat…
MohammadHossein Rezaei
@mhrezaeics
Aug 17
Launching rsi-benchmark.com: The work of AI R&D has always belonged to humans. For the first time, though, it no longer seems certain that it always will. Recursive self-improvement is within a line of sight. It may still be far, but it is close enough that we should
4
Scale Labs
@ScaleAILabs
Sep 17
New research from Scale Labs: SteerDuplex and SteerBench measure how well voice models follow instructions for how a response should sound and be delivered. +44.5 points on audio steering over the best open baseline. Pause barge-ins cut from 26.5% to 9%. Plus a sharp result on
Utkarsh Tyagi
@utkarsh4430
Sep 17
Full-duplex voice models can listen and speak at the same time, but can they change how they speak when you ask them to? Introducing SteerDuplex + SteerBench: the first benchmark that measures steerable full-duplex dialogue. 65.1% audio-steering APR vs 20.6% for the best open
1