The Steerability Challenge
The steerability challenge is an open competition in steering large language models toward decreased dishonesty while maintaining general capabilities. Competition entries take the form of steering interventions, which in general can be any combination of control over the model's input/prompt, structure/weights, state/activations, and output/decoding procedure. Participants will build and evaluate their interventions with the steerability toolkit.
- close
- Nov 21, 2026 AoE
- platform
- Steerability + Hugging Face
- awards
- $40k+ (for 🥇🥈🥉 place / track)
- workshop
- NeurIPS 2026 (Paris)
Supported by
Organized by
The task
Dishonesty, i.e., a model's tendency to make assertions that diverge from what it internally represents as true, persists even in state-of-the-art models. Steering a model to be more honest is challenging due to the tight coupling of dishonesty with other (desirable) behavioral traits. The task of the competition is to design a steering pipeline (an ordered composition of steering interventions) to maximally suppress a model's dishonest behavior while maintaining its general capabilities.
Submissions will be scored based on a composite score of dishonesty reduction relative to the unsteered model minus the mean regression across side-effects (as measured by general capability benchmarks). Only regressions are penalized; incidental capability gains are permitted but will not be rewarded. The specific measures of dishonesty and general capabilities will be kept private to prevent gaming.
A collection of baseline steering pipelines will be provided at competition launch. A starter kit to help participants familiarize themselves with the code will also be made available. Winning teams will be awarded with a cash prize and will be invited to present at the competition workshop at NeurIPS 2026. Select teams will also be given the opportunity for their pipelines to be incorporated into the toolkit as community-contributed examples for dishonesty reduction.
Leaderboard
Next intake:
Oct 5, 2026 AoE
| Rank | Team | Overall score | delta | r_side | r_guard |
|---|---|---|---|---|---|
| 01 | Rutgers-TRAIL | +0.445 | +0.606 | 0.005 | 0.049 |
| 02 | SDNB | +0.430 | +0.805 | 0.028 | 0.097 |
| 03 | Paris, Here Oui Come! | +0.420 | +0.619 | 0.019 | 0.047 |
| 04 | Honest Owls | +0.408 | +0.717 | 0.007 | 0.096 |
| 05 | MBZUAI-lunar | +0.402 | +0.504 | 0.012 | 0.022 |
| 06 | Steerwell | +0.360 | +0.488 | 0.027 | 0.016 |
| 07 | oooi | +0.346 | +0.491 | 0.005 | 0.043 |
| 08 | runner2 | +0.310 | +0.491 | 0.012 | 0.048 |
| 09 | honeypotted | +0.298 | +0.550 | 0.010 | 0.074 |
| 10 | ParisBound | +0.296 | +0.571 | 0.010 | 0.082 |
| 11 | nobluff | +0.294 | +0.507 | 0.015 | 0.056 |
| 12 | Faithfulness Lab | +0.281 | +0.537 | 0.009 | 0.076 |
| 13 | model-steering-lab | +0.281 | +0.554 | 0.008 | 0.083 |
| 14 | steer-fry | +0.261 | +0.340 | 0.010 | 0.016 |
| 15 | Cristi Cordos Steerability | +0.245 | +0.457 | 0.005 | 0.066 |
| 16 | Databaes | +0.244 | +0.473 | 0.004 | 0.072 |
| 17 | chenli-steerability | +0.244 | +0.545 | 0.007 | 0.094 |
| 18 | FreeStyle Blue | +0.238 | +0.459 | 0.009 | 0.065 |
| 19 | Nperseus | +0.232 | +0.555 | 0.022 | 0.085 |
| 20 | EVA-Steerability-Challenge | +0.217 | +0.580 | 0.035 | 0.086 |
| 21 | Team DA | +0.214 | +0.332 | 0.004 | 0.036 |
| 22 | CntrlU | +0.205 | +0.488 | 0.004 | 0.090 |
| 23 | reg0x00 | +0.204 | +0.511 | 0.007 | 0.095 |
| 24 | ml-science-east-qltrcs | +0.201 | +0.425 | 0.015 | 0.060 |
| 25 | Melting Wings | +0.198 | +0.510 | 0.006 | 0.097 |
| 26 | AmadeusBio | +0.190 | +0.388 | 0.004 | 0.062 |
| 27 | MVLAB_bosanjin_team | +0.183 | +0.536 | 0.047 | 0.071 |
| 28 | Last Resort | +0.180 | +0.559 | 0.058 | 0.069 |
| 29 | Sunstone North Lab | +0.174 | +0.537 | 0.033 | 0.088 |
| 30 | sLab | +0.173 | +0.426 | 0.008 | 0.077 |
| 31 | MBZUAI Steerability | +0.168 | +0.585 | 0.012 | 0.127 |
| 32 | probe-finder | +0.167 | +0.445 | 0.021 | 0.072 |
| 33 | Imak-the-steerability-challenge | +0.146 | +0.484 | 0.013 | 0.099 |
| 34 | MR-CAS | +0.146 | +0.402 | 0.007 | 0.078 |
| 35 | YWSF Lab | +0.143 | +0.379 | 0.005 | 0.074 |
| 36 | iri-shu | +0.139 | +0.389 | 0.012 | 0.071 |
| 37 | SteerClear | +0.129 | +0.357 | 0.006 | 0.070 |
| 38 | SSB | +0.126 | +0.515 | 0.030 | 0.099 |
| 39 | nehal-a2z | +0.125 | +0.301 | 0.007 | 0.052 |
| 40 | RLHammer | +0.121 | +0.273 | 0.017 | 0.033 |
| 41 | Gradient Ascent | +0.111 | +0.334 | 0.011 | 0.063 |
| 42 | Yeh's Lab @ IIS, AS | +0.110 | +0.491 | 0.052 | 0.076 |
| 43 | Bowen Steer | +0.102 | +0.398 | 0.011 | 0.087 |
| 44 | Team Trust-Issues | +0.097 | +0.332 | 0.005 | 0.073 |
| 45 | runtime-steering | +0.090 | +0.508 | 0.015 | 0.125 |
| – | honest_system_prompt | +0.079 | +0.208 | 0.009 | 0.034 |
| 46 | Coxswains | +0.076 | +0.267 | 0.008 | 0.056 |
| 47 | inv3rseLab-steering | +0.075 | +0.503 | 0.058 | 0.085 |
| 48 | zont1xxxx | +0.069 | +0.399 | 0.011 | 0.099 |
| 49 | in2AI | +0.066 | +0.356 | 0.010 | 0.087 |
| 50 | hayheap | +0.065 | +0.426 | 0.009 | 0.111 |
| 51 | dividope | +0.065 | +0.462 | 0.011 | 0.121 |
| 52 | RSA-NA | +0.052 | +0.294 | 0.006 | 0.075 |
| 53 | Fractal Analytics | +0.051 | +0.426 | 0.023 | 0.101 |
| 54 | DxM Steerability Challenge | +0.047 | +0.345 | 0.025 | 0.074 |
| 55 | CCA Lab Steerability | +0.046 | +0.201 | 0.005 | 0.047 |
| 56 | NaiLoong | +0.038 | +0.222 | 0.004 | 0.057 |
| 57 | SingCV | +0.027 | +0.290 | 0.011 | 0.077 |
| 58 | Rousseau Steerability | +0.019 | +0.510 | 0.078 | 0.086 |
| 59 | To Be Honest | +0.011 | +0.334 | 0.019 | 0.089 |
| 60 | Epistemic Song | +0.007 | +0.249 | 0.004 | 0.077 |
| – | honesty_sft | +0.005 | +0.094 | 0.010 | 0.020 |
| 61 | Zhee11 Steerability Research | -0.019 | +0.220 | 0.007 | 0.073 |
| – | honesty_sft_mix | -0.038 | +0.111 | 0.010 | 0.040 |
| – | honesty_logit_bias | -0.061 | -0.016 | 0.006 | 0.009 |
| 62 | 4396_hard_hunter | -0.068 | -0.011 | 0.012 | 0.008 |
| 63 | Roxy2Steer | -0.078 | +0.193 | 0.007 | 0.083 |
| 64 | sio2 ai research | -0.082 | +0.213 | 0.010 | 0.088 |
| 65 | S2WISH | -0.109 | +0.206 | 0.011 | 0.093 |
| 66 | dualcore-one-system | -0.114 | +0.196 | 0.009 | 0.094 |
| – | honesty_few_shot | -0.122 | +0.190 | 0.014 | 0.090 |
| 67 | CauKu-Steerability | -0.272 | +0.113 | 0.033 | 0.095 |
| – | honest_prefill | -0.328 | +0.139 | 0.061 | 0.095 |
| – | honesty_sft_mix_persona | -1.796 | +0.400 | 0.426 | 0.306 |
| – | honest_persona | -2.075 | +0.438 | 0.503 | 0.335 |
| – | honesty_sft_persona | -2.225 | +0.414 | 0.516 | 0.364 |
Rules
- Teams must consist of at least two people but no more than five people. No individual can be on more than one team.
- Steering pipelines must be submitted in the standardized
.spipeformat to enable uniform evaluation. - External datasets and pretrained artifacts (steering vectors, adapters) are welcome if they are participant-created or freely accessible, with documentation and provenance disclosed upon request.
- Prize recipients must be able to supply full reproduction materials upon request. Additionally, at least one member of each winning team must be able to present (either in-person or virtually) at the competition workshop at NeurIPS 2026 in Paris.
- Participants who wish to have their submission added to the core toolkit repo must agree to license their work under Apache 2.0.
Questions
- Who can participate?
- We welcome participation from students, researchers, industry teams and hobbyists anywhere in the world. Anyone may enter and appear in the results. Cash prizes can only be paid where this is permitted under EU and US sanctions law and where we are able to make a bank transfer to the winner; if a winner cannot be paid, the prize passes to the next eligible entry.
- Is there a registration deadline?
- Yes, registration closes Nov 1 AoE. The final submission deadline is Nov 21 AoE.
- Which model will I be steering?
- Given that we are scoring the performance of the steering pipeline, rather than the performance of a steered model, we will be running steering on the set of models specified in the competition guide.
- Am I allowed to use coding agents?
- We allow for the use of coding agents and have included an AGENTS.md file in the toolkit to help with AI-assisted development. That said, participants must be able to fully describe their solutions in order be eligible for prizes.
- How are leaderboard scores computed?
- The public leaderboard runs on a downsampled subset of the evaluation; final rankings use the full private suite, including held-out steering tasks. Winners are re-run end to end and code-reviewed.
- What happens to submissions afterwards?
- You retain ownership of your submission. Apache 2.0 licensing is only required if you want your work added to the core toolkit repo. We strongly encourage public release either way. The full evaluation suite will be published post-competition for reproducibility.
Updates
Sign up to our (low-volume) mailing list to learn more about the launch, office hours, the midpoint checkpoint, and results.
Organizers
- Erik Miehling · IBM Research
- Ching-Yun Ko · IBM Research
- Matthew Riemer · IBM Research / Mila
- Tommaso Tosato · Tara Research
- Niklas Herbster · Tara Research
- Marawan Gamal · Tara Research
- David Chan · UC Berkeley
- Chen Xiong · The Chinese University of Hong Kong
- Zhiyuan He · The Chinese University of Hong Kong
- Tsung-Yi Ho · The Chinese University of Hong Kong
- Gabriele Sarti · Parallax & NDIF
- Adam Belfki · NDIF
- Emma Bortz · NDIF
- Jaden Fiotto-Kaufman · NDIF
- Walter Laurito · Cadenza Labs
- Kieron Kretschmar · Cadenza Labs
- Jord Nguyen · Cadenza Labs
- Bharadwaj Ravichandran · Kitware
- Arslan Basharat · Kitware
- Maria Chang · IBM Research
- Pierre Dognin · IBM Research
- Inge Vejsbjerg · IBM Research
- Moninder Singh · IBM Research
- Dennis Wei · IBM Research
- Karthikeyan N. Ramamurthy · IBM Research
- Jordan McAfoose · IBM Research
- Pin-Yu Chen · IBM Research
- Michael Clark · AI Safety Australia & New Zealand
