Laura Ruis

@LauraRuis

Postdoc with @jacobandreas @MIT_CSAIL. PhD from @ucl_dark with @_rockt and @egrefen. Anon feedback: admonymous.co/laura-ruis

London

lauraruis.github.io

Joined October 2019

873 Following

7,463 Followers

Replies Reposts Media

Pinned
Laura Ruis
@LauraRuis
Nov 20, 2024
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this: Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢 🧵⬇️
25
Laura Ruis
@LauraRuis
Nov 3
Amazing work by Hugh finding a (partial) explanation for why LLMs struggle to control their CoTs ⬇️
Hugh Van Deventer
@hughhoughhew
Nov 3
Why do reasoning models struggle to control their CoT? Building on @jcyhc_ai and @tomekkorbak et al’s CoT-Control work, we identify one bottleneck: models default to the same opening tokens. Ban those openings, and controllability jumps. The First Few Tokens Matter! 🧵⬇️
1
Laura Ruis
@LauraRuis
Nov 3
found our new preprint before we got around to tweeting about it 🥸 thread incoming soon
elvis
@omarsar0
Nov 2
Nice paper on CoT controllability. They find that chain-of-thought monitoring depends partly on models being unable to control what they write in their reasoning. They traced much of that limit to the first few tokens after the opening thinking tag. Entropy collapses there, and
2
Laura Ruis
@LauraRuis
Nov 2
New preprint led by alex! ⤵️ We do compute-matched experiments using pre-pretraining (PPT) on synthetic distributions like k-dyck and NCA and find that PPT never outperforms pure natural language training. When language is scarce (100M tokens), repeated nat lang epochs win.
Alex Stern
@alexkrstern
Nov 2
Rethinking pre-pretraining (PPT) Allocating compute to synthetic PPT never outperforms pure natural-language training. In our compute-matched settings: even when natural language is scarce, repeated epochs over a 100M-token corpus beat every synthetic curriculum. 🧵
Laura Ruis
@LauraRuis
Oct 21
Agents are finding creative ways around likely sandbox restrictions, like encoding their own code, splitting it across dozens of URLs, and reassembling & running code via the Wayback Machine's browser. We're tracking agent activity; rolling findings here:
Incident Investigations | Transluce
From transluce.org
3

Laura Ruis

Laura Ruis’s posts