We designed a small scale randomized controlled experiment to determine how much LLMs really help speed up a specific task, with a somewhat humbling outcome - AI-assisted runs were more than twice as fast
(collaboration with @PrincetonCITP)
Can AI agents help researchers reproduce research more quickly? We conducted an uplift study. The answer is yes: researchers reproduced papers > 2x faster using Codex with GPT-5.4 xhigh. In a new paper, we show many other results.
![[Prompt]
Write a rap battle between Wikidata and Wikipedia, where Wikipedia disses Wikidata for lacking references, and Wikidata disses Wikipedia for its age and lack of structure
[ChatGPT]
(Verse 1: Wikipedia)
I'm the OG, the one you know,
With my encyclopedic knowledge, I'm the real pro.
I've been around for years, and I've seen it all,
From ancient history to the rise and fall.
But you, Wikidata, you're just a copycat,
Lacking in depth, and that's a fact.
You may have structured data, but where's your proof?
Your entries lack citations, that's the truth.
(Verse 2: Wikidata)
You may be older, Wikipedia, but that don't impress,
With your outdated info, you're just a mess.
I may not have references, but I've got structure,
Organized and neat, without a single fracture.
And sure, you may have depth, but who can sift through it all?
Your endless pages, they're just a ball.
But me, I'm concise, and easy to navigate,
With my structured data, I dominate.
(Verse 3: Wikipedia)
You [...]](https://pbs.twimg.com/media/Fi_99tDVsAAfNoX?format=webp&name=medium)
