Cheng-Yuan (Sam) Lee

@cl571128

Research @cognition | prev intern @windsurf | 2x ICPC World Finalist | from Taiwan 🇹🇼

College Park, MD

Joined July 2023

Replies Reposts Media

Cheng-Yuan (Sam) Lee
@cl571128
Sep 3
In our early testings, we found that GPT-6 Astra sometimes creates extra test files and ignores codebase conventions. But it follows instructions so well that adding a few sentences to the system prompt fixed it.
Cognition
@cognition
Sep 3
GPT-6 Astra is coming to Devin. On FrontierCode 1.1, Astra performs within 0.4 points of Fable 5 at a 64% lower cost. It also sets a new SOTA on our internal testing benchmark, generating more comprehensive tests, clearer reports, and better video evidence.
11
Cheng-Yuan (Sam) Lee
@cl571128
Sep 2
We have updated FrontierCode leaderboard to include more models! Check it out! frontiercode.com
9
Cheng-Yuan (Sam) Lee
@cl571128
Aug 19
Since I started using Devin a lot late last year, I haven’t been able to go back. The interaction just feels natural, and it makes getting things done so easy. Do it all with Devin!
Scott Wu
@ScottWu46
Aug 19
At Cognition, one of our values is to go for it all: when faced with a tradeoff, pick the ambition-maximizing direction. When we launched Devin in 2024, we envisioned a future where every team had an infinite army of junior engineers. We were early and Devin wasn't good enough.
Cheng-Yuan (Sam) Lee
@cl571128
Jul 24
We've received several questions about the Opus 5 FrontierCode results, where scores decline as reasoning effort increases. In fact, the behavior is expected under the benchmark design. FrontierCode evaluates merge-ability rather than correctness alone, incorporating criteria
27
Cheng-Yuan (Sam) Lee
@cl571128
Jul 8
One day, all of the research team spent hours in a room together manually solving the RL tasks we used to evaluate our models. I remember solving one of the tasks and realized that the tests are not even testing what the agent was asked to do. Since then, data has become one of
Cognition
@cognition
Jul 8
Introducing SWE-1.7, the most capable model we’ve trained yet. It scores within a few points of the strongest frontier models at a fraction of the cost, and is now available at 1000 tok/s. RL is not hitting its limit: after refining our recipe, we keep seeing gains as we scale
16