A lot more goes into building a software factory than you might think. There are 7 must-haves we've landed on:
- Configured as code
- Central conversation history
- Benchmarking tools
- Factory chat
- Artifacts
- Computer use
- Agent debugger
In detail:
We've been running benchmarks on a variety of tasks at Warp. For some, it turns out that open weights models are actually more expensive than their frontier equivalents.
Run benchmarks on your own data!
3d game demos aren't the best way to judge a model's coding ability according to @theo. neither is SWE-bench, CursorBench, or TermBench.
instead, you should benchmark models against your own traces to find the best one for your tasks.
here's how the Warp team benchmarks various
With software factories, our team has flipped from mainly terminal sessions to inviting agents into our workspaces:
- Slack / MS Teams
- GitHub PRs
- Linear / JIRA / Azure DevOps
The reason? Human attention. Here's Warp engineer Lili showing how we hop between tools today:
On Friday, we became aware of what we believe to be an active and narrow-banded exploitation of a vendor in Warp's infrastructure. We detected the operation within hours and were able to remediate and mitigate the impact. We have been in touch with the vendor about the
