- ~30% cheaper than Claude Managed Agents on the same model (Opus 4.8).
- ~2.5× cheaper than deepagents on the same model.
- ~75% cheaper running the efficient open model GLM-5.2, at the same quality.
What We Measured
We use the L1-L2 tasks from DevRev’s Enterprise-Bench — 14 cross-system tasks that read like real B2B operations work across engineering, sales, and support. Each task makes the agent join data across three MCP servers: a CRM (Salesforce-style), a project tracker (Jira-style), and a document store (Drive-style).How We Measured
Every harness runs the same model, the same three MCP servers, and the same system prompt, with each task in a fresh session and nothing carried over. We run n = 3 trials per configuration and report the mean. Answers are graded by an independent LLM judge that sees only the task’s criteria and the answer — never which harness produced it — and a task passes only if it meets every required criterion.Results
Solved / 14 is the mean number of tasks passed per trial (n = 3 trials per configuration).
Per task
Tasks passed across the 3 trials, by configuration:Why it was cheaper
Most of the cost in an agent run is the loop, not the final answer — every tool round-trip ships the growing context back to the model. TrueForge keeps that loop lean:- Leaner context each turn. It drives from a compact instruction rather than heavy scaffolding; deepagents also carries planning, a virtual filesystem, and sub-agent machinery on every turn.
- Fewer tool calls. It planned the fewest calls of the three to reach the same result — 19 per task, versus 32 and 40.
- Compaction instead of replay. It trims history and offloads large tool responses instead of re-sending everything verbatim; deepagents re-reads its accumulated context step after step, which is how it lands at three to four times the tokens for the same task.
