Toolathlon
General capabilityVerifiedLast updated 2026-10-04
The Tool Decathlon: long-horizon tasks across 32 apps and 604 tools.
Toolathlon spans 108 hand-crafted tasks averaging about 20 turns across 32 applications and 604 tools, and the best model reaches only 38.6% (Claude-4.5-Sonnet) with the best open-source model at 20.1%.
Key facts
| 2026 frontier | Best 38.6% (Claude-4.5-Sonnet); best open-source 20.1% (DeepSeek-V3.2-Exp) |
|---|---|
| Credibility | Shows how far agents are from long-horizon work: 108 hand-crafted tasks averaging about 20 turns each, spanning 32 applications and 604 tools, and no system reaches 40%. The best open-source result is roughly half the best closed result, which is the widest such gap in this list. |
| Verification | Verified |
| Source | Toolathlon (arXiv:2510.25726) |
FAQ
What does Toolathlon measure?
The Tool Decathlon: long-horizon tasks across 32 apps and 604 tools.
What is the 2026 frontier for Toolathlon?
Best 38.6% (Claude-4.5-Sonnet); best open-source 20.1% (DeepSeek-V3.2-Exp)
Is Toolathlon credible?
Shows how far agents are from long-horizon work: 108 hand-crafted tasks averaging about 20 turns each, spanning 32 applications and 604 tools, and no system reaches 40%. The best open-source result is roughly half the best closed result, which is the widest such gap in this list.