tau2-bench
Multi-turn customer-service benchmark measuring reliability across repetitions, not just best-case success.
On tau2-bench's Telecom domain the top reported score reached 99.3% pass^1 by February 2026, but the metric that maps better to production is pass^k — succeeding k times in a row — not the single-run number.
Key facts
| 2026 frontier | Telecom domain up to 99.3% (pass^1, reported Feb 2026) |
|---|---|
| Credibility | The top of the Telecom leaderboard is saturated — the top three sit within 0.3 points. More useful is the pass^k metric, which asks whether an agent succeeds k times in a row and therefore approximates production reliability better than a single-run score. The 99.3% figure is vendor-reported rather than independently reproduced. |
| Verification | Verified |
| Source | tau2-bench score ledger |
FAQ
What does tau2-bench measure?
Multi-turn customer-service benchmark measuring reliability across repetitions, not just best-case success.
What is the 2026 frontier for tau2-bench?
Telecom domain up to 99.3% (pass^1, reported Feb 2026)
Is tau2-bench credible?
The top of the Telecom leaderboard is saturated — the top three sit within 0.3 points. More useful is the pass^k metric, which asks whether an agent succeeds k times in a row and therefore approximates production reliability better than a single-run score. The 99.3% figure is vendor-reported rather than independently reproduced.