The company says the benchmark tests whether an agent can search a knowledge base, reason over retrieved material, execute multi-step tool calls, and continue a live user conversation.
The initial domain, tau-Banking, is a fintech-style customer-support environment. It contains 698 documents across 21 product categories, totaling about 195,000 tokens. The materials include product information such as account fees, APY rates and card rewards, alongside internal procedures for disputes, replacement cards, retention offers and identity verification.
Sierra reports that a tau-knowledge approach matched a baseline's accuracy while using 64% fewer tokens. The supplied source does not identify the baseline in this material, establish whether the result transfers to production customer-support systems, or provide an independent replication of the measurement.