Relevance AI adds cost breakdowns and Workforce coverage to agent evals

Relevance AI updated Evals, its testing feature for AI agents, on June 29, 2026. Each completed Eval run now shows a cost breakdown panel that itemizes credit and action use by component, including the Scenario Runner, Agent Execution and Checks. Workforce runs that call sub-agents or tools now have each of those interactions recorded and evaluated in the same way as single-agent runs. Users open any completed Eval run to see the panel.

The details

• Checks account for one action per run across every check type, including LLM as Judge and Tool Usage.
• On June 12, Relevance AI added Google's Gemini 3.5 Flash for all users, with a 1M-token context window and up to 64K output tokens.
• Gemini 3.5 Flash offers four thinking levels, with minimal as the default.
• On July 22, the company added concurrency charts showing queued runs, live slot counts and per-project capacity for project and organization admins.

Context

The Evals update is one of several cost and usage visibility changes in Relevance AI's mid-2026 changelog, followed in July by the concurrency monitoring view on the Analytics and Plan and Billing pages. Relevance AI sells a platform for building AI agents and multi-agent Workforces. In September 2026 the company released a new version of Invent, its builder assistant. The June changelog does not state whether the cost panel applies to Eval runs completed before the update.

Not disclosed

• Credit prices per component
• Plan availability