ServiceNow has launched EVA (Evaluation of Voice Agents), a new framework for evaluating voice-based AI agents. The solution is designed to measure both accuracy and user experience in conversations.
EVA – New Framework for Evaluating Voice-Based Agents
EVA is a comprehensive evaluation framework that analyzes complete, multi-step voice conversations. It provides two main scores: EVA-A, which measures task completion accuracy, and EVA-X, which assesses the quality of the user experience. Together, these offer a more complete understanding of how an AI agent actually performs in practice.
The framework stands out by combining evaluation of both task completion and conversation quality in a single model. Previous methods have often assessed these aspects separately, making it difficult to get a holistic view of performance.
EVA also includes a dataset with 50 scenarios from the aviation industry, developed to test various types of conversational interactions and complex user needs.
Source: Hugging Face
Read also: Meta AI Introduces Hyperagents for Self-Improvement
