The Quality Assurance Dashboard helps you understand the AI's performance across different products and profiles. Key metrics include Average Conversation Duration, Average Interactions per Conversation, Recognition Rate and the Recognition Rate Over Time.

Metrics Computation

• All user interactions with the AI are logged in real-time, as well as handovers to live agents and situations where the AI Cloud/Assistant fails to respond.
• In the dashboard you can view metrics for the last 7, 14, or 30 days, by using the dropdown menu at the top left of the page.
• Note that metrics are available from May 23, 2024, onwards (or from your account creation date if later).

Key metrics

The following metrics/statistics are available:

• Total Conversations: The total number of conversations for the selected time frame.
• Average Conversation Duration: The average length of a conversation for the selected period. A shorter duration suggests users are satisfied with quicker responses.
• Average Interactions per Conversation: The average number of interactions (messages from user or AI) per conversation. Fewer interactions imply users get the needed information faster.
• Recognition Rate: The percentage of questions the AI answers within the selected time frame. A higher rate indicates better performance.
• Recognition Rate Over Time Line-Chart: Shows the day-to-day change in recognition rate. Days without conversations are marked as 0.

Example Analysis

As an example, consider the picture above which shows the following metrics for a selected 14-day period:

• Conversations: 4,063
• Average Duration: 2 minutes and 54 seconds
• Average Interactions: 1.5 message per conversation
• Trends:
• Average Conversation Duration increased by 43.4%, suggesting the user is spending more time in a conversation.
• Average Interactions decreased by 24.4%, indicating less message exchanges per conversation.
• Recognition Rates:
• CAIC: 76.4% (improved with 52.8% from the previous period)
• MSC: 78.0% (unchanged) 

These metrics imply that users are spending more time in conversations but with fewer messages exchanged. Recognition rates are decent, though about 1 in 4 user messages still go unanswered.

---