Working with Intent Accuracy Report
Contents
Learn to understand and use the Intent Accuracy Report feature.
What is the Intent Accuracy Report?
The intent accuracy report is a visual representation that depicts intents that can be mistaken for others intents by your bot. The intent accuracy report shows an overview of the predicted intent vs the expected intent. It can help you understand why the bot predicted an intent as Intent X instead of predicted it as Intent Y. The number of correct and incorrect predictions are summarized with count values and broken down by each intent. The expected intents are shown as rows, the predicted intents are shown as columns.
When to use the Intent Accuracy Report
You can use the Intent Accuracy Report when you create a bot for the first time. As the report provides you an insight into how your bot will perform in the real world. Based on the training utterances, the report will show if the bot will have some confusion between intents when working with live data. If you find that the utterances used by customers are different from your training utterances, this report will not be accurate. To overcome this, ensure that you provide some good training utterances.
You can also use the Intent Accuracy Report whenever you add more utterances or newer intents. This report will indicate how your bot will perform when the new changes are included.
Understanding the Intent Accuracy Report
In the intent accuracy report, the expected intents are shown as rows, the predicted intents are shown as columns. The diagonal cells indicate that the predicted and expected intents match. Each cell in the matrix is colored according to the intent accuracy.
- Intents that are highly likely to be mistaken from another intent and require immediate attention are marked as Critical and are highlighted in red.
- Intents that have a likelihood to be mistaken for another intent are marked as Advisory and are highlighted in yellow.
- Intents that have a low likelihood to be mistaken for another intent are marked as Minimal and are marked in dark grey.
To view more information on how the bot classified utterances to that intent, click any cell to display a detailed breakdown of how the intent was classified. For example, in the following example, The Query Rebate Status intent was classified as View Previous Rebates intent in 20% of cases and as File Rebate Returns in 20% of cases.
You can toggle the display of the Critical, Advisory, and Minimal highlights. You can view the percentages for each marked cell by toggling the Percentages option.
For example, if you want to view only Critical intents and the Percentages:
How to improve the Intent Accuracy Report
If your intent accuracy report shows that you may need to make improvements by showing Critical or Advisory intents, you can try these top tips:
| Top tip | Action to take |
| Do you have enough utterances for each intent? | Ensure you have at least 10 or 20 utterances for each intent. |
| Are you training your bot evenly? | Ensure you have an even distribution of utterances across intents. For example, ensure all intents have around 10 utterances. Do not have one intent that has more utterances than other intents. |
| Are you using a similar phrasing in utterances across different intents? | If Intent A is being mistaken as Intent B, you can go to the utterances in Intent A and Intent B and modify them, so they are not similar in both intents. |
| Should any intents be merged? | It may make sense for you to merge two intents, for example, you may have two intents for making payments that you can merge into one single intent. You can combine Intent A and Intent B by copying the utterances from Intent A into Intent B and deleting Intent A. |
| Have you thought of your prompts for the intents? Can they be improved to help users understand what the bot is asking for? | You can update your prompts by updating:
|