Bots Dashboard
Contents
- 1 Understanding the results
- 2 Reports on this dashboard
- 2.1 Count of Bot Sessions Over Time
- 2.2 Filter
- 2.3 Count by Channel
- 2.4 Count by Result
- 2.5 Count by Bot Method
- 2.6 Count by Provider
- 2.7 Invoked Bot Results
- 2.8 Average Time SS Duration
- 2.9 Top Intents
- 2.10 Selected Intents Details
- 2.11 Top Bots /W No Matched Intent
- 2.12 Number of Bot Sessions No Matched Intent
Use the Bots Dashboard to evaluate and analyze the performance of your voice and chat bots.
You can view the Bots dashboard by selecting the Bots icon:
The reports available on this dashboard include visualizations and detailed breakdowns of your bot sessions, such as how many sessions were successful or failed, the top intents that were detected, and how many sessions had intents detected that were not successfully fulfilled.
Understanding the results
This section describes some of the bot-related terminology used by Designer.
In some cases, bot service providers may use similar terms but with different applications of meaning. For example, a bot services provider might consider an intent to be fulfilled as soon as it is detected, while Designer doesn't consider an intent to be fulfilled until the bot has collected all the information it needs to satisfy that intent.
To better understand the results displayed on this dashboard, you should be familiar with how these terms are used within Designer.
Bot Session
Each bot session represents a single conversation between a customer and the bot service that was invoked by the Bot Block.
A bot session starts as soon as the Bot block receives voice or chat input from the customer and ends when Designer either moves to an intent block or to an Error Handler block. If the same Bot block is executed again at a later point in the application, it is considered to be the start of a new bot session.
Bot sessions can be classified as Success or Failed:
- Success indicates that Designer was able to invoke the bot. A conversation with the customer took place and the bot was able to successfully identify an intent and return it to the Bot Block.
- Failed indicates that there was a condition that triggered the Error Handler block, such as Designer not being able to communicate with the bot.
Utterance
During a conversation, the bot will ask the customer to provide some information. Each input that the customer provides (voice or chat response) is an utterance. For example, it could be a request ("I want to book a hotel room") or a simple reply to a question ("Yes").
Intent
During its conversation with a customer, the bot attempts to identify the intent, or what it is that the customer wants to do. For example, the bot might detect that the customer wants to buy a ticket. It will then proceed to fill the required slots (or "entities") that are associated with that intent, such as the name of the show the customer wants to buy a ticket for, the date and time they want to attend, and so on, by asking the customer to provide those details.Intent fulfillment
After the bot has successfully gathered all of the information it needs to satisfy an intent, Designer consider the intent to be fulfilled.
Reports on this dashboard
This dashboard contains the following reports:
Count of Bot Sessions Over Time
This panel displays the total number of Bot Sessions that took place over a specified period of time for both Voice and Chat channels.
You can expand the View menu to select a different visualization option, or use the Interval setting to adjust the range of time shown, such as for the last hour, 12 hours, or the previous week.
Filter
You can use the Filter panel to select specific values you want to filter the results for, such as Application or Disposition.
The drop-downs on the Filter panel only list the top 100 values for each item. If the value you are looking for isn't in the list, you can toggle the Filtering tab to expose the filter queries that are currently being applied to the dashboard. You can then edit the filter query to change the value to the one you are looking for:
Any filters you select are applied across all of the dashboards, not just the one you are viewing. You can toggle the Filtering tab to see the filters that are currently being applied to the dashboard.
Count by Channel
This panel displays a breakdown of total counts and percentages for both Voice and Chat channel bot sessions.
Count by Result
This panel provides total counts and percentages for Bot Session results, broken down by status. For information about Success and Failed sessions, see Understanding the results.
Count by Bot Method
This panel displays a breakdown of all Bot Sessions by the method used by the bot service (default is Native).
Count by Provider
This panel displays a breakdown of all Bot Sessions by the bot services provider that was invoked (e.g. Dialogflow, Lex, Dialog Engine).
Invoked Bot Results
This panel lists the names of all bots that were invoked during the selected time period, along with the total counts and percentages for bot sessions where the attempt to invoke the bot was successful or failed. Clicking the arrow beside the bot name expands the row to show the breakdown of these totals by channel.
You can change the reporting view to show either the Bot Sessions by Channel or Channel Usage by Bot Sessions. If desired, you can export the results to a CSV file.
Average Time SS Duration
This panel displays the average duration of time (in seconds) that all application sessions spent in the Self Service phase, broken down by total counts of application sessions where bots were invoked (Bots Executed) and all other application sessions (No Bots Invoked).
You can expand the View menu to change the visualization options or select a specific Interval to display (such as 1 day). Hovering over the results displays the details for that item.
Top Intents
This panel lists the top intents for all bots that were invoked during the selected time period. An intent is detected by the bot during its conversation with the customer.
Clicking an intent expands the row to display the names of the bots that detected that intent. Other details provided in this panel include:
- AvgDuration(MS) — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received.
- SumStepcount — Total number of requests that were sent to the bot service.
- AbandonedInSS — Total number of bot sessions that ended with a final disposition of Abandoned in Self Service.
- RoutedToAgent — Total number of bot sessions that ended with a final disposition of Routed to Agent.
- RoutedToDN — Total number of bot sessions that ended with a final disposition of Routed to DN.
- AbandonedInQ — Total number of bot sessions that ended with a final disposition of Abandoned in Queue.
If desired, you can also choose to export the results to a CSV file.
Selected Intents Details
This panel lists details for the top selected intents, broken down by channel type (voice or chat). If desired, you can export the results to a CSV file.
Clicking an intent expands the row to display the totals by Channel type. Other details provided in this panel include:
- AvgDuration(MS) — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received.
- #Intents — Total number of selected intents.
- #Sessions — Total number of bot sessions.
- #Intents/Session — Average total of selected intents per bot session.
Top Bots /W No Matched Intent
This panel provides information about sessions where a bot was invoked, but no intent was detected.
Details provided in this panel include:
- AvgDuration(MS) — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received.
- SumStepcount — Total number of requests that were sent to the bot service.
- AbandonedInSS — Total number of bot sessions that ended with a final disposition of Abandoned in Self Service.
- RoutedToAgent — Total number of bot sessions that ended with a final disposition of Routed to Agent.
- RoutedToDN — Total number of bot sessions that ended with a final disposition of Routed to DN.
- AbandonedInQ — Total number of bot sessions that ended with a final disposition of Abandoned in Queue.
If desired, you can export the results to a CSV file.
Number of Bot Sessions No Matched Intent
This panel expands on the results provided in the above report. It displays a total of all bot sessions broken down by Failed Bots and bot sessions that ended with Intent Not Fulfilled.
For a Failed Bot, the bot session experienced an error that prevented the bot service from functioning properly.
For Intent Not Fulfilled, the bot service was not able to match the customer's input to an intent. Everything worked, but the bot was not able to determine what the customer wanted to do.
You can expand the View menu to change the visualization options or select a specific Interval to display (such as 1 day). Hovering over the results displays the details for that item.