Difference between revisions of "DES/Current/Designer/BotsDash"
(Published) |
m (Ed.jamer@genesys.com moved page PEC-ROU/Current/Designer/BotsDash to DES/Current/Designer/BotsDash: Text replacement - "PEC-ROU/Current/Designer/" to "DES/Current/Designer/") |
||
| (4 intermediate revisions by one other user not shown) | |||
| Line 22: | Line 22: | ||
|structuredtext=This section describes some of the bot-related terminology used by Designer. | |structuredtext=This section describes some of the bot-related terminology used by Designer. | ||
| − | In some cases, bot service providers may use similar terms but with different applications of meaning. For example, a bot services provider might consider an intent to be ''fulfilled'' as soon as it is detected, while Designer doesn't consider an intent to be fulfilled until the | + | In some cases, bot service providers may use similar terms but with different applications of meaning. For example, a bot services provider might consider an intent to be ''fulfilled'' as soon as it is detected, while Designer doesn't consider an intent to be fulfilled until the bot has collected all the information it needs to satisfy that intent. |
To better understand the results displayed on this dashboard, you should be familiar with how these terms are used within Designer.{{AnchorDiv|session}} | To better understand the results displayed on this dashboard, you should be familiar with how these terms are used within Designer.{{AnchorDiv|session}} | ||
| Line 28: | Line 28: | ||
===Bot Session=== | ===Bot Session=== | ||
{{AnchorDiv|session}} | {{AnchorDiv|session}} | ||
| − | Each ''' | + | Each '''bot session''' represents a single conversation between a customer and the bot service that was invoked by the {{Link-SomewhereInThisVersion|manual=Designer|topic=BotBlock}}. |
| − | A session starts as soon as the '''Bot''' block receives voice or chat input from the customer and ends when Designer either moves to an intent block or to an '''Error Handler''' block. If the same '''Bot''' block is executed again at a later point in the application, it is considered to be the start of a new session. | + | A bot session starts as soon as the '''Bot''' block receives voice or chat input from the customer and ends when Designer either moves to an intent block or to an '''Error Handler''' block. If the same '''Bot''' block is executed again at a later point in the application, it is considered to be the start of a new bot session. |
| − | Bot | + | Bot sessions can be classified as '''Success''' or '''Failed''': |
*'''Success''' indicates that Designer was able to invoke the bot. A conversation with the customer took place and the bot was able to successfully identify an intent and return it to the {{Link-SomewhereInThisVersion|manual=Designer|topic=BotBlock}}. | *'''Success''' indicates that Designer was able to invoke the bot. A conversation with the customer took place and the bot was able to successfully identify an intent and return it to the {{Link-SomewhereInThisVersion|manual=Designer|topic=BotBlock}}. | ||
| Line 41: | Line 41: | ||
{{AnchorDiv|intent}} | {{AnchorDiv|intent}} | ||
===Intent=== | ===Intent=== | ||
| − | During its conversation with a customer, the bot attempts to identify the '''intent''', or what it is that the customer wants to do. For example, the bot might detect that the customer wants to buy a ticket. It will then proceed to fill the required slots | + | During its conversation with a customer, the bot attempts to identify the '''intent''', or what it is that the customer wants to do. For example, the bot might detect that the customer wants to buy a ticket. It will then proceed to fill the required slots (or "entities") that are associated with that intent, such as the name of the show the customer wants to buy a ticket for, the date and time they want to attend, and so on, by asking the customer to provide those details. {{AnchorDiv|fulfill}} |
| − | |||
| − | |||
| − | |||
| − | |||
| − | {{AnchorDiv|fulfill}} | ||
===Intent fulfillment=== | ===Intent fulfillment=== | ||
| − | After the | + | After the bot has successfully gathered all of the information it needs to satisfy an intent, Designer consider the intent to be fulfilled. |
| − | |||
| − | |||
|Status=No | |Status=No | ||
}}{{Section | }}{{Section | ||
| Line 56: | Line 49: | ||
|anchor=reports | |anchor=reports | ||
|alignment=Vertical | |alignment=Vertical | ||
| − | |structuredtext= | + | |structuredtext=This dashboard contains the following reports: |
===Count of Bot Sessions Over Time=== | ===Count of Bot Sessions Over Time=== | ||
| Line 96: | Line 89: | ||
===Invoked Bot Results=== | ===Invoked Bot Results=== | ||
| − | This panel lists the names of all bots that were invoked during the selected time period, along with the total counts and percentages for sessions where the attempt to invoke the bot was successful or failed. Clicking the arrow beside the bot name expands the row to show the breakdown of these totals by channel. | + | This panel lists the names of all bots that were invoked during the selected time period, along with the total counts and percentages for bot sessions where the attempt to invoke the bot was successful or failed. Clicking the arrow beside the bot name expands the row to show the breakdown of these totals by channel. |
You can change the reporting view to show either the '''Bot Sessions by Channel''' or '''Channel Usage by Bot Sessions'''. If desired, you can export the results to a CSV file. | You can change the reporting view to show either the '''Bot Sessions by Channel''' or '''Channel Usage by Bot Sessions'''. If desired, you can export the results to a CSV file. | ||
| Line 103: | Line 96: | ||
===Average Time SS Duration=== | ===Average Time SS Duration=== | ||
| − | This panel displays the average duration of time (in seconds) that all sessions spent in the '''Self Service''' phase, broken down by total counts of sessions where bots were invoked ('''Bots Executed''') and all other sessions ('''No Bots Invoked'''). | + | This panel displays the average duration of time (in seconds) that all application sessions spent in the '''Self Service''' phase, broken down by total counts of application sessions where bots were invoked ('''Bots Executed''') and all other application sessions ('''No Bots Invoked'''). |
You can expand the '''View''' menu to change the visualization options or select a specific '''Interval''' to display (such as 1 day). Hovering over the results displays the details for that item. | You can expand the '''View''' menu to change the visualization options or select a specific '''Interval''' to display (such as 1 day). Hovering over the results displays the details for that item. | ||
| Line 110: | Line 103: | ||
===Top Intents=== | ===Top Intents=== | ||
| − | This panel lists the top intents for all bots that were invoked during the selected time period. | + | This panel lists the top intents for all bots that were invoked during the selected time period. An '''intent''' is detected by the bot during its conversation with the customer. |
Clicking an intent expands the row to display the names of the bots that detected that intent. Other details provided in this panel include: | Clicking an intent expands the row to display the names of the bots that detected that intent. Other details provided in this panel include: | ||
| − | *'''AvgDuration(MS)''' — Average duration of time (in milliseconds) that the | + | *'''AvgDuration(MS)''' — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received. |
*'''SumStepcount''' — Total number of requests that were sent to the bot service. | *'''SumStepcount''' — Total number of requests that were sent to the bot service. | ||
| − | *'''AbandonedInSS''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Self Service'''. | + | *'''AbandonedInSS''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Self Service'''. |
| − | *'''RoutedToAgent''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to Agent'''. | + | *'''RoutedToAgent''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to Agent'''. |
| − | *'''RoutedToDN''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to DN'''. | + | *'''RoutedToDN''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to DN'''. |
| − | *'''AbandonedInQ''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Queue'''. | + | *'''AbandonedInQ''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Queue'''. |
[[File:des_bots_dash_top_intents.png]] | [[File:des_bots_dash_top_intents.png]] | ||
| Line 132: | Line 125: | ||
Clicking an intent expands the row to display the totals by '''Channel''' type. Other details provided in this panel include: | Clicking an intent expands the row to display the totals by '''Channel''' type. Other details provided in this panel include: | ||
| − | *'''AvgDuration(MS)''' — Average duration of time (in milliseconds) that the | + | *'''AvgDuration(MS)''' — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received. |
*'''#Intents''' — Total number of selected intents. | *'''#Intents''' — Total number of selected intents. | ||
| − | *'''#Sessions''' — Total number of sessions. | + | *'''#Sessions''' — Total number of bot sessions. |
| − | *'''#Intents/Session''' — Average total of selected intents per session. | + | *'''#Intents/Session''' — Average total of selected intents per bot session. |
===Top Bots /W No Matched Intent=== | ===Top Bots /W No Matched Intent=== | ||
| Line 144: | Line 137: | ||
Details provided in this panel include: | Details provided in this panel include: | ||
| − | *'''AvgDuration(MS)''' — Average duration of time (in milliseconds) that the | + | *'''AvgDuration(MS)''' — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received. |
*'''SumStepcount''' — Total number of requests that were sent to the bot service. | *'''SumStepcount''' — Total number of requests that were sent to the bot service. | ||
| − | *'''AbandonedInSS''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Self Service'''. | + | *'''AbandonedInSS''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Self Service'''. |
| − | *'''RoutedToAgent''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to Agent'''. | + | *'''RoutedToAgent''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to Agent'''. |
| − | *'''RoutedToDN''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to DN'''. | + | *'''RoutedToDN''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Routed to DN'''. |
| − | *'''AbandonedInQ''' — Total number of sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Queue'''. | + | *'''AbandonedInQ''' — Total number of bot sessions that ended with a {{Link-SomewhereInThisVersion|manual=Designer|topic=ApplicationPhases|anchor=dispositions|display text=final disposition}} of '''Abandoned in Queue'''. |
If desired, you can export the results to a CSV file. | If desired, you can export the results to a CSV file. | ||
| Line 155: | Line 148: | ||
===Number of Bot Sessions No Matched Intent=== | ===Number of Bot Sessions No Matched Intent=== | ||
| − | This panel displays | + | This panel expands on the results provided in the above report. It displays a total of all bot sessions broken down by '''Failed Bots''' and bot sessions that ended with '''Intent Not Fulfilled'''. |
| + | |||
| + | For a '''Failed Bot''', the bot session experienced an error that prevented the bot service from functioning properly. | ||
| + | |||
| + | For '''Intent Not Fulfilled''', the bot service was not able to match the customer's input to an intent. Everything worked, but the bot was not able to determine what the customer wanted to do. | ||
| − | [[File: | + | [[File:des_bots_dash_no_matched_intent_3.png]] |
You can expand the '''View''' menu to change the visualization options or select a specific '''Interval''' to display (such as 1 day). Hovering over the results displays the details for that item. | You can expand the '''View''' menu to change the visualization options or select a specific '''Interval''' to display (such as 1 day). Hovering over the results displays the details for that item. | ||
Latest revision as of 13:59, April 21, 2021
Contents
- 1 Understanding the results
- 2 Reports on this dashboard
- 2.1 Count of Bot Sessions Over Time
- 2.2 Filter
- 2.3 Count by Channel
- 2.4 Count by Result
- 2.5 Count by Bot Method
- 2.6 Count by Provider
- 2.7 Invoked Bot Results
- 2.8 Average Time SS Duration
- 2.9 Top Intents
- 2.10 Selected Intents Details
- 2.11 Top Bots /W No Matched Intent
- 2.12 Number of Bot Sessions No Matched Intent
Use the Bots Dashboard to evaluate and analyze the performance of your voice and chat bots.
You can view the Bots dashboard by selecting the Bots icon:
The reports available on this dashboard include visualizations and detailed breakdowns of your bot sessions, such as how many sessions were successful or failed, the top intents that were detected, and how many sessions had intents detected that were not successfully fulfilled.
Understanding the results
This section describes some of the bot-related terminology used by Designer.
In some cases, bot service providers may use similar terms but with different applications of meaning. For example, a bot services provider might consider an intent to be fulfilled as soon as it is detected, while Designer doesn't consider an intent to be fulfilled until the bot has collected all the information it needs to satisfy that intent.
To better understand the results displayed on this dashboard, you should be familiar with how these terms are used within Designer.
Bot Session
Each bot session represents a single conversation between a customer and the bot service that was invoked by the Bot Block.
A bot session starts as soon as the Bot block receives voice or chat input from the customer and ends when Designer either moves to an intent block or to an Error Handler block. If the same Bot block is executed again at a later point in the application, it is considered to be the start of a new bot session.
Bot sessions can be classified as Success or Failed:
- Success indicates that Designer was able to invoke the bot. A conversation with the customer took place and the bot was able to successfully identify an intent and return it to the Bot Block.
- Failed indicates that there was a condition that triggered the Error Handler block, such as Designer not being able to communicate with the bot.
Utterance
During a conversation, the bot will ask the customer to provide some information. Each input that the customer provides (voice or chat response) is an utterance. For example, it could be a request ("I want to book a hotel room") or a simple reply to a question ("Yes").
Intent
During its conversation with a customer, the bot attempts to identify the intent, or what it is that the customer wants to do. For example, the bot might detect that the customer wants to buy a ticket. It will then proceed to fill the required slots (or "entities") that are associated with that intent, such as the name of the show the customer wants to buy a ticket for, the date and time they want to attend, and so on, by asking the customer to provide those details.Intent fulfillment
After the bot has successfully gathered all of the information it needs to satisfy an intent, Designer consider the intent to be fulfilled.
Reports on this dashboard
This dashboard contains the following reports:
Count of Bot Sessions Over Time
This panel displays the total number of Bot Sessions that took place over a specified period of time for both Voice and Chat channels.
You can expand the View menu to select a different visualization option, or use the Interval setting to adjust the range of time shown, such as for the last hour, 12 hours, or the previous week.
Filter
You can use the Filter panel to select specific values you want to filter the results for, such as Application or Disposition.
The drop-downs on the Filter panel only list the top 100 values for each item. If the value you are looking for isn't in the list, you can toggle the Filtering tab to expose the filter queries that are currently being applied to the dashboard. You can then edit the filter query to change the value to the one you are looking for:
Any filters you select are applied across all of the dashboards, not just the one you are viewing. You can toggle the Filtering tab to see the filters that are currently being applied to the dashboard.
Count by Channel
This panel displays a breakdown of total counts and percentages for both Voice and Chat channel bot sessions.
Count by Result
This panel provides total counts and percentages for Bot Session results, broken down by status. For information about Success and Failed sessions, see Understanding the results.
Count by Bot Method
This panel displays a breakdown of all Bot Sessions by the method used by the bot service (default is Native).
Count by Provider
This panel displays a breakdown of all Bot Sessions by the bot services provider that was invoked (e.g. Dialogflow, Lex, Dialog Engine).
Invoked Bot Results
This panel lists the names of all bots that were invoked during the selected time period, along with the total counts and percentages for bot sessions where the attempt to invoke the bot was successful or failed. Clicking the arrow beside the bot name expands the row to show the breakdown of these totals by channel.
You can change the reporting view to show either the Bot Sessions by Channel or Channel Usage by Bot Sessions. If desired, you can export the results to a CSV file.
Average Time SS Duration
This panel displays the average duration of time (in seconds) that all application sessions spent in the Self Service phase, broken down by total counts of application sessions where bots were invoked (Bots Executed) and all other application sessions (No Bots Invoked).
You can expand the View menu to change the visualization options or select a specific Interval to display (such as 1 day). Hovering over the results displays the details for that item.
Top Intents
This panel lists the top intents for all bots that were invoked during the selected time period. An intent is detected by the bot during its conversation with the customer.
Clicking an intent expands the row to display the names of the bots that detected that intent. Other details provided in this panel include:
- AvgDuration(MS) — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received.
- SumStepcount — Total number of requests that were sent to the bot service.
- AbandonedInSS — Total number of bot sessions that ended with a final disposition of Abandoned in Self Service.
- RoutedToAgent — Total number of bot sessions that ended with a final disposition of Routed to Agent.
- RoutedToDN — Total number of bot sessions that ended with a final disposition of Routed to DN.
- AbandonedInQ — Total number of bot sessions that ended with a final disposition of Abandoned in Queue.
If desired, you can also choose to export the results to a CSV file.
Selected Intents Details
This panel lists details for the top selected intents, broken down by channel type (voice or chat). If desired, you can export the results to a CSV file.
Clicking an intent expands the row to display the totals by Channel type. Other details provided in this panel include:
- AvgDuration(MS) — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received.
- #Intents — Total number of selected intents.
- #Sessions — Total number of bot sessions.
- #Intents/Session — Average total of selected intents per bot session.
Top Bots /W No Matched Intent
This panel provides information about sessions where a bot was invoked, but no intent was detected.
Details provided in this panel include:
- AvgDuration(MS) — Average duration of time (in milliseconds) that the bot sessions were active, based on the period of time between the first request sent to the invoked bot service and the last response that was received.
- SumStepcount — Total number of requests that were sent to the bot service.
- AbandonedInSS — Total number of bot sessions that ended with a final disposition of Abandoned in Self Service.
- RoutedToAgent — Total number of bot sessions that ended with a final disposition of Routed to Agent.
- RoutedToDN — Total number of bot sessions that ended with a final disposition of Routed to DN.
- AbandonedInQ — Total number of bot sessions that ended with a final disposition of Abandoned in Queue.
If desired, you can export the results to a CSV file.
Number of Bot Sessions No Matched Intent
This panel expands on the results provided in the above report. It displays a total of all bot sessions broken down by Failed Bots and bot sessions that ended with Intent Not Fulfilled.
For a Failed Bot, the bot session experienced an error that prevented the bot service from functioning properly.
For Intent Not Fulfilled, the bot service was not able to match the customer's input to an intent. Everything worked, but the bot was not able to determine what the customer wanted to do.
You can expand the View menu to change the visualization options or select a specific Interval to display (such as 1 day). Hovering over the results displays the details for that item.