Monitor predictor feature coverage
Contents
Gain insight into your Machine Learning model performance factors to ensure you always get the best predictions.
The AI Monitoring tab
The AI Monitoring tab provides a view into whether you have adequate data to make good predictions.
This tab is on the Predictors window:
Prerequisites: To use this tab, you must have created a Customer Profile dataset, an Agent Profile dataset, one or more Predictors, and have scoring data available.
Note: This page does not display real-time updates. GPR refreshes the data displayed once per day, at 3:00 am UTC.Configure the view
The AI Monitoring tab enables you to select the following information:
- The name and ID of the predictor. Use the selector to specify which predictor to view.
- The name and ID of the model. Use the selector to specify which model to view.
- The displayed date range. Use the year and month selectors to specify the month to view.
How to read the dials
The AI Monitoring tab presents six dials, each with a range of 0-1, which indicates the percent coverage of the specified performance factor. The number below the dial corresponds with the point indicated by the needle on the dial. That is, a dial setting of 0.59, as shown in the graphic, equals 59%.
The dials are colored red, yellow, and green to provide a quick visual indicator of whether the feature coverage for the associated parameter is satisfactory. Green indicates that you have good coverage, yellow means that you should review your coverage quality, and red indicates that coverage is not adequate and you should take action to improve predictor performance.
Metrics definitions
Feature coverage scores
Three of the dials show the percentage of features available when the selected model was trained that are also in the actual data used for scoring. One shows agent features, one customer features, and one shows the result for all features taken together.
The values range from 0 to 1, with higher being better. If all features in the training dataset used to create the model are also in the data used for scoring, the value is 1. A lower value shows that some data you expect might no longer be available.
If, over time, feature coverage drops but weighted feature coverage holds steady, the data in the missing features might be of low value in any case, since the weighted coverage value gives emphasis to the features with the highest impact on the KPI configured in the predictor.
Weighted feature coverage scores
The remaining three dials also show the percentage of features available when the selected model was trained that are also in the actual data used for scoring, but with that figure adjusted to weigh the most significant features higher.
- To view the feature weighting, see the bar graph showing the top features on the Predictors Simulator tab.
As a result of the weighting, if coverage is good for very significant features but poor for insignificant ones, the weighted result is still good. However, even if feature coverage is good for most features, if it is poor for one or more significant features, the weighted result is poor.
In these dials also, the values range from 0 to 1, with higher being better.
If weighted feature coverage drops but feature coverage is steady, the small amount of missing data must disproportionately affect key features.
View feature coverage trends
The feature coverage dials show data for the previous 15 minutes. To view feature coverage trends over a longer time, do the following steps:
- Click any of the six dials. A Trends window opens, with a graph showing all data points for the metric you clicked during the month you selected on the main window.
- To show a shorter period of time within the month, click the blue selector line below the main chart. This activates a slider function, enabling you to narrow the time period shown on the upper Trend graph.
- Hover your mouse over the Trend line to open a tooltip showing the exact date and feature coverage value for that point on the line.