Observability in Telemetry Service
Contents
Learn about the logs, metrics, and alerts you should monitor for Telemetry Service.
Monitoring
Private edition services expose metrics that can be scraped by Prometheus, to support monitoring operations and alerting.
- As described on Monitoring overview and approach, you can use a tool like Grafana to create dashboards that query the Prometheus metrics to visualize operational status.
- As described on Customizing Alertmanager configuration, you can configure Alertmanager to send notifications to notification providers such as PagerDuty, to notify you when an alert is triggered because a metric has exceeded a defined threshold.
The services expose a number of Genesys-defined and third-party metrics. The metrics that are defined in third-party software used by private edition services are available for you to use as long as the third-party provider still supports them. For descriptions of available Telemetry Service metrics, see:
See also System metrics.
Enable monitoring
Telemetry Service does not expose any specific metrics for monitoring. You can use standard Kubernetes metrics, as delivered by cAdvisor, of the kind that apply to any pod of the same nature.
| Service | CRD or annotations? | Port | Endpoint/Selector | Metrics update interval |
|---|---|---|---|---|
| n/a | /metrics |
Configure metrics
No additional service-level configuration is required to enable monitoring for Telemetry Service.
Alerting
Private edition services define a number of alerts based on Prometheus metrics thresholds.
For descriptions of available Telemetry Service alerts, see:
Configure alerts
Private edition services define a number of alerts by default (for Telemetry Service, see the pages linked to above). No further configuration is required.
The alerts are defined as PrometheusRule objects in a prometheus-rule.yaml file in the Helm charts. As described above, Telemetry Service does not support customizing the alerts or defining additional PrometheusRule objects to create alerts based on the service-provided metrics.
Logging
Telemetry Service sends logs to stdout.
Telemetry Service logs are structured so that log documents can be split into two distinct indexes: one for the Core Telemetry activity and the other for the Telemetry client logs.
The traceId attribute of the Telemetry log Contract is available to all Telemetry clients by default. This allows logs sent through Telemetry to meet the pre-condition for distributed tracing Observability goal.