Metrics

Learn which metrics you should monitor for <service_name> and when to sound the alarm.

Include a link to the "suite-level" documentation for monitoring:

Describe metrics (compatible with Prometheus endpoints) that customers can use to create their own monitoring dashboard in a tool like Grafana.

Make to identify any metrics that are important to monitor for alarming purposes, and include sample thresholds the customer should alarm on.

UCS-X provides internal monitoring metrics through a Prometheus endpoint on port 10052.

Common Performance Metrics

Name (type)	Key	Units	Additional keys	Alarm condition	Description	Common description
ucsx_performance (Gauge)	metric="cpuUsage"	%	'nodeId', 'pid'	> 95	CPU usage	basic metric, created with starting metrics service, cannot be disabled
	metric="loopDelay"	ms			event loop lag
ucsx_internal_queue (Gauge)	metric="queueSize"		'nodeId', 'pid', 'endpoint'		number requests waiting for processing
	metric="queueDelay"	ms		> 10000	request's waiting for processing time
ucsx_timings (Gauge)	metric="cpuTime"	s	'nodeId', 'pid'		CPU time used by process	basic metric, created with starting metrics service, cannot be disabled
	metric="sysTime"				Syetm mode time
	metric="userTime"				User mode time
	metric="upTime"				Process running time
ucsx_memory (Gauge)	metric="rss"	Mb	'nodeId', 'pid',	> 1024	Resident Set Size	basic metric, created with starting metrics service, cannot be disabled
	metric="heapTotal"
	metric="heapUsed"

Database Connection Metrics

Name (type)	Key	Units	Additional keys	Alarm condition	Description	Common description
ucsx_dbPool (Gauge)	metric="total"		'nodeId', 'pid', 'ccId', 'address'		Total count of connection to particular database (defined by key 'ccId')	basic metric, created with starting call-center-storage service, cannot be disabled, shows usage of connection pool
	metric="idle"				Total count of idle connection to particular database (defined by key 'ccId')
	metric="wait"				Total count of connection in 'waiting' state to particular database (defined by key 'ccId')
	metric="max"				Max available connection for this instance of database

Internal Functions Calls Monitoring

Name (type)	Key	Units	Additional keys	Description	Common description
ucsx_cache (Counter)	method="set"		'nodeId', 'pid', 'key'	Count writes to cache	Metrics of internal application cache
	method="get"			Count successful reads from cache
	method="delete"			Count delete operations from cache
	method="expired"			Count unsuccessful reads from cache due to data expiration
	method="remove"			Count remove operation by timer
	method="miss"			Count unsuccessful reads from cache due to data absence

SQL Monitoring

Name (type)	Key	Units	Additional keys	Description	Common description
ucsx_sql_bucket (Histogram)	le="10"	ms	'nodeId', 'pid', 'operation', 'ccId', 'ccName'	Count SQL request's of duration less 10ms	Metrics of raw SQL request duration. Key 'operation' might have values: 'update','insert','delete','query','rawSql'
	le="50"			Count SQL request's of duration less 50ms
	le="200"			Count SQL request's of duration less 200ms
	le="1000"			Count SQL request's of duration less 1000ms
	le="+Inf"			Count all SQL request's
ucsx_sql_sum (Histogram)				Total duration of all raw SQL requests
ucsx_sql_count (Histogram)				Total number of all raw SQL requests

Internal Cache Monitoring

Name (type)	Key	Units	Additional keys	Description	Common description
ucsx_cache (Counter)	method="set"		'nodeId', 'pid', 'key'	Count writes to cache	Metrics of internal application cache
	method="get"			Count successful reads from cache
	method="delete"			Count delete operations from cache
	method="expired"			Count unsuccessful reads from cache due to data expiration
	method="remove"			Count remove operation by timer
	method="miss"			Count unsuccessful reads from cache due to data absence

Elasticsearch Monitoring

Name (type)	Key	Units	Additional keys	Alarm condition	Description	Common description
ucsx_elastic_search_bucket (Histogram)	le="10"	ms	'nodeId', 'pid', 'operation', 'ccId', 'ccName'		Count ES request's of duration less 10ms	Metrics of ElasticSearch request duration. Key 'operation' can have values: 'read','write'.
	le="50"				Count ES request's of duration less 50ms
	le="200"				Count ES request's of duration less 200ms
	le="1000"				Count ES request's of duration less 1000ms
	le="+Inf"				Count all ES request's
ucsx_elastic_search_sum (Histogram)					Total duration of all ElasticSearch requests
ucsx_elastic_search_count (Histogram)					Total number of all ElasticSearch requests

Session Metrics

Name (type)	Key	Units	Additional keys	Description	Common description
ucsx_sessions (Gauge)	state="active"	counter	'nodeId', 'pid', 'state', 'session'	Count of sessions that have had some activity within configured interval of time.	Available session types are 'HTTP' and 'cometD'
	state="idle"			Count of sessions that have not had some activity within configured interval of time but still alive.
	state="newPerMinute"			Count of new opened sessions per last minute