Karl.Abraham: Published

2022-01-24T06:52:35Z

Published

New page

{{ArticlePEServiceMetrics
|IncludedServiceId=62ad5ef3-8215-4258-a2d6-cf010ffc92b3
|CRD=ServiceMonitor / PodMonitor
|Port=9116
|Endpoint=Metrics endpoint:
<source>
curl -v "http://<RS_POD_IP>:9116/snmp?target=127.0.0.1%3A1161&module=if_mib"
</source>

Enabling metrics:

Service/Pod Monitoring Settings

<source>
prometheus:
enabled: true
metric:
   port: 9116
</source>

Enable for Prometheus operator

<source>
podMonitor:
enabled: true
metric:
   path: /snmp
   module: [ if_mib ]
   target: [ 127.0.0.1:1161 ]
</source>

monitoring:

*prometheusRulesEnabled: true

* grafanaEnabled: true

monitor:

* monitorName: gvp-monitoring
|MetricsDefined=Yes
|PEMetric={{PEMetric
|Metric=rsQueueName
|Type=Gauge
|Unit=DisplayString
|MetricDescription=The name of the message queue

(rsQueueName{gvpConfigDBID="172",rsQueueIndex="1",rsQueueName="rs.queue.remote_cdr.rm"}
|SampleValue=1
|UsedFor=Information
}}{{PEMetric
|Metric=rsQueueSize
|Type=Gauge
|Unit=Unsigned32
|MetricDescription=Used to get the size of the message queue.

(rsQueueSize{gvpConfigDBID="172",rsQueueIndex="1"} )
|SampleValue=1
|UsedFor=Traffic
}}{{PEMetric
|Metric=rsDequeueCount
|Type=Counter
|Unit=Counter64
|MetricDescription=Used to get dequeue value of the message queue.

(rsDequeueCount{gvpConfigDBID="172",rsQueueIndex="1"} )
|SampleValue=0
|UsedFor=Traffic
}}{{PEMetric
|Metric=rsEnqueueCount
|Type=Counter
|Unit=Counter64
|MetricDescription=Used to get enqueue value of the message queue.

(rsEnqueueCount{gvpConfigDBID="172",rsQueueIndex="1"})
|SampleValue=4
|UsedFor=Traffic
}}{{PEMetric
|Metric=rsUptime
|Type=Gauge
|Unit=Unsigned32
|MetricDescription=The time (in hundredths of a second) since the RS was started.
|SampleValue=30619972
|UsedFor=Information
}}
|AlertsDefined=Yes
|PEAlert={{PEAlert
|Alert=PodStatusNotReady
|Severity=CRITICAL
|AlertDescription=The trigger will flag an alarm when RS pod status is Not ready for 30 mins and this will be controlled through override-value.yaml file.
|BasedOn=kube_pod_status_ready
|Threshold=30mins
}}{{PEAlert
|Alert=ContainerRestartedRepeatedly
|Severity=CRITICAL
|AlertDescription=The trigger will flag an alarm when the RS or RS SNMP container gets restarted 5 or more times within 15 mins
|BasedOn=kube_pod_container_status_restarts_total
|Threshold=15mins
}}{{PEAlert
|Alert=InitContainerFailingRepeatedly
|Severity=CRITICAL
|AlertDescription=The trigger will flag an alarm when the RS init container gets failed 5 or more times within 15 mins
|BasedOn=kube_pod_init_container_status_restarts_total
|Threshold=15mins
}}{{PEAlert
|Alert=ContainerCPUreached80percent
|Severity=HIGH
|AlertDescription=The trigger will flag an alarm when the RS container CPU utilization goes beyond 80% for 15 mins
|BasedOn=container_cpu_usage_seconds_total, container_spec_cpu_quota, container_spec_cpu_period
|Threshold=15mins
}}{{PEAlert
|Alert=ContainerMemoryUsage80percent
|Severity=HIGH
|AlertDescription=The trigger will flag an alarm when the RS container Memory utilization goes beyond 80% for 15 mins
|BasedOn=container_memory_usage_bytes, kube_pod_container_resource_limits_memory_bytes
|Threshold=15mins
}}{{PEAlert
|Alert=RSQueueSizeCritical
|Severity=HIGH
|AlertDescription=The trigger will flag an alarm when RS JMS message queue size goes beyond 15000 (3GB approx. backlog) for 15 mins
|BasedOn=rsQueueSize
|Threshold=15mins
}}{{PEAlert
|Alert=PVC50PercentFilled
|Severity=HIGH
|AlertDescription=This trigger will flag an alarm when the RS PVC size is 50% filled

<br />
|BasedOn=kubelet_volume_stats_used_bytes, kubelet_volume_stats_capacity_bytes
|Threshold=15mins
}}{{PEAlert
|Alert=PVC80PercentFilled
|Severity=CRITICAL
|AlertDescription=This trigger will flag an alarm when the RS PVC size is 80% filled

<br />
|BasedOn=kubelet_volume_stats_used_bytes, kubelet_volume_stats_capacity_bytes
|Threshold=5mins
}}
}}

GVP/Current/GVPPEGuide/Reporting Server Metrics - Revision history

Karl.Abraham: Published