Metrics
No sensitive data or PII is sent to the TrustX metrics server. The following data is sent to TrustX for the purpose of metrics and reporting.
VGSx Metrics
The xDeTECH Gateway service publishes metrics via a standard Prometheus endpoint with a path of /actuator/prometheus. This document describes the name and usage of the published metrics.
Two levels of metrics are published:
- System metrics: These metrics relate to system data which is not associated with a particular tenant such as data relating to audio calls being received from the SIPREC Server.
- Tenant-specific metrics: These metrics relate to tenant-specific calls to analyze call audio.
Two types of metrics are published:
- Counters: These are metrics that only increase over time. They are used to track cumulative data that can only increase over time (or be reset to zero).
- Gauges: These are metrics that can increase or decrease. They represent state data that can go up and down over time.
The subsequent sections details system metrics and then tenant-specific metrics, and within those categories describe metrics related to different aspects of the xDeTECH Gateway.
System Metrics
Call Metrics
These metrics relate to calls received via the VGSx /orktrack/command API which is used to announce calls starting and ending.
Counters
| Name | Description |
|---|---|
| call.announcements.start | The number of calls started. |
| call.announcements.stop | The number of calls stopped. |
| calls.withoutCallId | The number of calls that could not be processed because the call parameters did not contain either of the callid or nativecallid parameter. |
| calls.withoutCallState | The number of calls that could not be processed because the call parameters did not contain the stage parameter. |
Guages
By default, these call metrics are updated by VGSx every 5 seconds starting two hours after VGSx is started. You can reconfigure these properties to change when the task which gathers these metrics runs and how often. Refer to the auto$ for details on how to update properties. The relevant properties are gather.call.metrics.task.delay and gather.call.metrics.task.period.
For reference, the various states a call can be in are:
- PENDING: the call has been received but no processing has been requested on the call.
- PROCESSING: the call is currently being analyzed.
- STOPPED: processing on the call has completed but the call has not ended. The call can re-enter a processing state if a new analysis request is received.
- ENDING: the call with the user has ended but VGSx still has data which can be sent to the VCDx for analysis.
- ENDED: the call has ended and all the data has been analyzed.
- ERROR: an error has occurred while attempting to process the call.
| Name | Description |
|---|---|
| calls.pending | The number of calls known to VGSx which are currently in a PENDING state. |
| call.processing | The number of calls known to VGSx which are currently in a PROCESSING state. |
| calls.ending | The number of calls known to VGSx which are currently in an ENDING state. |
| calls.ended | The number of calls known to VGSx which are currently in an ENDED state. |
| calls.stopped | The number of calls known to VGSx which are currently in an STOPPED state. |
| calls.error | The number of calls known to VGSx which are currently in an ERROR state. |
| calls.current | The number of calls known to VGSx which are in progress and have not ended. |
| calls.active | The number of calls known to VGSx which are currently undergoing analysis using xDeTECH. |
Health Check Metrics
When the VGSx ping API is called it makes a health call to xDeTECH to check that it the service is alive.
| Name | Description |
|---|---|
| vgsx.calls.health | The number of successful calls made to check that the xDeTECH service is live. |
| vgsx.calls.health.failed | The number of failed calls made to check that the xDeTECH service is live. |
Voice Activity Detector Metrics
| Name | Description |
|---|---|
vgsx.analysis.vad .invocations | The number of times the Voice Activity Detector has been invoked over all calls processed by VGSx. |
Tenant-Specific Metrics
Audio Stream Metrics
Counters
| Name | Description |
|---|---|
| audiostreams.requested | The number of audio streaming connections that have been made. |
| audiostreams.completed | The number of audio streaming connections that have completed. |
| audiostreams.errors | The number of audio streaming connections that have failed. |
| audiobytes.received | The number of bytes received via audio streaming connections. |
| rawsamples.created | The number of raw audio samples to be processed that have been created from streamed audio data. |
Guages
| Name | Description |
|---|---|
| audiostreams.active | The number of audio streaming connections being analyzed. |
Speech Segment Metrics
Counters
| Name | Description |
|---|---|
analysis.speechSegments .received | The number of speech segments that have been prepared for dispatch. |
analysis.speechSegments .abandoned | The number of speech segments that have been prepared but not sent for analysis. |
Guages
| Name | Description |
|---|---|
analysis.speechSegments .pending | The number of speech segments that have been prepared and are awaiting analysis. |
xDeTECH Analysis Service Metrics
Counters
| Name | Description |
|---|---|
| calls.analysis | The number of successful calls made to the xDeTECH service to perform analysis. |
| calls.analysis.failed | The number of failed calls made to the xDeTECH service to perform analysis. |
calls.analysis.evaluation .failed | The number of calls made to the xDeTECH service to perform analysis which were not processed, that is resulted in a NOT_PROCESSED result. |
Voice Activity Detector Metrics
Counters
| Name | Description |
|---|---|
| speechsamples.created | The number of processed speech samples created. |
audiosamples.carryover .created | The number of audio samples created from remaining unprocessed audio at the end of an audio sample. |
VCDx Metrics
The xDeTECH servers on TrustX will expose a Prometheus endpoint and expose the following metrics:
The number of requests received (requests_received) with the following tags:
- tenant
- serviceName
- result
This will allow a customer, or Daon, to report on the number of audio chunks processed and to report on:
- The number that resulted in an error (error not null), per tenant and per service
- The number that could not be processed (result = NOT_PROCESSED) , per tenant and per service
- The number that reported no anomalies (result = NO_ANOMALY_DETECTED) , per tenant and per service
- The number that reported anomalies (result = ANOMALY_DETECTED) , per tenant and per service
- Average scores for each of replay and clone detection models, per tenant and per service
The number of requests received which resulted in an error (requests_received_error) with the following tags:
- tenant
- serviceName
This will allow a customer, or Daon, to report on the number of the number that resulted in an error (error not null), per tenant and per service
The number of audio bytes received (num_bytes_received) with the following tags:
- tenant
- serviceName
The length of audio received, in seconds, (num_audio_seconds_received) with the following tags:
- tenant
- serviceName
The number of seconds of audio processed (num_audio_seconds_processed) with the following tags:
- tenant
- serviceName
The number of unique audio streams (streams_received) with the following tags:
- tenant
- serviceName
The number of unique audio streams (streams_received_by_state) with the following tags:
- tenant
- serviceName
- state
Audio streams are identified by the stream-id HTTP header. This is used to group related requests together and to allow customers to identify the unique number of streams that have been processed. In the case of the voice gateway being used, the voice gateway will specify the stream-id and ensure that the ID is consistent between requests for the same audio call.
xDeTECH will use Redis to track the audio calls as there will be many xDeTECH servers running in parallel. Using Redis as a single source of truth between the xDeTECH servers will allow us to quickly determine if a stream was seen before and its previous state.
Tracking the state changes within the xDeTECH servers could give rise to race conditions so it is suggested that these are managed within Redis. Redis is capable of loading and running Lua scripts which are short scripts that execute within Redis and can manipulate the database entries within Redis.