Metrics (VCDx v2.0.0)

No sensitive data or PII is sent to the TrustX metrics server. The following data is sent to TrustX for the purpose of metrics and reporting.

VGSx Metrics

The xDeTECH Gateway service publishes metrics via a standard Prometheus endpoint with a path of /actuator/prometheus. This document describes the name and usage of the published metrics.

Two levels of metrics are published:

1. System metrics: These metrics relate to system data which is not associated with a particular tenant such as data relating to audio calls being received from the SIPREC Server.
2. Tenant-specific metrics: These metrics relate to tenant-specific calls to analyze call audio.

Two types of metrics are published:

• Counters: These are metrics that only increase over time. They are used to track cumulative data that can only increase over time (or be reset to zero).
• Gauges: These are metrics that can increase or decrease. They represent state data that can go up and down over time.

The subsequent sections details system metrics and then tenant-specific metrics, and within those categories describe metrics related to different aspects of the xDeTECH Gateway.

System Metrics

Call Metrics

These metrics relate to calls received via the VGSx /orktrack/command API which is used to announce calls starting and ending.

Counters

Name	Description
call.announcements.start	The number of calls started.
call.announcements.stop	The number of calls stopped.
calls.withoutCallId	The number of calls that could not be processed because the call parameters did not contain either of the callid or nativecallid parameter.
calls.withoutCallState	The number of calls that could not be processed because the call parameters did not contain the stage parameter.

Guages

By default, these call metrics are updated by VGSx every 5 seconds starting two hours after VGSx is started. You can reconfigure these properties to change when the task which gathers these metrics runs and how often. Refer to the auto$ for details on how to update properties. The relevant properties are gather.call.metrics.task.delay and gather.call.metrics.task.period.

For reference, the various states a call can be in are:

• PENDING: the call has been received but no processing has been requested on the call.
• PROCESSING: the call is currently being analyzed.
• STOPPED: processing on the call has completed but the call has not ended. The call can re-enter a processing state if a new analysis request is received.
• ENDING: the call with the user has ended but VGSx still has data which can be sent to the VCDx for analysis.
• ENDED: the call has ended and all the data has been analyzed.
• ERROR: an error has occurred while attempting to process the call.

Name	Description
calls.pending	The number of calls known to VGSx which are currently in a PENDING state.
call.processing	The number of calls known to VGSx which are currently in a PROCESSING state.
calls.ending	The number of calls known to VGSx which are currently in an ENDING state.
calls.ended	The number of calls known to VGSx which are currently in an ENDED state.
calls.stopped	The number of calls known to VGSx which are currently in an STOPPED state.
calls.error	The number of calls known to VGSx which are currently in an ERROR state.
calls.current	The number of calls known to VGSx which are in progress and have not ended.
calls.active	The number of calls known to VGSx which are currently undergoing analysis using xDeTECH.

Health Check Metrics

When the VGSx ping API is called it makes a health call to xDeTECH to check that it the service is alive.

Name	Description
vgsx.calls.health	The number of successful calls made to check that the xDeTECH service is live.
vgsx.calls.health.failed	The number of failed calls made to check that the xDeTECH service is live.

Voice Activity Detector Metrics

Name	Description
vgsx.analysis.vad .invocations	The number of times the Voice Activity Detector has been invoked over all calls processed by VGSx.

Tenant-Specific Metrics

Audio Stream Metrics

Counters

Name	Description
audiostreams.requested	The number of audio streaming connections that have been made.
audiostreams.completed	The number of audio streaming connections that have completed.
audiostreams.errors	The number of audio streaming connections that have failed.
audiobytes.received	The number of bytes received via audio streaming connections.
rawsamples.created	The number of raw audio samples to be processed that have been created from streamed audio data.

Guages

Name	Description
audiostreams.active	The number of audio streaming connections being analyzed.

Speech Segment Metrics

Counters

Name	Description
analysis.speechSegments .received	The number of speech segments that have been prepared for dispatch.
analysis.speechSegments .abandoned	The number of speech segments that have been prepared but not sent for analysis.

Guages

Name	Description
analysis.speechSegments .pending	The number of speech segments that have been prepared and are awaiting analysis.

xDeTECH Analysis Service Metrics

Counters

Name	Description
calls.analysis	The number of successful calls made to the xDeTECH service to perform analysis.
calls.analysis.failed	The number of failed calls made to the xDeTECH service to perform analysis.
calls.analysis.evaluation .failed	The number of calls made to the xDeTECH service to perform analysis which were not processed, that is resulted in a NOT_PROCESSED result.

Voice Activity Detector Metrics

Counters

Name	Description
speechsamples.created	The number of processed speech samples created.
audiosamples.carryover .created	The number of audio samples created from remaining unprocessed audio at the end of an audio sample.

VCDx Metrics

The xDeTECH servers on TrustX will expose a Prometheus endpoint and expose the following metrics:

The number of requests received (requests_received) with the following tags:

• tenant
• serviceName
• result
• streamResult
• version

This will allow a customer, or Daon, to report on the number of audio chunks processed and to report on:

• The number that resulted in an error (error not null), per tenant and per service
• The number that could not be processed (result = NOT_PROCESSED) , per tenant and per service
• The number that reported no anomalies (result = NO_ANOMALY_DETECTED) , per tenant and per service
• The number that reported anomalies (result = ANOMALY_DETECTED) , per tenant and per service

The number of requests received which resulted in an error (requests_received_error) with the following tags:

• tenant
• serviceName
• version

This will allow a customer, or Daon, to report on the number of the number that resulted in an error (error not null), per tenant and per service

The number of audio bytes received (num_bytes_received) with the following tags:

• tenant
• serviceName
• version

The length of audio received, in seconds, (num_audio_seconds_received) with the following tags:

• tenant
• serviceName
• version

The number of seconds of audio processed (num_audio_seconds_processed) with the following tags:

• tenant
• serviceName
• version

The number of unique audio streams (streams_received_by_state) with the following tags:

• tenant
• serviceName
• state
• result
• executionMode (e.g. PARALLEL_QC_FIRST)
• averagingMode (e.g. LAST_N_SAMPLES)
• windowSize
• qcTemplateUpdateRatio
• inferenceDuration
• version

Average processing time for quality check (processing_time_qc) with following tags:

• tenant
• serviceName
• version

Average processing time for replay check (processing_time_replay) with following tags:

• tenant
• serviceName
• version

Average processing time for clone check (processing_time_clone) with following tags:

• tenant
• serviceName
• version

Average total processing time for request (processing_time_total) with following tags:

• tenant
• serviceName
• version

Replay R1 scores (score_replay_r1) with following tags:

• tenant
• serviceName
• threshold
• belowThreshold
• bucket
• order
• streamResult
• version

Replay R2 scores (score_replay_r2) with following tags:

• tenant
• serviceName
• threshold
• belowThreshold
• bucket
• order
• streamResult
• version

Clone V1 scores (score_clone_v1) with following tags:

• tenant
• serviceName
• threshold
• belowThreshold
• bucket
• order
• streamResult
• version

Clone V2 scores (score_clone_v2) with following tags:

• tenant
• serviceName
• threshold
• belowThreshold
• bucket
• order
• streamResult
• version

Clone V3 scores (score_clone_v3) with following tags:

• tenant
• serviceName
• threshold
• belowThreshold
• bucket
• order
• streamResult
• version

Clone V4 scores (score_clone_v4) with following tags:

• tenant
• serviceName
• threshold
• belowThreshold
• bucket
• order
• streamResult
• version

Spoof count (count_spoof) with following tags:

• tenant
• serviceName
• version

Replay count (count_replay) with following tags:

• tenant
• serviceName
• version

Failed QC count (count_qc_failed) with following tags:

• tenant
• serviceName
• errorMessage
• version

Average QC speech duration (speech_duration_qc) with following tags:

• tenant
• serviceName
• version

Average clone speech duration (speech_duration_clone) with following tags:

• tenant
• serviceName
• version

Average replay speech duration (speech_duration_replay) with following tags:

• tenant
• serviceName
• version

Average number of chunks per stream (stream_count_chunks) with following tags:

• tenant
• serviceName
• result
• executionMode
• averagingMode
• windowSize
• qcTemplateUpdateRatio
• inferenceDuration
• version

Average audio duration per stream (stream_duration_audio) with following tags:

• tenant
• serviceName
• result
• executionMode
• averagingMode
• windowSize
• qcTemplateUpdateRatio
• inferenceDuration
• version

Average speech duration per stream (stream_duration_speech) with following tags:

• tenant
• serviceName
• result
• executionMode
• averagingMode
• windowSize
• qcTemplateUpdateRatio
• inferenceDuration
• version

Aggregated speech in QC template per chunk (qc_tpl_aggregated_speech) with following tags:

• tenane
• serviceName
• streamResult
• executionMode
• bucket
• order
• version

SNR QC scores (score_qc_snr) with following tags:

• tenant
• serviceName
• streamResult
• qcFailed
• threshold
• bucket
• order
• version

STOI QC scores (score_qc_stoi) with following tags:

• tenant
• serviceName
• streamResult
• qcFailed
• threshold
• bucket
• order
• version

PESQ QC scores (score_qc_pesq) with following tags:

• tenant
• serviceName
• streamResult
• qcFailed
• threshold
• bucket
• order
• version

SI-SDR QC scores (score_qc_si_sdr) with following tags:

• tenant
• serviceName
• streamResult
• qcFailed
• threshold
• bucket
• order
• version

Loudness 1 QC scores (score_qc_loudness_1) with following tags:

• tenant
• serviceName
• streamResult
• qcFailed
• threshold
• bucket
• order
• version

Loudness 2 QC scores (score_qc_loudness_2) with following tags:

• tenant
• serviceName
• streamResult
• qcFailed
• threshold
• bucket
• order
• version

Chunk speech duration from QC snapin (qc_chunk_duration_speech) with following tags:

• tenant
• serviceName
• belowThreshold
• bucket
• order
• version

Chunk audio duration from QC snapin (qc_chunk_duration_audio) with following tags:

• tenant
• serviceName
• bucket
• order
• version

QC codec count (qc_codec_count) with following tags:

• tenant
• serviceName
• codec
• codecDiscard
• qcFailed
• version

Audio streams are identified by the stream-id HTTP header. This is used to group related requests together and to allow customers to identify the unique number of streams that have been processed. In the case of the voice gateway being used, the voice gateway will specify the stream-id and ensure that the ID is consistent between requests for the same audio call.

xDeTECH will use Redis to track the audio calls as there will be many xDeTECH servers running in parallel. Using Redis as a single source of truth between the xDeTECH servers will allow us to quickly determine if a stream was seen before and its previous state.

Tracking the state changes within the xDeTECH servers could give rise to race conditions so it is suggested that these are managed within Redis. Redis is capable of loading and running Lua scripts which are short scripts that execute within Redis and can manipulate the database entries within Redis.