Monitoring Stack — Setup Guide

This guide walks you through deploying the KMS observability stack step by step. For a full reference of OTLP options and metrics, see → Metrics & Traces (OTLP)

Table of contents

1. Download
2. Prerequisites
3. Local mode — KMS included
4. External mode — existing KMS
5. Explore Grafana
6. Proven integrations
7. Troubleshooting

Architecture overview

sequenceDiagram
    participant KMS as KMS Server
    participant OTel as OTel Collector
    participant VM as VictoriaMetrics
    participant Grafana as Grafana

    KMS->>OTel: gRPC OTLP metrics (port 4317)
    Note over KMS,OTel: kms_* metric families pushed continuously
    OTel->>VM: remote_write (Prometheus protocol, every 10s)
    Grafana->>VM: PromQL queries (port 8428)
    VM-->>Grafana: metric series
    Note over KMS,OTel: KMS also exposes a Prometheus /metrics endpoint

The collector bridges KMS OTLP push to VictoriaMetrics remote-write. Grafana queries VictoriaMetrics directly — it never contacts the KMS server.

Download

The monitoring stack is distributed as a standalone archive containing the Docker Compose file, configuration templates, and helper scripts.

Download the archive from the KMS GitHub Releases page (cosmian-kms-monitoring.zip), or use wget with the version you are deploying:

wget https://package.cosmian.com/kms/<version>/monitoring/cosmian-kms-monitoring.zip
unzip cosmian-kms-monitoring.zip
cd monitoring/

Replace <version> with the KMS version you are deploying (e.g. 5.28.0). The list of available versions is at https://package.cosmian.com/kms/.

Alternatively, clone the KMS repository and navigate to the monitoring/ directory.

Prerequisites

Before getting started, make sure the following tools are installed on your machine.

Required

• Docker Engine ≥ 20.10 Compose profiles (COMPOSE_PROFILES) require Docker Compose v2 and Docker Engine 20.10+. Verify your version:

  docker --version
  docker compose version
  

• Docker Compose V2 (included in Docker Desktop 3.4+) The stack uses the docker compose command (with a space), not the legacy docker-compose.

Local mode only

• openssl — used by generate-demo-cert.sh to generate the demo TLS certificate. Verify it is available:

  openssl version
  

  On macOS it is pre-installed. On Linux: apt install openssl or dnf install openssl.

External mode only

• Your existing KMS must be configured to export telemetry via OTLP to the collector:

  # gRPC (recommended)
  KMS_OTLP_URL=http://<collector-host>:4317
  # or HTTP
  KMS_OTLP_URL=http://<collector-host>:4318
  

  See → Metrics & Traces (OTLP) — configuration

Ports availability

The following ports must be free on the host before starting the stack:

Local mode — KMS included

In this mode, the full stack starts together: KMS, OTel Collector, VictoriaMetrics, and Grafana. Use this mode for local development or to evaluate the stack end-to-end.

All files referenced below (docker-compose.yml, .env, generate-demo-cert.sh) are located in the monitoring/ directory of the KMS repository. Navigate there first:

cd monitoring/

1. Configure .env

Open the .env file — this is the only file you need to edit.

Make sure the following values are set:

# ── KMS Mode / Docker profiles ───────────────────────────────────
COMPOSE_PROFILES=kms-local
KMS_MODE=local

# ── TLS certificate password (used by generate-demo-cert.sh) ─────
KMS_P12_PASSWORD=testpassword

# ── KMS topology metadata ────────────────────────────────────────
KMS_CLUSTER=cosmian-kms-local

# ── Metadata for metrics/traces ──────────────────────────────────
KMS_VERSION=latest
ENVIRONMENT=demo

# ── Grafana ──────────────────────────────────────────────────────
GRAFANA_ADMIN_USER=demo
GRAFANA_ADMIN_PASSWORD=demo

# ── Metrics retention (VictoriaMetrics, in months) ───────────────
METRICS_RETENTION_MONTHS=12

✏️ Screenshot placeholder — .env file open in editor, COMPOSE_PROFILES and KMS_MODE highlighted.

2. Generate the demo TLS certificate

The KMS container requires a PKCS#12 certificate to serve its API over TLS. The provided script generates a self-signed certificate valid for 10 years, using the password defined in KMS_P12_PASSWORD:

bash generate-demo-cert.sh

Expected output:

Generating a 4096 bit RSA private key
.............................................++++
.............................................++++
.............................................++++
writing new private key to './certs/kms-demo.key'
-----
PKCS#12 demo certificate generated at: ./certs/kms.p12
Password: testpassword

⚠️ This certificate is for demo purposes only. Do not use it in production.

3. Start the stack

docker compose up -d

Docker will pull the required images and start all containers. On first run, this may take a minute depending on your network speed.

4. Verify the stack is healthy

docker compose ps

All containers should show Up or Up (healthy):

NAME                STATUS
kms                 Up (healthy)
otel-collector      Up
victoria-metrics    Up
grafana             Up (healthy)

You can also check the OTel Collector health endpoint:

curl http://localhost:13133

Expected response: {"status":"Server available"}

External mode — existing KMS

Use this mode when you already have a running KMS instance and only want to attach the observability stack to it.

All files referenced below are located in the monitoring/ directory of the KMS repository. Navigate there first:

cd monitoring/

1. Configure .env

Only a few lines change compared to local mode:

-COMPOSE_PROFILES=kms-local
-KMS_MODE=local
+COMPOSE_PROFILES=
+KMS_MODE=external

-# KMS_P12_PASSWORD is not needed in external mode
-KMS_P12_PASSWORD=testpassword

 KMS_CLUSTER=cosmian-kms-local
 KMS_VERSION=latest
-ENVIRONMENT=demo
+ENVIRONMENT=production

 GRAFANA_ADMIN_USER=demo
 GRAFANA_ADMIN_PASSWORD=demo
 METRICS_RETENTION_MONTHS=12

2. Point your KMS to the collector

On your existing KMS, enable OTLP export toward the collector host:

# via environment variable
KMS_OTLP_URL=http://<collector-host>:4317

# or via CLI argument
./cosmian_kms_server --otlp http://<collector-host>:4317 --enable-metering

If the KMS runs on the same machine as the stack, use http://localhost:4317.

3. Start the stack (without KMS)

docker compose up -d

With COMPOSE_PROFILES left empty, Docker Compose will start only OTel Collector, VictoriaMetrics, and Grafana — the KMS container is skipped.

4. Verify the stack is healthy

docker compose ps

Expected output (no kms container):

NAME                STATUS
otel-collector      Up
victoria-metrics    Up
grafana             Up (healthy)

Verify the collector is receiving data from your KMS:

# Check that otelcol_receiver_accepted_spans is incrementing
curl -s http://localhost:8888/metrics | grep otelcol_receiver_accepted_spans

Or

Explore Grafana

Once the stack is running, open Grafana in your browser:

http://localhost:3000

Login with:

Pre-provisioned dashboards are available under Dashboards → Browse:

After few seconds metrics will be available

Proven integrations

All tests use the product's official Docker image and fail if no evidence is found. These products consume KMS metrics, not audit events; see SIEM integration for audit event integrations.

Troubleshooting

Containers not starting

docker compose logs -f otel-collector
docker compose logs -f kms          # local mode only

No data in Grafana dashboards

1. Confirm the OTel Collector is up:

   curl http://localhost:13133
   

2. Confirm VictoriaMetrics is receiving data:

   curl http://localhost:8428/health
   

3. Check that your KMS is sending OTLP — look for incoming spans in collector logs:

   docker compose logs otel-collector | grep "traces"
   

KMS container fails to start (local mode)

Verify the certificate was generated:

ls -la .certs/kms.p12

If missing, re-run:

bash generate-demo-cert.sh

Port conflict

If a port is already in use, identify the process:

# macOS / Linux
lsof -i :<port>

# Windows
netstat -ano | findstr :<port>

Then either stop the conflicting process or edit the port mapping in docker-compose.yml.

For a complete reference of all OTLP options, log levels, and pipeline internals, see → Metrics & Traces (OTLP)