Monitoring Stack — Setup Guide

This guide walks you through deploying the KMS observability stack step by step. For a full reference of OTLP options and pipeline internals, see → KMS Telemetry

Table of contents

1. Download
2. Prerequisites
3. Local mode — KMS included
4. External mode — existing KMS
5. Explore Grafana
6. Troubleshooting

Download

The monitoring stack is distributed as a standalone archive containing the Docker Compose file, configuration templates, and helper scripts.

Download the archive from the KMS GitHub Releases page (cosmian-kms-monitoring.zip), or use wget with the version you are deploying:

wget https://package.cosmian.com/kms/<version>/monitoring/cosmian-kms-monitoring.zip
unzip cosmian-kms-monitoring.zip
cd monitoring/

Replace <version> with the KMS version you are deploying (e.g. 5.25.0). The list of available versions is at https://package.cosmian.com/kms/.

Alternatively, clone the KMS repository and navigate to the monitoring/ directory.

Prerequisites

Before getting started, make sure the following tools are installed on your machine.

Required

• Docker Engine ≥ 20.10 Compose profiles (COMPOSE_PROFILES) require Docker Compose v2 and Docker Engine 20.10+. Verify your version:

  docker --version
  docker compose version
  

• Docker Compose V2 (included in Docker Desktop 3.4+) The stack uses the docker compose command (with a space), not the legacy docker-compose.

Local mode only

• openssl — used by generate-demo-cert.sh to generate the demo TLS certificate. Verify it is available:

  openssl version
  

  On macOS it is pre-installed. On Linux: apt install openssl or dnf install openssl.

External mode only

• Your existing KMS must be configured to export telemetry via OTLP to the collector:

  # gRPC (recommended)
  KMS_OTLP_URL=http://<collector-host>:4317
  # or HTTP
  KMS_OTLP_URL=http://<collector-host>:4318
  

  See → KMS Telemetry — OTLP configuration

Ports availability

The following ports must be free on the host before starting the stack:

Local mode — KMS included

In this mode, the full stack starts together: KMS, OTel Collector, VictoriaMetrics, and Grafana. Use this mode for local development or to evaluate the stack end-to-end.

All files referenced below (docker-compose.yml, .env, generate-demo-cert.sh) are located in the monitoring/ directory of the KMS repository. Navigate there first:

cd monitoring/

1. Configure .env

Open the .env file — this is the only file you need to edit.

Make sure the following values are set:

# ── KMS Mode / Docker profiles ───────────────────────────────────
COMPOSE_PROFILES=kms-local
KMS_MODE=local

# ── TLS certificate password (used by generate-demo-cert.sh) ─────
KMS_P12_PASSWORD=testpassword

# ── KMS topology metadata ────────────────────────────────────────
KMS_CLUSTER=cosmian-kms-local

# ── Metadata for metrics/traces ──────────────────────────────────
KMS_VERSION=latest
ENVIRONMENT=demo

# ── Grafana ──────────────────────────────────────────────────────
GRAFANA_ADMIN_USER=demo
GRAFANA_ADMIN_PASSWORD=demo

# ── Metrics retention (VictoriaMetrics, in months) ───────────────
METRICS_RETENTION_MONTHS=12

✏️ Screenshot placeholder — .env file open in editor, COMPOSE_PROFILES and KMS_MODE highlighted.

2. Generate the demo TLS certificate

The KMS container requires a PKCS#12 certificate to serve its API over TLS. The provided script generates a self-signed certificate valid for 10 years, using the password defined in KMS_P12_PASSWORD:

bash generate-demo-cert.sh

Expected output:

Generating a 4096 bit RSA private key
.............................................++++
.............................................++++
.............................................++++
writing new private key to './certs/kms-demo.key'
-----
PKCS#12 demo certificate generated at: ./certs/kms.p12
Password: testpassword

⚠️ This certificate is for demo purposes only. Do not use it in production.

3. Start the stack

docker compose up -d

Docker will pull the required images and start all containers. On first run, this may take a minute depending on your network speed.

4. Verify the stack is healthy

docker compose ps

All containers should show Up or Up (healthy):

NAME                STATUS
kms                 Up (healthy)
otel-collector      Up
victoria-metrics    Up
grafana             Up (healthy)

You can also check the OTel Collector health endpoint:

curl http://localhost:13133

Expected response: {"status":"Server available"}

External mode — existing KMS

Use this mode when you already have a running KMS instance and only want to attach the observability stack to it.

All files referenced below are located in the monitoring/ directory of the KMS repository. Navigate there first:

cd monitoring/

1. Configure .env

Only a few lines change compared to local mode:

-COMPOSE_PROFILES=kms-local
-KMS_MODE=local
+COMPOSE_PROFILES=
+KMS_MODE=external

-# KMS_P12_PASSWORD is not needed in external mode
-KMS_P12_PASSWORD=testpassword

 KMS_CLUSTER=cosmian-kms-local
 KMS_VERSION=latest
-ENVIRONMENT=demo
+ENVIRONMENT=production

 GRAFANA_ADMIN_USER=demo
 GRAFANA_ADMIN_PASSWORD=demo
 METRICS_RETENTION_MONTHS=12

2. Point your KMS to the collector

On your existing KMS, enable OTLP export toward the collector host:

# via environment variable
KMS_OTLP_URL=http://<collector-host>:4317

# or via CLI argument
./cosmian_kms_server --otlp http://<collector-host>:4317 --enable-metering

If the KMS runs on the same machine as the stack, use http://localhost:4317.

3. Start the stack (without KMS)

docker compose up -d

With COMPOSE_PROFILES left empty, Docker Compose will start only OTel Collector, VictoriaMetrics, and Grafana — the KMS container is skipped.

4. Verify the stack is healthy

docker compose ps

Expected output (no kms container):

NAME                STATUS
otel-collector      Up
victoria-metrics    Up
grafana             Up (healthy)

Verify the collector is receiving data from your KMS:

# Check that otelcol_receiver_accepted_spans is incrementing
curl -s http://localhost:8888/metrics | grep otelcol_receiver_accepted_spans

Or

Explore Grafana

Once the stack is running, open Grafana in your browser:

http://localhost:3000

Login with:

Pre-provisioned dashboards are available under Dashboards → Browse:

After few seconds metrics will be available

Troubleshooting

Containers not starting

docker compose logs -f otel-collector
docker compose logs -f kms          # local mode only

No data in Grafana dashboards

1. Confirm the OTel Collector is up:

   curl http://localhost:13133
   

2. Confirm VictoriaMetrics is receiving data:

   curl http://localhost:8428/health
   

3. Check that your KMS is sending OTLP — look for incoming spans in collector logs:

   docker compose logs otel-collector | grep "traces"
   

KMS container fails to start (local mode)

Verify the certificate was generated:

ls -la .certs/kms.p12

If missing, re-run:

bash generate-demo-cert.sh

Port conflict

If a port is already in use, identify the process:

# macOS / Linux
lsof -i :<port>

# Windows
netstat -ano | findstr :<port>

Then either stop the conflicting process or edit the port mapping in docker-compose.yml.

For a complete reference of all OTLP options, log levels, and pipeline internals, see → KMS Telemetry