KMS Cache Mechanisms

The database layer uses three in-memory caches backed by moka::future::Cache: ObjectCache, UnwrappedCache, and RotateNameCache. This page documents those caches and inventories the other runtime cache mechanisms used by the server. Unless stated otherwise, cache state is process-local and is not shared between KMS replicas.

ObjectCache and UnwrappedCache serve object retrieval and key unwrapping. RotateNameCache serves keyset-generation resolution, especially for delegated HSM operations.

Runtime cache inventory

The detailed sections below describe the three database caches. The other runtime caches are described in the HSM, authentication/JWKS, certificate, and client cache hint sections.

Architecture overview

graph TD
    CALLER -->|retrieve_object| DB[Database]
    CALLER -->|get_unwrapped| GU[get_unwrapped]
    CALLER -->|find_by_rotate_name| RNC[RotateNameCache]

    DB -->|get| OC[ObjectCache]
    OC -->|miss| BS[(Backing Store<br/>SQLite / Postgres)]

    GU -->|peek| UC[UnwrappedCache]
    UC -->|miss| CRYPTO[unwrap_object<br/>KEK unwrap]

    RNC -->|miss| RESOLVE[Keyset resolution<br/>SQL / HSM scan]

    OC -->|stores| OC_VAL["wrapped ObjectWithMetadata<br/>+ fingerprint"]
    UC -->|stores| UC_VAL["unwrapped key material<br/>+ fingerprint of wrapped"]
    RNC -->|stores| RNC_VAL["matching UIDs + attributes"]

    style BS fill:#f9f,stroke:#333
    style CRYPTO fill:#f99,stroke:#333
    style RESOLVE fill:#f99,stroke:#333
    style OC fill:#9f9,stroke:#333
    style UC fill:#9f9,stroke:#333
    style RNC fill:#9f9,stroke:#333

RotateNameCache is an implementation cache; it has no operator-facing configuration. It is bounded to 10,000 entries and uses a 2-second TTL. The cache is populated by Database::find_by_rotate_name, which resolves a keyset name to its generations across the configured object stores. For HSM-backed objects, a miss can require a PKCS#11 C_FindObjects scan followed by C_GetAttributeValue calls; a hit avoids that scan on the delegated Sign/Verify/Encrypt/Decrypt hot path.

The cache key includes the requesting owner so one user's resolution cannot be reused for another user's objects. Explicit generation filters are also isolated from bare/latest lookups. After a rotation commits, the relevant bare/latest entry is invalidated immediately; the TTL remains a bounded staleness safeguard for paths that do not explicitly invalidate it. A rotation can therefore become visible immediately through the normal commit paths, while uncovered mutation paths are stale for at most the TTL.

ObjectCache

Source: crate/server_database/src/core/object_cache.rs

Caches full ObjectWithMetadata as read from the database to eliminate repeated DB round-trips on the hot path.

Public API

Callers

• Database::retrieve_object() — get() on hot path, insert() on miss
• Database::retrieve_object_arc() — get(), insert_arc() on miss
• Database::update_object() — invalidate() on write
• Database::validate_cache() — validate_cache() after forced re-fetch

Scenarios

S1 — Cold start (first access)

sequenceDiagram
    participant C as Caller
    participant DB as Database
    participant OC as ObjectCache
    participant BS as Backing Store

    C->>DB: retrieve_object("key-1")
    DB->>OC: get("key-1")
    OC-->>DB: None
    DB->>BS: retrieve("key-1")
    BS-->>DB: ObjectWithMetadata (wrapped)
    DB->>OC: insert("key-1", owm)
    DB-->>C: owm

S2 — Hot path (cache hit)

sequenceDiagram
    participant C as Caller
    participant DB as Database
    participant OC as ObjectCache

    C->>DB: retrieve_object("key-1")
    DB->>OC: get("key-1")
    OC-->>DB: cached entry found
    DB-->>C: owm via Arc unwrap_or_clone

S3 — retrieve_object_arc (zero-copy Arc path)

sequenceDiagram
    participant C as Caller
    participant DB as Database
    participant OC as ObjectCache
    participant BS as Backing Store

    C->>DB: retrieve_object_arc("key-1")
    DB->>OC: get("key-1")
    alt Cache hit
        OC-->>DB: cached Arc found
        DB-->>C: Arc clone (pointer bump)
    else Cache miss
        OC-->>DB: None
        DB->>BS: retrieve("key-1")
        BS-->>DB: owm
        DB->>OC: insert_arc("key-1", Arc.new(owm))
        DB-->>C: Arc
    end

S4 — Out-of-band mutation detected

sequenceDiagram
    participant DB as Database
    participant OC as ObjectCache

    Note over DB: Another process mutated "key-1"
    DB->>OC: validate_cache("key-1", current_db_object)
    OC->>OC: fingerprint(cached) != fingerprint(current)?
    OC->>OC: invalidate("key-1")
    Note over OC: Next get() triggers fresh DB fetch

S5 — Object update invalidates cache

sequenceDiagram
    participant C as Caller
    participant DB as Database
    participant OC as ObjectCache

    C->>DB: update_object("key-1", new_obj, new_attrs)
    DB->>OC: invalidate("key-1")
    DB->>DB: persist to backing store
    Note over OC: Entry removed, next get() is miss

S6 — Eviction (automatic)

moka eviction requires no explicit call:

RotateNameCache

Source: crate/server_database/src/core/rotate_name_cache.rs

Caches the result of Database::find_by_rotate_name, the shared keyset lookup used to resolve RotateName and generation selectors. This is distinct from ObjectCache: it caches a multi-object keyset query, not an individual object.

Why this cache exists

Every delegated Sign/Verify/Encrypt/Decrypt call that names a bare key UID (no @N generation suffix) must first answer "which generation is the latest?". For an HSM-backed store that answer is not free: it requires a full C_FindObjects scan of the slot followed by a C_GetAttributeValue round-trip for every object found, on every call — even for a key that has never been rotated.

Without the cache, that scan sits directly on the hot path and, because the lookup fans through the same Database::find_by_rotate_name chokepoint, concurrent operations all pay it simultaneously. RotateNameCache remembers the resolution for a short TTL (2 seconds) so repeat calls skip the scan entirely; correctness is preserved by eagerly invalidating the relevant entry the moment a local rotation commits (see below).

Behavior

On a miss, the existing multi-store lookup runs unchanged. On a hit, the database query or HSM slot scan is skipped. The owner is part of the key to preserve access isolation, and generation is part of the key to prevent an explicit historical lookup from sharing the bare/latest result.

Invalidation and consistency

Every local write through Database invalidates the affected entries eagerly:

• creating an object with a RotateName (including a re-key's new generation) clears every entry for that keyset name, for all owners and generation filters;
• updating, re-labelling (HSM CKA_LABEL), changing the state of (revoke, destroy) or deleting an object clears every entry that lists it as a member.

Empty results are not cached, so a newly created keyset is visible at once.

Writes performed by other KMS nodes sharing the same database are only seen once the 2-second TTL expires. Paths whose correctness depends on the current keyset state — re-key eligibility (enforce_keyset_latest) and rotation-time generation allocation (selecting which generation to increment on the next re-key) — therefore bypass the cache through Database::find_by_rotate_name_uncached. The delegated crypto hot path (Sign/Verify/Encrypt/Decrypt) does not bypass the cache.

RotateNameCache is internal and is not configurable through the server configuration file or command-line options.

UnwrappedCache

Source: crate/server_database/src/core/unwrapped_cache.rs

Caches unwrapped key material to avoid repeated KEK-unwrap cryptographic operations. The fingerprint of the wrapped object is stored alongside the unwrapped payload so stale entries (after key re-wrapping or KEK rotation) are silently rejected.

Public API

Security guard: insert returns Err if wrapped == unwrapped (defense-in-depth against plaintext-persistence bugs).

Callers

• KMS::get_unwrapped() — peek() + insert() on miss
• KMIP ReKey / ReKeyKeyPair — clear_cache() after rotation
• Database::validate_cache() chain — validate_cache() for defense-in-depth

Scenarios

U1 — First unwrap (cold)

sequenceDiagram
    participant G as get_unwrapped()
    participant UC as UnwrappedCache
    participant CR as Crypto

    G->>UC: peek("key-1", wrapped_obj)
    UC-->>G: None (no entry)
    G->>CR: unwrap_object(wrapped_obj)
    Note over CR: KEK unwrap
    CR-->>G: unwrapped Object
    G->>UC: insert("key-1", wrapped_obj, unwrapped_obj)
    Note over UC: Stores fingerprint(wrapped) + unwrapped
    G-->>G: return unwrapped

U2 — Subsequent unwrap (hot, zero crypto)

sequenceDiagram
    participant G as get_unwrapped()
    participant UC as UnwrappedCache

    G->>UC: peek("key-1", wrapped_obj)
    UC->>UC: fingerprint matches?
    UC-->>G: Some(unwrapped_obj)
    Note over G: No cryptographic operation
    G-->>G: return unwrapped

U3 — Key re-wrapped (KEK rotation)

sequenceDiagram
    participant G as get_unwrapped()
    participant UC as UnwrappedCache
    participant CR as Crypto

    Note over G: wrapped_obj changed (new KEK after rotation)
    G->>UC: peek("key-1", new_wrapped_obj)
    UC->>UC: fingerprint mismatch
    UC-->>G: None (stale entry rejected)
    G->>CR: unwrap_object(new_wrapped_obj)
    CR-->>G: unwrapped Object
    G->>UC: insert("key-1", new_wrapped_obj, unwrapped)
    Note over UC: New fingerprint replaces old entry
    G-->>G: return unwrapped

U4 — Security guard (wrapped == unwrapped)

sequenceDiagram
    participant G as get_unwrapped()
    participant UC as UnwrappedCache

    Note over G: Bug: caller passes already-unwrapped object
    G->>UC: insert("key-1", obj, obj)
    UC->>UC: wrapped == unwrapped?
    UC-->>G: Err("wrapped and unwrapped objects should be different")
    Note over UC: Prevents plaintext-persistence bugs

U5 — Explicit invalidation on key rotation

sequenceDiagram
    participant R as ReKey / Rekeyer
    participant UC as UnwrappedCache

    R->>UC: clear_cache("key-1")
    Note over UC: Entry removed
    Note over UC: Next get_unwrapped() triggers crypto unwrap

U6 — Concurrent workers (double-unwrap acceptable)

sequenceDiagram
    participant W1 as Worker 1
    participant W2 as Worker 2
    participant UC as UnwrappedCache
    participant CR as Crypto

    W1->>UC: peek("key-1", wrapped)
    UC-->>W1: None (cold)
    W2->>UC: peek("key-1", wrapped)
    UC-->>W2: None (W1 has not inserted yet)

    W1->>CR: unwrap_object(...)
    CR-->>W1: unwrapped
    W1->>UC: insert("key-1", wrapped, unwrapped)

    Note over W2: Received None, must also unwrap
    W2->>CR: unwrap_object(...)
    CR-->>W2: unwrapped
    W2->>UC: insert("key-1", wrapped, unwrapped)
    Note over UC: Double unwrap on cold start is acceptable

End-to-end: JOSE decrypt

Full trace of POST /v1/crypto/decrypt with alg: RSA-OAEP. The RSA wrapping key (persistent DB object, identified by kid) passes through both caches. The RSA-OAEP CEK is cached as a separate UnwrappedCache entry, keyed by a SHA-256 digest of the JWE encrypted_key field.

sequenceDiagram
    participant J as JOSE /decrypt
    participant RF as retrieve_object_for_operation
    participant DB as Database
    participant OC as ObjectCache
    participant GU as get_unwrapped()
    participant UC as UnwrappedCache
    participant CR as Crypto
    participant OS as OpenSSL

    J->>RF: retrieve_object_for_operation(kid, Decrypt)
    RF->>DB: retrieve_objects(kid)

    alt ObjectCache hit
        DB->>OC: get(kid)
        OC-->>DB: cached (wrapped)
    else ObjectCache miss
        DB->>OC: get(kid)
        OC-->>DB: None
        DB->>DB: fetch from SQLite or Postgres
        DB->>OC: insert(kid, owm)
    end

    DB-->>RF: owm (wrapped)

    RF->>GU: get_unwrapped(kid, wrapped_obj, user)

    alt UnwrappedCache hit
        GU->>UC: peek(kid, wrapped_obj)
        UC-->>GU: Some(unwrapped_obj)
        Note over GU: No crypto
    else UnwrappedCache miss
        GU->>UC: peek(kid, wrapped_obj)
        UC-->>GU: None
        GU->>CR: unwrap_object(wrapped_obj)
        Note over CR: KEK unwrap
        CR-->>GU: unwrapped Object
        GU->>UC: insert(kid, wrapped_obj, unwrapped_obj)
    end

    GU-->>RF: unwrapped Object
    RF->>RF: owm.set_object(unwrapped)
    RF-->>J: owm with plaintext key

    J->>OS: kmip_private_key_to_openssl(owm.object())
    OS-->>J: OpenSSL PKey Private

    J->>OS: RSA private decrypt(encrypted_key)
    OS-->>J: CEK (Content Encryption Key)

    J->>OS: AES-GCM decrypt(ciphertext, CEK)
    OS-->>J: plaintext

    J->>UC: insert_cek(sha256(encrypted_key), cek)
    Note over J,UC: CEK cached in UnwrappedCache for subsequent decrypt calls

Per-scenario cache behavior

JOSE CEK Cache (RSA-OAEP path)

The /v1/crypto/encrypt and /v1/crypto/decrypt endpoints support a RSA-OAEP / RSA-OAEP-256 key-management algorithm (RFC 7516). Each decryption requires an RSA private-key operation to unwrap the ephemeral Content Encryption Key (CEK). When the same JWE token is decrypted repeatedly, this operation is repeated on every call.

The CEK is stored in UnwrappedCache after the first successful unwrap so subsequent requests for the same token skip the RSA operation entirely.

Cache key

"jose_cek_{SHA-256-hex(encrypted_key_bytes)}"

The JWE encrypted_key field (the RSA-OAEP ciphertext of the CEK) uniquely identifies a token because RSA-OAEP encryption is randomised. SHA-256 keeps the map key compact: 64 hex chars regardless of RSA key size.

Fingerprint source

The RSA private-key KMIP Object is passed as the fingerprint source at insert() and peek() time. If the key is re-imported or re-wrapped (its serialised form changes), the fingerprint stored in the cache entry no longer matches and peek() returns None, forcing a fresh RSA-OAEP unwrap and re-population.

/encrypt → /decrypt cross-path caching

When /encrypt is called, the freshly generated CEK is inserted into the cache immediately after wrapping, keyed by the encrypted_key returned in the JWE response. The first /decrypt call for that token will get a cache hit if both calls share the same server instance and the key has not changed.

If /encrypt uses a public-key KMIP object while /decrypt uses the linked private-key KMIP object, the fingerprints differ. In that case the first decrypt call replaces the stale entry with a correctly fingerprinted one; all subsequent decrypts hit the cache.

TTL and sizing

CEK cache entries share the same UnwrappedCache TTL and LRU capacity as wrapped KMS keys:

RFC 7516 §11.5 — implicit rejection preserved

Only genuinely unwrapped CEKs are inserted. The random substitute key generated when RSA-OAEP decryption fails (padding oracle countermeasure) is never cached. This preserves the constant-time rejection guarantee: an attacker submitting an invalid encrypted_key still causes AES-GCM to fail at tag verification, and no cache entry is written.

dir mode

The dir key-management algorithm uses a KMS-stored symmetric key directly. That path delegates to the KMIP Encrypt/Decrypt pipeline which already passes through UnwrappedCache via get_unwrapped(). No additional caching is required.

Configuration

Both caches are configured via the server configuration file or CLI flags:

TTL vs TTI: max_age (time-to-idle) resets on every cache hit. A frequently accessed key is never evicted by TTI alone. max_ttl is an absolute ceiling independent of access frequency. For compliance environments that require a maximum plaintext-key exposure window, always set max_ttl.

Security considerations

1. ObjectCache never stores plaintext keys. It mirrors the database: objects are stored as-retrieved (wrapped). All callers provide the DB state.

2. UnwrappedCache validates against wrapped fingerprint. peek() compares the caller's wrapped_object fingerprint against the stored fingerprint. If the wrapped object changed (re-wrapped, corrupted), the entry is silently rejected.

3. Security guard rejects wrapped == unwrapped. insert() returns an error if the caller passes an already-unwrapped object as the wrapped argument. This is defense-in-depth against plaintext-persistence bugs.

4. UnwrappedCache does NOT participate in persistence. Unlike ObjectCache (checked by retrieve_object before every DB read), UnwrappedCache is only consulted by get_unwrapped(). Objects from UnwrappedCache are consumed in-memory for cryptographic operations and never written back to the database.

5. Concurrent cold-start double-unwrap is acceptable. If two workers call get_unwrapped() simultaneously for a cold key, both perform the cryptographic unwrap. The second insert() overwrites the first. This wastes one unwrap but avoids lock contention — a tradeoff favoring throughput.

6. TTI amplifies plaintext exposure window. With the default 15-second time-to-idle, a continuously accessed key's plaintext material stays resident indefinitely. For compliance environments with a bounded key-exposure requirement (e.g. FIPS operational security policies, PCI-DSS, or NIS2), set --unwrapped-cache-max-ttl to enforce an absolute ceiling:

   cosmian_kms --unwrapped-cache-max-ttl 60  # evict plaintext after at most 60 s
   

7. Memory-zeroization on eviction. When an entry is evicted from UnwrappedCache, the plaintext Object is explicitly zeroized before deallocation. All Zeroizing<Vec<u8>> byte buffers (symmetric keys, private-key components) and SafeBigInt fields are overwritten with zeroes. This reduces the window during which sensitive key material could be recovered from freed memory pages.

8. CPU-level side-channel risk (CacheFX / Prime+Probe). Plaintext key material held in the UnwrappedCache occupies resident DRAM pages. Modern CPU microarchitectural attacks (Prime+Probe, Flush+Reload) can potentially exfiltrate AES round-key material and RSA/EC private-key bits from shared L3 cache lines. These attacks require local code execution on the same physical host. Mitigations:

   • Keep --unwrapped-cache-max-ttl low (≤ 60 s) to limit the attack window.
   • Use --disable-unwrapped-cache in high-security deployments where KEK unwrap latency is acceptable.
   • Run the KMS on dedicated (non-shared) hardware or inside a hardware-enforced enclave (SEV-SNP, TDX) to eliminate co-tenant cache-sharing.

Disabling the UnwrappedCache

The UnwrappedCache trades security for performance: every cache hit saves a full KEK-unwrap operation but keeps plaintext key material in process memory between calls.

To disable it entirely — so that every operation performs a fresh cryptographic unwrap and no plaintext key material is ever cached — use:

# CLI flag
cosmian_kms --disable-unwrapped-cache

# Environment variable
KMS_DISABLE_UNWRAPPED_CACHE=true cosmian_kms

# Configuration file  (kms.toml / server.toml)
disable_unwrapped_cache = true

When disabled:

• peek() always returns None, causing every get_unwrapped() to perform the full KEK-unwrap cryptographic operation.
• insert() is a no-op; plaintext key material is never stored in the cache.
• The ObjectCache (wrapped objects) is not affected: database round-trips are still cached for performance.

Trade-off: Disabling the UnwrappedCache increases per-operation latency by one full asymmetric-key operation (RSA-OAEP or ECDH unwrap). For a 2048-bit RSA KEK on a modern server, this is typically 0.5–2 ms per call. For symmetric KEKs (AES-256 key-wrap), the overhead is under 0.1 ms.

The ObjectCache can be configured to very short TTI or very small capacity to reduce wrapped-object retention, but it cannot be disabled via a flag — set --cache-max-age 1 --cache-max-size 1 to minimise its footprint if needed.

HSM slot caches

Source: crate/hsm/base_hsm/src/slots.rs and crate/hsm/base_hsm/src/session/session_impl.rs.

SlotManager creates one object-handle cache and one OAEP hash support cache for each HSM slot. The caches are in memory and shared with sessions opened for that slot.

ObjectHandlesCache

ObjectHandlesCache maps an object identifier byte vector to a CK_OBJECT_HANDLE. Session::get_object_handle() checks the cache before querying PKCS#11. On a miss, it searches CKA_ID, falls back to CKA_LABEL if needed, then inserts the discovered handle.

The cache is a 100-entry LruCache protected by a Mutex. A hit updates LRU recency and therefore still takes the mutex. There is no time-based expiry. The KMS removes the entry after destroying its object; clear_object_handles() clears the cache explicitly.

OAEP hash support cache

supported_oaep_hash_cache is a slot-shared Mutex<Option<Vec<CK_MECHANISM_TYPE>>>. On its first call, Session::get_supported_oaep_hash() creates a temporary RSA-2048 key pair and tests RSA-OAEP encryption with SHA-1, SHA-256, SHA-384, and SHA-512. It stores the mechanisms that succeed and reuses the list for later calls. The cache has no TTL or separate capacity and is discarded with its SlotManager.

Key metadata is not cached

The current base_hsm source has no KeyMetadataCache type or field. Session::get_key_metadata() reads the relevant metadata from PKCS#11 when called; ObjectHandlesCache caches handles, not key metadata.

The SlotManager session pool is a resource-reuse pool, not a cache of key or object data.

Authentication and JWKS caches

SPIRE token validation

Source: crate/server/src/middlewares/spire_token.rs.

SpireTokenCache stores successful auth-verifier lookup results under the SHA-256 hash of the raw token. The cached value is the resolved entity and policies; the raw bearer token is not stored in the cache. Failed validations are not cached.

vault_token_cache_ttl_secs controls entry lifetime, defaults to 30 seconds, and can be set to 0 to disable cache hits. When the map reaches 100,000 entries, insertion triggers an expired-entry sweep; live entries are not evicted to enforce a hard capacity. A still-valid cached token can therefore remain accepted until its TTL expires after the auth-verifier revokes it. See the SPIRE/SPIFFE integration guide for the operational setting.

JWKS verification keys

Source: crate/server/src/middlewares/jwt/jwks.rs.

JwksManager keeps parsed JWK sets in a process-local map keyed by JWKS URI. Initialization fetches the configured sets. Calls to refresh() fetch no more often than once every 60 seconds; a forced refresh bypasses that throttle but has a 5-second cooldown. A refresh replaces the in-memory map with the sets fetched successfully in that refresh. The manager has no entry-count bound or disk persistence.

Certificate response caches

CRL caches

The server uses two distinct CRL caches:

• GENERATED_CRL_CACHE stores the latest locally generated CRL by issuer certificate UID. An entry is used for up to 60 seconds, after which the public CRL endpoint reloads the shared crls database table and refreshes its local copy. If the database read fails, the handler may serve the previous in-memory copy.
• CRL_CACHE_MAP stores downloaded CRLs by distribution-point URI for certificate validation. An entry is reused for at most 300 seconds and only while the CRL remains valid through nextUpdate. Expired entries are purged when a fetched CRL is inserted.

Both caches are process-local hash maps protected by Tokio RwLocks. They have no explicit entry-count limit.

Sources: crate/server/src/core/operations/generate_crl.rs and crate/server/src/core/operations/validate.rs.

OCSP response cache

Source: crate/server/src/routes/ocsp/handler.rs.

OCSP_CACHE stores signed DER responses by CA UID and the complete CertID (serial, hash algorithm, issuer-name hash, and issuer-key hash). The configured ocsp_cache_ttl_secs controls both entry expiry and the response's advertised cache lifetime; its default is 86,400 seconds. The cache holds at most 10,000 entries and purges expired entries before inserting at capacity.

Only single-CertID requests without an honored nonce are cacheable. Unknown certificate statuses and responses under a compromised CA are not cached. The Revoke operation immediately evicts the response for the revoked certificate; other entries expire by TTL.

Client-side cache hints

The Microsoft DKE endpoint includes a cache.exp value in its public-key response, set to one day in the future. This tells the Office client how long it may cache the public key; the KMS does not keep a corresponding in-memory DKE key cache. See the Microsoft DKE integration guide.