Security

Secrets Management

Problem

LLM API keys in .env files or code lead to leaks, unrotated keys, and no audit of who used which key.

Pattern

Store secrets in a vault (HashiCorp Vault, AWS Secrets Manager). Fetch at startup or per-request with short-lived tokens. Rotate frequently. Never log secrets.

Implementation

secrets.pypython

from functools import lru_cache
import boto3

@lru_cache(maxsize=1)
def get_openai_key() -> str:
    client = boto3.client("secretsmanager")
    resp = client.get_secret_value(SecretId="prod/openai/key")
    return resp["SecretString"]

def call_llm(messages):
    # Key fetched from vault, never from env
    key = get_openai_key()
    return openai.chat.completions.create(api_key=key, ...)

# Rotate keys without redeploying apps — vault returns new value
# Audit access via CloudTrail

Trade-offs

Centralised, audited secrets
Rotation without redeploy
Vault adds a dependency
Latency to fetch (cache to mitigate)

Production checklist

No secrets in env vars or code
Vault for production
Automatic rotation
Access audit
Secrets never logged
Separate keys per environment