Tracing
Recording the full execution path of an LLM call (retrieval, generation, tool calls) for debugging.
Simple explanation
A trace is a record of every step your AI system took: what was retrieved, what prompt was sent, what the model returned, how long each step took. Without traces, production AI is a black box.
Engineering explanation
Tracing uses OpenTelemetry spans: root span (request) → child spans (retrieve, rerank, generate, tool_call). Each span has attributes (model, token_count, cost, latency, scores). Export to LangSmith, Honeycomb, Jaeger. Production: 100% error traces, 1% success sampling, dashboards on P50/P95/P99 latency + cost + quality.
Example
Slow query (4.2s) → trace shows retrieve=3.4s (bottleneck) → fix: rebuild HNSW index.
When to use it
Every production LLM system. Non-negotiable for debugging, cost attribution and quality monitoring.
Common misconception
Tracing is NOT optional. 'I'll add it later' means you can't debug the first production incident. Add tracing from day one.