All challenges
PerformanceIntermediate
Cost Blowup After Launch
Your AI feature launched 2 weeks ago. Costs were $200/day in week 1. Today is $4,800/day. Traffic is up 3x but cost is up 24x. Find the leak.
Symptoms
- Requests up 3x, cost up 24x
- P95 latency up from 1.2s to 4.8s
- Most cost on the frontier model, not the small model
- Cache hit rate dropped from 60% to 8%
Evidence
Cost breakdown
Week 1: $200/day (small model 80%, frontier 20%) Week 2: $4,800/day (small model 12%, frontier 88%)
Recent changes
- Removed caching layer 'to simplify' - Bumped default model from mini to frontier - Added conversation history (full) to every prompt
Average prompt size
Week 1: 450 tokens Week 2: 8,200 tokens (history included)
Tasks
- 1Identify all cost drivers
- 2Propose a recovery plan
- 3Design cost guardrails going forward