All challenges
RAGAdvanced
Embedding Model Version Drift
You upgraded your embedding model from text-embedding-3-small to text-embedding-3-large. Search quality dropped 40% overnight despite the new model being 'better'.
Symptoms
- Recall@5 dropped 0.88 → 0.53
- New model is objectively better on benchmarks
- Drop happened immediately after upgrade
- Old queries return wrong results
Evidence
Upgrade process
- Updated embed() to call new model - Did NOT re-embed existing 50k documents - Vector index still contains old embeddings
Vector dimensions
Old: 1536-dim New: 3072-dim Index schema: 1536-dim
Tasks
- 1Diagnose the failure
- 2Design the correct upgrade path
- 3Add guardrails to prevent recurrence