All challenges
PerformanceIntermediate
Vision Model Misreads Charts
Your multimodal assistant answers questions about uploaded charts. It's confident but wrong on ~30% of numeric values. Users lose trust.
Symptoms
- 30% of numeric values are wrong
- Model is confident even when wrong
- Bar charts and line charts affected
- Large numbers (millions, billions) most affected
Evidence
Sample failure
Chart shows revenue: $4.2M (Q1), $4.8M (Q2), $5.1M (Q3) Model answer: 'Revenue grew from $42M to $51M across the quarters'
Prompt
What does this chart show?
Tasks
- 1Diagnose the failure mode
- 2Improve the prompt
- 3Add a verification step