The customer says the system must be 100% accurate or it is useless. How do you handle that?
Think first
Find out what the current human process achieves, and design for errors to be caught.
Strong answer
Do not argue; ask. "What accuracy does the current process achieve, and how do you know?" Usually nobody has measured it, and humans are well below 100%. Then reframe around consequence: which errors are costly and which are cheap? Design so costly errors are caught, through citations users can check, confidence thresholds that route uncertain cases to people, and human approval on irreversible actions. Agree a measurable target against a baseline (e.g. match expert answers on 95% of a labelled set, with zero unsupported answers on the safety-critical subset) and an evaluation the customer helps build. Accuracy becomes a number both sides own instead of an absolute.
Weak-answer indicators
- Promises 100%
- Dismisses the concern
- No discussion of error consequences
Likely follow-up questions
- How would you build the labelled evaluation set with the customer?