Engineering Challenges

Your RAG system over internal docs has excellent retrieval scores (recall@5 = 0.91) but users complain the answers are wrong, vague or cite the wrong source. Diagnose the problem.

Your research agent occasionally runs for 40+ minutes and costs $15 per run before timing out. The agent keeps calling the same tool with slightly different arguments. Diagnose and fix.

Your RAG system over user-uploaded docs started producing outputs that reveal the system prompt and refuse to answer normally. One user uploaded a document that 'broke' the assistant.

Your AI feature launched 2 weeks ago. Costs were $200/day in week 1. Today is $4,800/day. Traffic is up 3x but cost is up 24x. Find the leak.

You upgraded gpt-4o to gpt-4o-2025-01. Production metrics look fine but eval scores dropped 15%. Users haven't complained yet but you're worried.

Your agent depends on a weather API. When the API returns 500s, the agent retries 8 times, then crashes with an unhandled exception instead of answering with available information.

Your multimodal assistant answers questions about uploaded charts. It's confident but wrong on ~30% of numeric values. Users lose trust.

A customer wants to deploy an internal AI assistant across 50,000 internal policy, procedure and support documents for 12,000 employees. Design the solution.

Your streaming agent occasionally crashes with 'Invalid tool call: missing arguments'. It only happens for long tool calls and is intermittent.

You upgraded your embedding model from text-embedding-3-small to text-embedding-3-large. Search quality dropped 40% overnight despite the new model being 'better'.