Intermediate40 minBuilding LLM Applications

Call an LLM with Structured Output

Call an LLM and parse a structured JSON response with validation, error handling and retries. The bedrock of every generative AI application.

Scenario

Build the function that every downstream feature will use: a reliable, validated, retried structured LLM call.

Objective

Implement call_structured() that returns a validated Pydantic model from an LLM, with retry on validation failure.

Starter code

Implement the TODOs to complete the lab.

llm.pypython

from pydantic import BaseModel
from openai import OpenAI

client = OpenAI()

class Answer(BaseModel):
    summary: str
    confidence: float

def call_structured(query: str, context: str) -> Answer:
    """Call the LLM and return a validated Answer.
    - Use response_format json_object
    - Retry up to 3 times on validation failure
    - Raise on persistent failure
    """
    # TODO: implement
    pass

Solution hints

1Use response_format={'type': 'json_object'}
2Parse with Answer.model_validate_json()
3Catch ValidationError, retry up to 3x
4Add a system prompt that specifies the JSON schema

Validation steps

Your implementation should pass these checks.

Returns valid Answer for a normal query
Retries on malformed JSON (test by injecting bad output)
Raises after 3 failures
Confidence is between 0 and 1

Run validation

This is a simulated validation environment. In production, this would run your code against the validation steps.