AI Platform Engineering

Run the shared infrastructure every AI team builds on

Serve models at scale, put a secure gateway in front of every model with routing, budgets and attribution, and release prompt and model changes safely with evaluation gates, shadow traffic and canaries.

Skills you'll gain

Model ServingCapacity PlanningLLM GatewaysModel RoutingRelease EngineeringEvaluation Gates

Prerequisites

LLM APIsDockerProduction AI basics

Chapters

Serving Models at ScaleAdvanced

What drives the latency, throughput and cost of model serving, when to self-host, and how to measure and scale an inference service.

30m

LLM Gateways & RoutingAdvanced

Put one secure, observable entry point in front of every model, and route each request to the cheapest model that is good enough.

30m

Evaluation & Release PipelinesAdvanced

Release prompt, model and retrieval changes like code: versioned, gated by evaluations, rolled out gradually and easy to roll back.

30m