Docker & Deployment
Containerise, ship and deploy AI services reproducibly.
Docker & deployment
Docker packages your app + dependencies + runtime into a reproducible image that runs identically on your laptop, CI and production. For AI engineering, this matters more than usual: Python version, system libs (for ML), model SDK versions — all must be pinned and reproducible. Docker is how you ship AI without 'works on my machine'.
Why Docker for AI
AI dependencies are notoriously fragile: a CUDA version mismatch, a missing system lib, an SDK version drift — and your model breaks in prod but works locally. Docker eliminates this by packaging everything. CI/CD then automates build, test and deploy. Without this, AI deployments are manual and error-prone.
Deployment architecture
Code → Dockerfile (build image) → CI (build + test + push to registry) → CD (deploy to prod) → container orchestration (Kubernetes/ECS) → load balancer → users. Health checks, rolling updates and rollbacks are built in.
A production Dockerfile + CI
# Multi-stage build for smaller image
FROM python:3.12-slim AS builder
WORKDIR /app
# Install build deps (for ML libs)
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential curl && rm -rf /var/lib/apt/lists/*
# Install deps separately (cached layer)
COPY pyproject.toml uv.lock ./
RUN pip install uv && uv sync --frozen --no-dev
# Copy app
COPY . .
# --- Runtime stage ---
FROM python:3.12-slim AS runtime
WORKDIR /app
# Runtime deps only
RUN apt-get update && apt-get install -y --no-install-recommends \
libgomp1 && rm -rf /var/lib/apt/lists/*
# Copy from builder
COPY --from=builder /app /app
COPY --from=builder /app/.venv /app/.venv
ENV PATH="/app/.venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1
# Non-root user
RUN useradd -m app && chown -R app /app
USER app
# Health check
HEALTHCHECK --interval=30s --timeout=3s \
CMD curl -f http://localhost:8000/health || exit 1
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
# .github/workflows/deploy.yml
# name: Deploy
# on:
# push:
# branches: [main]
# jobs:
# build-and-deploy:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# - run: docker build -t myapp:SHA .
# - run: docker push registry/myapp:SHA
# - run: kubectl set image deployment/myapp myapp=registry/myapp:SHA
# - run: kubectl rollout status deployment/myappExperiment: image optimisation
See how Dockerfile choices affect image size and security.
What to observe
Image size affects deploy speed and cost. Security posture affects blast radius. Multi-stage build strips build deps, halving size. Non-root user limits blast radius on escape. Alpine seems small but breaks ML libs — use slim. Cache the deps layer for fast rebuilds.
Production deployment
Production deployment: multi-stage Dockerfile, non-root user, health check, multi-replica behind a load balancer, rolling updates (no downtime), rollback on health-check failure, secrets from a vault (not env in image), resource limits (CPU/memory), and observability (logs, metrics, traces exported).
Challenge
Your AI service deploys fine but crashes on startup in prod with 'libgomp.so.1 not found'. It works locally. Diagnose and fix the Dockerfile.
Production checklist
Production checklist
0 of 10 checked
Knowledge check
Your container runs as root. What's the risk?
Complete
You can now containerise and deploy AI services reproducibly. This completes the AI Engineering Foundations series.
Mark this chapter as complete
Track your progress and unlock the next chapter.