Command Palette

Search for a command to run...

Chapter 2·34 min read·Expert

Solution Architecture

Designing AI systems under real customer constraints.

Solution architecture

Solution architecture is designing a system that meets the customer's requirements under their constraints. It's not the 'best' architecture in the abstract — it's the best architecture FOR THESE CONSTRAINTS. A bank needs RBAC + audit; a hospital needs BAA + asymmetric safety; a startup needs cost efficiency. Same patterns, different emphases.

Why architecture under constraints

Generic architectures fail in production because they ignore constraints. 'Best practice RAG' without RBAC fails a bank's security review. 'Best practice agents' without a cost ceiling bankrupts a startup. The FDE's job is to find the architecture that satisfies ALL constraints simultaneously — which usually means trade-offs, not 'best' choices.

Architecture process

Requirements + constraints → candidate architectures → trade-off analysis (cost, latency, reliability, security, complexity) → select → document (diagram + components + data flow + failure modes) → validate with security/infra → build. The architecture is a decision log, not a diagram.

Architecture decision record

adr.pypython
from dataclasses import dataclass

@dataclass
class Decision:
    id: str
    title: str
    context: str        # why this decision is needed
    options: list[dict] # {name, pros, cons, cost, latency, security}
    chosen: str
    rationale: str      # why this option, given constraints

decisions = [
    Decision(
        id="ADR-001",
        title="Vector database choice",
        context="Need vector search over 50k documents with metadata filtering for RBAC.",
        options=[
            {"name": "Pinecone (managed)", "pros": "no ops", "cons": "vendor lock-in, $$", "cost": "$$$", "latency": "low", "security": "data leaves env"},
            {"name": "pgvector (Postgres)", "pros": "RBAC in SQL, no new infra, relational + vector in one", "cons": "self-managed", "cost": "$", "latency": "low", "security": "data stays in env"},
            {"name": "Qdrant (self-hosted)", "pros": "fast, no vendor lock-in", "cons": "new infra to operate", "cost": "$$", "latency": "low", "security": "data stays in env"},
        ],
        chosen="pgvector",
        rationale="Customer constraint: on-prem preferred, RBAC mandatory, existing Postgres. pgvector keeps vectors next to metadata, enabling RBAC filter at SQL level. Lowest cost, no new infra, data stays in env.",
    ),
    Decision(
        id="ADR-002",
        title="Model provider",
        context="Need BAA-covered endpoint for PHI / sensitive data.",
        options=[
            {"name": "Public OpenAI API", "pros": "easy", "cons": "no BAA, data leaves env", "cost": "$", "latency": "low", "security": "fails compliance"},
            {"name": "OpenAI Azure (enterprise)", "pros": "BAA available, regional", "cons": "enterprise pricing", "cost": "$$", "latency": "low", "security": "compliant"},
            {"name": "On-prem Llama", "pros": "data never leaves", "cons": "ops burden, lower quality", "cost": "$$", "latency": "higher", "security": "most compliant"},
        ],
        chosen="OpenAI Azure (enterprise)",
        rationale="BAA required for compliance. On-prem Llama quality gap too large for the use case. Azure enterprise offers BAA + regional residency + acceptable quality.",
    ),
]

def architecture_doc(decisions: list[Decision]) -> str:
    return "\n\n".join(
        f"## {d.id}: {d.title}\nContext: {d.context}\nChosen: {d.chosen}\nRationale: {d.rationale}"
        for d in decisions
    )

Experiment: constraint-driven architecture

See how different constraints drive different architecture choices.

Set constraints. See how the architecture changes.

What to observe

Constraints drive architecture. HIPAA forces BAA-covered models. On-prem-only forces self-hosted Llama. 100k users force replicas + caching. $0.005/query forces aggressive caching + small models. The 'best' architecture is constraint-dependent — there is no universal best.

Production architecture

Production FDE architecture: an architecture decision record (ADR) per major decision, a diagram showing components + data flow, a failure-mode analysis, a security review, an infra cost projection, and validation with the customer's security/infra teams BEFORE build. The ADR is the artifact — the diagram is a view of it.

Challenge

A hospital wants an AI triage assistant. PHI in scope. High-acuity false-negative rate < 1%. 99.95% uptime. Design the architecture and write the ADRs. (See the Healthcare FDE scenario.)

Production checklist

Production checklist

0 of 10 checked

Knowledge check

A healthcare customer needs PHI-safe AI. Which model choice is correct?

Complete

You can now design architecture under constraints. Next: prototyping and productionisation — from POC to production.

Mark this chapter as complete

Track your progress and unlock the next chapter.

Continue learning