Solution Architecture
Designing AI systems under real customer constraints.
Solution architecture
Solution architecture is designing a system that meets the customer's requirements under their constraints. It's not the 'best' architecture in the abstract — it's the best architecture FOR THESE CONSTRAINTS. A bank needs RBAC + audit; a hospital needs BAA + asymmetric safety; a startup needs cost efficiency. Same patterns, different emphases.
Why architecture under constraints
Generic architectures fail in production because they ignore constraints. 'Best practice RAG' without RBAC fails a bank's security review. 'Best practice agents' without a cost ceiling bankrupts a startup. The FDE's job is to find the architecture that satisfies ALL constraints simultaneously — which usually means trade-offs, not 'best' choices.
Architecture process
Requirements + constraints → candidate architectures → trade-off analysis (cost, latency, reliability, security, complexity) → select → document (diagram + components + data flow + failure modes) → validate with security/infra → build. The architecture is a decision log, not a diagram.
Architecture decision record
from dataclasses import dataclass
@dataclass
class Decision:
id: str
title: str
context: str # why this decision is needed
options: list[dict] # {name, pros, cons, cost, latency, security}
chosen: str
rationale: str # why this option, given constraints
decisions = [
Decision(
id="ADR-001",
title="Vector database choice",
context="Need vector search over 50k documents with metadata filtering for RBAC.",
options=[
{"name": "Pinecone (managed)", "pros": "no ops", "cons": "vendor lock-in, $$", "cost": "$$$", "latency": "low", "security": "data leaves env"},
{"name": "pgvector (Postgres)", "pros": "RBAC in SQL, no new infra, relational + vector in one", "cons": "self-managed", "cost": "$", "latency": "low", "security": "data stays in env"},
{"name": "Qdrant (self-hosted)", "pros": "fast, no vendor lock-in", "cons": "new infra to operate", "cost": "$$", "latency": "low", "security": "data stays in env"},
],
chosen="pgvector",
rationale="Customer constraint: on-prem preferred, RBAC mandatory, existing Postgres. pgvector keeps vectors next to metadata, enabling RBAC filter at SQL level. Lowest cost, no new infra, data stays in env.",
),
Decision(
id="ADR-002",
title="Model provider",
context="Need BAA-covered endpoint for PHI / sensitive data.",
options=[
{"name": "Public OpenAI API", "pros": "easy", "cons": "no BAA, data leaves env", "cost": "$", "latency": "low", "security": "fails compliance"},
{"name": "OpenAI Azure (enterprise)", "pros": "BAA available, regional", "cons": "enterprise pricing", "cost": "$$", "latency": "low", "security": "compliant"},
{"name": "On-prem Llama", "pros": "data never leaves", "cons": "ops burden, lower quality", "cost": "$$", "latency": "higher", "security": "most compliant"},
],
chosen="OpenAI Azure (enterprise)",
rationale="BAA required for compliance. On-prem Llama quality gap too large for the use case. Azure enterprise offers BAA + regional residency + acceptable quality.",
),
]
def architecture_doc(decisions: list[Decision]) -> str:
return "\n\n".join(
f"## {d.id}: {d.title}\nContext: {d.context}\nChosen: {d.chosen}\nRationale: {d.rationale}"
for d in decisions
)Experiment: constraint-driven architecture
See how different constraints drive different architecture choices.
What to observe
Constraints drive architecture. HIPAA forces BAA-covered models. On-prem-only forces self-hosted Llama. 100k users force replicas + caching. $0.005/query forces aggressive caching + small models. The 'best' architecture is constraint-dependent — there is no universal best.
Production architecture
Production FDE architecture: an architecture decision record (ADR) per major decision, a diagram showing components + data flow, a failure-mode analysis, a security review, an infra cost projection, and validation with the customer's security/infra teams BEFORE build. The ADR is the artifact — the diagram is a view of it.
Challenge
A hospital wants an AI triage assistant. PHI in scope. High-acuity false-negative rate < 1%. 99.95% uptime. Design the architecture and write the ADRs. (See the Healthcare FDE scenario.)
Production checklist
Production checklist
0 of 10 checked
Knowledge check
A healthcare customer needs PHI-safe AI. Which model choice is correct?
Complete
You can now design architecture under constraints. Next: prototyping and productionisation — from POC to production.
Mark this chapter as complete
Track your progress and unlock the next chapter.