AI Engineering Concepts
Definitions of the key concepts in AI engineering — each with a simple explanation, engineering depth, example, and the common misconceptions to avoid.
RAG (Retrieval Augmented Generation)
A pattern that grounds LLM responses in retrieved documents, reducing hallucination and enabling citations.
Embeddings
Dense vector representations of text that capture semantic meaning for similarity search.
Chunking
Splitting long documents into smaller units for embedding and retrieval.
Vector Database
A database optimised for storing and searching high-dimensional vectors (embeddings).
Reranker
A cross-encoder model that re-scores retrieval candidates for higher precision.
Hybrid Search
Combining vector (semantic) and BM25 (keyword) search for more robust retrieval.
AI Agent
An LLM with tools, memory and a loop that decides, acts and observes until a task is complete.
ReAct (Reason + Act)
The canonical agent loop pattern: reason about the task, take an action, observe, repeat.
Tool Calling
The mechanism by which an LLM decides to invoke a function you've defined.
Function Calling
Synonym for tool calling — the LLM API feature that returns structured function invocations.
MCP (Model Context Protocol)
An open standard for connecting AI applications to external tools and data sources.
Agent Memory
Persisting context across turns (short-term) and sessions (long-term) for agents.
Token
The unit of text an LLM processes — roughly 4 characters or 0.75 words for English.
Context Window
The maximum number of tokens an LLM can attend to in a single call.
Hallucination
When an LLM generates plausible-sounding but factually incorrect or unsupported content.
Grounding
Constraining LLM output to be supported by provided context or verified data.
Faithfulness
An eval metric measuring whether every claim in an answer is supported by the retrieved context.
LLM-as-Judge
Using a model to score the outputs of another model for scalable evaluation.
Guardrail
Input/output safety layers that filter, validate and redact before content reaches the model or user.
Prompt Injection
An attack where untrusted text (user input or retrieved docs) overrides the LLM's instructions.
Structured Output
Constraining LLM output to a defined schema (JSON) for downstream reliability.
VLM (Vision-Language Model)
An LLM extended to process images alongside text — GPT-4o, Claude 3.5, Gemini.
STT (Speech-to-Text)
Converting audio to text — Whisper, Deepgram, AssemblyAI.
TTS (Text-to-Speech)
Converting text to natural-sounding audio — OpenAI TTS, ElevenLabs.
Fine-Tuning
Adapting a pre-trained model's weights on your specific data for customised behaviour.
LoRA (Low-Rank Adaptation)
A parameter-efficient fine-tuning method that trains a tiny fraction of model weights.
Quantization
Reducing model precision (e.g., 16-bit to 4-bit) to save memory and enable local inference.
Semantic Search
Searching by meaning rather than exact keywords, using embeddings.
Tracing
Recording the full execution path of an LLM call (retrieval, generation, tool calls) for debugging.
Observability
The ability to see inside your AI system as it runs — traces, metrics, logs.