Embeddings
Dense vector representations of text that capture semantic meaning for similarity search.
Simple explanation
An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meanings have similar vectors. This lets you search by meaning, not just keywords.
Engineering explanation
Embeddings are produced by a trained encoder model (e.g., text-embedding-3-small produces 1536-dim vectors). Similarity is computed via cosine similarity or dot product. Production systems use ANN indexes (HNSW, IVFFlat) for sub-linear search. Embedding model choice affects retrieval quality more than LLM choice in RAG systems.
Example
embed('How do I get a refund?') ≈ embed('What is the return process?') — high cosine similarity despite different words.
When to use it
Semantic search, RAG retrieval, clustering, deduplication, recommendation. Anywhere meaning matters more than exact keywords.
Common misconception
Embeddings from different models are NOT compatible — you cannot compare a 1536-dim OpenAI embedding with a 768-dim sentence-transformers embedding. Mixing models breaks retrieval.