What are embeddings? How do they work and why are they useful?
Think first
Think about: vectors, semantic similarity, encoding, dimensionality, use cases.
Strong answer
Embeddings are dense vector representations of text (or images) that capture semantic meaning. A trained encoder model (e.g., text-embedding-3-small) converts text into a fixed-length vector (e.g., 1536 dimensions). Texts with similar meanings have vectors that are close together (high cosine similarity). This enables semantic search — find items by meaning, not keywords. Use cases: RAG retrieval, semantic search, clustering, deduplication, recommendation, classification. Key engineering points: (1) Embeddings from different models are NOT compatible (different dimensions, different spaces). (2) Dimensionality affects cost and quality — higher = more precise but slower. (3) Store in a vector database with HNSW index for sub-linear search. (4) Always pin the embedding model version — mixing models breaks retrieval.
Weak-answer indicators
- Says 'embeddings are like a database index'
- Doesn't mention cosine similarity
- Skips the incompatibility between models
- Can't name a use case beyond search
Likely follow-up questions
- How do you choose an embedding model?
- What's the difference between dense and sparse vectors?
- How do you handle multilingual embeddings?