Embeddings: Turning Text into Vectors
On this page
A user types "I forgot my password" into your help-centre search. The right article is titled "Recover access to your account". Keyword search finds nothing in common between the two and returns "Password policy for admins" instead, because that one contains the word "password". The search compares spelling; the user meant something. Embeddings let a system compare meaning.
How it works

Text becomes a point in space
An embedding model reads a piece of text and outputs a fixed-length list of numbers, a vector, often a few hundred to a few thousand numbers long. The model is trained so that texts with similar meaning get vectors that point in similar directions, and unrelated texts end up far apart.

No single number means anything on its own ("dimension 17 is not 'about passwords'"). Meaning lives in the direction of the whole vector.
How the model learns this
Most text embedding models are transformer encoders trained with a contrastive objective: show the model pairs that should match (a question and its answer, a title and its article, two paraphrases) and push their vectors together, while pushing apart vectors of texts that don't match. After training on many such pairs, "forgot my password" and "reset login" end up close, because they kept appearing as matches.
Measuring similarity
To compare two embeddings, systems usually use cosine similarity: the cosine of the angle between the vectors. It is 1 when they point the same way and near 0 when they are unrelated. If vectors are normalised to length 1, cosine similarity is just the dot product, which is fast to compute.

import math
def cosine(a, b):
dot = sum(x * y for x, y in zip(a, b))
na = math.sqrt(sum(x * x for x in a))
nb = math.sqrt(sum(y * y for y in b))
return dot / (na * nb)
print(round(cosine([1.0, 2.0, 0.0], [2.0, 4.0, 0.0]), 2)) # 1.0 same direction
print(round(cosine([1.0, 0.0, 0.0], [0.0, 1.0, 0.0]), 2)) # 0.0 unrelated
Search with embeddings
- Offline: embed every document (usually split into chunks) and store the vectors in a vector index.
- Online: embed the query with the same model, find the stored vectors with the highest similarity, and return their text.
Comparing against every vector works for thousands of items; for millions, systems use approximate nearest neighbour indexes such as HNSW or IVF (see Vector Search).
Trade-offs and costs
- Meaning versus exact terms. Embeddings handle paraphrases and synonyms well, but can miss exact identifiers: error codes, SKUs, names, version numbers. Keyword search (BM25) is better there, which is why many systems combine both (see Hybrid Search and Reranking).
- Dimensions. More dimensions can capture more nuance but cost more storage and slower search. 1 million vectors of 1,536 float32 numbers take about 6 GB (1,000,000 × 1,536 × 4 bytes) before index overhead.
- Model choice. General-purpose models are a fine start; domain-specific text (legal, medical, code) may need a domain model or fine-tuning. Some models support shortening vectors with little quality loss, which saves storage.
- Bi-encoder versus cross-encoder. Embedding query and document separately (a bi-encoder) is fast because documents are embedded once. A cross-encoder reads the query and document together and is more accurate but too slow to run on every document, so it is used to rerank a short list.
In production
- Same model, always. Query and documents must be embedded by the same model and version. Changing the model means re-embedding the whole corpus, so plan for it as a migration with a parallel index.
- Latency. Embedding a short query is fast (milliseconds on a GPU, or one API call); the bigger costs are bulk-embedding the corpus and keeping it fresh as documents change.
- Cost. Embedding is far cheaper per token than generation, but a large corpus re-embedded often still adds up.
- Failure modes: stale vectors after documents change; mismatched models after an upgrade; chunks too big (a vague average of many topics) or too small (no context); similarity thresholds copied from another model, since score ranges differ between models.
When to reach for it
- Semantic search and the retrieval step of RAG.
- Finding duplicates or near-duplicates (support tickets, questions).
- Clustering and topic discovery.
- Recommendations ("similar articles").
- Semantic caching: reusing an answer for a question that means the same thing.
Example
A help centre has 20,000 articles split into 120,000 chunks. Each chunk is embedded once (offline) into a 768-number vector: about 120,000 × 768 × 4 bytes ≈ 370 MB, small enough for one machine's memory. A query is embedded at request time, the 20 most similar chunks are fetched, and a reranker picks the best 5 to show or to pass to an LLM. "I forgot my password" now returns "Recover access to your account" first.
Common mistakes
- Embedding queries and documents with different models, or upgrading the model without re-embedding.
- Using embeddings alone for exact-match needs like product codes or error messages.
- Embedding whole long documents into one vector, which blurs many topics together.
- Using a fixed similarity cutoff from a blog post; calibrate thresholds on your own data.
- Forgetting to normalise when the index expects dot product on unit vectors.
Say this in the interview
"An embedding model maps text to a vector so that similar meanings point in similar directions. It's trained contrastively on matching pairs. For search, I embed document chunks offline, embed the query with the same model online, and find nearest neighbours by cosine similarity using an ANN index. Embeddings are great for paraphrases but weak on exact identifiers, so I'd add keyword search and a reranker for precision."
Interview questions on this
- Why cosine similarity instead of Euclidean distance? It compares direction, ignoring length; for normalised vectors the two give the same ranking.
- What happens when you change the embedding model? Old and new vectors are not comparable; you must re-embed everything, ideally into a new index and switch over.
- When would keyword search beat embeddings? Exact strings: names, codes, IDs, rare technical terms.
- Bi-encoder vs cross-encoder? Bi-encoder embeds separately and is fast for retrieval; cross-encoder reads both together, is more accurate, used for reranking.
- How much memory do 10 million 1,024-dimension float32 vectors need? 10M × 1,024 × 4 bytes ≈ 41 GB before index overhead; quantising to int8 cuts that to about a quarter.
Used in
- AI Engineering: Vector Search (HNSW, IVF); RAG End to End; Hybrid Search and Reranking; Semantic Caching; Implement Vector Top-k Retrieval (ML coding).
- AI System Design: Embeddings and vector search; Design Perplexity; Design Enterprise RAG.