Embeddings and Vector Search, From Cosine to ANN
How text becomes geometry, why cosine similarity works, and how approximate nearest neighbor search makes it fast at scale.
Meaning as geometry
An embedding model maps text to a point in high-dimensional space so that similar meanings land near each other. Semantic search then becomes a geometry problem: find the stored vectors closest to your query vector.
Why cosine similarity
Most embedding spaces care about **direction**, not magnitude — two documents about the same topic point the same way even if one is longer. Cosine similarity measures exactly that angle.
import numpy as np
def cosine(a, b):
a = a / np.linalg.norm(a)
b = b / np.linalg.norm(b)
return float(a @ b)If you normalize your vectors once at write time, cosine similarity reduces to a plain dot product and every query gets cheaper.
Exact search does not scale
Comparing a query against millions of vectors one by one is linear and slow. Approximate nearest neighbor (ANN) indexes — HNSW graphs, IVF partitions — trade a sliver of recall for orders-of-magnitude speedups.
Tuning the recall/latency knob
HNSW exposes `ef_search`: raise it for higher recall, lower it for speed. Measure recall against a brute-force baseline on a sample before you trust the index in production.
A note on chunking
Half of retrieval quality is decided before search ever runs, in how you split documents. Chunks that are too large blur the signal; too small and they lose context. Start around a few hundred tokens with slight overlap and tune against your own eval set.
Written by the AI Blog editorial team
Deep dives written by ML engineers and researchers who ship models in production. Replace this bio with your own — a line about your background and the systems you build goes a long way with technical readers.