nestonexStart a project What are embeddings, and when is semantic search actually better than keyword search?
An embedding is a list of numbers representing a piece of text so that similar meanings sit close together. It lets you find passages that mean the same thing in different words. It is worse than keyword search at exact identifiers, negation and rare domain terms, which is why production systems usually run both.
A model reads a piece of text and outputs a fixed-length list of numbers — a few hundred to a few thousand of them. The numbers have no individual meaning. What matters is position: texts the model considers similar end up close together in that space, and texts it considers unrelated end up far apart.
So 'how do I cancel my subscription' and 'stopping a recurring payment' land near each other despite sharing almost no words. That is the entire value proposition, and it is a real one — it is the thing keyword search fundamentally cannot do.
The catch is that 'similar' is defined by the model that produced the vectors, trained on text that was probably not yours. Similarity in your domain is its guess at similarity, and that guess is better in some domains than others.
Exact identifiers. Part number SKU-44192, an error code, an invoice reference. The embedding of an identifier is close to the embedding of other identifiers, which is precisely the wrong neighbourhood. Keyword matching finds it instantly.
Negation. 'Deployments that do not require downtime' and 'deployments that require downtime' embed close together, because they are about the same subject. Similarity captures topic far more reliably than it captures polarity.
Rare domain terms. A term that barely appeared in the training data has a vague position, so retrieval on it is noisy. Ironically these are often the terms that matter most in a specialist corpus.
Short queries. Two or three words carry little signal, and a vague vector retrieves vaguely. Hybrid retrieval — semantic and keyword, with the results merged — covers all four cases, and is the default we would start from rather than the optimisation we would reach for later.
Every chunk is embedded once at write time and re-embedded whenever the text changes, which is a per-document cost you can estimate up front. Every query is embedded once, which is a per-query cost and usually the smaller number.
The larger operational cost is the re-embedding you do not plan for. Changing the embedding model means re-embedding the entire corpus, because vectors from two different models are not comparable — they are points in different spaces. Treat the model choice as a migration, not a setting.
Storage is usually unremarkable until it is not. A few hundred thousand chunks at typical dimensions is real memory, and that is normally the wall a hand-rolled store hits first.
Fix chunking first. It is the highest-leverage change available and costs nothing but a re-index. Splitting on section and word boundaries with overlap, rather than on a character count, prevents the answer from being sliced in half.
Embed the right text. Prepending a document title or section heading to a chunk gives it context it otherwise lacks, and often does more for retrieval than a larger model would.
Normalise. Unit-length vectors turn cosine similarity into a dot product, so retrieval becomes one matrix operation.
And measure against a baseline. Build the TF-IDF version and compare. If keyword matching is within a few points on your corpus, the embeddings are buying complexity rather than relevance.
Our RAG pipeline puts both approaches behind one embedder interface — local sentence-transformers and a TF-IDF baseline — so switching between them is a configuration change rather than a rewrite. Being able to answer 'are the embeddings actually winning here' in an afternoon is worth more than any single model choice.
The vectors are L2-normalised at write time, which makes similarity a dot product and removes the need for a dedicated vector service at this scale. The whole pipeline runs offline, with no external API call anywhere in it.
No, unless the pair was explicitly trained together as an asymmetric encoder. Vectors from two independently trained models sit in different spaces, so the distances between them are meaningless. Changing the model means re-embedding everything.
Not reliably. Dimension count affects capacity and cost, not domain fit. A smaller model trained on text resembling yours routinely beats a larger general one, and it is cheaper to run. Test on your own corpus rather than trusting a benchmark table.
Chunks, in almost every case. A whole-document vector averages everything the document says, so a long document matches everything weakly and nothing strongly. Chunking keeps each vector about one thing, which is what makes retrieval precise.
No. Embeddings are the vectors; a vector database is one place to keep and search them. At small to moderate scale an in-process array does the same job, and the database becomes worthwhile when you need scale, concurrent writers or filtered search.