Retrieval-augmented generation, built from the ground up.
Normalise your vectors and the database stops being necessary. Once every embedding has unit length, cosine similarity is just a dot product, and a hand-rolled in-process store handles retrieval with no external service. This build chunks on word boundaries, embeds through one swappable interface, and runs entirely offline.
To understand it. A managed pipeline hides the decisions that determine whether retrieval works, and those decisions are where most RAG systems actually fail: chunk size, chunk boundaries, what gets embedded, how similarity is computed.
This was an R&D project rather than client work, built to make every one of those steps visible and adjustable. Running it fully offline was a constraint chosen on purpose, since anything that phones out is a step you cannot inspect.
Cosine similarity divides the dot product of two vectors by the product of their magnitudes. If both vectors are already normalised to unit length, those magnitudes are 1, and the division does nothing.
So normalising at write time removes work from every subsequent query. It is a small piece of arithmetic that turns retrieval into a single matrix operation, which is most of the reason a dedicated vector database is optional at this scale.
Because splitting a document every N characters cuts words in half, and half a word embeds as noise. It is the kind of detail a managed pipeline handles silently and a hand-rolled one has to get right.
Chunk size and overlap are both tunable. Overlap matters because a fact that straddles a boundary is otherwise retrievable from neither chunk.
The ability to change your mind. Local sentence-transformers give proper semantic embeddings; a TF-IDF baseline gives keyword matching with no model at all. Both sit behind the same interface.
Having the baseline is not a fallback so much as a control. If semantic embeddings are not beating TF-IDF on your corpus, that is worth knowing before you build anything else on top of them.
Source and offset. A retrieved chunk that cannot be traced back to where it came from produces an answer nobody can verify, which defeats the purpose of retrieval augmentation.
Top-k chunks are assembled into the prompt with their sources attached, and generation is optional and local, so the whole pipeline can run with no network access at all.
A reference RAG pipeline that runs entirely offline and stays fully inspectable at every step.
| Chunking | Word boundaries, tunable size and overlap |
|---|---|
| Embedders | sentence-transformers, TF-IDF baseline, one interface |
| Normalisation | L2, so cosine similarity becomes a dot product |
| Store | Hand-rolled, in-process, no external vector database |
| Metadata | Source and offset kept with each vector |
| Retrieval | Top-k by dot product over normalised vectors |
| Generation | Optional and local |
| Network | None required, fully offline |
Not at small to moderate scale. If embeddings are L2-normalised, similarity is a dot product and retrieval is one matrix operation over an in-process array. A dedicated vector database earns its place when the corpus outgrows memory or needs concurrent writers, not by default.
Because cosine similarity divides the dot product by both magnitudes, and normalising makes those magnitudes 1. Doing it once at write time removes that division from every query afterwards, turning retrieval into a single dot product.
It depends on the corpus, which is the argument for making it tunable rather than fixed. What matters more universally is splitting on word boundaries rather than character counts, and using overlap so a fact straddling a boundary stays retrievable.
As a control, yes. If semantic embeddings are not measurably beating a TF-IDF baseline on your corpus, that is important to discover early. Keeping both behind one interface makes the comparison cheap to run.
A B2B marketplace for construction and industrial materials.