Vol. 09 · Retrieval & RAG
RAG & Vector Search
Full route: why words cannot be compared directly → Bag of Words → TF-IDF → Word2Vec → contextual embeddings → cosine distance → how exactly the chunk “matches” with the query → HNSW → hybrid search → the full RAG pipeline parsed down to the byte.
Retrieval answer
Full route: why words cannot be compared directly → Bag of Words → TF-IDF → Word2Vec → contextual embeddings → cosine distance → how exactly the chunk “matches” with the query → HNSW → hybrid search → the full RAG pipeline parsed down to the byte. The computer works with numbers. This New Runtime record is an evidence-linked retrieval unit.
Sparse methods: from words to numbers
The computer works with numbers. The strings “cat” and “cat” are different byte sequences with no default proximity between them. The entire history of information retrieval is a history of attempts to turn the meaning of text into numbers that can be compared mathematically.
Bag of Words is the first way to turn text into vector. We build a dictionary of all words in the corpus, representing each document as a vector: how many times each word appears. Word order is ignored - hence the name "bag".
Source-complete notesBoW problems
BoW problems
- “and”, “in”, “on” - frequent, but useless
- vector size = dictionary size (100k+)
- 99% of values = 0 (sparse vector)
- word order lost: “the cat eats the fish” = “the fish eats the cat”
TF-IDF solves the problem of stop words: a word is important if it occurs frequently in the document (TF) and rarely in the rest of the corpus (IDF). “Neural network” in a text about ML is more important than “is” - although both can appear 5 times.
Source-complete notesformula and intuition · used in
texTF(word, document) = count(word in doc) / len(doc)
IDF(word) = log( N / df(word) )
N = number of documents in the corpus
df = in how many documents the word appears
TF-IDF(word, doc) = TF × IDF
Example, corpus = 1000 documents:
word "and": df=1000 → IDF=log(1000/1000)=0
word "algorithm": df=50 → IDF=log(1000/50) =3.0
word "catboost": df=3 → IDF=log(1000/3) =5.8
→ a rare and specific word receives high weight
→ stop words will automatically be zeroedused in
- BM25 - improved TF-IDF (Elasticsearch)
- sparse retrieval in hybrid search
- still doesn't understand synonyms
- “cat” and “cat” are completely different words
BM25 (Best Match 25) - improved TF-IDF with normalization by document length. A long document does not benefit simply from size. This is a standard in Elasticsearch, Opensearch and the baseline for RAG retrieval.
Source-complete notesformula BM25 · when BM25 beats vectors
texBM25(q, d) = Σ_{word∈q} IDF(word) ·
TF(word,d) · (k₁+1)
─────────────────────────────────
TF(word,d) + k₁·(1-b+b·|d|/avgdl)
k₁ ∈ [1.2, 2.0] – TF saturation
b ∈ [0, 1] - length normalization (usually 0.75)
|d| = document length, avgdl = average length
Key properties:
✓ TF saturation: 100 mentions ≠ 10× more important than 10
✓ long docs do not automatically dominate
✓ fast: inverted index → O(|q|) searchwhen BM25 beats vectors
- exact names: “WB-article 12345678”
- codes, abbreviations, proper names
- rare specific terms
- synonyms, paraphrases - does not see
Dense Embeddings: meaning in space
Embedding is a vector of several hundred or thousand numbers that encodes the meaning of the text so that semantically similar texts appear close in vector space. This is the key idea: distance in space equals semantic distance in language.
Source-complete noteskey properties
key properties
- “cat” and “cat” - side by side in space
- “pet” and “cat” are semantically close
- dense: all 768 values are non-zero
- size: 384–3072 depending on model
Word2Vec (2013, Google) is the first widely used neural embedding approach. Idea: a word is determined by the company in which it occurs. “Cat” and “cat” appear in similar contexts → their vectors will be similar.
Source-complete notesSkip-gram architecture
sqlTask: predict neighbors based on the central word
Window = 2:
"the big [cat] is sitting on the sofa"
→ predict: "big", "sitting" from the word "cat"
Neural network (2 layers):
input: one-hot(cat) ∈ ℝ^vocab_size
hidden: W_embed · one-hot → ℝ^300 ← embedding!
output: softmax → P(each word as a neighbor)
After training on a billion tokens:
king − man + woman ≈ queen
→ the arithmetic of meanings works!
Word2Vec problem:
"onion" (vegetable) = "onion" (weapon)
→ one word = one vector, context not taken into accountBERT (2018) solved the problem of polysemy: the embedding of a word depends on the context. The “bow” in “bought a bow for soup” and “took a bow and arrow” receives different vectors. For RAG, Bi-Encoder is used: we encode the request and the document separately, and compare them with cosine.
The embedding model produces a vector for each token. How to make one document vector from N token vectors? This is called pooling - and the choice of strategy affects the quality of the search.
Source-complete notespooling strategies
textEntry: "the cat is sitting at home"
Tokens: [CLS] cat sitting at home [SEP]
Model output: vector for each token ℝ^768
[CLS] pooling (BERT-style):
doc_emb = hidden_state[CLS]
↑ a special token is trained to aggregate meaning
↑ used in BERT fine-tuned models
Mean pooling (Sentence-BERT, E5):
doc_emb = mean(hidden_states[1:-1])
↑ average for all tokens (without CLS and SEP)
↑ in practice it is better for long documents
Weighted mean pooling:
↑ tokens with attention weights are strengthened
↑ used in some modern models
The result of any pooling:
doc_emb ∈ ℝ^768 ← one vector per documentHow Embedding Models are Trained for Search
The embedding model for search is trained so that relevant pairs (query, document) are close in space, and irrelevant ones are far away. This is called contrastive learning, and it explains why “cat price” and “kitten price” give high similarity.
Source-complete notestraining data
training data
- MS-MARCO: 500k query-passage pairs
- NQ, TriviaQA: question and answer from Wikipedia
- synthetic pairs via LLM (for domains)
- hard negatives: similar but irrelevant
Query and document are texts of different nature. Query is short, incomplete, contains intent. The document is long, self-contained, and contains facts. Modern embedding models are trained on special prefixes in order to process them differently.
Source-complete notesinstruction-prefixes (E5, nomic)
text#E5-instruct:
query_emb = encode(
"query: " + "how to treat a cat for a cold"
)
doc_emb = encode(
"passage: " + "For viral infections in cats..."
)
# nomic-embed-text:
query_emb = encode("search_query: " + text)
doc_emb = encode("search_document: " + text)
# Without the correct prefix → quality degradation
# Read the documentation for each model!
Asymmetric matryoshka (OpenAI text-embedding-3):
→ you can trim the vector to the desired length
3072 → 1536 → 512 → 256 without retrainingIt's not magic, it's statistics. The words "cat" and "cat" appear in the same contexts in billions of sentences. After contrastive training, the model learned: if the texts are about the same thing, their vectors should be close. Synonymy arises as a side effect of these statistics.
Source-complete notesfrom statistics to space
sqlIn a body of 100M sentences:
"my cat meows" meets with the same neighbors as
"my cat is meowing"
→ contrastive loss: these two phrases = positive pair
→ gradient descent moves their vectors closer
Likewise:
"cost of goods" ↔ "price of goods" → close
"acquire" ↔ "buy" → close
"expensive" ↔ "cheap" → FAR
(antonyms)
Bottom line: vector space is
“map of meanings” learned from language statisticsVector Search: how to find the nearest neighbor
Cosine similarity measures the angle between two vectors—not the distance between points, but the direction. This is important: two vectors can be far apart in absolute units, but pointing in the same direction means talking about the same thing.
Source-complete notesalternative distance metrics
alternative distance metrics
- cosine - standard for text
- dot product - faster if vectors are normalized
- L2 (Euclidean) - for images, not text
- for normalized vectors all three are equivalent
An exact search of all vectors in the database (Exact KNN) guarantees the correct answer, but it is slow for large collections. ANN - approximate search algorithms: sometimes they miss the perfect result, but hundreds of times faster.
Source-complete notescomparison of approaches
pythonExact KNN (brute force):
for doc in all_docs:
score = cosine(query_emb, doc_emb)
return top_k(scores)
n=1M docs, d=768 → 768M multiplications per request
→ ~2-5 seconds on CPU. Unacceptable.
ANN (HNSW, IVF, ScaNN, FAISS):
→ build the index in advance (~minutes/hours)
→ search: ~1-10ms even for 100M vectors
→ recall@10 = 95-99% (we skip 1-5% of accurate answers)
Trade-off:
accuracy ↑ ↔ speed ↓
memory ↑ ↔ accuracy ↑HNSW (Hierarchical Navigable Small World) is a standard ANN algorithm in most vector databases (Qdrant, Weaviate, pgvector). Builds a multi-level graph: the top level is large jumps, the bottom is fine tuning.
Vector database = HNSW/IVF index + metadata storage + filtering + API. Different databases are optimized for different trade-offs in terms of speed, memory and scaling.
Source-complete notestable
| Base | Algorithm | Filtration | Feature |
|---|---|---|---|
| Qdrant | HNSW | during search | Rust, payload filters |
| pgvector | HNSW / IVFFlat | SQL WHERE | in PostgreSQL |
| Weaviate | HNSW | GraphQL | hybrid built |
| Chroma | HNSW | post-filter | lightweight, dev |
| FAISS | IVF/PQ/HNSW | no | library, not database |
Chunking, Metadata and Matching
Chunk is a unit that we encode into a vector and which we return to the LLM context. A chunk that is too small loses context. Too large - it dilutes the signal, embedding becomes “average in a hospital”. The size and chunking strategy is one of the main parameters of RAG quality.
Source-complete notesempirical recommendations
empirical recommendations
- Q&A: 256–512 tokens
- technical documents: 512–1024
- overlap: 10–20% of chunk_size
- eval in different sizes - no one size fits all
Next to each vector, metadata is stored in the database - structured fields for filtering and for adding context to LLM. The right metadata allows you to combine semantic search with precise filters.
Source-complete notestypical payload structure · use in search
json{
# Identification
"doc_id": "wb-seller-guide-2024",
"chunk_id": "wb-seller-guide-2024-ch-047",
# Document navigation
"title": "WB Seller's Guide",
"section": "3.2 Pricing",
"page": 47,
# Filtering
"category": "pricing",
"language": "ru",
"date": "2024-11-01",
# Content
"text": "When calculating the price, take into account...",
"summary": "About WB pricing methods"
}use in search
- filter: category="pricing" AND date>"2024"
- title in LLM context: "from section 3.2"
- summary for hybrid search
The title of a document or section is a powerful signal. If both the title and the body of the chunk are included in the embedding, then a search for the topic “WB pricing” will find the chunk “3.2 Pricing,” even if the word “pricing” does not appear explicitly in the body of the text.
Source-complete notesheader inclusion strategies
text# 1. Prepend title into chunk text before embedding:
text_for_embedding = f"""
Section: {section_title}
Document: {doc_title}
{chunk_text}"""
# → title affects embedding through pooling
# 2. Separate “title embedding” + weighted sum:
emb_title = encode(title)
emb_body = encode(body)
emb_final = 0.4 emb_title + 0.6 emb_body
# → explicit weighting
#3. HyDE (Hypothetical Document Embedding):
# query → LLM generates a hypothetical answer
# embed a hypothetical response, not a request
# → embedding is already in the “document space”Step-by-step analysis of real matching: the query is received → encoded → compared with each vector in the database → the one closest in cosine wins. It is here that it is clear what is being compared to what.
Hybrid search: Dense + Sparse
Dense search is good for semantic proximity, but loses precision terms. BM25 is good for exact matches, but doesn't understand synonyms. Hybrid search combines both signals and consistently outperforms each individual signal.
Reciprocal Rank Fusion is a simple and reliable way to combine two lists of results without normalizing the absolute scores. The document that gets to the top of both methods rises to the top.
Source-complete notesRRF algorithm
texRRF_score(doc) = Σ_methods 1 / (k + rank_method(doc))
k = 60 (constant, reduces the influence of top 1)
Example:
BM25_rank Dense_rank RRF_score
chunk_A: 1 3 1/61 + 1/63 = 0.032
chunk_B: 5 1 1/65 + 1/61 = 0.032
chunk_C: 2 2 1/62 + 1/62 = 0.032
chunk_D: 100 50 1/160 + 1/110 = 0.015
chunk_C (good at both) → up!
chunk_D (bad in both) -> down
In Qdrant: built-in hybrid search with RRF
In pgvector: manually via UNION + RRF calculationBi-encoder is fast, but crude - it doesn't compare query and document together. Reranker (Cross-Encoder) takes the top K candidates and evaluates each more accurately: it sees query and document simultaneously through full attention.
Source-complete notestwo-stage retrieval
pythonStage 1: fast retrieval (bi-encoder + ANN)
K = 100 candidates in ~5ms
↑ approximately, but quickly
Stage 2: reranking (cross-encoder)
for each candidate_chunk:
score = cross_encoder(query, chunk)
# query and chunk together via BERT → more precisely
top_k_reranked = sorted by score[:k]
↑ slow (100 × forward pass), but accurate
Popular rerankers:
cohere rerank-3 ← API
bge-reranker-v2-m3 ← self-hosted
ms-marco-MiniLM-L-6 ← fast, easy
NDCG@10 gain: +5–15% over bi-encoderComplete RAG Pipeline
Before searching, you need to prepare the database: upload documents, cut them into chunks, enrich them with metadata, encode them into vectors and write them into a vector database. This is an offline process that runs when documents are updated.
This is a complete breakdown of what happens when a user asks a question in the RAG system. Each step is specific: what it takes as input, what it returns, what it is compared with. This is where everything the previous cards have said comes together.
Source-complete noteswhere quality is lost
where quality is lost
- chunk is too big → embedding is blurry
- the title was not included in the embedding text
- no metadata → cannot be filtered
- no reranker → bi-encoder misses nuances
- embed_model is not trained on the domain → low recall
- hybrid + rerank → best result
No dead end
Keep moving through the map.
Continue in sequence, switch to a related guide, or return to the seven-track learning map.