Retrieval¶

Retrieval is the step that finds the most relevant chunks for a query. The quality of retrieval directly determines answer quality — you can't generate a good answer from bad context. This covers similarity search, diversity, hybrid methods, and retrieval optimization.

Table of Contents¶

Installation / Setup
Core Syntax
Common Operations
Practical Examples
Common Patterns
Best Practices
Common Mistakes
Troubleshooting
Related Cheat Sheets
Official Documentation

Installation / Setup¶

pip install langchain langchain-openai faiss-cpu

Core Syntax¶

Retrieval methods¶

Method	How it works	Best for
Similarity	Cosine similarity ranking	Default, most use cases
MMR (Maximal Marginal Relevance)	Balance relevance + diversity	Avoid redundant chunks
Hybrid	Combine semantic + keyword search	Precise queries + broad coverage
Multi-query	Generate multiple queries, merge results	Complex questions
Parent document	Retrieve small chunks, return full context	When context is needed
Self-RAG	Model decides when to retrieve	When some queries don't need retrieval

Similarity search (default)¶

# basic retrieval
retriever = vectorstore.as_retriever(
    search_type="similarity",
    search_kwargs={"k": 5}
)
docs = retriever.invoke("What is machine learning?")

MMR (diversity)¶

# MMR balances relevance with diversity
retriever = vectorstore.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 5, "fetch_k": 20, "lambda_mult": 0.7}
)
# fetch_k: candidates to consider
# lambda_mult: 1.0 = pure relevance, 0.0 = pure diversity

Score-based retrieval (with thresholds)¶

# return chunks above a similarity threshold
docs = vectorstore.similarity_search_with_score(
    "query", k=10, score_threshold=0.7
)
# filter low-quality results

Hybrid search (semantic + keyword)¶

# FAISS doesn't support hybrid natively — combine manually
from langchain.retrievers import EnsembleRetriever
from langchain_community.retrievers import BM25Retriever

# keyword retriever
bm25 = BM25Retriever.from_documents(chunks)
bm25.k = 5

# semantic retriever
faiss_retriever = vectorstore.as_retriever(search_kwargs={"k": 5})

# combine with equal weights
hybrid = EnsembleRetriever(
    retrievers=[bm25, faiss_retriever],
    weights=[0.5, 0.5],
)

Multi-query retrieval¶

from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain_openai import ChatOpenAI

multi_retriever = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(),
    llm=ChatOpenAI(model="gpt-4o-mini"),
)
# generates 3 diverse queries, retrieves for each, deduplicates

Common Operations¶

Top-K selection¶

K value	Coverage	Speed	Token usage
1–2	Narrow	Fastest	Low
3–5	Balanced (default)	Fast	Medium
6–10	Broad	Slower	High
10+	Comprehensive	Slowest	Very high

Rule of thumb: K=3–5 for most use cases. Increase for complex questions; decrease for simple factoid queries.

Retrieval quality metrics¶

# measure retrieval quality
def evaluate_retrieval(retriever, queries_and_expected):
    hits = 0
    total = len(queries_and_expected)
    for query, expected_keywords in queries_and_expected:
        docs = retriever.invoke(query)
        retrieved_text = " ".join(d.page_content for d in docs).lower()
        if any(kw.lower() in retrieved_text for kw in expected_keywords):
            hits += 1
    return hits / total    # hit rate

Practical Examples¶

Example 1 — Optimized RAG with MMR¶

from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

# MMR retriever for diverse, non-redundant context
retriever = vectorstore.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 5, "fetch_k": 20, "lambda_mult": 0.7}
)

prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer based on context:\n{context}"),
    ("human", "{question}"),
])

rag = (
    {"context": retriever | (lambda d: "\n\n".join(x.page_content for x in d)),
     "question": RunnablePassthrough()}
    | prompt | ChatOpenAI(model="gpt-4o") | StrOutputParser()
)

Example 2 — Score-threshold retrieval¶

# only return high-confidence matches
results = vectorstore.similarity_search_with_score("query", k=10, score_threshold=0.75)

if not results:
    answer = "I don't have relevant information to answer this."
else:
    context = "\n\n".join(doc.page_content for doc, score in results)
    answer = llm.invoke(f"Based on:\n{context}\n\nQ: query")

Example 3 — Parent document retrieval¶

from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
from langchain.text_splitter import RecursiveCharacterTextSplitter

# small chunks for precise retrieval
child_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=30)
# large chunks for context
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=200)

retriever = ParentDocumentRetriever(
    vectorstore=vectorstore,
    docstore=InMemoryStore(),
    child_splitter=child_splitter,
    parent_splitter=parent_splitter,
)
retriever.add_documents(documents)
# retrieves small chunks but returns full parent documents

Common Patterns¶

Pattern: conditional retrieval¶

def should_retrieve(query, llm):
    """Check if query needs document retrieval."""
    response = llm.invoke(
        f"Can this be answered from general knowledge? yes/no:\n{query}")
    return "no" in response.content.lower()

Pattern: retrieval + reranking¶

# 1) retrieve many candidates
candidates = vectorstore.similarity_search(query, k=20)
# 2) rerank top results
reranked = reranker.rerank(query, candidates, top_k=5)

Pattern: cache retrieval results¶

import hashlib, json

def cached_retrieval(query, retriever, cache_file="retrieval_cache.json"):
    cache = json.load(open(cache_file)) if os.path.exists(cache_file) else {}
    key = hashlib.md5(query.encode()).hexdigest()
    if key in cache:
        return cache[key]
    results = retriever.invoke(query)
    cache[key] = [d.page_content for d in results]
    json.dump(cache, open(cache_file, "w"))
    return cache[key]

Best Practices¶

Start with similarity search, K=3 — it works well for most cases.
Use MMR when retrieved chunks are too similar to each other.
Hybrid search is the best for production — combine semantic + keyword.
Score thresholds prevent returning irrelevant results.
Multi-query helps with complex or ambiguous questions.
Parent document retrieval gives precise retrieval with full context.
Test with real queries — retrieval quality is measured by answer quality.

Common Mistakes¶

Mistake	Fix
Using K=1	Too narrow — use K=3–5 for better coverage
Not filtering by score	Add score_threshold to avoid irrelevant chunks
Ignoring diversity	Use MMR or hybrid search
Using only semantic search	Add BM25 for keyword-based queries
Same K for all queries	Tune K based on query complexity

Troubleshooting¶

Symptom	Cause / Fix
Retrieving irrelevant chunks	Embedding model mismatch or poor chunking
Missing relevant documents	K too small or wrong retrieval method
Too many redundant chunks	Use MMR for diversity
Slow retrieval	Use ANN index (FAISS, Qdrant) instead of brute force
No results above threshold	Lower threshold or improve embedding model

rag.md — the complete pipeline
embeddings.md — vector representations
reranking.md — post-retrieval reranking
chunking.md — what to retrieve
faiss.md — local vector store
chroma.md — developer vector DB

Official Documentation¶

LangChain retrievers — https://python.langchain.com/docs/how_to/#retrievers
MMR — https://python.langchain.com/docs/how_to/maximal_marginal_relevance/

Last Updated¶

2026-09-06 — Initial version.

Retrieval¶

Table of Contents¶

Installation / Setup¶

Core Syntax¶

Retrieval methods¶

Similarity search (default)¶

MMR (diversity)¶

Score-based retrieval (with thresholds)¶

Hybrid search (semantic + keyword)¶

Multi-query retrieval¶

Common Operations¶

Top-K selection¶

Retrieval quality metrics¶

Practical Examples¶

Example 1 — Optimized RAG with MMR¶

Example 2 — Score-threshold retrieval¶

Example 3 — Parent document retrieval¶

Common Patterns¶

Pattern: conditional retrieval¶

Pattern: retrieval + reranking¶

Pattern: cache retrieval results¶

Best Practices¶

Common Mistakes¶

Troubleshooting¶

Related Cheat Sheets¶

Official Documentation¶

Last Updated¶