RAG Pipeline¶

RAG (Retrieval-Augmented Generation) grounds LLM answers in your data. Instead of relying on the model's training knowledge, you retrieve relevant documents and inject them into the prompt. This sheet covers the complete pipeline end-to-end.

Table of Contents¶

Installation / Setup
Core Syntax
Common Operations
Practical Examples
Common Patterns
Best Practices
Common Mistakes
Troubleshooting
Related Cheat Sheets
Official Documentation

Installation / Setup¶

pip install langchain langchain-openai langchain-community faiss-cpu
# or: uv add langchain langchain-openai faiss-cpu

Core Syntax¶

The complete RAG pipeline¶

Documents
  → Loading        (PDF, HTML, DB, API)
  → Cleaning       (strip HTML, normalize whitespace)
  → Chunking       (split into overlapping segments)
  → Embeddings     (convert text → vectors)
  → Vector Store   (index vectors for fast search)
  → Retrieval      (find relevant chunks for a query)
  → Reranking      (reorder by relevance to query)
  → Context        (assemble top-K chunks into prompt)
  → LLM            (generate answer from context)
  → Answer         (return grounded response)
  → Evaluation     (measure quality: faithfulness, relevance)

Each step has its own cheat sheet — see Related Cheat Sheets.

Minimal RAG (3 lines)¶

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

# 1. Index documents
vectorstore = FAISS.from_texts(
    ["Python is great for ML", "JavaScript runs in browsers"],
    OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

# 2. Build RAG chain
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer based on context:\n{context}"),
    ("human", "{question}"),
])

def format_docs(docs):
    return "\n\n".join(d.page_content for d in docs)

rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt | ChatOpenAI(model="gpt-4o") | StrOutputParser()
)

# 3. Query
answer = rag_chain.invoke("What is Python used for?")

Common Operations¶

Pipeline decision table¶

Step	Options	Decision
Loading	PDF, HTML, Markdown, DB, API	Match your source format
Cleaning	Strip HTML, normalize, dedup	Always clean; extent depends on quality
Chunking	Fixed-size, recursive, semantic	Recursive is best default
Embeddings	OpenAI, Cohere, sentence-transformers	OpenAI for quality; local for privacy
Vector Store	FAISS, Chroma, Qdrant, Pinecone	FAISS for local; cloud for scale
Retrieval	Similarity, MMR, hybrid	Similarity default; MMR for diversity
Reranking	Cross-encoder, Cohere, ColBERT	Add for quality; skip for speed
Context	Top-K chunks, max tokens	K=3–5; watch token budget
LLM	GPT-4o, Claude, Gemini	Match quality needs and budget

Quality vs speed trade-offs¶

Approach	Quality	Speed	Cost
Simple RAG (no reranking)	Good	Fast	Low
RAG + reranking	Better	Slower	Medium
RAG + hybrid search + reranking	Best	Slowest	High
Fine-tuned embeddings	Better	Same	Higher

Practical Examples¶

Example 1 — Full RAG over PDFs¶

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

# Load
loader = PyPDFLoader("report.pdf")
documents = loader.load()

# Chunk
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_documents(documents)

# Embed + Index
vectorstore = FAISS.from_documents(chunks, OpenAIEmbeddings())
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

# RAG chain
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer the question using only the provided context.\n\nContext:\n{context}"),
    ("human", "{question}"),
])

rag = (
    {"context": retriever | (lambda docs: "\n\n".join(d.page_content for d in docs)),
     "question": RunnablePassthrough()}
    | prompt | ChatOpenAI(model="gpt-4o") | StrOutputParser()
)

answer = rag.invoke("What are the key findings?")

Example 2 — RAG with metadata filtering¶

# add metadata during loading
for doc in documents:
    doc.metadata["source"] = "report.pdf"
    doc.metadata["year"] = "2025"

# filter during retrieval
retriever = vectorstore.as_retriever(
    search_kwargs={"k": 5, "filter": {"year": "2025"}}
)

Example 3 — RAG with source citations¶

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", """Answer based on context. Cite sources.
Context:
{context}"""),
    ("human", "{question}"),
])

rag = (
    {"context": retriever | format_docs_with_sources, "question": RunnablePassthrough()}
    | prompt | ChatOpenAI(model="gpt-4o") | StrOutputParser()
)

def format_docs_with_sources(docs):
    parts = []
    for i, doc in enumerate(docs):
        source = doc.metadata.get("source", "unknown")
        parts.append(f"[{i+1}] ({source})\n{doc.page_content}")
    return "\n\n".join(parts)

Common Patterns¶

Pattern: query transformation¶

# HyDE: generate a hypothetical answer, then retrieve
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

def hyde_transform(question):
    response = llm.invoke(f"Write a detailed answer to: {question}")
    return response.content

# use as a RunnableLambda before retrieval
hyde_retriever = RunnableLambda(hyde_transform) | vectorstore.as_retriever()

Pattern: multi-query RAG¶

def generate_queries(question, llm):
    response = llm.invoke(f"Generate 3 diverse search queries for: {question}")
    return response.content.split("\n")

# retrieve for each query, deduplicate, then answer

Pattern: self-RAG (conditional retrieval)¶

def should_retrieve(question, llm):
    response = llm.invoke(f"Is this question answerable from documents? yes/no:\n{question}")
    return "yes" in response.content.lower()

if should_retrieve(question, llm):
    context = retriever.invoke(question)
else:
    context = ""

Pattern: save and load vector store¶

# save
vectorstore.save_local("my_index")

# load
vectorstore = FAISS.load_local("my_index", OpenAIEmbeddings())

Best Practices¶

Always clean and chunk — don't embed raw documents.
Chunk size matters — 500–1000 tokens is the sweet spot for most use cases.
Overlap prevents context loss — use 10–20% overlap between chunks.
Start simple — basic RAG (similarity + top-3) works surprisingly well.
Add reranking only when you need better quality — it adds latency.
Use metadata — source, date, category enable filtering and citations.
Evaluate — faithfulness (is the answer grounded?) and relevance (did we retrieve the right docs?) are the two key metrics.
Cache embeddings — don't re-embed the same documents.
Watch token budgets — too many retrieved chunks exceed context windows.

Common Mistakes¶

Mistake	Fix
Embedding entire documents	Chunk into 500–1000 token segments
No chunk overlap	Add 100–200 token overlap
Using the wrong embedding model	Match model to your domain and language
Top-K too small (K=1)	Use K=3–5 for better coverage
No metadata for filtering	Add source/date/category during loading
Ignoring chunk quality	Clean HTML, normalize whitespace, deduplicate
Not evaluating	Track faithfulness and retrieval relevance
Using GPT-4o for everything	Use GPT-4o-mini for simple Q&A

Troubleshooting¶

Symptom	Cause / Fix
Answers don't match documents	Chunking too aggressive or wrong retrieval
"I don't know" responses	Context not relevant — check retrieval quality
Slow retrieval	Index is too large — add filtering or use ANN
High token usage	Reduce top-K or use smaller chunks
Missing relevant docs	Chunk size too large — documents split poorly
Wrong answers despite good retrieval	LLM ignoring context — improve prompt

embeddings.md — embedding models and dimensions
chunking.md — splitting strategies
retrieval.md — retrieval methods
reranking.md — reranking models
rag-evaluation.md — evaluation metrics
faiss.md — local vector store
chroma.md — developer-friendly vector DB
qdrant.md — production vector DB
pinecone.md — managed vector DB
weaviate.md — vector DB with hybrid search
milvus.md — high-performance vector DB
langchain — LLM framework
llamaindex — data framework for RAG
sentence-transformers — open embeddings

Official Documentation¶

LangChain RAG guide — https://python.langchain.com/docs/tutorials/rag/
LlamaIndex RAG — https://docs.llamaindex.ai/en/stable/getting_started/starter_example/

Last Updated¶

2026-09-06 — Initial version.

RAG Pipeline¶

Table of Contents¶

Installation / Setup¶

Core Syntax¶

The complete RAG pipeline¶

Minimal RAG (3 lines)¶

Common Operations¶

Pipeline decision table¶

Quality vs speed trade-offs¶

Practical Examples¶

Example 1 — Full RAG over PDFs¶

Example 2 — RAG with metadata filtering¶

Example 3 — RAG with source citations¶

Common Patterns¶

Pattern: query transformation¶

Pattern: multi-query RAG¶

Pattern: self-RAG (conditional retrieval)¶

Pattern: save and load vector store¶

Best Practices¶

Common Mistakes¶

Troubleshooting¶

Related Cheat Sheets¶

Official Documentation¶

Last Updated¶