Chunking

Chunking splits documents into retrievable segments. The right chunk size balances context (too small loses meaning) and precision (too large retrieves irrelevant text). This is the most underrated step in RAG — bad chunking breaks retrieval.

Table of Contents

• Installation / Setup
• Core Syntax
• Common Operations
• Practical Examples
• Common Patterns
• Best Practices
• Common Mistakes
• Troubleshooting
• Related Cheat Sheets
• Official Documentation

Installation / Setup

pip install langchain langchain-text-splitters    # or: uv add langchain-text-splitters

Core Syntax

Chunking strategies

Strategy	How it works	Best for
Fixed-size	Split every N characters/tokens	Simple, predictable
Recursive character	Split on \n\n → \n → . →	General purpose (best default)
Semantic	Split when meaning changes	High quality, slower
Document-aware	Respect headers, code blocks, tables	Structured documents
Markdown-aware	Split on markdown headers	Markdown files
Code-aware	Split on function/class boundaries	Code files

Recursive character splitter (recommended default)

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,        # max characters per chunk
    chunk_overlap=200,      # overlap between chunks
    separators=["\n\n", "\n", ". ", " ", ""],   # split priority
    length_function=len,    # or use tiktoken for token counting
)

chunks = splitter.split_text(document)

Token-based splitting

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.callbacks import get_openai_callback

# split by tokens (more accurate for LLM context windows)
splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
    encoding_name="cl100k_base",
    chunk_size=512,         # tokens, not characters
    chunk_overlap=50,
)

Markdown-aware splitting

from langchain.text_splitter import MarkdownTextSplitter

splitter = MarkdownTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
)
# respects ## headers as split points

Code-aware splitting

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_text_splitters import Language

splitter = RecursiveCharacterTextSplitter.from_language(
    language=Language.PYTHON,
    chunk_size=1000,
    chunk_overlap=200,
)
# splits on function/class boundaries

Common Operations

Chunk size decision table

Use case	Chunk size (tokens)	Overlap	Why
Q&A over docs	500–1000	100–200	Balance context and precision
Summarization	1000–2000	200	Need more context per chunk
Code retrieval	200–500	50	Functions should stay together
Legal/contracts	500–800	100	Preserve clause boundaries
Chat messages	100–300	20	Short, focused context

Overlap rules

chunk_overlap should be 10–20% of chunk_size

chunk_size=1000 → chunk_overlap=100–200
chunk_size=500  → chunk_overlap=50–100
chunk_size=250  → chunk_overlap=25–50

Overlap prevents: - Important context split across chunk boundaries - Losing the "connection" between adjacent chunks

Document-type chunking

# PDF with sections
from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

docs = PyPDFLoader("paper.pdf").load()
splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200,
    separators=["\n\n\n", "\n\n", "\n", ". "],  # respect double newlines
)
chunks = splitter.split_documents(docs)

# Markdown with headers
from langchain.text_splitter import MarkdownTextSplitter

splitter = MarkdownTextSplitter(chunk_size=1500, chunk_overlap=200)
chunks = splitter.split_text(markdown_text)

Practical Examples

Example 1 — Compare chunk sizes

from langchain.text_splitter import RecursiveCharacterTextSplitter

text = open("document.md").read()
for size in [256, 512, 1000, 2000]:
    splitter = RecursiveCharacterTextSplitter(chunk_size=size, chunk_overlap=size//10)
    chunks = splitter.split_text(text)
    print(f"chunk_size={size}: {len(chunks)} chunks, avg {sum(len(c) for c in chunks)//len(chunks)} chars")

Example 2 — Chunk with metadata preservation

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_documents(documents)

# add chunk index metadata
for i, chunk in enumerate(chunks):
    chunk.metadata["chunk_index"] = i
    chunk.metadata["total_chunks"] = len(chunks)

Example 3 — Semantic chunking

# semantic chunking splits when meaning changes
# (requires an embedding model)
from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings

splitter = SemanticChunker(
    OpenAIEmbeddings(),
    breakpoint_threshold_type="percentile",
    breakpoint_percentile_threshold=85,
)
chunks = splitter.split_text(document)

Common Patterns

Pattern: hierarchical chunking

# keep parent document for context, retrieve smaller chunks
from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
from langchain.text_splitter import RecursiveCharacterTextSplitter

parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=200)
child_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)

retriever = ParentDocumentRetriever(
    vectorstore=vectorstore,
    docstore=InMemoryStore(),
    child_splitter=child_splitter,
    parent_splitter=parent_splitter,
)
retriever.add_documents(documents)
# retrieves small chunks but returns full parent documents

Pattern: test chunking quality

def evaluate_chunks(chunks, sample_queries):
    """Check if chunks contain answerable information."""
    for q in sample_queries:
        found = False
        for c in chunks:
            if any(word in c.page_content.lower() for word in q.lower().split()):
                found = True
                break
        print(f"{'✓' if found else '✗'} {q}")

Best Practices

• Recursive character splitter is the best default — handles most document types.
• Chunk size 500–1000 tokens is the sweet spot for most RAG systems.
• Overlap 10–20% prevents context loss at boundaries.
• Test with real queries — chunk quality is measured by retrieval success.
• Use document-aware splitting when structure matters (headers, code, tables).
• Token-based splitting is more accurate than character-based for LLM context.
• Metadata preservation — keep source, section, and page info in each chunk.
• Hierarchical chunking — retrieve small chunks, return full context.

Common Mistakes

Mistake	Fix
chunk_size too large (>2000 tokens)	Reduces retrieval precision — use 500–1000
chunk_size too small (<200 tokens)	Loses context — chunks become meaningless
No overlap	Add 10–20% overlap to prevent context loss
Ignoring document structure	Use Markdown/code-aware splitters
Same chunk_size for all documents	Tune per document type
Not testing with real queries	Validate chunk quality against sample questions

Troubleshooting

Symptom	Cause / Fix
Chunks too small	Increase chunk_size
Chunks cut mid-sentence	Use document-aware splitter
Poor retrieval quality	Chunk size may be wrong — test different sizes
Too many chunks	Increase chunk_size or reduce document count
Memory issues	Process documents in batches

Related Cheat Sheets

• rag.md — the complete RAG pipeline
• embeddings.md — what to do after chunking
• retrieval.md — searching chunks
• langchain — text splitters
• nlp-preprocessing — text cleaning

Official Documentation

• LangChain text splitters — https://python.langchain.com/docs/how_to/#text-splitting
• Semantic chunking — https://github.com/FullStackRetrieval/com RetrievalTutorial

Last Updated

2026-09-06 — Initial version.