Skip to content

FAISS

FAISS (Facebook AI Similarity Search) is the gold standard for fast local vector similarity search. It's not a database — it's a C++ library with Python bindings that stores vectors in memory and searches them at millions of vectors per second. Use FAISS when you need fast, local vector search without the overhead of a database server.

Table of Contents

Installation / Setup

pip install faiss-cpu                # CPU-only (most common)
pip install faiss-gpu                # GPU-accelerated
# or: uv add faiss-cpu

Core Syntax

Index types

Index Type Speed Memory Best for
IndexFlatL2 Brute force ★★★ ★★★★★ <100K vectors
IndexFlatIP Brute force (inner product) ★★★ ★★★★★ <100K, cosine sim
IndexIVFFlat Inverted index ★★★★★ ★★★★ 100K–10M vectors
IndexIVFPQ Inverted + product quantization ★★★★★ ★★★ >10M vectors
IndexHNSW Graph-based ★★★★★ ★★★ Fast, good quality

Basic FAISS usage

import faiss
import numpy as np

# create index
dimension = 1536   # embedding dimension
index = faiss.IndexFlatL2(dimension)

# add vectors
vectors = np.random.randn(1000, dimension).astype("float32")
index.add(vectors)

# search
query = np.random.randn(1, dimension).astype("float32")
distances, indices = index.search(query, k=5)
print(indices[0])    # indices of nearest vectors
print(distances[0])  # distances (L2)

Inner product index (cosine similarity)

# normalize vectors for cosine similarity
faiss.normalize_L2(vectors)

index = faiss.IndexFlatIP(dimension)
index.add(vectors)
# search returns inner product (1.0 = identical for normalized vectors)

LangChain + FAISS

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

# create from documents
vectorstore = FAISS.from_documents(chunks, OpenAIEmbeddings())

# search
docs = vectorstore.similarity_search("query", k=5)

# save/load
vectorstore.save_local("faiss_index")
vectorstore = FAISS.load_local("faiss_index", OpenAIEmbeddings())

IVF index (for large datasets)

import faiss

dimension = 1536
nlist = 100           # number of clusters
quantizer = faiss.IndexFlatL2(dimension)
index = faiss.IndexIVFFlat(quantizer, dimension, nlist)

# train on your data (required for IVF)
vectors = np.random.randn(10000, dimension).astype("float32")
index.train(vectors)
index.add(vectors)

# search (use nprobe to control accuracy/speed)
index.nprobe = 10    # search 10 clusters
distances, indices = index.search(query, k=5)

Common Operations

Persistence

# save to disk
faiss.write_index(index, "my_index.faiss")

# load from disk
index = faiss.read_index("my_index.faiss")

Adding vectors incrementally

# add new vectors to existing index
new_vectors = np.random.randn(100, dimension).astype("float32")
index.add(new_vectors)

GPU acceleration

import faiss

# move index to GPU
res = faiss.StandardGpuResources()
gpu_index = faiss.index_cpu_to_gpu(res, 0, index)

# search on GPU
distances, indices = gpu_index.search(query, k=5)

With metadata (via LangChain)

# LangChain FAISS supports metadata
vectorstore = FAISS.from_documents(
    chunks,
    OpenAIEmbeddings(),
    metadatas=[{"source": "doc1"}, {"source": "doc2"}]
)

# filter by metadata
docs = vectorstore.similarity_search("query", k=5, filter={"source": "doc1"})

Practical Examples

Example 1 — Build a semantic search engine

import faiss
import numpy as np
from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY")

def embed(texts):
    response = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return np.array([e.embedding for e in response.data]).astype("float32")

# index documents
documents = ["Python is great for ML", "JavaScript runs in browsers", "R is for statistics"]
doc_vectors = embed(documents)

index = faiss.IndexFlatIP(1536)
faiss.normalize_L2(doc_vectors)
index.add(doc_vectors)

# search
query_vector = embed(["data analysis tools"])[0].reshape(1, -1)
faiss.normalize_L2(query_vector)
distances, indices = index.search(query_vector, k=2)
print([documents[i] for i in indices[0]])

Example 2 — LangChain FAISS RAG

from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

# build index
docs = DirectoryLoader("./docs", glob="**/*.md", loader_cls=TextLoader).load()
chunks = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50).split_documents(docs)
vectorstore = FAISS.from_documents(chunks, OpenAIEmbeddings())
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

# RAG chain
rag = (
    {"context": retriever | (lambda d: "\n\n".join(x.page_content for x in d)),
     "question": RunnablePassthrough()}
    | ChatPromptTemplate.from_template("Answer:\n{context}\n\nQ: {question}")
    | ChatOpenAI(model="gpt-4o")
    | StrOutputParser()
)

Common Patterns

Pattern: combine multiple FAISS indices

# merge two indices
index1 = faiss.read_index("index1.faiss")
index2 = faiss.read_index("index2.faiss")
index1.add(index2.reconstruct_n(0, index2.ntotal))

Pattern: filter after retrieval

# FAISS doesn't support metadata filtering natively
# retrieve more, filter in Python
docs_with_scores = vectorstore.similarity_search_with_score(query, k=20)
filtered = [(doc, score) for doc, score in docs_with_scores
            if doc.metadata.get("source") == "desired_source"]
top_docs = [doc for doc, _ in filtered[:5]]

Best Practices

  • Use IndexFlatL2 for <100K vectors (simple, exact).
  • Use IndexIVFFlat for 100K–10M vectors (fast, approximate).
  • Normalize vectors for cosine similarity (IndexFlatIP).
  • Save to disk after building — don't rebuild every time.
  • Use LangChain FAISS wrapper for easy integration with RAG.
  • GPU acceleration helps for >1M vectors.

Common Mistakes

Mistake Fix
Not normalizing for cosine similarity Use faiss.normalize_L2() + IndexFlatIP
Using IVF without training IVF indexes must be trained on data first
No persistence Save with faiss.write_index()
Trying to filter by metadata Filter after retrieval; FAISS doesn't support metadata filters

Troubleshooting

Symptom Cause / Fix
Dimension mismatch Query and index dimensions must match
Poor search quality Use IndexFlatIP for cosine; check normalization
Slow indexing Use IVF for large datasets; requires training
OOM on large datasets Use IndexIVFPQ for compression
No metadata filtering Filter results in Python after search

Official Documentation

Last Updated

2026-09-06 — Initial version.