Skip to content

Sentence Transformers

Turn any sentence into a dense vector for similarity, search, and clustering. Sentence Transformers builds on Hugging Face models to produce high-quality sentence embeddings — the foundation of modern semantic search, retrieval-augmented generation (RAG), and document clustering.

Table of Contents

Installation / Setup

pip install sentence-transformers         # or: uv add sentence-transformers
Model Dimensions Speed Best for
all-MiniLM-L6-v2 384 ★★★★★ Fast, good general-purpose
all-mpnet-base-v2 768 ★★★★ Best quality/speed trade-off
multi-qa-MiniLM-L6-cos-v1 384 ★★★★★ Semantic search/QA
all-MiniLM-L12-v2 384 ★★★★ Better accuracy than L6
BAAI/bge-large-en-v1.5 1024 ★★★ High quality search
BAAI/bge-small-en-v1.5 384 ★★★★★ Fast, good quality
paraphrase-mpnet-base-v2 768 ★★★★ Paraphrase detection

Core Syntax

Encode sentences

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

# single sentence
vec = model.encode("Python is great for data science")
print(vec.shape)    # (384,)

# batch
sentences = ["I love cats", "Dogs are loyal", "Python is powerful"]
embeddings = model.encode(sentences, batch_size=32, show_progress_bar=True)
print(embeddings.shape)  # (3, 384)

Compute similarity

from sentence_transformers import util

a = model.encode("The cat sat on the mat")
b = model.encode("A kitten was sitting on a rug")
print(util.cos_sim(a, b).item())  # ~0.75 (example)

Or using numpy directly:

import numpy as np

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-10)

score = cosine(model.encode("hello"), model.encode("hi"))

Common Operations

Semantic search — find the most similar document

import numpy as np
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")

documents = [
    "Python is a versatile programming language",
    "JavaScript runs in web browsers",
    "Machine learning requires large datasets",
    "Deep learning is a subset of machine learning",
    "CSS styles HTML documents",
]

doc_embeddings = model.encode(documents)

def search(query, top_k=3):
    q_emb = model.encode(query)
    scores = util.cos_sim(q_emb, doc_embeddings)[0]
    top_indices = scores.argsort(descending=True)[:top_k]
    return [(documents[i], scores[i].item()) for i in top_indices]

results = search("artificial intelligence training data")
for doc, score in results:
    print(f"{score:.3f} | {doc}")

Find duplicates or similar pairs

def find_duplicates(texts, threshold=0.85):
    embeddings = model.encode(texts)
    sim_matrix = util.cos_sim(embeddings)
    pairs = []
    for i in range(len(texts)):
        for j in range(i + 1, len(texts)):
            if sim_matrix[i][j] > threshold:
                pairs.append((i, j, sim_matrix[i][j].item()))
    return pairs

dups = find_duplicates([
    "Python is great for ML",
    "Python excels at machine learning tasks",
    "JavaScript is used for web development",
    "JS powers the web frontend",
])
print(dups)
# [(0, 1, 0.92), (2, 3, 0.78)]  (example)

Clustering documents

from sklearn.cluster import KMeans
from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = model.encode(documents)

kmeans = KMeans(n_clusters=3, random_state=0, n_init=10).fit(embeddings)
labels = kmeans.labels_
for doc, label in zip(documents, labels):
    print(f"Cluster {label}: {doc[:60]}")

Cross-encoder re-ranking

from sentence_transformers import CrossEncoder

cross_encoder = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

query = "how does machine learning work"
# candidate documents (from bi-encoder search)
candidates = [doc for doc, _ in search(query, top_k=10)]

pairs = [(query, doc) for doc in candidates]
scores = cross_encoder.predict(pairs)
ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)

for doc, score in ranked[:3]:
    print(f"{score:.3f} | {doc}")

Two-stage search: bi-encoder retrieves top-k candidates quickly, cross-encoder re-ranks accurately.

Practical Examples

Example 1 — Build a semantic search index

import faiss
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = model.encode(documents, normalize_embeddings=True)

index = faiss.IndexFlatIP(embeddings.shape[1])    # inner product on normalized = cosine
index.add(embeddings.astype("float32"))

def search_faiss(query, top_k=5):
    q = model.encode([query], normalize_embeddings=True).astype("float32")
    scores, indices = index.search(q, top_k)
    return [(documents[i], float(s)) for s, i in zip(scores[0], indices[0])]

results = search_faiss("data science tools")

Example 2 — Compare two documents

from sentence_transformers import util

doc1 = open("resume_alice.txt").read()
doc2 = open("resume_bob.txt").read()

emb1 = model.encode(doc1)
emb2 = model.encode(doc2)
similarity = util.cos_sim(emb1, emb2).item()
print(f"Similarity: {similarity:.3f}")

Example 3 — Map sentences to clusters and summarize

from sklearn.cluster import AgglomerativeClustering
import numpy as np

embeddings = model.encode(documents)
clustering = AgglomerativeClustering(n_clusters=None, distance_threshold=0.5).fit(embeddings)

clusters = {}
for doc, label in zip(documents, clustering.labels_):
    clusters.setdefault(label, []).append(doc)

for label, docs in clusters.items():
    print(f"\nCluster {label} ({len(docs)} docs):")
    print("  " + docs[0][:80])

Common Patterns

Pattern: sentence-to-sentence similarity matrix

import torch
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
sentences = ["A cat", "A kitten", "A dog", "A puppy"]
embs = model.encode(sentences, convert_to_tensor=True)

sim_matrix = util.cos_sim(embs, embs)  # symmetric matrix
print(sim_matrix)

Pattern: encoding with metadata (for RAG)

def index_documents(docs, model):
    texts = [d["text"] for d in docs]
    embeddings = model.encode(texts, batch_size=64)
    return {"embeddings": embeddings, "documents": docs}

index = index_documents(large_corpus, model)
# embeddings[i] corresponds to index["documents"][i]

Pattern: batch processing with progress

embeddings = model.encode(
    documents,
    batch_size=256,
    show_progress_bar=True,
    convert_to_numpy=True,     # return numpy array, not torch
    normalize_embeddings=True,  # cosine-ready immediately
)

Best Practices

  • all-MiniLM-L6-v2 is the default recommendation: fast, 384-dim, good for most tasks.
  • all-mpnet-base-v2 when quality matters more than speed.
  • Always normalize embeddings if using inner-product similarity (normalize_embeddings=True).
  • For search, use FAISS (faiss) or hnswlib for fast approximate nearest neighbor — not brute-force cosine.
  • Use bi-encoder for retrieval (fast), cross-encoder for re-ranking (accurate).
  • Batch encoding with model.encode(list, batch_size=256) is fast; avoid per-sentence calls.
  • For multilingual search, use paraphrase-multilingual-MiniLM-L12-v2.
  • Fine-tune with SentenceTransformer + contrastive loss on domain data for best retrieval quality.

Common Mistakes

Mistake Fix
Using cosine similarity on un-normalized embeddings Normalize with normalize_embeddings=True or manual L2 normalization
Comparing embeddings from different models Always use the same model for all embeddings in a similarity comparison
Using model.encode() on a single string in a loop Batch with model.encode(list_of_strings, batch_size=256)
Confusing bi-encoder and cross-encoder Bi-encoder = fast retrieval; cross-encoder = accurate re-ranking
Expecting embeddings to transfer across domains Fine-tune on domain-specific data for production search
Large memory usage with FAISS Use approximate nearest neighbor (faiss.IndexIVFFlat) for large corpora
cos_sim returns a matrix, not a scalar Use [0][0] or .item() to extract a single score

Troubleshooting

Symptom Cause / Fix
ImportError: No module named 'sentence_transformers' pip install sentence-transformers
Similarity scores always near 0 Model output not normalized — add normalize_embeddings=True
CUDA out of memory Lower batch_size or move to CPU for small tasks
Semantic search returns irrelevant results Domain mismatch — fine-tune on your data or try a search-specific model
model.encode() is slow Increase batch_size, use show_progress_bar=True, check device
Cross-encoder crashes on long documents Truncate input to 512 tokens (cross-encoder limit)
faiss import error pip install faiss-cpu (CPU) or faiss-gpu (GPU)

Official Documentation

Last Updated

2026-09-06 — Initial version.