Skip to content

spaCy

Industrial-strength NLP. spaCy is the go-to library for fast, production-ready text processing: efficient tokenization, POS tagging, dependency parsing, NER, and built-in transformer integration. Use it when NLTK is too slow or you need a single nlp() call that does everything.

Table of Contents

Installation / Setup

pip install spacy                        # or: uv add spacy
python -m spacy download en_core_web_sm  # small English model (~12 MB)
python -m spacy download en_core_web_md  # medium (~40 MB)
python -m spacy download en_core_web_lg  # large (~560 MB, word vectors)
Model Speed Vectors NER Best for
en_core_web_sm ★★★★★ No Basic Scripts, CI, quick tests
en_core_web_md ★★★★ 300-dim GloVe Good General-purpose projects
en_core_web_lg ★★★ 300-dim GloVe Best Best accuracy + similarity
en_core_web_trf ★★★ Transformer-based Best Transformer accuracy (needs GPU)

Core Syntax

The one-call pipeline

import spacy

nlp = spacy.load("en_core_web_sm")      # load a trained pipeline
doc = nlp("Apple is looking at buying a startup in San Francisco for $1 billion.")

for token in doc:
    print(token.text, token.pos_, token.dep_)
# Apple     PROPN nsubj
# is        AUX   aux
# looking   VERB  ROOT
# ...

The Doc / Token / Span hierarchy

doc = nlp("The quick brown fox jumps.")
# doc = Document object
# doc[0] = Token("The")
# doc[0:3] = Span("The quick brown")

# Tokens
token = doc[3]       # "jumps"
print(token.text, token.pos_, token.dep_, token.lemma_, token.is_stop)
# jumps VERB ROOT jump False

# Sentences
for sent in doc.sents:
    print(sent)
Object Contains Key attributes
Doc All tokens text, sents, ents, noun_chunks
Token One word text, pos_, tag_, dep_, head, lemma_, is_stop
Span Slice of Doc text, label_ (for entities), start, end

Common Operations

Tokenization + POS tagging

doc = nlp("We're studying NLP models — they're amazing!")
print([(t.text, t.pos_, t.tag_) for t in doc])
# [("We", "PRON", "PRP"), ("'re", "AUX", "VBP"), ("studying", "VERB", "VBG"), ...]

Key POS tags: NN noun, VB verb, JJ adjective, RB adverb, DET determiner, ADP preposition, PROPN proper noun, AUX auxiliary, CCONJ conjunction.

Dependency parsing

for token in doc:
    print(f"{token.text:15s} {token.dep_:10s} {token.head.text}")
# She             nsubj      took
# took            ROOT       took
# the             det        cat
# cat             dobj       took
# out             prt        took

Useful for: finding the root verb of a sentence, extracting subject–object pairs, building knowledge graphs.

Named Entity Recognition (NER)

doc = nlp("Elon Musk founded SpaceX in Hawthorne, California in 2002.")

for ent in doc.ents:
    print(ent.text, ent.label_, ent.start_char, ent.end_char)
# Elon Musk   PERSON       0      9
# SpaceX       ORG          18     24
# Hawthorne    GPE          28     37
# California   GPE          39     49
# 2002         DATE         54     58
Label Meaning Example
PERSON Person Elon Musk
ORG Organization SpaceX, Google
GPE Geopolitical entity California, France
DATE Date / time January 5, 2023
MONEY Monetary value $1 billion
PRODUCT Product iPhone
EVENT Event World Cup
NORP Nationality / religious American, Buddhist 
# extract as structured data
ents = [{"text": e.text, "label": e.label_} for e in doc.ents]

Text similarity

doc1 = nlp("The cat sat on the mat")
doc2 = nlp("A kitten was sitting on a rug")
print(doc1.similarity(doc2))     # cosine of word vectors → 0.82 (example)

Requires a model with word vectors (en_core_web_md or lg). The sm model has no vectors.

Rule-based matching (Matcher)

from spacy.matcher import Matcher

matcher = Matcher(nlp.vocab)
matcher.add("TECH", [[{"LOWER": "python"}, {"LOWER": "?", "OP": "?"},
                       {"POS": "NOUN"}]])
matcher.add("ORG_WITH_SPACE", [[{"ENT_TYPE": "ORG"}, {"TEXT": "Inc", "OP": "?"}]])

doc = nlp("Python is great. I use Python every day at OpenAI Inc.")
for match_id, start, end in matcher(doc):
    print(doc[start:end].text)
# Python
# Python
# OpenAI Inc

Matcher patterns use token attributes (LOWER, POS, ENT_TYPE, TEXT, etc.) — powerful for extracting recurring phrases that NER misses.

PhraseMatcher (exact phrases)

from spacy.matcher import PhraseMatcher

phrase_matcher = PhraseMatcher(nlp.vocab, attr="LOWER")
phrases = ["machine learning", "deep learning", "neural network"]
phrase_matcher.add("TECH", [nlp(p) for p in phrases])

doc = nlp("I study machine learning and deep learning daily.")
for _, start, end in phrase_matcher(doc):
    print(doc[start:end].text)
# machine learning
# deep learning

Text classification with spaCy

# spaCy v3 uses textcat component in a pipeline
# Train from scratch with thinc or load from a package
from spacy.tokens import DocBin

# Using a pre-trained textcat model (e.g. en_core_web_sm has one)
doc = nlp("This movie was absolutely fantastic and brilliantly acted!")
print(doc.cats)    # {'POSITIVE': 0.98, 'NEGATIVE': 0.02}

For training your own: use spacy.TextCategorizer or the v3 textcat / textcat_multilabel component with spacy train.

Practical Examples

Example 1 — Extract key facts from a news article

import spacy

nlp = spacy.load("en_core_web_lg")
doc = nlp(open("article.txt").read())

# people, organizations, dates
print("People:", [e.text for e in doc.ents if e.label_ == "PERSON"])
print("Orgs:",   [e.text for e in doc.ents if e.label_ == "ORG"])
print("Dates:",  [e.text for e in doc.ents if e.label_ == "DATE"])

# subject-verb-object triples
for sent in doc.sents:
    root = [t for t in sent if t.dep_ == "ROOT"]
    if root:
        subj = [t.text for t in root[0].lefts if t.dep_ in ("nsubj", "nsubjpass")]
        obj = [t.text for t in root[0].rights if t.dep_ in ("dobj", "attr", "pobj")]
        if subj and obj:
            print(f"{subj[0]} -> {root[0].lemma_} -> {obj[0]}")

Example 2 — Keyword extraction using noun chunks

from collections import Counter

doc = nlp("""
Natural language processing enables machines to understand human language.
Text classification and sentiment analysis are common NLP tasks.
Deep learning has improved NLP performance dramatically.
""")
# filter out stopwords/short chunks
chunks = [chunk.lemma_.lower() for chunk in doc.noun_chunks
          if len(chunk) > 2 and not chunk.root.is_stop]
freq = Counter(chunks).most_common(5)
print(freq)
# [('natural language processing', 1), ('human language', 1), ...]

Example 3 — Batch-process a CSV with multiprocessing

import spacy
import pandas as pd
from spacy.language import Language

nlp = spacy.load("en_core_web_sm")

def extract_entities(text):
    doc = nlp(text)
    return {e.label_: list({e.text for e in doc.ents if e.label_ == e.label_})
            for e in doc.ents}

# spacy.pipe is faster than individual calls
texts = df["text"].tolist()
entities = list(nlp.pipe(texts, batch_size=100))
df["ents"] = [dict(Counter(e.label_ for e in doc.ents)) for doc in entities]

nlp.pipe() batches processing — much faster than calling nlp(text) in a loop on large datasets.

Common Patterns

Pattern: a production NLP pipeline

import spacy
from spacy.matcher import Matcher

nlp = spacy.load("en_core_web_lg")

def process(texts):
    for doc in nlp.pipe(texts, batch_size=200, n_process=-1):
        yield {
            "text": doc.text,
            "tokens": [t.lemma_ for t in doc if t.is_alpha and not t.is_stop],
            "entities": [{"text": e.text, "label": e.label_} for e in doc.ents],
            "noun_chunks": [chunk.lemma_ for chunk in doc.noun_chunks],
        }

Pattern: add custom components to a pipeline

@Language.component("custom_lowercase")
def custom_lowercase(doc):
    for token in doc:
        token._.lower_ = token.text.lower()
    return doc

nlp.add_pipe("custom_lowercase", after="tok2vec")

Pattern: efficient similarity search across documents

import numpy as np

docs = list(nlp.pipe(texts))
doc_vectors = np.array([d.vector for d in docs])    # averaged word vectors

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-10)

# find the most similar document to a query
query_vec = nlp("deep learning").vector
sims = [cosine(query_vec, v) for v in doc_vectors]
ranked = sorted(enumerate(sims), key=lambda x: x[1], reverse=True)

Best Practices

  • Use nlp.pipe() for batch processing — it's dramatically faster than calling nlp(text) in a loop.
  • Choose the right model: sm for scripts, md for similarity, lg for best accuracy, trf for SOTA.
  • Always check for word vectors before using doc.similarity() — sm has no vectors.
  • Matcher patterns are more maintainable than regex for structured text extraction.
  • For named entities, the built-in NER is decent but domain-specific (medical, legal) may need custom models.
  • Disable components you don't need for speed: nlp = spacy.load("en_core_web_sm", disable=["parser", "ner"])
  • Save custom models with nlp.to_disk().
  • spaCy v3 uses config files (config.cfg) for training — avoid hard-coding hyperparameters.

Common Mistakes

Mistake Fix
Using doc.text for lemmatized output Use t.lemma_ for each token
Tokenizing with doc.split() instead of nlp() nlp(text) handles tokenizer + POS + NER in one call
Assuming sm model has word vectors Use md or lg for doc.similarity()
Forgetting nlp.pipe() and calling nlp() in a loop nlp.pipe(batch_size=200) is 5-10× faster
Matcher returns wrong matches Check token attributes: use LOWER for case-insensitive, POS for grammar
Importing spacy before downloading a model Download first: python -m spacy download en_core_web_sm
Expecting spaCy to outperform transformers on accuracy spaCy is fast; for best accuracy use Hugging Face transformers

Troubleshooting

Symptom Cause / Fix
OSError: Can't find model 'en_core_web_sm' python -m spacy download en_core_web_sm
doc.similarity returns weird values Model lacks word vectors — use en_core_web_md/lg
Slow nlp() on single texts Batch with nlp.pipe() or disable unused components
NER misses domain entities Train a custom NER model (e.g. spacy train)
Matcher too broad/too narrow Refine patterns with POS, DEP, LIKE_URL, etc.
AttributeError: 'Doc' object has no attribute 'ents' NER component disabled — re-enable or use disable= correctly
Memory blow-up with nlp.pipe() on huge corpus Process in smaller batches, use streaming from disk

Official Documentation

Last Updated

2026-09-06 — Initial version.