Skip to content

pytest

pytest is the de facto test runner for Python projects — discovery, fixtures, parametrize, rich assertions, and plugins make it the default choice for most data and AI projects.

Quick Reference

Feature Use for
test_*.py / *_test.py Discovery convention
@pytest.fixture Reusable setup, shared state, teardown
@pytest.mark.parametrize Run one test against many inputs
assert Plain assertions — pytest rewrites them for great tracebacks
tmp_path / tmpdir Auto-cleaned temporary filesystem
monkeypatch Safely replace attributes/functions during a test
pytest.raises Assert that a block raises a specific exception
@pytest.mark.skip / xfail Skip tests or mark expected failures
-v / -q / -k Verbose, quiet, filter by name expression
--lf / --ff Run last failed, or run failed first
conftest.py Shared fixtures and hooks per directory

Installation

pip install pytest

Optionally add useful plugins:

pip install pytest-cov pytest-mock pytest-xdist

The Smallest Test

# test_math.py
def add(a, b):
    return a + b

def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0

Run:

pytest              # discover and run
pytest -v           # verbose
pytest test_math.py # one file
pytest -k add       # filter by name

pytest discovers tests by naming convention and runs any function starting with test_.

Assertions

pytest uses plain assert. When an assertion fails, pytest rewrites the failure message to show the values that mattered.

def test_inventory():
    stock = {"apples": 5, "oranges": 2}
    assert stock["apples"] == 5
    assert "bananas" not in stock
    assert len(stock) >= 2

For non-scalar comparisons, pytest's output is often enough. When you need more control:

def test_approx():
    import math
    assert math.pi == pytest.approx(3.14, rel=1e-2)

Use pytest.approx for floats instead of exact equality.

Test Layout

A common layout:

project/
├── src/
│   └── analysis.py
└── tests/
    ├── __init__.py
    ├── conftest.py
    ├── test_analysis.py
    └── test_integration.py
  • conftest.py is for fixtures and hooks shared across the tests/ tree.
  • Keep one test file per module, or per feature area.
  • Prefix everything with test_.

Fixtures

Fixtures handle setup, shared state, and teardown. They are the heart of readable pytest suites.

Basic fixture

# conftest.py
import pytest

@pytest.fixture
def sample_series():
    return [10, 20, 30, 40, 50]

# test_analysis.py
def mean(values):
    return sum(values) / len(values)

def test_mean(sample_series):
    assert mean(sample_series) == 30.0

pytest injects the fixture by name. No explicit import needed.

Fixture scope

@pytest.fixture(scope="function")   # default — fresh per test
@pytest.fixture(scope="class")
@pytest.fixture(scope="module")
@pytest.fixture(scope="session")
def db_connection():
    conn = setup_db()
    yield conn
    conn.close()                     # teardown runs after the scope ends

yield lets you run teardown. The part before yield is setup; the part after is cleanup.

Fixture with teardown

@pytest.fixture
def temp_file(tmp_path):
    path = tmp_path / "data.csv"
    path.write_text("x,y\n1,2\n3,4\n")
    yield path
    # tmp_path is cleaned automatically; explicit cleanup here if needed

Parametrize

Run the same test logic over many inputs without copy-paste.

import pytest

@pytest.mark.parametrize("value,expected", [
    (5, 0, 10, 5),
    (-3, 0, 10, 0),
    (12, 0, 10, 10),
    (0, 0, 10, 0),
    (10, 0, 10, 10),
])
def test_clip(value, lo, hi, expected):
    assert clip(value, lo, hi) == expected

You can stack multiple parametrize decorators to form a Cartesian product:

@pytest.mark.parametrize("a", [1, 2])
@pytest.mark.parametrize("b", [3, 4])
def test_add(a, b):
    assert add(a, b) == a + b

This runs 4 tests: (1,3), (1,4), (2,3), (2,4).

Id for readable names

@pytest.mark.parametrize("text,expected", [
    pytest.param("", 0, id="empty"),
    pytest.param("hello", 5, id="ascii"),
    pytest.param("café", 4, id="unicode"),
])
def test_char_count(text, expected):
    assert char_count(text) == expected

Good id values make failures easier to reason about.

tmp_path and tmpdir

Never write test artifacts into the real working directory.

def test_write_csv(tmp_path):
    csv_path = tmp_path / "out.csv"
    csv_path.write_text("a,b\n1,2\n")
    assert csv_path.exists()
    assert csv_path.read_text() == "a,b\n1,2\n"

tmp_path is a pathlib.Path. tmpdir is a legacy py.path. Prefer tmp_path.

pytest.raises

Assert that code raises the expected exception.

def divide(a, b):
    return a / b

def test_divide_by_zero():
    with pytest.raises(ZeroDivisionError):
        divide(1, 0)

def test_divide_message():
    with pytest.raises(ValueError, match="negative"):
        validate_age(-5)

match accepts a regex against the exception message.

Mocking

Two good options: monkeypatch (built-in) and pytest-mock (a thin wrapper around unittest.mock).

monkeypatch

def test_read_config(monkeypatch, tmp_path):
    config_path = tmp_path / "config.ini"
    config_path.write_text("[section]\nkey=value\n")

    monkeypatch.setattr("myapp.config._load_path", lambda: config_path)
    cfg = load_config()
    assert cfg["key"] == "value"

Use monkeypatch for replacing attributes, environment variables, and sys.path during a test. It is automatically reverted after each test.

def test_env_flag(monkeypatch):
    monkeypatch.setenv("DEBUG", "1")
    assert is_debug() is True

pytest-mock

pip install pytest-mock

def test_fetch(mocker):
    mock_get = mocker.patch("requests.get")
    mock_get.return_value.status_code = 200
    mock_get.return_value.json.return_value = {"result": 42}

    data = fetch_results()
    assert data == {"result": 42}
    mock_get.assert_called_once()

mocker is a fixture provided by pytest-mock. It wraps unittest.mock and auto-cleans up.

When to mock

  • Mock I/O, network calls, and time when you want deterministic fast tests.
  • Do not mock the thing you are trying to test.
  • Prefer real small integrations over mocking everything.

Skip and xfail

import pytest
import sys

@pytest.mark.skip
def test_old_feature():
    ...

@pytest.mark.skipif(sys.version_info < (3, 10), reason="needs 3.10+")
def test_new_syntax():
    ...

@pytest.mark.xfail
def test_known_bug():
    assert False  # expected to fail

xfail marks tests that are expected to fail. If they pass, pytest reports them as XPASS, which is often a signal to remove the marker.

Markers

@pytest.mark.slow
def test_large_dataset():
    ...

@pytest.mark.integration
def test_api_roundtrip():
    ...

Run filtered subsets:

pytest -m "not slow"            # skip slow tests
pytest -m "integration"         # only integration tests
pytest -m "not slow and integration"  # combined

Register markers in pyproject.toml or pytest.ini to avoid warnings:

[tool.pytest.ini_options]
markers = [
    "slow: long-running tests",
    "integration: requires external services",
]

command line essentials

pytest                      # run everything
pytest -v                   # verbose
pytest -q                   # quiet
pytest -v --tb=short       # short tracebacks
pytest -k "normalize"       # name filter
pytest tests/test_analysis.py::test_clip_boundaries  # one test
pytest --lf                  # last failed
pytest --ff                  # failed first
pytest --collect-only       # list tests without running
pytest -x                    # stop on first failure
pytest --maxfail=2          # stop after N failures
pytest --cov=src            # coverage (with pytest-cov)
pytest -n auto              # parallel (with pytest-xdist)

conftest.py patterns

conftest.py lives alongside the tests it serves (or at a higher directory to share downward).

# tests/conftest.py
import pytest

@pytest.fixture
def clean_env(monkeypatch):
    """Reset environment variables to a known state."""
    monkeypatch.delenv("API_KEY", raising=False)
    monkeypatch.setenv("DEBUG", "0")
    yield
    # monkeypatch cleanup is automatic

Fixtures defined in conftest.py are available to all test files in that tree.

Hooks (brief)

For deeper customization, pytest exposes hooks in conftest.py or plugins:

  • pytest_configure(config) — early configuration
  • pytest_collection_modifyitems(session, config, items) — reorder or mark tests
  • pytest_runtest_makereport(item, call) — customize reporting

Most projects never need hooks. Use them when you need cross-cutting behavior.

Reading Test Output

When a test fails, pytest shows: 1. The test node id (file::test_name). 2. The line that failed. 3. A short diff for assertion rewrites. 4. The stdout/stderr captured during the test (when applicable).

For long tracebacks, --tb=short often makes failures easier to scan.

Common Mistakes

  • Shared mutable fixtures — a session-scoped mutable fixture can leak state across tests. Reset in teardown or use function scope.
  • Asserting too much in one test — hard to tell which behavior broke.
  • Mocking the unit under test — defeats the purpose.
  • Ignoring parametrize for clear cases — copy-paste tests diverge over time.
  • Using tmp_path outside the test — the directory is cleaned up; don't expect it to persist.
  • Tests that depend on order — each test should be isolatable and runnable alone.
  • Skipping without a reason — skip without reason hides intent.

Practical Example — Test a Data Cleaning Function

A small data-cleaning helper:

def clean_numeric(values, drop_none=True, fill_none=0.0):
    """Convert values to float, handling None."""
    out = []
    for v in values:
        if v is None:
            if drop_none:
                continue
            v = fill_none
        out.append(float(v))
    return out

pytest tests:

import pytest

@pytest.mark.parametrize("values,expected", [
    ([1, 2, 3], [1.0, 2.0, 3.0]),
    ([1, None, 3], [1.0, 3.0]),
    ([1, None, 3], [1.0, 0.0, 3.0],),
    ([None], []),
    ([None], [0.0]),
    (["1", "2"], [1.0, 2.0]),
])
def test_clean_numeric(values, expected, drop_none=True, fill_none=0.0):
    # parametrize can't pass fixture-like defaults directly; handle in test
    pass

A cleaner approach with one parametrize and explicit branches:

@pytest.mark.parametrize("values,expected", [
    ([1, 2, 3], [1.0, 2.0, 3.0]),
    ([1, None, 3], [1.0, 3.0]),
    (["1", "2"], [1.0, 2.0]),
])
def test_clean_numeric_drop(values, expected):
    assert clean_numeric(values, drop_none=True) == expected

def test_clean_numeric_fill():
    assert clean_numeric([1, None, 3], drop_none=False, fill_none=0.0) == [1.0, 0.0, 3.0]

def test_clean_numeric_empty():
    assert clean_numeric([]) == []

Practical Example — Test an ML-Like Transform

Say you have a simple scaler that centers and scales.

def scale(values):
    """Center to zero mean, scale to unit variance."""
    n = len(values)
    if n == 0:
        raise ValueError("empty")
    mean = sum(values) / n
    centered = [v - mean for v in values]
    var = sum(c * c for c in centered) / n
    if var == 0:
        return centered          # constant input
    scale = var ** 0.5
    return [c / scale for c in centered]

Tests:

import pytest

def test_scale_basic():
    result = scale([1, 2, 3])
    assert result == pytest.approx([-1.22474487, 0.0, 1.22474487])

def test_scale_constant():
    assert scale([5, 5, 5]) == [0.0, 0.0, 0.0]

def test_scale_empty():
    with pytest.raises(ValueError):
        scale([])

def test_scale_one_element():
    assert scale([42]) == [0.0]

This is exactly the kind of small deterministic unit test that keeps ML preprocessing code trustworthy.

Official Documentation

Last Updated

2026-09-06 — Initial version.