Debugging Python

Debugging is the skill of locating the gap between what the code does and what you expected. The tools are straightforward; the craft is in forming and testing hypotheses quickly.

The Debugging Mindset

1. Reproduce — find a minimal input and steps that trigger the failure reliably.
2. Localize — narrow the failure to one function, one branch, or one data case.
3. Observe — inspect state at the right point, not everywhere at once.
4. Hypothesize — form one clear explanation.
5. Test the hypothesis — change one thing, or inspect one more value.
6. Fix and verify — the fix should make the failing case pass and not break existing behavior.

A failing test is already reproduction and localization. Use it.

Reading Tracebacks

Python tracebacks point to the failure, but the root cause can be earlier.

Traceback (most recent call last):
  File "analyze.py", line 40, in <module>
    results = process(data)
  File "analyze.py", line 18, in process
    clean = clean_numeric(rows)
  File "clean.py", line 7, in clean_numeric
    out.append(float(v))
ValueError: could not convert string to float: 'N/A'

Read bottom-up for the immediate error, then top-down for the call path. The real question is often: why did 'N/A' reach float()?

Useful habits: - Look at the last line for the exception type and message. - Walk up to the first line of your code, not library internals. - Ask what input reached the failing line.

print Debugging

print() is fast and sometimes enough for small problems.

def scale(values):
    print("scale input:", values)          # observe
    mean = sum(values) / len(values)
    print("mean:", mean)
    ...

Better form:

import logging
logging.debug("scale input=%r mean=%r", values, mean)

print is fine for a quick probe. Logging is better when you want to keep diagnostics around, filter them, or avoid leaking debug output into production.

Common print-debugging patterns: - Print the value that should not be there. - Print the type or length when the value looks plausible but is wrong. - Print a short path marker when control flow is unclear.

pdb — The Interactive Debugger

pdb lets you pause execution and inspect state interactively.

Breakpoint everywhere (Python 3.7+)

breakpoint()

When execution reaches breakpoint(), you drop into the debugger. Set PYTHONBREAKPOINT=0 to disable all breakpoints without removing them.

Step-by-step example

def find_outliers(values, threshold=2.0):
    mean = sum(values) / len(values)
    dev = [abs(v - mean) for v in values]
    return [v for v, d in zip(values, dev) if d > threshold * (sum(dev) / len(dev))]

Insert a breakpoint:

def find_outliers(values, threshold=2.0):
    breakpoint()
    mean = sum(values) / len(values)
    ...

Then run and interact:

(Pdb) values
[1, 2, 3, 100]
(Pdb) mean
26.5
(Pdb) dev
[25.5, 24.5, 23.5, 73.5]
(Pdb) sum(dev) / len(dev)
36.75
(Pdb) threshold * (sum(dev) / len(dev))
73.5
(Pdb) quit

The key insight: breakpoint() lets you inspect the actual state at the moment of interest.

pdb commands

Command	Action
n / next	Execute current line, stop at next
s / step	Step into a call
c / continue	Continue to next breakpoint or end
q / quit	Quit the debugger (and the program)
p expr	Print the value of an expression
pp expr	Pretty-print the value
l / list	Show nearby source
w / where	Show the stack
u / up	Move up the stack
d / down	Move down the stack
! stmt	Run a Python statement (not a debugger command)
h / help	Show help

Use n to move line by line, s to follow a function call into its internals, and c to jump to the next breakpoint.

Remote breakpoint with condition

if len(values) > 1000:
    breakpoint()

Conditional breakpoints are useful for rare or large-input bugs.

Post-Mortem Debugging

When a script crashes, you can drop into the debugger at the failure point.

python -m pdb analyze.py

Run c to continue. When the exception hits, you are dropped into post-mortem mode at the failing frame.

Inside a running session, pdb.pm() enters post-mortem debugging after an exception.

import pdb

try:
    process(data)
except Exception:
    pdb.pm()

This is very effective when a traceback alone is not enough to understand the bad state.

Debugging from a Test

A failing test gives you a controlled failure path.

pytest -v -s tests/test_analysis.py

-s disables output capture so you can see print() and logging output during the test.

For interactive debugging inside a test:

def test_scale():
    values = [1, 2, 3]
    breakpoint()
    result = scale(values)
    assert result == pytest.approx(...)

Run with pytest -s so the interactive session works.

IDE Debuggers

GUI debuggers (VS Code, PyCharm, etc.) give you breakpoints, watches, and step controls with a visual interface. The concepts map directly from pdb:

• Breakpoints pause execution at a line.
• Step over is next.
• Step into is step.
• Step out returns from the current function.
• Watches/inspectors are p / pp.

For Data Science workflows, IDE debuggers are often more comfortable than command-line pdb because you can inspect dataframes, arrays, and plots visually.

Debugging Data Issues

For data-heavy code, generic debugging is not enough. Inspect the data, not just the control flow.

def clean_numeric(values):
    out = []
    for i, v in enumerate(values):
        if v is None:
            continue
        out.append(float(v))
    return out

A quick data-oriented probe:

print("n=", len(values), "n_none=", sum(1 for v in values if v is None))
print("types:", set(type(v).__name__ for v in values))
print("sample:", values[:10])

For larger datasets, sampling is your friend.

import random
sample = random.sample(values, k=min(10, len(values)))
print(sample)

Common Causes and How to Probe Them

Symptom	Likely cause	First probe
Wrong result with no error	Bad assumption, edge case, silent conversion	print input and output shapes/types
None where a value is expected	Missing return, failed condition	trace the code path that should set the value
Exception only on some inputs	Edge case, type mismatch	reproduce with the smallest failing input
Flaky failure	randomness, time, shared state	freeze randomness, isolate state
Slow or hanging	infinite loop, large data, blocking IO	add progress prints or timeout guard
Error deep in a library	bad input reaching the library	inspect the actual arguments at the call site

A Small Debugging Workflow Example

A trimmed-mean function returns a wrong value for one case.

1. Reproduce: trimmed_mean([1, 1, 1, 1], trim=1) returns the wrong number.
2. Localize: add a breakpoint or print inside the function.
3. Observe: sorted_vals[trim:-trim] is the suspect line.
4. Hypothesize: the slice is wrong when trim removes too much.
5. Test: print sorted_vals, trim, and the slice result.
6. Fix: add the guard for insufficient data; verify with the failing case and a few neighbors.

Debugging is usually this small loop, repeated.

Defensive Practices That Help Debugging

• Fail fast — validate inputs early so bad state is caught near the source.
• one assertion per behavior — easier to locate the failure.
• deterministic tests — no hidden randomness or clock dependence.
• small functions — easier to isolate and inspect.
• type and shape checks at boundaries — especially useful for data pipelines.
• logging at decision points — record why a branch was taken, not just that it was taken.

Common Mistakes

• Changing multiple things at once — you learn nothing about the cause.
• Assuming the bug is where the exception is raised — the real issue can be earlier.
• Debugging without a reproducible case — random exploration is slow.
• Printing everything — too much output hides the signal.
• Ignoring types — many subtle bugs are type or shape mismatches.
• Using a debugger only for crashes — it is also for understanding correct-but-wrong behavior.

Related Cheat Sheets

• python-testing — write tests so failures are reproducible by default
• pytest — run and filter tests, inspect failures with rich output
• logging — structured diagnostics that survive the debug session
• ml-preprocessing — debugging transform pipelines often comes down to input/output inspection

Official Documentation

• pdb
• Built-in exceptions
• logging

Last Updated

2026-09-06 — Initial version.