69. Debugging: pdb, faulthandler, py-spy, and remote attach
69.1 breakpoint() and pdb
def mean_latency(samples: list[float]) -> float:
total = sum(samples)
breakpoint() # drops into pdb here (disable with PYTHONBREAKPOINT=0)
return total / len(samples)
| pdb command | Does |
|---|---|
l / ll |
list source around the line / the whole function |
p expr, pp expr |
print / pretty-print an expression |
n / s / c |
next line / step into / continue |
w, u, d |
where (the stack), up, down a frame |
b file:line, cond |
set a (conditional) breakpoint |
interact |
a full REPL with the current locals |
python -m pdb -c continue script.py runs a script and stops in the
debugger at the point of an uncaught exception, with every local variable
available: post-mortem debugging, the fastest way to understand a crash.
In pytest: pytest --pdb (stop at the failure) and pytest --lf (rerun
only the last failures). Your editor's debugger is pdb with a UI; learn
both.
69.2 When the process hangs or crashes
# Where is it stuck? Dump every thread's Python stack, without stopping it:
uv tool run py-spy dump --pid 12345
# Is it using CPU, and in which functions? (live, top-like)
uv tool run py-spy top --pid 12345
# Crashes in C code (segfault in an extension, deadlock in native code):
python -X faulthandler app.py # prints the Python stack of every thread on SIGSEGV/SIGABRT
kill -SIGUSR1 12345 # with faulthandler.register(signal.SIGUSR1) in the app: dump on demand
3.14 adds remote debugging (PEP 768): python -m pdb -p PID
attaches a full pdb session to a running process with no code changes and
no restart, and sys.remote_exec(pid, script) runs a script inside a live
interpreter. 3.14 also ships python -m asyncio ps PID and
python -m asyncio pstree PID to show every asyncio task in a running
program and what it is awaiting, which is the tool you want when an async
service stops responding.
And when it is the OS, not Python: strace -f -p PID (Ch 33),
lsof -p PID (leaked file descriptors), /proc/PID/status (memory), and
OS Ch 42's
method for "why is this stuck?".