HowToCodePage

Debugging and the hidden-state trap

Post-mortem debugging, breakpoints, logging, and the techniques that expose a notebook whose results depend on cells you already deleted.

Debugging inside the kernel

%pdb on          # drop into the debugger at any uncaught exception

# after a crash, without %pdb:
%debug           # inspect the traceback frame by frame

def parse(rows):
    breakpoint()                 # Python 3.7+: enters pdb right here
    return [r for r in rows if r["ok"]]
(Pdb) l            list source around the current line
(Pdb) p rows[0]    print an expression
(Pdb) w            show the call stack
(Pdb) u / d        move up / down the stack
(Pdb) q            quit the debugger and the cell
  • %debug works on the most recent exception, even after the cell finished, as long as the kernel has not restarted.
  • breakpoint() respects the PYTHONBREAKPOINT environment variable, so you can disable it in production.
  • Variable edits you make in the debugger persist in the kernel afterwards, which is useful and also dangerous.

Detecting stale state

SymptomLikely cause
NameError after a restartA cell you deleted or never ran defined that name
Results change with no code changeRandomness, or a variable reassigned in an old cell
Fix works locally, fails for othersThe notebook was never run top-to-bottom
Imported module seems not to updateCached import; use %autoreload 2 or restart
# list everything the kernel currently holds
%who DataFrame
%whos

# prove a variable was never defined by the code you can see
import inspect
print(inspect.getsource(parse))
💡
The only reliable test of a notebook is Restart Kernel and Run All, followed by nbconvert --execute for the version you share. Hidden state is not a bug you fix once; it is a habit you keep.

Structure for testability

import logging
from pathlib import Path

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(message)s",
)

def build_features(df):
    """Pure function: no prints, no globals, easy to unit-test."""
    return df.assign(ratio=df["a"] / df["b"])

# in the notebook, just orchestrate
df = read_csv(Path("data/raw/sales.csv"))
features = build_features(df)
logging.info("rows=%d cols=%d", *features.shape)

Logging beats printing: messages carry timestamps and levels, survive %%capture handling, and can be redirected to a file when the notebook runs as a batch job.

FAQ

Why does the same cell give different results each run?
Almost always randomness or hidden state. Seed your generators, then restart the kernel and run all cells. If the difference persists, print the inputs and check for a variable reassigned earlier in the session.
How do I debug a cell that takes ten minutes to reach the failure?
Isolate the slow part behind a cached artefact: save the intermediate DataFrame to disk in the earlier cell, then load it while iterating on the failing cell. Restart only when you need to verify the whole chain.

Notebook fundamentals Magics: line, cell and shell commands

Last refreshed 2026-09-18.