HowToCodePage

A notebook workflow that scales

Project layout, imports, reproducibility and handing a notebook to someone else without surprises.

Structure of a project

project/
  notebooks/
    01-explore.ipynb
    02-report.ipynb
  src/
    __init__.py
    load.py          # reusable functions the notebook imports
  data/
    raw/             # never edited by hand
    processed/
  requirements.txt
  README.md

The single most valuable habit: the moment a cell contains logic you might reuse, move it into src/ and import it. Notebooks stay short, and the logic becomes testable.

import sys
sys.path.append("..")          # so "import src.load" works from notebooks/
from src.load import read_sales

df = read_sales("../data/raw/sales.csv")

Reproducibility

  • Record the environment: pip freeze > requirements.txt (or use conda/uv lockfiles).
  • Seed randomness: np.random.default_rng(0) — otherwise "the results changed" with no code change.
  • Keep data paths relative to the notebook (or a config constant), never absolute C:\Users\you\….
  • Run top-to-bottom before every commit.
from pathlib import Path

ROOT = Path.cwd().parent
DATA = ROOT / "data" / "raw" / "sales.csv"
⚠️
A notebook holding credentials, API keys or personal data gets committed by accident more often than any other file type. Keep secrets in environment variables and add .ipynb_checkpoints/ to .gitignore.

Handing it over

jupyter nbconvert --to html --execute report.ipynb
jupyter nbconvert --to python 01-explore.ipynb    # diff-friendly review
papermill report.ipynb out.ipynb -p month 2026-08  # parameterised runs

--execute reruns the whole notebook in a fresh kernel while converting, so the HTML you send cannot be an artefact of your session state. papermill turns a notebook into a repeatable batch job with parameters.

FAQ

Should notebooks be in version control?
Yes, but strip outputs and avoid merge conflicts: keep notebooks short and keep logic in modules. Never let two people edit the same notebook cell-by-cell on separate branches.
How do I run the same analysis for twelve months of data?
Parameterise with papermill, or move the logic into a script/module and drive it from a loop. Copy-pasting notebooks does not scale.

Notebook fundamentals Reading and writing data

Last refreshed 2026-09-18.