Notebooks vs scripts: jupytext, nbconvert and papermill
Keep the analysis in a notebook and the review in a text file, export to any format, and run the same notebook once per parameter set.
Paired scripts with jupytext
jupytext makes an .ipynb file and a readable text file two views of the same document. You edit either; the other is regenerated on save.
pip install jupytext
# create the pairing once, per notebook
jupytext --set-formats ipynb,py:percent notebooks/analysis.ipynb
# or convert in a batch
jupytext --to py:percent notebooks/*.ipynb
# produce a Markdown view for review
jupytext --to md notebooks/analysis.ipynbThe py:percent format marks cells with # %% comments, which most editors and linters already understand. Reviewing a pull request then means reading a diff of Python, not of JSON.
nbconvert exporters
| Command | Output | Typical use |
|---|---|---|
--to html | Self-contained HTML | Sharing a report by email or a static page |
--to pdf | Formal delivery; needs a LaTeX or Chromium engine | |
--to slides | Reveal.js HTML | Turning an analysis into a presentation |
--to markdown | Markdown plus assets | Publishing to a docs site |
--to python | Plain script | Diff-friendly review or a production starting point |
jupyter nbconvert --to html --execute --ExecutePreprocessor.timeout=600 analysis.ipynb
jupyter nbconvert --to slides --post serve talk.ipynb--execute re-runs the notebook in a fresh kernel during conversion. This is the difference between a document that describes a result and a document that can reproduce it.
Parameterised runs with papermill
# a cell tagged "parameters" in the notebook defines the defaults
month = "2026-08"
region = "all"pip install papermill papermill-batch
papermill report.ipynb out/2026-08.ipynb -p month 2026-08 -p region emea
# run many parameter sets from a file
papermill --batch report.ipynb out/ -f params.yamlpapermill injects the parameters into the tagged cell, executes the notebook and writes a copy with the outputs recorded. That turns one notebook into a monthly job without copy-pasting it twelve times.
FAQ
Should the .py or the .ipynb be the source of truth?
.py as the reviewed artefact and regenerate notebooks on demand; others commit both. The tooling does not care, but a mixed convention produces constant noise.How do I fail a batch run when a check fails?
sys.exit(1) after logging. papermill propagates a non-zero exit code, which your scheduler or CI can then act on.Related
A notebook workflow that scales Version control, diffs and reproducible output
Last refreshed 2026-09-18.