Batch image processing scripts and performance

Process a directory of images correctly, decode faster with draft and reduce, and use processes rather than threads for CPU-bound work.

Walking a directory safely

from pathlib import Path
from PIL import Image, ImageOps, UnidentifiedImageError

SRC = Path("photos")
DST = Path("build/photos")
DST.mkdir(parents=True, exist_ok=True)
EXTS = {".jpg", ".jpeg", ".png", ".webp", ".tif", ".tiff"}

def convert(src: Path, dst: Path, box=(1600, 1600)) -> str:
    try:
        with Image.open(src) as im:
            im = ImageOps.exif_transpose(im)
            im = ImageOps.contain(im.convert("RGB"), box, Image.Resampling.LANCZOS)
            dst.parent.mkdir(parents=True, exist_ok=True)
            im.save(dst.with_suffix(".webp"), quality=82, method=5)
        return "ok"
    except UnidentifiedImageError:
        return "not an image"
    except OSError as exc:
        return f"error: {exc}"

for src in sorted(SRC.rglob("*")):
    if src.suffix.lower() in EXTS:
        rel = src.relative_to(SRC)
        print(rel, convert(src, DST / rel))

The with block matters: Pillow keeps the file handle open lazily, and a long batch without closing descriptors will hit the operating system limit.

Decoding less

Technique	Saving	Limit
`draft()`	Decodes JPEG at 1/2, 1/4 or 1/8 scale	JPEG only, and must be called before load
`reduce()`	Factors of two for the same purpose	Same restriction
`thumbnail()`	Resizes in place with a fast filter	Lossy for repeated use
`Image.Resampling.LANCZOS`	Best quality downscale	Slower than BILINEAR or BICUBIC
`ImageOps.contain`	Never upscales	Does not square an image

with Image.open("huge.jpg") as im:
    im.draft("RGB", (1200, 1200))       # decode at a reduced scale: 4x to 64x less work
    im = im.resize((1200, int(1200 * im.height / im.width)), Image.Resampling.LANCZOS)
    im.save("thumb.jpg", quality=85, optimize=True)

# Pillow's own decoder threads help for PNG and are ignored for others
from PIL import Image as I
I.MAX_IMAGE_PIXELS = 200_000_000        # allow larger inputs, bounded

⚠️

Image decoding and filtering run in C and release the GIL for parts of the work, but the Python-level loop still serialises under threads. For a large batch use concurrent.futures.ProcessPoolExecutor; use threads only when the work is dominated by I/O such as downloading.

Memory

from concurrent.futures import ProcessPoolExecutor
from pathlib import Path

def job(path_str: str) -> str:
    p = Path(path_str)
    src = Path("photos") / p
    dst = Path("build/photos") / p
    return convert(src, dst)

if __name__ == "__main__":               # required on Windows and macOS spawn start
    files = [str(p.relative_to("photos")) for p in Path("photos").rglob("*.jpg")]
    with ProcessPoolExecutor(max_workers=4) as pool:
        for name, status in zip(files, pool.map(job, files)):
            print(name, status)

One worker decoding a 100 MP image can need well over a gigabyte; size max_workers against available RAM, not just cores.
Every transform returns a new image, so a chain of five operations holds several full-size copies at once. Rebind names so intermediates can be collected.
Image.fromarray(arr, copy=False) avoids duplicating a NumPy buffer when you own it and will not mutate it.

FAQ

Why is my batch so much slower than a single conversion?

Usually because the whole file is decoded at full resolution before the resize. Call draft() first for JPEGs, and make sure you are not saving as PNG when WebP or JPEG would do.

Is it safe to run Pillow across processes?

Yes. Each process decodes independently, and Pillow holds no shared mutable global state you need to synchronise, other than the module-level limits such as MAX_IMAGE_PIXELS, which each child inherits at start.

Reading and writing EXIF and image metadata Debugging Pillow: mode errors, bombs and truncated files

Last refreshed 2026-09-18.

Batch image processing scripts and performance

Walking a directory safely

Decoding less

Memory

FAQ

Related