Screenshots, logs and debugging flaky tests

Capture the artefacts that make a CI failure diagnosable - screenshots, page source, browser logs, stack traces - and use them to find the real cause.

Capturing on failure

import pathlib, time
from selenium.common.exceptions import WebDriverException

ARTIFACTS = pathlib.Path("artifacts")

def capture(driver, name):
    ARTIFACTS.mkdir(exist_ok=True)
    stamp = time.strftime("%Y%m%d-%H%M%S")
    base = ARTIFACTS / (name + "-" + stamp)

    try:
        driver.save_screenshot(str(base) + ".png")
    except WebDriverException:
        pass

    try:
        (pathlib.Path(str(base) + ".html")).write_text(
            driver.page_source, encoding="utf-8"
        )
    except WebDriverException:
        pass

    try:
        logs = driver.get_log("browser")
        (pathlib.Path(str(base) + ".log")).write_text(
            "\n".join(e["level"] + " " + e["message"] for e in logs),
            encoding="utf-8",
        )
    except WebDriverException:
        pass

    return str(base)

• Screenshot, page source and console log together answer most 'what was on screen' questions without a re-run.
• Wrap each capture in its own try block: if the session has already died, the screenshot call will raise and you will lose the log too.
• Timestamp the filenames so a parallel run does not overwrite its own artefacts.
• element.screenshot() is also available and is often enough to see what a specific component rendered.

Logs that mean something

from selenium.webdriver.chrome.options import Options

options = Options()
options.set_capability("goog:loggingPrefs", {
    "browser": "ALL",        # console output and page errors
    "performance": "ALL",
})

# after the test, fail on unexpected console errors
errors = [e for e in driver.get_log("browser") if e["level"] == "SEVERE"]
assert not errors, errors

Asserting on console errors is one of the highest-value checks you can add: it catches failed asset loads, unhandled promise rejections and framework warnings that no assertion would otherwise notice.

Diagnosing intermittent failures

1. Reproduce locally with the same options as CI, including headless and the same window size.
2. Run it in a loop and count the failures; a test that fails one time in twenty is a real bug with a low hit rate.
3. Read the captured screenshot from the failing run before changing any code.
4. Look for a wait on the wrong condition - presence instead of clickability, or a URL instead of ready state.
5. Check for shared state: a reused browser profile, a database row another test also writes, or a fixed test account.
6. Only then add a retry, and record that you did - a retry that hides a real race is a bug you scheduled for later.

# loop until it fails, then keep the artefacts
for i in $(seq 1 20); do
  python -m pytest tests/test_checkout.py -q || break
done

# retries as a diagnostic, not a fix
python -m pytest --reruns 3 --reruns-delay 1 -q

# a wait that means what the test needs
from selenium.webdriver.support import expected_conditions as EC

# wrong: the element exists but is not yet interactive
WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "pay"))
)

# right: it is there, visible and enabled
WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.ID, "pay"))
)

FAQ

Related

Browser options and headless execution Structuring a suite: page objects, pytest and parallelism

Last refreshed 2026-09-18.