Handling Subprocess Timeouts and Exit Codes

subprocess.run(argv, timeout=30) looks like a complete answer to "what if the child hangs?" It is not. It kills only the process you started, so a npm run build or sh -c wrapper dies while the real work carries on underneath, holding ports and file locks. And once the child is gone you still have to decide what your own CLI exits with: pass the child's code through, map it, or collapse everything to 1? This guide covers enforcing timeouts that actually stop the work, reading return codes correctly — including the negative ones — and turning all of it into exit codes a calling script can rely on. It extends the helper from calling external commands safely with subprocess.

Prerequisites

• Python 3.10+, on Linux or macOS for the process-group examples (Windows differences are noted where they matter).
• A CLI built with Typer or Click.
• A basic understanding of signals — at least that SIGTERM asks a process to stop and SIGKILL forces it.

What timeout= really does

When the deadline passes, subprocess.run() calls kill() on the child, waits for it to exit, and raises TimeoutExpired. On POSIX that is SIGKILL: immediate, uncatchable, no cleanup. The exception carries whatever output was captured so far in exc.stdout and exc.stderr (as bytes, even if you asked for text in some Python versions — decode defensively).

Two things follow. First, the child gets no chance to clean up — temporary files stay, a half-written output file stays half-written. Second, and worse, grandchildren survive. If you ran ["npm", "run", "build"], npm is killed, but the node process it spawned is re-parented to init and keeps running. The next invocation of your tool then fails with "port already in use" or finds a lock file it cannot explain.

The recipe: a deadline that kills the whole tree

The robust approach is to start the child in its own process group (a new session), and when the deadline passes, signal the group: first SIGTERM so well-behaved programs can clean up, then SIGKILL after a grace period.

# src/mytool/deadline.py
from __future__ import annotations

import os
import signal
import subprocess
import sys
from dataclasses import dataclass

GRACE_SECONDS = 5.0


@dataclass(frozen=True)
class Outcome:
    returncode: int
    timed_out: bool
    stdout: str
    stderr: str


def _terminate_tree(proc: subprocess.Popen[str]) -> None:
    """Ask the whole group to stop, then force it."""
    if sys.platform == "win32":
        proc.kill()  # TerminateProcess; use a job object for true trees
        return
    try:
        os.killpg(proc.pid, signal.SIGTERM)
    except ProcessLookupError:
        return
    try:
        proc.wait(timeout=GRACE_SECONDS)
    except subprocess.TimeoutExpired:
        os.killpg(proc.pid, signal.SIGKILL)


def run_with_deadline(argv: list[str], timeout: float) -> Outcome:
    popen_kwargs: dict = dict(
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        stdin=subprocess.DEVNULL,
        text=True,
        encoding="utf-8",
        errors="replace",
    )
    if sys.platform == "win32":
        popen_kwargs["creationflags"] = subprocess.CREATE_NEW_PROCESS_GROUP
    else:
        popen_kwargs["start_new_session"] = True

    proc = subprocess.Popen(argv, **popen_kwargs)
    try:
        out, err = proc.communicate(timeout=timeout)
        return Outcome(proc.returncode, False, out, err)
    except subprocess.TimeoutExpired:
        _terminate_tree(proc)
        out, err = proc.communicate()
        return Outcome(proc.returncode, True, out, err)
    except BaseException:
        _terminate_tree(proc)   # Ctrl+C or any error: never leave the tree running
        proc.communicate()
        raise

start_new_session=True makes the child the leader of a new process group whose ID equals its PID, so os.killpg(proc.pid, ...) reaches every process it spawns — unless one of those deliberately starts its own session, which daemons do. communicate(timeout=...) reads both pipes concurrently while it waits, so it cannot deadlock the way sequential reads can (see streaming subprocess output in real time for why that matters).

The except BaseException branch is easy to overlook and important. If the user presses Ctrl+C while you wait, KeyboardInterrupt is raised in your process — but because the child is in its own session, the terminal's SIGINT did not reach it. Without that branch the child runs on after your CLI exits. The broader pattern is in handling KeyboardInterrupt cleanly.

On Windows, CREATE_NEW_PROCESS_GROUP lets you send CTRL_BREAK_EVENT, but killing a whole tree reliably needs a Job Object, which the standard library does not wrap. For most CLIs, proc.kill() plus documenting the limitation is acceptable; tools that orchestrate large trees on Windows usually depend on psutil to walk and kill children.

Reading the return code

A returncode is not just "zero or not". Python encodes three different situations in it, and your CLI should tell them apart.

• 0 — success.
• Positive — the program exited on its own with that status. Its meaning is defined by that program: grep uses 1 for "no match", diff uses 1 for "files differ", and both use 2 for real errors.
• Negative — on POSIX, the child was killed by signal -returncode. -9 is SIGKILL (often the out-of-memory killer), -15 is SIGTERM, -11 is a segmentation fault.

Shells report signal deaths as 128 + n, so when you pass a signal death through, convert it. Here is the translation layer, wired into a Typer command:

# src/mytool/cli.py
import signal
import sys

import typer

from mytool.deadline import run_with_deadline

app = typer.Typer()

EXIT_TIMEOUT = 124


def exit_code_for(returncode: int) -> int:
    if returncode < 0:
        return 128 + (-returncode)
    return returncode


def describe(returncode: int) -> str:
    if returncode < 0:
        try:
            return f"killed by {signal.Signals(-returncode).name}"
        except ValueError:
            return f"killed by signal {-returncode}"
    return f"exited with status {returncode}"


@app.callback()
def main() -> None:
    """Task runner."""


@app.command()
def test(timeout: float = typer.Option(600, help="Seconds before the run is stopped.")) -> None:
    """Run the test suite with a deadline."""
    outcome = run_with_deadline([sys.executable, "-m", "pytest", "-q"], timeout=timeout)
    if outcome.timed_out:
        typer.secho(f"error: tests did not finish within {timeout:g}s", fg="red", err=True)
        raise typer.Exit(EXIT_TIMEOUT)
    if outcome.returncode != 0:
        typer.secho(f"error: pytest {describe(outcome.returncode)}", fg="red", err=True)
        typer.echo(outcome.stdout[-2000:], err=True)
        raise typer.Exit(exit_code_for(outcome.returncode))
    typer.echo(outcome.stdout.strip().splitlines()[-1])


if __name__ == "__main__":
    app()

Pass through, map, or collapse?

There is no single right policy; there is a right policy per command, and it should be written down.

• Pass through when your command is a thin wrapper and callers think of it as the child. A mytool test that wraps pytest should exit with pytest's codes — CI systems already understand them.
• Map when the child's codes carry meaning your callers need but under different numbers. grep's 1 ("no match") might become your 0 with an empty result.
• Collapse to 1 when the child is an implementation detail. Nobody calling mytool deploy should need to know that exit 23 came from rsync's "partial transfer".

Whatever you choose, reserve your own codes for your own conditions — usage errors (2), timeouts (124), missing programs (127) — as described in choosing exit codes for CLI tools.

UX considerations

• Name the limit in the message. "did not finish within 600s" tells the user both what happened and which knob to turn. Expose the timeout as an option so they can turn it.
• Distinguish "failed" from "was stopped". A timeout and a crash call for different next steps; say which one happened. "killed by SIGKILL" with no timeout of your own usually means the out-of-memory killer, which is worth hinting at.
• Show partial output on timeout. The last lines captured before the deadline usually show where the child was stuck.
• Pick a grace period that fits the child. Five seconds is enough for most tools to flush and exit; a database migration might need thirty. Too short, and you turn every timeout into a hard kill with no cleanup.
• Never exit 0 after a timeout. Even with --keep-going semantics, the overall exit code should report that something did not complete.

Testing the behaviour

Test with small Python children so the suite runs anywhere, and assert on the outcome rather than on timing. The tree-kill test starts a child that spawns a grandchild, then checks the grandchild is gone:

# tests/test_deadline.py
import os
import sys
import time

import pytest

from mytool.cli import exit_code_for
from mytool.deadline import run_with_deadline

posix_only = pytest.mark.skipif(sys.platform == "win32", reason="process groups")


def test_success_passes_through():
    out = run_with_deadline([sys.executable, "-c", "print('ok')"], timeout=10)
    assert (out.returncode, out.timed_out, out.stdout) == (0, False, "ok\n")


def test_timeout_is_reported():
    out = run_with_deadline([sys.executable, "-c", "import time; time.sleep(30)"], timeout=0.5)
    assert out.timed_out


@posix_only
def test_grandchild_is_killed(tmp_path):
    pidfile = tmp_path / "grandchild.pid"
    parent = (
        "import subprocess, sys, time\n"
        "p = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)'])\n"
        f"open({str(pidfile)!r}, 'w').write(str(p.pid))\n"
        "time.sleep(60)\n"
    )
    out = run_with_deadline([sys.executable, "-c", parent], timeout=1.0)
    assert out.timed_out
    grandchild = int(pidfile.read_text())
    time.sleep(0.2)
    with pytest.raises(ProcessLookupError):
        os.kill(grandchild, 0)


@pytest.mark.parametrize(("code", "expected"), [(0, 0), (3, 3), (-15, 143), (-9, 137)])
def test_exit_code_mapping(code, expected):
    assert exit_code_for(code) == expected

os.kill(pid, 0) sends no signal; it only checks the process exists, raising ProcessLookupError once it is gone. Because the grandchild was re-parented to init after its parent died, it is reaped promptly and the check is reliable. For the patterns behind isolating tests like these, see mocking filesystem and network in CLI tests.

Conclusion

A timeout is only as good as what it stops. Start children in their own process group, give them a short grace period with SIGTERM, then SIGKILL the group; do the same on Ctrl+C. Read return codes with their sign in mind, convert signal deaths to the 128 + n convention, and decide per command whether to pass codes through, map them or collapse them. Your CLI then fails in ways that scripts, CI systems and people can all act on.

Frequently asked questions

Related

• Up: Running subprocesses from Python CLIs
• Calling external commands safely with subprocess
• Streaming subprocess output in real time
• Choosing exit codes for CLI tools
• Handling SIGTERM and graceful shutdown