Python CLI Toolcraft

Input & UX

Handling Broken Pipe and SIGPIPE

Stop a Python CLI printing a traceback under head - catch BrokenPipeError at the boundary, silence the shutdown warning, and exit cleanly across platforms.

Updated

mytool list | head -n 10 is an ordinary thing to type, and for many Python CLIs it produces a BrokenPipeError traceback followed by a second warning at shutdown. Nothing went wrong — the downstream program got what it needed and left — but the output says otherwise.

TL;DR

  • When a downstream reader exits, your next write raises BrokenPipeError.
  • Catch it at the boundary, not in each command.
  • Redirect stdout to devnull before exiting, or Python prints a second warning during shutdown.
  • Exit 0: the pipeline behaved as designed.
  • Do not restore the default SIGPIPE handler unless you understand what it does to your cleanup.

Prerequisites

A CLI that produces more than a screen of output, and the single main() boundary described in error handling and exit codes.

What actually happens

What happens under head When a downstream command exits early, the next write raises a broken pipe error, which a command line tool should treat as a normal end rather than a crash. What happens under head Your tool Pipe head User writes line 1…10 head -n 10 exits, closes the read end writes line 11 → BrokenPipeError exit quietly, status 0 Printing a traceback here makes a completely normal shell idiom look like a crash.

In a shell pipeline the two processes are joined by a pipe. When head has printed its ten lines it exits and closes its end. The next time your process writes, the operating system signals that nobody is listening.

For a C program the default is that SIGPIPE terminates the process silently — which is why yes | head does not produce an error. Python changes this: it sets SIGPIPE to be ignored, so the write raises BrokenPipeError instead, and an unhandled exception prints a traceback.

That is a reasonable default for a library, because a server should not die because one client disconnected. For a CLI in a pipeline it is exactly wrong, so you handle it.

The fix, in full

Handling a closed downstream cleanly A broken pipe is caught at the boundary, standard error is closed to suppress the interpreter warning, and the process exits without a traceback. Handling a closed downstream cleanly BrokenPipeError raised on the next write Caught at the boundary not in the command devnull stdout, exit no traceback, no warning propagates then Python also prints a warning at shutdown unless stdout is redirected before exiting. 
import os
import sys

def main() -> None:
    try:
        app()
    except BrokenPipeError:
        # The downstream reader exited. Nothing failed; leave quietly.
        #
        # Python flushes stdout during interpreter shutdown, which would raise again and print
        # "Exception ignored in: <_io.TextIOWrapper name='<stdout>'>". Pointing the file
        # descriptor at devnull first makes that flush a no-op.
        devnull = os.open(os.devnull, os.O_WRONLY)
        os.dup2(devnull, sys.stdout.fileno())
        sys.exit(0)

Both halves are needed. Catching the exception stops the traceback; the dup2 stops the second message, which Python prints during shutdown when it tries to flush a stream it cannot write to. Many implementations of this fix omit the second half and leave users with a stray warning after an otherwise clean run.

Exit 0 because nothing failed. A non-zero status makes mytool list | head look like an error to every wrapper script that checks it, and set -o pipefail will propagate that failure.

Platform differences

How a closed pipe surfaces, per platform A comparison of how a closed downstream pipe appears to a Python program on Linux and macOS versus Windows, and what each requires. How a closed pipe surfaces, per platform Platform You see You must Linux / macOS BrokenPipeError on write catch it, then redirect stdout Windows OSError with EINVAL/EPIPE catch OSError too Any, at shutdown a second warning on stderr dup2 devnull before exiting Python restores SIGPIPE to raising an exception rather than killing the process, which is why this is your job.

On Linux and macOS the failed write raises BrokenPipeError, which is a subclass of OSError. On Windows there is no SIGPIPE; a closed pipe surfaces as an OSError with EINVAL or EPIPE depending on the situation. Catching both covers everything:

    except (BrokenPipeError, OSError) as exc:
        if isinstance(exc, BrokenPipeError) or exc.errno in (errno.EPIPE, errno.EINVAL):
            devnull = os.open(os.devnull, os.O_WRONLY)
            os.dup2(devnull, sys.stdout.fileno())
            sys.exit(0)
        raise

Be careful not to swallow every OSError — a disk full or a permission problem is a genuine failure that should exit non-zero with a message. Checking errno keeps the clause narrow.

What about restoring the default signal handler?

You will find advice to do this:

import signal
signal.signal(signal.SIGPIPE, signal.SIG_DFL)      # think twice

It restores the C behaviour: the process is killed immediately by the signal when a write fails. It is short, and it has consequences worth knowing.

Your process dies at the write, so nothing after it runs — no finally blocks, no context manager cleanup, no temporary file removal, no flushing of a partially written output file elsewhere. For a tool that only prints, that is acceptable. For a tool that holds a lock, writes a state file, or has a half-finished download in a temporary directory, it means leaving mess behind.

The exception-based approach lets your cleanup run and costs six lines. Prefer it unless you have specifically decided that immediate termination is correct.

Where the handler goes

At the boundary, wrapping the whole application, and nowhere else. A try/except BrokenPipeError inside a command means every command needs one, they drift, and a command added next month has none.

The one exception is a long generator writing directly to stdout in a loop, where you may want to stop iterating rather than unwind through the whole call stack. Even then, re-raise or return promptly and let the boundary do the exiting:

for row in core.collect(period):
    try:
        typer.echo(json.dumps(row.as_dict()))
    except BrokenPipeError:
        break            # stop producing; the boundary handles the exit

UX considerations

Say nothing. A broken pipe is not worth a message, not even on stderr. head users are not surprised that the producer stopped, and a warning trains people to ignore your stderr output — which is where your real errors go.

Do not treat it as a partial failure. Some tools exit 141 (128 + SIGPIPE) to mimic the shell. Unless your users specifically expect that, exit 0: the alternative surprises set -e scripts for no benefit.

Flush deliberately for long output. When stdout is a pipe, Python buffers in blocks, so a consumer using head -n 1 may wait for a full buffer before seeing anything. Flushing every few hundred lines makes the tool feel responsive without giving up throughput.

Testing the behaviour

This is one of the few behaviours that genuinely needs a subprocess, because the pipe is the point:

import subprocess
import sys

def test_survives_a_closed_downstream():
    tool = subprocess.Popen(
        [sys.executable, "-m", "mytool", "list"],
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    )
    head = subprocess.Popen(["head", "-n", "5"], stdin=tool.stdout, stdout=subprocess.DEVNULL)
    tool.stdout.close()          # let the tool see EPIPE when head exits
    head.wait()

    _, stderr = tool.communicate()

    assert tool.returncode == 0
    assert b"BrokenPipeError" not in stderr
    assert b"Exception ignored" not in stderr        # the shutdown warning, too

The last assertion is the one that catches an incomplete fix. Plenty of tools catch the exception and still print Exception ignored in: <_io.TextIOWrapper …> afterwards, which looks just as broken to a user.

Mark the test as Unix-only if your CI matrix includes Windows, where head may not exist; the behaviour there is worth its own test using a Python reader.

Conclusion

Six lines at the boundary turn a traceback into silence: catch the error, redirect stdout so the shutdown flush cannot fail, exit 0. The alternative — restoring the default signal handler — is shorter and skips your cleanup, which is a trade worth making deliberately rather than by copying a snippet.

Frequently asked questions

Why do I get two messages, not one?

The first is the unhandled BrokenPipeError from your write. The second comes from the interpreter flushing sys.stdout during shutdown, which fails the same way and prints "Exception ignored". The dup2 to devnull is what prevents the second.

Does this affect stderr as well?

It can, if stderr is also piped into something that exits. It is much rarer, and the same handler covers it — but do not redirect stderr to devnull in the fix, or you will suppress genuine error messages on the way out.

Should I use exit code 141?

Only if your users expect shell-like semantics. 141 is what a shell reports when a child is killed by SIGPIPE, and reproducing it from Python surprises scripts that check for zero. Exit 0 is the friendlier default for a tool.

Does Click or Typer handle this for me?

Not by default. Click has historically included some handling around its own echo path, but the exception can still escape from your own writes, and neither framework silences the shutdown warning. The boundary handler is yours to add.

Can I avoid the problem by not writing so much?

No — the failure is about the reader leaving, not about volume. Even a short command can meet it if the consumer is head -n 1. Handle it once at the boundary and stop thinking about it.

Is this worth handling in a tool that prints one line?

Yes, because the cost is six lines at a boundary you already have. A single-line command still meets | head -n 0, and more importantly the handler is written once for the whole application rather than per command — so a future command that streams thousands of records is covered before it exists.

Does the same problem apply to writing to a file?

No. A closed pipe is specific to a reader that went away; writing to a file fails for different reasons — no space, no permission — which are genuine errors deserving a message and a non-zero exit. That is exactly why the handler checks the error type rather than swallowing every OSError.