Reporting Machine-Readable Errors in JSON Mode

A --json flag is a promise to scripts: "you can parse what I print". Most CLIs keep that promise only on the happy path. On failure, the same command prints Error: site "nope" does not exist in a coloured box, or a Python traceback, or a usage message — and the script that was about to call json.loads on the output has to fall back to matching English sentences with regular expressions. The fix is to make failures part of the JSON contract: when JSON mode is on, every error — expected failures, parse errors and bugs alike — is reported as one small JSON document with a stable code field that scripts can branch on. This guide defines that document, decides where it goes, implements it in a single entry point so no command has to think about it, and tests every kind of failure. It builds on designing an exception hierarchy for a CLI and belongs to the error handling and exit codes topic.

Prerequisites

• Python 3.10+ and a Typer or Click CLI with a --json output mode, as in emitting JSON output for scripting.
• An error class hierarchy, or willingness to add one.

Where errors go

The rule that makes everything else simple: stdout carries results and nothing else, in JSON mode as in human mode. The error document goes to stderr, and the exit code is non-zero. A script therefore never has to guess whether the JSON on stdout is a result or an error — if the exit code is 0, stdout is the result; if not, stderr holds the error document and stdout is empty.

Some tools put errors on stdout in JSON mode, wrapped in an envelope such as {"ok": false, "error": ...}. That works if every consumer checks the envelope, but it breaks the idiom mytool --json list | jq ... — jq happily processes the error object as if it were data. Keeping errors on stderr also means the same stream discipline applies in both modes, which is what users of structured JSON logs already expect.

The error document

The document has one top-level key, error, so it is unambiguous even when stderr also carries log lines. Inside it:

• code is the contract: a short, stable, lowercase identifier such as not_found or auth_required. Scripts branch on it. It never changes wording, unlike the message.
• message and hint are the same sentences a human sees, so a script can pass them on to its own user.
• exit_code repeats the process exit status, so a captured error log is self-contained.
• details holds the values involved ({"site": "nope"}), so a script does not have to parse them out of the message.
• schema_version is bumped only when the document changes incompatibly.

The recipe

Each error class gets a code next to its exit code, and accepts details as keyword arguments:

# src/mytool/errors.py
from __future__ import annotations


class MytoolError(Exception):
    """An expected failure. `code` is a stable identifier scripts can branch on."""

    code = "error"
    exit_code = 1

    def __init__(self, message: str, *, hint: str | None = None, **details: object) -> None:
        super().__init__(message)
        self.message = message
        self.hint = hint
        self.details = details


class NotFound(MytoolError):
    code = "not_found"


class AuthError(MytoolError):
    code = "auth_required"
    exit_code = 77

One small module builds the document and writes it in either format:

# src/mytool/report.py
from __future__ import annotations

import json
import sys
from typing import Any

SCHEMA_VERSION = 1


def error_payload(code: str, message: str, exit_code: int, *,
                  hint: str | None = None, details: dict[str, Any] | None = None) -> dict[str, Any]:
    return {"error": {"code": code, "message": message, "hint": hint,
                      "exit_code": exit_code, "details": details or {},
                      "schema_version": SCHEMA_VERSION}}


def report(payload: dict[str, Any], *, as_json: bool) -> None:
    """Write one error to stderr: a JSON document or two human lines."""
    err = payload["error"]
    if as_json:
        print(json.dumps(payload), file=sys.stderr)
        return
    print(f"error: {err['message']}", file=sys.stderr)
    if err["hint"]:
        print(f"hint: {err['hint']}", file=sys.stderr)

Commands raise errors with details and never format them:

# src/mytool/cli.py (command)
@app.command()
def show(name: str) -> None:
    """Show one site."""
    if name == "secret":
        raise AuthError("this site needs a token", hint='run "mytool auth login"')
    if name not in SITES:
        raise NotFound(f'site "{name}" does not exist', hint='run "mytool site list"', site=name)
    typer.echo(json.dumps(SITES[name]))

Covering parse errors and bugs too

The hard part is the failures that happen before or outside your commands: a missing argument is detected by the parser before the --json option has been processed, and a bug can happen anywhere. The entry point handles both by deciding the output mode from the raw arguments first, and running the app with standalone_mode=False so that parse errors reach it as exceptions:

# src/mytool/cli.py (entry point)
def wants_json(argv: list[str]) -> bool:
    """Decide the error format before parsing, so parse errors can use it too."""
    return "--json" in argv or os.environ.get("MYTOOL_OUTPUT") == "json"


def is_framework_error(exc: BaseException) -> bool:
    """Click's usage and file errors, whichever copy of Click raised them."""
    return isinstance(getattr(exc, "exit_code", None), int) and hasattr(exc, "format_message")


def run(argv: list[str] | None = None) -> None:
    argv = sys.argv[1:] if argv is None else argv
    as_json = wants_json(argv)
    try:
        code = app(args=argv, prog_name="mytool", standalone_mode=False)
    except MytoolError as exc:
        report(error_payload(exc.code, exc.message, exc.exit_code, hint=exc.hint,
                             details=exc.details), as_json=as_json)
        sys.exit(exc.exit_code)
    except typer.Abort:
        report(error_payload("interrupted", "interrupted", 130), as_json=as_json)
        sys.exit(130)
    except Exception as exc:  # noqa: BLE001
        if is_framework_error(exc):
            name = "usage" if exc.exit_code == 2 else "error"
            report(error_payload(name, exc.format_message(), exc.exit_code,
                                 hint="see 'mytool --help'"), as_json=as_json)
            sys.exit(exc.exit_code)
        report(error_payload("internal", f"{type(exc).__name__}: {exc}", 70,
                             hint="this is a bug; please report it"), as_json=as_json)
        sys.exit(70)
    sys.exit(code or 0)

Three details matter here. The mode is decided from argv and the environment, not from the parsed option, because a parse error means the option was never parsed. Framework errors are recognised by shape, not by class: recent Typer releases raise exceptions from their own bundled copy of Click, so isinstance(exc, click.UsageError) with the standalone click package can miss them, while checking for exit_code and format_message works with any version. And standalone_mode=False returns the exit code for --help and typer.Exit, which the last line passes on.

A script can now branch on the reason for a failure without parsing prose:

UX considerations

• Codes are an API. Document them next to the exit codes, add new ones freely, and never rename or reuse one; a script matching not_found must keep working. Treat a change as breaking, per semantic versioning policy for CLI tools.
• One document per failure. Print exactly one JSON line for the error, so jq or json.loads can read it without framing. If logs also go to stderr in JSON mode, they have different top-level keys, and consumers pick out error.
• An environment variable for wrappers. MYTOOL_OUTPUT=json lets a CI system or an editor plugin request JSON everywhere without editing every command line.
• Keep messages human. The message in JSON mode is the same sentence a person would read; do not make it terser because a machine is listening. Scripts often show it to their own users.
• No tracebacks in the document. Bugs report their type and message; the traceback belongs in the debug log, as in friendly error messages and tracebacks.

Testing the behaviour

The tests run the real entry point and check both streams and the exit code for every class of outcome — success, expected error, usage error, environment-selected JSON, human mode, a bug, and --help:

# tests/test_json_errors.py
import json

import pytest

from mytool import cli


def invoke(argv, capsys):
    with pytest.raises(SystemExit) as info:
        cli.run(argv)
    captured = capsys.readouterr()
    return info.value.code, captured.out, captured.err


def test_success_writes_only_data(capsys):
    code, out, err = invoke(["--json", "show", "web"], capsys)
    assert code == 0 and json.loads(out) == {"name": "web", "status": "live"} and err == ""


def test_expected_error_is_one_json_document_on_stderr(capsys):
    code, out, err = invoke(["--json", "show", "nope"], capsys)
    assert code == 1 and out == ""
    error = json.loads(err)["error"]
    assert error["code"] == "not_found" and error["details"] == {"site": "nope"}
    assert error["exit_code"] == code


def test_usage_errors_are_json_too(capsys):
    code, out, err = invoke(["--json", "show"], capsys)
    error = json.loads(err)["error"]
    assert code == 2 and error["code"] == "usage" and "'name'" in error["message"]


def test_env_var_selects_json(capsys, monkeypatch):
    monkeypatch.setenv("MYTOOL_OUTPUT", "json")
    code, _, err = invoke(["show", "secret"], capsys)
    assert code == 77 and json.loads(err)["error"]["code"] == "auth_required"


def test_humans_get_text(capsys):
    code, _, err = invoke(["show", "nope"], capsys)
    assert err == 'error: site "nope" does not exist\nhint: run "mytool site list"\n'


def test_bugs_are_reported_as_internal(capsys, monkeypatch):
    monkeypatch.setattr(cli, "SITES", None)
    code, _, err = invoke(["--json", "show", "web"], capsys)
    assert code == 70 and json.loads(err)["error"]["code"] == "internal"


def test_help_still_exits_0(capsys):
    code, out, _ = invoke(["--help"], capsys)
    assert code == 0 and "Manage sites" in out

json.loads(err) doubles as an assertion: if anything other than one JSON document reached stderr — a stray warning, a Rich-formatted usage box — the test fails with a decode error. The usage-error test is the one most implementations fail the first time, because the parser rejects the command before any code that knows about --json has run.

Conclusion

A JSON mode is only as useful as its worst failure. Report every error — your own, the parser's and genuine bugs — as one JSON document on stderr with a stable code, the human message and hint, the exit code and structured details; keep stdout for results only; and decide the mode from the raw arguments and an environment variable so that even parse errors honour it. Implement it once in the entry point, pin it with tests for each kind of outcome, and scripts can finally tell "not found" from "not logged in" without reading English.

Frequently asked questions

Related

• Up: Error handling and exit codes
• Designing an exception hierarchy for a CLI
• Choosing exit codes for CLI tools
• Emitting JSON output for scripting
• Detecting CI environments and non-interactive shells