Usage-String Driven Python CLIs with docopt-ng

docopt starts from a simple observation: every command-line tool already has a precise description of its interface — the usage message — so why write that description a second time as parser code? With docopt you write the help text in a conventional format, and the library turns it into a parser. The help can never drift from the behaviour, because it is the behaviour. The original docopt package is no longer maintained; docopt-ng is the maintained fork, a drop-in replacement with type hints and fixes. This guide builds a small backup tool with it, adds the validation layer docopt leaves to you, fixes its exit code, and tests the result. It belongs to the alternative Python CLI frameworks topic.

Prerequisites

• Python 3.10+ and docopt-ng (uv add docopt-ng; you still from docopt import docopt).
• Familiarity with POSIX usage syntax — [optional], (required | alternatives), <argument>, ... for repetition. Following POSIX and GNU argument conventions is a good refresher.

How a usage string becomes a parser

docopt reads two sections of the docstring. Lines under Usage: are patterns: each line is one valid shape of the command line. Lines under Options: declare each option's short and long forms, whether it takes a value, and its default. Everything else — the description, examples — is free text that only appears in help.

Parsing returns a dictionary with one key per command word, argument and option across all patterns. Command words are True/False, arguments and option values are strings or None, repeated elements are lists, and flags are booleans (or counts, for repeatable flags such as -v...).

The recipe

# src/backup/cli.py
"""Back up directories to a destination.

Usage:
  backup run <source>... --to=<dest> [--compress=<level>] [--exclude=<glob>]... [--dry-run]
  backup list [--limit=<n>] [--json]
  backup -h | --help
  backup --version

Options:
  -h --help            Show this help.
  --version            Show the version.
  --to=<dest>          Destination directory.
  --compress=<level>   Compression level, 0-9 [default: 6].
  --exclude=<glob>     Skip paths matching this glob; repeatable.
  --limit=<n>          Show at most n backups [default: 10].
  --json               Print JSON instead of a table.
  --dry-run            Show what would be copied without copying.

Examples:
  backup run ~/docs ~/photos --to=/mnt/backup --exclude='*.tmp'
  backup list --limit=3
"""
from __future__ import annotations

import sys
from dataclasses import dataclass, field
from pathlib import Path

from docopt import DocoptExit, docopt

__version__ = "1.0.0"


class UsageError(Exception):
    pass


@dataclass(frozen=True)
class RunArgs:
    sources: list[Path]
    dest: Path
    compress: int
    exclude: list[str] = field(default_factory=list)
    dry_run: bool = False


def _int_in_range(raw: str, name: str, lo: int, hi: int) -> int:
    try:
        value = int(raw)
    except ValueError:
        raise UsageError(f"{name} must be a whole number, got {raw!r}") from None
    if not lo <= value <= hi:
        raise UsageError(f"{name} must be between {lo} and {hi}, got {value}")
    return value


def to_run_args(args: dict) -> RunArgs:
    return RunArgs(
        sources=[Path(s) for s in args["<source>"]],
        dest=Path(args["--to"]),
        compress=_int_in_range(args["--compress"], "--compress", 0, 9),
        exclude=list(args["--exclude"]),
        dry_run=bool(args["--dry-run"]),
    )


def parse(argv: list[str] | None = None) -> dict:
    """Parse argv; exit 2 on a usage error, like argparse and Click."""
    try:
        return docopt(__doc__, argv=argv, version=f"backup {__version__}")
    except DocoptExit as exc:
        print(str(exc), file=sys.stderr)
        raise SystemExit(2) from None


def main(argv: list[str] | None = None) -> int:
    args = parse(argv)
    try:
        if args["run"]:
            run = to_run_args(args)
            verb = "would copy" if run.dry_run else "copying"
            for src in run.sources:
                print(f"{verb} {src} -> {run.dest} (level {run.compress})")
        elif args["list"]:
            limit = _int_in_range(args["--limit"], "--limit", 1, 1000)
            print(f"showing up to {limit} backups")
    except UsageError as exc:
        print(f"error: {exc}", file=sys.stderr)
        return 2
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The structure has three layers. parse() turns argv into docopt's dictionary and owns the exit-code fix. to_run_args() turns that untyped dictionary into a typed, validated dataclass — the layer docopt deliberately does not provide. And main() dispatches on command words and calls into the real work. Taking argv as a parameter is what makes the whole thing testable.

Why the exit code needs fixing

When the command line matches no pattern, docopt-ng raises DocoptExit, a SystemExit subclass whose code is the usage message. Python prints that message and exits with status 1 — the same status as "the backup failed". Click, Typer, argparse and most Unix tools use 2 for usage errors, and scripts rely on the difference; choosing exit codes for CLI tools explains why. Catching DocoptExit in one place restores the convention.

Pattern syntax worth knowing

Most docopt surprises come from a handful of syntax rules. Knowing them saves a lot of trial and error:

• [options] in a pattern is a shortcut for "any option from the Options section". It keeps patterns short but loosens validation, because every option becomes legal with every command. Spelling options out per pattern, as above, is stricter.
• ... repeats the element before it. <source>... means one or more sources; [<source>...] means zero or more. Repeated options ([--exclude=<glob>]...) arrive as lists.
• Parentheses group required alternatives. (--json | --csv) requires exactly one; [--json | --csv] allows at most one. This is docopt's answer to mutually exclusive options, which argparse handles with groups — compare mutually exclusive options in argparse.
• Option descriptions need two spaces. In the Options section, the option forms and the description must be separated by at least two spaces, or docopt reads the description as part of the option.
• Counting flags. A pattern element such as -v... makes the value an integer count, so -vvv gives 3 — handy for verbosity levels.
• -- ends options. Anything after a bare -- is treated as positional, which lets users pass file names that begin with a dash.

When a pattern does not behave as expected, print the parsed dictionary for a few sample command lines. Because the result contains every key from every pattern, it shows exactly which elements matched.

UX considerations

• Put examples in the docstring. They cost nothing — docopt ignores sections other than Usage: and Options: — and users read examples first. See adding examples and epilogs to help output.
• Write [default: …] in the options section. docopt fills in the default and the help shows it, with one source of truth.
• Be careful with pattern ambiguity. Two patterns that can match the same tokens produce confusing results. Keep one pattern per command word and let options be optional within it.
• Validate everything that is not a string. Numbers, ranges, paths that must exist and mutually exclusive options all belong in the conversion layer. The rule from advanced argument validation strategies applies: validate once, at the boundary, and pass typed values inward.
• Error messages are terse. docopt prints the usage section and nothing else for a non-matching command line. Keep the usage section short enough that this is helpful.

Testing the behaviour

Because main() and parse() accept argv, tests call them directly. Test the parsed dictionary for each pattern, the conversion layer for each rule, and the exit codes:

# tests/test_cli.py
import pytest

from backup import cli


def test_run_pattern_collects_repeated_values():
    args = cli.parse(["run", "a", "b", "--to=/mnt", "--exclude=*.tmp", "--exclude=*.log"])
    assert args["run"] and args["<source>"] == ["a", "b"]
    assert args["--exclude"] == ["*.tmp", "*.log"]
    assert args["--compress"] == "6"                     # default, still a string


def test_conversion_produces_typed_values():
    run = cli.to_run_args(cli.parse(["run", "a", "--to=/mnt", "--compress=9"]))
    assert run.compress == 9 and str(run.dest) == "/mnt"


@pytest.mark.parametrize("level", ["10", "-1", "fast"])
def test_bad_compress_level_is_a_usage_error(level, capsys):
    assert cli.main(["run", "a", "--to=/mnt", f"--compress={level}"]) == 2
    assert "--compress" in capsys.readouterr().err


def test_unmatched_command_line_exits_2(capsys):
    with pytest.raises(SystemExit) as exc:
        cli.main(["restore"])
    assert exc.value.code == 2
    assert "Usage:" in capsys.readouterr().err


def test_dry_run_output(capsys):
    assert cli.main(["run", "a", "--to=/mnt", "--dry-run"]) == 0
    assert capsys.readouterr().out == "would copy a -> /mnt (level 6)\n"

A useful extra test asserts that every example line in the docstring parses — cheap insurance that documentation and parser stay in step:

# tests/test_cli.py (continued)
import shlex


def test_docstring_examples_parse():
    examples = [line.strip() for line in cli.__doc__.split("Examples:")[1].splitlines()
                if line.strip().startswith("backup ")]
    for line in examples:
        cli.parse(shlex.split(line)[1:])

Migrating an old docopt project

Many long-lived internal tools were written with the original docopt around 2014–2018 and still run today. Moving them to docopt-ng is usually a one-line dependency change — replace docopt with docopt-ng in pyproject.toml and keep the imports — followed by running the test suite on current Python. If there is no test suite, the test_docstring_examples_parse test above is a fast way to build one: write down the command lines people actually use, assert each one parses, and you have a safety net before touching anything else. From there, adding the typed conversion layer and the exit-code wrapper can be done one command at a time.

Conclusion

docopt-ng is at its best for small tools whose interface is stable and whose authors want the help text to be the single source of truth. Write clear usage patterns and an options section with defaults, then add what docopt leaves out: a typed conversion layer, exit code 2 for usage errors and a main(argv) that tests can call. When the tool grows many commands, rich types or completion needs, a declarative framework such as Cyclopts or Typer will carry it further.

Frequently asked questions

Related

• Up: Alternative Python CLI frameworks
• Quick CLIs from functions with Python Fire
• Building a CLI with Cleo
• Writing help text users actually read
• Argparse subparsers for subcommands