Testing Shell Completion in Python CLIs

Shell completion is the feature most likely to be broken without anyone noticing. It is invisible in --help, nobody presses Tab in CI, and a refactor that renames an option, moves a command into a group or changes a callback's signature leaves completion silently returning nothing. Users notice weeks later, assume completion "just doesn't work" for your tool, and stop trying. Completion is also easy to test once you know how it works: the shell runs your program with a few environment variables set, and your program prints the candidates and exits. This guide tests completion at three levels — the callback, the completion protocol, and a real bash session — shows the differences between Typer's and Click's protocols that trip up most tests, and explains which level is worth running where. It belongs to the shell completion for Python CLIs topic.

Prerequisites

• A Typer or Click CLI with completion enabled, as in enabling tab completion in Click and Typer.
• pytest; bash for the optional end-to-end level.

How completion works

When you press Tab, the shell calls a small function that was installed by the completion script. That function runs your CLI with an environment variable named after the program — _MYTOOL_COMPLETE for mytool — plus variables describing the words typed so far. Your CLI notices the variable, parses the words, works out whether the cursor is on a command, an option or an option's value, calls the matching completion callback, prints one candidate per line and exits without running any command. The shell function turns those lines into the completion menu.

A test can play the part of the shell function: set the same variables, run the CLI, read the lines.

Three levels of test

Callbacks are plain functions, so test them directly for their filtering and caching logic, as in dynamic completion values from APIs and files. The protocol level runs the whole CLI through the framework's completion code, which catches wiring mistakes — a callback attached to the wrong option, a renamed command, a signature the framework no longer calls correctly. The real shell level sources the generated script into bash and checks COMPREPLY, which catches problems in the script itself and in how the program is installed.

The app under test

A small Typer app with two commands and a dynamic value for --env:

# src/mytool/cli.py
from __future__ import annotations

import typer

app = typer.Typer()

ENVIRONMENTS = {"prod-eu": "Production, Frankfurt", "prod-us": "Production, Virginia",
                "staging": "Staging"}


def complete_env(incomplete: str) -> list[tuple[str, str]]:
    return [(name, help) for name, help in ENVIRONMENTS.items() if name.startswith(incomplete)]


@app.callback()
def main() -> None:
    """Deploy things."""


@app.command()
def deploy(env: str = typer.Option(..., "--env", autocompletion=complete_env),
           force: bool = typer.Option(False, "--force")) -> None:
    """Deploy to ENV."""


@app.command()
def status() -> None:
    """Show status."""

The recipe: testing the protocol

Typer's protocol uses complete_bash, complete_zsh and complete_fish as values of _MYTOOL_COMPLETE. Bash passes the words in COMP_WORDS and the cursor position in COMP_CWORD; zsh and fish use _TYPER_COMPLETE_ARGS. A helper sets all of them and runs the app through CliRunner:

# tests/test_completion_typer.py
from typer.testing import CliRunner

from mytool.cli import app

runner = CliRunner()


def complete(line: str, shell: str = "bash") -> list[str]:
    """Ask the app for completions the way the shell script does, and return the lines."""
    words = line.split(" ")
    env = {"_MYTOOL_COMPLETE": f"complete_{shell}",
           "COMP_WORDS": line, "COMP_CWORD": str(len(words) - 1),   # bash
           "_TYPER_COMPLETE_ARGS": line}                               # zsh, fish
    if shell == "fish":
        env["_TYPER_COMPLETE_FISH_ACTION"] = "get-args"
    result = runner.invoke(app, [], env=env, prog_name="mytool")
    assert result.exit_code == 0, result.output
    return [line for line in result.output.splitlines() if line]   # bash ends with a blank line


def test_subcommands():
    assert complete("mytool ") == ["deploy", "status"]


def test_options_of_a_command():
    assert {"--env", "--force"} <= set(complete("mytool deploy --"))


def test_dynamic_values_are_filtered_by_prefix():
    assert complete("mytool deploy --env prod-") == ["prod-eu", "prod-us"]


def test_fish_gets_descriptions():
    assert complete("mytool deploy --env st", shell="fish") == ["staging\tStaging"]


def test_no_match_means_no_candidates():
    assert complete("mytool deploy --env zz") == []

Two details make or break these tests. prog_name="mytool" matters because the environment variable's name is derived from the program name; without it, the runner's program name is something else, the variable is ignored, and the CLI runs normally instead of completing. A trailing space in "mytool " means "complete a new word"; without it, the cursor is still on mytool itself.

The same checks by hand, which is also the quickest way to debug a failing test:

Plain Click has a public API for this

Click's protocol uses different values (bash_complete, zsh_complete, fish_complete), and it also exposes the machinery directly. ShellComplete.get_completions returns completion items with their help text, without environment variables or output parsing. Here mytool.clickcli is the same app written with Click, its --env option declared with shell_complete=complete_env and the callback returning CompletionItem(name, help=help) objects:

# tests/test_completion_click.py
from click.shell_completion import ShellComplete

from mytool.clickcli import cli


def completions(args: list[str], incomplete: str) -> list[tuple[str, str | None]]:
    comp = ShellComplete(cli, {}, "mytool", "_MYTOOL_COMPLETE")
    return [(item.value, item.help) for item in comp.get_completions(args, incomplete)]


def test_values_with_help():
    assert completions(["deploy", "--env"], "prod-") == [
        ("prod-eu", "Production, Frankfurt"), ("prod-us", "Production, Virginia")]


def test_commands():
    assert [v for v, _ in completions([], "")] == ["deploy", "status"]

Do not use this API with a Typer app: recent Typer releases build their command objects from their own bundled copy of Click, so standalone Click's completion classes are not the ones Typer runs. The environment-variable protocol above works for both, as long as the values match the framework.

The real shell, once

The protocol tests prove your Python side. What they cannot prove is that the script mytool --show-completion bash generates, sourced into a real shell, calls the installed program correctly. One end-to-end test covers that; it needs the CLI installed on PATH, so skip it where it is not:

# tests/test_completion_bash.py
import shutil
import subprocess

import pytest

pytestmark = pytest.mark.skipif(not shutil.which("mytool") or not shutil.which("bash"),
                                reason="needs the installed CLI and bash")

SCRIPT = """
source <(mytool --show-completion bash)
COMP_WORDS=({words})
COMP_CWORD={cword}
_mytool_completion mytool
printf '%s\\n' "${{COMPREPLY[@]}}"
"""


def bash_complete(*words: str) -> list[str]:
    script = SCRIPT.format(words=" ".join(words), cword=len(words) - 1)
    out = subprocess.run(["bash", "-c", script], capture_output=True, text=True, check=True).stdout
    return [line for line in out.splitlines() if line]


def test_real_bash_completes_values():
    assert bash_complete("mytool", "deploy", "--env", "prod-") == ["prod-eu", "prod-us"]

The function name _mytool_completion comes from the generated script; if you change the program name, the test fails loudly rather than silently. Run it in the job that installs the built wheel, as in end-to-end testing an installed CLI.

UX considerations

• Completion must be fast. A test that times the protocol call (time.perf_counter() around complete(...)) and fails above, say, 300 ms catches the import that made Tab feel sluggish; see CLI startup performance and lazy loading.
• Completion must be quiet. Anything a callback prints — a warning, a log line on stderr — can end up in the user's prompt. The protocol tests check stdout exactly; add an assertion that stderr is empty if your callbacks do I/O.
• Failing sources return nothing. A callback that raises breaks the whole completion; test that an unreachable API gives an empty list, not an exception.
• Hidden commands stay hidden. If you hide aliases from --help, assert they are absent from command completion too.

Conclusion

Completion breaks silently, so it needs tests more than most features — and it is easy to test, because the protocol is just environment variables in and lines out. Test callbacks as plain functions, test the protocol through CliRunner with prog_name set and the framework's own values (complete_bash for Typer, bash_complete for Click), and run one real-bash test against the installed CLI in CI. Together they keep Tab working through every refactor.

Frequently asked questions

Related

• Up: Shell completion for Python CLIs
• Enabling tab completion in Click and Typer
• Dynamic completion values from APIs and files
• Installing shell completion for bash, zsh and fish
• Testing Python CLI applications