Python CLI Toolcraft

Architecture

Reducing CLI Dependency Weight

Cut a Python CLI's import cost - audit the dependency tree, move heavy packages behind optional extras and lazy imports, and enforce the win in CI.

Updated

Every dependency you add is paid for on every invocation, including --help and every keystroke of tab completion. Most of that cost is avoidable — not by removing libraries, but by deciding which ones load at start-up and which load only when a command actually needs them.

TL;DR

  • Measure first: python -X importtime -m mytool --help and sort by the cumulative column.
  • Anything imported at module level costs on every invocation.
  • Move heavy, single-command dependencies behind an optional extra plus a lazy import.
  • Keep the top-level __init__.py empty — it runs on every import of anything in the package.
  • Enforce the result with a start-up budget test, or it regresses within a month.

Prerequisites

A working CLI and the measurement habits from profiling Python CLI startup time.

What each dependency actually costs

What each dependency costs at start-up A bar chart of the import cost of common command line dependencies, from a lightweight parser to a scientific stack. What each dependency costs at start-up click 12 ms typer + rich 48 ms httpx 70 ms pandas 310 ms Indicative import times measured with python -X importtime on a warm cache, Python 3.12. The first two are the price of a good interface; the last is a price only one command should pay.

The numbers are worth internalising because they set expectations. A parser costs ten milliseconds. A parser plus a rendering library costs fifty. An HTTP client costs about as much again. A scientific stack costs several hundred, and it is usually the only one that matters.

Against a 20 ms interpreter floor, that means a CLI with Typer, Rich and httpx starts in roughly 130 ms — noticeable but tolerable — and the same tool with a top-level import pandas starts in 440 ms, which is where tab completion begins to feel broken.

Auditing the tree

Auditing what your tool pulls in A dependency audit: list the resolved tree, measure import cost, and decide for each package whether it is runtime, optional or removable. Auditing what your tool pulls in uv tree what is actually installed -X importtime what each costs Classify runtime, extra, or gone Re-measure confirm the win inventory measure act The classification step is the work; measuring before and after is what keeps it honest.

Three commands give you the whole picture:

uv tree                                     # what is actually installed, and why
python -X importtime -m mytool --help 2>&1 | sort -k2 -rn | head -15
uv run --with tuna python -X importtime -m mytool --help 2> import.log && tuna import.log

uv tree answers "why is this here" — often the surprise is transitive: a small utility package pulling in a large one. The importtime output answers "what does it cost", sorted by cumulative microseconds so the top line is the subtree worth attacking.

Then classify every top-level import in the start-up path into one of three buckets: needed on every invocation, needed by one command, or not needed at all. The third bucket is smaller than people hope and never empty.

Moving a dependency out of the start-up path

Runtime dependency or optional extra? Guidance for deciding whether a package belongs in the runtime dependency list or behind an optional extra. Runtime dependency or optional extra? Make it an extra ✓ Only one command imports it ✓ It is large or slow to import ✓ It has heavy transitive dependencies ✓ A useful subset of users will never need it Keep it runtime ✗ The parser and output layer ✗ Anything imported at start-up ✗ Something every command path touches ✗ A tiny pure-Python helper An extra plus a lazy import plus a helpful error is three small changes that remove a large cost.

The pattern has three parts, and all three are needed for it to be pleasant.

Declare it as an extra, so a user who never runs that command never installs it:

[project]
dependencies = ["typer>=0.12", "rich>=13.7"]

[project.optional-dependencies]
analysis = ["pandas>=2.2"]
aws = ["boto3>=1.35"]

Import it inside the function that needs it:

@app.command()
def report(period: str) -> None:
    """Summarise activity for PERIOD."""
    import pandas as pd          # 310 ms, and only this command needs it

    frame = pd.DataFrame(core.collect(period))
    ...

Fail with an instruction when the extra is missing, because the default ImportError tells the user nothing about how to fix it:

def _load_pandas():
    try:
        import pandas
    except ImportError:  # pragma: no cover - depends on install extras
        raise typer.BadParameter(
            "the report command needs the analysis extra: pipx install 'mytool[analysis]'"
        ) from None
    return pandas

Three small changes, and the cost of the heaviest dependency moves from every invocation to the one command that needs it — and from every install to the users who asked for it.

The two structural wins

Beyond individual dependencies, two arrangements matter more than any single import.

Keep __init__.py empty. The top-level package module runs on every import of anything inside the package, including the entry point. A convenient re-export there — from .core.sync import sync_directory — quietly makes every invocation import the whole subtree. Limit it to __version__, or leave it empty entirely.

Resolve command modules on demand. Once the obvious dependencies are deferred, what remains is importing thirty command modules to build --help. A registry of dotted paths resolved when a command is chosen removes it, and is covered in lazy loading subcommands.

What you give up

Lazy imports have a real cost, and it is worth naming rather than discovering.

Import errors move. A broken dependency no longer fails at start-up; it fails when someone runs that command. Keep an import-all test in CI so the failure surfaces in the pull request instead:

def test_every_command_module_imports():
    package = importlib.import_module("mytool.commands")
    for module in pkgutil.iter_modules(package.__path__):
        importlib.import_module(f"mytool.commands.{module.name}")

Linters complain. A non-top-level import is flagged by default in most rule sets. Configure the exception rather than fighting it, and put a short comment on each deferred import saying what it costs — that comment is also the reason nobody moves it back.

Coverage shifts. Lazily imported modules are not measured unless a test invokes that command, which is another reason the import-all test earns its place.

UX considerations

Advertise the extras. A user who never learns mytool[analysis] exists concludes the report command is broken. Name the extra in the error, in the README install section, and in the command's help text.

Keep the base install useful. An extra should gate a genuinely optional capability, not the main one. If the most common command needs an extra, the split is in the wrong place.

Do not split too finely. Five extras is a matrix of install combinations you now support and test. Two or three, chosen around real user groups, is usually the right granularity.

Testing the behaviour

A budget test turns a one-off improvement into something that stays true:

import subprocess, sys, time

BUDGET_MS = 150

def test_help_starts_within_budget():
    best = min(_time_help() for _ in range(5))
    assert best < BUDGET_MS, f"--help took {best:.0f} ms (budget {BUDGET_MS} ms)"

def _time_help() -> float:
    start = time.perf_counter()
    subprocess.run([sys.executable, "-m", "mytool", "--help"], capture_output=True, check=True)
    return (time.perf_counter() - start) * 1000

Take the best of several runs rather than an average — you are measuring a floor, and CI runners are noisy. Set the budget slightly above your current number so it fails on a regression rather than on a bad afternoon, and lower it when the figure improves.

A second, sharper test asserts that specific modules are not imported at start-up:

def test_heavy_dependencies_are_not_imported_for_help():
    code = "import sys, mytool.cli; print('pandas' in sys.modules)"
    out = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True)
    assert out.stdout.strip() == "False"

That one names the offender directly, which makes the failure immediately actionable.

Conclusion

Measure, classify, then move: an extra for the packaging, a lazy import for the start-up cost, and a helpful error so the split is invisible to users who need the feature. Add a budget test and an import-all test, and the improvement survives the next six months of dependency additions.

Frequently asked questions

Is Rich too heavy for a CLI?

For most tools, no — roughly 40 ms buys tables, progress, colour and correct behaviour when output is redirected, all of which you would otherwise write. If you need --help under 60 ms, import it inside the rendering functions rather than dropping it.

Should I vendor a small dependency instead of depending on it?

Almost never. Vendoring means you own the security updates and the bug reports, and the import cost is the same either way. The exception is a handful of lines you would otherwise pull a whole package for — and even then, copying with attribution beats a dependency only if the code is genuinely stable.

How do I find which dependency pulled in a heavy transitive one?

uv tree shows the graph with the requester of each package; pipdeptree --reverse does the same for a pip environment. The usual discovery is that a convenience library depends on something substantial, and replacing the convenience removes both.

Do lazy imports slow down the command that uses them?

Marginally — the import happens on first use instead of at start-up, so that one command pays what every command used to. That is the trade, and it is nearly always worth it: the heavy command was going to do heavy work anyway.

What is a realistic target after this work?

Under 100 ms for --help on a tool with a normal dependency set. Getting below the 40 ms range usually means dropping the rendering library, which is a trade most tools should not make.

Does removing a dependency ever make things worse?

It can. Replacing a well-tested library with two hundred lines of your own moves the cost from import time to maintenance time, and your version will have the bugs theirs already fixed. The dependencies worth removing are the ones you barely use — a whole HTTP library for two GET requests, a date library for one format string — not the ones doing real work.