Python CLI Toolcraft

Project Setup

Speeding Up Slow pre-commit Hooks in a CLI Repository

Make pre-commit fast enough that nobody skips it: measure per-hook durations, filter files, move slow checks to pre-push or manual stages, fix mypy hooks and try prek.

Updated

A pre-commit setup has a time budget, and it is short. When a commit takes two seconds, nobody notices. When it takes twenty, people start typing git commit --no-verify, and the hooks protect nothing. Slow pre-commit runs usually have a handful of causes — a type checker run over the whole project on every commit, hooks scanning files they do not care about, a hook environment rebuilt more often than necessary, or a single test-running hook someone added in a moment of enthusiasm. Each has a targeted fix. This guide measures where the time goes, applies those fixes to a Python CLI repository, and shows how to keep the checks that are too slow for every commit without losing them. It belongs to the pre-commit topic.

Prerequisites

Measure first

--verbose prints each hook's duration, which is all the profiling you need:

pre-commit run --all-files --verbose 2>&1 | grep -E "^\S.*\.{3}|duration"

ruff check...............................................................Passed
- duration: 0.01s
ruff format..............................................................Passed
- duration: 0.01s
mypy.....................................................................Passed
- duration: 6.84s
pytest (fast).............................................................Passed
- duration: 11.2s

Run it twice: the first run after a change of hook versions includes building hook environments, which is a one-off cost; the second shows the steady state your colleagues live with. In almost every Python repository, the list is dominated by one or two hooks — usually the type checker and anything that runs tests.

Where commit time goes Example per-hook durations from pre-commit verbose output, showing type checking and tests dominating while linters take milliseconds. Where commit time goes ruff check 0.01 s ruff format 0.01 s mypy (whole project) 6.84 s pytest (fast subset) 11.2 s example figures from a mid-sized CLI; measure your own with --verbose Two hooks take 99% of the time — and neither needs to run on every commit.

The recipe

1. Run hooks only on files they care about

pre-commit passes each hook the staged files that match its filters. Hooks from good repositories set types: [python] already; local hooks often forget, and then run on every Markdown and YAML change too:

  - repo: local
    hooks:
      - id: cli-reference
        name: CLI reference is up to date
        entry: uv run python scripts/gen_cli_reference.py --check
        language: system
        files: ^src/mytool/(cli|commands)/.*\.py$
        pass_filenames: false

files (a regular expression), types/types_or (file kinds pre-commit detects) and exclude decide when a hook runs at all. A hook whose files pattern matches nothing in the commit is skipped instantly. Add a top-level exclude for vendored code, generated files and fixtures, so no hook wastes time on them.

2. Move slow checks to a later stage

Not every check must run on every commit. pre-commit supports several stages; the useful ones are pre-commit (the default), pre-push (when pushing — once per batch of commits) and manual (only when asked, typically in CI):

default_install_hook_types: [pre-commit, pre-push]
default_stages: [pre-commit]

repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.16.10
    hooks:
      - id: ruff-check
        args: [--fix]
      - id: ruff-format
  - repo: local
    hooks:
      - id: mypy
        name: mypy
        entry: uv run mypy
        language: system
        types: [python]
        require_serial: true
        stages: [pre-push]
      - id: pytest-fast
        name: fast tests
        entry: uv run pytest -q -m "not slow" -x
        language: system
        pass_filenames: false
        always_run: true
        stages: [manual]

Ruff stays on every commit — it takes milliseconds and fixes most of what it finds. mypy moves to pre-push, where a few seconds once per push is acceptable. Tests move to manual, run by CI as described in running pre-commit in CI and by developers on demand. default_install_hook_types makes pre-commit install set up both the commit and push hooks, so nobody has to remember the second command. default_stages matters just as much: a hook without its own stages key runs in every stage, so without it Ruff would run a second time on every push, alongside mypy. With default_stages: [pre-commit], unmarked hooks run on commit only, and the push runs just the checks that were moved there.

Which check runs when Assigning pre-commit hooks to stages: fast fixers on commit, type checking on push, tests in the manual stage run by CI. Which check runs when pre-commit stage every commit Ruff check --fix, Ruff format, whitespace fixers — milliseconds pre-push stage every push mypy from the project environment — once per push manual stage on demand fast tests, CLI reference check — CI runs these default_stages keeps unmarked hooks from running again on push.

3. Fix the mypy hook

The popular mirrors-mypy hook runs mypy in an isolated environment that contains only mypy — not your project's dependencies. mypy then cannot see Typer, httpx or Rich, treats them as Any (or reports missing imports), and you add a growing list of additional_dependencies that drifts from your lock file. It is slower and less accurate. A local hook with language: system and entry: uv run mypy uses the project's own environment instead, so mypy sees exactly the dependency versions you test with, and its incremental cache in .mypy_cache is shared with your editor and normal runs.

Two settings matter for whole-project tools. require_serial: true stops pre-commit splitting the file list across parallel processes, which would make mypy run several times with partial views of the code. And for checkers that should always analyse the whole package regardless of which files changed, set pass_filenames: false and let the tool's own configuration decide what to check.

4. Keep environments warm

pre-commit builds one environment per hook repository and revision, cached in ~/.cache/pre-commit. Rebuilds happen when a rev changes or the cache is cleared — slow, but rare. Avoid making them frequent: do not reference rev: main or other moving targets (pre-commit re-clones them), and batch hook version updates rather than updating one hook a day. In CI, cache that directory keyed on the config file.

5. Try prek

prek is a reimplementation of pre-commit in Rust that reads the same .pre-commit-config.yaml, installs hook environments with uv and runs hooks in parallel where it can. On a small test repository, a cold run (building the Ruff environment) took about 1.7 s with prek against 2.5 s with pre-commit, and warm runs were 0.05 s against 0.14 s. The difference is small for a few fast hooks and grows with the number of Python-based hook repositories. Because the configuration is shared, a team can try it without changing anything else: uvx prek install replaces the git hook, and uvx pre-commit install puts the original back.

UX considerations

  • Set a budget and say it out loud. "Commit hooks finish in under two seconds; anything slower goes to pre-push or CI" turns an argument into a rule.
  • Auto-fix on commit, check on push. Formatters and import sorters that fix files are pleasant on commit; checks that only report are better later, when the developer is ready to look.
  • Fail fast for expensive hooks. fail_fast: true at the top of the config stops at the first failing hook, which saves time when a formatter has already reported problems.
  • Make the escape hatch specific. SKIP=mypy git push skips one hook for one push; --no-verify skips everything. Document the former.
  • Keep CI as the safety net. Moving a check to manual is only safe if CI runs it on every pull request.
A commit that stays fast Terminal session where a commit runs only fast hooks and a push runs mypy. A commit that stays fast bash $ git commit -m "Add export command" ruff check...............................................Passed ruff format..............................................Passed $ git push mypy.....................................................Passed Fixers on commit, checkers on push, everything else in CI.

Testing the behaviour

Speed regressions creep in one hook at a time, so make the budget a check. A small script runs the commit-stage hooks against every file and fails if they exceed the budget:

# scripts/check_hook_budget.py
import subprocess
import sys
import time

BUDGET_SECONDS = 5.0

start = time.monotonic()
result = subprocess.run(["pre-commit", "run", "--all-files", "--hook-stage", "pre-commit"],
                        capture_output=True, text=True)
elapsed = time.monotonic() - start
print(result.stdout)
if elapsed > BUDGET_SECONDS:
    sys.exit(f"pre-commit stage took {elapsed:.1f}s, budget is {BUDGET_SECONDS:.0f}s; "
             "move slow hooks to pre-push or manual")
print(f"pre-commit stage took {elapsed:.1f}s (budget {BUDGET_SECONDS:.0f}s)")
sys.exit(result.returncode)

Running against all files gives a worst-case figure; real commits touch a few files and are faster. Run it in CI after the hook environments are cached, so environment building does not count against the budget.

Conclusion

Fast hooks are hooks people keep. Measure with --verbose, give every hook precise files and types filters, keep millisecond tools like Ruff on commit, move type checking to pre-push and tests to manual (with CI running them), run mypy from the project environment with require_serial, keep hook environments stable and cached, and consider prek for faster setup. Then put the budget in a script, so the next slow hook is caught before anyone reaches for --no-verify.

Frequently asked questions

Why not run the whole test suite as a hook?

Because it turns every commit into a test run, and the first thing people do is disable it. Tests belong in CI and in the developer's own workflow; a manual-stage hook gives a convenient shortcut without the tax.

Do pre-push hooks run in CI?

Only if CI asks for them: pre-commit run --all-files --hook-stage pre-push. Add that step (or run mypy separately) so pre-push checks are enforced, not just suggested.

Does pre-commit run hooks in parallel?

Within a hook, pre-commit splits the file list across processes unless require_serial is set. Different hooks run one after another. prek can run independent hooks concurrently, which is one source of its speed advantage.

My hook is fast locally but slow in CI — why?

Usually a cold cache: CI rebuilds hook environments because the cache key changed or caching is missing. Check the job log for "Installing environment" lines.