CI/CD Pipelines for Python CLIs

A command-line tool runs on machines you will never see: colleagues' Windows laptops, CI runners with an older Python, minimal containers, servers where it is installed by a configuration-management tool at 3 a.m. Your local test run proves the tool works in exactly one of those environments. Continuous integration is where it meets the rest, and continuous delivery is how a tested version reaches users without a maintainer running twine upload from their laptop and hoping they built from the right commit.

This topic covers the pipeline a Python CLI actually needs: linting and type-checking that fail fast, a test matrix across Python versions and operating systems, dependency caching so all of that stays quick, a smoke test of the built wheel rather than the source tree, releases driven by git tags, and publishing to PyPI with trusted publishing so no long-lived token exists anywhere. The examples use GitHub Actions and uv, but the structure transfers directly to GitLab CI, Buildkite or any other system.

TL;DR

• Stage the pipeline: lint and types first, then the test matrix, then build once, smoke-test the built wheel, and publish only on tags.
• Test where your users are: every supported Python on Linux, plus the oldest and newest on Windows and macOS.
• Cache uv's downloads keyed on uv.lock, and install with uv sync --locked so CI fails if the lockfile is stale.
• Test the artefact, not the checkout: install the wheel into a clean environment and run the command from another directory.
• Derive the version from the git tag and publish with PyPI trusted publishing from a protected environment.

The shape of the pipeline

A good pipeline for a CLI is a sequence of gates, each cheaper than the next, arranged so that the expensive steps only run when the cheap ones pass — and so that the thing that is published is exactly the thing that was tested.

Lint and type-check first: they take seconds, catch a large share of mistakes, and there is no point running fifteen test jobs on code that does not pass ruff check. The configuration for both is covered in linting and type-checking CLI code. Test across the matrix next. Then build once — a single job produces the sdist and wheel and uploads them as an artefact — and smoke-test that artefact in a clean environment. Finally, publish only when the run was triggered by a version tag, downloading the very same artefact rather than rebuilding.

Here is a complete workflow that implements it:

# .github/workflows/ci.yml
name: ci
on:
  push:
    branches: [main]
    tags: ["v*"]
  pull_request:

permissions:
  contents: read

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v6
        with: { enable-cache: true }
      - run: uv sync --locked --group dev
      - run: uv run ruff check .
      - run: uv run ruff format --check .
      - run: uv run mypy src

  test:
    needs: lint
    strategy:
      fail-fast: ${{ github.event_name == 'pull_request' }}
      matrix:
        os: [ubuntu-latest]
        python: ["3.10", "3.11", "3.12", "3.13", "3.14"]
        include:
          - { os: windows-latest, python: "3.10" }
          - { os: windows-latest, python: "3.14" }
          - { os: macos-latest, python: "3.10" }
          - { os: macos-latest, python: "3.14" }
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v6
        with: { enable-cache: true, python-version: "${{ matrix.python }}" }
      - run: uv sync --locked --group dev
      - run: uv run pytest -q

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }          # full history so the version can come from tags
      - uses: astral-sh/setup-uv@v6
      - run: uv build
      - uses: actions/upload-artifact@v4
        with: { name: dist, path: dist/ }

  smoke:
    needs: build
    strategy:
      matrix: { os: [ubuntu-latest, windows-latest, macos-latest] }
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/download-artifact@v4
        with: { name: dist, path: dist }
      - uses: astral-sh/setup-uv@v6
      - run: uv tool install --find-links dist mytool
        shell: bash
      - run: mytool --version && mytool --help
        shell: bash
        working-directory: ${{ runner.temp }}

  publish:
    if: startsWith(github.ref, 'refs/tags/v')
    needs: smoke
    runs-on: ubuntu-latest
    environment: pypi
    permissions:
      id-token: write
    steps:
      - uses: actions/download-artifact@v4
        with: { name: dist, path: dist }
      - uses: pypa/gh-action-pypi-publish@release/v1

Each job is the subject of one of the guides below, so the remaining sections explain why each piece looks the way it does rather than repeating it.

Testing where your users are

The test matrix earns its runner minutes by catching bugs that no amount of local testing would find. CLIs are especially exposed, because they interact directly with the operating system — paths, encodings, subprocesses, signals, terminals — in exactly the ways that differ between platforms.

A full matrix of five Python versions on three operating systems is fifteen jobs per push. Most projects get nearly all of the value for fewer: run every supported version on Linux, where runners are fastest and cheapest, and only the oldest and newest versions on Windows and macOS, where platform bugs appear regardless of the Python version.

Two refinements are worth the extra lines. Set fail-fast to cancel sibling jobs on pull requests, where a quick signal matters, but not on main, where you want the full picture of what is broken. And consider a job that installs your lowest declared dependency versions (uv sync --resolution lowest-direct), which catches the classic bug of using an API added after your stated lower bound. The details, including running the same matrix locally, are in testing a CLI across Python versions with GitHub Actions.

Keeping it fast

A pipeline that takes twenty minutes is a pipeline people learn to ignore. For Python projects, dependency installation is usually the biggest avoidable cost, and uv's cache removes most of it. astral-sh/setup-uv with enable-cache: true restores uv's cache at the start of the job and saves it at the end, keyed by default on the lockfile, so the cache is invalidated exactly when dependencies change. With a warm cache, uv sync --locked links packages from the cache instead of downloading them.

--locked matters for correctness as well as speed: it makes uv sync fail if uv.lock does not match pyproject.toml, instead of silently re-resolving. CI then tests exactly the dependency set that developers and releases use, and a forgotten uv lock after editing dependencies is caught in the first job. Caching uv dependencies in CI covers cache keys, pruning, other CI systems and Docker layer caching.

Test the wheel, not the checkout

Unit tests run against the source tree, where every file is present, every development dependency is installed, and import mytool finds the package in the current directory. Users get something else: a wheel, installed into a fresh environment, containing only what your build configuration included. The gap between the two produces a distinctive class of bug — a template file not included in the wheel, an entry point with a typo, a runtime dependency listed only in the dev group — which passes every test and fails on the first user's first command.

The fix is a smoke test of the built artefact: install the wheel into an isolated environment with uv tool install, change to a directory outside the checkout, and run the installed command. A doctor or self-check subcommand that loads every template, schema and plugin makes that smoke test thorough. Smoke-testing the built wheel in CI shows how to build such a command and wire it in, and building wheels and sdists for Python CLIs covers getting the build configuration right in the first place.

Releasing from tags

The most reliable release process is the one with the fewest manual steps. With the version derived from git tags — using hatch-vcs or setuptools-scm, as described in deriving versions from git tags with hatch-vcs — a release becomes a single command: push an annotated tag such as v1.5.0. The workflow's publish job only runs for tag pushes, and because the version comes from the tag, the package and the tag can never disagree.

A tag-driven release also makes rollbacks of a failed release simple. If the pipeline fails before publishing, nothing has reached PyPI; delete the tag, fix the problem, and push it again. Automating releases from git tags adds release notes from your changelog, a GitHub release with the wheel attached, and guards against tagging the wrong commit. The changelog side is covered in automating changelogs with conventional commits.

Publishing without secrets

For years, publishing from CI meant creating a PyPI API token and storing it as a repository secret — a long-lived credential with upload rights to your package, readable by any workflow that asks for it. Trusted publishing replaces it. You tell PyPI which repository, workflow file and environment are allowed to publish your project; the CI job requests a short-lived OpenID Connect token from GitHub proving its identity; PyPI exchanges it for an upload token valid for a few minutes. No secret is stored anywhere, so there is nothing to leak or rotate, and uploads carry attestations linking each file to the workflow run that built it.

The workflow above shows the only configuration needed on the CI side: permissions: id-token: write on the publish job, and the pypa/gh-action-pypi-publish action. Putting the job in a protected pypi environment with required reviewers adds a human approval before each release. Publishing to PyPI with trusted publishing walks through the PyPI side, TestPyPI dry runs and troubleshooting.

Beyond PyPI

Not every CLI's users install from PyPI. Standalone binaries built with PyInstaller or Nuitka, zipapps built with shiv, Homebrew formulae and Scoop manifests all slot into the same pipeline as additional jobs after the smoke test — built per platform in a matrix, attached to the GitHub release, and checked with the same "does --version run?" smoke test. The patterns are in building cross-platform release binaries in CI and Homebrew and Scoop packaging for Python CLIs.

Making the pipeline the gate

A pipeline only protects users if nothing can bypass it. Two settings in the repository turn it from advice into a rule. Branch protection on main requires the lint, test and smoke jobs to pass before a pull request can merge; list the individual matrix jobs, or add a final summary job that needs all of them and require that one, so adding a Python version to the matrix does not mean editing the protection rules. Tag protection (a ruleset restricting who can create v* tags) ensures that only maintainers can trigger a release, since pushing a tag is now the entire release process.

It is also worth deciding up front what happens when the pipeline is red on main. The healthiest convention is that a red main is the team's top priority: either fix forward within the hour or revert the change that broke it. A pipeline that is "usually a bit red" stops being read, and then the Windows-only encoding bug it caught last Tuesday ships anyway.

Flaky tests deserve particular suspicion in CLI projects. Tests that spawn subprocesses, bind ports, depend on timing or touch the real home directory are the usual culprits, and they tend to fail more on slower Windows and macOS runners. Quarantine a flaky test with a marker and an issue link rather than retrying the whole job until it passes; retry-until-green trains everyone to ignore failures. The isolation techniques in mocking filesystem and network in CLI tests remove most sources of flakiness at the root.

Finally, keep an eye on the matrix as time passes. Each October a new Python release arrives and the oldest supported version reaches end of life a year later; adding the new version to the matrix early — even as an allowed-to-fail job during its release candidates — surfaces deprecation warnings months before users hit them, and dropping the old one lets you use newer language features. Record the supported range in requires-python and the trove classifiers so the matrix, the metadata and the documentation agree, as described in writing pyproject.toml metadata for a CLI.

Security basics for CLI pipelines

A release pipeline can publish code to every one of your users, which makes it worth a few minutes of hardening:

• Default to read-only permissions (permissions: contents: read at the top) and grant more only to the job that needs it.
• Never publish from pull_request workflows, which can run code from forks.
• Pin third-party actions to a commit SHA, or at least a major version from a publisher you trust; Dependabot or Renovate can keep the pins current.
• Keep secrets out of test jobs. If integration tests need credentials, run them only on main and only in a job that does not execute untrusted code. The CLI-side practices are in secrets and credentials in Python CLIs.

Key takeaways

• Order the pipeline cheapest-first: lint and types, test matrix, build once, smoke test, publish on tags.
• Test every supported Python on Linux and the extremes on Windows and macOS.
• Install with uv sync --locked and let setup-uv cache by lockfile.
• Smoke-test the built wheel in a clean environment, from outside the checkout.
• Take the version from the git tag so a release is a single git push.
• Publish with trusted publishing from a protected environment; store no PyPI token at all.

Frequently asked questions

Related

• Up: Project Setup & Dependency Management
• Down: Testing a CLI across Python versions with GitHub Actions
• Down: Publishing to PyPI with trusted publishing
• Down: Caching uv dependencies in CI
• Down: Smoke-testing the built wheel in CI
• Down: Automating releases from git tags
• Sideways: Packaging Python CLIs for distribution
• Sideways: Testing Python CLI applications