Update Checks, Upgrade Commands and Telemetry for Python CLIs

A CLI installed on a laptop tends to stay at whatever version was installed. Nobody runs pipx upgrade-all on a schedule, internal tools are rarely re-installed until something breaks, and the bug you fixed three releases ago keeps generating support messages from people still on the old version. Browsers, editors and package managers solved this long ago by checking for updates and saying so. A Python CLI can do the same — but a careless update check is one of the fastest ways to make a tool feel slow, flaky or intrusive: an HTTP request on every run, a hang on an aeroplane, a notice in the middle of JSON output, or a self-upgrade that breaks the user's pipx environment.

This topic covers doing it well. It sits in the CLI Runtime & Systems Integration section because every part of it is a runtime concern: network calls with tight time budgets, caching state between runs, detecting how the tool was installed, and deciding what may be sent over the network at all. It also touches packaging — what "the latest version" means depends on how you publish, covered in publishing a Python CLI to PyPI.

TL;DR

• Check at most once a day, cache the answer, and never let a check add noticeable time to a command. A two-second network timeout on every run is a bug.
• Show the notice on stderr, after the command's output, and only when stderr is an interactive terminal. Never in CI, never in a pipe, never in JSON mode.
• Make it easy to turn off with an environment variable and a config setting, and turn it off automatically where it makes no sense.
• Compare versions with packaging.version, skip pre-releases unless the user runs one, and respect yanked releases.
• Upgrade through the installer that installed you. Detect pipx, uv, Homebrew or a plain virtual environment and run — or print — the right command. Never pip install into an environment you do not own.
• Telemetry is opt-in, minimal, documented and inspectable. If you cannot explain exactly what is sent, do not send it.

What an update check involves

An update check looks like a single HTTP request, but a robust one has five moving parts, and each one has a failure mode that users notice.

1. Should we check at all? Not in CI, not when output is piped, not when the user disabled it, not if we checked recently. Most runs should stop here, having done nothing but read a small cache file.
2. Fetch the latest version from PyPI's JSON API, a GitHub releases endpoint or your own server — with a short timeout and no retries.
3. Compare it with the installed version using real version semantics.
4. Record the result and the time in a cache file in the user's state directory, so the next run can skip the network entirely.
5. Tell the user — once, briefly, on stderr, after the command has done its job.

The ordering matters. Steps 1 and 5 run on every invocation and must be essentially free. Steps 2–4 run rarely and must never block the user's command. Checking PyPI for a newer version implements the fetch and comparison; showing non-blocking update notices handles the caching, threading and display.

Where version information comes from

For a package published on PyPI, the authoritative source is the JSON API at https://pypi.org/pypi/<name>/json. It returns the latest stable version in info.version and every release in releases, including which files are yanked. It needs no authentication, is served from a CDN, and is fast — but it is still a network call to a third party, and corporate networks sometimes block it or route it through a slow proxy.

Alternatives exist for tools that are not on PyPI. Internal tools published to a private index can query that index's JSON or Simple API. Tools distributed as binaries usually check GitHub's releases/latest endpoint, which is rate-limited for unauthenticated calls — another reason to cache. And tools with their own backend can expose a tiny /cli/latest endpoint that also returns a minimum supported version and a message, which lets you tell users "this version can no longer talk to the server; please upgrade".

Whichever source you use, keep the client code tolerant: any error — DNS failure, timeout, 500, unexpected JSON — means "no information", never an exception that reaches the user.

Comparing versions correctly

String comparison gets versions wrong ("1.10.0" < "1.9.0" is True), and splitting on dots fails on pre-releases and post-releases. Use packaging.version.Version, which implements the PEP 440 ordering that pip uses:

from packaging.version import InvalidVersion, Version


def is_newer(latest: str, current: str, *, allow_prerelease: bool = False) -> bool:
    try:
        new, cur = Version(latest), Version(current)
    except InvalidVersion:
        return False
    if new.is_prerelease and not (allow_prerelease or cur.is_prerelease):
        return False
    return new > cur

Two policies are worth deciding explicitly. Pre-releases: users on a stable version should not be told about 2.0.0rc1; users already running a pre-release usually want to hear about the next one. Major versions: some tools tell users about a new major version differently — "2.0 is available with breaking changes; see the changelog" — because upgrading is a decision rather than a chore. That ties into how you version the CLI in the first place, covered in semantic versioning policy for CLI tools.

The installed version should come from package metadata, importlib.metadata.version("mytool"), so it always matches what is actually installed; exposing version info and build metadata explains why a hard-coded __version__ drifts.

Never slow down the command

The cardinal rule is that an update check may never make a command noticeably slower or less reliable. There are three techniques, and good implementations combine them.

Cache with a long interval. Store the last check time and result in a JSON file in the state directory from storing app data with platformdirs. With a 24-hour interval, 99% of runs read a tiny file and do nothing else.

Check in the background. When a check is due, start it in a daemon thread at the beginning of the command and collect the result at the end, waiting at most a fraction of a second. If the network is slow, the result simply arrives on a later run, because the thread writes the cache when it finishes. A forked background process is an alternative that survives the command exiting, at the cost of more moving parts.

Use a tight timeout. Even in the background, cap the request at one or two seconds and do not retry. A check that cannot complete quickly is not worth completing.

What you must not do is check synchronously before the command runs. Users on slow or offline networks will experience it as "the tool hangs for two seconds every time", and they will be right.

The cache file

The cache is what turns an update check from "a network call per run" into "a file read per day", so its format deserves a moment of design. Keep it small, versioned and self-describing:

{
  "schema": 1,
  "checked_at": "2026-10-02T09:14:03+00:00",
  "current": "1.4.2",
  "latest": "1.6.0",
  "notified_at": "2026-10-02T09:14:05+00:00"
}

checked_at decides whether a new check is due. Storing current alongside latest means that after the user upgrades, the stale "1.6.0 is available" result is ignored automatically, because the installed version no longer matches. notified_at rate-limits the notice separately from the check, so a user who runs fifty commands a day sees it once. The schema field lets a later release change the format without crashing on an old file — if the schema is unknown, ignore the file and start over.

Read the file defensively and write it atomically:

import json
from datetime import datetime, timezone
from pathlib import Path


def load_cache(path: Path) -> dict:
    try:
        data = json.loads(path.read_text(encoding="utf-8"))
    except (OSError, ValueError):
        return {}
    return data if isinstance(data, dict) and data.get("schema") == 1 else {}


def is_due(cache: dict, current: str, *, now: datetime, hours: float = 24) -> bool:
    if cache.get("current") != current:
        return True                              # upgraded or downgraded since last check
    try:
        last = datetime.fromisoformat(cache["checked_at"])
    except (KeyError, TypeError, ValueError):
        return True
    return (now - last).total_seconds() >= hours * 3600

A corrupt or unreadable cache must never break the command — it simply means a check is due. Two concurrent runs racing to write the file are harmless if each write is atomic, as in writing files atomically in Python CLIs; the worst case is one extra check. Put the file in the state directory rather than the cache directory if you want the "already notified" marker to survive cache cleaners, and in the cache directory if you would rather it be disposable.

Showing the notice

A good notice is short, actionable and in the right place:

A new version of mytool is available: 1.4.2 → 1.6.0
Upgrade with: pipx upgrade mytool

Print it to stderr, after the command's own output, and only when stderr is a terminal — the same detection described in detecting CI environments and non-interactive shells. Show the upgrade command that matches how the tool was installed, which is why install-method detection matters even if you never upgrade automatically. And show it at most once per day, not on every run until the user upgrades; a notice that appears every time becomes noise that users learn to resent.

Respect the conventional switches. Many tools honour a tool-specific variable (MYTOOL_NO_UPDATE_CHECK=1), and some also honour a generic one such as NO_UPDATE_NOTIFIER. Add a config key for people who want it off permanently. Disable the check automatically when CI is set, when running under a test runner, and in any machine-readable output mode.

Upgrade commands

Telling users how to upgrade is good; offering mytool self-update is better — if, and only if, it uses the right installer. A Python CLI might be installed in at least six ways, and each has its own correct upgrade path:

Running pip install --upgrade mytool from inside the tool is wrong for almost all of them: it writes into a pipx- or uv-managed environment behind the manager's back, it fails in a Homebrew Cellar, and in a project virtual environment it changes a dependency that a lock file is supposed to control. The safe design detects the installer — pipx leaves pipx_metadata.json in its environment, uv tool leaves uv-receipt.toml, Homebrew installs under its Cellar, a PyInstaller binary has sys.frozen set — and then either runs that installer's upgrade command or prints it when the method is unknown. Self-upgrading a CLI installed with pipx or uv implements this detection and the command.

Telemetry, carefully

Once a tool phones home for version information, it is tempting to send a little more: which commands are used, how long they take, which errors occur. That data is genuinely useful for deciding what to fix and what to deprecate. It is also the fastest way to lose trust, especially in a tool that runs in CI with access to source code and credentials.

The responsible baseline is: off by default, enabled by an explicit command or config setting; minimal — command name, tool version, Python version, OS family, duration and an error class, but never arguments, paths, environment variables or hostnames; documented — a page listing every field; inspectable — a debug mode that prints the payload instead of sending it; disabled in CI unless explicitly enabled there; and anonymous, with a random installation ID the user can reset. Opt-in usage telemetry for Python CLIs builds a client that follows these rules and sends events without delaying the command.

Testing update logic

Update checks are easy to get wrong in ways that unit tests catch easily, provided you design for it:

• Inject the clock and the fetcher. A function that takes now and a fetch_latest callable can be tested for "due", "not due" and "network failed" without patching anything global.
• Mock HTTP at the transport. With httpx, respx returns canned PyPI responses, timeouts and errors. Assert that every failure is silent.
• Isolate the cache directory with tmp_path and an environment variable that overrides the state location.
• Assert on streams. The notice must appear on stderr only, and never when stderr is not a TTY or when JSON output is requested.

One end-to-end test that runs the real command with the check enabled and the network blocked is a good guard: it should complete exactly as fast as with the check disabled.

Common pitfalls

• Checking synchronously. The single most common mistake, and the one users complain about most.
• Notices in machine output. An update message on stdout breaks every script that parses JSON or CSV from the tool.
• Nagging. A notice on every run, or one that cannot be turned off, gets the tool uninstalled or aliased with the check disabled.
• Upgrading with the wrong installer. pip install --upgrade inside a pipx or uv environment corrupts the manager's view of it.
• Trusting the network. Proxies return HTML error pages with status 200; captive portals return anything. Validate the response shape and treat surprises as "no information".
• Telemetry by default. Even well-intentioned, it is the sort of surprise that ends up in a security review — or a news story.

Key takeaways

• Treat an update check as a background, best-effort feature: cached for a day, bounded by a short timeout, silent on failure.
• Compare versions with packaging.version, and decide policies for pre-releases and major versions.
• Show one short notice on stderr after the output, only for interactive terminals, with an upgrade command that matches the install method.
• Offer a self-update command only if it delegates to the installer that owns the environment.
• Make telemetry opt-in, minimal, documented and inspectable — or do not collect it.

Frequently asked questions

Related

• Up: CLI Runtime & Systems Integration
• Down: Checking PyPI for a newer version
• Down: Showing non-blocking update notices
• Down: Self-upgrading a CLI installed with pipx or uv
• Down: Opt-in usage telemetry for Python CLIs
• Sideways: Calling HTTP APIs from Python CLIs
• Sideways: Filesystem paths and atomic writes