Python CLI Toolcraft

Runtime

Building an API Client CLI with httpx

Build a Typer CLI over a REST API with httpx: one configured client, typed results, error mapping to exit codes, table and JSON output, and MockTransport tests.

Updated

You want a command-line front end for a web API — your company's deployment service, an issue tracker, a metrics backend — that the team can use from a terminal and from scripts. The goal is not just "it makes requests" but a tool that stays pleasant as it grows: commands that read like the domain, errors that tell people what to do, output that is readable for humans and stable for jq, and tests that run offline in milliseconds. This guide builds that tool end to end with httpx and Typer, using a small projects API as the running example. It is the hands-on companion to calling HTTP APIs from Python CLIs.

Prerequisites

  • Python 3.10+, httpx 0.27 or newer, typer and rich (uv add httpx typer rich).
  • An API to talk to. The examples assume GET /projects returns a JSON list of {"name", "owner", "build_count"} objects and GET /projects/{name} returns one.
  • An API token in the MYTOOL_TOKEN environment variable. Storing it properly is covered in storing tokens with keyring.

The shape of the tool

Three modules, each with one job. api.py knows HTTP and the API's JSON shapes and returns typed objects. cli.py knows arguments and output formats. A small settings.py (folded into the CLI callback here) knows where the base URL and token come from. The command asks the API module for objects; the API module reuses a single httpx.Client for every request in the run.

One command, one session A sequence where a command asks the API client for data, the client reuses one HTTP connection for several requests, and returns typed objects to the command. One command, one session Command API client httpx.Client API list_projects() GET /projects TLS once, keep-alive 200 JSON list[Project] Reusing one Client keeps the connection open, which saves a TLS handshake on every call.

The recipe: the API module

# src/mytool/api.py
from __future__ import annotations

from dataclasses import asdict, dataclass
from typing import Any
from urllib.parse import quote

import httpx

__version__ = "1.4.0"

EXIT_USAGE = 2
EXIT_AUTH = 4
EXIT_UNAVAILABLE = 69   # EX_UNAVAILABLE from sysexits.h


@dataclass(frozen=True)
class Project:
    name: str
    owner: str
    builds: int

    def to_json(self) -> dict[str, Any]:
        return asdict(self)


class ApiError(Exception):
    def __init__(self, message: str, exit_code: int = 1) -> None:
        super().__init__(message)
        self.exit_code = exit_code


def make_client(base_url: str, token: str | None, *,
                transport: httpx.BaseTransport | None = None) -> httpx.Client:
    headers = {"User-Agent": f"mytool/{__version__}", "Accept": "application/json"}
    if token:
        headers["Authorization"] = f"Bearer {token}"
    return httpx.Client(
        base_url=base_url,
        headers=headers,
        timeout=httpx.Timeout(30.0, connect=5.0),
        transport=transport or httpx.HTTPTransport(retries=2),
    )


class ProjectsApi:
    def __init__(self, client: httpx.Client) -> None:
        self._client = client

    def _get(self, path: str, **params: Any) -> Any:
        try:
            response = self._client.get(path, params=params or None)
        except httpx.TimeoutException:
            raise ApiError(f"{self._client.base_url.host} did not respond in time",
                           EXIT_UNAVAILABLE) from None
        except httpx.TransportError as exc:
            raise ApiError(f"cannot reach {self._client.base_url.host}: {exc}",
                           EXIT_UNAVAILABLE) from None
        return self._check(response)

    @staticmethod
    def _check(response: httpx.Response) -> Any:
        if response.is_success:
            return response.json()
        code = response.status_code
        detail = ""
        if "json" in response.headers.get("content-type", ""):
            detail = response.json().get("message", "")
        if code in (401, 403):
            raise ApiError("not authorised — set MYTOOL_TOKEN or run: mytool login", EXIT_AUTH)
        if code == 404:
            raise ApiError(detail or "not found")
        if code == 422:
            raise ApiError(detail or "the server rejected the request", EXIT_USAGE)
        if code >= 500:
            raise ApiError(f"the service returned {code}; try again shortly", EXIT_UNAVAILABLE)
        raise ApiError(f"unexpected response {code}: {detail}".rstrip(": "))

    def list_projects(self) -> list[Project]:
        return [Project(p["name"], p["owner"], p["build_count"]) for p in self._get("/projects")]

    def get_project(self, name: str) -> Project:
        p = self._get(f"/projects/{quote(name, safe='')}")   # "a/b" must not become a path
        return Project(p["name"], p["owner"], p["build_count"])

Three decisions are doing most of the work.

One error type with an exit code. Every failure — network, timeout, HTTP status — becomes ApiError, carrying the message the user should see and the exit code the process should use. The command layer then needs exactly one except clause. The codes follow the conventions in choosing exit codes for CLI tools: 2 for bad input, 69 (EX_UNAVAILABLE) when the service cannot be reached, and a distinct code for authentication so scripts can detect "needs login".

Typed objects out. Commands receive Project instances, never response dictionaries. When the API renames build_count, one line in api.py changes and every command keeps working. It also means your --json output is your schema, not a pass-through of the server's, so you control its stability.

An injectable transport. make_client accepts an optional transport. Production code passes nothing and gets real HTTP with connection retries; tests pass httpx.MockTransport and get no network at all.

Turning HTTP failures into CLI errors How an API client maps HTTP status codes and network failures to user-facing messages and exit codes. Turning HTTP failures into CLI errors Failure Message Exit 401 / 403 not logged in, or no access — run: mytool login 4 404 no project named "web" 1 422 the server's validation message 2 5xx after retries the service is having problems 69 ConnectError cannot reach api.example.com 69 Each message says what went wrong and what to do next; none of them is a traceback.

The recipe: the command layer

# src/mytool/cli.py
from __future__ import annotations

import json
import os
from collections.abc import Callable

import httpx
import typer
from rich.console import Console
from rich.table import Table

from mytool.api import ApiError, ProjectsApi, make_client

app = typer.Typer(no_args_is_help=True)
err = Console(stderr=True)

# Tests replace this factory to inject a mock transport.
client_factory: Callable[[str, str | None], httpx.Client] = make_client


@app.callback()
def main(
    ctx: typer.Context,
    api_url: str = typer.Option("https://api.example.com/v1", envvar="MYTOOL_API_URL"),
) -> None:
    """Command-line access to the projects API."""
    client = client_factory(api_url, os.environ.get("MYTOOL_TOKEN"))
    ctx.call_on_close(client.close)
    ctx.obj = ProjectsApi(client)


def fail(exc: ApiError) -> None:
    err.print(f"[red]error:[/red] {exc}")
    raise typer.Exit(exc.exit_code)


@app.command()
def projects(ctx: typer.Context, as_json: bool = typer.Option(False, "--json")) -> None:
    """List projects."""
    try:
        items = ctx.obj.list_projects()
    except ApiError as exc:
        fail(exc)
    if as_json:
        typer.echo(json.dumps([p.to_json() for p in items], indent=2))
        return
    table = Table("NAME", "OWNER", "BUILDS", box=None, header_style="bold")
    for p in sorted(items, key=lambda p: p.name):
        table.add_row(p.name, p.owner, str(p.builds))
    Console().print(table)


@app.command()
def show(ctx: typer.Context, name: str, as_json: bool = typer.Option(False, "--json")) -> None:
    """Show one project."""
    try:
        p = ctx.obj.get_project(name)
    except ApiError as exc:
        fail(exc)
    if as_json:
        typer.echo(json.dumps(p.to_json(), indent=2))
    else:
        typer.echo(f"{p.name}  owner={p.owner}  builds={p.builds}")


if __name__ == "__main__":
    app()

ctx.call_on_close(client.close) closes the connection pool when the command finishes, whether it succeeded or raised — the Click context's cleanup hook, which is more reliable than remembering a finally in every command. The client and API object travel on ctx.obj, the shared-state mechanism described in sharing state with Click context objects.

UX considerations

Same data, two audiences Terminal output of an API-backed command rendering a table for people and JSON for scripts from the same typed objects. Same data, two audiences bash $ mytool projects NAME OWNER BUILDS web ana 214 billing ops 87 $ mytool projects --json | jq ".[0].name" "web" Render from typed objects, never from raw response JSON, so the --json shape is yours to keep stable.
  • Human by default, machine on request. A table with sensible column order and no borders for people; --json with a stable, documented schema for scripts. Keep diagnostics on stderr so --json output can be piped straight into jq. The contract is spelled out in emitting JSON output for scripting.
  • Errors that end in an action. "not authorised — set MYTOOL_TOKEN or run: mytool login" is worth ten "401 Unauthorized"s. Prefer the server's own validation message for 422 responses; it usually names the offending field.
  • Show which server you are talking to. A --verbose line naming the base URL saves a lot of confusion between staging and production.
  • Do not hide slow calls. For requests that can take more than a second, a Rich console.status("Fetching projects...") spinner on stderr tells the user the tool is working, as covered in adding progress bars and spinners to Python CLIs.
  • Sort client-side when order matters. APIs often return items in insertion or arbitrary order. Sorting by name makes output diffable between runs.

Testing the behaviour

Replace the client factory with one that builds a client on httpx.MockTransport. The handler is an ordinary function that inspects the request and returns a response, so each test states exactly what the server does:

# tests/test_cli.py
import json

import httpx
import pytest
from typer.testing import CliRunner

from mytool import cli
from mytool.api import make_client

runner = CliRunner()


@pytest.fixture
def server(monkeypatch):
    routes: dict[str, httpx.Response] = {}

    def handler(request: httpx.Request) -> httpx.Response:
        assert request.headers["User-Agent"].startswith("mytool/")
        return routes.get(request.url.path, httpx.Response(404, json={"message": "no such thing"}))

    monkeypatch.setattr(cli, "client_factory",
                        lambda url, token: make_client(url, token, transport=httpx.MockTransport(handler)))
    return routes


def test_projects_table(server):
    server["/v1/projects"] = httpx.Response(200, json=[
        {"name": "web", "owner": "ana", "build_count": 214},
        {"name": "billing", "owner": "ops", "build_count": 87},
    ])
    result = runner.invoke(cli.app, ["projects"])
    assert result.exit_code == 0
    assert result.output.index("billing") < result.output.index("web")


def test_projects_json_is_our_schema(server):
    server["/v1/projects"] = httpx.Response(200, json=[{"name": "web", "owner": "ana", "build_count": 1}])
    result = runner.invoke(cli.app, ["projects", "--json"])
    assert json.loads(result.output) == [{"name": "web", "owner": "ana", "builds": 1}]


def test_auth_failure_exit_code(server):
    server["/v1/projects"] = httpx.Response(401)
    result = runner.invoke(cli.app, ["projects"])
    assert result.exit_code == 4
    assert "mytool login" in result.output


def test_not_found_uses_server_message(server):
    result = runner.invoke(cli.app, ["show", "nope"])
    assert result.exit_code == 1
    assert "no such thing" in result.output


def test_network_failure(monkeypatch):
    def boom(request):
        raise httpx.ConnectError("connection refused", request=request)

    monkeypatch.setattr(cli, "client_factory",
                        lambda url, token: make_client(url, token, transport=httpx.MockTransport(boom)))
    result = runner.invoke(cli.app, ["projects"])
    assert result.exit_code == 69

Note the JSON test asserts the renamed key builds, proving that the output schema is decoupled from the server's. For more on keeping CLI tests fast and offline, see mocking filesystem and network in CLI tests.

Conclusion

An API client CLI is three small, separate things: a configured httpx.Client, an API module that converts HTTP into typed objects and one error type, and a command layer that renders those objects for people or for scripts. Keep the boundaries firm, inject the transport so tests never touch the network, and give every failure a message that ends in something the user can do. From here, add retries and backoff for flaky services and pagination for lists that outgrow one response.

Frequently asked questions

Should I generate the client from an OpenAPI spec?

Generators such as openapi-python-client save typing for large APIs and keep models in sync with the spec. They also produce a lot of code and a dependency on the generator's style. For a CLI that uses a dozen endpoints, a hand-written module is usually smaller and easier to read; for one that wraps hundreds, generate the models and keep a hand-written layer on top.

Pydantic or dataclasses for the response models?

Dataclasses are enough when you control the parsing and want zero import cost. Pydantic earns its place when responses are deeply nested or you want validation errors that name the bad field. If startup time matters, measure: Pydantic adds noticeable import time.

Where should the base URL and token come from?

A flag, then an environment variable, then a config file, then a default — the standard precedence. Tokens should prefer the keychain over config files. Both are covered in supporting multiple profiles and accounts.

How do I avoid creating the client for commands that do not need it?

Create it lazily: store a factory on the context and build the client the first time a command asks for it. That keeps --help and purely local commands free of network setup.