Mocking the Filesystem and Network in CLI Tests

A test that touches the network is a test that fails on a train. A test that writes into the repository is a test that passes once. Isolating a CLI suite is mostly a question of choosing where to substitute something — and the usual instinct, patching the function under test, is the one place that proves nothing.

TL;DR

• Push the seam outwards: replace the HTTP transport, not your own request-building function.
• Use tmp_path for real files; a mocked open() behaves differently from a real filesystem.
• Inject time and randomness as parameters with sensible defaults instead of patching modules.
• Prefer a small hand-written fake over MagicMock — it fails loudly when the real interface changes.
• Clear your tool's environment variables in a fixture so a developer's shell cannot change results.

Prerequisites

A CLI whose logic lives outside the command functions, pytest, and httpx for the network examples (the same pattern applies to requests via responses, and to any client with a pluggable transport).

Where to put the seam

The rule of thumb: push the substitution outwards until all of your code is inside it. Patching mytool.core.deploy and asserting it was called proves that the command calls a function — which was never in doubt — while leaving the request construction, the error handling and the response parsing untested.

Replacing the transport keeps all of that under test and still never opens a socket:

import json
import httpx

def test_deploy_sends_the_expected_request():
    seen = {}

    def handler(request: httpx.Request) -> httpx.Response:
        seen["method"] = request.method
        seen["url"] = str(request.url)
        seen["body"] = json.loads(request.content)
        return httpx.Response(201, json={"id": "dep_123", "status": "queued"})

    client = httpx.Client(transport=httpx.MockTransport(handler), base_url="https://api.test")
    result = core.deploy(client, environment="prod", replicas=3)

    assert seen["method"] == "POST"
    assert seen["url"].endswith("/deployments")
    assert seen["body"] == {"environment": "prod", "replicas": 3}
    assert result.id == "dep_123"

For this to work, the client has to be something the caller can supply. A core function that constructs its own httpx.Client internally forces you back to patching — so make it a parameter with a default, which costs nothing at the call site:

def deploy(client: httpx.Client | None = None, *, environment: str, replicas: int) -> Deployment:
    client = client or httpx.Client(base_url=settings.api_url)
    ...

Testing error paths without a network

The valuable network tests are the failures, because that is the code least exercised during development. A transport handler can raise or return anything:

import pytest

def failing_transport(exc: Exception) -> httpx.MockTransport:
    def handler(request: httpx.Request) -> httpx.Response:
        raise exc
    return httpx.MockTransport(handler)

def test_connection_error_becomes_a_domain_error():
    client = httpx.Client(transport=failing_transport(httpx.ConnectError("no route")))

    with pytest.raises(RemoteUnavailable) as excinfo:
        core.deploy(client, environment="prod", replicas=1)

    assert "deploy API" in str(excinfo.value)      # the message names what was unreachable

@pytest.mark.parametrize("status,expected", [(401, AuthError), (429, RateLimited), (500, RemoteUnavailable)])
def test_http_errors_map_to_domain_errors(status, expected):
    client = httpx.Client(transport=httpx.MockTransport(lambda r: httpx.Response(status)))

    with pytest.raises(expected):
        core.deploy(client, environment="prod", replicas=1)

That parametrised test is the one that pays for itself. It pins the mapping from HTTP status to your own exception types, which is what determines the exit code and the message a user sees.

The filesystem: use real files

tmp_path gives each test its own directory with absolute paths, cleaned up automatically. It is almost always better than a mocked open(), because the code under test then meets real encodings, real permissions and the real FileNotFoundError.

def test_reads_config_from_the_project_directory(tmp_path):
    (tmp_path / ".mytool.toml").write_text('region = "eu-west-1"\n', encoding="utf-8")

    settings = load_settings(start=tmp_path)

    assert settings.region == "eu-west-1"

def test_unreadable_config_is_reported_clearly(tmp_path):
    config = tmp_path / ".mytool.toml"
    config.write_text("region = [unclosed\n")

    with pytest.raises(ConfigError) as excinfo:
        load_settings(start=tmp_path)

    assert ".mytool.toml" in str(excinfo.value)     # the message names the file

Two habits keep filesystem tests honest. Never write outside tmp_path — a test that writes into the repository leaves artifacts that later tests come to depend on. And pass the starting directory in rather than relying on the process working directory, so tests can run in parallel without fighting over chdir.

Time, randomness and identifiers

Anything non-deterministic should be a parameter with a default, not a module-level call. The default keeps production code readable; the parameter makes the test trivial.

from datetime import UTC, datetime
from typing import Callable

def retry(
    operation: Callable[[], T],
    *,
    attempts: int = 3,
    backoff: float = 1.0,
    sleep: Callable[[float], None] = time.sleep,
) -> T:
    for attempt in range(1, attempts + 1):
        try:
            return operation()
        except Transient:
            if attempt == attempts:
                raise
            sleep(backoff * 2 ** (attempt - 1))

def test_backoff_doubles_between_attempts():
    waits: list[float] = []
    calls = iter([Transient(), Transient(), "done"])

    def operation():
        item = next(calls)
        if isinstance(item, Exception):
            raise item
        return item

    assert retry(operation, sleep=waits.append) == "done"
    assert waits == [1.0, 2.0]

No patching, no waiting, and the test reads as a statement about the retry policy. The same shape covers now: Callable[[], datetime] = lambda: datetime.now(UTC) and new_id: Callable[[], str] = lambda: str(uuid4()).

The environment

Environment variables leak between tests and between machines, and the failures are baffling because nothing in the test mentions them. Two lines in a fixture remove the whole class:

@pytest.fixture(autouse=True)
def clean_environment(monkeypatch):
    for key in list(os.environ):
        if key.startswith("MYTOOL_"):
            monkeypatch.delenv(key, raising=False)
    monkeypatch.setenv("NO_COLOR", "1")            # stable output everywhere

autouse=True is justified here: no test wants a developer's exported variables, and forgetting to request the fixture is exactly the mistake it exists to prevent. monkeypatch undoes all of it afterwards, including in tests that fail.

UX considerations

Isolation choices show up in the quality of failure messages, which is worth optimising for.

A fake that raises AttributeError: 'FakeBucket' object has no attribute 'upload_many' tells you immediately that the real interface grew a method. A MagicMock in the same position returns another mock, the test passes, and the defect surfaces in production. That difference is the whole argument for hand-written fakes:

class FakeBucket:
    def __init__(self) -> None:
        self.uploaded: list[str] = []

    def upload(self, path, *, retries: int = 3) -> None:
        self.uploaded.append(str(path))

Ten lines, readable, and assertions run against state (fake.uploaded == [...]) rather than against call sequences — which is both easier to read and less coupled to how the code achieves the result.

Testing the behaviour

Once the seams are in place, the tests that matter become short:

def test_sync_uploads_only_changed_files(tmp_path):
    (tmp_path / "a.txt").write_text("x")
    (tmp_path / "b.txt").write_text("y")
    bucket = FakeBucket()
    bucket.current = {str(tmp_path / "a.txt")}

    result = sync_directory(tmp_path, bucket=bucket)

    assert result == SyncResult(uploaded=1, skipped=1)
    assert bucket.uploaded == [str(tmp_path / "b.txt")]

No patching, no network, one real directory, and every assertion is about the rule under test.

Conclusion

Isolation is a design question more than a testing one. Accept collaborators as parameters, keep non-determinism at the edges, replace the transport rather than your own code, and use real temporary files. The result is a suite that runs offline, in parallel, in any order, and fails for reasons that point at the defect.

Frequently asked questions

Related

• Up: Testing Python CLI applications
• Sideways: Testing Click commands with CliRunner
• Sideways: Measuring CLI test coverage
• Related: Sharing state with Click context objects
• Related: Handling configuration files and environment variables