Cancelling Async Tasks on Ctrl+C in Python CLIs

An async command is mirroring 120 files with eight concurrent downloads. The user realises they pointed it at the wrong bucket and presses Ctrl+C. What should happen is obvious: stop starting new downloads, abandon the ones in progress, remove their partial files, say what was done, and exit — within a second. What often happens instead is a wall of Task was destroyed but it is pending! warnings, a CancelledError traceback, half-written files left behind, or a tool that ignores the first Ctrl+C entirely because some except Exception swallowed the cancellation. This guide explains how cancellation actually flows through asyncio, and how to write async CLI code that stops cleanly and reports honestly. It is part of the concurrency and async topic.

Prerequisites

• Python 3.11+ — the behaviour described relies on asyncio.run's signal handling and TaskGroup, both improved in 3.11.
• An async command, bridged into Typer or Click as in running async code in Typer and Click.
• Familiarity with the synchronous story in handling KeyboardInterrupt cleanly.

How Ctrl+C travels through asyncio

When the terminal sends SIGINT, Python normally raises KeyboardInterrupt wherever the main thread happens to be — which, in an async program, is usually deep inside the event loop's machinery, where an exception can corrupt its state. Since Python 3.11, asyncio.run() installs its own SIGINT handler instead. On the first Ctrl+C it cancels the main task; the cancellation propagates down through everything that task is awaiting; once the main task has finished unwinding, asyncio.run() raises KeyboardInterrupt to its caller.

Cancellation arrives inside each task as a CancelledError raised at the await where the task is suspended. That means cleanup code in finally blocks and async with exits runs normally, as long as nothing catches and discards the CancelledError on its way up. If the user presses Ctrl+C a second time before shutdown finishes, asyncio.run() stops waiting and raises KeyboardInterrupt immediately — the escape hatch for cleanup that hangs.

asyncio.TaskGroup completes the picture: when the task running the group is cancelled, the group cancels every child task and waits for all of them to finish before letting the cancellation continue. No child can outlive the async with block, so there are no orphaned tasks and no "destroyed but pending" warnings.

The recipe

The example mirrors a list of URLs into a directory. Each download writes to a .part file and renames it on success; cancellation removes the partial file. A small state object tracks what happened, so the command can report it even after an interrupt.

# src/mytool/mirror.py
from __future__ import annotations

import asyncio
import os
from dataclasses import dataclass, field
from pathlib import Path

import httpx


@dataclass
class MirrorState:
    total: int
    done: list[str] = field(default_factory=list)
    cancelled: list[str] = field(default_factory=list)
    failed: dict[str, str] = field(default_factory=dict)

    @property
    def not_started(self) -> int:
        return self.total - len(self.done) - len(self.cancelled) - len(self.failed)


async def fetch_one(client: httpx.AsyncClient, url: str, dest: Path, state: MirrorState) -> None:
    part = dest.with_name(dest.name + ".part")
    try:
        async with client.stream("GET", url) as response:
            response.raise_for_status()
            with part.open("wb") as fh:
                async for chunk in response.aiter_bytes(64 * 1024):
                    fh.write(chunk)
        # The rename is quick and must not be half-done: protect it from cancellation.
        await asyncio.shield(asyncio.to_thread(os.replace, part, dest))
        state.done.append(url)
    except asyncio.CancelledError:
        state.cancelled.append(url)
        raise                                   # never swallow cancellation
    except httpx.HTTPError as exc:
        state.failed[url] = str(exc) or type(exc).__name__
    finally:
        part.unlink(missing_ok=True)            # gone on success (renamed) or failure


async def mirror(urls: list[str], out: Path, jobs: int, state: MirrorState) -> None:
    sem = asyncio.Semaphore(jobs)
    out.mkdir(parents=True, exist_ok=True)
    async with httpx.AsyncClient(timeout=httpx.Timeout(30.0, connect=5.0),
                                 follow_redirects=True) as client:

        async def guarded(url: str) -> None:
            async with sem:
                name = httpx.URL(url).path.rsplit("/", 1)[-1] or "index.html"
                await fetch_one(client, url, out / name, state)

        async with asyncio.TaskGroup() as tg:
            for url in urls:
                tg.create_task(guarded(url))

# src/mytool/cli.py
import asyncio
from pathlib import Path

import typer

from mytool.mirror import MirrorState, mirror

app = typer.Typer()


@app.callback()
def main() -> None:
    """Mirroring tools."""


@app.command("mirror")
def mirror_cmd(
    urls_file: typer.FileText,
    out: Path = typer.Option(Path("mirror"), "--out", "-o"),
    jobs: int = typer.Option(8, "--jobs", "-j", min=1, max=64),
) -> None:
    """Download every URL listed in URLS_FILE."""
    urls = [u.strip() for u in urls_file if u.strip()]
    state = MirrorState(total=len(urls))
    try:
        asyncio.run(mirror(urls, out, jobs, state))
    except KeyboardInterrupt:
        typer.echo(
            f"\ninterrupted: {len(state.done)} done, {len(state.cancelled)} cancelled, "
            f"{state.not_started} not started",
            err=True,
        )
        typer.echo("partial files removed; re-run to continue", err=True)
        raise typer.Exit(130)
    for url, why in state.failed.items():
        typer.echo(f"✗ {url}: {why}", err=True)
    typer.echo(f"mirrored {len(state.done)}/{state.total}", err=True)
    raise typer.Exit(1 if state.failed else 0)


if __name__ == "__main__":
    app()

Why each piece is there

except asyncio.CancelledError: ... raise. Recording the cancellation is fine; swallowing it is not. If a task catches CancelledError and returns normally, the task group believes it finished, and the cancellation that was meant to stop the program is lost — the classic "Ctrl+C does nothing" bug. Since Python 3.8, CancelledError inherits from BaseException, so except Exception no longer catches it by accident; bare except: and except BaseException: still do.

Cleanup in finally. The partial file is removed whether the download succeeded (it was renamed, so unlink is a no-op), failed, or was cancelled. Anything that must happen on every exit path belongs there, or in an async with.

asyncio.shield for a short critical section. Shielding protects an awaitable from cancellation so it runs to completion, while the surrounding task still sees the CancelledError. Use it only for brief, essential steps — here, the rename that makes a finished download visible. Shielding long operations defeats the purpose of Ctrl+C.

HTTP errors are data, cancellation is control flow. httpx.HTTPError is recorded per URL and the task returns normally, so one bad URL does not cancel the others. Only cancellation and unexpected bugs propagate through the task group.

Bounding shutdown time

Cleanup that awaits the network — closing connections, notifying a server that a job was aborted — can itself hang. Wrap it in a timeout so Ctrl+C always ends the program promptly:

async def notify_aborted(client, job_id: str) -> None:
    try:
        async with asyncio.timeout(2):
            await client.post(f"/jobs/{job_id}/abort")
    except (TimeoutError, httpx.HTTPError):
        pass   # best effort: never block shutdown on a courtesy call

The user's second Ctrl+C is the final backstop, but a tool that needs it has already disappointed them.

UX considerations

• Report the state you left things in. "37 done, 8 cancelled, 75 not started" tells the user exactly what a re-run will do. Keep the counts in a state object outside the coroutine so they survive the interrupt.
• Make re-running safe. Because partial files are always removed and complete ones are renamed atomically, a second run can skip what exists and fetch the rest.
• Exit with 130. By convention, 128 + SIGINT(2). Shells and CI systems recognise it as "interrupted", not "failed".
• No tracebacks for Ctrl+C. Catch KeyboardInterrupt around asyncio.run and print one line. A traceback on an intentional interrupt looks like a crash.
• Respect SIGTERM too. asyncio.run only handles SIGINT. Under a supervisor, container runtime or CI timeout, you get SIGTERM; add a handler with loop.add_signal_handler(signal.SIGTERM, main_task.cancel) on POSIX so it gets the same clean shutdown — see handling SIGTERM and graceful shutdown.

Testing the behaviour

Cancellation logic is testable without signals: cancel the task yourself and assert on the aftermath. For the full path — a real SIGINT to a real process — one integration test using subprocess and send_signal is worth having on POSIX:

# tests/test_cancel.py
import asyncio
import sys

import httpx
import pytest

from mytool import mirror as m


def slow_server(delay_for: set[str]):
    async def handler(request):
        if request.url.path in delay_for:
            await asyncio.sleep(10)
        return httpx.Response(200, content=b"x" * 1000)
    return httpx.MockTransport(handler)


@pytest.fixture
def patched_client(monkeypatch):
    def install(transport):
        real = httpx.AsyncClient
        monkeypatch.setattr(m.httpx, "AsyncClient", lambda **kw: real(transport=transport, **kw))
    return install


def test_cancel_cleans_up_and_records(tmp_path, patched_client):
    patched_client(slow_server({"/b.bin", "/c.bin"}))
    urls = [f"https://files.test/{n}.bin" for n in "abc"]
    state = m.MirrorState(total=3)

    async def scenario():
        task = asyncio.create_task(m.mirror(urls, tmp_path, 3, state))
        await asyncio.sleep(0.2)          # let a.bin finish, b and c hang
        task.cancel()
        with pytest.raises(asyncio.CancelledError):
            await task

    asyncio.run(scenario())
    assert state.done == ["https://files.test/a.bin"]
    assert sorted(state.cancelled) == ["https://files.test/b.bin", "https://files.test/c.bin"]
    assert sorted(p.name for p in tmp_path.iterdir()) == ["a.bin"]   # no .part files


@pytest.mark.skipif(sys.platform == "win32", reason="POSIX signals")
def test_real_sigint_exits_130(tmp_path):
    import signal
    import subprocess
    import textwrap
    import time

    script = tmp_path / "prog.py"
    script.write_text(textwrap.dedent("""
        import asyncio, sys
        async def main():
            try:
                await asyncio.sleep(30)
            finally:
                print("cleaned up", flush=True)
        try:
            asyncio.run(main())
        except KeyboardInterrupt:
            sys.exit(130)
    """))
    proc = subprocess.Popen([sys.executable, str(script)], stdout=subprocess.PIPE, text=True)
    time.sleep(0.5)
    proc.send_signal(signal.SIGINT)
    out, _ = proc.communicate(timeout=5)
    assert proc.returncode == 130
    assert "cleaned up" in out

The first test proves the state object and file cleanup are right under cancellation; the second proves that asyncio.run really turns SIGINT into cancellation that runs finally blocks, which guards against a future refactor that installs its own signal handling incorrectly.

Conclusion

In modern asyncio, Ctrl+C is cancellation: asyncio.run cancels the main task, TaskGroup fans the cancellation out to every child, and each task unwinds through its finally blocks. Your job is to not get in the way — never swallow CancelledError, clean up in finally, shield only brief critical steps, time-box cleanup that touches the network — and then to tell the user exactly what state the work was left in before exiting with 130.

Frequently asked questions

Related

• Up: Concurrency and async in Python CLIs
• Running async code in Typer and Click
• Handling KeyboardInterrupt cleanly
• Handling SIGTERM and graceful shutdown
• Downloading files with progress in Python