Reading Arguments from Files with argparse’s fromfile_prefix_chars

Some command lines get long. A build tool invoked with three hundred source files exceeds the Windows command-line length limit of about 32,000 characters. A nightly job runs the same fifteen flags every time, and the cron line has become unreadable. A team shares a set of "standard" options that everyone is supposed to pass. Compilers and linkers solved this decades ago with response files: write the arguments into a file and pass @args.txt instead. argparse supports exactly this with one constructor argument, fromfile_prefix_chars. Its defaults are minimal, though — one argument per line, no comments, no quoting — and it has a sharp edge where real values begin with @. This guide enables response files, makes them pleasant to write, explains the edge cases, and tests them. It belongs to the argparse topic.

Prerequisites

• Python 3.10+ and an argparse-based CLI.
• Familiarity with shell quoting; response-file lines will follow the same rules.

How response files work

When fromfile_prefix_chars="@" is set, any command-line argument that starts with @ is replaced by the arguments read from the named file, in place, before parsing. The replacement can contain further @file references, which are expanded too, so presets can include other presets.

By default, each line of the file is exactly one argument. That is unambiguous, but it means --to /mnt/backup must be written on two lines, and a # comment line becomes a literal positional argument named # comment. Both are easy to fix.

The recipe

# src/backup/cli.py
from __future__ import annotations

import argparse
import shlex


class ResponseFileParser(argparse.ArgumentParser):
    """Response files with shell-style quoting and # comments."""

    def convert_arg_line_to_args(self, arg_line: str) -> list[str]:
        return shlex.split(arg_line, comments=True)


def build_parser() -> argparse.ArgumentParser:
    parser = ResponseFileParser(
        prog="backup",
        fromfile_prefix_chars="@",
        description="Back up directories. Arguments can be read from a file with @FILE.",
    )
    parser.add_argument("sources", nargs="*", metavar="SOURCE")
    parser.add_argument("--to", metavar="DEST")
    parser.add_argument("--exclude", action="append", default=[], metavar="GLOB")
    parser.add_argument("--notify", metavar="CHANNEL")
    return parser


def main(argv: list[str] | None = None) -> int:
    args = build_parser().parse_args(argv)
    print(f"{len(args.sources)} source(s) -> {args.to}, excluding {args.exclude}")
    return 0

convert_arg_line_to_args is the documented hook for changing how lines become arguments. shlex.split(..., comments=True) gives each line shell semantics: several arguments per line, quotes for values with spaces, and # starting a comment. A preset file now reads naturally:

# nightly.args — standard options for the nightly backup
--to /mnt/backup            # local disk; the S3 copy runs separately
--exclude "*.tmp" --exclude "my docs/cache"
@notify.args                # shared notification settings

backup @nightly.args ~/projects ~/documents

Arguments from the file are inserted where @nightly.args appears, so options given after it on the command line override the preset — backup @nightly.args --to /mnt/other ~/projects uses /mnt/other for a single-value option. For append options such as --exclude, both sources accumulate.

The @value gotcha

Expansion applies to every argument that starts with the prefix — not just ones that look like file names. A perfectly reasonable value fails:

$ backup --notify @alice
backup: error: [Errno 2] No such file or directory: 'alice'

Users can work around it with the = form, which makes the value part of the option argument and therefore does not start with @: backup --notify=@alice works. Better, avoid the conflict by design: if your CLI takes values that commonly start with @ — user handles, npm scopes, decorator names — choose a different prefix character, or document the = form prominently in the option's help text. fromfile_prefix_chars accepts several characters (for example "@+") if you need to support an established convention alongside your own.

Encoding and Windows

Response files are most useful on Windows, where the command-line limit bites first. Since Python 3.12, argparse reads them with the filesystem encoding and error handler (UTF-8 on modern systems), rather than the locale encoding used by earlier versions. If your users generate response files with tools that write UTF-16 — some Windows editors and PowerShell's older Out-File default — they will see decoding errors; tell them to save files as UTF-8, as discussed in fixing Unicode and encoding errors on Windows.

The same feature for Click and Typer

Click and Typer have no response-file support, but expansion is simple to do yourself before the framework sees the arguments — and doing it yourself lets you fix the -- behaviour too:

# src/mytool/respfiles.py
from __future__ import annotations

import shlex
from pathlib import Path


def expand(argv: list[str], prefix: str = "@", _depth: int = 0) -> list[str]:
    """Replace @FILE arguments with the shell-split contents of FILE (before any '--')."""
    if _depth > 10:
        raise ValueError("response files nested too deeply")
    out: list[str] = []
    for i, arg in enumerate(argv):
        if arg == "--":
            return out + argv[i:]               # everything after -- is literal
        if arg.startswith(prefix) and len(arg) > 1:
            text = Path(arg[1:]).read_text(encoding="utf-8")
            out += expand(shlex.split(text, comments=True), prefix, _depth + 1)
        else:
            out.append(arg)
    return out


def main() -> None:
    import sys

    from mytool.cli import app
    app(args=expand(sys.argv[1:]), prog_name="mytool")

Passing args= to the Typer or Click app replaces sys.argv[1:], and prog_name keeps usage messages naming the command correctly. The depth limit stops a file that includes itself from recursing forever, and stopping at -- gives users a way to pass a literal @alice as a positional. Turn a missing file into a usage error (exit code 2) rather than a traceback by catching FileNotFoundError in main().

UX considerations

• Mention it in help. The description above says "Arguments can be read from a file with @FILE". Nobody discovers response files on their own.
• Keep paths relative to the working directory — that is how argparse resolves them, and it matches what users expect from compilers. A preset that includes another preset by relative path works only from the right directory; document that, or use absolute paths in shared presets.
• Prefer config files for settings, response files for invocations. A TOML config with precedence rules, as in config precedence: flags, env, files, defaults, is better for long-lived settings; response files shine for long or generated argument lists and named presets of a particular run.
• Show the expanded command line when verbose. Printing shlex.join(expanded_args) at -v makes presets debuggable.
• Never treat response files as trusted input if they can come from someone else; they are exactly as powerful as typing the arguments.

Testing the behaviour

Response files are easy to test with tmp_path and monkeypatch.chdir, since paths resolve against the working directory:

# tests/test_response_files.py
import pytest

from backup.cli import build_parser


@pytest.fixture
def in_tmp(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    return tmp_path


def test_lines_support_quotes_comments_and_nesting(in_tmp):
    (in_tmp / "notify.args").write_text("--notify ops\n")
    (in_tmp / "nightly.args").write_text(
        "# preset\n--to /mnt/backup   # local\n--exclude \"*.tmp\" --exclude \"my docs/cache\"\n"
        "@notify.args\n"
    )
    args = build_parser().parse_args(["@nightly.args", "src"])
    assert args.to == "/mnt/backup"
    assert args.exclude == ["*.tmp", "my docs/cache"]
    assert args.notify == "ops" and args.sources == ["src"]


def test_later_arguments_override_the_preset(in_tmp):
    (in_tmp / "p.args").write_text("--to /mnt/backup\n")
    assert build_parser().parse_args(["@p.args", "--to", "/elsewhere"]).to == "/elsewhere"


def test_at_values_need_the_equals_form(in_tmp, capsys):
    with pytest.raises(SystemExit):
        build_parser().parse_args(["--notify", "@alice"])
    assert "No such file or directory" in capsys.readouterr().err
    assert build_parser().parse_args(["--notify=@alice"]).notify == "@alice"

The last test documents the gotcha so a future refactor does not "fix" it in a way that silently changes behaviour — and makes the trade-off visible to whoever decides on the prefix character.

Conclusion

fromfile_prefix_chars="@" gives an argparse CLI response files for free: long argument lists that dodge command-line limits, and named presets that make scheduled jobs readable. Override convert_arg_line_to_args with shlex.split(..., comments=True) so files can use quoting, several arguments per line and comments; remember that options after the @file override it; document the --opt=@value form or choose another prefix if values legitimately start with @; and test expansion from a temporary directory.

Frequently asked questions

Related

• Up: Command-line parsing with argparse
• Argument groups and help formatting in argparse
• Writing custom argparse actions
• Following POSIX and GNU argument conventions
• Running a CLI on a schedule with cron and systemd