Reading Secrets from Env Vars and Files in CLIs

A person at a terminal can log in once and let the keychain remember. Automation cannot: a GitHub Actions job, a Kubernetes CronJob or a Docker container has to receive its credentials from outside, from whatever secret manager the platform uses. Those platforms deliver secrets in two standard shapes — as environment variables and as files mounted into the filesystem — and a CLI that supports both, correctly, drops into any pipeline without wrappers. This guide implements a small secret-reading helper that handles MYTOOL_TOKEN, the MYTOOL_TOKEN_FILE convention and --token-stdin, checks file permissions, strips the trailing newline that breaks so many tokens, and keeps the secrets out of child processes. It is part of the secrets and credentials topic.

Prerequisites

• Python 3.10+ and Typer.
• A CLI that needs one or more credentials — API tokens, a database password, a signing key.
• For the interactive side of the same story, see storing tokens with keyring.

Why not just a --token flag?

Because arguments are the most public place a secret can be. On Linux and macOS, every user on the machine can list every process's full command line for as long as it runs; shells save commands to history files; CI systems echo the commands they run into logs that many people can read.

Environment variables and files avoid all three. A process's environment is readable only by the same user and root, and it is not written to history. A file's visibility is governed by its permissions, and it is not inherited by child processes at all.

Environment variable or file?

Both are worth supporting, because different platforms favour different ones:

Environment variables are universal and convenient: every CI system can set them from its secret store, and they are trivial to use locally (MYTOOL_TOKEN=... mytool deploy). Their weaknesses are inheritance — every subprocess gets them — and the fact that they are fixed for the lifetime of the process.

Files are what container orchestrators prefer. Docker secrets appear under /run/secrets/<name>; Kubernetes can mount secrets as files anywhere; Vault Agent and similar sidecars write rendered secrets to disk and refresh them on rotation. A long-running process that re-reads the file picks up a rotated credential without a restart.

The convention that ties them together comes from official Docker images such as postgres and mysql: for any secret variable NAME, also accept NAME_FILE containing a path to read the value from. Operators recognise it immediately.

The recipe

# src/mytool/secrets.py
from __future__ import annotations

import os
import stat
import sys
import warnings
from pathlib import Path


class Secret:
    __slots__ = ("_v",)

    def __init__(self, value: str) -> None:
        self._v = value

    def reveal(self) -> str:
        return self._v

    def __repr__(self) -> str:
        return "Secret('********')"

    __str__ = __repr__


class MissingSecret(Exception):
    pass


def _read_file(path: Path) -> str:
    try:
        info = path.stat()
    except FileNotFoundError:
        raise MissingSecret(f"secret file {path} does not exist") from None
    if os.name == "posix" and info.st_mode & (stat.S_IRWXG | stat.S_IRWXO):
        warnings.warn(f"{path} is readable by other users; chmod 600 it", stacklevel=3)
    value = path.read_text(encoding="utf-8")
    return value.rstrip("\r\n")                # editors and `echo` add a newline


def read_secret(name: str, *, stdin: bool = False, required: bool = True) -> Secret | None:
    """Resolve secret NAME from stdin, NAME_FILE or NAME, in that order."""
    if stdin:
        value = sys.stdin.readline().rstrip("\r\n")
        if not value:
            raise MissingSecret(f"expected {name} on stdin, got nothing")
        return Secret(value)
    if file_path := os.environ.get(f"{name}_FILE"):
        if name in os.environ:
            warnings.warn(f"both {name} and {name}_FILE are set; using {name}_FILE", stacklevel=2)
        return Secret(_read_file(Path(file_path)))
    if value := os.environ.get(name):
        return Secret(value.strip())
    if required:
        raise MissingSecret(f"set {name} or {name}_FILE (or pass it on stdin)")
    return None


def child_env(*, keep: tuple[str, ...] = (), drop_prefixes: tuple[str, ...] = ("MYTOOL_",)) -> dict[str, str]:
    """A copy of the environment without our secrets, for subprocesses."""
    return {
        k: v for k, v in os.environ.items()
        if k in keep or not k.startswith(drop_prefixes)
    }

And a command that uses it:

# src/mytool/cli.py
import subprocess

import typer

from mytool.secrets import MissingSecret, child_env, read_secret

app = typer.Typer()


@app.callback()
def main() -> None:
    """Deployment tool."""


@app.command()
def deploy(
    env: str,
    token_stdin: bool = typer.Option(False, "--token-stdin", help="Read the API token from stdin."),
) -> None:
    """Deploy the current build to ENV."""
    try:
        token = read_secret("MYTOOL_TOKEN", stdin=token_stdin)
    except MissingSecret as exc:
        typer.echo(f"error: {exc}", err=True)
        raise typer.Exit(2)
    headers = {"Authorization": f"Bearer {token.reveal()}"}
    typer.echo(f"deploying to {env} (auth header prepared: {len(headers)} header)", err=True)
    # Build tools run with our secrets removed from their environment.
    subprocess.run(["npm", "run", "build"], env=child_env(), check=True)


if __name__ == "__main__":
    app()

The details that matter

Strip the trailing newline — and only that. echo "$TOKEN" > token.txt, most editors and many secret managers add a final newline. A token with \n on the end fails authentication with a baffling "invalid token" error. rstrip("\r\n") removes line endings without touching legitimate whitespace inside a password.

_FILE wins over the plain variable. When both are set, the file is almost certainly the operator's deliberate choice (a mounted secret) and the variable a leftover default. Warn so the ambiguity is visible.

Check permissions on POSIX. A secret file readable by the group or others is a misconfiguration worth flagging. A warning rather than a refusal keeps the tool usable on platforms and mounts where permission bits are not meaningful — some container volume types report 0644 regardless.

Scrub the environment for children. child_env() removes your tool's own variables before running subprocesses. Third-party build scripts, package install hooks and linters do not need your deploy token. For stricter isolation, build the child environment from an allow-list instead — PATH, HOME, LANG and whatever the child actually needs — as discussed in calling external commands safely with subprocess.

Wrap immediately. The value is a Secret from the moment it is read, so a stray print(token) or a traceback prints asterisks. reveal() appears exactly where the header is built.

UX considerations

• Name every option in the error. "set MYTOOL_TOKEN or MYTOOL_TOKEN_FILE (or pass it on stdin)" tells a CI engineer exactly what to configure. List these variables in --help too — Typer's envvar= shows them automatically for options that support it.
• Offer --token-stdin. It lets users pipe from a password manager (pass show mytool | mytool deploy prod --token-stdin) without the secret touching argv or the environment.
• Document the precedence alongside your other settings, using the same order as in config precedence: flags, env, files and defaults: explicit flag, then _FILE, then plain variable, then stored credential.
• Do not load secrets from .env files implicitly. Auto-loading a .env from the current directory is convenient for development and surprising in production, where a stray file can override a real secret. If you support it, make it opt-in with a flag.
• Never echo the value in errors. "token rejected" and "token from MYTOOL_TOKEN_FILE (/run/secrets/mytool) was rejected" are both fine; including the token is not.

Testing the behaviour

monkeypatch controls the environment and tmp_path provides files with known permissions, so every branch is testable in isolation:

# tests/test_secrets.py
import os
import stat
import sys

import pytest

from mytool.secrets import MissingSecret, child_env, read_secret


@pytest.fixture(autouse=True)
def clean_env(monkeypatch):
    for var in ("MYTOOL_TOKEN", "MYTOOL_TOKEN_FILE"):
        monkeypatch.delenv(var, raising=False)


def test_plain_variable(monkeypatch):
    monkeypatch.setenv("MYTOOL_TOKEN", "abc123")
    assert read_secret("MYTOOL_TOKEN").reveal() == "abc123"


def test_file_strips_newline(monkeypatch, tmp_path):
    f = tmp_path / "token"
    f.write_text("s3cret-with-trailing-newline\n")
    f.chmod(0o600)
    monkeypatch.setenv("MYTOOL_TOKEN_FILE", str(f))
    assert read_secret("MYTOOL_TOKEN").reveal() == "s3cret-with-trailing-newline"


def test_file_beats_variable_with_warning(monkeypatch, tmp_path):
    f = tmp_path / "token"
    f.write_text("from-file")
    f.chmod(0o600)
    monkeypatch.setenv("MYTOOL_TOKEN_FILE", str(f))
    monkeypatch.setenv("MYTOOL_TOKEN", "from-env")
    with pytest.warns(UserWarning, match="both"):
        assert read_secret("MYTOOL_TOKEN").reveal() == "from-file"


@pytest.mark.skipif(sys.platform == "win32", reason="POSIX permissions")
def test_world_readable_file_warns(monkeypatch, tmp_path):
    f = tmp_path / "token"
    f.write_text("x")
    f.chmod(0o644)
    monkeypatch.setenv("MYTOOL_TOKEN_FILE", str(f))
    with pytest.warns(UserWarning, match="chmod 600"):
        read_secret("MYTOOL_TOKEN")


def test_missing_names_both_options():
    with pytest.raises(MissingSecret, match="MYTOOL_TOKEN_FILE"):
        read_secret("MYTOOL_TOKEN")


def test_child_env_drops_our_secrets(monkeypatch):
    monkeypatch.setenv("MYTOOL_TOKEN", "abc")
    monkeypatch.setenv("PATH", os.environ.get("PATH", "/usr/bin"))
    env = child_env()
    assert "MYTOOL_TOKEN" not in env and "PATH" in env


def test_secret_repr_is_masked(monkeypatch):
    monkeypatch.setenv("MYTOOL_TOKEN", "abc123")
    assert "abc123" not in repr(read_secret("MYTOOL_TOKEN"))

For command-level tests, CliRunner.invoke(app, [...], env={"MYTOOL_TOKEN": "x"}) sets variables for one invocation, and input="token\n" feeds --token-stdin. See testing interactive prompts and stdin for more on driving stdin in tests.

Conclusion

Automation delivers secrets through environment variables and mounted files, and a CLI that accepts both — plus stdin for piping from password managers — fits every platform without glue scripts. Support the _FILE convention, strip only the trailing newline, warn about loose permissions, wrap values in a masking type the moment you read them, and keep your secrets out of the environment of every child process. None of it is complicated; all of it prevents a real, recurring leak.

Frequently asked questions

Related

• Up: Secrets and credentials in Python CLIs
• Storing tokens with keyring
• Prompting for passwords securely
• Redacting secrets from CLI output and logs
• Publishing to PyPI with trusted publishing