Config Precedence: Flags, Env, Files, Defaults

A CLI reads the same setting from four places — a flag you typed, an environment variable in your shell, a config file on disk, and a built-in default — and the interesting question is never "how do I read one," it's "which one wins." Get the precedence wrong and users hit the maddening bug where --port 9000 is silently ignored because a config file quietly overrode it. This page fixes the order once, gives you a runnable resolver that merges all four layers, and shows how to make --help reveal where each value actually came from.

TL;DR

• The canonical order, highest to lowest: flags > environment variables > config file > defaults.
• Merge layers low-to-high into one dict so higher sources overwrite lower ones key by key.
• Only override with values a source actually set — an unset flag (None) must not clobber a real config value.
• Click can do much of this for you with auto_envvar_prefix and default_map.
• Make precedence testable by passing argv, environ, and file contents in as arguments instead of reading globals.

The canonical order, and why

Precedence follows one principle: the closer a value is to the moment of invocation, the more it should win. A flag is the most immediate expression of intent — you typed it just now, for this run — so it beats everything. An env var is scoped to your shell session or deployment. A config file is the most persistent and shared, so it yields to both. Defaults are the last resort when nobody said anything.

This is the order users expect because every well-behaved tool they already use — git, docker, kubectl — works this way. Violate it and the surprise is expensive: nothing is more confusing than a flag that appears to do nothing. (If your file layer itself splits into a project file and a user file, slot them between env and defaults; the parent config guide walks through that five-level chain.)

A runnable layered resolver

The mechanic is a chain of dict.update calls from lowest priority to highest, so each layer overwrites the ones before it. The one rule that makes it correct: a layer contributes only the keys it actually defines. This program runs as-is:

from __future__ import annotations
import os

DEFAULTS = {"host": "localhost", "port": 8000, "timeout": 10, "verbose": False}

ENV_MAP = {
    "MYCLI_HOST": "host",
    "MYCLI_PORT": "port",
    "MYCLI_TIMEOUT": "timeout",
    "MYCLI_VERBOSE": "verbose",
}


def from_env(environ: dict[str, str]) -> dict:
    return {key: environ[name] for name, key in ENV_MAP.items() if name in environ}


def from_flags(flags: dict) -> dict:
    # Drop unset options so `--port` left off doesn't overwrite lower layers.
    return {key: value for key, value in flags.items() if value is not None}


def resolve(file_cfg: dict, environ: dict[str, str], flags: dict) -> dict:
    """Merge low -> high: defaults < file < env < flags."""
    merged: dict = {}
    merged.update(DEFAULTS)          # lowest
    merged.update(file_cfg)          # config file
    merged.update(from_env(environ)) # environment
    merged.update(from_flags(flags)) # flags win
    return merged


if __name__ == "__main__":
    file_cfg = {"host": "db.internal", "port": 5432, "timeout": 30}
    environ = {"MYCLI_PORT": "9000", "MYCLI_VERBOSE": "true"}
    flags = {"host": "cli-host", "port": None, "timeout": None, "verbose": None}
    print(resolve(file_cfg, environ, flags))

{'host': 'cli-host', 'port': '9000', 'timeout': 30, 'verbose': 'true'}

Trace each key: host came from the flag, port from the env var (overriding the file's 5432), timeout from the file (nothing higher set it), and verbose from the env. That's the precedence table, executed. The from_flags filter is the load-bearing detail — without it, every option your parser defaults to None would stomp the config file with a blank.

Coerce types after merging, not before

Env vars and many file formats hand you strings: port above ends up as '9000', not 9000. Resist coercing inside each layer — do it once, at the end, against a schema. That keeps every source honest against the same types and unknown-key rules. Feed the merged dict into a Pydantic v2 model exactly as the parent config guide does:

from pydantic import BaseModel, ConfigDict

class AppConfig(BaseModel):
    model_config = ConfigDict(extra="forbid")
    host: str = "localhost"
    port: int = 8000
    timeout: int = 10
    verbose: bool = False

config = AppConfig.model_validate(resolve(file_cfg, environ, flags))
# port -> 9000 (int), verbose -> True (bool)

Validating after the merge means "9000" becomes 9000 and "true" becomes True in one place, and a typo'd key fails loudly instead of being silently dropped. The YAML side of that file layer — loading it safely — is covered in loading YAML configs safely.

Letting Click layer it for you

If you're on Click, two built-ins cover the env and file layers so you write less merge code. auto_envvar_prefix reads MYCLI_PORT for a --port option automatically, and default_map (set from a loaded config file in the group callback) supplies file-level defaults. Click's own resolution order is exactly the canonical one: an explicit flag beats the env var, which beats default_map, which beats the option's default.

import click

def load_file_config() -> dict:
    # Read + parse your config.yaml/toml here; return {} if absent.
    return {"port": 5432, "host": "db.internal"}

@click.group(context_settings={"auto_envvar_prefix": "MYCLI"})
@click.pass_context
def cli(ctx: click.Context) -> None:
    ctx.default_map = load_file_config()   # file layer feeds option defaults

@cli.command()
@click.option("--host", default="localhost")
@click.option("--port", type=int, default=8000)
def serve(host: str, port: int) -> None:
    click.echo(f"{host}:{port}")

Now serve resolves port from --port if given, else MYCLI_PORT, else the file's 5432, else 8000 — no manual merge. The default_map is keyed by command name for groups ({"serve": {"port": ...}}), so nest it accordingly. Sharing that loaded config across subcommands is a natural fit for a Click context object.

Show users where a value came from

Precedence is invisible until it bites, so the best CLIs make the effective source discoverable. Instead of only resolving the value, resolve the origin alongside it, and expose it behind a --show-config flag or in --help's epilog:

def resolve_with_source(file_cfg, environ, flags) -> dict[str, tuple]:
    env_cfg, flag_cfg = from_env(environ), from_flags(flags)
    result = {}
    for key in DEFAULTS:
        if key in flag_cfg:
            result[key] = (flag_cfg[key], "flag")
        elif key in env_cfg:
            result[key] = (env_cfg[key], "env")
        elif key in file_cfg:
            result[key] = (file_cfg[key], "file")
        else:
            result[key] = (DEFAULTS[key], "default")
    return result

$ mycli --show-config
host    = cli-host   (flag)
port    = 9000       (env)
timeout = 30         (file)
verbose = true       (env)

That table turns "why is my flag ignored?" support tickets into a five-second self-diagnosis. It's cheap to build because it reuses the same per-layer dicts your resolver already computes.

Per-key versus whole-file override

Decide explicitly whether a higher layer overrides a config file per key or wholesale. The dict.update approach above is per-key and shallow: setting MYCLI_PORT overrides only port, leaving the file's host intact — which is almost always what users want. The trap is nesting. If your config has a nested table, a shallow update replaces the entire sub-table, silently dropping sibling keys:

file_cfg = {"db": {"host": "a", "port": 1}}
override = {"db": {"port": 2}}
# shallow: {"db": {"port": 2}} — "host" is GONE

If you support nested config, deep-merge the mappings recursively so db.host survives while db.port is overridden. For flat config, the shallow merge is correct and simpler — don't reach for recursion you don't need.

Production notes

• Flags default to a sentinel, not a value. Give options a None default (or click's automatic one) so "unset" is distinguishable from "set to the default." Otherwise you can't tell whether to override a lower layer.
• Document the order in --help. State "flags > env > config > defaults" once so users never have to reverse-engineer it.
• Test precedence with a table. Because resolve takes file_cfg, environ, and flags as arguments, a pytest.mark.parametrize sweep over combinations pins every rule; don't read os.environ directly inside the resolver.
• Verbosity flows through here too. A --verbose flag should beat MYCLI_VERBOSE should beat a config value — wire it through this same chain rather than special-casing it, as noted in adding verbose and quiet flags.
• TOML is identical. Swap the file parser for tomllib (stdlib since 3.11); the merge and validation layers are unchanged.

Related

• Up: Handling config files and env vars in CLIs
• Sideways: Loading YAML configs safely in CLI apps
• Sideways: Adding verbose and quiet logging flags