Discovering Project Config Files by Walking Up Directories

A tool used inside code repositories usually needs project settings as well as personal ones: the deploy region for this service, the lint rules for this codebase, the API endpoint this team uses. Those belong in a file committed to the repository, where every contributor and CI job sees the same values. The catch is that people run the tool from anywhere in the repository — the root, src/, services/api/tests/ — so the tool has to find the file by itself. git, Ruff, pytest, pre-commit and Black all solve this the same way: start in the current directory and walk up through the parents until a config file is found or a boundary is reached. This guide implements that search, supports a [tool.mytool] table in pyproject.toml, decides where to stop, fits the result into the precedence chain alongside user config, environment variables and flags, and shows users which file won. It belongs to the handling configuration files and environment variables topic.

Prerequisites

• Python 3.11+ for tomllib (or the tomli backport on 3.10), as in reading TOML config with tomllib.
• A user-level config location, typically from storing app data with platformdirs.

How the search works

From the starting directory, check for the tool's config file; if it is not there, move to the parent and check again. Stop at the first match — the nearest config wins, so a subproject can override its parent — or at a boundary. Three boundaries are sensible: the repository root (a directory containing .git), so a config file lying in some unrelated parent directory is never picked up; the home directory, beyond which project configuration makes no sense; and the filesystem root, as a last resort.

The recipe

# src/mytool/discovery.py
from __future__ import annotations

import os
import tomllib
from dataclasses import dataclass
from pathlib import Path
from typing import Any

CONFIG_NAMES = ("mytool.toml", ".mytool.toml")


@dataclass(frozen=True)
class Found:
    path: Path
    data: dict[str, Any]


def _read_candidate(directory: Path) -> Found | None:
    for name in CONFIG_NAMES:
        path = directory / name
        if path.is_file():
            with path.open("rb") as fh:
                return Found(path, tomllib.load(fh))
    pyproject = directory / "pyproject.toml"
    if pyproject.is_file():
        with pyproject.open("rb") as fh:
            table = tomllib.load(fh).get("tool", {}).get("mytool")
        if table is not None:                       # only if it has our table
            return Found(pyproject, table)
    return None


def find_project_config(start: Path | None = None, stop_at: Path | None = None) -> Found | None:
    """Walk up from `start` to the first directory with our config.

    Stops at (and includes) the repository root (a directory containing .git), at `stop_at`
    (default: the home directory), or at the filesystem root.
    """
    if env := os.environ.get("MYTOOL_PROJECT_CONFIG"):
        path = Path(env)
        with path.open("rb") as fh:
            return Found(path, tomllib.load(fh))
    here = (start or Path.cwd()).resolve()
    stop = (stop_at or Path.home()).resolve()
    for directory in (here, *here.parents):
        found = _read_candidate(directory)
        if found:
            return found
        if (directory / ".git").exists() or directory == stop:
            return None
    return None


def merge(defaults: dict[str, Any], *layers: tuple[str, dict[str, Any]]) -> tuple[dict[str, Any], dict[str, str]]:
    """Later layers win. Returns the merged values and where each came from."""
    values, sources = dict(defaults), {k: "default" for k in defaults}
    for label, layer in layers:
        for key, value in layer.items():
            values[key] = value
            sources[key] = label
    return values, sources

Design decisions

Two file names, one winner per directory. mytool.toml is visible; .mytool.toml suits people who prefer dot-files. Checking both in a fixed order keeps the behaviour predictable. Supporting both is common; supporting five is confusing.

pyproject.toml only counts if it has our table. Many directories in a monorepo have a pyproject.toml for unrelated reasons. Treating any pyproject.toml as "found" would stop the search early at a subpackage and ignore the real configuration higher up. Only a file containing [tool.mytool] is a match — the rule Ruff and Black follow as well.

The repository root is a hard boundary. Checking for .git after reading each directory means the root's own config is found, but nothing above it is. That prevents a stray mytool.toml in ~/src/ from silently configuring every repository beneath it.

An explicit override. MYTOOL_PROJECT_CONFIG=path (or a --config flag feeding the same function) bypasses discovery entirely. CI jobs and unusual layouts need a way to say "use this file", and tests can use it too.

Merging keeps sources. merge applies layers in order and records which layer supplied each key, which is what makes a useful config show possible.

Where project config sits in the precedence chain

Project configuration describes the repository, and is shared by everyone who works in it, so it should beat each person's user-level config. Environment variables and flags remain above both, because they express the intent of a specific shell session or a single invocation. In code, the chain is a single call:

from mytool.discovery import find_project_config, merge

DEFAULTS = {"region": "us-east-1", "timeout": 30}

project = find_project_config()
values, sources = merge(
    DEFAULTS,
    ("user config", user_config),                       # ~/.config/mytool/config.toml
    (f"project config ({project.path})", project.data) if project else ("project config", {}),
    ("environment", env_overrides),                     # MYTOOL_* variables, already parsed
    ("command line", flag_overrides),                   # only flags actually given
)

The general rules for building those layers, and for telling given flags from defaults, are covered in config precedence: flags, env, files and defaults.

UX considerations

• Show which files were used. A config show command that prints the discovered project file, the user file and each value with its source ends almost every "why is it using that region?" conversation in one step.
• Print relative paths when inside the repo. mytool.toml (repo root) is easier to read than a long absolute path; see cross-platform paths with pathlib.
• Warn about near misses. If the search passes a file with a close-but-wrong name (mytool.yaml, my-tool.toml), a one-line hint saves a lot of confusion.
• Treat project config as trusted, but not blindly. Anyone who can commit to the repository controls this file. Never execute commands from it without making that trust explicit in the documentation, as with pre-commit hooks and task runners.
• Keep discovery cheap. It runs on every invocation. A handful of stat calls per directory level is negligible; parsing large files is not — read only the candidate that matched.

Testing the behaviour

Build a small directory tree in tmp_path, with a .git marker for the repository root, and exercise each rule:

# tests/test_discovery.py
from pathlib import Path

import pytest

from mytool.discovery import find_project_config, merge


@pytest.fixture
def repo(tmp_path: Path) -> Path:
    root = tmp_path / "home" / "src" / "shop"
    (root / ".git").mkdir(parents=True)
    (root / "services" / "api").mkdir(parents=True)
    return root


def test_found_in_an_ancestor(repo, monkeypatch):
    monkeypatch.delenv("MYTOOL_PROJECT_CONFIG", raising=False)
    (repo / "mytool.toml").write_text('region = "eu-west-1"\n')
    found = find_project_config(repo / "services" / "api", stop_at=repo.parent.parent)
    assert found.path == repo / "mytool.toml" and found.data == {"region": "eu-west-1"}


def test_nearest_wins(repo, monkeypatch):
    monkeypatch.delenv("MYTOOL_PROJECT_CONFIG", raising=False)
    (repo / "mytool.toml").write_text('region = "outer"\n')
    (repo / "services" / ".mytool.toml").write_text('region = "inner"\n')
    assert find_project_config(repo / "services" / "api").data["region"] == "inner"


def test_pyproject_table_counts_only_if_present(repo, monkeypatch):
    monkeypatch.delenv("MYTOOL_PROJECT_CONFIG", raising=False)
    (repo / "services" / "api" / "pyproject.toml").write_text('[project]\nname = "api"\n')
    (repo / "pyproject.toml").write_text('[tool.mytool]\nregion = "from-pyproject"\n')
    found = find_project_config(repo / "services" / "api")
    assert found.path == repo / "pyproject.toml" and found.data["region"] == "from-pyproject"


def test_stops_at_the_repository_root(repo, monkeypatch):
    monkeypatch.delenv("MYTOOL_PROJECT_CONFIG", raising=False)
    (repo.parent / "mytool.toml").write_text('region = "outside the repo"\n')
    assert find_project_config(repo / "services" / "api") is None


def test_env_override(tmp_path, monkeypatch):
    cfg = tmp_path / "explicit.toml"
    cfg.write_text('region = "explicit"\n')
    monkeypatch.setenv("MYTOOL_PROJECT_CONFIG", str(cfg))
    assert find_project_config(tmp_path).data["region"] == "explicit"


def test_merge_reports_sources():
    values, sources = merge({"region": "us-east-1", "timeout": 30},
                            ("user config", {"timeout": 45}),
                            ("project config", {"region": "eu-west-1"}),
                            ("MYTOOL_TIMEOUT", {"timeout": 60}))
    assert values == {"region": "eu-west-1", "timeout": 60}
    assert sources == {"region": "project config", "timeout": "MYTOOL_TIMEOUT"}

The "stops at the repository root" test is the important safety property: it places a config file outside the repository and asserts it is not used. Clearing MYTOOL_PROJECT_CONFIG in each test keeps a developer's own environment from leaking in.

Conclusion

Project configuration should be found, not pointed at. Walk up from the current directory to the nearest mytool.toml, .mytool.toml or pyproject.toml with a [tool.mytool] table; stop at the repository root or home directory; offer an explicit override for CI and unusual layouts; place the result above user config and below environment variables and flags; and record where every value came from so config show can explain it. That is the behaviour users already know from git, Ruff and pytest — and it makes the tool work the same from every directory in the project.

Frequently asked questions

Related

• Up: Handling configuration files and environment variables
• Reading TOML config with tomllib
• Typed settings with pydantic-settings
• Config precedence: flags, env, files and defaults
• Supporting multiple profiles and accounts