[{"data":1,"prerenderedAt":2646},["ShallowReactive",2],{"page-\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fmultiprocessing-for-cpu-bound-cli-tasks\u002F":3,"content-directory":2099},{"id":4,"title":5,"body":6,"date":2084,"description":2085,"difficulty":2086,"draft":2087,"extension":2088,"meta":2089,"navigation":130,"path":2090,"seo":2091,"stem":2092,"tags":2093,"updated":2084,"__hash__":2098},"content\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fmultiprocessing-for-cpu-bound-cli-tasks\u002Findex.md","Multiprocessing for CPU-Bound CLI Tasks",{"type":7,"value":8,"toc":2066},"minimark",[9,29,34,56,60,63,67,82,86,89,994,1412,1417,1431,1434,1440,1462,1484,1487,1497,1501,1552,1556,1563,1939,1945,1949,1960,1964,1974,1994,1998,2001,2005,2020,2024,2027,2031,2062],[10,11,12,13,17,18,22,23,28],"p",{},"Your command parses 5,000 log files, validates a large dataset, renders hundreds of templates or computes statistics over millions of rows — in pure Python — and it pins one CPU core at 100% for four minutes while the rest of the machine idles. You try a thread pool and it gets slightly ",[14,15,16],"em",{},"slower",". That is the global interpreter lock at work: in standard CPython only one thread executes Python bytecode at a time, so threads help with waiting but not with computing. To use every core, a CLI needs multiple processes. This guide shows when that is worth it, how to structure a ",[19,20,21],"code",{},"ProcessPoolExecutor"," so it works identically on Linux, macOS and Windows, how to keep pickling costs from eating the gains, and how to stop cleanly on Ctrl+C. It is part of the ",[24,25,27],"a",{"href":26},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002F","concurrency and async topic",".",[30,31,33],"h2",{"id":32},"prerequisites","Prerequisites",[35,36,37,41,53],"ul",{},[38,39,40],"li",{},"Python 3.10+ and a Typer or Click CLI installed as a package (with a console-script entry point).",[38,42,43,44,48,49,52],{},"Work that is genuinely ",[45,46,47],"strong",{},"CPU-bound in Python",". Confirm it first: if ",[19,50,51],{},"--jobs 4"," with threads is no faster than one thread, and the CPU is at 100% on one core, it is.",[38,54,55],{},"Work that splits into independent pieces — per file, per record batch, per template.",[30,57,59],{"id":58},"why-threads-do-not-help-here","Why threads do not help here",[10,61,62],{},"A thread pool shines when each task mostly waits: during a network call or disk read, the thread releases the GIL and others run. Pure-Python computation never waits, so threads take turns holding the lock and the total time stays the same — plus the overhead of switching.",[64,65],"inline-diagram",{"name":66},"cc-gil-timeline",[10,68,69,70,73,74,77,78,81],{},"Two caveats before reaching for processes. First, many \"CPU-heavy\" libraries do their work in C and release the GIL — ",[19,71,72],{},"hashlib"," on large buffers, ",[19,75,76],{},"zlib",", NumPy, Pillow's resizing, ",[19,79,80],{},"lxml"," parsing. For those, a thread pool may already scale; measure before switching. Second, free-threaded CPython builds (3.13t and later) remove the GIL entirely. They are promising but still opt-in and not what your users have installed, so for distributed tools processes remain the portable answer.",[30,83,85],{"id":84},"the-recipe","The recipe",[10,87,88],{},"The example command computes per-file statistics over a directory of log files: request counts, error counts and the slowest endpoints. The parsing is regular-expression and string work — pure Python, CPU-bound.",[90,91,96],"pre",{"className":92,"code":93,"language":94,"meta":95,"style":95},"language-python shiki shiki-themes github-light github-dark","# src\u002Fmytool\u002Fstats.py\nfrom __future__ import annotations\n\nimport os\nimport re\nfrom collections import Counter\nfrom concurrent.futures import ProcessPoolExecutor\nfrom dataclasses import dataclass, field\nfrom pathlib import Path\n\nLINE = re.compile(r'\"(?P\u003Cmethod>[A-Z]+) (?P\u003Cpath>\\S+) [^\"]*\" (?P\u003Cstatus>\\d{3}) .* (?P\u003Cms>\\d+)ms$')\n\n\n@dataclass\nclass Stats:\n    requests: int = 0\n    errors: int = 0\n    slow: Counter[str] = field(default_factory=Counter)\n\n    def merge(self, other: Stats) -> Stats:\n        self.requests += other.requests\n        self.errors += other.errors\n        self.slow.update(other.slow)\n        return self\n\n\ndef analyse_file(path: str) -> Stats:\n    \"\"\"Runs in a worker process: must be importable and take\u002Freturn picklable values.\"\"\"\n    stats = Stats()\n    with open(path, encoding=\"utf-8\", errors=\"replace\") as fh:\n        for line in fh:\n            m = LINE.search(line)\n            if not m:\n                continue\n            stats.requests += 1\n            if m[\"status\"].startswith(\"5\"):\n                stats.errors += 1\n            if int(m[\"ms\"]) > 1000:\n                stats.slow[m[\"path\"]] += 1\n    return stats\n\n\ndef default_jobs() -> int:\n    try:\n        return len(os.sched_getaffinity(0))   # respects container CPU limits on Linux\n    except AttributeError:\n        return os.cpu_count() or 1\n\n\ndef analyse(paths: list[Path], jobs: int) -> Stats:\n    total = Stats()\n    if jobs == 1:\n        for p in paths:\n            total.merge(analyse_file(str(p)))\n        return total\n    chunksize = max(1, len(paths) \u002F\u002F (jobs * 4))\n    with ProcessPoolExecutor(max_workers=jobs) as pool:\n        try:\n            for stats in pool.map(analyse_file, map(str, paths), chunksize=chunksize):\n                total.merge(stats)\n        except KeyboardInterrupt:\n            pool.shutdown(wait=False, cancel_futures=True)\n            raise\n    return total\n","python","",[19,97,98,107,125,132,141,149,162,175,188,201,206,313,318,323,330,342,356,368,395,400,412,427,440,448,457,462,467,484,490,501,541,555,569,581,587,598,618,628,653,669,678,683,688,703,711,731,742,755,760,765,780,790,807,820,831,839,877,898,906,938,944,955,981,987],{"__ignoreMap":95},[99,100,103],"span",{"class":101,"line":102},"line",1,[99,104,106],{"class":105},"sJ8bj","# src\u002Fmytool\u002Fstats.py\n",[99,108,110,114,118,121],{"class":101,"line":109},2,[99,111,113],{"class":112},"szBVR","from",[99,115,117],{"class":116},"sj4cs"," __future__",[99,119,120],{"class":112}," import",[99,122,124],{"class":123},"sVt8B"," annotations\n",[99,126,128],{"class":101,"line":127},3,[99,129,131],{"emptyLinePlaceholder":130},true,"\n",[99,133,135,138],{"class":101,"line":134},4,[99,136,137],{"class":112},"import",[99,139,140],{"class":123}," os\n",[99,142,144,146],{"class":101,"line":143},5,[99,145,137],{"class":112},[99,147,148],{"class":123}," re\n",[99,150,152,154,157,159],{"class":101,"line":151},6,[99,153,113],{"class":112},[99,155,156],{"class":123}," collections ",[99,158,137],{"class":112},[99,160,161],{"class":123}," Counter\n",[99,163,165,167,170,172],{"class":101,"line":164},7,[99,166,113],{"class":112},[99,168,169],{"class":123}," concurrent.futures ",[99,171,137],{"class":112},[99,173,174],{"class":123}," ProcessPoolExecutor\n",[99,176,178,180,183,185],{"class":101,"line":177},8,[99,179,113],{"class":112},[99,181,182],{"class":123}," dataclasses ",[99,184,137],{"class":112},[99,186,187],{"class":123}," dataclass, field\n",[99,189,191,193,196,198],{"class":101,"line":190},9,[99,192,113],{"class":112},[99,194,195],{"class":123}," pathlib ",[99,197,137],{"class":112},[99,199,200],{"class":123}," Path\n",[99,202,204],{"class":101,"line":203},10,[99,205,131],{"emptyLinePlaceholder":130},[99,207,209,212,215,218,221,225,229,232,236,239,242,245,248,251,254,256,258,261,264,267,270,273,275,278,281,284,286,289,291,293,296,298,300,302,305,308,310],{"class":101,"line":208},11,[99,210,211],{"class":116},"LINE",[99,213,214],{"class":112}," =",[99,216,217],{"class":123}," re.compile(",[99,219,220],{"class":112},"r",[99,222,224],{"class":223},"sZZnC","'",[99,226,228],{"class":227},"sA_wV","\"",[99,230,231],{"class":116},"(",[99,233,235],{"class":234},"s9eBZ","?P\u003Cmethod>",[99,237,238],{"class":116},"[A-Z]",[99,240,241],{"class":112},"+",[99,243,244],{"class":116},")",[99,246,247],{"class":116}," (",[99,249,250],{"class":234},"?P\u003Cpath>",[99,252,253],{"class":116},"\\S",[99,255,241],{"class":112},[99,257,244],{"class":116},[99,259,260],{"class":116}," [",[99,262,263],{"class":112},"^",[99,265,266],{"class":116},"\"]",[99,268,269],{"class":112},"*",[99,271,272],{"class":227},"\" ",[99,274,231],{"class":116},[99,276,277],{"class":234},"?P\u003Cstatus>",[99,279,280],{"class":116},"\\d",[99,282,283],{"class":112},"{3}",[99,285,244],{"class":116},[99,287,288],{"class":116}," .",[99,290,269],{"class":112},[99,292,247],{"class":116},[99,294,295],{"class":234},"?P\u003Cms>",[99,297,280],{"class":116},[99,299,241],{"class":112},[99,301,244],{"class":116},[99,303,304],{"class":227},"ms",[99,306,307],{"class":116},"$",[99,309,224],{"class":223},[99,311,312],{"class":123},")\n",[99,314,316],{"class":101,"line":315},12,[99,317,131],{"emptyLinePlaceholder":130},[99,319,321],{"class":101,"line":320},13,[99,322,131],{"emptyLinePlaceholder":130},[99,324,326],{"class":101,"line":325},14,[99,327,329],{"class":328},"sScJk","@dataclass\n",[99,331,333,336,339],{"class":101,"line":332},15,[99,334,335],{"class":112},"class",[99,337,338],{"class":328}," Stats",[99,340,341],{"class":123},":\n",[99,343,345,348,351,353],{"class":101,"line":344},16,[99,346,347],{"class":123},"    requests: ",[99,349,350],{"class":116},"int",[99,352,214],{"class":112},[99,354,355],{"class":116}," 0\n",[99,357,359,362,364,366],{"class":101,"line":358},17,[99,360,361],{"class":123},"    errors: ",[99,363,350],{"class":116},[99,365,214],{"class":112},[99,367,355],{"class":116},[99,369,371,374,377,380,383,386,390,392],{"class":101,"line":370},18,[99,372,373],{"class":123},"    slow: Counter[",[99,375,376],{"class":116},"str",[99,378,379],{"class":123},"] ",[99,381,382],{"class":112},"=",[99,384,385],{"class":123}," field(",[99,387,389],{"class":388},"s4XuR","default_factory",[99,391,382],{"class":112},[99,393,394],{"class":123},"Counter)\n",[99,396,398],{"class":101,"line":397},19,[99,399,131],{"emptyLinePlaceholder":130},[99,401,403,406,409],{"class":101,"line":402},20,[99,404,405],{"class":112},"    def",[99,407,408],{"class":328}," merge",[99,410,411],{"class":123},"(self, other: Stats) -> Stats:\n",[99,413,415,418,421,424],{"class":101,"line":414},21,[99,416,417],{"class":116},"        self",[99,419,420],{"class":123},".requests ",[99,422,423],{"class":112},"+=",[99,425,426],{"class":123}," other.requests\n",[99,428,430,432,435,437],{"class":101,"line":429},22,[99,431,417],{"class":116},[99,433,434],{"class":123},".errors ",[99,436,423],{"class":112},[99,438,439],{"class":123}," other.errors\n",[99,441,443,445],{"class":101,"line":442},23,[99,444,417],{"class":116},[99,446,447],{"class":123},".slow.update(other.slow)\n",[99,449,451,454],{"class":101,"line":450},24,[99,452,453],{"class":112},"        return",[99,455,456],{"class":116}," self\n",[99,458,460],{"class":101,"line":459},25,[99,461,131],{"emptyLinePlaceholder":130},[99,463,465],{"class":101,"line":464},26,[99,466,131],{"emptyLinePlaceholder":130},[99,468,470,473,476,479,481],{"class":101,"line":469},27,[99,471,472],{"class":112},"def",[99,474,475],{"class":328}," analyse_file",[99,477,478],{"class":123},"(path: ",[99,480,376],{"class":116},[99,482,483],{"class":123},") -> Stats:\n",[99,485,487],{"class":101,"line":486},28,[99,488,489],{"class":223},"    \"\"\"Runs in a worker process: must be importable and take\u002Freturn picklable values.\"\"\"\n",[99,491,493,496,498],{"class":101,"line":492},29,[99,494,495],{"class":123},"    stats ",[99,497,382],{"class":112},[99,499,500],{"class":123}," Stats()\n",[99,502,504,507,510,513,516,518,521,524,527,529,532,535,538],{"class":101,"line":503},30,[99,505,506],{"class":112},"    with",[99,508,509],{"class":116}," open",[99,511,512],{"class":123},"(path, ",[99,514,515],{"class":388},"encoding",[99,517,382],{"class":112},[99,519,520],{"class":223},"\"utf-8\"",[99,522,523],{"class":123},", ",[99,525,526],{"class":388},"errors",[99,528,382],{"class":112},[99,530,531],{"class":223},"\"replace\"",[99,533,534],{"class":123},") ",[99,536,537],{"class":112},"as",[99,539,540],{"class":123}," fh:\n",[99,542,544,547,550,553],{"class":101,"line":543},31,[99,545,546],{"class":112},"        for",[99,548,549],{"class":123}," line ",[99,551,552],{"class":112},"in",[99,554,540],{"class":123},[99,556,558,561,563,566],{"class":101,"line":557},32,[99,559,560],{"class":123},"            m ",[99,562,382],{"class":112},[99,564,565],{"class":116}," LINE",[99,567,568],{"class":123},".search(line)\n",[99,570,572,575,578],{"class":101,"line":571},33,[99,573,574],{"class":112},"            if",[99,576,577],{"class":112}," not",[99,579,580],{"class":123}," m:\n",[99,582,584],{"class":101,"line":583},34,[99,585,586],{"class":112},"                continue\n",[99,588,590,593,595],{"class":101,"line":589},35,[99,591,592],{"class":123},"            stats.requests ",[99,594,423],{"class":112},[99,596,597],{"class":116}," 1\n",[99,599,601,603,606,609,612,615],{"class":101,"line":600},36,[99,602,574],{"class":112},[99,604,605],{"class":123}," m[",[99,607,608],{"class":223},"\"status\"",[99,610,611],{"class":123},"].startswith(",[99,613,614],{"class":223},"\"5\"",[99,616,617],{"class":123},"):\n",[99,619,621,624,626],{"class":101,"line":620},37,[99,622,623],{"class":123},"                stats.errors ",[99,625,423],{"class":112},[99,627,597],{"class":116},[99,629,631,633,636,639,642,645,648,651],{"class":101,"line":630},38,[99,632,574],{"class":112},[99,634,635],{"class":116}," int",[99,637,638],{"class":123},"(m[",[99,640,641],{"class":223},"\"ms\"",[99,643,644],{"class":123},"]) ",[99,646,647],{"class":112},">",[99,649,650],{"class":116}," 1000",[99,652,341],{"class":123},[99,654,656,659,662,665,667],{"class":101,"line":655},39,[99,657,658],{"class":123},"                stats.slow[m[",[99,660,661],{"class":223},"\"path\"",[99,663,664],{"class":123},"]] ",[99,666,423],{"class":112},[99,668,597],{"class":116},[99,670,672,675],{"class":101,"line":671},40,[99,673,674],{"class":112},"    return",[99,676,677],{"class":123}," stats\n",[99,679,681],{"class":101,"line":680},41,[99,682,131],{"emptyLinePlaceholder":130},[99,684,686],{"class":101,"line":685},42,[99,687,131],{"emptyLinePlaceholder":130},[99,689,691,693,696,699,701],{"class":101,"line":690},43,[99,692,472],{"class":112},[99,694,695],{"class":328}," default_jobs",[99,697,698],{"class":123},"() -> ",[99,700,350],{"class":116},[99,702,341],{"class":123},[99,704,706,709],{"class":101,"line":705},44,[99,707,708],{"class":112},"    try",[99,710,341],{"class":123},[99,712,714,716,719,722,725,728],{"class":101,"line":713},45,[99,715,453],{"class":112},[99,717,718],{"class":116}," len",[99,720,721],{"class":123},"(os.sched_getaffinity(",[99,723,724],{"class":116},"0",[99,726,727],{"class":123},"))   ",[99,729,730],{"class":105},"# respects container CPU limits on Linux\n",[99,732,734,737,740],{"class":101,"line":733},46,[99,735,736],{"class":112},"    except",[99,738,739],{"class":116}," AttributeError",[99,741,341],{"class":123},[99,743,745,747,750,753],{"class":101,"line":744},47,[99,746,453],{"class":112},[99,748,749],{"class":123}," os.cpu_count() ",[99,751,752],{"class":112},"or",[99,754,597],{"class":116},[99,756,758],{"class":101,"line":757},48,[99,759,131],{"emptyLinePlaceholder":130},[99,761,763],{"class":101,"line":762},49,[99,764,131],{"emptyLinePlaceholder":130},[99,766,768,770,773,776,778],{"class":101,"line":767},50,[99,769,472],{"class":112},[99,771,772],{"class":328}," analyse",[99,774,775],{"class":123},"(paths: list[Path], jobs: ",[99,777,350],{"class":116},[99,779,483],{"class":123},[99,781,783,786,788],{"class":101,"line":782},51,[99,784,785],{"class":123},"    total ",[99,787,382],{"class":112},[99,789,500],{"class":123},[99,791,793,796,799,802,805],{"class":101,"line":792},52,[99,794,795],{"class":112},"    if",[99,797,798],{"class":123}," jobs ",[99,800,801],{"class":112},"==",[99,803,804],{"class":116}," 1",[99,806,341],{"class":123},[99,808,810,812,815,817],{"class":101,"line":809},53,[99,811,546],{"class":112},[99,813,814],{"class":123}," p ",[99,816,552],{"class":112},[99,818,819],{"class":123}," paths:\n",[99,821,823,826,828],{"class":101,"line":822},54,[99,824,825],{"class":123},"            total.merge(analyse_file(",[99,827,376],{"class":116},[99,829,830],{"class":123},"(p)))\n",[99,832,834,836],{"class":101,"line":833},55,[99,835,453],{"class":112},[99,837,838],{"class":123}," total\n",[99,840,842,845,847,850,852,855,857,860,863,866,869,871,874],{"class":101,"line":841},56,[99,843,844],{"class":123},"    chunksize ",[99,846,382],{"class":112},[99,848,849],{"class":116}," max",[99,851,231],{"class":123},[99,853,854],{"class":116},"1",[99,856,523],{"class":123},[99,858,859],{"class":116},"len",[99,861,862],{"class":123},"(paths) ",[99,864,865],{"class":112},"\u002F\u002F",[99,867,868],{"class":123}," (jobs ",[99,870,269],{"class":112},[99,872,873],{"class":116}," 4",[99,875,876],{"class":123},"))\n",[99,878,880,882,885,888,890,893,895],{"class":101,"line":879},57,[99,881,506],{"class":112},[99,883,884],{"class":123}," ProcessPoolExecutor(",[99,886,887],{"class":388},"max_workers",[99,889,382],{"class":112},[99,891,892],{"class":123},"jobs) ",[99,894,537],{"class":112},[99,896,897],{"class":123}," pool:\n",[99,899,901,904],{"class":101,"line":900},58,[99,902,903],{"class":112},"        try",[99,905,341],{"class":123},[99,907,909,912,915,917,920,923,925,927,930,933,935],{"class":101,"line":908},59,[99,910,911],{"class":112},"            for",[99,913,914],{"class":123}," stats ",[99,916,552],{"class":112},[99,918,919],{"class":123}," pool.map(analyse_file, ",[99,921,922],{"class":116},"map",[99,924,231],{"class":123},[99,926,376],{"class":116},[99,928,929],{"class":123},", paths), ",[99,931,932],{"class":388},"chunksize",[99,934,382],{"class":112},[99,936,937],{"class":123},"chunksize):\n",[99,939,941],{"class":101,"line":940},60,[99,942,943],{"class":123},"                total.merge(stats)\n",[99,945,947,950,953],{"class":101,"line":946},61,[99,948,949],{"class":112},"        except",[99,951,952],{"class":116}," KeyboardInterrupt",[99,954,341],{"class":123},[99,956,958,961,964,966,969,971,974,976,979],{"class":101,"line":957},62,[99,959,960],{"class":123},"            pool.shutdown(",[99,962,963],{"class":388},"wait",[99,965,382],{"class":112},[99,967,968],{"class":116},"False",[99,970,523],{"class":123},[99,972,973],{"class":388},"cancel_futures",[99,975,382],{"class":112},[99,977,978],{"class":116},"True",[99,980,312],{"class":123},[99,982,984],{"class":101,"line":983},63,[99,985,986],{"class":112},"            raise\n",[99,988,990,992],{"class":101,"line":989},64,[99,991,674],{"class":112},[99,993,838],{"class":123},[90,995,997],{"className":92,"code":996,"language":94,"meta":95,"style":95},"# src\u002Fmytool\u002Fcli.py\nfrom pathlib import Path\n\nimport typer\n\nfrom mytool.stats import analyse, default_jobs\n\napp = typer.Typer()\n\n\n@app.callback()\ndef main() -> None:\n    \"\"\"Log analysis.\"\"\"\n\n\n@app.command()\ndef stats(\n    directory: Path = typer.Argument(..., exists=True, file_okay=False),\n    jobs: int = typer.Option(0, \"--jobs\", \"-j\", min=0, help=\"Worker processes (0 = one per core).\"),\n) -> None:\n    \"\"\"Summarise every *.log file under DIRECTORY.\"\"\"\n    paths = sorted(directory.rglob(\"*.log\"))\n    try:\n        total = analyse(paths, jobs or default_jobs())\n    except KeyboardInterrupt:\n        typer.echo(\"interrupted\", err=True)\n        raise typer.Exit(130)\n    typer.echo(f\"{len(paths)} files, {total.requests} requests, {total.errors} server errors\")\n    for path, count in total.slow.most_common(5):\n        typer.echo(f\"  {count:>6} slow  {path}\")\n\n\nif __name__ == \"__main__\":\n    app()\n",[19,998,999,1004,1014,1018,1025,1029,1041,1045,1055,1059,1063,1071,1085,1090,1094,1098,1105,1115,1149,1194,1203,1208,1226,1232,1247,1255,1274,1287,1332,1350,1383,1387,1391,1407],{"__ignoreMap":95},[99,1000,1001],{"class":101,"line":102},[99,1002,1003],{"class":105},"# src\u002Fmytool\u002Fcli.py\n",[99,1005,1006,1008,1010,1012],{"class":101,"line":109},[99,1007,113],{"class":112},[99,1009,195],{"class":123},[99,1011,137],{"class":112},[99,1013,200],{"class":123},[99,1015,1016],{"class":101,"line":127},[99,1017,131],{"emptyLinePlaceholder":130},[99,1019,1020,1022],{"class":101,"line":134},[99,1021,137],{"class":112},[99,1023,1024],{"class":123}," typer\n",[99,1026,1027],{"class":101,"line":143},[99,1028,131],{"emptyLinePlaceholder":130},[99,1030,1031,1033,1036,1038],{"class":101,"line":151},[99,1032,113],{"class":112},[99,1034,1035],{"class":123}," mytool.stats ",[99,1037,137],{"class":112},[99,1039,1040],{"class":123}," analyse, default_jobs\n",[99,1042,1043],{"class":101,"line":164},[99,1044,131],{"emptyLinePlaceholder":130},[99,1046,1047,1050,1052],{"class":101,"line":177},[99,1048,1049],{"class":123},"app ",[99,1051,382],{"class":112},[99,1053,1054],{"class":123}," typer.Typer()\n",[99,1056,1057],{"class":101,"line":190},[99,1058,131],{"emptyLinePlaceholder":130},[99,1060,1061],{"class":101,"line":203},[99,1062,131],{"emptyLinePlaceholder":130},[99,1064,1065,1068],{"class":101,"line":208},[99,1066,1067],{"class":328},"@app.callback",[99,1069,1070],{"class":123},"()\n",[99,1072,1073,1075,1078,1080,1083],{"class":101,"line":315},[99,1074,472],{"class":112},[99,1076,1077],{"class":328}," main",[99,1079,698],{"class":123},[99,1081,1082],{"class":116},"None",[99,1084,341],{"class":123},[99,1086,1087],{"class":101,"line":320},[99,1088,1089],{"class":223},"    \"\"\"Log analysis.\"\"\"\n",[99,1091,1092],{"class":101,"line":325},[99,1093,131],{"emptyLinePlaceholder":130},[99,1095,1096],{"class":101,"line":332},[99,1097,131],{"emptyLinePlaceholder":130},[99,1099,1100,1103],{"class":101,"line":344},[99,1101,1102],{"class":328},"@app.command",[99,1104,1070],{"class":123},[99,1106,1107,1109,1112],{"class":101,"line":358},[99,1108,472],{"class":112},[99,1110,1111],{"class":328}," stats",[99,1113,1114],{"class":123},"(\n",[99,1116,1117,1120,1122,1125,1128,1130,1133,1135,1137,1139,1142,1144,1146],{"class":101,"line":370},[99,1118,1119],{"class":123},"    directory: Path ",[99,1121,382],{"class":112},[99,1123,1124],{"class":123}," typer.Argument(",[99,1126,1127],{"class":116},"...",[99,1129,523],{"class":123},[99,1131,1132],{"class":388},"exists",[99,1134,382],{"class":112},[99,1136,978],{"class":116},[99,1138,523],{"class":123},[99,1140,1141],{"class":388},"file_okay",[99,1143,382],{"class":112},[99,1145,968],{"class":116},[99,1147,1148],{"class":123},"),\n",[99,1150,1151,1154,1156,1158,1161,1163,1165,1168,1170,1173,1175,1178,1180,1182,1184,1187,1189,1192],{"class":101,"line":397},[99,1152,1153],{"class":123},"    jobs: ",[99,1155,350],{"class":116},[99,1157,214],{"class":112},[99,1159,1160],{"class":123}," typer.Option(",[99,1162,724],{"class":116},[99,1164,523],{"class":123},[99,1166,1167],{"class":223},"\"--jobs\"",[99,1169,523],{"class":123},[99,1171,1172],{"class":223},"\"-j\"",[99,1174,523],{"class":123},[99,1176,1177],{"class":388},"min",[99,1179,382],{"class":112},[99,1181,724],{"class":116},[99,1183,523],{"class":123},[99,1185,1186],{"class":388},"help",[99,1188,382],{"class":112},[99,1190,1191],{"class":223},"\"Worker processes (0 = one per core).\"",[99,1193,1148],{"class":123},[99,1195,1196,1199,1201],{"class":101,"line":402},[99,1197,1198],{"class":123},") -> ",[99,1200,1082],{"class":116},[99,1202,341],{"class":123},[99,1204,1205],{"class":101,"line":414},[99,1206,1207],{"class":223},"    \"\"\"Summarise every *.log file under DIRECTORY.\"\"\"\n",[99,1209,1210,1213,1215,1218,1221,1224],{"class":101,"line":429},[99,1211,1212],{"class":123},"    paths ",[99,1214,382],{"class":112},[99,1216,1217],{"class":116}," sorted",[99,1219,1220],{"class":123},"(directory.rglob(",[99,1222,1223],{"class":223},"\"*.log\"",[99,1225,876],{"class":123},[99,1227,1228,1230],{"class":101,"line":442},[99,1229,708],{"class":112},[99,1231,341],{"class":123},[99,1233,1234,1237,1239,1242,1244],{"class":101,"line":450},[99,1235,1236],{"class":123},"        total ",[99,1238,382],{"class":112},[99,1240,1241],{"class":123}," analyse(paths, jobs ",[99,1243,752],{"class":112},[99,1245,1246],{"class":123}," default_jobs())\n",[99,1248,1249,1251,1253],{"class":101,"line":459},[99,1250,736],{"class":112},[99,1252,952],{"class":116},[99,1254,341],{"class":123},[99,1256,1257,1260,1263,1265,1268,1270,1272],{"class":101,"line":464},[99,1258,1259],{"class":123},"        typer.echo(",[99,1261,1262],{"class":223},"\"interrupted\"",[99,1264,523],{"class":123},[99,1266,1267],{"class":388},"err",[99,1269,382],{"class":112},[99,1271,978],{"class":116},[99,1273,312],{"class":123},[99,1275,1276,1279,1282,1285],{"class":101,"line":469},[99,1277,1278],{"class":112},"        raise",[99,1280,1281],{"class":123}," typer.Exit(",[99,1283,1284],{"class":116},"130",[99,1286,312],{"class":123},[99,1288,1289,1292,1295,1297,1300,1303,1306,1309,1312,1315,1317,1320,1322,1325,1327,1330],{"class":101,"line":486},[99,1290,1291],{"class":123},"    typer.echo(",[99,1293,1294],{"class":112},"f",[99,1296,228],{"class":223},[99,1298,1299],{"class":116},"{len",[99,1301,1302],{"class":123},"(paths)",[99,1304,1305],{"class":116},"}",[99,1307,1308],{"class":223}," files, ",[99,1310,1311],{"class":116},"{",[99,1313,1314],{"class":123},"total.requests",[99,1316,1305],{"class":116},[99,1318,1319],{"class":223}," requests, ",[99,1321,1311],{"class":116},[99,1323,1324],{"class":123},"total.errors",[99,1326,1305],{"class":116},[99,1328,1329],{"class":223}," server errors\"",[99,1331,312],{"class":123},[99,1333,1334,1337,1340,1342,1345,1348],{"class":101,"line":492},[99,1335,1336],{"class":112},"    for",[99,1338,1339],{"class":123}," path, count ",[99,1341,552],{"class":112},[99,1343,1344],{"class":123}," total.slow.most_common(",[99,1346,1347],{"class":116},"5",[99,1349,617],{"class":123},[99,1351,1352,1354,1356,1359,1361,1364,1367,1369,1372,1374,1377,1379,1381],{"class":101,"line":503},[99,1353,1259],{"class":123},[99,1355,1294],{"class":112},[99,1357,1358],{"class":223},"\"  ",[99,1360,1311],{"class":116},[99,1362,1363],{"class":123},"count",[99,1365,1366],{"class":112},":>6",[99,1368,1305],{"class":116},[99,1370,1371],{"class":223}," slow  ",[99,1373,1311],{"class":116},[99,1375,1376],{"class":123},"path",[99,1378,1305],{"class":116},[99,1380,228],{"class":223},[99,1382,312],{"class":123},[99,1384,1385],{"class":101,"line":543},[99,1386,131],{"emptyLinePlaceholder":130},[99,1388,1389],{"class":101,"line":557},[99,1390,131],{"emptyLinePlaceholder":130},[99,1392,1393,1396,1399,1402,1405],{"class":101,"line":571},[99,1394,1395],{"class":112},"if",[99,1397,1398],{"class":116}," __name__",[99,1400,1401],{"class":112}," ==",[99,1403,1404],{"class":223}," \"__main__\"",[99,1406,341],{"class":123},[99,1408,1409],{"class":101,"line":583},[99,1410,1411],{"class":123},"    app()\n",[1413,1414,1416],"h3",{"id":1415},"the-rules-that-make-it-portable","The rules that make it portable",[10,1418,1419,1422,1423,1426,1427,1430],{},[45,1420,1421],{},"Worker functions live at module level."," The pool sends work to other processes by pickling a ",[14,1424,1425],{},"reference"," to the function — its module and name — plus the arguments. Lambdas, nested functions and methods of unpicklable objects cannot be sent. ",[19,1428,1429],{},"analyse_file"," is a top-level function in an importable module, so any worker can find it.",[64,1432],{"name":1433},"cc-pickle-boundary",[10,1435,1436,1439],{},[45,1437,1438],{},"Arguments and results are small and simple."," Everything crosses the boundary by pickling, so send a path, not the file's contents, and return a compact summary, not every parsed row. Here each worker reads its own file and returns a few integers and a counter. If you find yourself pickling megabytes per task, the copying can cost more than the computation saves.",[10,1441,1442,1445,1446,1449,1450,1453,1454,1457,1458,1461],{},[45,1443,1444],{},"Chunk the work."," With thousands of small items, one round trip per item is dominated by overhead. ",[19,1447,1448],{},"pool.map(..., chunksize=n)"," sends items in batches; roughly ",[19,1451,1452],{},"len(items) \u002F (jobs * 4)"," keeps workers busy while still balancing load. With ",[19,1455,1456],{},"submit","\u002F",[19,1459,1460],{},"as_completed",", batch items yourself.",[10,1463,1464,1467,1468,1471,1472,1475,1476,1479,1480,1483],{},[45,1465,1466],{},"Guard the entry point."," On macOS and Windows, and on Linux from Python 3.14, workers are started with ",[19,1469,1470],{},"spawn"," or ",[19,1473,1474],{},"forkserver",": a fresh interpreter that ",[14,1477,1478],{},"imports"," your main module. If that module runs the CLI at import time, every worker would start the CLI again. The ",[19,1481,1482],{},"if __name__ == \"__main__\":"," guard prevents it, and console-script entry points generated by pip or uv already call your function from a guarded wrapper, so an installed CLI is safe by construction.",[64,1485],{"name":1486},"cc-start-methods",[10,1488,1489,1496],{},[45,1490,1491,1492,1495],{},"Keep ",[19,1493,1494],{},"--jobs 1"," sequential."," Running in-process for one job makes debugging normal — breakpoints work, tracebacks are local — and avoids process start-up for tiny inputs.",[30,1498,1500],{"id":1499},"ux-considerations","UX considerations",[35,1502,1503,1517,1523,1536,1546],{},[38,1504,1505,1508,1509,1512,1513,1516],{},[45,1506,1507],{},"Default to the cores the process may actually use."," ",[19,1510,1511],{},"os.sched_getaffinity(0)"," respects CPU pinning and container limits on Linux, where ",[19,1514,1515],{},"os.cpu_count()"," would report the host's 64 cores to a job limited to 2.",[38,1518,1519,1522],{},[45,1520,1521],{},"Expect a start-up cost."," Spawning workers takes tens to hundreds of milliseconds, since each re-imports your package. For a handful of small files, sequential is faster; consider switching automatically below a size threshold.",[38,1524,1525,1528,1529,1531,1532,1535],{},[45,1526,1527],{},"Show progress by completed chunks."," With ",[19,1530,1460],{}," over batches, a Rich progress bar advanced per batch gives honest progress; ",[19,1533,1534],{},"pool.map"," yields in order, so a slow first file stalls the display.",[38,1537,1538,1541,1542,1545],{},[45,1539,1540],{},"Memory multiplies."," Each worker is a full interpreter with your imports loaded. Eight workers each holding a 500 MB dataset is 4 GB; let ",[19,1543,1544],{},"--jobs"," bring it down on smaller machines, and say so in the help text.",[38,1547,1548,1551],{},[45,1549,1550],{},"Errors in workers arrive re-raised in the parent",", with the original traceback attached as a cause. Catch the expected ones inside the worker and return them as data so one malformed file does not abort the run.",[30,1553,1555],{"id":1554},"testing-the-behaviour","Testing the behaviour",[10,1557,1558,1559,1562],{},"The key tests prove that parallel and sequential runs produce identical results, and that the worker function behaves on its own. Running with ",[19,1560,1561],{},"jobs=2"," in tests exercises the real pickling path; keep the inputs small so the suite stays fast:",[90,1564,1566],{"className":92,"code":1565,"language":94,"meta":95,"style":95},"# tests\u002Ftest_stats.py\nfrom pathlib import Path\n\nfrom mytool.stats import analyse, analyse_file\n\nSAMPLE = [\n    '1.2.3.4 - - [18\u002FSep\u002F2026:10:00:00] \"GET \u002Fapi\u002Fusers HTTP\u002F1.1\" 200 512 12ms',\n    '1.2.3.4 - - [18\u002FSep\u002F2026:10:00:01] \"GET \u002Fapi\u002Freport HTTP\u002F1.1\" 500 80 2400ms',\n    '1.2.3.4 - - [18\u002FSep\u002F2026:10:00:02] \"POST \u002Fapi\u002Freport HTTP\u002F1.1\" 200 64 1800ms',\n    \"garbage line\",\n]\n\n\ndef write_logs(tmp_path: Path, n: int) -> list[Path]:\n    paths = []\n    for i in range(n):\n        p = tmp_path \u002F f\"app-{i}.log\"\n        p.write_text(\"\\n\".join(SAMPLE) + \"\\n\", encoding=\"utf-8\")\n        paths.append(p)\n    return paths\n\n\ndef test_worker_function(tmp_path):\n    [path] = write_logs(tmp_path, 1)\n    s = analyse_file(str(path))\n    assert (s.requests, s.errors, s.slow[\"\u002Fapi\u002Freport\"]) == (3, 1, 2)\n\n\ndef test_parallel_matches_sequential(tmp_path):\n    paths = write_logs(tmp_path, 12)\n    seq = analyse(paths, jobs=1)\n    par = analyse(paths, jobs=2)\n    assert (par.requests, par.errors, par.slow) == (seq.requests, seq.errors, seq.slow)\n    assert par.requests == 36\n",[19,1567,1568,1573,1583,1587,1598,1602,1612,1620,1627,1634,1641,1646,1650,1654,1669,1678,1693,1721,1759,1764,1771,1775,1779,1789,1803,1818,1849,1853,1857,1866,1879,1898,1915,1927],{"__ignoreMap":95},[99,1569,1570],{"class":101,"line":102},[99,1571,1572],{"class":105},"# tests\u002Ftest_stats.py\n",[99,1574,1575,1577,1579,1581],{"class":101,"line":109},[99,1576,113],{"class":112},[99,1578,195],{"class":123},[99,1580,137],{"class":112},[99,1582,200],{"class":123},[99,1584,1585],{"class":101,"line":127},[99,1586,131],{"emptyLinePlaceholder":130},[99,1588,1589,1591,1593,1595],{"class":101,"line":134},[99,1590,113],{"class":112},[99,1592,1035],{"class":123},[99,1594,137],{"class":112},[99,1596,1597],{"class":123}," analyse, analyse_file\n",[99,1599,1600],{"class":101,"line":143},[99,1601,131],{"emptyLinePlaceholder":130},[99,1603,1604,1607,1609],{"class":101,"line":151},[99,1605,1606],{"class":116},"SAMPLE",[99,1608,214],{"class":112},[99,1610,1611],{"class":123}," [\n",[99,1613,1614,1617],{"class":101,"line":164},[99,1615,1616],{"class":223},"    '1.2.3.4 - - [18\u002FSep\u002F2026:10:00:00] \"GET \u002Fapi\u002Fusers HTTP\u002F1.1\" 200 512 12ms'",[99,1618,1619],{"class":123},",\n",[99,1621,1622,1625],{"class":101,"line":177},[99,1623,1624],{"class":223},"    '1.2.3.4 - - [18\u002FSep\u002F2026:10:00:01] \"GET \u002Fapi\u002Freport HTTP\u002F1.1\" 500 80 2400ms'",[99,1626,1619],{"class":123},[99,1628,1629,1632],{"class":101,"line":190},[99,1630,1631],{"class":223},"    '1.2.3.4 - - [18\u002FSep\u002F2026:10:00:02] \"POST \u002Fapi\u002Freport HTTP\u002F1.1\" 200 64 1800ms'",[99,1633,1619],{"class":123},[99,1635,1636,1639],{"class":101,"line":203},[99,1637,1638],{"class":223},"    \"garbage line\"",[99,1640,1619],{"class":123},[99,1642,1643],{"class":101,"line":208},[99,1644,1645],{"class":123},"]\n",[99,1647,1648],{"class":101,"line":315},[99,1649,131],{"emptyLinePlaceholder":130},[99,1651,1652],{"class":101,"line":320},[99,1653,131],{"emptyLinePlaceholder":130},[99,1655,1656,1658,1661,1664,1666],{"class":101,"line":325},[99,1657,472],{"class":112},[99,1659,1660],{"class":328}," write_logs",[99,1662,1663],{"class":123},"(tmp_path: Path, n: ",[99,1665,350],{"class":116},[99,1667,1668],{"class":123},") -> list[Path]:\n",[99,1670,1671,1673,1675],{"class":101,"line":332},[99,1672,1212],{"class":123},[99,1674,382],{"class":112},[99,1676,1677],{"class":123}," []\n",[99,1679,1680,1682,1685,1687,1690],{"class":101,"line":344},[99,1681,1336],{"class":112},[99,1683,1684],{"class":123}," i ",[99,1686,552],{"class":112},[99,1688,1689],{"class":116}," range",[99,1691,1692],{"class":123},"(n):\n",[99,1694,1695,1698,1700,1703,1705,1708,1711,1713,1716,1718],{"class":101,"line":358},[99,1696,1697],{"class":123},"        p ",[99,1699,382],{"class":112},[99,1701,1702],{"class":123}," tmp_path ",[99,1704,1457],{"class":112},[99,1706,1707],{"class":112}," f",[99,1709,1710],{"class":223},"\"app-",[99,1712,1311],{"class":116},[99,1714,1715],{"class":123},"i",[99,1717,1305],{"class":116},[99,1719,1720],{"class":223},".log\"\n",[99,1722,1723,1726,1728,1731,1733,1736,1738,1740,1742,1745,1747,1749,1751,1753,1755,1757],{"class":101,"line":370},[99,1724,1725],{"class":123},"        p.write_text(",[99,1727,228],{"class":223},[99,1729,1730],{"class":116},"\\n",[99,1732,228],{"class":223},[99,1734,1735],{"class":123},".join(",[99,1737,1606],{"class":116},[99,1739,534],{"class":123},[99,1741,241],{"class":112},[99,1743,1744],{"class":223}," \"",[99,1746,1730],{"class":116},[99,1748,228],{"class":223},[99,1750,523],{"class":123},[99,1752,515],{"class":388},[99,1754,382],{"class":112},[99,1756,520],{"class":223},[99,1758,312],{"class":123},[99,1760,1761],{"class":101,"line":397},[99,1762,1763],{"class":123},"        paths.append(p)\n",[99,1765,1766,1768],{"class":101,"line":402},[99,1767,674],{"class":112},[99,1769,1770],{"class":123}," paths\n",[99,1772,1773],{"class":101,"line":414},[99,1774,131],{"emptyLinePlaceholder":130},[99,1776,1777],{"class":101,"line":429},[99,1778,131],{"emptyLinePlaceholder":130},[99,1780,1781,1783,1786],{"class":101,"line":442},[99,1782,472],{"class":112},[99,1784,1785],{"class":328}," test_worker_function",[99,1787,1788],{"class":123},"(tmp_path):\n",[99,1790,1791,1794,1796,1799,1801],{"class":101,"line":450},[99,1792,1793],{"class":123},"    [path] ",[99,1795,382],{"class":112},[99,1797,1798],{"class":123}," write_logs(tmp_path, ",[99,1800,854],{"class":116},[99,1802,312],{"class":123},[99,1804,1805,1808,1810,1813,1815],{"class":101,"line":459},[99,1806,1807],{"class":123},"    s ",[99,1809,382],{"class":112},[99,1811,1812],{"class":123}," analyse_file(",[99,1814,376],{"class":116},[99,1816,1817],{"class":123},"(path))\n",[99,1819,1820,1823,1826,1829,1831,1833,1835,1838,1840,1842,1844,1847],{"class":101,"line":464},[99,1821,1822],{"class":112},"    assert",[99,1824,1825],{"class":123}," (s.requests, s.errors, s.slow[",[99,1827,1828],{"class":223},"\"\u002Fapi\u002Freport\"",[99,1830,644],{"class":123},[99,1832,801],{"class":112},[99,1834,247],{"class":123},[99,1836,1837],{"class":116},"3",[99,1839,523],{"class":123},[99,1841,854],{"class":116},[99,1843,523],{"class":123},[99,1845,1846],{"class":116},"2",[99,1848,312],{"class":123},[99,1850,1851],{"class":101,"line":469},[99,1852,131],{"emptyLinePlaceholder":130},[99,1854,1855],{"class":101,"line":486},[99,1856,131],{"emptyLinePlaceholder":130},[99,1858,1859,1861,1864],{"class":101,"line":492},[99,1860,472],{"class":112},[99,1862,1863],{"class":328}," test_parallel_matches_sequential",[99,1865,1788],{"class":123},[99,1867,1868,1870,1872,1874,1877],{"class":101,"line":503},[99,1869,1212],{"class":123},[99,1871,382],{"class":112},[99,1873,1798],{"class":123},[99,1875,1876],{"class":116},"12",[99,1878,312],{"class":123},[99,1880,1881,1884,1886,1889,1892,1894,1896],{"class":101,"line":543},[99,1882,1883],{"class":123},"    seq ",[99,1885,382],{"class":112},[99,1887,1888],{"class":123}," analyse(paths, ",[99,1890,1891],{"class":388},"jobs",[99,1893,382],{"class":112},[99,1895,854],{"class":116},[99,1897,312],{"class":123},[99,1899,1900,1903,1905,1907,1909,1911,1913],{"class":101,"line":557},[99,1901,1902],{"class":123},"    par ",[99,1904,382],{"class":112},[99,1906,1888],{"class":123},[99,1908,1891],{"class":388},[99,1910,382],{"class":112},[99,1912,1846],{"class":116},[99,1914,312],{"class":123},[99,1916,1917,1919,1922,1924],{"class":101,"line":571},[99,1918,1822],{"class":112},[99,1920,1921],{"class":123}," (par.requests, par.errors, par.slow) ",[99,1923,801],{"class":112},[99,1925,1926],{"class":123}," (seq.requests, seq.errors, seq.slow)\n",[99,1928,1929,1931,1934,1936],{"class":101,"line":583},[99,1930,1822],{"class":112},[99,1932,1933],{"class":123}," par.requests ",[99,1935,801],{"class":112},[99,1937,1938],{"class":116}," 36\n",[10,1940,1941,1942,1944],{},"Tests that spawn processes are slower than ordinary unit tests — each pool start re-imports your package — so keep one or two of them and test the logic through the worker function directly. If you use pytest-xdist, process pools inside tests work fine, but avoid combining huge ",[19,1943,1544],{}," values with a parallel test runner on small CI machines.",[30,1946,1948],{"id":1947},"conclusion","Conclusion",[10,1950,1951,1952,1954,1955,1959],{},"When a CLI is CPU-bound in pure Python, processes are the portable way to use every core. Keep worker functions at module level, send small arguments and return small summaries, chunk many small tasks, guard the entry point, and keep a sequential path for ",[19,1953,1494],{},". Default to the CPUs the process is allowed to use, watch memory, and prove with a test that parallel and sequential runs agree. For work that mostly waits rather than computes, a ",[24,1956,1958],{"href":1957},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fparallelising-cli-work-with-thread-pools\u002F","thread pool"," is simpler and just as fast.",[30,1961,1963],{"id":1962},"frequently-asked-questions","Frequently asked questions",[1413,1965,1967,1968,1471,1971,1973],{"id":1966},"should-i-use-multiprocessingpool-or-processpoolexecutor","Should I use ",[19,1969,1970],{},"multiprocessing.Pool",[19,1972,21],{},"?",[10,1975,1976,1978,1979,1982,1983,1986,1987,1989,1990,1993],{},[19,1977,21],{}," for new code: it shares the futures API with ",[19,1980,1981],{},"ThreadPoolExecutor",", so switching between threads and processes is a one-word change, and it handles worker crashes by raising ",[19,1984,1985],{},"BrokenProcessPool"," instead of hanging. ",[19,1988,1970],{}," offers a few extras such as ",[19,1991,1992],{},"imap_unordered"," with chunking.",[1413,1995,1997],{"id":1996},"why-does-my-cli-print-its-banner-once-per-worker","Why does my CLI print its banner once per worker?",[10,1999,2000],{},"Something runs at import time of your main module — a print, a config load with output, or an unguarded call to the app. Workers started with spawn import that module. Move side effects into functions called from the guarded entry point.",[1413,2002,2004],{"id":2003},"can-workers-share-a-large-read-only-dataset","Can workers share a large read-only dataset?",[10,2006,2007,2008,2011,2012,2015,2016,2019],{},"Under ",[19,2009,2010],{},"fork",", children inherit the parent's memory copy-on-write, but that start method is no longer the default on most platforms. Portable options are to have each worker load the data in an ",[19,2013,2014],{},"initializer",", or to put it in ",[19,2017,2018],{},"multiprocessing.shared_memory"," for large numeric arrays.",[1413,2021,2023],{"id":2022},"is-it-worth-trying-the-free-threaded-build","Is it worth trying the free-threaded build?",[10,2025,2026],{},"For internal tools where you control the Python version, experimenting with a 3.13t or 3.14t build can let a plain thread pool use every core with no pickling. Check that your dependencies support it first; C extensions built without free-threading support re-enable the GIL.",[30,2028,2030],{"id":2029},"related","Related",[35,2032,2033,2039,2044,2050,2056],{},[38,2034,2035,2036],{},"Up: ",[24,2037,2038],{"href":26},"Concurrency and async in Python CLIs",[38,2040,2041],{},[24,2042,2043],{"href":1957},"Parallelising CLI work with thread pools",[38,2045,2046],{},[24,2047,2049],{"href":2048},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fcancelling-async-tasks-on-ctrl-c\u002F","Cancelling async tasks on Ctrl+C",[38,2051,2052],{},[24,2053,2055],{"href":2054},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fprofiling-python-cli-startup-time\u002F","Profiling Python CLI startup time",[38,2057,2058],{},[24,2059,2061],{"href":2060},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fprocessing-large-files-and-ndjson-streams\u002F","Processing large files and NDJSON streams",[2063,2064,2065],"style",{},"html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sA_wV, html code.shiki .sA_wV{--shiki-default:#032F62;--shiki-dark:#DBEDFF}html pre.shiki code .s9eBZ, html code.shiki .s9eBZ{--shiki-default:#22863A;--shiki-dark:#85E89D}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .s4XuR, html code.shiki .s4XuR{--shiki-default:#E36209;--shiki-dark:#FFAB70}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":95,"searchDepth":109,"depth":109,"links":2067},[2068,2069,2070,2073,2074,2075,2076,2083],{"id":32,"depth":109,"text":33},{"id":58,"depth":109,"text":59},{"id":84,"depth":109,"text":85,"children":2071},[2072],{"id":1415,"depth":127,"text":1416},{"id":1499,"depth":109,"text":1500},{"id":1554,"depth":109,"text":1555},{"id":1947,"depth":109,"text":1948},{"id":1962,"depth":109,"text":1963,"children":2077},[2078,2080,2081,2082],{"id":1966,"depth":127,"text":2079},"Should I use multiprocessing.Pool or ProcessPoolExecutor?",{"id":1996,"depth":127,"text":1997},{"id":2003,"depth":127,"text":2004},{"id":2022,"depth":127,"text":2023},{"id":2029,"depth":109,"text":2030},"2026-09-18","Use every core from a Python CLI with ProcessPoolExecutor: when threads will not help, picklable workers, chunking, start methods, Ctrl+C and testing.","advanced",false,"md",{},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fmultiprocessing-for-cpu-bound-cli-tasks",{"title":5,"description":2085},"cli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fmultiprocessing-for-cpu-bound-cli-tasks\u002Findex",[2094,2095,2096,2097],"multiprocessing","performance","concurrency","gil","1W55U4YkwQDTrlBSnaV16DmY5WevOgKNMW2jYHenvtE",[2100,2103,2106,2109,2112,2115,2118,2121,2124,2127,2130,2133,2136,2139,2142,2145,2148,2151,2154,2157,2160,2163,2166,2169,2172,2175,2178,2181,2184,2187,2190,2193,2196,2199,2202,2205,2208,2211,2214,2217,2220,2223,2226,2229,2232,2235,2238,2241,2244,2247,2250,2253,2256,2259,2262,2265,2268,2271,2274,2277,2280,2283,2286,2289,2290,2293,2296,2299,2302,2305,2308,2311,2314,2317,2320,2323,2326,2329,2332,2335,2338,2341,2344,2347,2350,2353,2356,2359,2362,2365,2368,2371,2373,2376,2379,2382,2385,2388,2391,2394,2397,2400,2403,2406,2409,2412,2415,2418,2421,2424,2427,2430,2433,2436,2439,2442,2445,2448,2451,2454,2457,2460,2463,2466,2469,2472,2475,2478,2481,2484,2487,2490,2493,2496,2499,2502,2505,2508,2511,2514,2517,2520,2523,2526,2529,2532,2535,2538,2541,2544,2547,2550,2553,2556,2559,2562,2565,2568,2571,2574,2577,2580,2583,2586,2589,2592,2595,2598,2601,2604,2607,2610,2613,2616,2619,2622,2625,2628,2631,2634,2637,2640,2643],{"path":2101,"title":2102},"\u002Fabout","About Python CLI Toolcraft",{"path":2104,"title":2105},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies","Advanced Argument Validation Strategies",{"path":2107,"title":2108},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fparsing-nested-json-arguments-in-python-clis","Parsing Nested JSON Args in Python CLIs",{"path":2110,"title":2111},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-dependent-and-conflicting-options","Validating Dependent and Conflicting CLI Options",{"path":2113,"title":2114},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-file-and-directory-paths-in-clis","Validating File and Directory Paths in CLIs",{"path":2116,"title":2117},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fwriting-custom-click-parameter-types","Writing Custom Click Parameter Types",{"path":2119,"title":2120},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fbuilding-your-first-textual-app","Building Your First Textual App for a Python CLI",{"path":2122,"title":2123},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fchoosing-between-a-cli-a-prompt-flow-and-a-tui","Choosing Between a CLI, a Prompt Flow and a TUI",{"path":2125,"title":2126},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual","Building Terminal UIs with Textual for Python CLIs",{"path":2128,"title":2129},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Ftesting-textual-apps-with-pilot","Testing Textual Apps with Pilot",{"path":2131,"title":2132},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fadding-examples-and-epilogs-to-help-output","Adding Examples and Epilogs to Help Output",{"path":2134,"title":2135},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fgenerating-man-pages-and-docs-from-a-cli","Generating Man Pages and Docs from a CLI",{"path":2137,"title":2138},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation","CLI Help Output and Documentation",{"path":2140,"title":2141},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fversioning-and-deprecating-cli-flags","Versioning and Deprecating CLI Flags",{"path":2143,"title":2144},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fwriting-help-text-users-actually-read","Writing Help Text Users Actually Read",{"path":2146,"title":2147},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fadapting-output-to-terminal-width","Adapting Python CLI Output to Terminal Width",{"path":2149,"title":2150},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fdetecting-ci-environments-and-non-interactive-shells","Detecting CI Environments and Non-Interactive Shells",{"path":2152,"title":2153},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Ffixing-unicode-and-encoding-errors-on-windows","Fixing Unicode and Encoding Errors on Windows in Python CLIs",{"path":2155,"title":2156},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility","Cross-Platform Terminal Compatibility for Python CLIs",{"path":2158,"title":2159},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Frespecting-no-color-and-force-color","Respecting NO_COLOR and FORCE_COLOR in Python CLIs",{"path":2161,"title":2162},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fchoosing-exit-codes-for-cli-tools","Choosing Exit Codes for CLI Tools",{"path":2164,"title":2165},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fdesigning-an-exception-hierarchy-for-a-cli","Designing an Exception Hierarchy for a Python CLI",{"path":2167,"title":2168},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Ffriendly-error-messages-and-tracebacks","Friendly Error Messages and Tracebacks",{"path":2170,"title":2171},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fhandling-keyboard-interrupt-cleanly","Handling Keyboard Interrupt Cleanly",{"path":2173,"title":2174},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes","Error Handling and Exit Codes for CLIs",{"path":2176,"title":2177},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Freporting-machine-readable-errors-in-json-mode","Reporting Machine-Readable Errors in JSON Mode",{"path":2179,"title":2180},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fconfig-precedence-flags-env-files-defaults","Config Precedence: Flags, Env, Files, Defaults",{"path":2182,"title":2183},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fdiscovering-project-config-files-by-walking-up-directories","Discovering Project Config Files by Walking Up Directories",{"path":2185,"title":2186},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars","Handling Config Files and Env Vars in CLIs",{"path":2188,"title":2189},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Floading-yaml-configs-safely-in-cli-apps","Loading YAML configs safely in CLI apps",{"path":2191,"title":2192},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Freading-toml-config-with-tomllib","Reading TOML Config with tomllib in Python CLIs",{"path":2194,"title":2195},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Ftyped-settings-with-pydantic-settings","Typed Settings with pydantic-settings in Python CLIs",{"path":2197,"title":2198},"\u002Fadvanced-input-parsing-user-experience","Advanced Input Parsing for Python CLIs",{"path":2200,"title":2201},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fadding-progress-bars-and-spinners-to-python-clis","Progress Bars and Spinners for Python CLIs",{"path":2203,"title":2204},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fbuilding-interactive-prompts-and-menus","Building Interactive Prompts and Menus in Python CLIs",{"path":2206,"title":2207},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich","Interactive Terminal UI with Rich",{"path":2209,"title":2210},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Flive-dashboards-with-rich-live","Live Dashboards with Rich Live in Python CLIs",{"path":2212,"title":2213},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Frendering-tables-and-json-with-rich","Rendering Tables and JSON with Rich",{"path":2215,"title":2216},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Ftheming-rich-output-consistently","Theming Rich Output Consistently in Python CLIs",{"path":2218,"title":2219},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fdynamic-completion-values-from-apis-and-files","Dynamic Completion Values from APIs and Files",{"path":2221,"title":2222},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fenabling-tab-completion-in-click-and-typer","Enabling Tab Completion in Click and Typer",{"path":2224,"title":2225},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis","Shell Completion for Python CLIs",{"path":2227,"title":2228},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Finstalling-shell-completion-for-bash-zsh-fish","Installing Shell Completion for bash, zsh, fish",{"path":2230,"title":2231},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Ftesting-shell-completion-in-python-clis","Testing Shell Completion in Python CLIs",{"path":2233,"title":2234},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fadding-trace-ids-and-context-to-cli-logs","Adding Trace IDs and Context to Python CLI Logs",{"path":2236,"title":2237},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fadding-verbose-and-quiet-logging-flags","Adding Verbose and Quiet Logging Flags",{"path":2239,"title":2240},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps","Structured Logging for CLI Apps",{"path":2242,"title":2243},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fstructured-json-logging-in-python-clis","Structured JSON Logging in Python CLIs",{"path":2245,"title":2246},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fwriting-rotating-log-files-from-a-cli","Writing Rotating Log Files from a Python CLI",{"path":2248,"title":2249},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fdetecting-tty-and-adapting-output","Detecting a TTY and Adapting Output",{"path":2251,"title":2252},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Femitting-json-output-for-scripting","Emitting JSON Output for Scripting",{"path":2254,"title":2255},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fhandling-broken-pipe-and-sigpipe","Handling Broken Pipe and SIGPIPE",{"path":2257,"title":2258},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes","Working with stdin, stdout and Pipes",{"path":2260,"title":2261},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fprocessing-large-files-and-ndjson-streams","Processing Large Files and NDJSON Streams in Python CLIs",{"path":2263,"title":2264},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Freading-piped-input-in-python-clis","Reading Piped Input in Python CLIs",{"path":2266,"title":2267},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fbuilding-an-api-client-cli-with-httpx","Building an API Client CLI with httpx",{"path":2269,"title":2270},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fdownloading-files-with-progress-in-python","Downloading Files with Progress in Python CLIs",{"path":2272,"title":2273},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis","Calling HTTP APIs from Python CLIs",{"path":2275,"title":2276},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Foauth-device-flow-login-for-clis","OAuth Device Flow Login for Python CLIs",{"path":2278,"title":2279},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fpaginating-api-results-in-a-cli","Paginating API Results in a Python CLI",{"path":2281,"title":2282},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fretries-and-backoff-for-cli-http-calls","Retries and Backoff for CLI HTTP Calls",{"path":2284,"title":2285},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fcancelling-async-tasks-on-ctrl-c","Cancelling Async Tasks on Ctrl+C in Python CLIs",{"path":2287,"title":2288},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis","Concurrency and Async in Python CLIs",{"path":2090,"title":5},{"path":2291,"title":2292},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fparallelising-cli-work-with-thread-pools","Parallelising CLI Work with Thread Pools",{"path":2294,"title":2295},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Frate-limiting-concurrent-requests-in-clis","Rate-Limiting Concurrent Requests in Python CLIs",{"path":2297,"title":2298},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Frunning-async-code-in-typer-and-click","Running Async Code in Typer and Click",{"path":2300,"title":2301},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fcross-platform-paths-with-pathlib","Cross-Platform Paths with pathlib in CLIs",{"path":2303,"title":2304},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Ffile-locking-for-concurrent-cli-runs","File Locking for Concurrent CLI Runs in Python",{"path":2306,"title":2307},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes","Filesystem Paths and Atomic Writes for CLIs",{"path":2309,"title":2310},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fsafe-temporary-files-and-directories","Safe Temporary Files and Directories in CLIs",{"path":2312,"title":2313},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fstoring-app-data-with-platformdirs","Storing CLI App Data with platformdirs",{"path":2315,"title":2316},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fwriting-files-atomically-in-python-clis","Writing Files Atomically in Python CLIs",{"path":2318,"title":2319},"\u002Fcli-runtime-systems-integration","CLI Runtime & Systems Integration for Python",{"path":2321,"title":2322},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fbuilding-a-watch-mode-with-watchfiles","Building a Watch Mode with watchfiles in Python",{"path":2324,"title":2325},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fhandling-sigterm-and-graceful-shutdown","Handling SIGTERM and Graceful Shutdown in CLIs",{"path":2327,"title":2328},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fhealth-checks-and-heartbeats-for-long-running-clis","Health Checks and Heartbeats for Long-Running CLIs",{"path":2330,"title":2331},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis","Long-Running and Watch-Mode Python CLIs",{"path":2333,"title":2334},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Frunning-a-cli-on-a-schedule-with-cron-and-systemd","Running a Python CLI on a Schedule with cron and systemd",{"path":2336,"title":2337},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Favoiding-shell-injection-in-python-clis","Avoiding Shell Injection in Python CLIs",{"path":2339,"title":2340},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fcalling-external-commands-safely-with-subprocess","Calling External Commands Safely with subprocess",{"path":2342,"title":2343},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fhandling-subprocess-timeouts-and-exit-codes","Handling Subprocess Timeouts and Exit Codes",{"path":2345,"title":2346},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis","Running Subprocesses from Python CLIs",{"path":2348,"title":2349},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fstreaming-subprocess-output-in-real-time","Streaming Subprocess Output in Real Time",{"path":2351,"title":2352},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fwrapping-git-and-other-tools-from-a-python-cli","Wrapping git and Other Tools from a Python CLI",{"path":2354,"title":2355},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis","Secrets and Credentials in Python CLIs",{"path":2357,"title":2358},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fprompting-for-passwords-securely","Prompting for Passwords Securely in Python CLIs",{"path":2360,"title":2361},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Freading-secrets-from-env-and-files","Reading Secrets from Env Vars and Files in CLIs",{"path":2363,"title":2364},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fredacting-secrets-from-cli-output-and-logs","Redacting Secrets from CLI Output and Logs",{"path":2366,"title":2367},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fstoring-tokens-with-keyring","Storing CLI Tokens Securely with keyring",{"path":2369,"title":2370},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fsupporting-multiple-profiles-and-accounts","Supporting Multiple Profiles and Accounts in CLIs",{"path":1457,"title":2372},"Python CLI Toolcraft",{"path":2374,"title":2375},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fcaching-expensive-work-between-cli-runs","Caching Expensive Work Between Python CLI Runs",{"path":2377,"title":2378},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading","CLI Startup Performance and Lazy Loading",{"path":2380,"title":2381},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Flazy-loading-subcommands-for-faster-startup","Lazy Loading Subcommands for Faster Startup",{"path":2383,"title":2384},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fprofiling-python-cli-startup-time","Profiling Python CLI Startup Time",{"path":2386,"title":2387},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Freducing-cli-dependency-weight","Reducing CLI Dependency Weight",{"path":2389,"title":2390},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargparse-subparsers-for-subcommands","argparse Subparsers for Subcommands",{"path":2392,"title":2393},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargparse-vs-click-vs-typer-comparison","argparse vs Click vs Typer Compared",{"path":2395,"title":2396},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse","Command-Line Parsing with argparse",{"path":2398,"title":2399},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fmigrating-from-argparse-to-typer","Migrating from argparse to Typer",{"path":2401,"title":2402},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fmutually-exclusive-options-in-argparse","Mutually Exclusive Options in argparse",{"path":2404,"title":2405},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fwriting-custom-argparse-actions","Writing Custom argparse Actions in Python",{"path":2407,"title":2408},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fadding-dry-run-and-confirmation-to-destructive-commands","Adding Dry-Run and Confirmation to Destructive Commands",{"path":2410,"title":2411},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Ffollowing-posix-and-gnu-argument-conventions","Following POSIX and GNU Argument Conventions in Python",{"path":2413,"title":2414},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fglobal-options-vs-per-command-options","Global Options vs Per-Command Options in Python CLIs",{"path":2416,"title":2417},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions","Designing CLI Interfaces and Conventions in Python",{"path":2419,"title":2420},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fnaming-commands-and-flags-consistently","Naming Commands and Flags Consistently in Python CLIs",{"path":2422,"title":2423},"\u002Fmodern-python-cli-frameworks-architecture","Python CLI Frameworks and Architecture",{"path":2425,"title":2426},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fdiscovering-plugins-with-entry-points","Discovering Plugins with Entry Points in Python CLIs",{"path":2428,"title":2429},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fhook-based-plugins-with-pluggy","Hook-Based Plugins for Python CLIs with pluggy",{"path":2431,"title":2432},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis","Plugin Architectures for Extensible CLIs",{"path":2434,"title":2435},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fversioning-a-plugin-api","Versioning a Plugin API for a Python CLI",{"path":2437,"title":2438},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fwriting-a-plugin-for-an-existing-cli","Writing a Plugin for an Existing CLI",{"path":2440,"title":2441},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fbest-practices-for-python-cli-entry-points","Best practices for Python CLI entry points",{"path":2443,"title":2444},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fdependency-injection-patterns-for-cli-commands","Dependency Injection Patterns for CLI Commands",{"path":2446,"title":2447},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fhow-to-structure-a-large-python-cli-project","Structuring a Large Python CLI Project",{"path":2449,"title":2450},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis","Structuring Multi-Command Python CLIs",{"path":2452,"title":2453},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fsharing-common-options-across-commands","Sharing Common Options Across Python CLI Commands",{"path":2455,"title":2456},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fsharing-state-with-click-context-objects","Sharing State with Click Context Objects",{"path":2458,"title":2459},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fend-to-end-testing-an-installed-cli","End-to-End Testing an Installed Python CLI",{"path":2461,"title":2462},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications","Testing Python CLI Applications",{"path":2464,"title":2465},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fmeasuring-cli-test-coverage","Measuring CLI Test Coverage",{"path":2467,"title":2468},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fmocking-filesystem-and-network-in-cli-tests","Mocking the Filesystem and Network in CLI Tests",{"path":2470,"title":2471},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fproperty-based-testing-cli-arguments-with-hypothesis","Property-Based Testing CLI Arguments with Hypothesis",{"path":2473,"title":2474},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fsnapshot-testing-cli-output","Snapshot Testing CLI Output",{"path":2476,"title":2477},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-click-commands-with-clirunner","Testing Click Commands with CliRunner",{"path":2479,"title":2480},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-interactive-prompts-and-stdin","Testing Interactive Prompts and stdin",{"path":2482,"title":2483},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fbuilding-a-cli-with-subcommands-in-click","Building a CLI with subcommands in Click",{"path":2485,"title":2486},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fbuilding-dynamic-commands-in-click","Building Dynamic Commands in Click",{"path":2488,"title":2489},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fconverting-a-click-app-to-typer","Converting a Click App to Typer",{"path":2491,"title":2492},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each","Typer vs Click: When to Use Each",{"path":2494,"title":2495},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Ftyper-callback-functions-explained","Typer callback functions explained",{"path":2497,"title":2498},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fusing-annotated-options-in-typer","Using Annotated Options in Typer",{"path":2500,"title":2501},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fautomating-releases-from-git-tags","Automating Python CLI Releases from Git Tags",{"path":2503,"title":2504},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fcaching-uv-dependencies-in-ci","Caching uv Dependencies in CI for Python CLIs",{"path":2506,"title":2507},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis","CI\u002FCD Pipelines for Python CLIs",{"path":2509,"title":2510},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fpublishing-to-pypi-with-trusted-publishing","Publishing a CLI to PyPI with Trusted Publishing",{"path":2512,"title":2513},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fsmoke-testing-the-built-wheel-in-ci","Smoke-Testing the Built Wheel of a Python CLI in CI",{"path":2515,"title":2516},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Ftesting-a-cli-across-python-versions-with-github-actions","Testing a CLI Across Python Versions in GitHub Actions",{"path":2518,"title":2519},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fbuilding-a-cookiecutter-template-for-typer-clis","Building a Cookiecutter Template for Typer CLIs",{"path":2521,"title":2522},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fcopier-vs-cookiecutter-for-cli-templates","Copier vs Cookiecutter for CLI Templates",{"path":2524,"title":2525},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter","CLI Project Scaffolding with Cookiecutter",{"path":2527,"title":2528},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fpost-generation-hooks-in-cli-templates","Post-Generation Hooks in Python CLI Templates",{"path":2530,"title":2531},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fbuilding-cross-platform-release-binaries-in-ci","Building Cross-Platform Release Binaries in CI",{"path":2533,"title":2534},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fbundling-a-python-cli-with-pyinstaller","Bundling a Python CLI with PyInstaller",{"path":2536,"title":2537},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fhomebrew-and-scoop-packaging-for-python-clis","Homebrew and Scoop Packaging for Python CLIs",{"path":2539,"title":2540},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries","Distributing CLIs as Standalone Binaries",{"path":2542,"title":2543},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fnuitka-vs-pyinstaller-for-python-clis","Nuitka vs PyInstaller for Python CLI Binaries",{"path":2545,"title":2546},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fshipping-a-cli-as-a-zipapp-with-shiv","Shipping a CLI as a Zipapp with shiv",{"path":2548,"title":2549},"\u002Fproject-setup-dependency-management","Project Setup & Dependency Management",{"path":2551,"title":2552},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Fconfiguring-ruff-for-a-cli-project","Configuring Ruff for a Python CLI Project",{"path":2554,"title":2555},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Fenforcing-import-boundaries-in-a-cli-codebase","Enforcing Import Boundaries in a Python CLI Codebase",{"path":2557,"title":2558},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code","Linting and Type-Checking Python CLI Code",{"path":2560,"title":2561},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Ftype-checking-click-and-typer-code-with-mypy","Type-Checking Click and Typer Code with mypy",{"path":2563,"title":2564},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fautomating-changelogs-with-conventional-commits","Automating Changelogs with Conventional Commits",{"path":2566,"title":2567},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fderiving-versions-from-git-tags-with-hatch-vcs","Deriving CLI Versions from Git Tags with hatch-vcs",{"path":2569,"title":2570},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fexposing-version-info-and-build-metadata","Exposing Version Info and Build Metadata",{"path":2572,"title":2573},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs","Managing CLI Versioning & Changelogs",{"path":2575,"title":2576},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fsemantic-versioning-policy-for-cli-tools","A Semantic Versioning Policy for CLI Tools",{"path":2578,"title":2579},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fbuilding-wheels-and-sdists-for-python-clis","Building Wheels and sdists for Python CLIs",{"path":2581,"title":2582},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fbundling-data-files-with-importlib-resources","Bundling Data Files with importlib.resources in CLIs",{"path":2584,"title":2585},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution","Packaging Python CLIs for Distribution",{"path":2587,"title":2588},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Finstalling-and-distributing-clis-with-pipx","Installing and Distributing CLIs with pipx",{"path":2590,"title":2591},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fpublishing-a-python-cli-to-pypi","Publishing a Python CLI to PyPI",{"path":2593,"title":2594},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fwriting-pyproject-toml-metadata-for-a-cli","Writing pyproject.toml Metadata for a Python CLI",{"path":2596,"title":2597},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development","Poetry Workflows for CLI Development",{"path":2599,"title":2600},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fmigrating-a-cli-from-poetry-to-uv","Migrating a Python CLI from Poetry to uv",{"path":2602,"title":2603},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpoetry-dependency-groups-for-cli-tooling","Poetry Dependency Groups for CLI Tooling",{"path":2605,"title":2606},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpoetry-entry-points-and-scripts-for-clis","Poetry Entry Points and Scripts for CLIs",{"path":2608,"title":2609},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects","Pre-commit Hooks for CLI Projects",{"path":2611,"title":2612},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fsetting-up-pre-commit-for-python-cli-repos","Setting up pre-commit for Python CLI repos",{"path":2614,"title":2615},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fshipping-your-cli-as-a-pre-commit-hook","Shipping Your Python CLI as a pre-commit Hook",{"path":2617,"title":2618},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fwriting-local-pre-commit-hooks-in-python","Writing Local pre-commit Hooks in Python",{"path":2620,"title":2621},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management","uv for Python CLI Dependency Management",{"path":2623,"title":2624},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Frunning-one-off-cli-scripts-with-uv-run","Running One-Off CLI Scripts with uv run and PEP 723",{"path":2626,"title":2627},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-init-vs-poetry-init-for-cli-tools","uv init vs poetry init for CLI tools",{"path":2629,"title":2630},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-tool-install-vs-pipx-for-clis","uv tool install vs pipx for CLIs",{"path":2632,"title":2633},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-workspaces-for-multi-package-clis","uv Workspaces for Multi-Package Python CLIs",{"path":2635,"title":2636},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices","Python CLI Env Isolation Best Practices",{"path":2638,"title":2639},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fmanaging-virtual-environments-for-cross-platform-clis","Managing Python CLI Virtual Environments",{"path":2641,"title":2642},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fpinning-the-python-version-for-a-cli","Pinning the Python Version for a CLI",{"path":2644,"title":2645},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fsupporting-multiple-python-versions-with-nox","Supporting Multiple Python Versions with nox",1789736905048]