[{"data":1,"prerenderedAt":3024},["ShallowReactive",2],{"page-\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fpiping-between-subprocesses-in-python\u002F":3,"content-directory":2179},{"id":4,"title":5,"body":6,"date":2163,"description":2164,"difficulty":2165,"draft":2166,"extension":2167,"meta":2168,"navigation":140,"path":2169,"seo":2170,"stem":2171,"tags":2172,"updated":2163,"__hash__":2178},"content\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fpiping-between-subprocesses-in-python\u002Findex.md","Piping Between Subprocesses in Python Without a Shell",{"type":7,"value":8,"toc":2139},"minimark",[9,49,54,72,76,80,95,99,999,1002,1119,1124,1148,1152,1179,1183,1212,1215,1219,1234,1238,1265,1269,1309,1312,1316,1363,1367,1370,2008,2011,2015,2027,2031,2039,2045,2049,2066,2070,2076,2080,2090,2094,2102,2106,2135],[10,11,12,13,17,18,21,22,25,26,29,30,34,35,38,39,42,43,48],"p",{},"Sooner or later a CLI needs the equivalent of ",[14,15,16],"code",{},"git log --format=%ae | sort | uniq -c",", ",[14,19,20],{},"pg_dump mydb | gzip",", or ",[14,23,24],{},"tar -c dir | ssh host tar -x",". The tempting shortcut is ",[14,27,28],{},"subprocess.run(\"cmd1 | cmd2\", shell=True)",". It works until a file name contains a space or a quote, a user-supplied value turns into shell syntax, or the first command fails and the shell reports success because only the ",[31,32,33],"em",{},"last"," command’s status counts. Building the pipeline in Python — one ",[14,36,37],{},"Popen"," per stage, connected by OS pipes — avoids the shell entirely, keeps arguments as lists, and lets you check every stage’s exit status. The details are fiddly: which pipe ends to close, how to avoid deadlocks on unread stderr, what to do when a downstream command exits early, and how to clean up on timeouts. This guide writes a small ",[14,40,41],{},"run_pipeline"," helper that handles them, and tests each failure mode. It belongs to the ",[44,45,47],"a",{"href":46},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002F","running subprocesses from Python CLIs topic",".",[50,51,53],"h2",{"id":52},"prerequisites","Prerequisites",[55,56,57,69],"ul",{},[58,59,60,64,65,48],"li",{},[44,61,63],{"href":62},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fcalling-external-commands-safely-with-subprocess\u002F","Calling external commands safely with subprocess"," and ",[44,66,68],{"href":67},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Favoiding-shell-injection-in-python-clis\u002F","avoiding shell injection in Python CLIs",[58,70,71],{},"A Unix-like system for the examples; the helper itself also runs on Windows with Windows commands.",[50,73,75],{"id":74},"how-a-pipeline-is-wired","How a pipeline is wired",[77,78],"inline-diagram",{"name":79},"pipe-wiring",[10,81,82,83,86,87,90,91,94],{},"A shell pipeline is nothing magic: the shell creates an OS pipe, starts the first command with its stdout connected to the pipe’s write end, starts the second with its stdin connected to the read end, and waits for both. Data streams between the processes directly, at full speed, without passing through the shell. Python can do the same: ",[14,84,85],{},"Popen(..., stdout=PIPE)"," creates the pipe, and passing that process’s ",[14,88,89],{},".stdout"," as the next process’s ",[14,92,93],{},"stdin"," connects them. Your own process only reads the final stage’s output.",[50,96,98],{"id":97},"the-recipe","The recipe",[100,101,106],"pre",{"className":102,"code":103,"language":104,"meta":105,"style":105},"language-python shiki shiki-themes github-light github-dark","# src\u002Fmytool\u002Fpipeline.py\nfrom __future__ import annotations\n\nimport subprocess\nimport tempfile\nfrom collections.abc import Sequence\nfrom dataclasses import dataclass\n\n\n@dataclass\nclass PipelineError(Exception):\n    command: Sequence[str]\n    returncode: int\n    stderr: str\n\n    def __str__(self) -> str:\n        detail = f\": {self.stderr.strip().splitlines()[-1]}\" if self.stderr.strip() else \"\"\n        return f\"{self.command[0]} exited with status {self.returncode}{detail}\"\n\n\ndef run_pipeline(*commands: Sequence[str], input: bytes | None = None, timeout: float | None = None,\n                 ok_codes: dict[int, set[int]] | None = None) -> bytes:\n    \"\"\"Run cmd1 | cmd2 | ... without a shell and return the last command's stdout.\n\n    Every command's exit status is checked (like `set -o pipefail`); ok_codes lists extra\n    acceptable codes per position, e.g. {1: {1}} for a grep that may match nothing.\n    \"\"\"\n    ok_codes = ok_codes or {}\n    procs: list[subprocess.Popen] = []\n    # stdin and every stderr go through temporary files: pipes we are not reading can fill\n    # up and block a stage forever, files cannot.\n    stdin = tempfile.TemporaryFile() if input is not None else subprocess.DEVNULL\n    stderrs = [tempfile.TemporaryFile() for _ in commands]\n    try:\n        if input is not None:\n            stdin.write(input)\n            stdin.seek(0)\n        for i, command in enumerate(commands):\n            proc = subprocess.Popen(command, stdin=procs[-1].stdout if procs else stdin,\n                                    stdout=subprocess.PIPE, stderr=stderrs[i])\n            if procs:\n                procs[-1].stdout.close()          # the next stage now holds the only read end\n            procs.append(proc)\n        out, _ = procs[-1].communicate(timeout=timeout)\n        for proc in procs[:-1]:\n            proc.wait(timeout=timeout)\n    except BaseException:                          # timeout, Ctrl+C, failure to start a stage\n        for proc in procs:\n            proc.kill()\n            proc.wait()\n        raise\n    finally:\n        if input is not None:\n            stdin.close()\n    try:\n        for i, proc in enumerate(procs):\n            if proc.returncode != 0 and proc.returncode not in ok_codes.get(i, set()):\n                stderrs[i].seek(0)\n                raise PipelineError(commands[i], proc.returncode,\n                                    stderrs[i].read().decode(errors=\"replace\"))\n    finally:\n        for f in stderrs:\n            f.close()\n    return out\n","python","",[14,107,108,117,135,142,151,159,172,185,190,195,202,220,232,241,250,255,272,324,365,370,375,428,461,467,472,478,484,490,507,518,524,530,564,587,595,611,623,633,650,686,710,719,735,741,767,787,799,814,825,831,837,843,851,866,872,879,894,928,938,947,964,971,984,990],{"__ignoreMap":105},[109,110,113],"span",{"class":111,"line":112},"line",1,[109,114,116],{"class":115},"sJ8bj","# src\u002Fmytool\u002Fpipeline.py\n",[109,118,120,124,128,131],{"class":111,"line":119},2,[109,121,123],{"class":122},"szBVR","from",[109,125,127],{"class":126},"sj4cs"," __future__",[109,129,130],{"class":122}," import",[109,132,134],{"class":133},"sVt8B"," annotations\n",[109,136,138],{"class":111,"line":137},3,[109,139,141],{"emptyLinePlaceholder":140},true,"\n",[109,143,145,148],{"class":111,"line":144},4,[109,146,147],{"class":122},"import",[109,149,150],{"class":133}," subprocess\n",[109,152,154,156],{"class":111,"line":153},5,[109,155,147],{"class":122},[109,157,158],{"class":133}," tempfile\n",[109,160,162,164,167,169],{"class":111,"line":161},6,[109,163,123],{"class":122},[109,165,166],{"class":133}," collections.abc ",[109,168,147],{"class":122},[109,170,171],{"class":133}," Sequence\n",[109,173,175,177,180,182],{"class":111,"line":174},7,[109,176,123],{"class":122},[109,178,179],{"class":133}," dataclasses ",[109,181,147],{"class":122},[109,183,184],{"class":133}," dataclass\n",[109,186,188],{"class":111,"line":187},8,[109,189,141],{"emptyLinePlaceholder":140},[109,191,193],{"class":111,"line":192},9,[109,194,141],{"emptyLinePlaceholder":140},[109,196,198],{"class":111,"line":197},10,[109,199,201],{"class":200},"sScJk","@dataclass\n",[109,203,205,208,211,214,217],{"class":111,"line":204},11,[109,206,207],{"class":122},"class",[109,209,210],{"class":200}," PipelineError",[109,212,213],{"class":133},"(",[109,215,216],{"class":126},"Exception",[109,218,219],{"class":133},"):\n",[109,221,223,226,229],{"class":111,"line":222},12,[109,224,225],{"class":133},"    command: Sequence[",[109,227,228],{"class":126},"str",[109,230,231],{"class":133},"]\n",[109,233,235,238],{"class":111,"line":234},13,[109,236,237],{"class":133},"    returncode: ",[109,239,240],{"class":126},"int\n",[109,242,244,247],{"class":111,"line":243},14,[109,245,246],{"class":133},"    stderr: ",[109,248,249],{"class":126},"str\n",[109,251,253],{"class":111,"line":252},15,[109,254,141],{"emptyLinePlaceholder":140},[109,256,258,261,264,267,269],{"class":111,"line":257},16,[109,259,260],{"class":122},"    def",[109,262,263],{"class":126}," __str__",[109,265,266],{"class":133},"(self) -> ",[109,268,228],{"class":126},[109,270,271],{"class":133},":\n",[109,273,275,278,281,284,288,291,294,297,300,303,306,309,312,315,318,321],{"class":111,"line":274},17,[109,276,277],{"class":133},"        detail ",[109,279,280],{"class":122},"=",[109,282,283],{"class":122}," f",[109,285,287],{"class":286},"sZZnC","\": ",[109,289,290],{"class":126},"{self",[109,292,293],{"class":133},".stderr.strip().splitlines()[",[109,295,296],{"class":122},"-",[109,298,299],{"class":126},"1",[109,301,302],{"class":133},"]",[109,304,305],{"class":126},"}",[109,307,308],{"class":286},"\"",[109,310,311],{"class":122}," if",[109,313,314],{"class":126}," self",[109,316,317],{"class":133},".stderr.strip() ",[109,319,320],{"class":122},"else",[109,322,323],{"class":286}," \"\"\n",[109,325,327,330,332,334,336,339,342,344,346,349,351,354,357,360,362],{"class":111,"line":326},18,[109,328,329],{"class":122},"        return",[109,331,283],{"class":122},[109,333,308],{"class":286},[109,335,290],{"class":126},[109,337,338],{"class":133},".command[",[109,340,341],{"class":126},"0",[109,343,302],{"class":133},[109,345,305],{"class":126},[109,347,348],{"class":286}," exited with status ",[109,350,290],{"class":126},[109,352,353],{"class":133},".returncode",[109,355,356],{"class":126},"}{",[109,358,359],{"class":133},"detail",[109,361,305],{"class":126},[109,363,364],{"class":286},"\"\n",[109,366,368],{"class":111,"line":367},19,[109,369,141],{"emptyLinePlaceholder":140},[109,371,373],{"class":111,"line":372},20,[109,374,141],{"emptyLinePlaceholder":140},[109,376,378,381,384,386,389,392,394,397,400,403,406,409,411,414,417,419,421,423,425],{"class":111,"line":377},21,[109,379,380],{"class":122},"def",[109,382,383],{"class":200}," run_pipeline",[109,385,213],{"class":133},[109,387,388],{"class":122},"*",[109,390,391],{"class":133},"commands: Sequence[",[109,393,228],{"class":126},[109,395,396],{"class":133},"], input: ",[109,398,399],{"class":126},"bytes",[109,401,402],{"class":122}," |",[109,404,405],{"class":126}," None",[109,407,408],{"class":122}," =",[109,410,405],{"class":126},[109,412,413],{"class":133},", timeout: ",[109,415,416],{"class":126},"float",[109,418,402],{"class":122},[109,420,405],{"class":126},[109,422,408],{"class":122},[109,424,405],{"class":126},[109,426,427],{"class":133},",\n",[109,429,431,434,437,440,442,445,448,450,452,454,457,459],{"class":111,"line":430},22,[109,432,433],{"class":133},"                 ok_codes: dict[",[109,435,436],{"class":126},"int",[109,438,439],{"class":133},", set[",[109,441,436],{"class":126},[109,443,444],{"class":133},"]] ",[109,446,447],{"class":122},"|",[109,449,405],{"class":126},[109,451,408],{"class":122},[109,453,405],{"class":126},[109,455,456],{"class":133},") -> ",[109,458,399],{"class":126},[109,460,271],{"class":133},[109,462,464],{"class":111,"line":463},23,[109,465,466],{"class":286},"    \"\"\"Run cmd1 | cmd2 | ... without a shell and return the last command's stdout.\n",[109,468,470],{"class":111,"line":469},24,[109,471,141],{"emptyLinePlaceholder":140},[109,473,475],{"class":111,"line":474},25,[109,476,477],{"class":286},"    Every command's exit status is checked (like `set -o pipefail`); ok_codes lists extra\n",[109,479,481],{"class":111,"line":480},26,[109,482,483],{"class":286},"    acceptable codes per position, e.g. {1: {1}} for a grep that may match nothing.\n",[109,485,487],{"class":111,"line":486},27,[109,488,489],{"class":286},"    \"\"\"\n",[109,491,493,496,498,501,504],{"class":111,"line":492},28,[109,494,495],{"class":133},"    ok_codes ",[109,497,280],{"class":122},[109,499,500],{"class":133}," ok_codes ",[109,502,503],{"class":122},"or",[109,505,506],{"class":133}," {}\n",[109,508,510,513,515],{"class":111,"line":509},29,[109,511,512],{"class":133},"    procs: list[subprocess.Popen] ",[109,514,280],{"class":122},[109,516,517],{"class":133}," []\n",[109,519,521],{"class":111,"line":520},30,[109,522,523],{"class":115},"    # stdin and every stderr go through temporary files: pipes we are not reading can fill\n",[109,525,527],{"class":111,"line":526},31,[109,528,529],{"class":115},"    # up and block a stage forever, files cannot.\n",[109,531,533,536,538,541,544,547,550,553,555,558,561],{"class":111,"line":532},32,[109,534,535],{"class":133},"    stdin ",[109,537,280],{"class":122},[109,539,540],{"class":133}," tempfile.TemporaryFile() ",[109,542,543],{"class":122},"if",[109,545,546],{"class":126}," input",[109,548,549],{"class":122}," is",[109,551,552],{"class":122}," not",[109,554,405],{"class":126},[109,556,557],{"class":122}," else",[109,559,560],{"class":133}," subprocess.",[109,562,563],{"class":126},"DEVNULL\n",[109,565,567,570,572,575,578,581,584],{"class":111,"line":566},33,[109,568,569],{"class":133},"    stderrs ",[109,571,280],{"class":122},[109,573,574],{"class":133}," [tempfile.TemporaryFile() ",[109,576,577],{"class":122},"for",[109,579,580],{"class":133}," _ ",[109,582,583],{"class":122},"in",[109,585,586],{"class":133}," commands]\n",[109,588,590,593],{"class":111,"line":589},34,[109,591,592],{"class":122},"    try",[109,594,271],{"class":133},[109,596,598,601,603,605,607,609],{"class":111,"line":597},35,[109,599,600],{"class":122},"        if",[109,602,546],{"class":126},[109,604,549],{"class":122},[109,606,552],{"class":122},[109,608,405],{"class":126},[109,610,271],{"class":133},[109,612,614,617,620],{"class":111,"line":613},36,[109,615,616],{"class":133},"            stdin.write(",[109,618,619],{"class":126},"input",[109,621,622],{"class":133},")\n",[109,624,626,629,631],{"class":111,"line":625},37,[109,627,628],{"class":133},"            stdin.seek(",[109,630,341],{"class":126},[109,632,622],{"class":133},[109,634,636,639,642,644,647],{"class":111,"line":635},38,[109,637,638],{"class":122},"        for",[109,640,641],{"class":133}," i, command ",[109,643,583],{"class":122},[109,645,646],{"class":126}," enumerate",[109,648,649],{"class":133},"(commands):\n",[109,651,653,656,658,661,664,666,669,671,673,676,678,681,683],{"class":111,"line":652},39,[109,654,655],{"class":133},"            proc ",[109,657,280],{"class":122},[109,659,660],{"class":133}," subprocess.Popen(command, ",[109,662,93],{"class":663},"s4XuR",[109,665,280],{"class":122},[109,667,668],{"class":133},"procs[",[109,670,296],{"class":122},[109,672,299],{"class":126},[109,674,675],{"class":133},"].stdout ",[109,677,543],{"class":122},[109,679,680],{"class":133}," procs ",[109,682,320],{"class":122},[109,684,685],{"class":133}," stdin,\n",[109,687,689,692,694,697,700,702,705,707],{"class":111,"line":688},40,[109,690,691],{"class":663},"                                    stdout",[109,693,280],{"class":122},[109,695,696],{"class":133},"subprocess.",[109,698,699],{"class":126},"PIPE",[109,701,17],{"class":133},[109,703,704],{"class":663},"stderr",[109,706,280],{"class":122},[109,708,709],{"class":133},"stderrs[i])\n",[109,711,713,716],{"class":111,"line":712},41,[109,714,715],{"class":122},"            if",[109,717,718],{"class":133}," procs:\n",[109,720,722,725,727,729,732],{"class":111,"line":721},42,[109,723,724],{"class":133},"                procs[",[109,726,296],{"class":122},[109,728,299],{"class":126},[109,730,731],{"class":133},"].stdout.close()          ",[109,733,734],{"class":115},"# the next stage now holds the only read end\n",[109,736,738],{"class":111,"line":737},43,[109,739,740],{"class":133},"            procs.append(proc)\n",[109,742,744,747,749,752,754,756,759,762,764],{"class":111,"line":743},44,[109,745,746],{"class":133},"        out, _ ",[109,748,280],{"class":122},[109,750,751],{"class":133}," procs[",[109,753,296],{"class":122},[109,755,299],{"class":126},[109,757,758],{"class":133},"].communicate(",[109,760,761],{"class":663},"timeout",[109,763,280],{"class":122},[109,765,766],{"class":133},"timeout)\n",[109,768,770,772,775,777,780,782,784],{"class":111,"line":769},45,[109,771,638],{"class":122},[109,773,774],{"class":133}," proc ",[109,776,583],{"class":122},[109,778,779],{"class":133}," procs[:",[109,781,296],{"class":122},[109,783,299],{"class":126},[109,785,786],{"class":133},"]:\n",[109,788,790,793,795,797],{"class":111,"line":789},46,[109,791,792],{"class":133},"            proc.wait(",[109,794,761],{"class":663},[109,796,280],{"class":122},[109,798,766],{"class":133},[109,800,802,805,808,811],{"class":111,"line":801},47,[109,803,804],{"class":122},"    except",[109,806,807],{"class":126}," BaseException",[109,809,810],{"class":133},":                          ",[109,812,813],{"class":115},"# timeout, Ctrl+C, failure to start a stage\n",[109,815,817,819,821,823],{"class":111,"line":816},48,[109,818,638],{"class":122},[109,820,774],{"class":133},[109,822,583],{"class":122},[109,824,718],{"class":133},[109,826,828],{"class":111,"line":827},49,[109,829,830],{"class":133},"            proc.kill()\n",[109,832,834],{"class":111,"line":833},50,[109,835,836],{"class":133},"            proc.wait()\n",[109,838,840],{"class":111,"line":839},51,[109,841,842],{"class":122},"        raise\n",[109,844,846,849],{"class":111,"line":845},52,[109,847,848],{"class":122},"    finally",[109,850,271],{"class":133},[109,852,854,856,858,860,862,864],{"class":111,"line":853},53,[109,855,600],{"class":122},[109,857,546],{"class":126},[109,859,549],{"class":122},[109,861,552],{"class":122},[109,863,405],{"class":126},[109,865,271],{"class":133},[109,867,869],{"class":111,"line":868},54,[109,870,871],{"class":133},"            stdin.close()\n",[109,873,875,877],{"class":111,"line":874},55,[109,876,592],{"class":122},[109,878,271],{"class":133},[109,880,882,884,887,889,891],{"class":111,"line":881},56,[109,883,638],{"class":122},[109,885,886],{"class":133}," i, proc ",[109,888,583],{"class":122},[109,890,646],{"class":126},[109,892,893],{"class":133},"(procs):\n",[109,895,897,899,902,905,908,911,913,916,919,922,925],{"class":111,"line":896},57,[109,898,715],{"class":122},[109,900,901],{"class":133}," proc.returncode ",[109,903,904],{"class":122},"!=",[109,906,907],{"class":126}," 0",[109,909,910],{"class":122}," and",[109,912,901],{"class":133},[109,914,915],{"class":122},"not",[109,917,918],{"class":122}," in",[109,920,921],{"class":133}," ok_codes.get(i, ",[109,923,924],{"class":126},"set",[109,926,927],{"class":133},"()):\n",[109,929,931,934,936],{"class":111,"line":930},58,[109,932,933],{"class":133},"                stderrs[i].seek(",[109,935,341],{"class":126},[109,937,622],{"class":133},[109,939,941,944],{"class":111,"line":940},59,[109,942,943],{"class":122},"                raise",[109,945,946],{"class":133}," PipelineError(commands[i], proc.returncode,\n",[109,948,950,953,956,958,961],{"class":111,"line":949},60,[109,951,952],{"class":133},"                                    stderrs[i].read().decode(",[109,954,955],{"class":663},"errors",[109,957,280],{"class":122},[109,959,960],{"class":286},"\"replace\"",[109,962,963],{"class":133},"))\n",[109,965,967,969],{"class":111,"line":966},61,[109,968,848],{"class":122},[109,970,271],{"class":133},[109,972,974,976,979,981],{"class":111,"line":973},62,[109,975,638],{"class":122},[109,977,978],{"class":133}," f ",[109,980,583],{"class":122},[109,982,983],{"class":133}," stderrs:\n",[109,985,987],{"class":111,"line":986},63,[109,988,989],{"class":133},"            f.close()\n",[109,991,993,996],{"class":111,"line":992},64,[109,994,995],{"class":122},"    return",[109,997,998],{"class":133}," out\n",[10,1000,1001],{},"Used from a command:",[100,1003,1005],{"className":102,"code":1004,"language":104,"meta":105,"style":105},"from mytool.pipeline import run_pipeline\n\nauthors = run_pipeline(\n    [\"git\", \"log\", \"--format=%ae\"],\n    [\"sort\"],\n    [\"uniq\", \"-c\"],\n    timeout=60,\n)\nfor line in authors.decode().splitlines():\n    print(line.strip())\n",[14,1006,1007,1019,1023,1033,1060,1069,1083,1095,1099,1111],{"__ignoreMap":105},[109,1008,1009,1011,1014,1016],{"class":111,"line":112},[109,1010,123],{"class":122},[109,1012,1013],{"class":133}," mytool.pipeline ",[109,1015,147],{"class":122},[109,1017,1018],{"class":133}," run_pipeline\n",[109,1020,1021],{"class":111,"line":119},[109,1022,141],{"emptyLinePlaceholder":140},[109,1024,1025,1028,1030],{"class":111,"line":137},[109,1026,1027],{"class":133},"authors ",[109,1029,280],{"class":122},[109,1031,1032],{"class":133}," run_pipeline(\n",[109,1034,1035,1038,1041,1043,1046,1048,1051,1054,1057],{"class":111,"line":144},[109,1036,1037],{"class":133},"    [",[109,1039,1040],{"class":286},"\"git\"",[109,1042,17],{"class":133},[109,1044,1045],{"class":286},"\"log\"",[109,1047,17],{"class":133},[109,1049,1050],{"class":286},"\"--format=",[109,1052,1053],{"class":126},"%a",[109,1055,1056],{"class":286},"e\"",[109,1058,1059],{"class":133},"],\n",[109,1061,1062,1064,1067],{"class":111,"line":153},[109,1063,1037],{"class":133},[109,1065,1066],{"class":286},"\"sort\"",[109,1068,1059],{"class":133},[109,1070,1071,1073,1076,1078,1081],{"class":111,"line":161},[109,1072,1037],{"class":133},[109,1074,1075],{"class":286},"\"uniq\"",[109,1077,17],{"class":133},[109,1079,1080],{"class":286},"\"-c\"",[109,1082,1059],{"class":133},[109,1084,1085,1088,1090,1093],{"class":111,"line":174},[109,1086,1087],{"class":663},"    timeout",[109,1089,280],{"class":122},[109,1091,1092],{"class":126},"60",[109,1094,427],{"class":133},[109,1096,1097],{"class":111,"line":187},[109,1098,622],{"class":133},[109,1100,1101,1103,1106,1108],{"class":111,"line":192},[109,1102,577],{"class":122},[109,1104,1105],{"class":133}," line ",[109,1107,583],{"class":122},[109,1109,1110],{"class":133}," authors.decode().splitlines():\n",[109,1112,1113,1116],{"class":111,"line":197},[109,1114,1115],{"class":126},"    print",[109,1117,1118],{"class":133},"(line.strip())\n",[1120,1121,1123],"h3",{"id":1122},"closing-the-parents-copy-of-each-pipe","Closing the parent’s copy of each pipe",[10,1125,1126,1127,1130,1131,1134,1135,1139,1140,1143,1144,1147],{},"After starting stage ",[31,1128,1129],{},"n+1"," with stage ",[31,1132,1133],{},"n","’s stdout as its stdin, the helper closes ",[1136,1137,1138],"strong",{},"its own"," handle to that pipe. This line is the one most hand-written pipelines miss, and it matters in both directions. A pipe’s reader sees end-of-file only when every write end is closed, and a writer gets ",[14,1141,1142],{},"SIGPIPE"," only when every read end is closed. If your Python process keeps a copy of the read end open, a downstream command that exits early (",[14,1145,1146],{},"head -n1",") does not cause the upstream command to stop — it keeps producing into a pipe nobody reads, fills it, and blocks forever. Closing the parent’s copy leaves the next stage as the only reader, which is how the shell does it.",[1120,1149,1151],{"id":1150},"checking-every-stage-like-pipefail","Checking every stage, like pipefail",[10,1153,1154,1155,1158,1159,1162,1163,1166,1167,1170,1171,1174,1175,1178],{},"A shell pipeline’s exit status is the last command’s, so ",[14,1156,1157],{},"false | sort"," succeeds. Bash’s ",[14,1160,1161],{},"set -o pipefail"," changes that; the helper does the equivalent by checking every process’s return code and raising ",[14,1164,1165],{},"PipelineError"," for the first failing stage, including the last line of its stderr. Some non-zero codes are not failures: ",[14,1168,1169],{},"grep"," exits 1 when nothing matches, ",[14,1172,1173],{},"diff"," exits 1 when files differ. ",[14,1176,1177],{},"ok_codes"," lets callers list acceptable codes per stage position rather than disabling checks for the whole pipeline.",[1120,1180,1182],{"id":1181},"early-exits-and-sigpipe","Early exits and SIGPIPE",[10,1184,1185,1186,1188,1189,1192,1193,1196,1197,1199,1200,1202,1203,1206,1207,1211],{},"When a downstream command quits before reading everything, the upstream one is killed by ",[14,1187,1142],{}," the next time it writes: ",[14,1190,1191],{},"yes | head -n1"," ends with ",[14,1194,1195],{},"yes"," reporting status −13 in Python’s terms (killed by signal 13). That is normal behaviour, not an error, for producers whose output is deliberately cut short. ",[14,1198,37],{}," restores the default ",[14,1201,1142],{}," handling in children (",[14,1204,1205],{},"restore_signals=True","), so ordinary Unix tools behave exactly as they do in a shell. Allow −13 for producer stages you expect to be cut off, and see ",[44,1208,1210],{"href":1209},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fhandling-broken-pipe-and-sigpipe\u002F","handling broken pipe and SIGPIPE"," for the same issue in your own output.",[77,1213],{"name":1214},"pipe-terminal",[1120,1216,1218],{"id":1217},"avoiding-deadlocks","Avoiding deadlocks",[10,1220,1221,1222,1225,1226,1229,1230,1233],{},"A process blocks when it writes into a full pipe whose reader is not reading. Python only reads the final stage’s stdout, so any ",[31,1223,1224],{},"other"," pipe it holds — an intermediate stage’s stderr, or the first stage’s stdin — can fill up and block the whole chain. The helper sidesteps this by using ",[1136,1227,1228],{},"temporary files"," for every stage’s stderr and for the input data: files never fill, so no stage can block on them, and the stderr content is still available for the error message. ",[14,1231,1232],{},"communicate()"," on the last stage reads its stdout to the end, after which the earlier stages have finished or are about to.",[1120,1235,1237],{"id":1236},"timeouts-and-cleanup","Timeouts and cleanup",[10,1239,1240,1243,1244,1247,1248,1251,1252,1255,1256,1259,1260,1264],{},[14,1241,1242],{},"communicate(timeout=...)"," raises ",[14,1245,1246],{},"TimeoutExpired"," when the pipeline takes too long. The ",[14,1249,1250],{},"except BaseException"," block then kills ",[1136,1253,1254],{},"every"," stage and reaps it, so no orphaned processes keep running in the background — the same applies to Ctrl+C, which arrives as ",[14,1257,1258],{},"KeyboardInterrupt",". The approach is the one described in ",[44,1261,1263],{"href":1262},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fhandling-subprocess-timeouts-and-exit-codes\u002F","handling subprocess timeouts and exit codes",", extended to several processes.",[50,1266,1268],{"id":1267},"when-to-use-python-instead-of-a-stage","When to use Python instead of a stage",[10,1270,1271,1272,1275,1276,1279,1280,1283,1284,1286,1287,1290,1291,1294,1295,17,1298,17,1301,1304,1305,1308],{},"Not every stage needs to be a process. ",[14,1273,1274],{},"sort | uniq -c"," is a ",[14,1277,1278],{},"collections.Counter","; ",[14,1281,1282],{},"gzip"," is the ",[14,1285,1282],{}," module; ",[14,1288,1289],{},"grep pattern"," is a list comprehension. Replacing stages with Python code removes dependencies on external tools (and their platform differences — BSD versus GNU ",[14,1292,1293],{},"sort",", missing tools on Windows) and makes errors easier to report. Keep external stages for what Python cannot do as well: ",[14,1296,1297],{},"pg_dump",[14,1299,1300],{},"ssh",[14,1302,1303],{},"tar"," with special file handling, compressors like ",[14,1306,1307],{},"zstd",", or tools the user expects to see invoked.",[77,1310],{"name":1311},"pipe-choices",[50,1313,1315],{"id":1314},"ux-considerations","UX considerations",[55,1317,1318,1324,1334,1350],{},[58,1319,1320,1323],{},[1136,1321,1322],{},"Report which stage failed."," “sort exited with status 2: sort: cannot read: …” beats “pipeline failed”.",[58,1325,1326,1329,1330,1333],{},[1136,1327,1328],{},"Stream large outputs."," The helper collects the final output in memory, which is fine for reports; for gigabytes, read ",[14,1331,1332],{},"procs[-1].stdout"," incrementally or connect the last stage directly to a file opened by your CLI.",[58,1335,1336,1342,1343,1346,1347,1349],{},[1136,1337,1338,1339,48],{},"Show the equivalent shell command at ",[14,1340,1341],{},"-v"," ",[14,1344,1345],{},"shlex.join"," each stage and join with ",[14,1348,447],{},", so users can reproduce problems by hand.",[58,1351,1352,1342,1355,1358,1359,1362],{},[1136,1353,1354],{},"Check that tools exist first.",[14,1356,1357],{},"shutil.which(\"pg_dump\")"," before starting gives a clear “pg_dump is not installed” instead of a ",[14,1360,1361],{},"FileNotFoundError"," halfway through.",[50,1364,1366],{"id":1365},"testing-the-behaviour","Testing the behaviour",[10,1368,1369],{},"Use the Python interpreter itself to simulate stages, so the tests run anywhere and exercise exactly the failure modes that matter — upstream failures, allowed codes, early exits, noisy stderr and timeouts:",[100,1371,1373],{"className":102,"code":1372,"language":104,"meta":105,"style":105},"# tests\u002Ftest_pipeline.py\nimport shutil\nimport subprocess\nimport sys\n\nimport pytest\n\nfrom mytool.pipeline import PipelineError, run_pipeline\n\nPY = sys.executable\npytestmark = pytest.mark.skipif(not shutil.which(\"sort\"), reason=\"needs sort\")\n\n\ndef test_two_stage_pipeline_with_input():\n    out = run_pipeline([PY, \"-c\", \"import sys; sys.stdout.write(sys.stdin.read().upper())\"],\n                       [\"sort\"], input=b\"b\\na\\nc\\n\")\n    assert out == b\"A\\nB\\nC\\n\"\n\n\ndef test_failure_in_first_stage_is_not_hidden():\n    with pytest.raises(PipelineError) as info:\n        run_pipeline([PY, \"-c\", \"import sys; print('partial'); sys.exit('boom')\"], [\"sort\"])\n    assert info.value.returncode == 1 and \"boom\" in str(info.value)\n\n\ndef test_allowed_exit_codes():\n    out = run_pipeline([PY, \"-c\", \"print('x')\"],\n                       [PY, \"-c\", \"import sys; sys.stdin.read(); sys.exit(1)\"],\n                       ok_codes={1: {1}})\n    assert out == b\"\"\n\n\ndef test_early_exit_downstream_does_not_hang():\n    # The consumer quits after one line; the producer then fails writing (EPIPE), which we accept.\n    producer = [PY, \"-c\", \"import sys\\nfor i in range(10**6): print(i)\"]\n    consumer = [PY, \"-c\", \"import sys; print(sys.stdin.readline().strip())\"]\n    out = run_pipeline(producer, consumer, ok_codes={0: {1, -13}}, timeout=20)\n    assert out == b\"0\\n\"\n\n\ndef test_timeout_kills_every_stage():\n    with pytest.raises(subprocess.TimeoutExpired):\n        run_pipeline([PY, \"-c\", \"import time; time.sleep(30)\"], [\"sort\"], timeout=0.5)\n\n\ndef test_noisy_stderr_does_not_deadlock():\n    noisy = [PY, \"-c\", \"import sys\\nfor i in range(20000): print('warn', i, file=sys.stderr)\\nprint('ok')\"]\n    assert run_pipeline(noisy, [\"sort\"], timeout=20) == b\"ok\\n\"\n",[14,1374,1375,1380,1387,1393,1400,1404,1411,1415,1426,1430,1440,1470,1474,1478,1488,1511,1547,1578,1582,1586,1595,1609,1633,1658,1662,1666,1675,1696,1713,1733,1746,1750,1754,1763,1768,1796,1818,1858,1875,1879,1883,1892,1899,1929,1933,1937,1946,1977],{"__ignoreMap":105},[109,1376,1377],{"class":111,"line":112},[109,1378,1379],{"class":115},"# tests\u002Ftest_pipeline.py\n",[109,1381,1382,1384],{"class":111,"line":119},[109,1383,147],{"class":122},[109,1385,1386],{"class":133}," shutil\n",[109,1388,1389,1391],{"class":111,"line":137},[109,1390,147],{"class":122},[109,1392,150],{"class":133},[109,1394,1395,1397],{"class":111,"line":144},[109,1396,147],{"class":122},[109,1398,1399],{"class":133}," sys\n",[109,1401,1402],{"class":111,"line":153},[109,1403,141],{"emptyLinePlaceholder":140},[109,1405,1406,1408],{"class":111,"line":161},[109,1407,147],{"class":122},[109,1409,1410],{"class":133}," pytest\n",[109,1412,1413],{"class":111,"line":174},[109,1414,141],{"emptyLinePlaceholder":140},[109,1416,1417,1419,1421,1423],{"class":111,"line":187},[109,1418,123],{"class":122},[109,1420,1013],{"class":133},[109,1422,147],{"class":122},[109,1424,1425],{"class":133}," PipelineError, run_pipeline\n",[109,1427,1428],{"class":111,"line":192},[109,1429,141],{"emptyLinePlaceholder":140},[109,1431,1432,1435,1437],{"class":111,"line":197},[109,1433,1434],{"class":126},"PY",[109,1436,408],{"class":122},[109,1438,1439],{"class":133}," sys.executable\n",[109,1441,1442,1445,1447,1450,1452,1455,1457,1460,1463,1465,1468],{"class":111,"line":204},[109,1443,1444],{"class":133},"pytestmark ",[109,1446,280],{"class":122},[109,1448,1449],{"class":133}," pytest.mark.skipif(",[109,1451,915],{"class":122},[109,1453,1454],{"class":133}," shutil.which(",[109,1456,1066],{"class":286},[109,1458,1459],{"class":133},"), ",[109,1461,1462],{"class":663},"reason",[109,1464,280],{"class":122},[109,1466,1467],{"class":286},"\"needs sort\"",[109,1469,622],{"class":133},[109,1471,1472],{"class":111,"line":222},[109,1473,141],{"emptyLinePlaceholder":140},[109,1475,1476],{"class":111,"line":234},[109,1477,141],{"emptyLinePlaceholder":140},[109,1479,1480,1482,1485],{"class":111,"line":243},[109,1481,380],{"class":122},[109,1483,1484],{"class":200}," test_two_stage_pipeline_with_input",[109,1486,1487],{"class":133},"():\n",[109,1489,1490,1493,1495,1498,1500,1502,1504,1506,1509],{"class":111,"line":252},[109,1491,1492],{"class":133},"    out ",[109,1494,280],{"class":122},[109,1496,1497],{"class":133}," run_pipeline([",[109,1499,1434],{"class":126},[109,1501,17],{"class":133},[109,1503,1080],{"class":286},[109,1505,17],{"class":133},[109,1507,1508],{"class":286},"\"import sys; sys.stdout.write(sys.stdin.read().upper())\"",[109,1510,1059],{"class":133},[109,1512,1513,1516,1518,1521,1523,1525,1528,1531,1534,1536,1538,1541,1543,1545],{"class":111,"line":257},[109,1514,1515],{"class":133},"                       [",[109,1517,1066],{"class":286},[109,1519,1520],{"class":133},"], ",[109,1522,619],{"class":663},[109,1524,280],{"class":122},[109,1526,1527],{"class":122},"b",[109,1529,1530],{"class":286},"\"b",[109,1532,1533],{"class":126},"\\n",[109,1535,44],{"class":286},[109,1537,1533],{"class":126},[109,1539,1540],{"class":286},"c",[109,1542,1533],{"class":126},[109,1544,308],{"class":286},[109,1546,622],{"class":133},[109,1548,1549,1552,1555,1558,1561,1564,1566,1569,1571,1574,1576],{"class":111,"line":274},[109,1550,1551],{"class":122},"    assert",[109,1553,1554],{"class":133}," out ",[109,1556,1557],{"class":122},"==",[109,1559,1560],{"class":122}," b",[109,1562,1563],{"class":286},"\"A",[109,1565,1533],{"class":126},[109,1567,1568],{"class":286},"B",[109,1570,1533],{"class":126},[109,1572,1573],{"class":286},"C",[109,1575,1533],{"class":126},[109,1577,364],{"class":286},[109,1579,1580],{"class":111,"line":326},[109,1581,141],{"emptyLinePlaceholder":140},[109,1583,1584],{"class":111,"line":367},[109,1585,141],{"emptyLinePlaceholder":140},[109,1587,1588,1590,1593],{"class":111,"line":372},[109,1589,380],{"class":122},[109,1591,1592],{"class":200}," test_failure_in_first_stage_is_not_hidden",[109,1594,1487],{"class":133},[109,1596,1597,1600,1603,1606],{"class":111,"line":377},[109,1598,1599],{"class":122},"    with",[109,1601,1602],{"class":133}," pytest.raises(PipelineError) ",[109,1604,1605],{"class":122},"as",[109,1607,1608],{"class":133}," info:\n",[109,1610,1611,1614,1616,1618,1620,1622,1625,1628,1630],{"class":111,"line":430},[109,1612,1613],{"class":133},"        run_pipeline([",[109,1615,1434],{"class":126},[109,1617,17],{"class":133},[109,1619,1080],{"class":286},[109,1621,17],{"class":133},[109,1623,1624],{"class":286},"\"import sys; print('partial'); sys.exit('boom')\"",[109,1626,1627],{"class":133},"], [",[109,1629,1066],{"class":286},[109,1631,1632],{"class":133},"])\n",[109,1634,1635,1637,1640,1642,1645,1647,1650,1652,1655],{"class":111,"line":463},[109,1636,1551],{"class":122},[109,1638,1639],{"class":133}," info.value.returncode ",[109,1641,1557],{"class":122},[109,1643,1644],{"class":126}," 1",[109,1646,910],{"class":122},[109,1648,1649],{"class":286}," \"boom\"",[109,1651,918],{"class":122},[109,1653,1654],{"class":126}," str",[109,1656,1657],{"class":133},"(info.value)\n",[109,1659,1660],{"class":111,"line":469},[109,1661,141],{"emptyLinePlaceholder":140},[109,1663,1664],{"class":111,"line":474},[109,1665,141],{"emptyLinePlaceholder":140},[109,1667,1668,1670,1673],{"class":111,"line":480},[109,1669,380],{"class":122},[109,1671,1672],{"class":200}," test_allowed_exit_codes",[109,1674,1487],{"class":133},[109,1676,1677,1679,1681,1683,1685,1687,1689,1691,1694],{"class":111,"line":486},[109,1678,1492],{"class":133},[109,1680,280],{"class":122},[109,1682,1497],{"class":133},[109,1684,1434],{"class":126},[109,1686,17],{"class":133},[109,1688,1080],{"class":286},[109,1690,17],{"class":133},[109,1692,1693],{"class":286},"\"print('x')\"",[109,1695,1059],{"class":133},[109,1697,1698,1700,1702,1704,1706,1708,1711],{"class":111,"line":492},[109,1699,1515],{"class":133},[109,1701,1434],{"class":126},[109,1703,17],{"class":133},[109,1705,1080],{"class":286},[109,1707,17],{"class":133},[109,1709,1710],{"class":286},"\"import sys; sys.stdin.read(); sys.exit(1)\"",[109,1712,1059],{"class":133},[109,1714,1715,1718,1720,1723,1725,1728,1730],{"class":111,"line":509},[109,1716,1717],{"class":663},"                       ok_codes",[109,1719,280],{"class":122},[109,1721,1722],{"class":133},"{",[109,1724,299],{"class":126},[109,1726,1727],{"class":133},": {",[109,1729,299],{"class":126},[109,1731,1732],{"class":133},"}})\n",[109,1734,1735,1737,1739,1741,1743],{"class":111,"line":520},[109,1736,1551],{"class":122},[109,1738,1554],{"class":133},[109,1740,1557],{"class":122},[109,1742,1560],{"class":122},[109,1744,1745],{"class":286},"\"\"\n",[109,1747,1748],{"class":111,"line":526},[109,1749,141],{"emptyLinePlaceholder":140},[109,1751,1752],{"class":111,"line":532},[109,1753,141],{"emptyLinePlaceholder":140},[109,1755,1756,1758,1761],{"class":111,"line":566},[109,1757,380],{"class":122},[109,1759,1760],{"class":200}," test_early_exit_downstream_does_not_hang",[109,1762,1487],{"class":133},[109,1764,1765],{"class":111,"line":589},[109,1766,1767],{"class":115},"    # The consumer quits after one line; the producer then fails writing (EPIPE), which we accept.\n",[109,1769,1770,1773,1775,1778,1780,1782,1784,1786,1789,1791,1794],{"class":111,"line":597},[109,1771,1772],{"class":133},"    producer ",[109,1774,280],{"class":122},[109,1776,1777],{"class":133}," [",[109,1779,1434],{"class":126},[109,1781,17],{"class":133},[109,1783,1080],{"class":286},[109,1785,17],{"class":133},[109,1787,1788],{"class":286},"\"import sys",[109,1790,1533],{"class":126},[109,1792,1793],{"class":286},"for i in range(10**6): print(i)\"",[109,1795,231],{"class":133},[109,1797,1798,1801,1803,1805,1807,1809,1811,1813,1816],{"class":111,"line":613},[109,1799,1800],{"class":133},"    consumer ",[109,1802,280],{"class":122},[109,1804,1777],{"class":133},[109,1806,1434],{"class":126},[109,1808,17],{"class":133},[109,1810,1080],{"class":286},[109,1812,17],{"class":133},[109,1814,1815],{"class":286},"\"import sys; print(sys.stdin.readline().strip())\"",[109,1817,231],{"class":133},[109,1819,1820,1822,1824,1827,1829,1831,1833,1835,1837,1839,1841,1843,1846,1849,1851,1853,1856],{"class":111,"line":625},[109,1821,1492],{"class":133},[109,1823,280],{"class":122},[109,1825,1826],{"class":133}," run_pipeline(producer, consumer, ",[109,1828,1177],{"class":663},[109,1830,280],{"class":122},[109,1832,1722],{"class":133},[109,1834,341],{"class":126},[109,1836,1727],{"class":133},[109,1838,299],{"class":126},[109,1840,17],{"class":133},[109,1842,296],{"class":122},[109,1844,1845],{"class":126},"13",[109,1847,1848],{"class":133},"}}, ",[109,1850,761],{"class":663},[109,1852,280],{"class":122},[109,1854,1855],{"class":126},"20",[109,1857,622],{"class":133},[109,1859,1860,1862,1864,1866,1868,1871,1873],{"class":111,"line":635},[109,1861,1551],{"class":122},[109,1863,1554],{"class":133},[109,1865,1557],{"class":122},[109,1867,1560],{"class":122},[109,1869,1870],{"class":286},"\"0",[109,1872,1533],{"class":126},[109,1874,364],{"class":286},[109,1876,1877],{"class":111,"line":652},[109,1878,141],{"emptyLinePlaceholder":140},[109,1880,1881],{"class":111,"line":688},[109,1882,141],{"emptyLinePlaceholder":140},[109,1884,1885,1887,1890],{"class":111,"line":712},[109,1886,380],{"class":122},[109,1888,1889],{"class":200}," test_timeout_kills_every_stage",[109,1891,1487],{"class":133},[109,1893,1894,1896],{"class":111,"line":721},[109,1895,1599],{"class":122},[109,1897,1898],{"class":133}," pytest.raises(subprocess.TimeoutExpired):\n",[109,1900,1901,1903,1905,1907,1909,1911,1914,1916,1918,1920,1922,1924,1927],{"class":111,"line":737},[109,1902,1613],{"class":133},[109,1904,1434],{"class":126},[109,1906,17],{"class":133},[109,1908,1080],{"class":286},[109,1910,17],{"class":133},[109,1912,1913],{"class":286},"\"import time; time.sleep(30)\"",[109,1915,1627],{"class":133},[109,1917,1066],{"class":286},[109,1919,1520],{"class":133},[109,1921,761],{"class":663},[109,1923,280],{"class":122},[109,1925,1926],{"class":126},"0.5",[109,1928,622],{"class":133},[109,1930,1931],{"class":111,"line":743},[109,1932,141],{"emptyLinePlaceholder":140},[109,1934,1935],{"class":111,"line":769},[109,1936,141],{"emptyLinePlaceholder":140},[109,1938,1939,1941,1944],{"class":111,"line":789},[109,1940,380],{"class":122},[109,1942,1943],{"class":200}," test_noisy_stderr_does_not_deadlock",[109,1945,1487],{"class":133},[109,1947,1948,1951,1953,1955,1957,1959,1961,1963,1965,1967,1970,1972,1975],{"class":111,"line":801},[109,1949,1950],{"class":133},"    noisy ",[109,1952,280],{"class":122},[109,1954,1777],{"class":133},[109,1956,1434],{"class":126},[109,1958,17],{"class":133},[109,1960,1080],{"class":286},[109,1962,17],{"class":133},[109,1964,1788],{"class":286},[109,1966,1533],{"class":126},[109,1968,1969],{"class":286},"for i in range(20000): print('warn', i, file=sys.stderr)",[109,1971,1533],{"class":126},[109,1973,1974],{"class":286},"print('ok')\"",[109,1976,231],{"class":133},[109,1978,1979,1981,1984,1986,1988,1990,1992,1994,1997,1999,2001,2004,2006],{"class":111,"line":816},[109,1980,1551],{"class":122},[109,1982,1983],{"class":133}," run_pipeline(noisy, [",[109,1985,1066],{"class":286},[109,1987,1520],{"class":133},[109,1989,761],{"class":663},[109,1991,280],{"class":122},[109,1993,1855],{"class":126},[109,1995,1996],{"class":133},") ",[109,1998,1557],{"class":122},[109,2000,1560],{"class":122},[109,2002,2003],{"class":286},"\"ok",[109,2005,1533],{"class":126},[109,2007,364],{"class":286},[10,2009,2010],{},"The early-exit test would hang without the “close the parent’s copy” line, which is why it has a timeout; the noisy-stderr test would hang without temporary files for stderr. Both turn subtle deadlocks into fast, deterministic test failures.",[50,2012,2014],{"id":2013},"conclusion","Conclusion",[10,2016,2017,2018,2020,2021,2023,2024,2026],{},"Pipelines do not need a shell. Start one ",[14,2019,37],{}," per stage, pass each stage’s stdout as the next stage’s stdin and close the parent’s copy, send stderr and input through temporary files so nothing can deadlock, read the final output with ",[14,2022,1242],{},", check every stage’s exit status with per-stage exceptions for codes like ",[14,2025,1169],{},"’s 1 and SIGPIPE’s −13, kill and reap every stage on timeouts and Ctrl+C, and replace stages with Python where that is simpler.",[50,2028,2030],{"id":2029},"frequently-asked-questions","Frequently asked questions",[1120,2032,2034,2035,2038],{"id":2033},"is-shelltrue-ever-acceptable-for-pipelines","Is ",[14,2036,2037],{},"shell=True"," ever acceptable for pipelines?",[10,2040,2041,2042,2044],{},"For a fixed command string with no user-supplied parts, run in a context you control, it is safe — but you still lose per-stage exit codes unless you add ",[14,2043,1161],{},", and it requires a POSIX shell. The explicit form is safer by construction and portable.",[1120,2046,2048],{"id":2047},"how-do-i-write-the-pipelines-output-straight-to-a-file","How do I write the pipeline’s output straight to a file?",[10,2050,2051,2052,2055,2056,2058,2059,2062,2063,2065],{},"Pass an open file as the last stage’s ",[14,2053,2054],{},"stdout"," instead of ",[14,2057,699],{},": ",[14,2060,2061],{},"open(\"dump.sql.gz\", \"wb\")",". Then ",[14,2064,1232],{}," returns no data, and the file receives the output without passing through Python.",[1120,2067,2069],{"id":2068},"can-i-feed-a-large-input-without-temporary-files","Can I feed a large input without temporary files?",[10,2071,2072,2073,2075],{},"Yes, by writing to the first stage’s stdin from a separate thread while the main thread reads the last stage’s stdout — which is what ",[14,2074,1232],{}," does internally for a single process. A temporary file is simpler and fast enough for most CLIs.",[1120,2077,2079],{"id":2078},"does-this-work-with-asyncio","Does this work with asyncio?",[10,2081,2082,2085,2086,2089],{},[14,2083,2084],{},"asyncio.create_subprocess_exec"," accepts the same idea, but connecting one process’s output to another’s input needs ",[14,2087,2088],{},"os.pipe()"," file descriptors passed explicitly. For most CLIs, running the synchronous helper in a thread is simpler.",[1120,2091,2093],{"id":2092},"what-about-windows","What about Windows?",[10,2095,2096,2098,2099,2101],{},[14,2097,37],{}," pipes work the same way on Windows; the commands differ. There is no ",[14,2100,1142],{},", so an upstream process usually gets a write error instead and exits with its own error code.",[50,2103,2105],{"id":2104},"related","Related",[55,2107,2108,2114,2118,2124,2129],{},[58,2109,2110,2111],{},"Up: ",[44,2112,2113],{"href":46},"Running subprocesses from Python CLIs",[58,2115,2116],{},[44,2117,63],{"href":62},[58,2119,2120],{},[44,2121,2123],{"href":2122},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fstreaming-subprocess-output-in-real-time\u002F","Streaming subprocess output in real time",[58,2125,2126],{},[44,2127,2128],{"href":1262},"Handling subprocess timeouts and exit codes",[58,2130,2131],{},[44,2132,2134],{"href":2133},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Flaunching-the-users-editor-from-a-cli\u002F","Launching the user’s editor from a CLI",[2136,2137,2138],"style",{},"html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .s4XuR, html code.shiki .s4XuR{--shiki-default:#E36209;--shiki-dark:#FFAB70}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":105,"searchDepth":119,"depth":119,"links":2140},[2141,2142,2143,2150,2151,2152,2153,2154,2162],{"id":52,"depth":119,"text":53},{"id":74,"depth":119,"text":75},{"id":97,"depth":119,"text":98,"children":2144},[2145,2146,2147,2148,2149],{"id":1122,"depth":137,"text":1123},{"id":1150,"depth":137,"text":1151},{"id":1181,"depth":137,"text":1182},{"id":1217,"depth":137,"text":1218},{"id":1236,"depth":137,"text":1237},{"id":1267,"depth":119,"text":1268},{"id":1314,"depth":119,"text":1315},{"id":1365,"depth":119,"text":1366},{"id":2013,"depth":119,"text":2014},{"id":2029,"depth":119,"text":2030,"children":2155},[2156,2158,2159,2160,2161],{"id":2033,"depth":137,"text":2157},"Is shell=True ever acceptable for pipelines?",{"id":2047,"depth":137,"text":2048},{"id":2068,"depth":137,"text":2069},{"id":2078,"depth":137,"text":2079},{"id":2092,"depth":137,"text":2093},{"id":2104,"depth":119,"text":2105},"2026-10-02","Build cmd1 | cmd2 pipelines in a Python CLI without shell=True: chained Popen objects, closed pipe ends, pipefail-style checks, SIGPIPE, timeouts and tests.","advanced",false,"md",{},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fpiping-between-subprocesses-in-python",{"title":5,"description":2164},"cli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fpiping-between-subprocesses-in-python\u002Findex",[2173,2174,2175,2176,2177],"subprocess","pipes","popen","sigpipe","error-handling","gLPGZ-dDF80slKNngm-osH7s5kJJJwOxEpwy48vulyE",[2180,2183,2186,2189,2192,2195,2198,2201,2204,2207,2210,2213,2216,2219,2222,2225,2228,2231,2234,2237,2240,2243,2246,2249,2252,2255,2258,2261,2264,2267,2270,2273,2276,2279,2282,2285,2288,2291,2294,2297,2300,2303,2306,2309,2312,2315,2318,2321,2324,2327,2330,2333,2336,2339,2342,2345,2348,2351,2354,2357,2360,2363,2366,2369,2372,2375,2378,2381,2384,2387,2390,2393,2396,2399,2402,2405,2408,2411,2414,2417,2420,2423,2426,2429,2432,2435,2438,2441,2444,2447,2450,2453,2456,2459,2462,2465,2468,2471,2474,2477,2480,2483,2486,2489,2492,2495,2498,2501,2504,2507,2510,2513,2516,2519,2522,2525,2528,2531,2534,2537,2540,2543,2546,2549,2552,2553,2556,2559,2562,2565,2568,2571,2574,2577,2580,2583,2586,2589,2592,2595,2598,2601,2604,2607,2610,2613,2616,2619,2622,2625,2628,2631,2634,2637,2640,2643,2646,2649,2652,2655,2658,2661,2664,2667,2670,2673,2676,2679,2682,2685,2688,2691,2694,2697,2700,2703,2706,2709,2712,2715,2718,2721,2724,2727,2730,2733,2736,2739,2742,2745,2748,2751,2754,2757,2760,2763,2766,2769,2772,2775,2778,2781,2784,2787,2790,2793,2796,2799,2802,2805,2808,2811,2814,2817,2820,2823,2826,2829,2832,2835,2838,2841,2844,2847,2850,2853,2856,2859,2862,2865,2868,2871,2874,2877,2880,2883,2886,2889,2892,2895,2898,2901,2904,2907,2910,2913,2916,2919,2922,2925,2928,2931,2934,2937,2940,2943,2946,2949,2952,2955,2958,2961,2964,2967,2970,2973,2976,2979,2982,2985,2988,2991,2994,2997,3000,3003,3006,3009,3012,3015,3018,3021],{"path":2181,"title":2182},"\u002Fabout","About Python CLI Toolcraft",{"path":2184,"title":2185},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies","Advanced Argument Validation Strategies",{"path":2187,"title":2188},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fparsing-nested-json-arguments-in-python-clis","Parsing Nested JSON Args in Python CLIs",{"path":2190,"title":2191},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-dates-and-durations-in-cli-arguments","Validating Dates and Durations in Python CLI Arguments",{"path":2193,"title":2194},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-dependent-and-conflicting-options","Validating Dependent and Conflicting CLI Options",{"path":2196,"title":2197},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-file-and-directory-paths-in-clis","Validating File and Directory Paths in CLIs",{"path":2199,"title":2200},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-urls-hosts-and-ports","Validating URLs, Hosts and Ports in Python CLI Arguments",{"path":2202,"title":2203},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fwriting-custom-click-parameter-types","Writing Custom Click Parameter Types",{"path":2205,"title":2206},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fbrowsing-records-with-a-textual-datatable","Browsing Records with a Textual DataTable Picker",{"path":2208,"title":2209},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fbuilding-your-first-textual-app","Building Your First Textual App for a Python CLI",{"path":2211,"title":2212},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fchoosing-between-a-cli-a-prompt-flow-and-a-tui","Choosing Between a CLI, a Prompt Flow and a TUI",{"path":2214,"title":2215},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual","Building Terminal UIs with Textual for Python CLIs",{"path":2217,"title":2218},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Frunning-background-work-in-textual-with-workers","Running Background Work in Textual with Workers",{"path":2220,"title":2221},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fstyling-textual-apps-with-tcss","Styling Textual Apps with TCSS",{"path":2223,"title":2224},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Ftesting-textual-apps-with-pilot","Testing Textual Apps with Pilot",{"path":2226,"title":2227},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fadding-examples-and-epilogs-to-help-output","Adding Examples and Epilogs to Help Output",{"path":2229,"title":2230},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fdocumenting-environment-variables-in-help","Documenting Environment Variables in CLI Help",{"path":2232,"title":2233},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fgenerating-man-pages-and-docs-from-a-cli","Generating Man Pages and Docs from a CLI",{"path":2235,"title":2236},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation","CLI Help Output and Documentation",{"path":2238,"title":2239},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Frich-formatted-help-with-rich-click","Rich-Formatted Help for Click CLIs with rich-click",{"path":2241,"title":2242},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fversioning-and-deprecating-cli-flags","Versioning and Deprecating CLI Flags",{"path":2244,"title":2245},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fwriting-help-text-users-actually-read","Writing Help Text Users Actually Read",{"path":2247,"title":2248},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fadapting-output-to-terminal-width","Adapting Python CLI Output to Terminal Width",{"path":2250,"title":2251},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fdetecting-ci-environments-and-non-interactive-shells","Detecting CI Environments and Non-Interactive Shells",{"path":2253,"title":2254},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Ffixing-unicode-and-encoding-errors-on-windows","Fixing Unicode and Encoding Errors on Windows in Python CLIs",{"path":2256,"title":2257},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fhandling-ansi-escape-codes-on-windows-consoles","Handling ANSI Escape Codes on Windows Consoles",{"path":2259,"title":2260},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility","Cross-Platform Terminal Compatibility for Python CLIs",{"path":2262,"title":2263},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Frespecting-no-color-and-force-color","Respecting NO_COLOR and FORCE_COLOR in Python CLIs",{"path":2265,"title":2266},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fsupporting-dumb-terminals-and-screen-readers","Supporting Dumb Terminals and Screen Readers in a Python CLI",{"path":2268,"title":2269},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fchoosing-exit-codes-for-cli-tools","Choosing Exit Codes for CLI Tools",{"path":2271,"title":2272},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fdesigning-an-exception-hierarchy-for-a-cli","Designing an Exception Hierarchy for a Python CLI",{"path":2274,"title":2275},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fdid-you-mean-suggestions-for-mistyped-input","Did You Mean…? Suggestions for Mistyped CLI Input",{"path":2277,"title":2278},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Ffriendly-error-messages-and-tracebacks","Friendly Error Messages and Tracebacks",{"path":2280,"title":2281},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fhandling-keyboard-interrupt-cleanly","Handling Keyboard Interrupt Cleanly",{"path":2283,"title":2284},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes","Error Handling and Exit Codes for CLIs",{"path":2286,"title":2287},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Freporting-machine-readable-errors-in-json-mode","Reporting Machine-Readable Errors in JSON Mode",{"path":2289,"title":2290},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fwriting-crash-reports-users-can-send","Writing Crash Reports Users Can Send",{"path":2292,"title":2293},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fconfig-precedence-flags-env-files-defaults","Config Precedence: Flags, Env, Files, Defaults",{"path":2295,"title":2296},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fdiscovering-project-config-files-by-walking-up-directories","Discovering Project Config Files by Walking Up Directories",{"path":2298,"title":2299},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars","Handling Config Files and Env Vars in CLIs",{"path":2301,"title":2302},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Floading-yaml-configs-safely-in-cli-apps","Loading YAML configs safely in CLI apps",{"path":2304,"title":2305},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Freading-toml-config-with-tomllib","Reading TOML Config with tomllib in Python CLIs",{"path":2307,"title":2308},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Ftyped-settings-with-pydantic-settings","Typed Settings with pydantic-settings in Python CLIs",{"path":2310,"title":2311},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fvalidating-config-files-with-json-schema","Validating Config Files with JSON Schema in a Python CLI",{"path":2313,"title":2314},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fwriting-a-config-init-and-edit-command","Writing a Config Init and Edit Command for a Python CLI",{"path":2316,"title":2317},"\u002Fadvanced-input-parsing-user-experience","Advanced Input Parsing for Python CLIs",{"path":2319,"title":2320},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fadding-progress-bars-and-spinners-to-python-clis","Progress Bars and Spinners for Python CLIs",{"path":2322,"title":2323},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fbuilding-interactive-prompts-and-menus","Building Interactive Prompts and Menus in Python CLIs",{"path":2325,"title":2326},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fbuilding-tree-views-with-rich","Building Tree Views with Rich in a Python CLI",{"path":2328,"title":2329},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich","Interactive Terminal UI with Rich",{"path":2331,"title":2332},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Flive-dashboards-with-rich-live","Live Dashboards with Rich Live in Python CLIs",{"path":2334,"title":2335},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Frendering-markdown-and-syntax-highlighting-with-rich","Rendering Markdown and Syntax Highlighting with Rich",{"path":2337,"title":2338},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Frendering-tables-and-json-with-rich","Rendering Tables and JSON with Rich",{"path":2340,"title":2341},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Ftheming-rich-output-consistently","Theming Rich Output Consistently in Python CLIs",{"path":2343,"title":2344},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fadding-a-format-flag-for-table-json-and-csv","Adding a Format Flag for Table, JSON and CSV Output",{"path":2346,"title":2347},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fcustom-output-templates-with-a-format-string","Custom Output Templates with a Format String in Python CLIs",{"path":2349,"title":2350},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fexporting-cli-results-to-files","Exporting CLI Results to Files from a Python CLI",{"path":2352,"title":2353},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis","Output Formats for Data-Heavy Python CLIs",{"path":2355,"title":2356},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fselecting-fields-and-columns-from-cli-output","Selecting Fields and Columns from Python CLI Output",{"path":2358,"title":2359},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fwriting-csv-and-tsv-output-correctly","Writing CSV and TSV Output Correctly from a Python CLI",{"path":2361,"title":2362},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fargcomplete-for-argparse-clis","Tab Completion for argparse CLIs with argcomplete",{"path":2364,"title":2365},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fdynamic-completion-values-from-apis-and-files","Dynamic Completion Values from APIs and Files",{"path":2367,"title":2368},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fenabling-tab-completion-in-click-and-typer","Enabling Tab Completion in Click and Typer",{"path":2370,"title":2371},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis","Shell Completion for Python CLIs",{"path":2373,"title":2374},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Finstalling-shell-completion-for-bash-zsh-fish","Installing Shell Completion for bash, zsh, fish",{"path":2376,"title":2377},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fshipping-completion-scripts-with-packages","Shipping Shell Completion Scripts with a Python CLI",{"path":2379,"title":2380},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Ftesting-shell-completion-in-python-clis","Testing Shell Completion in Python CLIs",{"path":2382,"title":2383},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fadding-trace-ids-and-context-to-cli-logs","Adding Trace IDs and Context to Python CLI Logs",{"path":2385,"title":2386},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fadding-verbose-and-quiet-logging-flags","Adding Verbose and Quiet Logging Flags",{"path":2388,"title":2389},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps","Structured Logging for CLI Apps",{"path":2391,"title":2392},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Flogging-with-structlog-in-a-cli","Logging with structlog in a Python CLI",{"path":2394,"title":2395},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fseparating-logs-from-program-output","Separating Logs from Program Output in a Python CLI",{"path":2397,"title":2398},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fstructured-json-logging-in-python-clis","Structured JSON Logging in Python CLIs",{"path":2400,"title":2401},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fwriting-rotating-log-files-from-a-cli","Writing Rotating Log Files from a Python CLI",{"path":2403,"title":2404},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fdetecting-tty-and-adapting-output","Detecting a TTY and Adapting Output",{"path":2406,"title":2407},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Femitting-json-output-for-scripting","Emitting JSON Output for Scripting",{"path":2409,"title":2410},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fhandling-broken-pipe-and-sigpipe","Handling Broken Pipe and SIGPIPE",{"path":2412,"title":2413},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes","Working with stdin, stdout and Pipes",{"path":2415,"title":2416},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fnull-delimited-input-and-xargs-compatibility","Null-Delimited Input and xargs Compatibility in Python CLIs",{"path":2418,"title":2419},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fprocessing-large-files-and-ndjson-streams","Processing Large Files and NDJSON Streams in Python CLIs",{"path":2421,"title":2422},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Freading-piped-input-in-python-clis","Reading Piped Input in Python CLIs",{"path":2424,"title":2425},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fbuilding-an-api-client-cli-with-httpx","Building an API Client CLI with httpx",{"path":2427,"title":2428},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fdownloading-files-with-progress-in-python","Downloading Files with Progress in Python CLIs",{"path":2430,"title":2431},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis","Calling HTTP APIs from Python CLIs",{"path":2433,"title":2434},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fmocking-http-in-cli-tests-with-respx","Mocking HTTP in Python CLI Tests with respx",{"path":2436,"title":2437},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Foauth-device-flow-login-for-clis","OAuth Device Flow Login for Python CLIs",{"path":2439,"title":2440},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fpaginating-api-results-in-a-cli","Paginating API Results in a Python CLI",{"path":2442,"title":2443},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fretries-and-backoff-for-cli-http-calls","Retries and Backoff for CLI HTTP Calls",{"path":2445,"title":2446},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fuploading-files-with-multipart-and-progress","Uploading Files with Multipart and Progress in a Python CLI",{"path":2448,"title":2449},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fcancelling-async-tasks-on-ctrl-c","Cancelling Async Tasks on Ctrl+C in Python CLIs",{"path":2451,"title":2452},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis","Concurrency and Async in Python CLIs",{"path":2454,"title":2455},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fmultiprocessing-for-cpu-bound-cli-tasks","Multiprocessing for CPU-Bound CLI Tasks",{"path":2457,"title":2458},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fparallelising-cli-work-with-thread-pools","Parallelising CLI Work with Thread Pools",{"path":2460,"title":2461},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Frate-limiting-concurrent-requests-in-clis","Rate-Limiting Concurrent Requests in Python CLIs",{"path":2463,"title":2464},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Frunning-async-code-in-typer-and-click","Running Async Code in Typer and Click",{"path":2466,"title":2467},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fshowing-progress-for-concurrent-tasks","Showing Progress for Concurrent Tasks in a Python CLI",{"path":2469,"title":2470},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fstructured-concurrency-with-taskgroups-in-clis","Structured Concurrency with TaskGroups in Python CLIs",{"path":2472,"title":2473},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fcross-platform-paths-with-pathlib","Cross-Platform Paths with pathlib in CLIs",{"path":2475,"title":2476},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Ffile-locking-for-concurrent-cli-runs","File Locking for Concurrent CLI Runs in Python",{"path":2478,"title":2479},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fhandling-file-permissions-and-umask-in-clis","Handling File Permissions and umask in Python CLIs",{"path":2481,"title":2482},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes","Filesystem Paths and Atomic Writes for CLIs",{"path":2484,"title":2485},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fsafe-temporary-files-and-directories","Safe Temporary Files and Directories in CLIs",{"path":2487,"title":2488},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fstoring-app-data-with-platformdirs","Storing CLI App Data with platformdirs",{"path":2490,"title":2491},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fwalking-directory-trees-with-ignore-rules","Walking Directory Trees with Ignore Rules in a Python CLI",{"path":2493,"title":2494},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fwriting-files-atomically-in-python-clis","Writing Files Atomically in Python CLIs",{"path":2496,"title":2497},"\u002Fcli-runtime-systems-integration","CLI Runtime & Systems Integration for Python",{"path":2499,"title":2500},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Fcaching-http-responses-on-disk-in-a-cli","Caching HTTP Responses on Disk in a Python CLI",{"path":2502,"title":2503},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis","Local State and SQLite in Python CLIs",{"path":2505,"title":2506},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Fmigrating-a-cli-sqlite-schema","Migrating a CLI’s SQLite Schema Between Releases",{"path":2508,"title":2509},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Frecording-and-querying-cli-run-history","Recording and Querying CLI Run History in SQLite",{"path":2511,"title":2512},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Fstoring-cli-state-in-sqlite","Storing CLI State in SQLite with a Small Repository Class",{"path":2514,"title":2515},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fbuilding-a-watch-mode-with-watchfiles","Building a Watch Mode with watchfiles in Python",{"path":2517,"title":2518},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fhandling-sigterm-and-graceful-shutdown","Handling SIGTERM and Graceful Shutdown in CLIs",{"path":2520,"title":2521},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fhealth-checks-and-heartbeats-for-long-running-clis","Health Checks and Heartbeats for Long-Running CLIs",{"path":2523,"title":2524},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis","Long-Running and Watch-Mode Python CLIs",{"path":2526,"title":2527},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Freloading-config-on-sighup","Reloading Configuration on SIGHUP in a Python CLI",{"path":2529,"title":2530},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Frunning-a-cli-as-a-systemd-service","Running a Python CLI as a systemd Service",{"path":2532,"title":2533},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Frunning-a-cli-on-a-schedule-with-cron-and-systemd","Running a Python CLI on a Schedule with cron and systemd",{"path":2535,"title":2536},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fsending-cli-logs-to-journald-and-syslog","Sending Python CLI Logs to journald and syslog",{"path":2538,"title":2539},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Favoiding-shell-injection-in-python-clis","Avoiding Shell Injection in Python CLIs",{"path":2541,"title":2542},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fcalling-external-commands-safely-with-subprocess","Calling External Commands Safely with subprocess",{"path":2544,"title":2545},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fhandling-subprocess-timeouts-and-exit-codes","Handling Subprocess Timeouts and Exit Codes",{"path":2547,"title":2548},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis","Running Subprocesses from Python CLIs",{"path":2550,"title":2551},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Flaunching-the-users-editor-from-a-cli","Launching the User’s Editor from a Python CLI",{"path":2169,"title":5},{"path":2554,"title":2555},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fstreaming-subprocess-output-in-real-time","Streaming Subprocess Output in Real Time",{"path":2557,"title":2558},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fwrapping-git-and-other-tools-from-a-python-cli","Wrapping git and Other Tools from a Python CLI",{"path":2560,"title":2561},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis","Secrets and Credentials in Python CLIs",{"path":2563,"title":2564},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fprompting-for-passwords-securely","Prompting for Passwords Securely in Python CLIs",{"path":2566,"title":2567},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Freading-secrets-from-env-and-files","Reading Secrets from Env Vars and Files in CLIs",{"path":2569,"title":2570},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fredacting-secrets-from-cli-output-and-logs","Redacting Secrets from CLI Output and Logs",{"path":2572,"title":2573},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Frefreshing-expired-tokens-automatically","Refreshing Expired Tokens Automatically in a Python CLI",{"path":2575,"title":2576},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fstoring-tokens-with-keyring","Storing CLI Tokens Securely with keyring",{"path":2578,"title":2579},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fsupporting-multiple-profiles-and-accounts","Supporting Multiple Profiles and Accounts in CLIs",{"path":2581,"title":2582},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fchecking-pypi-for-a-newer-version","Checking PyPI for a Newer Version of Your Python CLI",{"path":2584,"title":2585},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis","Update Checks, Upgrade Commands and Telemetry for Python CLIs",{"path":2587,"title":2588},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fopt-in-usage-telemetry-for-python-clis","Opt-In Usage Telemetry for Python CLIs Done Responsibly",{"path":2590,"title":2591},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fself-upgrading-a-cli-installed-with-pipx-or-uv","Self-Upgrading a Python CLI Installed with pipx or uv",{"path":2593,"title":2594},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fshowing-non-blocking-update-notices","Showing Non-Blocking Update Notices in a Python CLI",{"path":2596,"title":2597},"\u002F","Python CLI Toolcraft",{"path":2599,"title":2600},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fbuilding-a-cli-with-cleo","Building a Python CLI with Cleo Command Classes",{"path":2602,"title":2603},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fbuilding-a-cli-with-cyclopts","Building a Type-Hinted Python CLI with Cyclopts",{"path":2605,"title":2606},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks","Beyond Click and Typer: Alternative Python CLI Frameworks",{"path":2608,"title":2609},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fquick-clis-from-functions-with-python-fire","Quick CLIs from Functions with Python Fire",{"path":2611,"title":2612},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fusage-string-driven-clis-with-docopt-ng","Usage-String Driven Python CLIs with docopt-ng",{"path":2614,"title":2615},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Favoiding-import-time-side-effects","Avoiding Import-Time Side Effects in a Python CLI",{"path":2617,"title":2618},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fcaching-expensive-work-between-cli-runs","Caching Expensive Work Between Python CLI Runs",{"path":2620,"title":2621},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fguarding-startup-with-import-tests","Guarding CLI Startup with Import Tests",{"path":2623,"title":2624},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading","CLI Startup Performance and Lazy Loading",{"path":2626,"title":2627},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Flazy-loading-subcommands-for-faster-startup","Lazy Loading Subcommands for Faster Startup",{"path":2629,"title":2630},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fprofiling-python-cli-startup-time","Profiling Python CLI Startup Time",{"path":2632,"title":2633},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Freducing-cli-dependency-weight","Reducing CLI Dependency Weight",{"path":2635,"title":2636},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargparse-subparsers-for-subcommands","argparse Subparsers for Subcommands",{"path":2638,"title":2639},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargparse-vs-click-vs-typer-comparison","argparse vs Click vs Typer Compared",{"path":2641,"title":2642},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargument-groups-and-help-formatting-in-argparse","Argument Groups and Help Formatting in argparse",{"path":2644,"title":2645},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse","Command-Line Parsing with argparse",{"path":2647,"title":2648},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fmigrating-from-argparse-to-typer","Migrating from argparse to Typer",{"path":2650,"title":2651},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fmutually-exclusive-options-in-argparse","Mutually Exclusive Options in argparse",{"path":2653,"title":2654},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Freading-arguments-from-files-with-fromfile-prefix-chars","Reading Arguments from Files with argparse’s fromfile_prefix_chars",{"path":2656,"title":2657},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fwriting-custom-argparse-actions","Writing Custom argparse Actions in Python",{"path":2659,"title":2660},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fadding-dry-run-and-confirmation-to-destructive-commands","Adding Dry-Run and Confirmation to Destructive Commands",{"path":2662,"title":2663},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fdesigning-idempotent-commands","Designing Idempotent Commands in a Python CLI",{"path":2665,"title":2666},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Ffollowing-posix-and-gnu-argument-conventions","Following POSIX and GNU Argument Conventions in Python",{"path":2668,"title":2669},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fglobal-options-vs-per-command-options","Global Options vs Per-Command Options in Python CLIs",{"path":2671,"title":2672},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions","Designing CLI Interfaces and Conventions in Python",{"path":2674,"title":2675},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fnaming-commands-and-flags-consistently","Naming Commands and Flags Consistently in Python CLIs",{"path":2677,"title":2678},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fpositional-arguments-vs-options","Positional Arguments vs Options: Designing a CLI Signature",{"path":2680,"title":2681},"\u002Fmodern-python-cli-frameworks-architecture","Python CLI Frameworks and Architecture",{"path":2683,"title":2684},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fdiscovering-plugins-with-entry-points","Discovering Plugins with Entry Points in Python CLIs",{"path":2686,"title":2687},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fhook-based-plugins-with-pluggy","Hook-Based Plugins for Python CLIs with pluggy",{"path":2689,"title":2690},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis","Plugin Architectures for Extensible CLIs",{"path":2692,"title":2693},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fisolating-plugin-failures","Isolating Plugin Failures in an Extensible Python CLI",{"path":2695,"title":2696},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Ftesting-plugins-against-the-host-cli","Testing Plugins Against the Host CLI",{"path":2698,"title":2699},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fversioning-a-plugin-api","Versioning a Plugin API for a Python CLI",{"path":2701,"title":2702},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fwriting-a-plugin-for-an-existing-cli","Writing a Plugin for an Existing CLI",{"path":2704,"title":2705},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fbest-practices-for-python-cli-entry-points","Best practices for Python CLI entry points",{"path":2707,"title":2708},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fdependency-injection-patterns-for-cli-commands","Dependency Injection Patterns for CLI Commands",{"path":2710,"title":2711},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fhow-to-structure-a-large-python-cli-project","Structuring a Large Python CLI Project",{"path":2713,"title":2714},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis","Structuring Multi-Command Python CLIs",{"path":2716,"title":2717},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Foffering-a-python-api-alongside-your-cli","Offering a Python API Alongside Your CLI",{"path":2719,"title":2720},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fregistering-commands-from-modules-automatically","Registering CLI Commands from Modules Automatically",{"path":2722,"title":2723},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fsharing-common-options-across-commands","Sharing Common Options Across Python CLI Commands",{"path":2725,"title":2726},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fsharing-state-with-click-context-objects","Sharing State with Click Context Objects",{"path":2728,"title":2729},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fend-to-end-testing-an-installed-cli","End-to-End Testing an Installed Python CLI",{"path":2731,"title":2732},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications","Testing Python CLI Applications",{"path":2734,"title":2735},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fmeasuring-cli-test-coverage","Measuring CLI Test Coverage",{"path":2737,"title":2738},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fmocking-filesystem-and-network-in-cli-tests","Mocking the Filesystem and Network in CLI Tests",{"path":2740,"title":2741},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fproperty-based-testing-cli-arguments-with-hypothesis","Property-Based Testing CLI Arguments with Hypothesis",{"path":2743,"title":2744},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fsnapshot-testing-cli-output","Snapshot Testing CLI Output",{"path":2746,"title":2747},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-click-commands-with-clirunner","Testing Click Commands with CliRunner",{"path":2749,"title":2750},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-commands-that-call-subprocesses","Testing Python CLI Commands That Call Subprocesses",{"path":2752,"title":2753},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-interactive-prompts-and-stdin","Testing Interactive Prompts and stdin",{"path":2755,"title":2756},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftranscript-tests-for-cli-commands","Transcript Tests for Python CLI Commands",{"path":2758,"title":2759},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fbuilding-a-cli-with-subcommands-in-click","Building a CLI with subcommands in Click",{"path":2761,"title":2762},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fbuilding-dynamic-commands-in-click","Building Dynamic Commands in Click",{"path":2764,"title":2765},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fchoices-and-enums-in-typer-and-click","Choices and Enums in Typer and Click Options",{"path":2767,"title":2768},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fclick-option-callbacks-and-eager-options","Click Option Callbacks and Eager Options Explained",{"path":2770,"title":2771},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fconverting-a-click-app-to-typer","Converting a Click App to Typer",{"path":2773,"title":2774},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each","Typer vs Click: When to Use Each",{"path":2776,"title":2777},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Frich-markup-and-help-panels-in-typer","Rich Markup and Help Panels in Typer",{"path":2779,"title":2780},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Ftyper-callback-functions-explained","Typer callback functions explained",{"path":2782,"title":2783},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fusing-annotated-options-in-typer","Using Annotated Options in Typer",{"path":2785,"title":2786},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fautomating-releases-from-git-tags","Automating Python CLI Releases from Git Tags",{"path":2788,"title":2789},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fcaching-uv-dependencies-in-ci","Caching uv Dependencies in CI for Python CLIs",{"path":2791,"title":2792},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis","CI\u002FCD Pipelines for Python CLIs",{"path":2794,"title":2795},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fpublishing-a-cli-docker-image-from-ci","Publishing a Python CLI as a Docker Image from CI",{"path":2797,"title":2798},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fpublishing-to-pypi-with-trusted-publishing","Publishing a CLI to PyPI with Trusted Publishing",{"path":2800,"title":2801},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Frunning-cli-tests-on-windows-and-macos-runners","Running Python CLI Tests on Windows and macOS Runners",{"path":2803,"title":2804},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fsmoke-testing-the-built-wheel-in-ci","Smoke-Testing the Built Wheel of a Python CLI in CI",{"path":2806,"title":2807},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Ftesting-a-cli-across-python-versions-with-github-actions","Testing a CLI Across Python Versions in GitHub Actions",{"path":2809,"title":2810},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fbuilding-a-cookiecutter-template-for-typer-clis","Building a Cookiecutter Template for Typer CLIs",{"path":2812,"title":2813},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fcopier-vs-cookiecutter-for-cli-templates","Copier vs Cookiecutter for CLI Templates",{"path":2815,"title":2816},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter","CLI Project Scaffolding with Cookiecutter",{"path":2818,"title":2819},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fpost-generation-hooks-in-cli-templates","Post-Generation Hooks in Python CLI Templates",{"path":2821,"title":2822},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Ftemplate-variables-and-conditional-files","Template Variables and Conditional Files in CLI Templates",{"path":2824,"title":2825},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Ftesting-a-project-template-with-pytest","Testing a CLI Project Template with pytest",{"path":2827,"title":2828},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fupdating-generated-projects-with-copier-update","Updating Generated CLI Projects with copier update",{"path":2830,"title":2831},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fbuilding-cross-platform-release-binaries-in-ci","Building Cross-Platform Release Binaries in CI",{"path":2833,"title":2834},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fbundling-a-python-cli-with-pyinstaller","Bundling a Python CLI with PyInstaller",{"path":2836,"title":2837},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fcode-signing-and-notarizing-cli-binaries","Code Signing and Notarizing Python CLI Binaries",{"path":2839,"title":2840},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fhomebrew-and-scoop-packaging-for-python-clis","Homebrew and Scoop Packaging for Python CLIs",{"path":2842,"title":2843},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries","Distributing CLIs as Standalone Binaries",{"path":2845,"title":2846},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fnuitka-vs-pyinstaller-for-python-clis","Nuitka vs PyInstaller for Python CLI Binaries",{"path":2848,"title":2849},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Freducing-pyinstaller-binary-size","Reducing the Size of a PyInstaller CLI Binary",{"path":2851,"title":2852},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fshipping-a-cli-as-a-zipapp-with-shiv","Shipping a CLI as a Zipapp with shiv",{"path":2854,"title":2855},"\u002Fproject-setup-dependency-management","Project Setup & Dependency Management",{"path":2857,"title":2858},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Fconfiguring-ruff-for-a-cli-project","Configuring Ruff for a Python CLI Project",{"path":2860,"title":2861},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Fenforcing-import-boundaries-in-a-cli-codebase","Enforcing Import Boundaries in a Python CLI Codebase",{"path":2863,"title":2864},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Ffinding-unused-code-and-dependencies-with-vulture-and-deptry","Finding Unused Code and Dependencies with vulture and deptry",{"path":2866,"title":2867},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code","Linting and Type-Checking Python CLI Code",{"path":2869,"title":2870},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Frunning-pyright-in-strict-mode-on-a-cli","Running Pyright in Strict Mode on a Python CLI",{"path":2872,"title":2873},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Frunning-ruff-and-mypy-in-ci-with-annotations","Running Ruff and mypy in CI with Inline Annotations",{"path":2875,"title":2876},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Ftype-checking-click-and-typer-code-with-mypy","Type-Checking Click and Typer Code with mypy",{"path":2878,"title":2879},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fautomating-changelogs-with-conventional-commits","Automating Changelogs with Conventional Commits",{"path":2881,"title":2882},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fbumping-versions-consistently-across-a-cli-project","Bumping Versions Consistently Across a Python CLI Project",{"path":2884,"title":2885},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fderiving-versions-from-git-tags-with-hatch-vcs","Deriving CLI Versions from Git Tags with hatch-vcs",{"path":2887,"title":2888},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fexposing-version-info-and-build-metadata","Exposing Version Info and Build Metadata",{"path":2890,"title":2891},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs","Managing CLI Versioning & Changelogs",{"path":2893,"title":2894},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fsemantic-versioning-policy-for-cli-tools","A Semantic Versioning Policy for CLI Tools",{"path":2896,"title":2897},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fshipping-pre-releases-and-release-candidates","Shipping Pre-Releases and Release Candidates of a Python CLI",{"path":2899,"title":2900},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fbuilding-wheels-and-sdists-for-python-clis","Building Wheels and sdists for Python CLIs",{"path":2902,"title":2903},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fbundling-data-files-with-importlib-resources","Bundling Data Files with importlib.resources in CLIs",{"path":2905,"title":2906},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fchoosing-a-build-backend-for-a-python-cli","Choosing a Build Backend for a Python CLI",{"path":2908,"title":2909},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution","Packaging Python CLIs for Distribution",{"path":2911,"title":2912},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Finstalling-and-distributing-clis-with-pipx","Installing and Distributing CLIs with pipx",{"path":2914,"title":2915},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Foptional-dependencies-and-extras-for-clis","Optional Dependencies and Extras for Python CLIs",{"path":2917,"title":2918},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fpublishing-a-python-cli-to-pypi","Publishing a Python CLI to PyPI",{"path":2920,"title":2921},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fwriting-pyproject-toml-metadata-for-a-cli","Writing pyproject.toml Metadata for a Python CLI",{"path":2923,"title":2924},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fdynamic-versioning-with-poetry-plugins","Dynamic Versioning for a Poetry CLI from Git Tags",{"path":2926,"title":2927},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development","Poetry Workflows for CLI Development",{"path":2929,"title":2930},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fmanaging-poetry-lock-files-for-cli-tools","Managing Poetry Lock Files for CLI Tools",{"path":2932,"title":2933},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fmigrating-a-cli-from-poetry-to-uv","Migrating a Python CLI from Poetry to uv",{"path":2935,"title":2936},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpoetry-dependency-groups-for-cli-tooling","Poetry Dependency Groups for CLI Tooling",{"path":2938,"title":2939},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpoetry-entry-points-and-scripts-for-clis","Poetry Entry Points and Scripts for CLIs",{"path":2941,"title":2942},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpublishing-a-cli-with-poetry","Building and Publishing a Python CLI with Poetry",{"path":2944,"title":2945},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects","Pre-commit Hooks for CLI Projects",{"path":2947,"title":2948},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fkeeping-hook-versions-current-with-autoupdate","Keeping pre-commit Hook Versions Current with autoupdate",{"path":2950,"title":2951},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Frunning-pre-commit-in-ci","Running pre-commit in CI for a Python CLI Repository",{"path":2953,"title":2954},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fsetting-up-pre-commit-for-python-cli-repos","Setting up pre-commit for Python CLI repos",{"path":2956,"title":2957},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fshipping-your-cli-as-a-pre-commit-hook","Shipping Your Python CLI as a pre-commit Hook",{"path":2959,"title":2960},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fspeeding-up-slow-pre-commit-hooks","Speeding Up Slow pre-commit Hooks in a CLI Repository",{"path":2962,"title":2963},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fwriting-local-pre-commit-hooks-in-python","Writing Local pre-commit Hooks in Python",{"path":2965,"title":2966},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fauditing-dependencies-with-pip-audit","Auditing a Python CLI’s Dependencies with pip-audit",{"path":2968,"title":2969},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fgenerating-an-sbom-for-a-python-cli","Generating an SBOM for a Python CLI Release",{"path":2971,"title":2972},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain","Securing the Supply Chain of a Python CLI",{"path":2974,"title":2975},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fpinning-dependencies-with-hashes","Pinning a Python CLI’s Dependencies with Hashes",{"path":2977,"title":2978},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fpublishing-attestations-and-verifying-releases","Publishing Attestations and Verifying Python CLI Releases",{"path":2980,"title":2981},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fbuilding-and-publishing-a-cli-with-uv","Building and Publishing a Python CLI with uv",{"path":2983,"title":2984},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management","uv for Python CLI Dependency Management",{"path":2986,"title":2987},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Flocking-and-syncing-cli-dependencies-with-uv","Locking and Syncing a Python CLI’s Dependencies with uv",{"path":2989,"title":2990},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Frunning-one-off-cli-scripts-with-uv-run","Running One-Off CLI Scripts with uv run and PEP 723",{"path":2992,"title":2993},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fusing-private-package-indexes-with-uv","Using Private Package Indexes with uv for Internal CLIs",{"path":2995,"title":2996},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-init-vs-poetry-init-for-cli-tools","uv init vs poetry init for CLI tools",{"path":2998,"title":2999},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-tool-install-vs-pipx-for-clis","uv tool install vs pipx for CLIs",{"path":3001,"title":3002},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-workspaces-for-multi-package-clis","uv Workspaces for Multi-Package Python CLIs",{"path":3004,"title":3005},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fdebugging-wrong-python-and-wrong-venv-problems","Debugging Wrong-Python and Wrong-Venv Problems in CLIs",{"path":3007,"title":3008},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fexternally-managed-environments-and-pep-668","PEP 668 and Python CLIs: the externally-managed-environment Error",{"path":3010,"title":3011},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices","Python CLI Env Isolation Best Practices",{"path":3013,"title":3014},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fisolating-cli-tools-from-project-dependencies","Isolating CLI Tools from Your Project’s Dependencies",{"path":3016,"title":3017},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fmanaging-virtual-environments-for-cross-platform-clis","Managing Python CLI Virtual Environments",{"path":3019,"title":3020},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fpinning-the-python-version-for-a-cli","Pinning the Python Version for a CLI",{"path":3022,"title":3023},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fsupporting-multiple-python-versions-with-nox","Supporting Multiple Python Versions with nox",1790967540101]