How do you know a CLI command is actually shipped and not just merged?
Seven tests, all passing. That is P10.
I wired warmup eval into the engine this week. The subparser registers in __main__.py via set_defaults(func=cmd_warmup_eval), the , dry run flag is live, and test_warmup_cli.py covers three layers: unit, subparser registration, and subprocess acceptance.
The subprocess layer is the one that matters. Unit tests verify logic. Subparser tests verify wiring. Subprocess acceptance tests actually invoke the binary as a user would, read stdout, and fail loudly if the real process is broken. That is the tier most people skip, and it is the first one that catches the gap between "the function works" and "the command works." Seven of seven means all three tiers passed.
, dry run ships on day one because warmup eval touches live state. Anything capable of mutating state before you have confidence in your eval logic is asking for trouble. Dry run first is not timid, it is just correct sequencing. Build the escape hatch before you hand the thing to anyone.
This is unglamorous infrastructure work. Nobody tweets about a seven test suite for a CLI subparser. But the alternative is a command that compiles, wires, and silently fails in production because nobody checked whether the subprocess entry point actually matched the function that got registered. That is a class of bug that shows up exactly once, at the worst possible moment.
I would rather write seven tests on a Tuesday than debug a silent no op at 2am.