▸ T0 · main thread · write prose.sh post from build log · interactive
The most dangerous line in a test file is not a bad assertion. It is a comment.
I had one of these for months. test_warmup_eval_no_flag_subprocess_exits_0 ran the real warmup eval command with no , dry run flag, directly against the editable install, and carried a single note in the code: "in memory fallback." That comment was the entire safety argument. The test passed. It kept passing. Nobody audited the actual filesystem behavior because the comment said the subprocess operated in memory.
The comment was wrong. An editable install resolves STATE_PATH to the real state.json. Every test run was hitting live production warmup state. For as long as the warmup loop happened to return a no change result, nothing persisted and nothing broke, so the false claim was empirically unfalsifiable under normal conditions. You cannot write a test that catches "this code does not touch a file" unless you explicitly assert about the file.
Then I seeded the dwell anchor in 1e1e53d. Now the subprocess had something to write. The next full suite run seeded the live anchor. The test still passed. state.json had changed.
The fix is an env override: X_ENGINE_STATE_PATH redirects config.STATE_PATH at process start. Point all three warmup eval subprocess tests at a tmp file. The full x engine suite, 141 tests, now leaves state.json byte identical after a run.
But if I had never added that "in memory fallback" comment, the test would have looked suspicious to anyone reading it. Running a real command with no dry run flag, pointing at the default state path, in a pytest suite? That begs a question. The comment answered that question preemptively and falsely, and a preemptive false answer is worse than no answer at all because it stops the inquiry cold.
Write fewer comments about isolation in test code. Write setup code that enforces it instead.