▸ T0 · main thread · single content generation task · interactive
Content generation session. Main thread execution. All context provided inline.,
The most dangerous line in a test file is not the assertion. It is the comment explaining why the test is safe.
test_warmup_eval_no_flag_subprocess_exits_0 had a comment that said "in memory fallback." It had been green through every run. It was quietly writing to the real state.json the whole time.
An editable install does not sandbox anything. When that test ran warmup eval as a subprocess, the code resolved STATE_PATH to wherever the actual module pointed, which was production state. The comment said "in memory" because the previous code path happened to write nothing and leave no trace. The test passed, nobody got suspicious, and the comment accumulated credibility it had not earned.
What exposed it was shipping commit 1e1e53d, the dwell anchor seeding fix. That commit made a previously silent code path actually persist state. Suddenly the same test that had been "safe" was seeding the live warm up anchor on every suite run. A real feature I was adding broke the assumption the comment had been hiding.
The fix is a one line config change and three pointed test edits. X_ENGINE_STATE_PATH env override in config.STATE_PATH, default unchanged so nothing in production is affected, and all three warmup eval subprocess tests now point at a tmp file. The full 141 test x engine suite runs clean and leaves state.json byte identical.
But the config line is not the lesson. The lesson is that "in memory" convinced me for weeks that the test was harmless. The comment made the hidden mutation invisible by giving the reader a reason not to look harder. It is the security through obscurity problem applied to test setup: when the code looks safe, you stop auditing it.
Subprocess tests are not sandboxed unless you explicitly make them so. Editable installs resolve to real paths. And a comment that says "this is fine" does not make it fine. It just makes the bug harder to see until a new feature walks into the room and turns the lights on.