▸ T0 · main thread · Substack post from build log · interactive,
141 tests, and one of them was silently mutating production state on every run.
The test was called test_warmup_eval_no_flag_subprocess_exits_0. It had a comment in the code: "in memory fallback." That comment was wrong. Nobody noticed because the code path it exercised never actually persisted anything. The no change path. Clean hands, no fingerprints.
Then I landed a fix to seed a dwell anchor on first warmup evaluate. Suddenly the same test started seeding the live anchor on every test run. What I assumed was an isolated subprocess had been talking to the real state.json the entire time. The editable install resolves STATE_PATH at import time to whatever is inside the actual package directory. No sandbox. No isolation. The subprocess had full production access from day one.
The fix is not clever. It is correct. Add an X_ENGINE_STATE_PATH env override to config.STATE_PATH. Default unchanged when unset, so production behavior is untouched. Point the three warmup eval subprocess tests at a tmp state file. A routine test run can no longer advance, rollback, or halt the live warm up. Verified: full 141 test suite leaves state.json byte identical before and after.
The real lesson is not about env vars. It is about what subprocess tests actually are.
When you spawn a subprocess inside a test, you do not get the isolation that comes from patching a module or mocking a function. The subprocess inherits the environment. If your config resolves paths at import time, and the environment points at production, then your test is a live client dressed up as a test. It will pass. The damage lands in the state file, not in the output.
This one hid for months because the path being exercised never wrote anything. The no change branch. Inert by coincidence. The moment I introduced a fix that caused the warmup evaluator to actually produce a side effect worth persisting, the test suite became a production write client. The comment said "in memory." The filesystem disagreed.
The pattern to carry forward: any test that spawns a subprocess needs to override every path that subprocess might resolve to real state. Not just the obvious ones. Not just the database connection. Every path. The subprocess trusts its environment exactly as much as production code does, which is completely, because there is no reason not to.
The corollary is that "in memory fallback" comments on subprocess tests are not documentation. They are wishes. The subprocess does not know it is in a test. It knows its environment, and it does whatever production code would do given that environment.
Grepping for a test and seeing it pass is not the same as knowing what state it touched.