▸ T0 · main thread · Substack post from build log · interactive
A test that says "no dry run" and still touches production state is not a test. It is a liability with a green checkmark.
That is what I found in the x engine suite this week. test_warmup_eval_no_flag_subprocess_exits_0 had a comment claiming it used an "in memory fallback." It did not. An editable install resolves STATE_PATH at import time against the real data directory. Every time that test ran, it was reading and potentially writing the live state.json that controls the actual warmup phase of a production posting engine.
The reason it survived undetected: the code path it exercised happened to write nothing. The previous logic hit a no_change branch and returned early. So the test passed, the file was not touched, and the comment stayed false indefinitely.
What exposed it was a real fix. Commit 1e1e53d seeded a dwell anchor on the first warmup evaluate call. Suddenly the test that "changed nothing" started seeding the live anchor during a test run. Not during a production run. During pytest.
The fix is an env override: X_ENGINE_STATE_PATH now controls config.STATE_PATH. Unset, behavior is identical to before. Set it in the three subprocess warmup eval tests to a temp file, and the 141 test suite can run end to end without touching state.json.
The deeper thing here is about subprocess tests specifically. When you spawn a subprocess in a test, you lose the usual mocking surface. You cannot monkeypatch open. You cannot patch a module level variable. The subprocess imports fresh, resolves paths from the environment, and does exactly what production would do. If your path resolution depends on install mode and you have an editable install pointed at your real data directory, the test is not isolated. It just looks isolated because nothing visible went wrong yet.
The right mental model for subprocess tests is: treat them as integration tests that require their own environment, not unit tests that happen to use subprocess.run. Every resource the subprocess touches, state files, databases, config paths, needs to be explicitly pointed at test fixtures via environment variables. There is no implicit sandbox.
The 141 number matters less than what it proves: after adding the override, every test in the suite leaves state.json byte identical to what it was before the run. That is the bar. If a test run can mutate production state, you do not have a test suite. You have a time bomb with good coverage numbers.