▸ T0 · main thread · write Substack post from build log · interactive
Most test isolation failures don't fail. That's the problem. They pass, every run, quietly doing damage you find later in some completely different context. That's what happened in the x engine warmup eval tests: test_warmup_eval_no_flag_subprocess_exits_0 had a comment claiming "in memory fallback" but was resolving STATE_PATH to the real state.json through the editable install. 141 tests, all green. Live warm up state mutating on every test run, silently, correctly, leaving no trace.
I added an X_ENGINE_STATE_PATH env override to config.STATE_PATH so subprocess tests can redirect to a tmp file. The full suite now runs and leaves state.json byte identical. Simple fix. The interesting part is why it took this long to surface.
The previous code path never wrote anything on the no change branch, so the mutation was harmless. Every test run resolved to real production state and then did nothing with it, so nobody noticed. It took an actual state write (dwell anchor seeding from a previous commit) to expose the bug. Which means the test was only "safe" by accident. One feature addition away from corruption.
Subprocess tests are their own category of isolation problem. When you shell out inside a test, you leave your test framework's jurisdiction. The subprocess inherits the real environment, resolves real paths, finds real config. Your test runner has no authority over what happens inside that child process unless you explicitly hand it some. An editable install makes this worse because the subprocess finds your actual package, your actual data directories, and your actual defaults.
The instinct when you see "in memory fallback" in a comment is to trust it. I trusted it. But a comment is not a guarantee. The thing to actually verify is what the path resolves to at runtime, inside the subprocess, after the editable install wires everything up. That is a different question entirely.
The lesson is not "add more mocks." It is: any test that shells out to your actual package under an editable install should be treated as if it has production access until proven otherwise.