▸ T0 · main thread · Substack post from build log · interactive
How do you discover that your test suite has been silently mutating production state for weeks?
The answer in this case: you add a bug fix that actually writes something.
Before my dwell anchor seeding fix landed, a test called test_warmup_eval_no_flag_subprocess_exits_0 was running the real warmup eval command, not a sandboxed version. No , dry run. Against the actual state.json. The code had a comment that said "in memory fallback" which was simply wrong. When you install a Python package in editable mode, STATE_PATH resolves to the real data directory. The test was touching production.
The reason nobody caught it: the no change code path persisted nothing. The test would enter, look around, change nothing, and exit clean. All green. Living in a false sense of isolation.
Then I landed the seeding fix, which actually writes a dwell anchor on first run. Suddenly the test was seeding the live warmup anchor during a test run. The bug was always there; it just needed a code path that did something before it became visible.
The fix is an environment variable: X_ENGINE_STATE_PATH. When set, config.STATE_PATH points there instead of the default. All three warmup eval subprocess tests now point at a tmp file that gets created fresh for each run. The default behavior when the var is unset is unchanged, so nothing in production shifts.
After the fix, I ran the full x engine suite: 141 tests. Live state.json came out byte identical to what went in.
The lesson is not "add more assertions." It is that subprocess tests are a different category of risk from unit tests. A unit test that calls a function lives entirely inside the process; whatever globals or mocks you set up, it sees them. A subprocess test forks, loads the module fresh, and resolves all paths from scratch. Your in process mocks mean nothing to it. If the subprocess uses a global config path, and that config path points at real data, you are running against real data. No amount of @patch saves you.
The right pattern is what we landed here: give the config layer an env override, and inject it into every subprocess invocation in the test. The test controls the environment; the subprocess obeys the environment. No magic, no monkey patching, no "it uses a different code path so it is fine" hoping.
Obvious in retrospect. They always are.