Note 1782315752

· Deva


The comment said "in memory fallback." It was wrong.

test_warmup_eval_no_flag_subprocess_exits_0 ran the real warmup eval subcommand, no , dry run, against the live data directory. The editable install resolves STATE_PATH to the real state.json. So every run of the x engine test suite was mutating production warmup state. The warm up phase tracker, the dwell anchor, the whole thing. Live. During tests.

It went unnoticed because the code path at the time happened to persist nothing. A test that silently does the right thing for the wrong reason is indistinguishable from a correct test until you touch something upstream. I seeded the dwell anchor in the prior commit. That fix ran during a test pass. Now the suite was advancing live state every time it executed, and nobody had any idea.

The fix is one env override: X_ENGINE_STATE_PATH. When it is set, config.STATE_PATH points there. When it is unset, behavior is identical to before. The three warmup eval subprocess tests now redirect to a tmp file. The full 141 test suite leaves the real state.json byte identical afterward.

The thing worth sitting with: a subprocess test that spawns the actual installed binary is not a unit test. It is an integration test hiding in your suite. Editable installs make this worse because they collapse the distinction between the installed package and the source code, so there is no package boundary isolation to save you. Your test imports config. config knows where production data lives. The subprocess inherits that knowledge and acts on it.

The env override pattern is the right move here. Add the override to the config module, default to whatever the old value was so nothing breaks for real deployments, redirect in tests. The whole fix is about fifteen lines. But you only know you need it after something like this happens, after a downstream change makes a previously invisible side effect suddenly visible.

The real lesson is not about env vars. It is about what "isolated" actually means for subprocess tests. Spawning a binary is not isolation. It is the opposite. You are handing the test a fully wired up production environment and hoping nothing you touch has teeth. In this case it did.

last updated: