Commit Graph
1 Commits
Author SHA1 Message Date
trs-80 0d99e5595d fix(tests): isolate persisted settings from developer state (#3449)
## Description

Clearing `HEADROOM_*` environment variables still leaves the developer's
persisted settings active. Tests that construct the proxy or use the
settings store can read `~/.headroom/settings.json`, and settings
endpoint tests can write back to it.

Give each test a temporary settings store through the existing
`HEADROOM_SETTINGS_PATH` override. Explicit path-precedence tests can
opt out or supply their own path.

Extracts the settings-isolation portion of #2524, including the CLI
precedence-test opt-outs, with focused regression coverage.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Set a fresh per-test settings path in `tests/conftest.py` after the
shared fixture scrubs developer environment variables.
- Keep both CLI settings-precedence tests in
`tests/test_cli_proxy_env.py` pointed at their deliberately seeded
temporary workspace.
- Add `tests/test_settings_isolation.py`: a regression that reads and
writes through the real settings store, verifies an empty initial state
for each case, and preserves a simulated developer settings file
byte-for-byte.

## Testing

- [x] Unit tests pass (`pytest`) — 368 targeted cases across settings,
paths, proxy endpoints, CLI configuration, and the related frozen-prefix
regression
- [x] Linting passes (`ruff check .`) — for the three changed files
- [x] Type checking passes (`mypy headroom`) — via the full-package mypy
commit hook
- [x] New tests added for new functionality
- [x] Manual testing performed — new regression observed failing before
the fixture change

### Test Output

```text
python -m pytest tests/test_settings_isolation.py tests/test_proxy_loopback_gating.py tests/test_proxy/test_settings_store.py tests/test_proxy_settings_endpoints.py tests/test_paths.py tests/test_proxy/test_settings_fresh_process_precedence.py -q --tb=short
233 passed

python -m pytest tests/test_cli_proxy_env.py tests/test_cli_proxy_improvements.py tests/test_paths_backward_compat.py -k 'not TestMissingProxyDepsError' -q --tb=short
134 passed, 2 deselected

python -m pytest tests/test_proxy_anthropic_cache_stability.py -k memory_context_avoids_system_mutation_when_prefix_frozen -q --tb=short
1 passed, 27 deselected

python -m pytest tests/test_settings_isolation.py tests/test_proxy_loopback_gating.py::test_settings_post_trusted_gateway_client_same_origin_allowed tests/test_cli_proxy_env.py::TestSettingsFileToEnv -q --tb=short
6 passed

ruff check tests/conftest.py tests/test_settings_isolation.py tests/test_cli_proxy_env.py
All checks passed!

ruff format --check tests/conftest.py tests/test_settings_isolation.py tests/test_cli_proxy_env.py
3 files already formatted

git diff --check
```

The broader CLI run encountered two dependency-check failures in
`TestMissingProxyDepsError`: those tests mock `builtins.__import__`,
while the dependency check uses `importlib.import_module`. The
missing-FastAPI case also fails in the unchanged checkout, so both are
pre-existing and are excluded from the targeted passing run above. An
existing Starlette TestClient deprecation warning remains.

## Real Behavior Proof

- Environment: macOS, Python 3.12, existing native extension and
installed test dependencies; clean worktree based on `e67b3c8a`.
- Exact command / steps: seed a simulated developer home with
`{"target_ratio": 0.9}`, run `python -m pytest
tests/test_settings_isolation.py -q` against the unchanged fixture, then
apply the `tests/conftest.py` change and rerun `python -m pytest
tests/test_settings_isolation.py
tests/test_proxy_loopback_gating.py::test_settings_post_trusted_gateway_client_same_origin_allowed
tests/test_cli_proxy_env.py::TestSettingsFileToEnv -q --tb=short`,
followed by the wider targeted runs listed under Test Output.
- Observed result: before the fixture change, both regression cases fail
because `settings_store.load()` returns the developer's persisted
`target_ratio` instead of an empty mapping. After the change, both cases
start empty, save and reload their own settings, and leave the simulated
developer file byte-for-byte unchanged; the formerly failing settings
endpoint test passes and the CLI precedence tests still read their own
seeded workspace (6 passed).
- Not tested: full repository suite, other operating systems, or live
providers. Coverage is limited to persisted settings; tool-home and
network isolation are separate follow-ups.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. Test-only change under `tests/`; no
shipped code path is touched.
- Minimum rollout channel: N/A — nothing ships to users from this
change.
- Stable/default behavior changed: No. Production behavior is unchanged;
only the pytest fixture environment gains a per-test
`HEADROOM_SETTINGS_PATH`.
- Kill switch / disable path: N/A — no runtime feature to disable. Tests
that need a specific path opt out of the fixture or set their own
`HEADROOM_SETTINGS_PATH`.
- Unsafe override required: No.
- Qualification impact: None. No new qualification gates; the suite
continues to run unchanged apart from stronger isolation.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] My changes generate no new warnings
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Test-only change: no documentation, dependency, or public configuration
updates apply. The two `TestMissingProxyDepsError` failures noted above
are pre-existing and unrelated to this fixture change.
2026-09-11 20:41:47 -07:00