mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description Clearing `HEADROOM_*` environment variables still leaves the developer's persisted settings active. Tests that construct the proxy or use the settings store can read `~/.headroom/settings.json`, and settings endpoint tests can write back to it. Give each test a temporary settings store through the existing `HEADROOM_SETTINGS_PATH` override. Explicit path-precedence tests can opt out or supply their own path. Extracts the settings-isolation portion of #2524, including the CLI precedence-test opt-outs, with focused regression coverage. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Set a fresh per-test settings path in `tests/conftest.py` after the shared fixture scrubs developer environment variables. - Keep both CLI settings-precedence tests in `tests/test_cli_proxy_env.py` pointed at their deliberately seeded temporary workspace. - Add `tests/test_settings_isolation.py`: a regression that reads and writes through the real settings store, verifies an empty initial state for each case, and preserves a simulated developer settings file byte-for-byte. ## Testing - [x] Unit tests pass (`pytest`) — 368 targeted cases across settings, paths, proxy endpoints, CLI configuration, and the related frozen-prefix regression - [x] Linting passes (`ruff check .`) — for the three changed files - [x] Type checking passes (`mypy headroom`) — via the full-package mypy commit hook - [x] New tests added for new functionality - [x] Manual testing performed — new regression observed failing before the fixture change ### Test Output ```text python -m pytest tests/test_settings_isolation.py tests/test_proxy_loopback_gating.py tests/test_proxy/test_settings_store.py tests/test_proxy_settings_endpoints.py tests/test_paths.py tests/test_proxy/test_settings_fresh_process_precedence.py -q --tb=short 233 passed python -m pytest tests/test_cli_proxy_env.py tests/test_cli_proxy_improvements.py tests/test_paths_backward_compat.py -k 'not TestMissingProxyDepsError' -q --tb=short 134 passed, 2 deselected python -m pytest tests/test_proxy_anthropic_cache_stability.py -k memory_context_avoids_system_mutation_when_prefix_frozen -q --tb=short 1 passed, 27 deselected python -m pytest tests/test_settings_isolation.py tests/test_proxy_loopback_gating.py::test_settings_post_trusted_gateway_client_same_origin_allowed tests/test_cli_proxy_env.py::TestSettingsFileToEnv -q --tb=short 6 passed ruff check tests/conftest.py tests/test_settings_isolation.py tests/test_cli_proxy_env.py All checks passed! ruff format --check tests/conftest.py tests/test_settings_isolation.py tests/test_cli_proxy_env.py 3 files already formatted git diff --check ``` The broader CLI run encountered two dependency-check failures in `TestMissingProxyDepsError`: those tests mock `builtins.__import__`, while the dependency check uses `importlib.import_module`. The missing-FastAPI case also fails in the unchanged checkout, so both are pre-existing and are excluded from the targeted passing run above. An existing Starlette TestClient deprecation warning remains. ## Real Behavior Proof - Environment: macOS, Python 3.12, existing native extension and installed test dependencies; clean worktree based on `e67b3c8a`. - Exact command / steps: seed a simulated developer home with `{"target_ratio": 0.9}`, run `python -m pytest tests/test_settings_isolation.py -q` against the unchanged fixture, then apply the `tests/conftest.py` change and rerun `python -m pytest tests/test_settings_isolation.py tests/test_proxy_loopback_gating.py::test_settings_post_trusted_gateway_client_same_origin_allowed tests/test_cli_proxy_env.py::TestSettingsFileToEnv -q --tb=short`, followed by the wider targeted runs listed under Test Output. - Observed result: before the fixture change, both regression cases fail because `settings_store.load()` returns the developer's persisted `target_ratio` instead of an empty mapping. After the change, both cases start empty, save and reload their own settings, and leave the simulated developer file byte-for-byte unchanged; the formerly failing settings endpoint test passes and the CLI precedence tests still read their own seeded workspace (6 passed). - Not tested: full repository suite, other operating systems, or live providers. Coverage is limited to persisted settings; tool-home and network isolation are separate follow-ups. ## Runtime Rollout Safety - Rollout-managed feature(s): None. Test-only change under `tests/`; no shipped code path is touched. - Minimum rollout channel: N/A — nothing ships to users from this change. - Stable/default behavior changed: No. Production behavior is unchanged; only the pytest fixture environment gains a per-test `HEADROOM_SETTINGS_PATH`. - Kill switch / disable path: N/A — no runtime feature to disable. Tests that need a specific path opt out of the fixture or set their own `HEADROOM_SETTINGS_PATH`. - Unsafe override required: No. - Qualification impact: None. No new qualification gates; the suite continues to run unchanged apart from stronger isolation. - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] My changes generate no new warnings - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Test-only change: no documentation, dependency, or public configuration updates apply. The two `TestMissingProxyDepsError` failures noted above are pre-existing and unrelated to this fixture change.
24 lines
947 B
Python
24 lines
947 B
Python
"""Shared test fixtures must keep persisted developer settings out of tests."""
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from headroom import settings_store
|
|
|
|
|
|
@pytest.mark.parametrize("target_ratio", [0.4, 0.6])
|
|
def test_settings_reads_and_writes_are_isolated(monkeypatch, tmp_path, target_ratio):
|
|
developer_home = tmp_path / "developer-home"
|
|
developer_settings = developer_home / ".headroom" / "settings.json"
|
|
developer_settings.parent.mkdir(parents=True)
|
|
original = '{"target_ratio": 0.9}\n'
|
|
developer_settings.write_text(original, encoding="utf-8")
|
|
monkeypatch.setattr(Path, "home", lambda: developer_home)
|
|
|
|
# Each test starts empty, regardless of developer settings or prior saves.
|
|
assert settings_store.load() == {}
|
|
settings_store.save({"target_ratio": target_ratio})
|
|
assert settings_store.load() == {"target_ratio": target_ratio}
|
|
assert developer_settings.read_text(encoding="utf-8") == original
|