mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(wrap): warn when reused proxy ignores compression settings (#4019)
## Description A healthy shared proxy is intentionally not restarted: doing so would interrupt other clients. Explicit startup compression settings must not be silently ignored when a wrapper reuses it. This implements the issue's warning-on-reuse alternative. Closes #3588. ## Type of Change - [x] Bug fix - [x] Documentation update ## Changes Made - Expose the effective tool exclusions alongside the existing effective minimum-token setting in `/health`. - Warn when explicitly supplied `HEADROOM_MIN_TOKENS` or `HEADROOM_EXCLUDE_TOOLS` differs from the running configuration. - Compare tool names case-insensitively and include built-in exclusions on both sides. Skip comparisons when older health responses lack the setting or the requested integer is invalid. - Preserve the existing mode warning and shared-proxy lifecycle. Document startup-only configuration and the restart/separate-port remedies. - Query the running settings for explicit compression environment variables even with `--no-proxy` and no explicit `HEADROOM_MODE`. ## Testing - [x] Focused unit tests: `python -m pytest -q tests/test_cli/test_wrap_bob.py` — **59 passed**. With current main's dashboard-license tests: **67 passed**. - [x] New consumer-visible warning regressions: **4 failed on unchanged baseline**; the additional no-start/MODE-unset cases produced **2 failures before the final fix**, passing afterwards. - [x] Changed-file Ruff and formatting checks; installed pre-commit hooks, including `mypy headroom` and Commitlint, passed. - [x] Manual testing performed. - [ ] Full-tree `ruff check .`: blocked locally by EXE002 on five unchanged OAuth2 plugin files checked out with executable filesystem permissions (Git tracks them as 100644). No unrelated plugin files or permissions changed. - [ ] Separate full dev-environment `mypy headroom`: found one `truthy-function` error in unchanged `headroom/integrations/langchain/chat_model.py:394`. The file is byte-identical to baseline (SHA256 `701a9fe8457907406a2b2810211c1534bc9dfcacf3d841ec4c27d56897209bc7`); the project's isolated pre-commit mypy environment passed. The mandatory pre-push `make ci-precheck` completed: workspace Rust fmt/clippy/tests, native extension build/import verification, Python gate **177 passed / 4 skipped**, and Commitlint all passed. The native model gate used `ORT_DYLIB_PATH` pointing to the installed CPU `libonnxruntime.so.1.26.0`; the cached Kompress parity suite matched **21 fixtures byte-for-byte**. Implicit runtime selection did not finish and was cancelled, not reported as a pass; explicit CPU runtime qualification completed without skipping the cached-model assertions. ### Test Output ```text [build_rust_extension] headroom._core build + install + verify: OK ================== 177 passed, 4 skipped, 1 warning in 7.29s =================== ✅ ci-precheck PASSED — safe to push. ``` ## Real Behavior Proof - Environment: Linux x86_64, CPython 3.13.14; original baseline `dea9092e19385f286b7f67a64f4d85de2708d756` and final fix rebased onto current main `1ad92f0cbc3dea27676e253588100f64a2af6994`; real CLI proxy and real health endpoint. - Exact command / steps: start the real proxy, then reuse it with `HEADROOM_MODE` unset and conflicting compression settings without starting/restarting it; commands below. ```sh HEADROOM_BEACON=off HEADROOM_MIN_TOKENS=120 HEADROOM_EXCLUDE_TOOLS='' \ headroom proxy --host 127.0.0.1 --port 18795 --stateless --no-optimize \ --no-cache --no-rate-limit --no-subscription-tracking --disable-kompress ``` - With `HEADROOM_MODE` unset, reuse it without starting/restarting a proxy: ```sh HEADROOM_MIN_TOKENS=2000 HEADROOM_EXCLUDE_TOOLS=Bash,Read,Grep,Glob \ python -c 'from headroom.cli.wrap import _ensure_proxy; print("RESULT", _ensure_proxy(18795, True))' ``` - Before the final no-start-path fix: only `RESULT (None, 18795)`; both explicit settings were silent. The original baseline was also silent. - Observed result: both explicit-setting warnings appeared after the fix; the same PID and running configuration remained unchanged. Output: ```text Warning: this session requested HEADROOM_MIN_TOKENS=2000 (running: 120); HEADROOM_EXCLUDE_TOOLS=['bash', 'glob', 'grep', 'read'] (running: []), but those settings are fixed at proxy startup. Restart the proxy, or use --port for a separate one. RESULT (None, 18795) {"proxy_pid_before": 1231619, "proxy_pid_after": 1231619, "min_tokens": 120, "exclude_tools": [], "no_proxy": true, "explicit_mode": null} ``` - Not tested: an external interactive agent session or paid upstream model request. The actual proxy process, health response, and wrapper reuse path were exercised. ## Runtime Rollout Safety - Rollout-managed feature(s): none; diagnostic-only wrapper fix. - Minimum rollout channel: stable; no rollout-managed feature is enabled. - Stable/default behavior changed: a warning only when explicit settings conflict with the running proxy; no configuration mutation or forced restart. - Kill switch / disable path: remove the conflicting explicit environment setting, or revert the commit; no new switch. - Unsafe override required: no. - Qualification impact: wrapper diagnostics only; no model/compression rollout change. - Rollback path: revert this commit; the extra health field is additive. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] Existing conventions reused; documentation updated. - [x] Regression tests cover mismatches, matching settings, built-in exclusions, missing fields, and invalid integers. - [x] `CHANGELOG.md` not edited; Conventional Commit title supplied for release-please. The intentional warning is the requested behavior, not a new unsolicited runtime warning. Full remote CI results will be reported by GitHub, not inferred from local checks.
This commit is contained in:
@@ -886,6 +886,18 @@ headroom wrap cursor
|
||||
headroom wrap grok-build
|
||||
```
|
||||
|
||||
Wrapping can reuse an existing proxy instead of starting a new process. Its
|
||||
startup-only settings do not change when you export different values in the
|
||||
calling shell. `wrap` warns when an explicit `HEADROOM_MODE`,
|
||||
`HEADROOM_MIN_TOKENS`, or `HEADROOM_EXCLUDE_TOOLS` disagrees with the settings
|
||||
reported by that proxy. It does not restart a shared proxy or interrupt other
|
||||
clients. Restart the proxy with the desired settings, or use `--port` to start
|
||||
a separate instance. Tool exclusions are compared case-insensitively using the
|
||||
effective router exclusions, including built-ins and tools protected by
|
||||
`HEADROOM_PROTECT_TOOL_RESULTS`. Older proxies that do not report a setting
|
||||
cannot be compared for that setting.
|
||||
|
||||
|
||||
`headroom wrap codex` keeps routing scoped to the Codex process it launches: it
|
||||
passes the proxy URL as a `--config` override and never writes it to
|
||||
`~/.codex/config.toml`. Codex runs any session started with command-line
|
||||
|
||||
+48
-5
@@ -5131,7 +5131,13 @@ def _ensure_proxy_unlocked(
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
elif not helpers._check_proxy(port):
|
||||
click.echo(f" Warning: No proxy detected on port {port}")
|
||||
elif vertex_api_url or clear_vertex_api_url or os.environ.get("HEADROOM_MODE"):
|
||||
elif (
|
||||
vertex_api_url
|
||||
or clear_vertex_api_url
|
||||
or os.environ.get("HEADROOM_MODE")
|
||||
or os.environ.get("HEADROOM_MIN_TOKENS") is not None
|
||||
or os.environ.get("HEADROOM_EXCLUDE_TOOLS") is not None
|
||||
):
|
||||
health_payload = helpers._query_proxy_health(port)
|
||||
running_config = helpers._proxy_health_config(health_payload)
|
||||
if running_config is None:
|
||||
@@ -9041,12 +9047,49 @@ def unwrap_zcode(port: int, no_stop_proxy: bool) -> None:
|
||||
|
||||
|
||||
def _warn_proxy_mode_mismatch(running_config: dict[str, Any] | None) -> None:
|
||||
"""Warn when a reused proxy runs a different mode than this session asked for.
|
||||
"""Warn when reuse ignores this session's startup-only settings.
|
||||
|
||||
Mode is fixed at proxy startup, so a requested HEADROOM_MODE (explicit, or
|
||||
a wrap target's default_mode) is silently ignored on reuse. Warning-only:
|
||||
other clients may be attached to the running proxy.
|
||||
Do not restart a shared proxy: other clients may be attached to it.
|
||||
"""
|
||||
config = running_config or {}
|
||||
mismatches: list[str] = []
|
||||
requested_min = os.environ.get("HEADROOM_MIN_TOKENS")
|
||||
running_min = config.get("min_tokens_to_crush")
|
||||
if requested_min is not None and isinstance(running_min, int):
|
||||
try:
|
||||
requested_min_value = int(requested_min)
|
||||
except ValueError:
|
||||
requested_min_value = None
|
||||
if requested_min_value is not None and requested_min_value != running_min:
|
||||
mismatches.append(f"HEADROOM_MIN_TOKENS={requested_min_value} (running: {running_min})")
|
||||
|
||||
requested_excludes = os.environ.get("HEADROOM_EXCLUDE_TOOLS")
|
||||
running_excludes = config.get("exclude_tools")
|
||||
if (
|
||||
requested_excludes is not None
|
||||
and isinstance(running_excludes, list)
|
||||
and all(isinstance(name, str) for name in running_excludes)
|
||||
):
|
||||
from headroom.config import DEFAULT_EXCLUDE_TOOLS
|
||||
|
||||
defaults = {name.lower() for name in DEFAULT_EXCLUDE_TOOLS}
|
||||
requested_names = {
|
||||
name.strip().lower() for name in requested_excludes.split(",") if name.strip()
|
||||
}
|
||||
running_names = {name.lower() for name in running_excludes}
|
||||
if requested_names | defaults != running_names | defaults:
|
||||
mismatches.append(
|
||||
f"HEADROOM_EXCLUDE_TOOLS={sorted(requested_names)!r} "
|
||||
f"(running: {sorted(running_names)!r})"
|
||||
)
|
||||
|
||||
if mismatches:
|
||||
click.echo(
|
||||
" Warning: this session requested "
|
||||
+ "; ".join(mismatches)
|
||||
+ ", but those settings are fixed at proxy startup. "
|
||||
"Restart the proxy, or use --port for a separate one."
|
||||
)
|
||||
requested = os.environ.get("HEADROOM_MODE")
|
||||
running = (running_config or {}).get("mode")
|
||||
if not requested or not isinstance(running, str):
|
||||
|
||||
@@ -3847,6 +3847,13 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
|
||||
"min_tokens_to_compress",
|
||||
config.min_tokens_to_crush,
|
||||
),
|
||||
"exclude_tools": sorted(
|
||||
{
|
||||
*DEFAULT_EXCLUDE_TOOLS,
|
||||
*(config.exclude_tools or ()),
|
||||
*(config.protect_tool_results or ()),
|
||||
}
|
||||
),
|
||||
"max_items_after_crush": profile_kwargs.get(
|
||||
"max_items_after_crush",
|
||||
config.max_items_after_crush,
|
||||
|
||||
@@ -291,6 +291,72 @@ class TestModeMismatchWarning:
|
||||
assert self._warnings(monkeypatch, running_config, requested) == []
|
||||
|
||||
|
||||
class TestStartupCompressionMismatchWarning:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _session_env(self, monkeypatch):
|
||||
for name in ("HEADROOM_MODE", "HEADROOM_MIN_TOKENS", "HEADROOM_EXCLUDE_TOOLS"):
|
||||
monkeypatch.delenv(name, raising=False)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("name", "value", "config"),
|
||||
[
|
||||
("HEADROOM_MIN_TOKENS", "2000", {"min_tokens_to_crush": 120}),
|
||||
("HEADROOM_MIN_TOKENS", "0", {"min_tokens_to_crush": 120}),
|
||||
("HEADROOM_EXCLUDE_TOOLS", "Bash,Read", {"exclude_tools": []}),
|
||||
("HEADROOM_EXCLUDE_TOOLS", "", {"exclude_tools": ["Bash"]}),
|
||||
],
|
||||
)
|
||||
def test_warns_when_reuse_ignores_explicit_compression_settings(
|
||||
self, monkeypatch, capsys, name, value, config
|
||||
):
|
||||
monkeypatch.setenv(name, value)
|
||||
_warn_proxy_mode_mismatch(config)
|
||||
output = capsys.readouterr().out
|
||||
assert name in output
|
||||
assert "Restart" in output and "--port" in output
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("env", "config"),
|
||||
[
|
||||
({}, {"min_tokens_to_crush": 120, "exclude_tools": ["Bash"]}),
|
||||
({"HEADROOM_MIN_TOKENS": "120"}, {"min_tokens_to_crush": 120}),
|
||||
({"HEADROOM_MIN_TOKENS": "invalid"}, {"min_tokens_to_crush": 120}),
|
||||
({"HEADROOM_MIN_TOKENS": "2000"}, {}),
|
||||
(
|
||||
{"HEADROOM_EXCLUDE_TOOLS": " Read, BASH, bash "},
|
||||
{"exclude_tools": ["Bash", "bash"]},
|
||||
),
|
||||
({"HEADROOM_EXCLUDE_TOOLS": "Read"}, {"exclude_tools": []}),
|
||||
({"HEADROOM_EXCLUDE_TOOLS": "Bash"}, {}),
|
||||
({"HEADROOM_EXCLUDE_TOOLS": "Bash"}, {"exclude_tools": None}),
|
||||
],
|
||||
)
|
||||
def test_does_not_warn_for_matching_unrequested_or_unknown_settings(
|
||||
self, monkeypatch, capsys, env, config
|
||||
):
|
||||
for name, value in env.items():
|
||||
monkeypatch.setenv(name, value)
|
||||
_warn_proxy_mode_mismatch(config)
|
||||
assert capsys.readouterr().out == ""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("name", "value"),
|
||||
[("HEADROOM_MIN_TOKENS", "2000"), ("HEADROOM_EXCLUDE_TOOLS", "Bash")],
|
||||
)
|
||||
def test_no_start_reuse_warns_without_an_explicit_mode(self, monkeypatch, capsys, name, value):
|
||||
monkeypatch.setenv(name, value)
|
||||
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda _port: True)
|
||||
monkeypatch.setattr(
|
||||
wrap_mod,
|
||||
"_query_proxy_health",
|
||||
lambda _port: {"config": {"min_tokens_to_crush": 120, "exclude_tools": []}},
|
||||
)
|
||||
|
||||
wrap_mod._ensure_proxy_unlocked(18795, True)
|
||||
|
||||
assert name in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_passthrough_handler_roots_profile_at_origin_and_strips_region_domain():
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
|
||||
@@ -541,6 +541,47 @@ def test_health_config_block_is_loopback_only(monkeypatch: pytest.MonkeyPatch) -
|
||||
assert "config" in local.json()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("exclude_tools", "requested"),
|
||||
[(None, "Bash"), ({"WebSearch"}, "Bash,WebSearch")],
|
||||
)
|
||||
def test_protected_tool_reuse_does_not_warn_about_missing_exclusions(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
exclude_tools: set[str] | None,
|
||||
requested: str,
|
||||
) -> None:
|
||||
from headroom.cli.wrap import _warn_proxy_mode_mismatch
|
||||
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
monkeypatch.delenv("HEADROOM_MODE", raising=False)
|
||||
monkeypatch.delenv("HEADROOM_MIN_TOKENS", raising=False)
|
||||
monkeypatch.setenv("HEADROOM_EXCLUDE_TOOLS", requested)
|
||||
app = create_app(
|
||||
ProxyConfig(
|
||||
optimize=False,
|
||||
cache_enabled=False,
|
||||
rate_limit_enabled=False,
|
||||
cost_tracking_enabled=False,
|
||||
log_requests=False,
|
||||
ccr_inject_tool=False,
|
||||
ccr_handle_responses=False,
|
||||
ccr_context_tracking=False,
|
||||
image_optimize=False,
|
||||
disable_kompress=True,
|
||||
exclude_tools=exclude_tools,
|
||||
protect_tool_results={"Bash"},
|
||||
)
|
||||
)
|
||||
client = TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 12345))
|
||||
response = client.get("/health")
|
||||
assert response.status_code == 200
|
||||
|
||||
capsys.readouterr()
|
||||
_warn_proxy_mode_mismatch(response.json()["config"])
|
||||
assert capsys.readouterr().out == ""
|
||||
|
||||
|
||||
def test_stats_per_request_metadata_is_loopback_only() -> None:
|
||||
"""/stats keeps aggregate counters public but restricts per-request metadata
|
||||
(recent_requests / request_logs) and `config` to loopback callers."""
|
||||
|
||||
Reference in New Issue
Block a user