Files
headroom/tests/test_agent_savings.py
Tejas ChopraandClaude Opus 5.5 73f59a32cc refactor(cli): remove test-only and dead helpers from wrap, proxy and mcp (#4118)
## Description

Removes private CLI helpers that no product code calls any more. Each
one lost its last caller in an earlier rewrite, so only its own unit
tests still called it. The tests that pinned the dead code go with it.

Refs #4136
## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `headroom/cli/wrap.py`: removed six dead helpers and constants.
- `_wrap_marker_proxy_is_dead`: the self-heal path now calls
`_wrap_proxy_alive` inline.
- `_agent_savings_config_mismatches` and `_AGENT_SAVINGS_TARGET_AGENTS`:
the live restart check only compares `savings_profile`.
- The `_normalize_openclaw_gateway_provider_ids` forwarder and its
import alias.
- A dead copy of `_opencode_home_dir`; the real one lives in
`providers/opencode/config.py`.
- The unused `_CONTEXT_1M_SUFFIX` and `_OPENCLAUDE_INSTRUCTIONS_FILE`
constants.
- `headroom/cli/proxy.py`: removed `_get_env_float_optional`, which had
no callers.
- `headroom/cli/mcp.py`: removed `get_headroom_command`,
`load_mcp_config`, `save_mcp_config`, `find_headroom_registration` and
the `CLAUDE_CONFIG_DIR` / `MCP_CONFIG_PATH` / `CLAUDE_JSON_PATH`
constants.
- `mcp install`, `mcp status` and `mcp uninstall` all go through
`headroom.mcp_registry` registrars now.
- `load_mcp_config` returned `{}` on corrupt JSON, and `save_mcp_config`
would then overwrite the file. That data-loss path goes away with them.
- Tests removed, because they only exercised the deleted code:
- `tests/test_cli/test_mcp_status.py`, all of which tested
`find_headroom_registration`;
- `TestMCPConfigFunctions` and its path fixtures in
`tests/test_cli/test_mcp.py`;
  - 3 `_wrap_marker_proxy_is_dead` tests;
  - 5 `_agent_savings_config_mismatches` tests;
  - 1 forwarder test.
- The remaining `mcp uninstall` / `mcp status` tests already patch the
registrars. They no longer take the path fixture.

Evidence: `rg -n -w '<name>' headroom crates plugins sdk examples
benchmarks scripts e2e docs wiki .github pyproject.toml tests <consumer
repos>` for each name finds only the definition and the removed tests.
Live behaviour stays covered:
- `test_wrap_dead_marker_selfheal.py`: `test_dead_port_*`,
`test_live_port_is_never_cleared` and the `_wrap_proxy_alive` tests;
- `tests/test_provider_openclaw_wrap.py`;
- `test_mcp.py::TestMCPStatusCommand` / `TestMCPUninstallCommand`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ pytest -q -p no:cacheprovider tests/test_cli/test_wrap_dead_marker_selfheal.py tests/test_agent_savings.py tests/test_cli/test_wrap_openclaw.py tests/test_cli/test_wrap_opencode.py tests/test_cli/test_mcp.py tests/test_cli/test_wrap_helpers.py tests/test_provider_openclaw_wrap.py tests/test_cli/test_main_help_version.py
253 passed in 33.08s
$ uvx ruff@0.16.8 check headroom/cli tests/test_cli tests/test_agent_savings.py
All checks passed!
$ mypy headroom/cli/wrap.py headroom/cli/mcp.py headroom/cli/proxy.py --ignore-missing-imports
Success: no issues found in 3 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, repo `.venv`, branch cut from
`origin/main` at a9815bbb.
- Exact command / steps: ran the pytest files above, which cover `wrap`,
`unwrap`, `mcp status` and `mcp uninstall` via CliRunner, plus `headroom
--help` / `--version`.
- Observed result: all 253 passed; ruff and mypy are clean.
- Smoke harness (real proxy started from this checkout against a local
fake upstream; temp HOME and workspace, offline):

```text
branch: cleanup/cli-integrations-dead-cli-helpers @ 5f450556
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines (<tmp>/ws/logs/proxy-63492.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: <tmp>)
```
- Direct runtime command on this PR's own path (temp HOME,
HEADROOM_WORKSPACE_DIR and CODEX_HOME):

```text
$ HOME=<tmp>/home HEADROOM_WORKSPACE_DIR=<tmp>/ws CODEX_HOME=<tmp>/codex headroom mcp install --agent claude
exit=0
--- stdout
Installing Headroom MCP server...
  claude: registered (restart the agent if it was already running)

Next steps:
  1. Start the Headroom proxy (if not running): headroom proxy
  2. Start your agent (e.g.) ANTHROPIC_BASE_URL=http://127.0.0.1:8787 claude
  3. Restart any agent that was already running so it picks up the new MCP server.
--- stderr
$ HOME=<tmp>/home HEADROOM_WORKSPACE_DIR=<tmp>/ws CODEX_HOME=<tmp>/codex headroom mcp status
exit=0
--- stdout
Headroom MCP Status
========================================
MCP SDK:        ✓ Installed
Agent Config:
  Antigravity IDE: ✗ Not detected
  Claude Code: ✓ Configured
$ HOME=<tmp>/home HEADROOM_WORKSPACE_DIR=<tmp>/ws CODEX_HOME=<tmp>/codex headroom mcp uninstall
exit=0
--- stdout
✓ Claude Code: removed headroom MCP server(s)
--- stderr
```
- Not tested: a live `headroom wrap` session against a real agent
binary. Only unreferenced private code was deleted, so no runtime path
changes.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A, no rollout-managed behaviour.
- Stable/default behavior changed: No. Only code with no callers is
removed.
- Kill switch / disable path: N/A.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.
- No new tests or docs are needed, because this is a pure removal and
none of the removed names were documented.
- `_DEFAULT_1M_MODEL` stays, because
`tests/test_cli/test_wrap_helpers.py` pins it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4118"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:01:00 -07:00

811 lines
28 KiB
Python

from __future__ import annotations
import json
import logging
from importlib import import_module
from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
from click.testing import CliRunner
from headroom.agent_savings import (
AGENT_90_PROFILE,
DEFAULT_PROFILE,
apply_agent_savings_env_defaults,
apply_agent_savings_profile,
get_agent_savings_profile,
proxy_pipeline_kwargs,
seed_proxy_env_defaults,
)
from headroom.cli import wrap as wrap_module
from headroom.cli.main import main
from headroom.compress import CompressConfig, compress
from headroom.proxy.models import ProxyConfig
from headroom.transforms.compression_units import (
CompressionUnit,
compress_unit_with_router,
)
from headroom.transforms.content_router import (
CompressionStrategy,
ContentRouter,
ContentRouterConfig,
RouterCompressionResult,
)
compress_module = import_module("headroom.compress")
def test_agent_90_profile_sets_accuracy_preserving_compress_config() -> None:
cfg = CompressConfig()
apply_agent_savings_profile(cfg, AGENT_90_PROFILE)
assert cfg.compress_user_messages is True
assert cfg.compress_system_messages is True
assert cfg.protect_recent == 2
assert cfg.protect_analysis_context is True
assert cfg.target_ratio == 0.10
assert cfg.min_tokens_to_compress == 120
def test_agent_90_profile_exports_cross_agent_proxy_env() -> None:
profile = get_agent_savings_profile(AGENT_90_PROFILE)
env = profile.proxy_env()
assert env["HEADROOM_MODE"] == "token"
assert env["HEADROOM_SAVINGS_PROFILE"] == "agent-90"
assert env["HEADROOM_SAVINGS_TARGET"] == "0.90"
assert env["HEADROOM_TARGET_RATIO"] == "0.10"
assert env["HEADROOM_COMPRESS_USER_MESSAGES"] == "1"
assert env["HEADROOM_COMPRESS_SYSTEM_MESSAGES"] == "1"
assert env["HEADROOM_MAX_ITEMS"] == "8"
assert env["HEADROOM_SMART_CRUSHER_COMPACTION"] == "0"
assert env["HEADROOM_FORCE_KOMPRESS"] == "1"
assert env["HEADROOM_ACCURACY_GUARD"] == "strict"
def test_coding_persona_compresses_recent_delta_and_stays_visible() -> None:
profile = get_agent_savings_profile("coding")
env = profile.proxy_env()
assert env["HEADROOM_SAVINGS_PROFILE"] == "coding"
assert env["HEADROOM_MODE"] == "cache" # delta-only compression at ~0 prefix-cache busts
assert env["HEADROOM_PROTECT_RECENT"] == "0" # reads guarded by type, not position
assert env["HEADROOM_MIN_TOKENS"] == "10" # low → even modest deltas are eligible
# Cache mode compresses the newest observation delta → compress_user must be ON.
assert env["HEADROOM_COMPRESS_USER_MESSAGES"] == "1"
assert env["HEADROOM_COMPRESS_SYSTEM_MESSAGES"] == "0" # system prompt is the hottest cache
assert env["HEADROOM_ACCURACY_GUARD"] == "strict"
assert "HEADROOM_TARGET_RATIO" not in env # unset → Kompress / ambient default decides
# Coding posture toggles seeded through the profile.
assert env["HEADROOM_TOOL_SEARCH"] == "1"
assert env["HEADROOM_DEDUPE"] == "1"
assert env["HEADROOM_LOSSLESS_THEN_LOSSY"] == "1"
assert env["HEADROOM_PROTECT_READS"] == "1"
assert env["HEADROOM_CODE_AWARE_ENABLED"] == "1"
assert env["HEADROOM_LOSSLESS"] == "0" # lossy enabled (CCR keeps it recoverable)
assert env["HEADROOM_MIN_CHARS_FOR_BLOCK"] == "25"
def test_coding_profile_couples_zero_protect_recent_with_type_read_guard() -> None:
"""Fidelity invariant behind #2145's protect_recent 2->0.
Dropping positional protection (protect_recent=0) is only safe because the
code working set stays byte-exact via the TYPE-based read guard
(protect_reads=True), and any *other* recent delta that does get compressed
stays losslessly recoverable via CCR (lossless=0 means lossy-with-CCR, not
silent loss) while the frozen prefix is left untouched (cache mode). This is
the honest boundary: recent file reads are verbatim, recent non-read deltas
are recoverable — not "nothing is ever touched". If a future edit drops the
read guard while keeping protect_recent=0, recent reads would silently
degrade, so this test fails closed on that pairing.
"""
profile = get_agent_savings_profile("coding")
# Positional protection is off ...
assert profile.protect_recent == 0
# ... so the byte-exact guarantee for reads MUST come from the type guard.
assert profile.protect_reads is True
env = profile.proxy_env()
assert env["HEADROOM_PROTECT_RECENT"] == "0"
assert env["HEADROOM_PROTECT_READS"] == "1"
# Lossy compression is on, but CCR keeps compressed deltas recoverable, and
# cache mode compresses only the newest delta (frozen prefix stays byte-stable).
assert env["HEADROOM_LOSSLESS"] == "0"
assert env["HEADROOM_MODE"] == "cache"
def test_general_persona_has_no_positional_code_protection() -> None:
profile = get_agent_savings_profile("general")
env = profile.proxy_env()
assert env["HEADROOM_PROTECT_RECENT"] == "0"
assert env["HEADROOM_MIN_TOKENS"] == "25"
assert "HEADROOM_TARGET_RATIO" not in env
def test_personas_omit_target_ratio_in_pipeline_kwargs() -> None:
# coding compresses the delta observation (cache mode) → compress_user True;
# general has no positional code working set and leaves user turns intact.
for name, expected_protect, expected_compress_user, expected_min_tokens in (
("coding", 0, True, 10),
("general", 0, False, 25),
):
kwargs = proxy_pipeline_kwargs(ProxyConfig(savings_profile=name))
assert kwargs["protect_recent"] == expected_protect
assert kwargs["read_protection_window"] == expected_protect
assert kwargs["min_tokens_to_compress"] == expected_min_tokens
assert kwargs["compress_user_messages"] is expected_compress_user
assert kwargs["compress_system_messages"] is False
assert kwargs["force_kompress"] is False
assert "target_ratio" not in kwargs # persona never pins a keep-ratio
def test_persona_apply_profile_leaves_target_ratio_untouched() -> None:
cfg = CompressConfig(target_ratio=0.42)
apply_agent_savings_profile(cfg, "coding")
assert cfg.protect_recent == 0
assert cfg.min_tokens_to_compress == 10
assert cfg.target_ratio == 0.42 # persona did not override an explicit ratio
def test_agent_savings_env_defaults_preserve_user_overrides() -> None:
env = {
"HEADROOM_TARGET_RATIO": "0.25",
"HEADROOM_MAX_ITEMS": "12",
}
apply_agent_savings_env_defaults(env, AGENT_90_PROFILE)
assert env["HEADROOM_SAVINGS_PROFILE"] == "agent-90"
assert env["HEADROOM_TARGET_RATIO"] == "0.25"
assert env["HEADROOM_MAX_ITEMS"] == "12"
assert env["HEADROOM_SMART_CRUSHER_COMPACTION"] == "0"
def test_seed_proxy_env_defaults_reports_only_generated_keys() -> None:
env = {"HEADROOM_MODE": "token", "HEADROOM_LOSSLESS": "1"}
seeded = seed_proxy_env_defaults(env)
assert "HEADROOM_MODE" not in seeded
assert "HEADROOM_LOSSLESS" not in seeded
assert "HEADROOM_SAVINGS_PROFILE" in seeded
assert env["HEADROOM_MODE"] == "token"
assert env["HEADROOM_LOSSLESS"] == "1"
def test_unknown_agent_savings_profile_falls_back_to_default(
caplog: pytest.LogCaptureFixture,
) -> None:
# An unknown profile must NOT raise: it's resolved during proxy startup, so
# raising takes the whole proxy down before it opens its port (desktop asked
# for a profile a fallback runtime predates).
#
# It must degrade to the DEFAULT profile, not to "balanced". The two are not
# interchangeable: balanced flips cache->token mode, turns cross-turn dedup
# and tool-search off, stops compressing user messages, and raises the
# message floor 25x (250 vs 10) and the block floor 20x (500 vs 25). A typo
# in HEADROOM_SAVINGS_PROFILE used to silently reconfigure the entire proxy
# into that posture, which is strictly worse than behaving as if the
# variable were unset.
with caplog.at_level(logging.WARNING):
profile = get_agent_savings_profile("missing")
assert profile is get_agent_savings_profile(None)
assert profile.name == DEFAULT_PROFILE
assert profile is not get_agent_savings_profile("balanced")
assert "unknown savings profile" in caplog.text
assert "missing" in caplog.text
# The warning has to name the resolved profile, so an operator reading it
# knows what they actually got rather than only what they asked for.
assert DEFAULT_PROFILE in caplog.text
def test_agent_savings_cli_renders_shell_exports() -> None:
result = CliRunner().invoke(main, ["agent-savings", "--profile", "agent-90"])
assert result.exit_code == 0
assert 'export HEADROOM_SAVINGS_PROFILE="agent-90"' in result.output
assert 'export HEADROOM_SAVINGS_TARGET="0.90"' in result.output
assert 'export HEADROOM_ACCURACY_GUARD="strict"' in result.output
def test_agent_savings_cli_renders_json() -> None:
result = CliRunner().invoke(
main,
["agent-savings", "--profile", "agent-90", "--format", "json"],
)
assert result.exit_code == 0
assert '"HEADROOM_TARGET_RATIO": "0.10"' in result.output
def test_compress_applies_agent_savings_profile_to_pipeline(monkeypatch) -> None:
captured: dict[str, object] = {}
messages = [{"role": "user", "content": "x" * 500}]
class Pipeline:
def apply(self, **kwargs):
captured.update(kwargs)
return SimpleNamespace(
messages=messages,
tokens_before=1000,
tokens_after=100,
transforms_applied=["test"],
)
monkeypatch.setattr(compress_module, "_get_pipeline", lambda: Pipeline())
config = CompressConfig()
apply_agent_savings_profile(config, AGENT_90_PROFILE)
result = compress(messages, config=config)
assert result.compression_ratio == 0.9
assert captured["compress_user_messages"] is True
assert captured["compress_system_messages"] is True
assert captured["protect_recent"] == 2
assert captured["protect_analysis_context"] is True
assert captured["target_ratio"] == 0.10
assert captured["min_tokens_to_compress"] == 120
def test_compress_savings_profile_does_not_mutate_supplied_config(monkeypatch) -> None:
captured: dict[str, object] = {}
messages = [{"role": "user", "content": "x" * 500}]
config = CompressConfig(
compress_user_messages=False,
compress_system_messages=False,
protect_recent=9,
protect_analysis_context=False,
target_ratio=None,
min_tokens_to_compress=999,
)
class Pipeline:
def apply(self, **kwargs):
captured.update(kwargs)
return SimpleNamespace(
messages=messages,
tokens_before=1000,
tokens_after=100,
transforms_applied=["test"],
)
monkeypatch.setattr(compress_module, "_get_pipeline", lambda: Pipeline())
compress(messages, config=config, savings_profile=AGENT_90_PROFILE)
assert captured["target_ratio"] == 0.10
assert captured["min_tokens_to_compress"] == 120
assert config.compress_user_messages is False
assert config.compress_system_messages is False
assert config.protect_recent == 9
assert config.protect_analysis_context is False
assert config.target_ratio is None
assert config.min_tokens_to_compress == 999
def test_force_kompress_falls_back_to_structural_compression_when_cold(monkeypatch):
"""A cold forced-ML profile must not hide safe content routing."""
import headroom.transforms.content_router as router_mod
router = ContentRouter(ContentRouterConfig())
compressor = SimpleNamespace(is_ready=lambda: False, ensure_background_load=MagicMock())
log_compressor = SimpleNamespace(
compress=lambda content, bias=1.0: SimpleNamespace(compressed="ERROR: compacted")
)
router._get_kompress = lambda: compressor
router._get_log_compressor = lambda: log_compressor
router._runtime_force_kompress = True
router._runtime_skip_kompress = False
router._runtime_target_ratio = None
router.config.enable_log_compressor = True
monkeypatch.setattr(
router_mod,
"_detect_content",
lambda content: router_mod.DetectionResult(router_mod.ContentType.BUILD_OUTPUT, 1.0, {}),
)
monkeypatch.setattr(router_mod, "is_mixed_content", lambda content: False)
result = router.compress("ERROR: repeated output\\n" * 40)
assert result.compressed == "ERROR: compacted"
assert result.strategy_used == CompressionStrategy.LOG
compressor.ensure_background_load.assert_called_once()
def test_wrap_agent_savings_profile_is_opt_in(monkeypatch) -> None:
monkeypatch.delenv("HEADROOM_SAVINGS_PROFILE", raising=False)
assert wrap_module._wrap_agent_savings_profile("codex") is None
monkeypatch.setenv("HEADROOM_SAVINGS_PROFILE", AGENT_90_PROFILE)
assert wrap_module._wrap_agent_savings_profile("codex") == AGENT_90_PROFILE
def test_start_proxy_does_not_inject_agent_savings_by_default(monkeypatch, tmp_path) -> None:
captured_env: dict[str, str] = {}
class Proc:
returncode = None
def poll(self) -> None:
return None
def popen(cmd, **kwargs): # noqa: ANN001
captured_env.update(kwargs["env"])
return Proc()
monkeypatch.delenv("HEADROOM_SAVINGS_PROFILE", raising=False)
monkeypatch.setattr(wrap_module.subprocess, "Popen", popen)
monkeypatch.setattr(wrap_module.time, "sleep", lambda seconds: None)
monkeypatch.setattr(wrap_module, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_module, "_get_log_path", lambda port=None: tmp_path / "proxy.log")
wrap_module._start_proxy(8787, agent_type="codex")
assert "HEADROOM_SAVINGS_PROFILE" not in captured_env
assert "HEADROOM_TARGET_RATIO" not in captured_env
def test_start_proxy_injects_explicit_agent_savings_profile(monkeypatch, tmp_path) -> None:
captured_env: dict[str, str] = {}
class Proc:
returncode = None
def poll(self) -> None:
return None
def popen(cmd, **kwargs): # noqa: ANN001
captured_env.update(kwargs["env"])
return Proc()
monkeypatch.setenv("HEADROOM_SAVINGS_PROFILE", AGENT_90_PROFILE)
monkeypatch.setattr(wrap_module.subprocess, "Popen", popen)
monkeypatch.setattr(wrap_module.time, "sleep", lambda seconds: None)
monkeypatch.setattr(wrap_module, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_module, "_get_log_path", lambda port=None: tmp_path / "proxy.log")
wrap_module._start_proxy(8787, agent_type="codex")
assert captured_env["HEADROOM_SAVINGS_PROFILE"] == AGENT_90_PROFILE
assert captured_env["HEADROOM_TARGET_RATIO"] == "0.10"
def test_agent_90_profile_applies_to_proxy_config_runtime_kwargs() -> None:
config = ProxyConfig(savings_profile="agent-90")
kwargs = proxy_pipeline_kwargs(config)
assert kwargs["compress_user_messages"] is True
assert kwargs["compress_system_messages"] is True
assert kwargs["protect_recent"] == 2
assert kwargs["protect_analysis_context"] is True
assert kwargs["target_ratio"] == 0.10
assert kwargs["min_tokens_to_compress"] == 120
assert kwargs["max_items_after_crush"] == 8
assert kwargs["smart_crusher_with_compaction"] is False
assert kwargs["force_kompress"] is True
assert kwargs["read_protection_window"] == 2
def test_proxy_explicit_config_overrides_agent_90_profile() -> None:
config = ProxyConfig(
savings_profile="agent-90",
target_ratio=0.25,
protect_recent=5,
min_tokens_to_crush=300,
)
kwargs = proxy_pipeline_kwargs(config)
assert kwargs["target_ratio"] == 0.25
assert kwargs["protect_recent"] == 5
assert kwargs["min_tokens_to_compress"] == 300
def test_proxy_explicit_compress_user_messages_off_overrides_coding_profile() -> None:
# coding turns user-message compression on; an explicit off
# (HEADROOM_COMPRESS_USER_MESSAGES=0) must win like every other override,
# and leaving it unset must keep the profile default.
explicit_off = ProxyConfig(savings_profile="coding", compress_user_messages=False)
unset = ProxyConfig(savings_profile="coding")
assert proxy_pipeline_kwargs(explicit_off)["compress_user_messages"] is False
assert proxy_pipeline_kwargs(unset)["compress_user_messages"] is True
def test_agent_90_router_uses_ccr_sampling_not_lossless_table() -> None:
router = ContentRouter(
ContentRouterConfig(
smart_crusher_max_items_after_crush=8,
smart_crusher_with_compaction=False,
)
)
crusher = router._get_smart_crusher()
assert crusher is not None
assert crusher.config.max_items_after_crush == 8
assert crusher._with_compaction is False
def test_router_lossless_only_flag_reaches_crusher() -> None:
# HEADROOM_LOSSLESS_ONLY=1 sets this field on the proxy router; it
# must flow through to the SmartCrusher so a real proxy session runs
# strict marker-free mode.
router = ContentRouter(ContentRouterConfig(smart_crusher_lossless_only=True))
crusher = router._get_smart_crusher()
assert crusher is not None
assert crusher._lossless_only is True
def test_router_lossless_only_defaults_off() -> None:
# Unset (None) must not force the flag — default crushers stay in
# the marker-emitting mode.
router = ContentRouter(ContentRouterConfig())
crusher = router._get_smart_crusher()
assert crusher is not None
assert crusher._lossless_only is False
def test_agent_90_router_json_tool_output_reaches_target_with_needle() -> None:
needle = "CRITICAL_NEEDLE_42"
rows = [
{
"id": i,
"status": "ok",
"message": "normal repeated telemetry payload",
"value": i % 7,
}
for i in range(1000)
]
rows.append(
{
"id": 99999,
"status": "error",
"message": f"{needle} root cause disk full",
"value": 999.99,
}
)
router = ContentRouter(
ContentRouterConfig(
smart_crusher_max_items_after_crush=8,
smart_crusher_with_compaction=False,
)
)
result = router.compress(json.dumps(rows), question=f"Find {needle}")
before = len(result.original.split())
after = len(result.compressed.split())
assert 1 - after / before >= 0.90
assert needle in result.compressed
assert "<<ccr:" in result.compressed
def test_proxy_cli_reads_agent_90_profile_env() -> None:
captured_config: dict[str, ProxyConfig] = {}
def mock_run_server(config: ProxyConfig, **kwargs: object) -> None:
captured_config["config"] = config
runner = CliRunner()
with pytest.MonkeyPatch.context() as mp:
mp.setattr("headroom.proxy.server.run_server", mock_run_server)
result = runner.invoke(
main,
["proxy"],
env={"HEADROOM_SAVINGS_PROFILE": "agent-90"},
catch_exceptions=False,
)
assert result.exit_code == 0, result.output
config = captured_config["config"]
assert config.savings_profile == "agent-90"
assert proxy_pipeline_kwargs(config)["target_ratio"] == 0.10
def test_unit_router_receives_agent_target_ratio() -> None:
seen: dict[str, object] = {}
class Tokenizer:
def count_text(self, text: str) -> int:
return len(text.split())
class Router:
_runtime_target_ratio = None
def compress(self, text: str, **kwargs: object) -> RouterCompressionResult:
seen["target_ratio"] = self._runtime_target_ratio
return RouterCompressionResult(
compressed="short text",
original=text,
strategy_used=CompressionStrategy.KOMPRESS,
strategy_chain=["kompress"],
)
unit = CompressionUnit(
text=("long text " * 40) + "\nRetrieve more: hash=abc123\n",
provider="openai",
endpoint="responses",
role="assistant",
item_type="message",
cache_zone="live",
mutable=True,
min_bytes=1,
metadata={"compress_assistant": "true"},
)
result = compress_unit_with_router(
unit,
router=Router(),
tokenizer=Tokenizer(),
target_ratio=0.10,
)
assert result.modified is True
assert seen["target_ratio"] == 0.10
def test_agent_savings_check_perf_and_accuracy_report_passes(
monkeypatch,
tmp_path,
) -> None:
from headroom.perf import analyzer
monkeypatch.setattr(analyzer, "parse_log_files", lambda last_n_hours: object())
monkeypatch.setattr(
analyzer,
"build_perf_summary",
lambda report: {"savings_pct": 92.0},
)
report = tmp_path / "eval.json"
report.write_text(json.dumps({"totals": {"accuracy_rate": 1.0}}))
result = CliRunner().invoke(
main,
[
"agent-savings",
"--profile",
"agent-90",
"--check-perf",
"--accuracy-report",
str(report),
],
)
assert result.exit_code == 0, result.output
assert "92.0% savings meets 90.0%" in result.output
assert "100.0% accuracy meets 90.0%" in result.output
def test_agent_savings_accuracy_report_below_threshold_fails(
monkeypatch,
tmp_path,
) -> None:
from headroom.perf import analyzer
monkeypatch.setattr(analyzer, "parse_log_files", lambda last_n_hours: object())
monkeypatch.setattr(
analyzer,
"build_perf_summary",
lambda report: {"savings_pct": 92.0},
)
report = tmp_path / "eval.json"
report.write_text(json.dumps({"totals": {"accuracy_rate": 0.89}}))
result = CliRunner().invoke(
main,
[
"agent-savings",
"--profile",
"agent-90",
"--check-perf",
"--accuracy-report",
str(report),
],
)
assert result.exit_code != 0
assert "89.0% accuracy below 90.0%" in result.output
def test_agent_savings_requires_each_agent_to_meet_target(monkeypatch) -> None:
from headroom.perf import analyzer
from headroom.perf.analyzer import PerfRecord, PerfReport
report = PerfReport(
perf_records=[
PerfRecord(
timestamp="2026-06-10 10:00:00,000",
request_id="claude-1",
model="claude-sonnet",
client="claude",
tokens_before=1000,
tokens_after=80,
tokens_saved=920,
),
PerfRecord(
timestamp="2026-06-10 10:01:00,000",
request_id="codex-1",
model="gpt-5",
client="codex",
tokens_before=1000,
tokens_after=90,
tokens_saved=910,
),
PerfRecord(
timestamp="2026-06-10 10:02:00,000",
request_id="cursor-1",
model="gpt-5",
client="cursor",
tokens_before=1000,
tokens_after=70,
tokens_saved=930,
),
]
)
monkeypatch.setattr(analyzer, "parse_log_files", lambda last_n_hours: report)
result = CliRunner().invoke(
main,
[
"agent-savings",
"--check-perf",
"--require-agents",
"claude,codex,cursor",
],
)
assert result.exit_code == 0, result.output
assert "claude: 92.0% savings meets 90.0%" in result.output
assert "codex: 91.0% savings meets 90.0%" in result.output
assert "cursor: 93.0% savings meets 90.0%" in result.output
def test_agent_savings_required_agent_missing_fails(monkeypatch) -> None:
from headroom.perf import analyzer
from headroom.perf.analyzer import PerfRecord, PerfReport
report = PerfReport(
perf_records=[
PerfRecord(
timestamp="2026-06-10 10:00:00,000",
request_id="claude-1",
model="claude-sonnet",
client="claude",
tokens_before=1000,
tokens_after=80,
tokens_saved=920,
),
PerfRecord(
timestamp="2026-06-10 10:01:00,000",
request_id="codex-1",
model="gpt-5",
client="codex",
tokens_before=1000,
tokens_after=90,
tokens_saved=910,
),
]
)
monkeypatch.setattr(analyzer, "parse_log_files", lambda last_n_hours: report)
result = CliRunner().invoke(
main,
[
"agent-savings",
"--check-perf",
"--require-agents",
"claude,codex,cursor",
],
)
assert result.exit_code != 0
assert "missing required agent traffic: cursor" in result.output
def test_agent_savings_writes_three_agent_smoke_fixture(tmp_path) -> None:
workspace = tmp_path / "workspace"
result = CliRunner().invoke(
main,
["agent-savings", "--write-smoke-fixture", str(workspace)],
)
assert result.exit_code == 0, result.output
assert (workspace / "logs" / "proxy.log").exists()
eval_report = workspace / "agent-90-eval.json"
assert eval_report.exists()
assert "--require-agents claude,codex,cursor" in result.output
def test_agent_savings_smoke_fixture_passes_real_gate(tmp_path) -> None:
workspace = tmp_path / "workspace"
runner = CliRunner()
write_result = runner.invoke(
main,
["agent-savings", "--write-smoke-fixture", str(workspace)],
)
assert write_result.exit_code == 0, write_result.output
gate_result = runner.invoke(
main,
[
"agent-savings",
"--check-perf",
"--hours",
"0",
"--require-agents",
"claude,codex,cursor",
"--accuracy-report",
str(workspace / "agent-90-eval.json"),
],
env={"HEADROOM_WORKSPACE_DIR": str(workspace)},
)
assert gate_result.exit_code == 0, gate_result.output
assert "claude: 92.0% savings meets 90.0%" in gate_result.output
assert "codex: 91.0% savings meets 90.0%" in gate_result.output
assert "cursor: 93.0% savings meets 90.0%" in gate_result.output
assert "100.0% accuracy meets 90.0%" in gate_result.output
def test_coding_profile_min_chars_block_reaches_router_without_env_seeding(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The block-char floor must travel on the config object, not env only.
Every other router pipeline kwarg this function builds travels on the config
object; ``min_chars_for_block`` alone was populated only from
``HEADROOM_MIN_CHARS_FOR_BLOCK`` (emitted by ``proxy_env()``). A proxy whose
config carried ``savings_profile="coding"`` but whose process env was never
seeded applied every sibling coding knob while this floor silently stayed at
``ContentRouterConfig.min_chars_for_block_compression`` (500) instead of the
profile's 25 — a 20x gap on the gate that governs tool_result blocks.
Note this does not make the profile fully config-deliverable: fields whose
consumers read ``os.environ`` directly (``cross_turn_dedup`` via
ContentRouter, ``tool_search`` via the Anthropic handler) never pass through
this function and remain seed-dependent.
"""
monkeypatch.delenv("HEADROOM_MIN_CHARS_FOR_BLOCK", raising=False)
class _Config:
savings_profile = "coding"
min_tokens_to_crush = 500
kwargs = proxy_pipeline_kwargs(_Config())
assert kwargs["min_chars_for_block_compression"] == 25
assert kwargs["min_tokens_to_compress"] == 10
def test_explicit_min_chars_block_env_overrides_the_profile(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""An explicit operator override still wins over the profile value."""
monkeypatch.setenv("HEADROOM_MIN_CHARS_FOR_BLOCK", "120")
class _Config:
savings_profile = "coding"
min_tokens_to_crush = 500
assert proxy_pipeline_kwargs(_Config())["min_chars_for_block_compression"] == 120