mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): report read maturation as inert in cache mode (#4198)
## Description
`--read-maturation` combined with `--mode cache` starts cleanly, and
`/stats.rollout` reports the feature as requested and enabled. It can
never fire, and nothing says so.
Here is why:
- In cache mode the Anthropic handler sets `frozen_message_count =
_strict_previous_turn_frozen_count(...)` (`handlers/anthropic.py`). That
freezes every message except the final user turn.
- Cache mode never reaches the token-mode `prepare_turn` clamp, so that
count is passed straight to `ReadMaturationManager.apply(...,
frozen_message_count=...)`.
- `apply` skips every message with `i < frozen_message_count`.
- The only Reads it ever examines are therefore the tool results in the
latest user turn. Those answer the assistant message just before, so
`quiet_turns = held_turns = 0`. That is below `quiesce_turns` (default
5) and `max_hold_turns` (default 25), so they are always held.
- By the time a Read could have quiesced, it sits in the frozen prefix.
The handler's per-request summary is logged at DEBUG, so a deployment
sees no output either way. One persistent proxy ran this combination for
over five weeks, about 51,000 real requests, with zero
`read_maturation:` transforms. The existing handler tests all use
`mode="token"`, so no test caught it.
Holding is harmless in cache mode, because a "held" Read is forwarded
verbatim. So this PR makes the inert state visible rather than changing
request handling:
- At startup, enabled read maturation is reported once: a **WARNING**
`Read maturation: INERT (cache mode freezes a Read before it can
quiesce); use --mode token or disable read_maturation` in cache mode,
otherwise an INFO line with the effective quiesce, hold and size
settings. The hint says "disable read_maturation" rather than naming a
flag, because the feature can come from `--read-maturation` or from
`HEADROOM_FEATURES`.
- `/stats` gains `read_maturation: {enabled, effective, inert_reason}`,
beside `rollout`, which only records that the feature was granted.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/modes.py`: `read_maturation_inert_reason(mode, *,
quiesce_turns, max_hold_turns)` returns the reason string under cache
mode, else `None`. A threshold of 0, which the env path
(`HEADROOM_READ_MATURATION_*_TURNS`) accepts even though the CLI
requires at least 1, matures a Read in the still-mutable latest turn, so
it is not reported as inert. The rule lives in one place for both
callers.
- `headroom/proxy/server.py` (startup banner): after the cache-mode
lines, log the INERT WARNING or an ENABLED INFO line with
`quiesce_turns` / `max_hold_turns` / `min_size_bytes`.
- `headroom/proxy/server.py` (`/stats`): add `read_maturation: {enabled,
effective, inert_reason}` through a small `_read_maturation_status(cfg)`
helper.
- `tests/test_proxy_stats_recent_requests.py`:
`test_read_maturation_reports_whether_it_can_fire`, parametrized over
cache, token, and cache with `quiesce_turns=0`. It asserts the `/stats`
fields and that exactly one `Read maturation:` startup record is logged,
at WARNING (inert) or INFO (effective).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_proxy_stats_recent_requests.py -k read_maturation -v
tests/test_proxy_stats_recent_requests.py::test_read_maturation_reports_whether_it_can_fire[cache-5-False-WARNING] PASSED [ 33%]
tests/test_proxy_stats_recent_requests.py::test_read_maturation_reports_whether_it_can_fire[token-5-True-INFO] PASSED [ 66%]
tests/test_proxy_stats_recent_requests.py::test_read_maturation_reports_whether_it_can_fire[cache-0-True-INFO] PASSED [100%]
================= 3 passed, 5 deselected, 2 warnings in 1.74s ==================
# all three cases fail on main (no "read_maturation" key in /stats, no startup record)
$ pytest tests/test_proxy_stats_recent_requests.py tests/test_read_maturation*.py
======================== 40 passed, 3 warnings in 4.86s ========================
$ ruff check headroom/proxy/modes.py headroom/proxy/server.py tests/test_proxy_stats_recent_requests.py
All checks passed!
$ ruff format --check headroom/proxy/modes.py headroom/proxy/server.py tests/test_proxy_stats_recent_requests.py
3 files already formatted
$ mypy headroom/proxy/modes.py headroom/proxy/server.py
Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: macOS, Python 3.13. A persistent `headroom proxy` (0.38 →
0.40.0) serving Anthropic Messages traffic from an agentic coding
client, with `--mode cache --lossless --code-aware --read-maturation`,
`HEADROOM_ROLLOUT_CHANNEL=beta` and default maturation thresholds.
- Exact command / steps: (1) `GET /stats` on the running proxy. (2)
Counted `PERF` records and `read_maturation:` transforms in the proxy's
archived PERF log from 2026-09-05 to 2026-10-13. (3) Traced
`frozen_message_count` from `_strict_previous_turn_frozen_count` into
`ReadMaturationManager.apply`.
- Observed result: (1) `rollout` reports `read_maturation:
requested=True, enabled=True, decision=explicit` on channel `beta`, so
gating is not the cause. (2) 50,970 PERF records and **0**
`read_maturation:` transforms from real traffic; the only 5 hits in the
window were test-fixture requests. (3) Only the final user turn is ever
unfrozen, so every Read examined is at 0 quiet turns and is held.
- Not tested: a full restart of that deployment on this branch. The
startup line and the `/stats` block were exercised through `create_app`
plus `TestClient` in the new test. I also did not try to make maturation
actually work in cache mode; see Additional Notes.
## Runtime Rollout Safety
- Rollout-managed feature(s): `read_maturation` (beta). Gating is
unchanged; this only reports on it.
- Minimum rollout channel: unchanged (beta).
- Stable/default behavior changed: no request-path change. `effective`
describes the mode and thresholds only; maturation runs in the Anthropic
messages handler and skips bypassed requests. One new startup log line
appears when read maturation is enabled, and `/stats` gains one additive
key.
- Kill switch / disable path: the existing one; drop `--read-maturation`
/ `HEADROOM_FEATURES`.
- Unsafe override required: no.
- Qualification impact: `/stats` gains `read_maturation`; existing keys,
including `rollout`, are unchanged. Black-box tools reading `rollout`
see the same payload.
- Rollback path: revert the commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A
## Additional Notes
- **Why a warning, not a refusal:** existing deployments run this
combination, and failing startup on upgrade would take them down for a
feature that is a no-op there. Happy to switch to refusing the
combination if maintainers prefer.
- **Possible follow-up, not in this PR:** cache mode does have one
moment when rewriting the prefix is free, which is cold-prefix
recompaction (`HEADROOM_COLD_RECOMPACT`, after the cache has lapsed).
Letting maturation run there would give it a real effect in cache mode.
That is a design change, and I'd rather discuss it first.
- **Also possible:** per-reason skip counters (frozen, below min size,
holding, already marked) in `MaturationResult`, surfaced in `/stats`. I
left this out to keep the PR small.
- **Docs:** I didn't find a page describing `--read-maturation`'s mode
interaction. Happy to add a line wherever maintainers want it.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4198"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -36,3 +36,17 @@ def is_token_mode(mode: str | None) -> bool:
|
||||
def is_cache_mode(mode: str | None) -> bool:
|
||||
"""Return True when mode resolves to cache mode."""
|
||||
return normalize_proxy_mode(mode) == PROXY_MODE_CACHE
|
||||
|
||||
|
||||
def read_maturation_inert_reason(
|
||||
mode: str | None, *, quiesce_turns: int, max_hold_turns: int
|
||||
) -> str | None:
|
||||
"""Why read maturation can never fire under ``mode``, else None.
|
||||
|
||||
Cache mode freezes every turn but the latest, and maturation skips frozen
|
||||
messages. A Read in the latest turn has had no quiet turns yet, so it is
|
||||
held unless a threshold is 0; by the time it could quiesce it is frozen.
|
||||
"""
|
||||
if is_cache_mode(mode) and quiesce_turns > 0 and max_hold_turns > 0:
|
||||
return "cache mode freezes a Read before it can quiesce"
|
||||
return None
|
||||
|
||||
@@ -163,6 +163,7 @@ from headroom.proxy.modes import (
|
||||
PROXY_MODE_TOKEN,
|
||||
is_token_mode,
|
||||
normalize_proxy_mode,
|
||||
read_maturation_inert_reason,
|
||||
)
|
||||
from headroom.proxy.orphan_watchdog import (
|
||||
orphan_grace_seconds,
|
||||
@@ -2092,6 +2093,25 @@ class HeadroomProxy(
|
||||
if self.config.mode == PROXY_MODE_CACHE:
|
||||
logger.info(" Prefix freeze: strict (all prior turns immutable)")
|
||||
logger.info(" Mutations: latest turn only")
|
||||
if self.config.read_maturation:
|
||||
_inert = read_maturation_inert_reason(
|
||||
self.config.mode,
|
||||
quiesce_turns=self.config.read_maturation_quiesce_turns,
|
||||
max_hold_turns=self.config.read_maturation_max_hold_turns,
|
||||
)
|
||||
if _inert:
|
||||
logger.warning(
|
||||
"Read maturation: INERT (%s); use --mode token or disable read_maturation",
|
||||
_inert,
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
"Read maturation: ENABLED (quiesce_turns=%d max_hold_turns=%d "
|
||||
"min_size_bytes=%d)",
|
||||
self.config.read_maturation_quiesce_turns,
|
||||
self.config.read_maturation_max_hold_turns,
|
||||
self.config.read_maturation_min_size_bytes,
|
||||
)
|
||||
# Effective compression posture, resolved (not merely requested). The
|
||||
# savings profile reaches the router through two paths — the config
|
||||
# object and the seeded process env — and `setdefault` semantics mean a
|
||||
@@ -5387,6 +5407,25 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
|
||||
# Effective state from the running process. This is the supported
|
||||
# black-box provenance surface for benchmark/qualification tools.
|
||||
"rollout": proxy.config.rollout.to_dict() if proxy.config.rollout else None,
|
||||
"read_maturation": _read_maturation_status(proxy.config),
|
||||
}
|
||||
|
||||
def _read_maturation_status(cfg: ProxyConfig) -> dict[str, Any]:
|
||||
# The rollout snapshot says whether the feature is enabled, not whether
|
||||
# it can act; cache mode leaves it enabled but unable to fire.
|
||||
inert = (
|
||||
read_maturation_inert_reason(
|
||||
cfg.mode,
|
||||
quiesce_turns=cfg.read_maturation_quiesce_turns,
|
||||
max_hold_turns=cfg.read_maturation_max_hold_turns,
|
||||
)
|
||||
if cfg.read_maturation
|
||||
else None
|
||||
)
|
||||
return {
|
||||
"enabled": cfg.read_maturation,
|
||||
"effective": cfg.read_maturation and inert is None,
|
||||
"inert_reason": inert,
|
||||
}
|
||||
|
||||
def _dashboard_config_payload() -> dict[str, Any]:
|
||||
|
||||
@@ -277,3 +277,50 @@ def test_stats_preserves_default_smart_crusher_compaction_state() -> None:
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.json()["config"]["smart_crusher_with_compaction"] is None
|
||||
|
||||
|
||||
def _read_maturation_app(mode: str, quiesce_turns: int = 5):
|
||||
return create_app(
|
||||
ProxyConfig(
|
||||
mode=mode,
|
||||
read_maturation=True,
|
||||
read_maturation_quiesce_turns=quiesce_turns,
|
||||
optimize=False,
|
||||
cache_enabled=False,
|
||||
rate_limit_enabled=False,
|
||||
cost_tracking_enabled=False,
|
||||
log_requests=False,
|
||||
ccr_inject_tool=False,
|
||||
ccr_handle_responses=False,
|
||||
ccr_context_tracking=False,
|
||||
http2=False,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("mode", "quiesce_turns", "effective", "startup_level"),
|
||||
[
|
||||
("cache", 5, False, "WARNING"),
|
||||
("token", 5, True, "INFO"),
|
||||
# A zero threshold matures a Read in the still-mutable latest turn.
|
||||
("cache", 0, True, "INFO"),
|
||||
],
|
||||
)
|
||||
def test_read_maturation_reports_whether_it_can_fire(
|
||||
mode, quiesce_turns, effective, startup_level, caplog
|
||||
):
|
||||
"""Cache mode freezes a Read before it can quiesce, so enabled maturation
|
||||
never fires there; startup and /stats must say so instead of reporting a
|
||||
silently inert feature as on."""
|
||||
app = _read_maturation_app(mode, quiesce_turns)
|
||||
|
||||
with caplog.at_level("INFO", logger="headroom.proxy"):
|
||||
with TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 12345)) as client:
|
||||
payload = client.get("/stats").json()["read_maturation"]
|
||||
|
||||
assert payload["enabled"] is True
|
||||
assert payload["effective"] is effective
|
||||
assert (payload["inert_reason"] is None) is effective
|
||||
banner = [r for r in caplog.records if r.getMessage().startswith("Read maturation:")]
|
||||
assert [r.levelname for r in banner] == [startup_level]
|
||||
|
||||
Reference in New Issue
Block a user