fix(proxy): report read maturation as inert in cache mode (#4198)

## Description

`--read-maturation` combined with `--mode cache` starts cleanly, and
`/stats.rollout` reports the feature as requested and enabled. It can
never fire, and nothing says so.

Here is why:

- In cache mode the Anthropic handler sets `frozen_message_count =
_strict_previous_turn_frozen_count(...)` (`handlers/anthropic.py`). That
freezes every message except the final user turn.
- Cache mode never reaches the token-mode `prepare_turn` clamp, so that
count is passed straight to `ReadMaturationManager.apply(...,
frozen_message_count=...)`.
- `apply` skips every message with `i < frozen_message_count`.
- The only Reads it ever examines are therefore the tool results in the
latest user turn. Those answer the assistant message just before, so
`quiet_turns = held_turns = 0`. That is below `quiesce_turns` (default
5) and `max_hold_turns` (default 25), so they are always held.
- By the time a Read could have quiesced, it sits in the frozen prefix.

The handler's per-request summary is logged at DEBUG, so a deployment
sees no output either way. One persistent proxy ran this combination for
over five weeks, about 51,000 real requests, with zero
`read_maturation:` transforms. The existing handler tests all use
`mode="token"`, so no test caught it.

Holding is harmless in cache mode, because a "held" Read is forwarded
verbatim. So this PR makes the inert state visible rather than changing
request handling:

- At startup, enabled read maturation is reported once: a **WARNING**
`Read maturation: INERT (cache mode freezes a Read before it can
quiesce); use --mode token or disable read_maturation` in cache mode,
otherwise an INFO line with the effective quiesce, hold and size
settings. The hint says "disable read_maturation" rather than naming a
flag, because the feature can come from `--read-maturation` or from
`HEADROOM_FEATURES`.
- `/stats` gains `read_maturation: {enabled, effective, inert_reason}`,
beside `rollout`, which only records that the feature was granted.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/modes.py`: `read_maturation_inert_reason(mode, *,
quiesce_turns, max_hold_turns)` returns the reason string under cache
mode, else `None`. A threshold of 0, which the env path
(`HEADROOM_READ_MATURATION_*_TURNS`) accepts even though the CLI
requires at least 1, matures a Read in the still-mutable latest turn, so
it is not reported as inert. The rule lives in one place for both
callers.
- `headroom/proxy/server.py` (startup banner): after the cache-mode
lines, log the INERT WARNING or an ENABLED INFO line with
`quiesce_turns` / `max_hold_turns` / `min_size_bytes`.
- `headroom/proxy/server.py` (`/stats`): add `read_maturation: {enabled,
effective, inert_reason}` through a small `_read_maturation_status(cfg)`
helper.
- `tests/test_proxy_stats_recent_requests.py`:
`test_read_maturation_reports_whether_it_can_fire`, parametrized over
cache, token, and cache with `quiesce_turns=0`. It asserts the `/stats`
fields and that exactly one `Read maturation:` startup record is logged,
at WARNING (inert) or INFO (effective).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_proxy_stats_recent_requests.py -k read_maturation -v
tests/test_proxy_stats_recent_requests.py::test_read_maturation_reports_whether_it_can_fire[cache-5-False-WARNING] PASSED [ 33%]
tests/test_proxy_stats_recent_requests.py::test_read_maturation_reports_whether_it_can_fire[token-5-True-INFO] PASSED [ 66%]
tests/test_proxy_stats_recent_requests.py::test_read_maturation_reports_whether_it_can_fire[cache-0-True-INFO] PASSED [100%]
================= 3 passed, 5 deselected, 2 warnings in 1.74s ==================
# all three cases fail on main (no "read_maturation" key in /stats, no startup record)

$ pytest tests/test_proxy_stats_recent_requests.py tests/test_read_maturation*.py
======================== 40 passed, 3 warnings in 4.86s ========================

$ ruff check headroom/proxy/modes.py headroom/proxy/server.py tests/test_proxy_stats_recent_requests.py
All checks passed!
$ ruff format --check headroom/proxy/modes.py headroom/proxy/server.py tests/test_proxy_stats_recent_requests.py
3 files already formatted
$ mypy headroom/proxy/modes.py headroom/proxy/server.py
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.13. A persistent `headroom proxy` (0.38 →
0.40.0) serving Anthropic Messages traffic from an agentic coding
client, with `--mode cache --lossless --code-aware --read-maturation`,
`HEADROOM_ROLLOUT_CHANNEL=beta` and default maturation thresholds.
- Exact command / steps: (1) `GET /stats` on the running proxy. (2)
Counted `PERF` records and `read_maturation:` transforms in the proxy's
archived PERF log from 2026-09-05 to 2026-10-13. (3) Traced
`frozen_message_count` from `_strict_previous_turn_frozen_count` into
`ReadMaturationManager.apply`.
- Observed result: (1) `rollout` reports `read_maturation:
requested=True, enabled=True, decision=explicit` on channel `beta`, so
gating is not the cause. (2) 50,970 PERF records and **0**
`read_maturation:` transforms from real traffic; the only 5 hits in the
window were test-fixture requests. (3) Only the final user turn is ever
unfrozen, so every Read examined is at 0 quiet turns and is held.
- Not tested: a full restart of that deployment on this branch. The
startup line and the `/stats` block were exercised through `create_app`
plus `TestClient` in the new test. I also did not try to make maturation
actually work in cache mode; see Additional Notes.

## Runtime Rollout Safety

- Rollout-managed feature(s): `read_maturation` (beta). Gating is
unchanged; this only reports on it.
- Minimum rollout channel: unchanged (beta).
- Stable/default behavior changed: no request-path change. `effective`
describes the mode and thresholds only; maturation runs in the Anthropic
messages handler and skips bypassed requests. One new startup log line
appears when read maturation is enabled, and `/stats` gains one additive
key.
- Kill switch / disable path: the existing one; drop `--read-maturation`
/ `HEADROOM_FEATURES`.
- Unsafe override required: no.
- Qualification impact: `/stats` gains `read_maturation`; existing keys,
including `rollout`, are unchanged. Black-box tools reading `rollout`
see the same payload.
- Rollback path: revert the commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- **Why a warning, not a refusal:** existing deployments run this
combination, and failing startup on upgrade would take them down for a
feature that is a no-op there. Happy to switch to refusing the
combination if maintainers prefer.
- **Possible follow-up, not in this PR:** cache mode does have one
moment when rewriting the prefix is free, which is cold-prefix
recompaction (`HEADROOM_COLD_RECOMPACT`, after the cache has lapsed).
Letting maturation run there would give it a real effect in cache mode.
That is a design change, and I'd rather discuss it first.
- **Also possible:** per-reason skip counters (frozen, below min size,
holding, already marked) in `MaturationResult`, surfaced in `/stats`. I
left this out to keep the PR small.
- **Docs:** I didn't find a page describing `--read-maturation`'s mode
interaction. Happy to add a line wherever maintainers want it.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4198"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
M Barry
2026-10-15 16:31:05 -05:00
committed by GitHub
co-authored by JD Davis
parent ca6ba3486c
commit ba824539d6
3 changed files with 100 additions and 0 deletions
+14
View File
@@ -36,3 +36,17 @@ def is_token_mode(mode: str | None) -> bool:
def is_cache_mode(mode: str | None) -> bool:
"""Return True when mode resolves to cache mode."""
return normalize_proxy_mode(mode) == PROXY_MODE_CACHE
def read_maturation_inert_reason(
mode: str | None, *, quiesce_turns: int, max_hold_turns: int
) -> str | None:
"""Why read maturation can never fire under ``mode``, else None.
Cache mode freezes every turn but the latest, and maturation skips frozen
messages. A Read in the latest turn has had no quiet turns yet, so it is
held unless a threshold is 0; by the time it could quiesce it is frozen.
"""
if is_cache_mode(mode) and quiesce_turns > 0 and max_hold_turns > 0:
return "cache mode freezes a Read before it can quiesce"
return None
+39
View File
@@ -163,6 +163,7 @@ from headroom.proxy.modes import (
PROXY_MODE_TOKEN,
is_token_mode,
normalize_proxy_mode,
read_maturation_inert_reason,
)
from headroom.proxy.orphan_watchdog import (
orphan_grace_seconds,
@@ -2092,6 +2093,25 @@ class HeadroomProxy(
if self.config.mode == PROXY_MODE_CACHE:
logger.info(" Prefix freeze: strict (all prior turns immutable)")
logger.info(" Mutations: latest turn only")
if self.config.read_maturation:
_inert = read_maturation_inert_reason(
self.config.mode,
quiesce_turns=self.config.read_maturation_quiesce_turns,
max_hold_turns=self.config.read_maturation_max_hold_turns,
)
if _inert:
logger.warning(
"Read maturation: INERT (%s); use --mode token or disable read_maturation",
_inert,
)
else:
logger.info(
"Read maturation: ENABLED (quiesce_turns=%d max_hold_turns=%d "
"min_size_bytes=%d)",
self.config.read_maturation_quiesce_turns,
self.config.read_maturation_max_hold_turns,
self.config.read_maturation_min_size_bytes,
)
# Effective compression posture, resolved (not merely requested). The
# savings profile reaches the router through two paths — the config
# object and the seeded process env — and `setdefault` semantics mean a
@@ -5387,6 +5407,25 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
# Effective state from the running process. This is the supported
# black-box provenance surface for benchmark/qualification tools.
"rollout": proxy.config.rollout.to_dict() if proxy.config.rollout else None,
"read_maturation": _read_maturation_status(proxy.config),
}
def _read_maturation_status(cfg: ProxyConfig) -> dict[str, Any]:
# The rollout snapshot says whether the feature is enabled, not whether
# it can act; cache mode leaves it enabled but unable to fire.
inert = (
read_maturation_inert_reason(
cfg.mode,
quiesce_turns=cfg.read_maturation_quiesce_turns,
max_hold_turns=cfg.read_maturation_max_hold_turns,
)
if cfg.read_maturation
else None
)
return {
"enabled": cfg.read_maturation,
"effective": cfg.read_maturation and inert is None,
"inert_reason": inert,
}
def _dashboard_config_payload() -> dict[str, Any]:
+47
View File
@@ -277,3 +277,50 @@ def test_stats_preserves_default_smart_crusher_compaction_state() -> None:
assert response.status_code == 200
assert response.json()["config"]["smart_crusher_with_compaction"] is None
def _read_maturation_app(mode: str, quiesce_turns: int = 5):
return create_app(
ProxyConfig(
mode=mode,
read_maturation=True,
read_maturation_quiesce_turns=quiesce_turns,
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
http2=False,
)
)
@pytest.mark.parametrize(
("mode", "quiesce_turns", "effective", "startup_level"),
[
("cache", 5, False, "WARNING"),
("token", 5, True, "INFO"),
# A zero threshold matures a Read in the still-mutable latest turn.
("cache", 0, True, "INFO"),
],
)
def test_read_maturation_reports_whether_it_can_fire(
mode, quiesce_turns, effective, startup_level, caplog
):
"""Cache mode freezes a Read before it can quiesce, so enabled maturation
never fires there; startup and /stats must say so instead of reporting a
silently inert feature as on."""
app = _read_maturation_app(mode, quiesce_turns)
with caplog.at_level("INFO", logger="headroom.proxy"):
with TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 12345)) as client:
payload = client.get("/stats").json()["read_maturation"]
assert payload["enabled"] is True
assert payload["effective"] is effective
assert (payload["inert_reason"] is None) is effective
banner = [r for r in caplog.records if r.getMessage().startswith("Read maturation:")]
assert [r.levelname for r in banner] == [startup_level]