mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(transforms): honour Subscription toin_read_only when the router calls SmartCrusher (#4493)
## Description
Subscription requests still wrote observations to the shared TOIN store
when the `ContentRouter` compressed a JSON array with SmartCrusher.
`CompressionPolicy.toin_read_only` says Subscription traffic must never
write to TOIN (`compression_policy.py`).
Root cause: SmartCrusher only checks `self._runtime_compression_policy`,
and only `SmartCrusher.apply()` sets that field. The router never calls
`crusher.apply()`. The registry adapter `_invoke_smart_crusher` calls
`crusher.crush()`, so the crusher's policy stays `None` and `crush()`
always reaches `_record_to_toin`. The router's own TOIN record is gated,
but it skips SmartCrusher because "SmartCrusher handles its own TOIN
recording".
Fix: `SmartCrusher.crush()` gets an optional `record_toin: bool = True`.
`_invoke_smart_crusher` reads the router's per-request policy
(ContextVar-backed since #3486) and passes `record_toin=False` when
`toin_read_only` is true. The gate is passed with each call and never
set on the crusher: one crusher instance serves every concurrent
request, so writing to its fields would leak state between requests. The
kwarg is only sent when gating, so duck-typed crushers with the plain
`crush(content, query, bias)` signature keep working.
Scope: this fixes only the first write path in the issue (router →
`SmartCrusher.crush()`). Not changed here:
- `store_kompress_in_ccr` (Kompress CCR store, the issue's second path)
has no policy parameter. It touches the Kompress and remote-Kompress
call sites and should be a separate PR.
- `ConfigCompressor` and `TabularCompressor` each create their own
`SmartCrusher` and call `crush()` without a policy in scope, so they
keep the existing default.
- Direct `crush()` callers keep today's write-enabled default.
Closes #4474
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/transforms/smart_crusher.py`: `crush()` accepts
`record_toin: bool = True` and skips `_record_to_toin` when it is false.
Compression output is unchanged.
- `headroom/transforms/content_router.py`: `_invoke_smart_crusher`
passes `record_toin=False` when the request's `CompressionPolicy` has
`toin_read_only=True`.
- `tests/test_compression_policy_toin_gate.py`: new tests run real
`ContentRouter.apply()` calls through SmartCrusher with Subscription and
PAYG policies. Subscription must make zero TOIN writes and PAYG must
still write. The tests also check that the shared crusher's
`_runtime_compression_policy` stays `None`, and that
`crush(record_toin=False)` skips the write while the default still
records.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
Lint and type checks ran on the changed files only (`ruff check`, `ruff
format --check`, and `mypy --ignore-missing-imports` on the two changed
modules). Before the fix, the new Subscription router test and the
`record_toin` test fail: the router path makes 1 write, and `crush()`
rejects the unknown kwarg. The PAYG control passes on both trees.
`tests/test_content_router_concurrency.py` errors on both the old and
the new tree in this offline sandbox, because it tries to download the
`o200k_base` tiktoken file, so it is not listed below.
### Test Output
```text
$ python -m pytest tests/test_compression_policy_toin_gate.py -q # before the fix (tests only)
FAILED tests/test_compression_policy_toin_gate.py::test_content_router_smart_crusher_path_honours_toin_read_only[subscription-False]
FAILED tests/test_compression_policy_toin_gate.py::test_smart_crusher_crush_record_toin_false_skips_write
2 failed, 8 passed in 0.85s
$ python -m pytest tests/test_compression_policy_toin_gate.py -q # with the fix
10 passed in 0.70s
$ python -m pytest tests/test_compression_policy_toin_gate.py tests/test_smart_crusher_toin_attachment.py tests/test_router_registry_smartcrusher.py tests/test_builtin_compressor_adapters.py tests/test_router_registry_dispatch.py tests/test_transforms_content_router.py tests/test_openai_responses_context_compaction.py tests/test_compression_policy.py -q
172 passed, 4 skipped in 9.19s # skips: tree-sitter / trafilatura not installed
$ ruff check headroom/transforms/content_router.py headroom/transforms/smart_crusher.py tests/test_compression_policy_toin_gate.py
All checks passed!
$ ruff format --check <same files>
3 files already formatted
$ mypy headroom/transforms/content_router.py headroom/transforms/smart_crusher.py --ignore-missing-imports
Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: Ubuntu 26.04 x86_64, Python 3.12, offline, no provider;
real `headroom._core` Rust extension
- Exact command / steps: a standalone script (`python repro.py`, outside
pytest) points TOIN at a fresh temp `toin.json`. It runs a real
`ContentRouter().apply()` on one `role=tool` message (a 60-item JSON
array) with `compression_policy=policy_for_mode(AuthMode.SUBSCRIPTION)`,
then again with `AuthMode.PAYG` as the control, and prints the TOIN
`total_compressions` sum after each. A second run used 6 tool messages
so the router's parallel worker path is exercised too.
- Observed result: before the fix, Subscription wrote to TOIN just like
PAYG. After the fix, Subscription writes nothing, PAYG still writes, and
the compression output is identical.
```text
# before (upstream main 3fef15d3)
subscription: toin_read_only=True tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=1
payg: toin_read_only=False tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=1
# after
subscription: toin_read_only=True tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=0
payg: toin_read_only=False tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=1
# after, 6 tool messages (parallel compression workers)
subscription: toin_read_only=True tokens 4482->948 transforms=['router:smart_crusher:0.21', ... x6] toin_writes=0
payg: toin_read_only=False tokens 4482->948 transforms=['router:smart_crusher:0.21', ... x6] toin_writes=6
```
- Not tested: a live proxy in Subscription mode with real provider
traffic; the Kompress `store_kompress_in_ccr` write path (out of scope,
see Description); `tests/test_content_router_concurrency.py` (needs a
tiktoken download, so it errors offline on both trees)
## Runtime Rollout Safety
- Rollout-managed feature(s): none; this is a bug fix on the default
router code path with no rollout flag
- Minimum rollout channel: none; there is no rollout gate, so it ships
with the next normal release
- Stable/default behavior changed: only for requests whose policy has
`toin_read_only=True` (Subscription): router-driven SmartCrusher
compressions stop writing TOIN observations, as documented. Compression
output and all other modes are unchanged.
- Kill switch / disable path: none added; the change only enforces the
existing `CompressionPolicy.toin_read_only` field
- Unsafe override required: no
- Qualification impact: none; compressed output is byte-identical, only
the TOIN learning write is skipped for Subscription requests
- Rollback path: revert this PR
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4493"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -360,9 +360,18 @@ def _invoke_smart_crusher(router: ContentRouter, inp: CompressInput) -> str | No
|
||||
crusher = router._get_smart_crusher()
|
||||
if crusher is None:
|
||||
return None
|
||||
# F2.2: the router never calls ``crusher.apply()``, so the crusher's own
|
||||
# policy field stays None. Forward the request's TOIN read-only gate per
|
||||
# call instead (the crusher is shared across concurrent requests). The
|
||||
# kwarg is only sent when gating, so crushers exposing the plain
|
||||
# ``crush(content, query, bias)`` signature keep working.
|
||||
policy = router._runtime_compression_policy
|
||||
gate: dict[str, bool] = {}
|
||||
if policy is not None and policy.toin_read_only:
|
||||
gate["record_toin"] = False
|
||||
# ``_get_*`` getters are typed ``Any``; pin the result to the contract type.
|
||||
compressed: str = crusher.crush(
|
||||
inp.content, query=inp.query, bias=_adapter_bias(inp)
|
||||
inp.content, query=inp.query, bias=_adapter_bias(inp), **gate
|
||||
).compressed
|
||||
return compressed
|
||||
|
||||
|
||||
@@ -497,6 +497,7 @@ class SmartCrusher(Transform):
|
||||
query: str = "",
|
||||
bias: float = 1.0,
|
||||
lossless_only: bool | None = None,
|
||||
record_toin: bool = True,
|
||||
) -> CrushResult:
|
||||
"""Crush a single JSON content string.
|
||||
|
||||
@@ -510,6 +511,12 @@ class SmartCrusher(Transform):
|
||||
but any path that would need a CCR marker (row-drop or
|
||||
opaque-blob offload) leaves the content uncompacted instead.
|
||||
`None` (default) uses the instance's configured value.
|
||||
|
||||
`record_toin=False` skips the TOIN write for this call only. The
|
||||
ContentRouter passes it when the request's ``CompressionPolicy``
|
||||
has ``toin_read_only=True`` (Subscription, F2.2): the crusher is
|
||||
shared across concurrent requests, so the gate has to travel with
|
||||
the call rather than through ``_runtime_compression_policy``.
|
||||
"""
|
||||
# Web search tools often return space-separated JSON objects
|
||||
# (``{...} {...} {...}``) rather than a real array. The Rust crusher
|
||||
@@ -548,7 +555,7 @@ class SmartCrusher(Transform):
|
||||
# JSON re-canonicalization (whitespace normalization) without
|
||||
# actually compressing — the strategy stays `"passthrough"` in
|
||||
# that case, and there's no learning value in recording it.
|
||||
if r.was_modified and r.strategy != "passthrough":
|
||||
if record_toin and r.was_modified and r.strategy != "passthrough":
|
||||
self._record_to_toin(
|
||||
original=content,
|
||||
compressed=r.compressed,
|
||||
|
||||
@@ -287,3 +287,55 @@ def test_content_router_payg_records_to_toin(fresh_toin):
|
||||
assert post >= pre, (
|
||||
"PAYG must not be blocked by the F2.2 gate — write should happen or fall through naturally"
|
||||
)
|
||||
|
||||
|
||||
# ─── ContentRouter → SmartCrusher.crush() (#4474) ───────────────────────
|
||||
|
||||
|
||||
@_skip_no_core
|
||||
@pytest.mark.parametrize(
|
||||
("mode", "expect_write"),
|
||||
[(AuthMode.SUBSCRIPTION, False), (AuthMode.PAYG, True)],
|
||||
)
|
||||
def test_content_router_smart_crusher_path_honours_toin_read_only(fresh_toin, mode, expect_write):
|
||||
"""The router compresses JSON arrays via the shared crusher's
|
||||
``crush()`` and never calls ``crusher.apply()``, so the crusher's
|
||||
own ``_runtime_compression_policy`` stays None. The router must
|
||||
forward the request's ``toin_read_only`` gate to that call.
|
||||
"""
|
||||
from headroom.transforms.content_router import ContentRouter
|
||||
|
||||
router = ContentRouter()
|
||||
messages = _wrap_in_tool_message(_bigger_array(60))
|
||||
pre = sum(p.total_compressions for p in fresh_toin._patterns.values())
|
||||
|
||||
result = router.apply(messages, _tokenizer(), compression_policy=policy_for_mode(mode))
|
||||
|
||||
assert any("smart_crusher" in t for t in result.transforms_applied), (
|
||||
"payload must route through SmartCrusher for this test to mean anything"
|
||||
)
|
||||
# The shared crusher's own field is untouched — the gate travels per call.
|
||||
assert router._get_smart_crusher()._runtime_compression_policy is None
|
||||
post = sum(p.total_compressions for p in fresh_toin._patterns.values())
|
||||
if expect_write:
|
||||
assert post > pre, "write-enabled policy must still feed TOIN via the router"
|
||||
else:
|
||||
assert post == pre, "Subscription MUST NOT write to TOIN via router → crush()"
|
||||
|
||||
|
||||
@_skip_no_core
|
||||
def test_smart_crusher_crush_record_toin_false_skips_write(fresh_toin):
|
||||
"""``crush(record_toin=False)`` compresses as usual but skips TOIN;
|
||||
the default keeps the legacy write-enabled behaviour."""
|
||||
from headroom.transforms.smart_crusher import SmartCrusher, SmartCrusherConfig
|
||||
|
||||
crusher = SmartCrusher(SmartCrusherConfig())
|
||||
payload = _bigger_array(60)
|
||||
|
||||
gated = crusher.crush(payload, record_toin=False)
|
||||
if not gated.was_modified or gated.strategy == "passthrough":
|
||||
pytest.skip("payload didn't trigger compression — bump the size")
|
||||
assert sum(p.total_compressions for p in fresh_toin._patterns.values()) == 0
|
||||
|
||||
crusher.crush(payload)
|
||||
assert sum(p.total_compressions for p in fresh_toin._patterns.values()) > 0
|
||||
|
||||
Reference in New Issue
Block a user