fix(transforms): honour Subscription toin_read_only when the router calls SmartCrusher (#4493)

## Description

Subscription requests still wrote observations to the shared TOIN store
when the `ContentRouter` compressed a JSON array with SmartCrusher.
`CompressionPolicy.toin_read_only` says Subscription traffic must never
write to TOIN (`compression_policy.py`).

Root cause: SmartCrusher only checks `self._runtime_compression_policy`,
and only `SmartCrusher.apply()` sets that field. The router never calls
`crusher.apply()`. The registry adapter `_invoke_smart_crusher` calls
`crusher.crush()`, so the crusher's policy stays `None` and `crush()`
always reaches `_record_to_toin`. The router's own TOIN record is gated,
but it skips SmartCrusher because "SmartCrusher handles its own TOIN
recording".

Fix: `SmartCrusher.crush()` gets an optional `record_toin: bool = True`.
`_invoke_smart_crusher` reads the router's per-request policy
(ContextVar-backed since #3486) and passes `record_toin=False` when
`toin_read_only` is true. The gate is passed with each call and never
set on the crusher: one crusher instance serves every concurrent
request, so writing to its fields would leak state between requests. The
kwarg is only sent when gating, so duck-typed crushers with the plain
`crush(content, query, bias)` signature keep working.

Scope: this fixes only the first write path in the issue (router →
`SmartCrusher.crush()`). Not changed here:
- `store_kompress_in_ccr` (Kompress CCR store, the issue's second path)
has no policy parameter. It touches the Kompress and remote-Kompress
call sites and should be a separate PR.
- `ConfigCompressor` and `TabularCompressor` each create their own
`SmartCrusher` and call `crush()` without a policy in scope, so they
keep the existing default.
- Direct `crush()` callers keep today's write-enabled default.

Closes #4474

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/transforms/smart_crusher.py`: `crush()` accepts
`record_toin: bool = True` and skips `_record_to_toin` when it is false.
Compression output is unchanged.
- `headroom/transforms/content_router.py`: `_invoke_smart_crusher`
passes `record_toin=False` when the request's `CompressionPolicy` has
`toin_read_only=True`.
- `tests/test_compression_policy_toin_gate.py`: new tests run real
`ContentRouter.apply()` calls through SmartCrusher with Subscription and
PAYG policies. Subscription must make zero TOIN writes and PAYG must
still write. The tests also check that the shared crusher's
`_runtime_compression_policy` stays `None`, and that
`crush(record_toin=False)` skips the write while the default still
records.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

Lint and type checks ran on the changed files only (`ruff check`, `ruff
format --check`, and `mypy --ignore-missing-imports` on the two changed
modules). Before the fix, the new Subscription router test and the
`record_toin` test fail: the router path makes 1 write, and `crush()`
rejects the unknown kwarg. The PAYG control passes on both trees.
`tests/test_content_router_concurrency.py` errors on both the old and
the new tree in this offline sandbox, because it tries to download the
`o200k_base` tiktoken file, so it is not listed below.

### Test Output

```text
$ python -m pytest tests/test_compression_policy_toin_gate.py -q   # before the fix (tests only)
FAILED tests/test_compression_policy_toin_gate.py::test_content_router_smart_crusher_path_honours_toin_read_only[subscription-False]
FAILED tests/test_compression_policy_toin_gate.py::test_smart_crusher_crush_record_toin_false_skips_write
2 failed, 8 passed in 0.85s

$ python -m pytest tests/test_compression_policy_toin_gate.py -q   # with the fix
10 passed in 0.70s

$ python -m pytest tests/test_compression_policy_toin_gate.py tests/test_smart_crusher_toin_attachment.py tests/test_router_registry_smartcrusher.py tests/test_builtin_compressor_adapters.py tests/test_router_registry_dispatch.py tests/test_transforms_content_router.py tests/test_openai_responses_context_compaction.py tests/test_compression_policy.py -q
172 passed, 4 skipped in 9.19s   # skips: tree-sitter / trafilatura not installed

$ ruff check headroom/transforms/content_router.py headroom/transforms/smart_crusher.py tests/test_compression_policy_toin_gate.py
All checks passed!
$ ruff format --check <same files>
3 files already formatted
$ mypy headroom/transforms/content_router.py headroom/transforms/smart_crusher.py --ignore-missing-imports
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: Ubuntu 26.04 x86_64, Python 3.12, offline, no provider;
real `headroom._core` Rust extension
- Exact command / steps: a standalone script (`python repro.py`, outside
pytest) points TOIN at a fresh temp `toin.json`. It runs a real
`ContentRouter().apply()` on one `role=tool` message (a 60-item JSON
array) with `compression_policy=policy_for_mode(AuthMode.SUBSCRIPTION)`,
then again with `AuthMode.PAYG` as the control, and prints the TOIN
`total_compressions` sum after each. A second run used 6 tool messages
so the router's parallel worker path is exercised too.
- Observed result: before the fix, Subscription wrote to TOIN just like
PAYG. After the fix, Subscription writes nothing, PAYG still writes, and
the compression output is identical.

```text
# before (upstream main 3fef15d3)
subscription: toin_read_only=True tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=1
payg: toin_read_only=False tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=1

# after
subscription: toin_read_only=True tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=0
payg: toin_read_only=False tokens 728->143 transforms=['router:smart_crusher:0.20'] toin_writes=1

# after, 6 tool messages (parallel compression workers)
subscription: toin_read_only=True tokens 4482->948 transforms=['router:smart_crusher:0.21', ... x6] toin_writes=0
payg: toin_read_only=False tokens 4482->948 transforms=['router:smart_crusher:0.21', ... x6] toin_writes=6
```

- Not tested: a live proxy in Subscription mode with real provider
traffic; the Kompress `store_kompress_in_ccr` write path (out of scope,
see Description); `tests/test_content_router_concurrency.py` (needs a
tiktoken download, so it errors offline on both trees)

## Runtime Rollout Safety

- Rollout-managed feature(s): none; this is a bug fix on the default
router code path with no rollout flag
- Minimum rollout channel: none; there is no rollout gate, so it ships
with the next normal release
- Stable/default behavior changed: only for requests whose policy has
`toin_read_only=True` (Subscription): router-driven SmartCrusher
compressions stop writing TOIN observations, as documented. Compression
output and all other modes are unchanged.
- Kill switch / disable path: none added; the change only enforces the
existing `CompressionPolicy.toin_read_only` field
- Unsafe override required: no
- Qualification impact: none; compressed output is byte-identical, only
the TOIN learning write is skipped for Subscription requests
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4493"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Ryrenz
2026-10-29 20:08:02 -05:00
committed by GitHub
co-authored by JD Davis
parent a2d4fe7238
commit 7c11d462e6
3 changed files with 70 additions and 2 deletions
+10 -1
View File
@@ -360,9 +360,18 @@ def _invoke_smart_crusher(router: ContentRouter, inp: CompressInput) -> str | No
crusher = router._get_smart_crusher()
if crusher is None:
return None
# F2.2: the router never calls ``crusher.apply()``, so the crusher's own
# policy field stays None. Forward the request's TOIN read-only gate per
# call instead (the crusher is shared across concurrent requests). The
# kwarg is only sent when gating, so crushers exposing the plain
# ``crush(content, query, bias)`` signature keep working.
policy = router._runtime_compression_policy
gate: dict[str, bool] = {}
if policy is not None and policy.toin_read_only:
gate["record_toin"] = False
# ``_get_*`` getters are typed ``Any``; pin the result to the contract type.
compressed: str = crusher.crush(
inp.content, query=inp.query, bias=_adapter_bias(inp)
inp.content, query=inp.query, bias=_adapter_bias(inp), **gate
).compressed
return compressed
+8 -1
View File
@@ -497,6 +497,7 @@ class SmartCrusher(Transform):
query: str = "",
bias: float = 1.0,
lossless_only: bool | None = None,
record_toin: bool = True,
) -> CrushResult:
"""Crush a single JSON content string.
@@ -510,6 +511,12 @@ class SmartCrusher(Transform):
but any path that would need a CCR marker (row-drop or
opaque-blob offload) leaves the content uncompacted instead.
`None` (default) uses the instance's configured value.
`record_toin=False` skips the TOIN write for this call only. The
ContentRouter passes it when the request's ``CompressionPolicy``
has ``toin_read_only=True`` (Subscription, F2.2): the crusher is
shared across concurrent requests, so the gate has to travel with
the call rather than through ``_runtime_compression_policy``.
"""
# Web search tools often return space-separated JSON objects
# (``{...} {...} {...}``) rather than a real array. The Rust crusher
@@ -548,7 +555,7 @@ class SmartCrusher(Transform):
# JSON re-canonicalization (whitespace normalization) without
# actually compressing — the strategy stays `"passthrough"` in
# that case, and there's no learning value in recording it.
if r.was_modified and r.strategy != "passthrough":
if record_toin and r.was_modified and r.strategy != "passthrough":
self._record_to_toin(
original=content,
compressed=r.compressed,
@@ -287,3 +287,55 @@ def test_content_router_payg_records_to_toin(fresh_toin):
assert post >= pre, (
"PAYG must not be blocked by the F2.2 gate — write should happen or fall through naturally"
)
# ─── ContentRouter → SmartCrusher.crush() (#4474) ───────────────────────
@_skip_no_core
@pytest.mark.parametrize(
("mode", "expect_write"),
[(AuthMode.SUBSCRIPTION, False), (AuthMode.PAYG, True)],
)
def test_content_router_smart_crusher_path_honours_toin_read_only(fresh_toin, mode, expect_write):
"""The router compresses JSON arrays via the shared crusher's
``crush()`` and never calls ``crusher.apply()``, so the crusher's
own ``_runtime_compression_policy`` stays None. The router must
forward the request's ``toin_read_only`` gate to that call.
"""
from headroom.transforms.content_router import ContentRouter
router = ContentRouter()
messages = _wrap_in_tool_message(_bigger_array(60))
pre = sum(p.total_compressions for p in fresh_toin._patterns.values())
result = router.apply(messages, _tokenizer(), compression_policy=policy_for_mode(mode))
assert any("smart_crusher" in t for t in result.transforms_applied), (
"payload must route through SmartCrusher for this test to mean anything"
)
# The shared crusher's own field is untouched — the gate travels per call.
assert router._get_smart_crusher()._runtime_compression_policy is None
post = sum(p.total_compressions for p in fresh_toin._patterns.values())
if expect_write:
assert post > pre, "write-enabled policy must still feed TOIN via the router"
else:
assert post == pre, "Subscription MUST NOT write to TOIN via router → crush()"
@_skip_no_core
def test_smart_crusher_crush_record_toin_false_skips_write(fresh_toin):
"""``crush(record_toin=False)`` compresses as usual but skips TOIN;
the default keeps the legacy write-enabled behaviour."""
from headroom.transforms.smart_crusher import SmartCrusher, SmartCrusherConfig
crusher = SmartCrusher(SmartCrusherConfig())
payload = _bigger_array(60)
gated = crusher.crush(payload, record_toin=False)
if not gated.was_modified or gated.strategy == "passthrough":
pytest.skip("payload didn't trigger compression — bump the size")
assert sum(p.total_compressions for p in fresh_toin._patterns.values()) == 0
crusher.crush(payload)
assert sum(p.total_compressions for p in fresh_toin._patterns.values()) > 0