feat(router): elide dense machine-generated lines no compressor can shrink (#3685)

## Description

Tool outputs that dump a fetched web page, a bundled asset or an encoded
blob (minified JS/CSS, base64, RSC payloads) arrive as a few very long
lines with almost no whitespace. No structural compressor can do
anything with them: `code_aware` is disabled, `log`/`text` return them
unchanged, and the HTML extractor returns either the whole thing or `""`
(which the unit layer then rejects as an empty block and keeps all of it
verbatim). The block rides through the router at ratio 1.0. On a Codex
web-research session these were the four largest tool outputs (75k
tokens) and the whole session compressed at 1.3% of new input.

This adds a last-resort pass in `_apply_strategy_to_content`: after the
strategy chain (and on the passthrough path), every line that is >= 300
chars with < 6% spaces keeps its head and tail and the middle becomes a
marker. Prose, code, logs, indented JSON, CSV and markdown all have far
more spaces and are never touched; JSON-shaped lines are left to
SmartCrusher. The pre-elision block goes into the CCR store and a
`Retrieve original: hash=` marker is appended, so the messages path's
marker-less lossy gate (#1307) keeps the result and the agent can
recover exact bytes. Runs only on strategies whose result is still plain
text (`html`, `log`, `text`, `code_aware`, `search`, `passthrough`),
never after SmartCrusher/tabular/config/diff/Kompress, and never in
lossless mode. New config flag `enable_dense_line_elision`, default on.

Also: an HTML extraction that comes back `""` on non-empty input is now
treated as "nothing extracted" (fall through) rather than "compressed to
zero tokens".

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Performance improvement

## Changes Made

- `headroom/transforms/dense_line_elider.py`: `is_dense_line` /
`elide_dense_lines`, byte-identical no-op when nothing is dense.
- `headroom/transforms/content_router.py`: `enable_dense_line_elision`
config, `_DENSE_ELIDE_AFTER` strategy set, `_elide_dense` (CCR store +
marker), hook after the strategy chain and on the passthrough path,
empty-HTML-extraction fallthrough.
- `tests/test_transforms/test_dense_line_elider.py`: 8 tests
(prose/code/JSON untouched, elision shape, router hook on/off, marker
retrievable from the CCR store, survives the messages-path lossy gate,
lossless mode never elides, search output keeps line structure).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run --frozen --extra dev pytest tests/test_transforms/test_dense_line_elider.py -q -> 8 passed
uv run --frozen --extra dev pytest tests/test_transforms/ tests/test_no_ccr_lossy.py tests/test_transforms_content_router.py tests/test_openai_responses_compression_units.py tests/test_code_aware_regressions.py tests/test_router_registry_dispatch.py tests/test_router_external_dispatch.py tests/test_compression_observability.py tests/test_openai_responses_context_compaction.py tests/test_agent_savings.py tests/test_openai_responses_t3_replay_regression.py -q -> 588 passed, 72 skipped
uv run --frozen --extra dev pytest tests/ --ignore=tests/e2e -q -> 9 failed, 12619 passed, 597 skipped (see note below)
uvx ruff check headroom/transforms tests/test_transforms/test_dense_line_elider.py  -> All checks passed!
uv run --frozen --extra dev mypy headroom/transforms/content_router.py headroom/transforms/dense_line_elider.py -> Success: no issues found in 2 source files
```

The 9 full-suite failures are unrelated to this change:
`tests/test_provider_grok.py` (2) and `tests/test_cli/test_wrap_grok.py`
fail because this shell exports `GROK_CLI_CHAT_PROXY_BASE_URL` (all pass
with it unset); the other 7 (`test_proxy_health.py` readyz/kompress x4,
`test_read_maturation_handler_nobust.py` x2,
`test_providers/test_deepseek.py` x1) are order-dependent, pass when run
in isolation, and fail identically with `upstream/main`'s
`content_router.py` swapped in.

## Real Behavior Proof

- Environment: this branch, `HeadroomProxy(ProxyConfig(stateless=True,
mode="token"))`, `HEADROOM_DISABLE_KOMPRESS=1`, two real
`~/.codex/sessions/*.jsonl` rollouts replayed through
`_compress_openai_responses_payload` one request per tool-output item
(`input=items[:i+1]`), novel `tokens_saved` (this pass minus previous
pass) over tiktoken of the items added since the previous request. Same
script with `elide_dense_lines` monkeypatched to a no-op for the "off"
arm.
- Exact command / steps: `HEADROOM_ELIDE_TEST=off|on uv run --frozen
--extra dev python replay_onoff.py <rollout.jsonl> <n_outputs> <model>`
- Observed result: see the block below (web-research session 1.3% ->
15.3% of new input, coding session unchanged at 34.6%).

```text
2026-09-09 gpt-5.6-sol elision off: novel=4377  new_input=344023 rate=1.3%
2026-09-09 gpt-5.6-sol elision on:  novel=52737 new_input=344023 rate=15.3%
2026-09-07 gpt-6-astra elision off: novel=51520 new_input=148924 rate=34.6%
2026-09-07 gpt-6-astra elision on:  novel=51520 new_input=148924 rate=34.6%
```

The web-research session (curl'd pages, truncated minified bundles) goes
1.3% -> 15.3%; the coding session (code-mode JSON chunk outputs, no
dense lines) is byte-for-byte unchanged. Across 14 local Codex sessions,
dense lines are 13.5% of all tool-output characters (0% to 54% per
session), so the fleet effect will be uneven and largest for
research-heavy users. What remains un-elided on the research session is
inside `embedded_json` spans (SSE `data: {...}` lines) and is excluded
on purpose: eliding inside those would corrupt the JSON.

- Not tested: live proxy traffic with a real Codex or Claude Code client
(replay only); Kompress enabled alongside elision (replays ran with
`HEADROOM_DISABLE_KOMPRESS=1`, matching the desktop's production
config); a CCR `/v1/retrieve` round trip over HTTP (the
store-and-retrieve check is in-process in the tests).

Desktop-side: measured 2026-09-20 on the `user_daily_savings` rollup,
Codex-dominant users sit at a median 6.5% strict input rate (n=341
user-days) vs 25.7% for Claude Code (n=478) at identical cache coverage;
this is the largest single lever found for that gap.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes.
`ContentRouterConfig.enable_dense_line_elision` defaults to on, so
blocks with dense machine-generated lines that previously passed through
at ratio 1.0 are now elided with a CCR retrieval marker. Blocks with no
such lines are byte-identical to before.
- Kill switch / disable path:
`ContentRouterConfig(enable_dense_line_elision=False)`; `lossless=True`
also disables it.
- Unsafe override required: No.
- Qualification impact: None; the elision is a deterministic post-pass
and does not touch routing decisions, strategy selection or the
frozen-prefix logic.
- Rollback path: Revert this PR, or set
`enable_dense_line_elision=False`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Documentation: no docs page lists `ContentRouterConfig` flags today, so
there is nothing to update; happy to add one if you want the flag
documented.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
gglucass
2026-09-21 09:38:51 -07:00
committed by GitHub
co-authored by Claude Fable 5.1
parent a7858fd2ae
commit 9263b42099
3 changed files with 304 additions and 0 deletions
+91
View File
@@ -83,6 +83,7 @@ from .content_detector import (
_try_detect_structured_config,
)
from .content_detector import detect_content_type as _regex_detect_content_type
from .dense_line_elider import elide_dense_lines
from .error_detection import content_has_strong_error_indicators
from .lossless_provider import (
get_lossless_generation,
@@ -1602,6 +1603,10 @@ class ContentRouterConfig:
enable_tabular_compressor: bool = True # CSV/TSV/markdown tables via SmartCrusher
enable_config_compressor: bool = True # YAML/TOML/INI structural compression
enable_html_extractor: bool = True # HTML content extraction
# Last-resort fallback for blocks no compressor could shrink: elide the
# middle of long, whitespace-free lines (minified JS/CSS, base64, RSC
# payloads). See ``dense_line_elider``.
enable_dense_line_elision: bool = True
enable_image_optimizer: bool = True # Image token optimization
# Routing preferences
@@ -1933,6 +1938,21 @@ class ContentRouter(Transform):
# the memo O(1) and unlockable at the cost of an occasional cold start.
_LOSSLESS_FIRST_MEMO_MAX = 256
# Strategies whose result is still plain text, so dense-line elision can
# run on top of them (see ``_elide_dense``).
_DENSE_ELIDE_AFTER = frozenset(
{
CompressionStrategy.HTML,
CompressionStrategy.LOG,
CompressionStrategy.TEXT,
CompressionStrategy.CODE_AWARE,
# grep-style output is line-oriented (path, then ``N:match``), so a
# dense match line can be elided without breaking the format.
CompressionStrategy.SEARCH,
CompressionStrategy.PASSTHROUGH,
}
)
def __init__(
self,
config: ContentRouterConfig | None = None,
@@ -3766,6 +3786,13 @@ class ContentRouter(Transform):
compressed = (
output.content if output is not None and output.compressed else None
)
# A page that is all <script>/<style> extracts to "".
# That is "nothing extracted" (fall through to the
# passthrough / dense-line path), not a compression
# to zero tokens: the unit layer rejects an empty
# block and keeps the original verbatim.
if compressed is not None and not compressed.strip():
compressed = None
# Estimate tokens from extracted text (simple word count)
compressed_tokens = _estimate_tokens(compressed) if compressed else 0
decision_reason = "html_extractor"
@@ -3944,6 +3971,21 @@ class ContentRouter(Transform):
# Re-narrow for mypy: all reassignments above produce str, but
# mypy 1.14.x widens after nested try/except/else reassignments.
assert compressed is not None
# Dense-line elision: whatever survived the chain, long
# whitespace-free lines (minified bundles, base64, RSC payloads)
# are still in it, because no structural compressor understands
# them. Runs on the RESULT so it composes with a partial win
# (code_aware at 0.95 on minified JS) as well as a no-op.
# Only after strategies that hand back the text AS TEXT. SmartCrusher,
# tabular, config and diff emit their own structured (and
# CCR-marked) forms; Kompress is lossy with its own marker. Eliding
# inside those would corrupt output another compressor owns.
if self.config.enable_dense_line_elision and actual_strategy in self._DENSE_ELIDE_AFTER:
dense = self._elide_dense(compressed, context)
if dense is not None and dense[1] < compressed_tokens:
compressed, compressed_tokens = dense
strategy_chain.append("dense_elide")
decision_reason = f"{decision_reason}_dense_elide"
if logger.isEnabledFor(logging.DEBUG):
_log_router_debug(
"content_router_strategy_result",
@@ -3978,6 +4020,21 @@ class ContentRouter(Transform):
return compressed, compressed_tokens, strategy_chain
# Fallback: return unchanged
if self.config.enable_dense_line_elision:
dense = self._elide_dense(content, context)
if dense is not None and dense[1] < original_tokens:
elided, elided_tokens = dense
strategy_chain.append("dense_elide")
self._record_to_toin(
strategy=strategy,
content=content,
compressed=elided,
original_tokens=original_tokens,
compressed_tokens=elided_tokens,
language=language,
context=context,
)
return elided, elided_tokens, strategy_chain
strategy_chain.append(CompressionStrategy.PASSTHROUGH.value)
if logger.isEnabledFor(logging.DEBUG):
_log_router_debug(
@@ -4001,6 +4058,40 @@ class ContentRouter(Transform):
)
return content, original_tokens, strategy_chain
def _elide_dense(self, text: str, context: str) -> tuple[str, int] | None:
"""Dense-line elision with a CCR retrieval marker; None when no line is dense.
The elided middle is lossy, so the pre-elision block is stored in the
CCR store and a ``Retrieve original: hash=`` marker is appended, the
same contract Kompress honours: the messages path discards a lossy
result that carries no marker (#1307), and the agent can get the
exact bytes back if it turns out to need them.
"""
# Lossy by nature: never in lossless mode, where nothing may be dropped.
if self.config.lossless:
return None
elided, n_dense = elide_dense_lines(text)
if not n_dense:
return None
if self.config.ccr_inject_marker:
try:
from ..cache.compression_store import get_compression_store
key = get_compression_store().store(
text,
elided,
original_tokens=_estimate_tokens(text),
compressed_tokens=_estimate_tokens(elided),
query_context=context or None,
compression_strategy="dense_elide",
)
noun = "line" if n_dense == 1 else "lines"
elided += f"\n[{n_dense} dense machine-generated {noun} elided. Retrieve original: hash={key}]"
except Exception as exc: # noqa: BLE001 - store optional; keep the block verbatim
logger.debug("dense-line elision: CCR store unavailable (%s); skipping", exc)
return None
return elided, _estimate_tokens(elided)
def _try_ml_compressor(
self,
content: str,
+74
View File
@@ -0,0 +1,74 @@
"""Elide dense machine-generated lines (minified JS/CSS, base64, RSC payloads).
Tool outputs that dump a fetched web page, a bundled asset or an encoded blob
arrive as a few very long lines with almost no whitespace. None of the
structural compressors can do anything with them: they are not code an AST
walker can prune, not a log, not a table, and the HTML extractor returns the
whole thing (or nothing) once the page is mostly ``<script>``. The result is a
multi-thousand-token block that rides through the router at ratio 1.0.
This module keeps a head and a tail of each such line and replaces the middle
with a one-line marker. It is deliberately dumb: a line is "dense" when it is
long AND has a tiny fraction of spaces. Prose, code, logs, JSON with any
indentation, CSV and markdown all have far more than 6% spaces, so they are
never touched, a JSON-shaped line is left to SmartCrusher, and a lone dense
value (JWT, signed URL, PATH) is below the per-block minimum. Measured on a replayed Codex web-research session against the
installed 0.37.0 wheel: novel savings went from 7.2% to 25.1% of new input, on
top of (not instead of) what the router already removed.
"""
from __future__ import annotations
MIN_LINE_CHARS = 300
MAX_SPACE_RATIO = 0.06
# A single dense line (a JWT, a signed URL, a PATH, an RSA modulus) is a value
# the agent asked for, not a dump: a block is only elided when its dense lines
# add up to at least this many chars. Real bundle dumps are tens of KB.
MIN_DENSE_TOTAL_CHARS = 2000
HEAD_CHARS = 160
TAIL_CHARS = 80
def is_dense_line(line: str) -> bool:
"""True when ``line`` is long, nearly whitespace-free, and not JSON-shaped.
A compact JSON value (or a run of them) on one line is also nearly
whitespace-free, but that is SmartCrusher's job and it does it losslessly;
the elider must never pre-empt it.
"""
n = len(line)
# Tabs: TSV / ``psql -A`` rows pass the space ratio but are data the agent
# asked for; minified assets and encoded blobs never carry tabs.
if n < MIN_LINE_CHARS or "\t" in line or (line.count(" ") / n) >= MAX_SPACE_RATIO:
return False
stripped = line.strip()
return not (stripped[:1] in "{[" and stripped[-1:] in "}]")
def elide_dense_lines(text: str) -> tuple[str, int]:
"""Return ``(text_with_dense_lines_elided, lines_elided)``.
Byte-identical to the input (and ``0``) when no line is dense, so callers
can use ``elided != text`` as the "did anything" signal. Line endings are
preserved: the split is on ``"\\n"`` only, so ``"\\r"`` stays on its line.
"""
if len(text) < MIN_LINE_CHARS:
return text, 0
lines = text.split("\n")
if sum(len(line) for line in lines if is_dense_line(line)) < MIN_DENSE_TOTAL_CHARS:
return text, 0
out: list[str] = []
n_elided = 0
for line in lines:
if is_dense_line(line):
omitted = len(line) - HEAD_CHARS - TAIL_CHARS
out.append(
f"{line[:HEAD_CHARS]} ...[{omitted} chars of dense "
f"machine-generated content elided]... {line[-TAIL_CHARS:]}"
)
n_elided += 1
else:
out.append(line)
if n_elided == 0:
return text, 0
return "\n".join(out), n_elided
@@ -0,0 +1,139 @@
"""Dense-line elision: minified/base64/RSC lines that no compressor can shrink."""
from __future__ import annotations
import pytest
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
from headroom.transforms.dense_line_elider import elide_dense_lines
@pytest.fixture
def tokenizer():
from headroom.providers import OpenAIProvider
from headroom.tokenizer import Tokenizer
return Tokenizer(OpenAIProvider().get_token_counter("gpt-4o"), "gpt-4o")
MINIFIED_JS = (
"!function(){var e=window,t=e.document,n=t.createElement('div');"
"n.className='x';for(var r=0;r<1e3;r++){n.appendChild(t.createTextNode(r))}"
"e.__x=n;var a=e.location.search.replace(/^\\?/,'').split('&');"
) * 16
BASE64 = (
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="
* 12
)
PROSE = "This is an ordinary paragraph with plenty of spaces in it, " * 12
CODE = "\n".join(f" value_{i} = compute(arg_{i}, other_{i}) # comment {i}" for i in range(40))
def test_prose_code_and_short_lines_untouched():
for text in (PROSE, CODE, "short", "a" * 299):
assert elide_dense_lines(text) == (text, 0)
def test_dense_lines_elided_others_kept():
text = f"Script completed\nOutput:\n{MINIFIED_JS}\n{PROSE}\n{BASE64}\n"
out, n = elide_dense_lines(text)
assert n == 2
lines = out.split("\n")
assert lines[0] == "Script completed" and lines[3] == PROSE and lines[5] == ""
assert lines[2].startswith(MINIFIED_JS[:160]) and lines[2].endswith(MINIFIED_JS[-80:])
assert "chars of dense machine-generated content elided" in lines[2]
assert len(out) < len(text) // 2
def test_router_applies_elision_when_no_compressor_wins():
html = (
"<!doctype html><html><head><script>" + MINIFIED_JS + "</script></head><body></body></html>"
)
router = ContentRouter(ContentRouterConfig(enable_kompress=False, min_section_tokens=10))
result = router.compress(html, context="tool_result")
assert "content elided" in result.compressed
assert len(result.compressed) < len(html) // 2
assert result.compressed.strip(), "must never blank a non-empty block"
off = ContentRouter(
ContentRouterConfig(
enable_kompress=False, enable_dense_line_elision=False, min_section_tokens=10
)
)
assert "content elided" not in off.compress(html, context="tool_result").compressed
def test_elided_block_carries_a_retrievable_marker():
from headroom.cache.compression_store import get_compression_store
html = "<html><head><script>" + MINIFIED_JS + "</script></head><body>hi there</body></html>"
router = ContentRouter(ContentRouterConfig(enable_kompress=False, min_section_tokens=10))
out = router.compress(html, context="tool_result").compressed
assert "Retrieve original: hash=" in out
key = out.rsplit("hash=", 1)[1].rstrip("]\n")
assert get_compression_store().retrieve(key) is not None
def test_messages_path_keeps_elided_tool_result(tokenizer):
"""The Claude path discards marker-less lossy results (#1307); elision must survive it."""
body = (
"<!doctype html><html><head><script>"
+ MINIFIED_JS * 3
+ "</script></head><body><p>x</p></body></html>"
)
messages = [
{"role": "user", "content": "check the site"},
{
"role": "assistant",
"content": [
{"type": "tool_use", "id": "t1", "name": "Bash", "input": {"command": "curl -s x"}}
],
},
{
"role": "user",
"content": [{"type": "tool_result", "tool_use_id": "t1", "content": body}],
},
{"role": "assistant", "content": "ok"},
{"role": "user", "content": "and now?"},
]
router = ContentRouter(ContentRouterConfig(enable_kompress=False, min_section_tokens=10))
result = router.apply(messages, tokenizer)
block = result.messages[2]["content"][0]["content"]
text = block if isinstance(block, str) else block[0]["text"]
assert "content elided" in text and "Retrieve original: hash=" in text
assert len(text) < len(body) // 2
def test_single_dense_values_and_tsv_are_kept():
"""A lone JWT / signed URL / PATH / modulus is a value the agent asked for; TSV is data."""
jwt = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9." + "eyJzdWIiOiIxMjM0NTY3ODkwIn0" * 40 + ".sig"
url = "https://d1.cloudfront.net/v.mp4?Policy=" + "A" * 500 + "&Signature=" + "b" * 128
path = "PATH=" + ":".join(f"/opt/homebrew/opt/package{i}/bin" for i in range(24))
modulus = "Modulus: " + ":".join("ab" for _ in range(256))
tsv = "\n".join("\t".join(f"column_{i}_value_{j}" for i in range(35)) for j in range(5))
for text in (jwt, url, path, modulus, tsv, "Output:\n" + tsv * 3):
assert elide_dense_lines(text) == (text, 0)
def test_compact_json_lines_are_left_to_smart_crusher():
objs = " ".join(
f'{{"file":"src/m_{i}.py","line":{i},"text":"repeated search payload"}}' for i in range(160)
)
assert elide_dense_lines(objs) == (objs, 0)
assert elide_dense_lines("[" + ",".join(f'{{"id":{i}}}' for i in range(200)) + "]")[1] == 0
def test_lossless_mode_never_elides():
html = "<html><head><script>" + MINIFIED_JS + "</script></head><body>hi</body></html>"
router = ContentRouter(
ContentRouterConfig(enable_kompress=False, lossless=True, min_section_tokens=10)
)
assert "content elided" not in router.compress(html, context="tool_result").compressed
def test_search_output_keeps_line_structure():
grep = "src/bundle.js\n1:" + MINIFIED_JS + "\n2:export default x\nsrc/app.py\n10:def main():\n"
out, n = elide_dense_lines(grep)
lines = out.split("\n")
assert n == 1 and lines[0] == "src/bundle.js" and lines[1].startswith("1:!function")
assert lines[2:] == ["2:export default x", "src/app.py", "10:def main():", ""]