mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(transforms): stop embedded JSON pre-pass short-circuiting HTML extraction (#3620)
## Description
The embedded-JSON pre-pass in `_apply_strategy_to_content` runs before
any strategy dispatch and returns immediately on a successful splice.
For a whole HTML document containing a JSON fragment, a small local
minify pre-empts whole-document extraction: the report measured a 2%
splice win blocking a 95.5% extraction (my reproduction: 12.4% blocking
45.9%). The parent format's stronger compressor never runs.
This defers the embedded-JSON route when the block's strategy is HTML:
extraction gets first refusal, and the splice runs in the passthrough
fallback only when extraction found nothing. This is option 3 from the
issue ("only allow embedded JSON compression to short-circuit routing
when there is no stronger parent-level strategy available"), scoped to
the HTML case the report measured. Non-HTML strategies keep the
immediate pre-pass untouched. Lossless mode never reaches the strategy
dispatch, so it keeps the immediate-return behavior (JSON minification
is itself byte-lossless).
While tracing the fallback I also fixed an adjacent hole in the same
branch: `trafilatura` returns `''` (not `None`) when nothing extracts,
and the HTML branch treated `''` as a successful compression, so an
unextractable page blanked the whole block and only the empty-output
guard in `compress()` caught it. Empty extraction now collapses to the
passthrough path like `None` does, which is also what makes the deferred
splice reachable for shell-only documents.
Closes #3609
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/transforms/content_router.py`: the embedded-JSON pre-pass
body moved unchanged into `_embedded_json_route()`; HTML blocks skip the
immediate splice and retry it in the passthrough fallback, appending
`embedded_json` to the existing chain (`["html", "embedded_json"]`) so
log readers can see the extraction miss. Empty HTML extraction (`''`)
now falls through to passthrough instead of being accepted as a
compression result.
- `tests/test_transforms_content_router.py`: two regression tests
(extractor runs on HTML with embedded JSON; deferred splice still fires
when extraction misses).
- `route_embedded_json` itself is unchanged; its unit tests pass
untouched.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_transforms_content_router.py tests/test_recursive_json.py -q
71 passed in 5.78s
$ uv run ruff check .
All checks passed!
$ uv run mypy headroom/transforms/content_router.py
Success: no issues found in 1 source file
```
Both new tests fail on base `35b1156` (chain `["embedded_json"]`,
`<html>` retained) and pass with the fix. I also ran every test file
that imports `ContentRouter` (86 files, `grep -rl`): `1462 passed, 41
skipped, 0 failed`.
## Real Behavior Proof
- Environment: WSL2 Ubuntu, Python 3.11 via uv, headroom 0.37.1-dev from
source (`uv sync --frozen --extra dev --extra html`, trafilatura 2.0.0),
pure-Python detection (no ONNX runtime installed).
- Exact command / steps: a real HTML document (30 varied prose
paragraphs x2 + one 60-row embedded JSON array in a single-line
`<script>`, 19,297 chars) through `ContentRouter().compress()`, run with
the patch and then with the patched file stashed (same interpreter):
```text
# WITH FIX
detection : ContentType.HTML
doc tokens: 4824 -> compressed tokens: 2610
strategy : html chain=['html']
reduction : 45.9%
<html> tag still present: False
# BASE 35b1156
detection : ContentType.HTML
doc tokens: 4824 -> compressed tokens: 4226
strategy : html chain=['embedded_json']
reduction : 12.4%
<html> tag still present: True
```
- Observed result: with the fix the extraction runs and the markup is
gone; on base the JSON splice short-circuits and the full HTML document
(markup included) survives.
- Not tested: proxy mode end-to-end (`/v1/compress` through a live LLM
provider); Windows/macOS; ONNX native detection (the pure-Python
fallback classifies these documents as HTML identically).
## Runtime Rollout Safety
- Rollout-managed feature(s): none — no flag/rollout system gates this
path.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: yes, for HTML-classified blocks
containing routable embedded JSON: the strategy chain changes from
`["embedded_json"]` to `["html"]` (extraction wins) or `["html",
"embedded_json"]` (extraction misses). Everything else is
byte-identical.
- Kill switch / disable path: `enable_html_extractor=false` routes HTML
blocks to passthrough; the deferred splice then fires exactly as the
pre-fix pre-pass did.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert the commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (n/a — no
user-facing docs cover route ordering)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
A tiny local JSON win should never block the parent document's real
compressor. Extraction is lossy and the splice is lossless, so deferring
is a behavior trade (extraction output replaces markup+JSON entirely);
the fallback keeps the splice whenever extraction has nothing to offer.
AI use: an AI agent located the short-circuit, ran the before/after
reproductions and the test suite, and drafted this description. The fix
approach follows option 3 suggested in the issue. I reviewed every line
of the diff and the reproduced numbers.
This commit is contained in:
@@ -3515,22 +3515,20 @@ class ContentRouter(Transform):
|
||||
# call is a one-shot re-entrancy guard (NOT a depth cap). Deterministic +
|
||||
# benefit-gated (no size/min thresholds) → prefix-cache- and CCR-store-
|
||||
# stable, and a strict no-op when the block has no embedded JSON.
|
||||
if _allow_embedded:
|
||||
from headroom.transforms.recursive_json import route_embedded_json
|
||||
|
||||
def _dispatch_span(span: str) -> str | None:
|
||||
strat = self._strategy_from_detection_type(_detect_content(span).content_type)
|
||||
text, _t, _c = self._apply_strategy_to_content(
|
||||
span,
|
||||
strat,
|
||||
context,
|
||||
question=question,
|
||||
bias=bias,
|
||||
_allow_embedded=False,
|
||||
)
|
||||
return text if text != span else None
|
||||
|
||||
routed = route_embedded_json(content, _dispatch_span, tok=_estimate_tokens)
|
||||
#
|
||||
# HTML blocks are the one exception (#3609): whole-document extraction
|
||||
# routinely beats a local JSON minify by orders of magnitude (the report
|
||||
# measured a 2% splice win pre-empting a 95% extraction), so the splice
|
||||
# must not short-circuit the parent strategy. Defer it: run the HTML
|
||||
# dispatch below first and compute the splice in the passthrough
|
||||
# fallback only when extraction found nothing. Lossless mode never
|
||||
# reaches that dispatch, so it keeps the immediate-return behavior
|
||||
# (JSON minification is itself byte-lossless).
|
||||
_defer_embedded_for_html = (
|
||||
_allow_embedded and strategy is CompressionStrategy.HTML and not self.config.lossless
|
||||
)
|
||||
if _allow_embedded and not _defer_embedded_for_html:
|
||||
routed = self._embedded_json_route(content, context, question, bias)
|
||||
if routed is not None:
|
||||
return routed, _estimate_tokens(routed), ["embedded_json"]
|
||||
|
||||
@@ -3830,9 +3828,15 @@ class ContentRouter(Transform):
|
||||
# ``result.extracted is None`` path (chain
|
||||
# ``[html, passthrough]``). A real extraction is
|
||||
# byte-identical to the historical ``result.extracted``.
|
||||
# An EMPTY extraction (trafilatura found no article text)
|
||||
# collapses to None too: a blank page is "nothing
|
||||
# extracts", and returning "" here would discard the
|
||||
# whole block instead of falling through (#3609).
|
||||
output = self._registry_compress("html", strategy, content, context, bias)
|
||||
compressed = (
|
||||
output.content if output is not None and output.compressed else None
|
||||
output.content
|
||||
if output is not None and output.compressed and output.content.strip()
|
||||
else None
|
||||
)
|
||||
# A page that is all <script>/<style> extracts to "".
|
||||
# That is "nothing extracted" (fall through to the
|
||||
@@ -4067,6 +4071,16 @@ class ContentRouter(Transform):
|
||||
)
|
||||
return compressed, compressed_tokens, strategy_chain
|
||||
|
||||
# #3609: deferred embedded-JSON splice. The HTML strategy ran above and
|
||||
# extracted nothing (``compressed`` is still None), so the local JSON
|
||||
# win — held back at the top to give whole-document extraction first
|
||||
# refusal — is now the best available outcome.
|
||||
if _defer_embedded_for_html:
|
||||
routed = self._embedded_json_route(content, context, question, bias)
|
||||
if routed is not None:
|
||||
strategy_chain.append("embedded_json")
|
||||
return routed, _estimate_tokens(routed), strategy_chain
|
||||
|
||||
# Fallback: return unchanged
|
||||
if self.config.enable_dense_line_elision:
|
||||
dense = self._elide_dense(content, context)
|
||||
@@ -4140,6 +4154,32 @@ class ContentRouter(Transform):
|
||||
return None
|
||||
return elided, _estimate_tokens(elided)
|
||||
|
||||
def _embedded_json_route(
|
||||
self, content: str, context: str, question: str | None, bias: float
|
||||
) -> str | None:
|
||||
"""Route embedded JSON spans inside ``content`` through the JSON compressors.
|
||||
|
||||
Extracted from the ``_apply_strategy_to_content`` pre-pass so the HTML
|
||||
strategy can defer the splice (#3609) and retry it in the passthrough
|
||||
fallback. Returns the spliced block, or ``None`` when nothing
|
||||
safe/smaller applied.
|
||||
"""
|
||||
from headroom.transforms.recursive_json import route_embedded_json
|
||||
|
||||
def _dispatch_span(span: str) -> str | None:
|
||||
strat = self._strategy_from_detection_type(_detect_content(span).content_type)
|
||||
text, _t, _c = self._apply_strategy_to_content(
|
||||
span,
|
||||
strat,
|
||||
context,
|
||||
question=question,
|
||||
bias=bias,
|
||||
_allow_embedded=False,
|
||||
)
|
||||
return text if text != span else None
|
||||
|
||||
return route_embedded_json(content, _dispatch_span, tok=_estimate_tokens)
|
||||
|
||||
def _try_ml_compressor(
|
||||
self,
|
||||
content: str,
|
||||
|
||||
@@ -2151,3 +2151,102 @@ def test_prompt_text_compresses_without_replay_guarantee(monkeypatch: pytest.Mon
|
||||
min_tokens_to_compress=1,
|
||||
)
|
||||
assert _TASK in calls
|
||||
|
||||
|
||||
def _html_doc_with_embedded_json(*, padded: bool = True) -> str:
|
||||
"""A whole HTML document with one embedded JSON array-of-objects.
|
||||
|
||||
The JSON sits in a single-line ``<script>`` so the mixed-content regex
|
||||
heuristics do not fire: the block detects as pure HTML and takes the
|
||||
``_compress_pure`` path. ``padded`` keeps the default ``json.dumps``
|
||||
separators so the array is minifiable — a compact array gives the JSON
|
||||
route no benefit and both passes no-op.
|
||||
"""
|
||||
rows = [
|
||||
{
|
||||
"id": i,
|
||||
"name": f"item-{i}",
|
||||
"tags": ["alpha", "beta"],
|
||||
"score": i * 3,
|
||||
"description": "a routine record with repeated descriptive fields",
|
||||
}
|
||||
for i in range(60)
|
||||
]
|
||||
embedded = json.dumps(rows) if padded else json.dumps(rows, separators=(",", ":"))
|
||||
paras = "".join(
|
||||
f"<p>Paragraph {i}: distributed systems rely on consensus protocols to "
|
||||
f"agree on state. Raft separates leader election from log replication, "
|
||||
f"which makes it easier to reason about.</p>\n"
|
||||
for i in range(30)
|
||||
)
|
||||
return (
|
||||
"<!DOCTYPE html>\n<html><head><title>Raft Notes</title></head>\n<body>\n"
|
||||
"<article><h1>Raft consensus notes</h1>\n"
|
||||
+ paras
|
||||
+ '<script type="application/json">'
|
||||
+ embedded
|
||||
+ "</script>\n"
|
||||
+ paras
|
||||
+ "</article>\n</body></html>\n"
|
||||
)
|
||||
|
||||
|
||||
def test_html_with_embedded_json_runs_extractor_not_json_shortcut() -> None:
|
||||
"""#3609: the embedded-JSON pre-pass must not short-circuit HTML extraction.
|
||||
|
||||
Before the fix, a small local JSON minify (~10% win) returned immediately
|
||||
from ``_apply_strategy_to_content`` and whole-document extraction (~36%
|
||||
win on this document) never ran; the issue reported the same shape at
|
||||
2% vs 95.5%. The splice is now deferred for HTML: extraction gets first
|
||||
refusal and the JSON route only fires when extraction finds nothing.
|
||||
"""
|
||||
pytest.importorskip("trafilatura")
|
||||
doc = _html_doc_with_embedded_json()
|
||||
assert _detect_content(doc).content_type is ContentType.HTML
|
||||
assert not is_mixed_content(doc)
|
||||
|
||||
result = ContentRouter().compress(doc)
|
||||
|
||||
assert result.strategy_used is CompressionStrategy.HTML
|
||||
assert result.strategy_chain == ["html"]
|
||||
# extraction ran: markup is gone and the document collapsed hard
|
||||
# (the pre-fix JSON shortcut left <html> in place at ~10% reduction)
|
||||
assert "<html>" not in result.compressed
|
||||
assert len(result.compressed) < len(doc) * 3 // 5
|
||||
|
||||
|
||||
def test_html_extraction_miss_still_gets_deferred_json_splice() -> None:
|
||||
"""Deferring must not LOSE the embedded-JSON win when extraction finds nothing.
|
||||
|
||||
A shell-only HTML document makes the extractor come back empty; the
|
||||
deferred splice then still compresses the embedded array. Before the fix
|
||||
this shape short-circuited at the top (chain ``["embedded_json"]``); now
|
||||
the block first proves the HTML strategy is a miss (chain
|
||||
``["html", "embedded_json"]``) with the same spliced output.
|
||||
"""
|
||||
pytest.importorskip("trafilatura")
|
||||
embedded = json.dumps(
|
||||
[
|
||||
{"id": i, "name": f"item-{i}", "tags": ["alpha", "beta"], "score": i * 3}
|
||||
for i in range(40)
|
||||
]
|
||||
)
|
||||
shell = (
|
||||
"<!DOCTYPE html>\n<html><head><title>t</title></head>\n<body>\n"
|
||||
'<script type="application/json">' + embedded + "</script>\n"
|
||||
"</body></html>\n"
|
||||
)
|
||||
router = ContentRouter()
|
||||
# sanity: extraction finds nothing on this shell (empty or None)
|
||||
extractor = router._get_html_extractor()
|
||||
assert extractor is not None
|
||||
assert not (extractor.extract(shell).extracted or "").strip()
|
||||
|
||||
text, _tokens, chain = router._apply_strategy_to_content(
|
||||
shell, CompressionStrategy.HTML, "", None, 1.0
|
||||
)
|
||||
|
||||
assert chain == ["html", "embedded_json"]
|
||||
assert len(text) < len(shell)
|
||||
json_marker = '"id":'
|
||||
assert json_marker not in text # the array was compressed away
|
||||
|
||||
Reference in New Issue
Block a user