fix(router): preserve nested JSON records through compression (#3680)

## Description

Kompress can alter JSON record boundaries while leaving a shorter
document parseable, as reported in #3673. Current main already protects
whole documents and embedded record arrays while allowing separate
JSONL/search objects to compress. This branch now extends that
protection to nested JSON inside a balanced but invalid wrapper and
prevents dense-line elision from silently rewriting prefixed JSON
values.

Fixes #3673.

## Changes Made

- Refresh against current main, preserving its narrower structural JSON
guard, forced/targeted-Kompress routing, relevance-segment protection,
and bounded bracket scanning.
- Reuse the bracket walk's memoized nested boundaries when validating
documents. If an outer candidate is invalid, inspect nested candidates
without rescanning suffixes. Accepted outer documents suppress duplicate
nested results.
- Bound cumulative JSON parsing by input length as well as bracket
scanning; exhausted budgets and recursion limits report an incomplete
scan so the existing safety guard declines lossy compression.
- Recognize parseable prefixed JSON in the dense-line elider. JSON
values stay unchanged; ordinary minified JavaScript/base64 controls
still elide.
- Add four regressions: two invalid wrappers around valid record arrays,
a prefixed dense JSON value, and deeply nested invalid-wrapper parse
work.

## Validation

All four regressions fail against current main before correction and
pass afterward. The
router/recursive-JSON/dense-elision/forced-Kompress/deadline suite
passes 188 tests with two existing warnings. Ruff lint/format,
whitespace checks, and Linux-target mypy across 566 files pass.

## Real Behavior Proof

- Setup: Windows, Python 3.12, real Kompress ONNX model
(`chopratejas/kompress-v2-base`) loaded with ONNX Runtime 1.23.2 CPU,
actual uvicorn proxy, and a separate upstream over loopback TCP. The
runtime replay is offline after cache preload. Python detection is used
because this ORT exposes an older C API than native detection requires.
Optimization and forced Kompress enabled; response caching, rate
limiting, cost tracking, and subscription polling disabled. Only network
destination is redirected.
- Steps: run `pr3680-resume20-runtime.py`; exercise the actual
router/model, then POST five tool-result histories through
`/v1/chat/completions` and inspect the upstream's captured tool content.
- The actual ONNX prose control compresses 3,680 characters to 2,609,
demonstrating the model is loaded and active.
- Whole JSON, prefixed record JSON, and valid record JSON inside invalid
array/object wrappers remain byte-identical at the Kompress boundary,
with zero calls to the actual model. A 3,500-character token value in
prefixed JSON also survives the full TEXT/dense-elision strategy
unchanged.
- All five HTTP requests return 200 and preserve forwarded tool content
byte-for-byte, including all five named records.
- Identical replay against main sends both invalid-wrapper documents to
the actual model and changes them. Their forwarded HTTP bodies shrink
from 11,148 characters to 7,547 and 7,507. The prefixed dense value is
also rewritten by the native TEXT strategy. This demonstrates the actual
unsafe routes without claiming every model run deletes the same record.
- Evidence: `pr3680-resume20-runtime.log` and
`pr3680-resume20-runtime-red.log`; no fake model or fake inference
result. The method observer only records calls and delegates to real
inference. No live LLM provider account or billing traffic was
exercised.

## Runtime Rollout Safety

JSON inside invalid wrappers and dense prefixed JSON now retain their
structural bytes; adversarial/deep input can decline compression when
bounded scanning cannot establish safety. This may reduce savings for
those inputs. Separate top-level JSONL/search records, prose
compression, and existing forced-Kompress deadline behavior remain
covered by controls. No new flag or unsafe override. Roll back by
reverting the nested-span and dense-line changes.

Current-head hosted CI and fresh review must qualify before approval. No
merge performed.

---------

Signed-off-by: wangzifei <wangzifei@cit.group.hk>
Co-authored-by: wangzifei <wangzifei@cit.group.hk>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
iuiuiu
2026-10-08 10:32:53 -05:00
committed by GitHub
co-authored by wangzifei JD Davis
parent a65455804c
commit fd067033da
3 changed files with 88 additions and 7 deletions
+8 -1
View File
@@ -42,7 +42,14 @@ def is_dense_line(line: str) -> bool:
if n < MIN_LINE_CHARS or "\t" in line or (line.count(" ") / n) >= MAX_SPACE_RATIO:
return False
stripped = line.strip()
return not (stripped[:1] in "{[" and stripped[-1:] in "}]")
if stripped[:1] in "{[" and stripped[-1:] in "}]":
return False
# A label can precede compact JSON. Eliding that line would silently
# rewrite its values even when the JSON record guard skipped Kompress.
from .recursive_json import scan_json_documents
spans, complete = scan_json_documents(line)
return complete and not spans
def elide_dense_lines(text: str) -> tuple[str, int]:
+28 -6
View File
@@ -119,7 +119,7 @@ def _scan_from(text: str, start: int, known: dict[int, int | None]) -> tuple[int
return None, n - start
def _scan_spans(text: str) -> tuple[list[tuple[int, int]], bool]:
def _scan_spans(text: str, *, include_nested: bool = False) -> tuple[list[tuple[int, int]], bool]:
"""``(spans, complete)``: the top-level balanced spans ``_spans`` returns, and
whether the walk covered the whole text within its budget.
@@ -148,6 +148,14 @@ def _scan_spans(text: str) -> tuple[list[tuple[int, int]], bool]:
i = end
continue
i += 1
if include_nested:
# The walk already records every nested bracket's verdict. Reuse it
# in source order without scanning any suffix again.
out = []
for start in range(n):
end = known.get(start)
if end is not None:
out.append((start, end))
return out, True
@@ -167,19 +175,33 @@ def json_document_spans(text: str) -> list[tuple[int, int]]:
def scan_json_documents(text: str) -> tuple[list[tuple[int, int]], bool]:
"""``json_document_spans`` plus whether the scan covered all of ``text``.
``complete`` is only ever False past the scan budget, i.e. on input built to
defeat the linear walk; a caller that must not miss a document treats that
as "may contain one".
``complete`` is False when the bracket walk or cumulative JSON validation
exhausts its work budget, or the decoder reaches its recursion limit.
A caller that must not miss a document treats any incomplete result as
"may contain one" and declines lossy compression.
"""
spans, complete = _scan_spans(text)
spans, complete = _scan_spans(text, include_nested=True)
out: list[tuple[int, int]] = []
covered_until = 0
parsed_chars = 0
parse_budget = _SCAN_BUDGET_PER_CHAR * len(text) + _SCAN_BUDGET_FLOOR
for a, b in spans:
if a < covered_until:
continue
# Invalid wrappers may contain valid JSON. Checking each nested
# candidate must not turn deeply nested malformed input quadratic.
if parsed_chars + b - a > parse_budget:
return out, False
parsed_chars += b - a
try:
parsed = json.loads(text[a:b])
except (ValueError, TypeError, RecursionError):
except RecursionError:
return out, False
except (ValueError, TypeError):
continue
if isinstance(parsed, dict | list):
out.append((a, b))
covered_until = b
return out, complete
+52
View File
@@ -0,0 +1,52 @@
from types import SimpleNamespace
import pytest
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
from headroom.transforms.dense_line_elider import elide_dense_lines
from headroom.transforms.recursive_json import json_document_spans, scan_json_documents
@pytest.mark.parametrize("wrapper", ["[response: {}]", "{{response: {}}}"])
def test_nested_record_document_survives_non_json_wrapper(monkeypatch, wrapper):
document = '{"items":[{"id":1},{"id":2}]}'
payload = wrapper.format(document)
calls = []
class Kompress:
def is_ready(self):
return True
def compress(self, content, **kwargs):
calls.append(content)
return SimpleNamespace(compressed="records deleted", compressed_tokens=2)
router = ContentRouter(ContentRouterConfig())
monkeypatch.setattr(router, "_get_kompress", lambda: Kompress())
out, _ = router._try_ml_compressor(payload, context="")
assert out == payload
assert calls == []
assert [payload[a:b] for a, b in json_document_spans(payload)] == [document]
def test_dense_prefixed_json_retains_its_value():
payload = 'Tool result: {"token":"' + "x" * 3500 + '"}'
assert elide_dense_lines(payload) == (payload, 0)
def test_invalid_wrapper_parsing_has_a_linear_character_budget(monkeypatch):
import headroom.transforms.recursive_json as recursive_json
text = "[" * 500 + "response: " + '{"items":[{"id":1},{"id":2}]}' + "]" * 500
parsed_chars = 0
real_loads = recursive_json.json.loads
def count_loads(value, *args, **kwargs):
nonlocal parsed_chars
parsed_chars += len(value)
return real_loads(value, *args, **kwargs)
monkeypatch.setattr(recursive_json.json, "loads", count_loads)
_spans, complete = scan_json_documents(text)
assert not complete
assert parsed_chars <= 4 * len(text) + 4096