mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(router): preserve nested JSON records through compression (#3680)
## Description Kompress can alter JSON record boundaries while leaving a shorter document parseable, as reported in #3673. Current main already protects whole documents and embedded record arrays while allowing separate JSONL/search objects to compress. This branch now extends that protection to nested JSON inside a balanced but invalid wrapper and prevents dense-line elision from silently rewriting prefixed JSON values. Fixes #3673. ## Changes Made - Refresh against current main, preserving its narrower structural JSON guard, forced/targeted-Kompress routing, relevance-segment protection, and bounded bracket scanning. - Reuse the bracket walk's memoized nested boundaries when validating documents. If an outer candidate is invalid, inspect nested candidates without rescanning suffixes. Accepted outer documents suppress duplicate nested results. - Bound cumulative JSON parsing by input length as well as bracket scanning; exhausted budgets and recursion limits report an incomplete scan so the existing safety guard declines lossy compression. - Recognize parseable prefixed JSON in the dense-line elider. JSON values stay unchanged; ordinary minified JavaScript/base64 controls still elide. - Add four regressions: two invalid wrappers around valid record arrays, a prefixed dense JSON value, and deeply nested invalid-wrapper parse work. ## Validation All four regressions fail against current main before correction and pass afterward. The router/recursive-JSON/dense-elision/forced-Kompress/deadline suite passes 188 tests with two existing warnings. Ruff lint/format, whitespace checks, and Linux-target mypy across 566 files pass. ## Real Behavior Proof - Setup: Windows, Python 3.12, real Kompress ONNX model (`chopratejas/kompress-v2-base`) loaded with ONNX Runtime 1.23.2 CPU, actual uvicorn proxy, and a separate upstream over loopback TCP. The runtime replay is offline after cache preload. Python detection is used because this ORT exposes an older C API than native detection requires. Optimization and forced Kompress enabled; response caching, rate limiting, cost tracking, and subscription polling disabled. Only network destination is redirected. - Steps: run `pr3680-resume20-runtime.py`; exercise the actual router/model, then POST five tool-result histories through `/v1/chat/completions` and inspect the upstream's captured tool content. - The actual ONNX prose control compresses 3,680 characters to 2,609, demonstrating the model is loaded and active. - Whole JSON, prefixed record JSON, and valid record JSON inside invalid array/object wrappers remain byte-identical at the Kompress boundary, with zero calls to the actual model. A 3,500-character token value in prefixed JSON also survives the full TEXT/dense-elision strategy unchanged. - All five HTTP requests return 200 and preserve forwarded tool content byte-for-byte, including all five named records. - Identical replay against main sends both invalid-wrapper documents to the actual model and changes them. Their forwarded HTTP bodies shrink from 11,148 characters to 7,547 and 7,507. The prefixed dense value is also rewritten by the native TEXT strategy. This demonstrates the actual unsafe routes without claiming every model run deletes the same record. - Evidence: `pr3680-resume20-runtime.log` and `pr3680-resume20-runtime-red.log`; no fake model or fake inference result. The method observer only records calls and delegates to real inference. No live LLM provider account or billing traffic was exercised. ## Runtime Rollout Safety JSON inside invalid wrappers and dense prefixed JSON now retain their structural bytes; adversarial/deep input can decline compression when bounded scanning cannot establish safety. This may reduce savings for those inputs. Separate top-level JSONL/search records, prose compression, and existing forced-Kompress deadline behavior remain covered by controls. No new flag or unsafe override. Roll back by reverting the nested-span and dense-line changes. Current-head hosted CI and fresh review must qualify before approval. No merge performed. --------- Signed-off-by: wangzifei <wangzifei@cit.group.hk> Co-authored-by: wangzifei <wangzifei@cit.group.hk> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
wangzifei
JD Davis
parent
a65455804c
commit
fd067033da
@@ -42,7 +42,14 @@ def is_dense_line(line: str) -> bool:
|
||||
if n < MIN_LINE_CHARS or "\t" in line or (line.count(" ") / n) >= MAX_SPACE_RATIO:
|
||||
return False
|
||||
stripped = line.strip()
|
||||
return not (stripped[:1] in "{[" and stripped[-1:] in "}]")
|
||||
if stripped[:1] in "{[" and stripped[-1:] in "}]":
|
||||
return False
|
||||
# A label can precede compact JSON. Eliding that line would silently
|
||||
# rewrite its values even when the JSON record guard skipped Kompress.
|
||||
from .recursive_json import scan_json_documents
|
||||
|
||||
spans, complete = scan_json_documents(line)
|
||||
return complete and not spans
|
||||
|
||||
|
||||
def elide_dense_lines(text: str) -> tuple[str, int]:
|
||||
|
||||
@@ -119,7 +119,7 @@ def _scan_from(text: str, start: int, known: dict[int, int | None]) -> tuple[int
|
||||
return None, n - start
|
||||
|
||||
|
||||
def _scan_spans(text: str) -> tuple[list[tuple[int, int]], bool]:
|
||||
def _scan_spans(text: str, *, include_nested: bool = False) -> tuple[list[tuple[int, int]], bool]:
|
||||
"""``(spans, complete)``: the top-level balanced spans ``_spans`` returns, and
|
||||
whether the walk covered the whole text within its budget.
|
||||
|
||||
@@ -148,6 +148,14 @@ def _scan_spans(text: str) -> tuple[list[tuple[int, int]], bool]:
|
||||
i = end
|
||||
continue
|
||||
i += 1
|
||||
if include_nested:
|
||||
# The walk already records every nested bracket's verdict. Reuse it
|
||||
# in source order without scanning any suffix again.
|
||||
out = []
|
||||
for start in range(n):
|
||||
end = known.get(start)
|
||||
if end is not None:
|
||||
out.append((start, end))
|
||||
return out, True
|
||||
|
||||
|
||||
@@ -167,19 +175,33 @@ def json_document_spans(text: str) -> list[tuple[int, int]]:
|
||||
def scan_json_documents(text: str) -> tuple[list[tuple[int, int]], bool]:
|
||||
"""``json_document_spans`` plus whether the scan covered all of ``text``.
|
||||
|
||||
``complete`` is only ever False past the scan budget, i.e. on input built to
|
||||
defeat the linear walk; a caller that must not miss a document treats that
|
||||
as "may contain one".
|
||||
``complete`` is False when the bracket walk or cumulative JSON validation
|
||||
exhausts its work budget, or the decoder reaches its recursion limit.
|
||||
A caller that must not miss a document treats any incomplete result as
|
||||
"may contain one" and declines lossy compression.
|
||||
"""
|
||||
spans, complete = _scan_spans(text)
|
||||
spans, complete = _scan_spans(text, include_nested=True)
|
||||
out: list[tuple[int, int]] = []
|
||||
covered_until = 0
|
||||
parsed_chars = 0
|
||||
parse_budget = _SCAN_BUDGET_PER_CHAR * len(text) + _SCAN_BUDGET_FLOOR
|
||||
for a, b in spans:
|
||||
if a < covered_until:
|
||||
continue
|
||||
# Invalid wrappers may contain valid JSON. Checking each nested
|
||||
# candidate must not turn deeply nested malformed input quadratic.
|
||||
if parsed_chars + b - a > parse_budget:
|
||||
return out, False
|
||||
parsed_chars += b - a
|
||||
try:
|
||||
parsed = json.loads(text[a:b])
|
||||
except (ValueError, TypeError, RecursionError):
|
||||
except RecursionError:
|
||||
return out, False
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
if isinstance(parsed, dict | list):
|
||||
out.append((a, b))
|
||||
covered_until = b
|
||||
return out, complete
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
|
||||
from headroom.transforms.dense_line_elider import elide_dense_lines
|
||||
from headroom.transforms.recursive_json import json_document_spans, scan_json_documents
|
||||
|
||||
|
||||
@pytest.mark.parametrize("wrapper", ["[response: {}]", "{{response: {}}}"])
|
||||
def test_nested_record_document_survives_non_json_wrapper(monkeypatch, wrapper):
|
||||
document = '{"items":[{"id":1},{"id":2}]}'
|
||||
payload = wrapper.format(document)
|
||||
calls = []
|
||||
|
||||
class Kompress:
|
||||
def is_ready(self):
|
||||
return True
|
||||
|
||||
def compress(self, content, **kwargs):
|
||||
calls.append(content)
|
||||
return SimpleNamespace(compressed="records deleted", compressed_tokens=2)
|
||||
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
monkeypatch.setattr(router, "_get_kompress", lambda: Kompress())
|
||||
out, _ = router._try_ml_compressor(payload, context="")
|
||||
assert out == payload
|
||||
assert calls == []
|
||||
assert [payload[a:b] for a, b in json_document_spans(payload)] == [document]
|
||||
|
||||
|
||||
def test_dense_prefixed_json_retains_its_value():
|
||||
payload = 'Tool result: {"token":"' + "x" * 3500 + '"}'
|
||||
assert elide_dense_lines(payload) == (payload, 0)
|
||||
|
||||
|
||||
def test_invalid_wrapper_parsing_has_a_linear_character_budget(monkeypatch):
|
||||
import headroom.transforms.recursive_json as recursive_json
|
||||
|
||||
text = "[" * 500 + "response: " + '{"items":[{"id":1},{"id":2}]}' + "]" * 500
|
||||
parsed_chars = 0
|
||||
real_loads = recursive_json.json.loads
|
||||
|
||||
def count_loads(value, *args, **kwargs):
|
||||
nonlocal parsed_chars
|
||||
parsed_chars += len(value)
|
||||
return real_loads(value, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(recursive_json.json, "loads", count_loads)
|
||||
_spans, complete = scan_json_documents(text)
|
||||
assert not complete
|
||||
assert parsed_chars <= 4 * len(text) + 4096
|
||||
Reference in New Issue
Block a user