Files
headroom/tests/test_json_protection_boundaries.py
fd067033da fix(router): preserve nested JSON records through compression (#3680)
## Description

Kompress can alter JSON record boundaries while leaving a shorter
document parseable, as reported in #3673. Current main already protects
whole documents and embedded record arrays while allowing separate
JSONL/search objects to compress. This branch now extends that
protection to nested JSON inside a balanced but invalid wrapper and
prevents dense-line elision from silently rewriting prefixed JSON
values.

Fixes #3673.

## Changes Made

- Refresh against current main, preserving its narrower structural JSON
guard, forced/targeted-Kompress routing, relevance-segment protection,
and bounded bracket scanning.
- Reuse the bracket walk's memoized nested boundaries when validating
documents. If an outer candidate is invalid, inspect nested candidates
without rescanning suffixes. Accepted outer documents suppress duplicate
nested results.
- Bound cumulative JSON parsing by input length as well as bracket
scanning; exhausted budgets and recursion limits report an incomplete
scan so the existing safety guard declines lossy compression.
- Recognize parseable prefixed JSON in the dense-line elider. JSON
values stay unchanged; ordinary minified JavaScript/base64 controls
still elide.
- Add four regressions: two invalid wrappers around valid record arrays,
a prefixed dense JSON value, and deeply nested invalid-wrapper parse
work.

## Validation

All four regressions fail against current main before correction and
pass afterward. The
router/recursive-JSON/dense-elision/forced-Kompress/deadline suite
passes 188 tests with two existing warnings. Ruff lint/format,
whitespace checks, and Linux-target mypy across 566 files pass.

## Real Behavior Proof

- Setup: Windows, Python 3.12, real Kompress ONNX model
(`chopratejas/kompress-v2-base`) loaded with ONNX Runtime 1.23.2 CPU,
actual uvicorn proxy, and a separate upstream over loopback TCP. The
runtime replay is offline after cache preload. Python detection is used
because this ORT exposes an older C API than native detection requires.
Optimization and forced Kompress enabled; response caching, rate
limiting, cost tracking, and subscription polling disabled. Only network
destination is redirected.
- Steps: run `pr3680-resume20-runtime.py`; exercise the actual
router/model, then POST five tool-result histories through
`/v1/chat/completions` and inspect the upstream's captured tool content.
- The actual ONNX prose control compresses 3,680 characters to 2,609,
demonstrating the model is loaded and active.
- Whole JSON, prefixed record JSON, and valid record JSON inside invalid
array/object wrappers remain byte-identical at the Kompress boundary,
with zero calls to the actual model. A 3,500-character token value in
prefixed JSON also survives the full TEXT/dense-elision strategy
unchanged.
- All five HTTP requests return 200 and preserve forwarded tool content
byte-for-byte, including all five named records.
- Identical replay against main sends both invalid-wrapper documents to
the actual model and changes them. Their forwarded HTTP bodies shrink
from 11,148 characters to 7,547 and 7,507. The prefixed dense value is
also rewritten by the native TEXT strategy. This demonstrates the actual
unsafe routes without claiming every model run deletes the same record.
- Evidence: `pr3680-resume20-runtime.log` and
`pr3680-resume20-runtime-red.log`; no fake model or fake inference
result. The method observer only records calls and delegates to real
inference. No live LLM provider account or billing traffic was
exercised.

## Runtime Rollout Safety

JSON inside invalid wrappers and dense prefixed JSON now retain their
structural bytes; adversarial/deep input can decline compression when
bounded scanning cannot establish safety. This may reduce savings for
those inputs. Separate top-level JSONL/search records, prose
compression, and existing forced-Kompress deadline behavior remain
covered by controls. No new flag or unsafe override. Roll back by
reverting the nested-span and dense-line changes.

Current-head hosted CI and fresh review must qualify before approval. No
merge performed.

---------

Signed-off-by: wangzifei <wangzifei@cit.group.hk>
Co-authored-by: wangzifei <wangzifei@cit.group.hk>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-08 10:32:53 -05:00

53 lines
1.9 KiB
Python

from types import SimpleNamespace
import pytest
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
from headroom.transforms.dense_line_elider import elide_dense_lines
from headroom.transforms.recursive_json import json_document_spans, scan_json_documents
@pytest.mark.parametrize("wrapper", ["[response: {}]", "{{response: {}}}"])
def test_nested_record_document_survives_non_json_wrapper(monkeypatch, wrapper):
document = '{"items":[{"id":1},{"id":2}]}'
payload = wrapper.format(document)
calls = []
class Kompress:
def is_ready(self):
return True
def compress(self, content, **kwargs):
calls.append(content)
return SimpleNamespace(compressed="records deleted", compressed_tokens=2)
router = ContentRouter(ContentRouterConfig())
monkeypatch.setattr(router, "_get_kompress", lambda: Kompress())
out, _ = router._try_ml_compressor(payload, context="")
assert out == payload
assert calls == []
assert [payload[a:b] for a, b in json_document_spans(payload)] == [document]
def test_dense_prefixed_json_retains_its_value():
payload = 'Tool result: {"token":"' + "x" * 3500 + '"}'
assert elide_dense_lines(payload) == (payload, 0)
def test_invalid_wrapper_parsing_has_a_linear_character_budget(monkeypatch):
import headroom.transforms.recursive_json as recursive_json
text = "[" * 500 + "response: " + '{"items":[{"id":1},{"id":2}]}' + "]" * 500
parsed_chars = 0
real_loads = recursive_json.json.loads
def count_loads(value, *args, **kwargs):
nonlocal parsed_chars
parsed_chars += len(value)
return real_loads(value, *args, **kwargs)
monkeypatch.setattr(recursive_json.json, "loads", count_loads)
_spans, complete = scan_json_documents(text)
assert not complete
assert parsed_chars <= 4 * len(text) + 4096