mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-23 02:49:17 +08:00
## Description Closes #3697. An unchanged SmartCrusher result now keeps the original bytes and reports no structural compression. Formatting-only savings come from the existing Rust JsonMinifier, not from pretending the JSON value changed. The minifier preserves duplicate keys, order, escapes and number literals. A mixed result cannot use one section's CCR marker to hide unmarked loss in another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Documentation update ## Changes Made - Keep SmartCrusher's exact-byte no-op behavior; route actual whitespace savings through JsonMinifier in Python and the eligible native live zone. - Keep all records and JSON literals; do not store originals or inject CCR markers for lossless minification. - Preserve mixed routing and reject uncovered lossy sections in both tool-result formats. Explicit no-CCR behavior is unchanged. - Replace incidental forwarding tests with native minification and complete-record regressions; document the distinction. ## Testing - [ ] Entire Python suite — not claimed. - [x] Four consumer/native modules: `python -m pytest -q tests/test_recursive_json.py tests/test_transforms_content_router.py tests/test_transforms/test_smart_crusher_rust_parity.py tests/test_transforms/test_smart_crusher_lossless_default.py` - [x] Changed-file Ruff checks; normal commit hooks, including mypy and commitlint. - [x] Manual public-API runs, with and without the HTML extra. ### Test Output Consumer/native modules: ```text collected 136 items ============================= 136 passed in 36.43s ============================= ``` Normal pre-push (fmt, clippy, 1,668 Rust tests with 3 ignored, native build, Python checks, commitlint): ```text ======================== 180 passed, 1 warning in 8.35s ======================== ✅ ci-precheck PASSED — safe to push. To https://github.com/oxura/headroom.git 64688217..0002af3e HEAD -> fix/3697-unchanged-json-passthrough ``` The first push stopped on an inherited commit footer because `origin/main` was stale. Fetching the actual upstream ref fixed the lint range; the normal push then passed. No source merge, lint-rule change or hook bypass. Hosted checks on `0002af3e` now pass: all four Python shards, Windows permissions, smoke tests, native wrappers, parity and Docker e2e. [Checks](https://github.com/headroomlabs-ai/headroom/pull/4044/checks). ## Real Behavior Proof - Environment: Linux x86-64, Python 3.13.14, Rust 1.95 and the actual extension built from this checkout. No provider billing calls. - Exact command / steps: the earlier localhost `/v1/compress` and TransformPipeline runs exercised the original fix. For this follow-up, call `ContentRouter.compress` on a data-only HTML page containing 60 complete records, with `smart_crusher_lossless_only=True`, `smart_crusher_with_compaction=False`, and `enable_kompress=False`. Run it once without trafilatura, then with the locked 2.0.0 extra. The new public-API test contains the full input/config. Separately call `route_embedded_json` on a compact array followed by a padded array, with Unicode and CRLF surroundings. - Observed result: both HTML modes keep all 60 records and the surrounding markup, with no unrecoverable section. The unchanged first JSON span does not stop the second span's compaction; surrounding text stays exact. Final Codecov is 96%; the two remaining lines are the deferred HTML fallback return. The real no-HTML-extra API run executed both, and local coverage also exercises the unchanged-span branch. - Not tested: live provider acceptance, invoices or production cache hit rates. No native eligibility or admission gates changed. ## Runtime Rollout Safety - Rollout-managed feature(s): none; existing JSON routing. - Minimum rollout channel: unchanged. - Stable/default behavior changed: eligible formatting savings return as json_minifier; standalone SmartCrusher remains byte-exact on no-ops. - Kill switch / disable path: existing routing configuration and HEADROOM_LIVE_ZONE_DISABLE_ARMS. - Unsafe override required: no. - Qualification impact: no-op, formatting, structural and CCR paths exercised separately. - Rollback path: revert the minifier integration; no persistence migration. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have made corresponding changes to the documentation - [x] I did not edit CHANGELOG.md — release-please owns it ## Additional Notes Addresses the [minifier review](https://github.com/headroomlabs-ai/headroom/pull/4044#issuecomment-6157583285). The maintainer’s main merge is retained. The latest follow-up only changes consumer tests. AI-assisted; no subagents. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
414 lines
15 KiB
Python
414 lines
15 KiB
Python
from __future__ import annotations
|
|
|
|
from types import SimpleNamespace
|
|
from typing import Any
|
|
|
|
from headroom.proxy.handlers.openai import (
|
|
OpenAIHandlerMixin,
|
|
_compact_openai_responses_tools,
|
|
)
|
|
from headroom.transforms.content_router import (
|
|
CompressionStrategy,
|
|
ContentRouter,
|
|
ContentRouterConfig,
|
|
_estimate_tokens,
|
|
)
|
|
|
|
|
|
def test_openai_tool_schema_compaction_preserves_invocation_shape() -> None:
|
|
verbose = " ".join(["Use this tool to read a file from the workspace."] * 40)
|
|
payload = {
|
|
"tools": [
|
|
{
|
|
"type": "function",
|
|
"name": "read_file",
|
|
"title": "Read File",
|
|
"description": verbose,
|
|
"parameters": {
|
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
"title": "ReadFileParameters",
|
|
"type": "object",
|
|
"properties": {
|
|
"path": {
|
|
"title": "Path",
|
|
"type": "string",
|
|
"description": verbose,
|
|
"examples": ["src/main.py"],
|
|
}
|
|
},
|
|
"required": ["path"],
|
|
"additionalProperties": False,
|
|
},
|
|
}
|
|
]
|
|
}
|
|
|
|
compacted, modified, before, after = _compact_openai_responses_tools(payload)
|
|
|
|
assert modified is True
|
|
assert after < before
|
|
tool = compacted["tools"][0]
|
|
assert tool["type"] == "function"
|
|
assert tool["name"] == "read_file"
|
|
assert "title" not in tool
|
|
assert tool["parameters"]["type"] == "object"
|
|
assert tool["parameters"]["required"] == ["path"]
|
|
assert tool["parameters"]["additionalProperties"] is False
|
|
assert tool["parameters"]["properties"]["path"]["type"] == "string"
|
|
assert "examples" not in tool["parameters"]["properties"]["path"]
|
|
assert tool["parameters"]["properties"]["path"]["description"] == " ".join(verbose.split())
|
|
|
|
|
|
def test_openai_tool_schema_compaction_preserves_property_named_title() -> None:
|
|
"""Issue #759: drop-key list must not strip property *names* under `properties`.
|
|
|
|
Schema annotations like ``title: "ReadFileParameters"`` on a schema object
|
|
are safe to drop. But a tool that has a field literally called ``title``
|
|
(or ``readOnly``, ``deprecated``, etc.) must survive compaction; removing
|
|
it while leaving ``required: ["title"]`` produces an invalid strict schema
|
|
that upstream (OpenAI / Codex) rejects.
|
|
"""
|
|
payload = {
|
|
"tools": [
|
|
{
|
|
"type": "function",
|
|
"name": "eval",
|
|
"description": "Evaluate cells.",
|
|
"parameters": {
|
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
"title": "EvalParameters",
|
|
"type": "object",
|
|
"properties": {
|
|
"cells": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"title": "CellItem",
|
|
"properties": {
|
|
"language": {"type": "string"},
|
|
"code": {"type": "string"},
|
|
"title": {"type": "string"},
|
|
},
|
|
"required": ["language", "code", "title"],
|
|
},
|
|
}
|
|
},
|
|
"required": ["cells"],
|
|
},
|
|
}
|
|
]
|
|
}
|
|
|
|
compacted, modified, before, after = _compact_openai_responses_tools(payload)
|
|
|
|
assert modified is True
|
|
assert after < before
|
|
|
|
params = compacted["tools"][0]["parameters"]
|
|
# Schema-level annotations are still dropped.
|
|
assert "title" not in params
|
|
assert "$schema" not in params
|
|
|
|
items = params["properties"]["cells"]["items"]
|
|
# "title" as a JSON Schema annotation on the items object is dropped.
|
|
assert "title" not in items
|
|
# "title" as a *property name* inside properties must be preserved.
|
|
assert "title" in items["properties"], (
|
|
"property named 'title' was incorrectly stripped by compaction"
|
|
)
|
|
assert items["required"] == ["language", "code", "title"]
|
|
|
|
|
|
def test_openai_tool_schema_compaction_is_deterministic() -> None:
|
|
payload = {
|
|
"tools": [
|
|
{
|
|
"type": "function",
|
|
"name": "mcp__serena__",
|
|
"description": " Semantic code tools.\n\nUse for symbol-aware edits. ",
|
|
"parameters": {
|
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
"$comment": "annotation only",
|
|
"type": "object",
|
|
"properties": {
|
|
"name_path_pattern": {
|
|
"type": "string",
|
|
"description": " Name path to match.\nKeeps full semantics. ",
|
|
"examples": ["Foo/bar"],
|
|
}
|
|
},
|
|
"required": ["name_path_pattern"],
|
|
"additionalProperties": False,
|
|
},
|
|
}
|
|
]
|
|
}
|
|
|
|
first, first_modified, first_before, first_after = _compact_openai_responses_tools(payload)
|
|
second, second_modified, second_before, second_after = _compact_openai_responses_tools(payload)
|
|
|
|
assert first_modified is True
|
|
assert second_modified is True
|
|
assert first_before == second_before
|
|
assert first_after == second_after
|
|
assert first == second
|
|
assert first["tools"][0]["description"] == ("Semantic code tools. Use for symbol-aware edits.")
|
|
prop = first["tools"][0]["parameters"]["properties"]["name_path_pattern"]
|
|
assert prop["description"] == "Name path to match. Keeps full semantics."
|
|
assert prop["type"] == "string"
|
|
assert "examples" not in prop
|
|
|
|
|
|
class _StubTokenizer:
|
|
def count_text(self, text: str) -> int:
|
|
return len(text.split())
|
|
|
|
|
|
class _StubProvider:
|
|
def get_token_counter(self, model: str) -> _StubTokenizer:
|
|
del model
|
|
return _StubTokenizer()
|
|
|
|
|
|
class _StubPipeline:
|
|
def __init__(self, router: ContentRouter):
|
|
self.transforms = [router]
|
|
|
|
|
|
class _HandlerHarness(OpenAIHandlerMixin):
|
|
"""Minimal subclass exposing just the deps the unit-extraction path
|
|
actually reads. The full HeadroomProxy ctor wires dozens of unrelated
|
|
services; this keeps the test focused on the gate behavior."""
|
|
|
|
def __init__(self, router: ContentRouter):
|
|
self.openai_pipeline: Any = _StubPipeline(router)
|
|
self.openai_provider: Any = _StubProvider()
|
|
|
|
|
|
def test_codex_input_list_payload_reaches_router_without_skip() -> None:
|
|
"""Codex's Responses payload uses `input=[...]` with no `messages` key.
|
|
The compression gate must accept either field as the items source —
|
|
otherwise the entire payload is silently passed through uncompressed,
|
|
which is the exact production bug surfaced in proxy.log analysis."""
|
|
|
|
router = ContentRouter(ContentRouterConfig())
|
|
handler = _HandlerHarness(router)
|
|
|
|
long_output = " ".join(["compressible"] * 200)
|
|
payload: dict[str, Any] = {
|
|
"type": "response.create",
|
|
"model": "gpt-5.5",
|
|
"input": [
|
|
{
|
|
"type": "function_call_output",
|
|
"call_id": "call_1",
|
|
"output": long_output,
|
|
}
|
|
],
|
|
# Note: no `messages` key at all — Codex doesn't send one.
|
|
}
|
|
|
|
updated, modified, _saved, _transforms, _units_by_cat, _chain, _attempted = (
|
|
handler._compress_openai_responses_live_text_units_with_router(
|
|
payload,
|
|
model="gpt-5.5",
|
|
request_id="hr_codex_test_0001",
|
|
)
|
|
)
|
|
|
|
# The gate must NOT have skipped the payload. If it had, `updated`
|
|
# would be the input payload identity-passed through with
|
|
# modified=False — but the deepcopy + splice always returns a new
|
|
# dict object when the path executes.
|
|
assert updated is not payload, "Codex-shape payload was skipped at the input/messages gate"
|
|
# Whether or not Kompress actually compresses 200 repeated words is
|
|
# not the point of this test; the point is that we *entered* the
|
|
# extraction loop. Modified may be True or False depending on
|
|
# Kompress availability in CI, so we only assert non-skip semantics.
|
|
assert isinstance(modified, bool)
|
|
|
|
|
|
def test_codex_payload_with_only_messages_field_also_reaches_router() -> None:
|
|
"""The Anthropic-style shape (messages=list, no input) must also
|
|
flow. This is the reverse of the Codex case and guards against a
|
|
future regression that swings the gate too far the other way."""
|
|
|
|
router = ContentRouter(ContentRouterConfig())
|
|
handler = _HandlerHarness(router)
|
|
|
|
payload: dict[str, Any] = {
|
|
"type": "response.create",
|
|
"model": "gpt-5.5",
|
|
"messages": [
|
|
{
|
|
"type": "function_call_output",
|
|
"call_id": "call_2",
|
|
"output": " ".join(["compressible"] * 200),
|
|
}
|
|
],
|
|
}
|
|
|
|
updated, _modified, _saved, _transforms, _units_by_cat, _chain, _attempted = (
|
|
handler._compress_openai_responses_live_text_units_with_router(
|
|
payload,
|
|
model="gpt-5.5",
|
|
request_id="hr_codex_test_0002",
|
|
)
|
|
)
|
|
|
|
assert updated is not payload, "messages-shape payload was skipped at the gate"
|
|
|
|
|
|
def test_compression_pass_debug_logs_are_suppressed(caplog) -> None:
|
|
"""Compression passes must not log per-pass payload events at INFO.
|
|
|
|
Re-entrant Codex websocket passes share one `request_id`; three passes
|
|
over two distinct payloads must emit no ``event=codex_compression_*``
|
|
records, which would carry request content.
|
|
"""
|
|
|
|
import logging as _logging
|
|
|
|
router = ContentRouter(ContentRouterConfig())
|
|
handler = _HandlerHarness(router)
|
|
|
|
payload_a: dict[str, Any] = {
|
|
"type": "response.create",
|
|
"model": "gpt-5.5",
|
|
"input": [
|
|
{
|
|
"type": "function_call_output",
|
|
"call_id": "call_1",
|
|
"output": " ".join(["alpha"] * 200),
|
|
}
|
|
],
|
|
}
|
|
payload_b: dict[str, Any] = {
|
|
"type": "response.create",
|
|
"model": "gpt-5.5",
|
|
"input": [
|
|
{
|
|
"type": "function_call_output",
|
|
"call_id": "call_1",
|
|
"output": " ".join(["bravo"] * 200),
|
|
}
|
|
],
|
|
}
|
|
|
|
caplog.set_level(_logging.INFO, logger="headroom.proxy")
|
|
handler._compress_openai_responses_payload(
|
|
payload_a, model="gpt-5.5", request_id="hr_shared_request"
|
|
)
|
|
handler._compress_openai_responses_payload(
|
|
payload_b, model="gpt-5.5", request_id="hr_shared_request"
|
|
)
|
|
# A repeat of the first payload, as a re-entrant pass would send.
|
|
handler._compress_openai_responses_payload(
|
|
payload_a, model="gpt-5.5", request_id="hr_shared_request"
|
|
)
|
|
|
|
assert not any("event=codex_compression_" in record.getMessage() for record in caplog.records)
|
|
|
|
|
|
def test_codex_payload_without_either_field_is_skipped() -> None:
|
|
"""The gate must still reject malformed payloads — `input` and
|
|
`messages` both absent (or non-list) is the genuine skip condition."""
|
|
|
|
router = ContentRouter(ContentRouterConfig())
|
|
handler = _HandlerHarness(router)
|
|
|
|
payload: dict[str, Any] = {
|
|
"type": "response.create",
|
|
"model": "gpt-5.5",
|
|
# No input, no messages — genuinely nothing to compress.
|
|
}
|
|
|
|
updated, modified, saved, transforms, units_by_cat, chain, attempted = (
|
|
handler._compress_openai_responses_live_text_units_with_router(
|
|
payload,
|
|
model="gpt-5.5",
|
|
request_id="hr_codex_test_0003",
|
|
)
|
|
)
|
|
|
|
assert updated is payload
|
|
assert modified is False
|
|
assert units_by_cat == {}
|
|
assert chain == []
|
|
assert attempted == 0
|
|
assert saved == 0
|
|
assert transforms == []
|
|
|
|
|
|
def test_content_router_retries_kompress_when_structured_strategy_noops(monkeypatch) -> None:
|
|
router = ContentRouter(ContentRouterConfig(enable_smart_crusher=True))
|
|
content = " ".join("x" for _ in range(200))
|
|
|
|
class NoopCrusher:
|
|
def crush(self, value: str, query: str = "", bias: float = 1.0):
|
|
return SimpleNamespace(compressed=value)
|
|
|
|
monkeypatch.setattr(router, "_get_smart_crusher", lambda: NoopCrusher())
|
|
monkeypatch.setattr(
|
|
router,
|
|
"_try_ml_compressor",
|
|
lambda value, context, question=None: ("short summary", 2),
|
|
)
|
|
|
|
compressed, compressed_tokens, strategy_chain, _reversible = router._apply_strategy_to_content(
|
|
content,
|
|
CompressionStrategy.SMART_CRUSHER,
|
|
context="",
|
|
)
|
|
|
|
assert compressed == "short summary"
|
|
# Measured with the router's estimator, the unit of the strategy result.
|
|
assert compressed_tokens == _estimate_tokens("short summary")
|
|
# The fallback chain must record both strategies it tried.
|
|
assert strategy_chain == ["smart_crusher", "kompress"]
|
|
|
|
|
|
def test_responses_turn_hook_message_fold_is_applied_and_counted() -> None:
|
|
"""On the Responses path a turn hook may fold the `input` items (in place),
|
|
not just tools. The fold must be written back to the outbound payload AND its
|
|
token saving added to tokens_saved — before, this path only wrote tools back,
|
|
so a message fold was silently dropped and uncounted."""
|
|
from headroom.proxy.turn_hooks import clear_turn_hooks, register_turn_hook
|
|
|
|
class FoldInput:
|
|
name = "fold_input"
|
|
|
|
def on_request(self, ctx: Any) -> None:
|
|
# Fold a big function_call_output IN PLACE (mutate the dict; identity
|
|
# of ctx.messages is unchanged) — the case an identity gate would miss.
|
|
for item in ctx.messages:
|
|
if isinstance(item, dict) and isinstance(item.get("output"), str):
|
|
item["output"] = "folded"
|
|
|
|
router = ContentRouter(ContentRouterConfig())
|
|
handler = _HandlerHarness(router)
|
|
payload: dict[str, Any] = {
|
|
"type": "response.create",
|
|
"model": "gpt-5.5",
|
|
"input": [
|
|
{
|
|
"type": "function_call_output",
|
|
"call_id": "c1",
|
|
"output": " ".join(["compressible"] * 300),
|
|
}
|
|
],
|
|
}
|
|
|
|
clear_turn_hooks()
|
|
register_turn_hook(FoldInput())
|
|
try:
|
|
working, _modified, tokens_saved, *_ = handler._compress_openai_responses_payload(
|
|
payload, model="gpt-5.5", request_id="hr_test"
|
|
)
|
|
finally:
|
|
clear_turn_hooks()
|
|
|
|
assert working["input"][0]["output"] == "folded" # fold applied to the outbound payload
|
|
assert tokens_saved > 0 # ...and the message-fold saving is counted
|
|
assert payload["input"][0]["output"] != "folded" # original untouched (deep-copied)
|