mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description Change `crush_object` in `crushers.rs` to preserve every property instead of treating independent object fields as interchangeable sampled records; remove the key-selection algorithm and update its inline tests, including the test that currently requires default-mode key loss. Keep its existing production call in `SmartCrusher::process_value_with_hook` in `crusher.rs`, which first recursively compresses nested values, so legitimate large-array compaction and CCR-backed row offload remain available without deleting their enclosing keys. In `smart_crush_content_with_hook`, return the original input bytes with an unmodified result when an object and its descendants were left unchanged, covering whitespace and escaping without falsely advertising compression; retain serialization and strategy attribution when descendants actually change. SmartCrusher removes the non-null `tags` and `subtasks` properties from an MCP task response while retaining nulls and small fields, without emitting a CCR marker or recovery hash. The issue contains complete before/after JSON and a `/v1/compress` reproduction, followed by a reporter retest on 0.37.0 confirming the same behavior. The full example is 813 bytes before compression, with 299 request tokens reported; the approximate 350-byte description and claimed 500-token default should not be used as implementation constants. The clone's Python and Rust defaults are 200 tokens, and `crush_object` uses a per-property estimate including overhead, explaining how this object can cross its gate. Closes #3634 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Change `crush_object` in `crushers.rs` to preserve every property instead of treating independent object fields as interchangeable sampled records; remove the key-selection algorithm and update its inline tests, including the test that currently requires default-mode key loss. Keep its existing production call in `SmartCrusher::process_value_with_hook` in `crusher.rs`, which first recursively compresses nested values, so legitimate large-array compaction and CCR-backed row offload remain available without deleting their enclosing keys. In `smart_crush_content_with_hook`, return the original input bytes with an unmodified result when an object and its descendants were left unchanged, covering whitespace and escaping without falsely advertising compression; retain serialization and strategy attribution when descendants actually change. ## Testing - [ ] Unit tests pass (`pytest`) Not run in this workspace, so this is left for CI to confirm. - [ ] Linting passes (`ruff check .`) Not run in this workspace, so this is left for CI to confirm. - [ ] Type checking passes (`mypy headroom`) Not run in this workspace, so this is left for CI to confirm. - [x] New tests added for new functionality - [ ] Manual testing performed Not verified: this needs a person on the named hardware or environment. - Embed the complete issue JSON as a local fixture literal, preserving its original order and encoded tags string. With default configuration, all properties and nested task records survive; compact input remains byte-identical, and no fake CCR hash or key-drop strategy is reported. - Exercise wide objects above the existing floor with arbitrary names, long strings, nested dictionaries, nulls, booleans, and empty arrays. ### Test Output ```text All 28 required checks are green one214a5e(18 workflows): test (1) (2) (3) (4), test-extras, test-agno, test (ubuntu), parity, audit, CodeQL, simulator e2e (ubuntu/macos/windows), python-314-wheels (ubuntu/macos), wheels (x86_64-apple-darwin, aarch64-apple-darwin, x86_64-unknown-linux-gnu). Run: https://github.com/headroomlabs-ai/headroom/pull/3648/checks test (1): https://github.com/headroomlabs-ai/headroom/actions/runs/35326929298/job/105542328074 ``` ## Real Behavior Proof - Environment: GitHub Actions hosted runners (ubuntu-latest, macos-latest, windows-latest) one214a5e. Not run on local hardware. - Exact command / steps: the repository's own CI matrix on this head; the new inline tests in `crushers.rs` embed the complete issue JSON from #3634 as a fixture literal and assert every property survives with default configuration. - Observed result: all properties and nested task records survive; compact input remains byte-identical; no CCR hash or key-drop strategy is reported when nothing changed. 28/28 checks green. - Not tested: no manual `/v1/compress` reproduction against a running server. The issue's own reproduction and the reporter's 0.37.0 retest stand as the field evidence; I did not re-run them. ## Runtime Rollout Safety - Rollout-managed feature(s): none. This is not behind a rollout flag. - Minimum rollout channel: n/a, ships with the code. - Stable/default behavior changed: yes. `crush_object` no longer drops object properties in default mode, and `smart_crush_content_with_hook` returns the original bytes unmodified when nothing changed. That is the bug fix; the prior behavior was silent data loss. - Kill switch / disable path: none. Revert the commit to restore prior behavior. - Unsafe override required: no. - Qualification impact: one existing inline test that asserted default-mode key loss was updated, since it encoded the bug as expected behavior. - Rollback path: `git revert` of this PR's merge commit. No migration, no persisted state, no config change. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) Not run in this workspace, so this is left for CI to confirm. ## Screenshots (if applicable) ## Additional Notes Nothing beyond what is described above. Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
323 lines
11 KiB
Python
323 lines
11 KiB
Python
"""Compact machine-generated JSON must not evade compression (token-estimate bug).
|
|
|
|
Whitespace-split token counting made a compact JSON payload (no spaces —
|
|
the default output of json.dumps with separators, JSON.stringify, boto3)
|
|
count as ~1 token, so compression ratios computed as ~1.0 and the
|
|
min_ratio gate rejected SmartCrusher's real output.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import random
|
|
|
|
import pytest
|
|
|
|
from headroom.transforms.content_router import (
|
|
ContentRouter,
|
|
ContentRouterConfig,
|
|
_estimate_tokens,
|
|
)
|
|
|
|
|
|
def test_estimate_tokens_monotone_on_compact_json() -> None:
|
|
small = json.dumps([{"a": 1}] * 5, separators=(",", ":"))
|
|
large = json.dumps([{"a": 1}] * 500, separators=(",", ":"))
|
|
assert _estimate_tokens(large) > _estimate_tokens(small) > 1
|
|
|
|
|
|
def test_compact_json_tool_result_compresses() -> None:
|
|
tokenizer = pytest.importorskip("headroom.tokenizers.estimator")
|
|
from headroom.tokenizer import Tokenizer
|
|
|
|
tok = Tokenizer(tokenizer.EstimatingTokenCounter())
|
|
random.seed(42)
|
|
rows = [
|
|
{
|
|
"serviceArn": f"arn:aws:ecs:us-east-1:123456789012:service/x/svc-{s:03d}",
|
|
"serviceName": f"svc-{s:03d}",
|
|
"status": "ACTIVE",
|
|
"desiredCount": random.randint(1, 6),
|
|
"runningCount": random.randint(0, 6),
|
|
}
|
|
for s in range(150)
|
|
]
|
|
payload = json.dumps(rows, separators=(",", ":"))
|
|
assert " " not in payload[:200] # genuinely compact
|
|
messages = [
|
|
{"role": "user", "content": "Investigate."},
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{"type": "text", "text": "Checking."},
|
|
{"type": "tool_use", "id": "toolu_1", "name": "list_services", "input": {}},
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [{"type": "tool_result", "tool_use_id": "toolu_1", "content": payload}],
|
|
},
|
|
{"role": "user", "content": "Summarize in one sentence."},
|
|
]
|
|
router = ContentRouter(ContentRouterConfig(skip_user_messages=False))
|
|
before = tok.count_messages(messages)
|
|
result = router.apply(
|
|
[json.loads(json.dumps(m)) for m in messages],
|
|
tok,
|
|
context="Summarize",
|
|
frozen_message_count=0,
|
|
)
|
|
after = tok.count_messages(result.messages)
|
|
assert after < before * 0.9, (before, after, result.transforms_applied[:5])
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Issue #3634: SmartCrusher must not drop object fields (tags/subtasks)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Exact MCP `saga-mcp_task_get` payload from the issue (813 bytes, original
|
|
# key order, tags encoded as a JSON string). Do not pretty-print or reorder.
|
|
MCP_TASK_GET_JSON = (
|
|
'{"id":79,"epic_id":9,"title":"test","description":null,'
|
|
'"status":"done","priority":"medium","sort_order":0,'
|
|
'"assigned_to":null,"estimated_hours":null,"actual_hours":null,'
|
|
'"due_date":null,"source_ref":null,"metadata":"{}",'
|
|
'"created_at":"2026-09-15 20:32:06","updated_at":"2026-09-16 11:59:27",'
|
|
'"description_locked":0,"is_deleted":0,"deleted_at":null,'
|
|
'"deleted_by":null,"delete_reason":null,"epic_name":"latency-routing",'
|
|
'"tags":"[\\"cherry-pick\\",\\"dedicated branch\\"]",'
|
|
'"subtasks":[{"id":163,"task_id":79,"title":"test2","status":"todo",'
|
|
'"sort_order":1,"created_at":"2026-09-15 20:32:13",'
|
|
'"updated_at":"2026-09-16 14:02:26"},'
|
|
'{"id":165,"task_id":79,"title":"test4","status":"todo","sort_order":2,'
|
|
'"created_at":"2026-09-15 20:38:47","updated_at":"2026-09-17 20:29:04"}],'
|
|
'"notes":[],"comments":[],"depends_on":[],"dependents":[]}'
|
|
)
|
|
|
|
_TOOL_CALL_ID = "call_task_get_79"
|
|
_TOOL_NAME = "saga-mcp_task_get"
|
|
|
|
|
|
def _offline_tokenizer():
|
|
tokenizer = pytest.importorskip("headroom.tokenizers.estimator")
|
|
from headroom.tokenizer import Tokenizer
|
|
|
|
return Tokenizer(tokenizer.EstimatingTokenCounter())
|
|
|
|
|
|
def _object_preserving_router() -> ContentRouter:
|
|
# Real ContentRouter + Rust SmartCrusher. Disable unrelated ML / log
|
|
# fallbacks so a no-savings crush cannot be rewritten by Kompress.
|
|
return ContentRouter(
|
|
ContentRouterConfig(
|
|
skip_user_messages=False,
|
|
enable_kompress=False,
|
|
enable_log_compressor=False,
|
|
enable_search_compressor=False,
|
|
relevance_split=False,
|
|
protect_analysis_context=False,
|
|
exclude_tools=set(),
|
|
)
|
|
)
|
|
|
|
|
|
def _openai_task_messages(payload: str) -> list[dict]:
|
|
return [
|
|
{"role": "user", "content": "read task 79"},
|
|
{
|
|
"role": "assistant",
|
|
"content": None,
|
|
"tool_calls": [
|
|
{
|
|
"id": _TOOL_CALL_ID,
|
|
"type": "function",
|
|
"function": {"name": _TOOL_NAME, "arguments": '{"id":79}'},
|
|
}
|
|
],
|
|
},
|
|
{
|
|
"role": "tool",
|
|
"tool_call_id": _TOOL_CALL_ID,
|
|
"content": payload,
|
|
},
|
|
]
|
|
|
|
|
|
def _anthropic_task_messages(payload: str) -> list[dict]:
|
|
return [
|
|
{"role": "user", "content": "read task 79"},
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": _TOOL_CALL_ID,
|
|
"name": _TOOL_NAME,
|
|
"input": {"id": 79},
|
|
}
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": _TOOL_CALL_ID,
|
|
"content": payload,
|
|
}
|
|
],
|
|
},
|
|
]
|
|
|
|
|
|
def _apply(messages: list[dict]):
|
|
router = _object_preserving_router()
|
|
tok = _offline_tokenizer()
|
|
return router.apply(
|
|
[json.loads(json.dumps(m)) for m in messages],
|
|
tok,
|
|
context="read task 79",
|
|
frozen_message_count=0,
|
|
protect_recent=0,
|
|
protect_analysis_context=False,
|
|
)
|
|
|
|
|
|
def _openai_tool_content(messages: list[dict]) -> str:
|
|
for msg in messages:
|
|
if msg.get("role") == "tool" and msg.get("tool_call_id") == _TOOL_CALL_ID:
|
|
content = msg.get("content")
|
|
assert isinstance(content, str), content
|
|
return content
|
|
raise AssertionError(f"OpenAI tool message {_TOOL_CALL_ID} missing")
|
|
|
|
|
|
def _anthropic_tool_content(messages: list[dict]) -> str:
|
|
for msg in messages:
|
|
content = msg.get("content")
|
|
if not isinstance(content, list):
|
|
continue
|
|
for block in content:
|
|
if block.get("type") == "tool_result" and block.get("tool_use_id") == _TOOL_CALL_ID:
|
|
inner = block.get("content")
|
|
assert isinstance(inner, str), inner
|
|
return inner
|
|
raise AssertionError(f"Anthropic tool_result {_TOOL_CALL_ID} missing")
|
|
|
|
|
|
def _assert_task_payload_preserved(original: str, compressed: str) -> dict:
|
|
assert "<<ccr:" not in compressed, compressed[:200]
|
|
parsed = json.loads(compressed)
|
|
expected = json.loads(original)
|
|
assert parsed == expected, (sorted(expected), sorted(parsed))
|
|
assert "tags" in parsed
|
|
assert "subtasks" in parsed
|
|
assert parsed["tags"] == '["cherry-pick","dedicated branch"]'
|
|
assert len(parsed["subtasks"]) == 2
|
|
assert parsed["subtasks"][0]["title"] == "test2"
|
|
assert parsed["subtasks"][1]["title"] == "test4"
|
|
return parsed
|
|
|
|
|
|
def _wide_renamed_task_json() -> str:
|
|
"""Renamed, oversized variant so a below-threshold skip cannot hide key drop.
|
|
|
|
Field names are arbitrary (not tags/subtasks) and expensive values sit at
|
|
the beginning, middle, and end so a boundary/stride selector cannot
|
|
accidentally keep them all.
|
|
"""
|
|
original = json.loads(MCP_TASK_GET_JSON)
|
|
expensive = {
|
|
"nested": {"label": "child", "note": "n" * 180},
|
|
"items": [],
|
|
"ok": True,
|
|
"missing": None,
|
|
}
|
|
wide: dict = {
|
|
"leading_blob": expensive,
|
|
"label_bundle": original.pop("tags"),
|
|
"child_records": original.pop("subtasks"),
|
|
}
|
|
wide.update(original)
|
|
for i in range(20):
|
|
wide[f"extra_field_{i:02d}"] = (
|
|
f"this is a relatively long value string for entry number {i} with content"
|
|
)
|
|
wide["trailing_blob"] = {
|
|
"nested": {"label": "tail", "note": "t" * 180},
|
|
"items": [],
|
|
"ok": False,
|
|
"missing": None,
|
|
}
|
|
payload = json.dumps(wide, separators=(",", ":"))
|
|
assert len(payload) > 1500
|
|
assert len(wide) > 8
|
|
return payload
|
|
|
|
|
|
def test_mcp_task_fixture_is_the_issue_payload() -> None:
|
|
assert len(MCP_TASK_GET_JSON) == 813
|
|
parsed = json.loads(MCP_TASK_GET_JSON)
|
|
assert list(parsed)[21] == "tags"
|
|
assert list(parsed)[22] == "subtasks"
|
|
assert parsed["tags"] == '["cherry-pick","dedicated branch"]'
|
|
|
|
|
|
def test_mcp_task_object_preserved_openai_tool_role() -> None:
|
|
messages = _openai_task_messages(MCP_TASK_GET_JSON)
|
|
result = _apply(messages)
|
|
content = _openai_tool_content(result.messages)
|
|
assert content == MCP_TASK_GET_JSON
|
|
_assert_task_payload_preserved(MCP_TASK_GET_JSON, content)
|
|
# Tool-call pairing stays intact.
|
|
tool_msg = next(m for m in result.messages if m.get("role") == "tool")
|
|
assistant = next(m for m in result.messages if m.get("role") == "assistant")
|
|
assert tool_msg["tool_call_id"] == _TOOL_CALL_ID
|
|
assert assistant["tool_calls"][0]["id"] == _TOOL_CALL_ID
|
|
assert assistant["tool_calls"][0]["function"]["name"] == _TOOL_NAME
|
|
assert not any("object:adaptive" in t for t in result.transforms_applied)
|
|
|
|
|
|
def test_mcp_task_object_preserved_anthropic_tool_result() -> None:
|
|
messages = _anthropic_task_messages(MCP_TASK_GET_JSON)
|
|
result = _apply(messages)
|
|
content = _anthropic_tool_content(result.messages)
|
|
assert content == MCP_TASK_GET_JSON
|
|
_assert_task_payload_preserved(MCP_TASK_GET_JSON, content)
|
|
assistant = next(m for m in result.messages if m.get("role") == "assistant")
|
|
assert assistant["content"][0]["id"] == _TOOL_CALL_ID
|
|
user_blocks = next(
|
|
m["content"]
|
|
for m in result.messages
|
|
if m.get("role") == "user" and isinstance(m.get("content"), list)
|
|
)
|
|
assert user_blocks[0]["tool_use_id"] == _TOOL_CALL_ID
|
|
assert not any("object:adaptive" in t for t in result.transforms_applied)
|
|
|
|
|
|
def test_wide_renamed_object_fields_preserved_openai_tool_role() -> None:
|
|
payload = _wide_renamed_task_json()
|
|
result = _apply(_openai_task_messages(payload))
|
|
content = _openai_tool_content(result.messages)
|
|
assert "<<ccr:" not in content
|
|
parsed = json.loads(content)
|
|
expected = json.loads(payload)
|
|
assert parsed == expected
|
|
assert "label_bundle" in parsed
|
|
assert "child_records" in parsed
|
|
assert "leading_blob" in parsed
|
|
assert "trailing_blob" in parsed
|
|
assert parsed["label_bundle"] == '["cherry-pick","dedicated branch"]'
|
|
assert len(parsed["child_records"]) == 2
|
|
assert parsed["leading_blob"]["nested"]["note"] == "n" * 180
|
|
assert parsed["trailing_blob"]["nested"]["note"] == "t" * 180
|
|
|
|
|
|
def test_malformed_json_tool_result_passes_through() -> None:
|
|
payload = "{" + "not-json, still a tool result. " * 40
|
|
assert len(payload) > 500
|
|
result = _apply(_openai_task_messages(payload))
|
|
content = _openai_tool_content(result.messages)
|
|
assert content == payload
|
|
assert "<<ccr:" not in content
|