mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description Enforce role, mutable, cache zone, and user/assistant message protection guards in small-unit batch compression. Previously, `compress_unit_with_router` enforced these guards on individual units, but `build_compression_batches` and `compress_batch_with_router` bypassed them for batched small units, allowing protected messages (e.g., system, developer, unpermitted user/assistant, immutable, non-live cache zones) to be compressed when grouped. Closes #4480 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Extracted `check_unit_protection(unit)` in `headroom/transforms/compression_units.py` to share early refusal checks between single-unit and batch paths. - Updated `build_compression_batches` in `headroom/transforms/compression_batches.py` to skip protected units, leaving them in `skipped` so callers process them through the single-unit path. - Updated `compress_batch_with_router` in `headroom/transforms/compression_batches.py` to refuse batches containing protected units, returning verbatim passthrough results with the corresponding refusal reasons and reason categories. - Added comprehensive regression tests in `tests/test_compression_batches.py` covering all 6 protection guard scenarios and verifying that permitted user and assistant messages can each be batched. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text ============================= test session starts ============================== platform linux -- Python 3.14.8, pytest-9.0.3, pluggy-1.6.0 collected 14 items tests/test_compression_batches.py::test_small_units_over_floor_form_one_batch PASSED [ 7%] tests/test_compression_batches.py::test_batch_compresses_entries_with_one_router_call PASSED [ 14%] tests/test_compression_batches.py::test_under_floor_tail_is_skipped_without_a_batch PASSED [ 21%] tests/test_compression_batches.py::test_sixteen_under_floor_entries_are_skipped_before_a_new_batch_starts PASSED [ 28%] tests/test_compression_batches.py::test_missing_batch_tags_passes_through_every_entry PASSED [ 35%] tests/test_compression_batches.py::test_many_small_entries_create_no_more_than_sixteen_per_batch PASSED [ 42%] tests/test_compression_batches.py::test_batch_keeps_structured_shell_output_without_a_ccr_marker PASSED [ 50%] tests/test_compression_batches.py::test_batch_preserves_ccr_markers_when_router_would_remove_them PASSED [ 57%] tests/test_compression_batches.py::test_batch_rejects_ccr_marker_moved_to_another_entry PASSED [ 64%] tests/test_compression_batches.py::test_batch_uses_utf8_bytes_for_cjk_small_units PASSED [ 71%] tests/test_compression_batches.py::test_batch_after_the_request_deadline_passes_through_without_routing PASSED [ 78%] tests/test_compression_batches.py::test_build_compression_batches_skips_protected_units PASSED [ 85%] tests/test_compression_batches.py::test_batch_refuses_protected_units_without_routing PASSED [ 92%] tests/test_compression_batches.py::test_batch_allows_permitted_user_and_assistant_units PASSED [100%] ======================== 14 passed, 1 warning in 0.38s ========================= ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 x86_64, Python 3.14.8, real `headroom._core` Rust extension, ContentRouter and TiktokenCounter - Exact command / steps: Executed standalone probe script initializing `ContentRouter()` and `TiktokenCounter('gpt-4o')` with 2 small developer messages (`role="developer"`, 396 bytes total). Ran `build_compression_batches` and `compress_batch_with_router`. - Observed result: `build_compression_batches` returned `batches=0, skipped=2`. `compress_batch_with_router` returned `modified=False, reason="protected_system_message", reason_category="protected_role"` for both slots without altering text. ```text build_compression_batches (developer): batches = 0 skipped = 2 slot=(0, None) modified=False reason=protected_system_message category=protected_role slot=(1, None) modified=False reason=protected_system_message category=protected_role ``` - Not tested: Live external LLM API endpoints requiring network credentials. ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: stable - Stable/default behavior changed: Protected units that are smaller than unit min_bytes will no longer be erroneously compressed when grouped into batches. - Kill switch / disable path: N/A - Unsafe override required: None - Qualification impact: None - Rollback path: Revert this PR ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes AI-assisted contribution: investigated, patched, and verified with Antigravity / Gemini. --------- Co-authored-by: pei711 <199601925+pei711@users.noreply.github.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
pei711
JD Davis
parent
7c11d462e6
commit
40cd7e7503
@@ -16,7 +16,9 @@ from .compression_units import (
|
||||
RoutedCompressionUnit,
|
||||
TokenCounterLike,
|
||||
UnitCompressionResult,
|
||||
_categorize_reason,
|
||||
_is_structured_shell_output,
|
||||
check_unit_protection,
|
||||
)
|
||||
from .content_router import RouterCompressionResult
|
||||
from .tag_protector import protect_tags, restore_tags
|
||||
@@ -104,7 +106,12 @@ def build_compression_batches(
|
||||
for entry in entries:
|
||||
entry_bytes = _text_bytes(entry.routed.unit.text)
|
||||
entry_key = _compatibility_key(entry)
|
||||
if entry_bytes >= min_batch_bytes or entry_bytes > max_batch_bytes:
|
||||
protection_reason = check_unit_protection(entry.routed.unit)
|
||||
if (
|
||||
protection_reason is not None
|
||||
or entry_bytes >= min_batch_bytes
|
||||
or entry_bytes > max_batch_bytes
|
||||
):
|
||||
flush()
|
||||
skipped.append(entry)
|
||||
continue
|
||||
@@ -197,7 +204,7 @@ def _passthrough_batch_results(
|
||||
batch: CompressionBatch,
|
||||
*,
|
||||
tokenizer: TokenCounterLike,
|
||||
reason: str,
|
||||
reason: str | None = None,
|
||||
router_result: RouterCompressionResult | None = None,
|
||||
) -> list[tuple[object, UnitCompressionResult]]:
|
||||
strategy = (
|
||||
@@ -217,11 +224,13 @@ def _passthrough_batch_results(
|
||||
tokens_saved=0,
|
||||
transforms_applied=[],
|
||||
strategy=strategy,
|
||||
reason=reason,
|
||||
reason=reason or check_unit_protection(entry.routed.unit) or "batch_invalid",
|
||||
router_result=router_result,
|
||||
text_bytes=_text_bytes(entry.routed.unit.text),
|
||||
min_bytes=entry.routed.unit.min_bytes,
|
||||
reason_category=reason,
|
||||
reason_category=_categorize_reason(
|
||||
reason or check_unit_protection(entry.routed.unit) or "batch_invalid"
|
||||
),
|
||||
),
|
||||
)
|
||||
for entry in batch.entries
|
||||
@@ -241,6 +250,9 @@ def compress_batch_with_router(
|
||||
``deadline_started_at`` works as in ``compress_unit_with_router``.
|
||||
"""
|
||||
|
||||
if any(check_unit_protection(entry.routed.unit) is not None for entry in batch.entries):
|
||||
return _passthrough_batch_results(batch, tokenizer=tokenizer)
|
||||
|
||||
if deadline_started_at is not None and not router.share_request_deadline(deadline_started_at):
|
||||
return _passthrough_batch_results(batch, tokenizer=tokenizer, reason="deadline_exceeded")
|
||||
|
||||
|
||||
@@ -218,6 +218,21 @@ def _compress_marker_free_text(
|
||||
)
|
||||
|
||||
|
||||
def check_unit_protection(unit: CompressionUnit) -> str | None:
|
||||
"""Return the early refusal reason if a unit is protected, else None."""
|
||||
if not unit.mutable:
|
||||
return "immutable"
|
||||
if unit.role == "user" and unit.metadata.get("compress_user") != "true":
|
||||
return "protected_user_message"
|
||||
if unit.role in {"system", "developer"}:
|
||||
return "protected_system_message"
|
||||
if unit.role == "assistant" and unit.metadata.get("compress_assistant") != "true":
|
||||
return "protected_assistant_message"
|
||||
if unit.cache_zone != "live":
|
||||
return f"cache_zone_{unit.cache_zone}"
|
||||
return None
|
||||
|
||||
|
||||
def compress_unit_with_router(
|
||||
unit: CompressionUnit,
|
||||
*,
|
||||
@@ -263,16 +278,9 @@ def compress_unit_with_router(
|
||||
kw["reason_category"] = _categorize_reason(reason_val)
|
||||
return replace(base, **kw) # type: ignore[arg-type]
|
||||
|
||||
if not unit.mutable:
|
||||
return _with_reason(reason="immutable")
|
||||
if unit.role == "user" and unit.metadata.get("compress_user") != "true":
|
||||
return _with_reason(reason="protected_user_message")
|
||||
if unit.role in {"system", "developer"}:
|
||||
return _with_reason(reason="protected_system_message")
|
||||
if unit.role == "assistant" and unit.metadata.get("compress_assistant") != "true":
|
||||
return _with_reason(reason="protected_assistant_message")
|
||||
if unit.cache_zone != "live":
|
||||
return _with_reason(reason=f"cache_zone_{unit.cache_zone}")
|
||||
protection_reason = check_unit_protection(unit)
|
||||
if protection_reason is not None:
|
||||
return _with_reason(reason=protection_reason)
|
||||
if text_bytes < unit.min_bytes:
|
||||
return _with_reason(reason="below_unit_floor")
|
||||
if deadline_started_at is not None and not router.share_request_deadline(deadline_started_at):
|
||||
|
||||
@@ -254,3 +254,165 @@ def test_batch_after_the_request_deadline_passes_through_without_routing(monkeyp
|
||||
|
||||
assert [result.compressed for _, result in results] == ["x" * 150] * 4
|
||||
assert {result.reason for _, result in results} == {"deadline_exceeded"}
|
||||
|
||||
|
||||
def test_build_compression_batches_skips_protected_units():
|
||||
from headroom.transforms.compression_units import compress_unit_with_router
|
||||
|
||||
cases = [
|
||||
# (kwargs for CompressionUnit, expected_reason, expected_category)
|
||||
({"mutable": False}, "immutable", "immutable"),
|
||||
({"role": "system"}, "protected_system_message", "protected_role"),
|
||||
({"role": "developer"}, "protected_system_message", "protected_role"),
|
||||
({"role": "user", "metadata": {}}, "protected_user_message", "protected_role"),
|
||||
({"role": "assistant", "metadata": {}}, "protected_assistant_message", "protected_role"),
|
||||
({"cache_zone": "prefix"}, "cache_zone_prefix", "cache_zone"),
|
||||
]
|
||||
|
||||
entries = []
|
||||
for idx, (kw, _expected_reason, _expected_category) in enumerate(cases):
|
||||
unit_kwargs = {
|
||||
"text": "x" * 150,
|
||||
"provider": "openai",
|
||||
"endpoint": "responses",
|
||||
"role": "tool",
|
||||
"item_type": "message",
|
||||
"cache_zone": "live",
|
||||
"mutable": True,
|
||||
"min_bytes": 512,
|
||||
}
|
||||
unit_kwargs.update(kw)
|
||||
unit = CompressionUnit(**unit_kwargs)
|
||||
entries.append(
|
||||
CompressionBatchEntry(
|
||||
entry_id=f"u{idx}",
|
||||
routed=RoutedCompressionUnit(unit=unit, slot=(idx, ("output", None))),
|
||||
)
|
||||
)
|
||||
|
||||
batches, skipped = build_compression_batches(entries, min_batch_bytes=512)
|
||||
|
||||
assert batches == []
|
||||
assert len(skipped) == len(cases)
|
||||
|
||||
router = _ShorteningRouter()
|
||||
tokenizer = _CharacterCounter()
|
||||
for entry, (_kw, expected_reason, expected_category) in zip(skipped, cases, strict=True):
|
||||
single_result = compress_unit_with_router(
|
||||
entry.routed.unit,
|
||||
router=router,
|
||||
tokenizer=tokenizer,
|
||||
)
|
||||
assert not single_result.modified
|
||||
assert single_result.reason == expected_reason
|
||||
assert single_result.reason_category == expected_category
|
||||
assert router.calls == 0
|
||||
|
||||
|
||||
def test_batch_refuses_protected_units_without_routing():
|
||||
from headroom.transforms.compression_batches import CompressionBatch
|
||||
|
||||
cases = [
|
||||
({"mutable": False}, "immutable", "immutable"),
|
||||
({"role": "system"}, "protected_system_message", "protected_role"),
|
||||
({"role": "developer"}, "protected_system_message", "protected_role"),
|
||||
({"role": "user", "metadata": {}}, "protected_user_message", "protected_role"),
|
||||
({"role": "assistant", "metadata": {}}, "protected_assistant_message", "protected_role"),
|
||||
({"cache_zone": "prefix"}, "cache_zone_prefix", "cache_zone"),
|
||||
]
|
||||
|
||||
for kw, expected_reason, expected_category in cases:
|
||||
entries = []
|
||||
for idx in range(2):
|
||||
unit_kwargs = {
|
||||
"text": "x" * 150,
|
||||
"provider": "openai",
|
||||
"endpoint": "responses",
|
||||
"role": "tool",
|
||||
"item_type": "message",
|
||||
"cache_zone": "live",
|
||||
"mutable": True,
|
||||
"min_bytes": 512,
|
||||
}
|
||||
unit_kwargs.update(kw)
|
||||
unit = CompressionUnit(**unit_kwargs)
|
||||
entries.append(
|
||||
CompressionBatchEntry(
|
||||
entry_id=f"u{idx}",
|
||||
routed=RoutedCompressionUnit(unit=unit, slot=(idx, ("output", None))),
|
||||
)
|
||||
)
|
||||
|
||||
batch = CompressionBatch(entries=tuple(entries), text_bytes=300)
|
||||
|
||||
def _not_called(*_args, **_kwargs):
|
||||
raise AssertionError("a protected batch must not enter the router")
|
||||
|
||||
class _AssertRouter:
|
||||
compress = _not_called
|
||||
|
||||
results = compress_batch_with_router(
|
||||
batch,
|
||||
router=_AssertRouter(),
|
||||
tokenizer=_CharacterCounter(),
|
||||
)
|
||||
|
||||
assert len(results) == 2
|
||||
for _slot, res in results:
|
||||
assert not res.modified
|
||||
assert res.compressed == "x" * 150
|
||||
assert res.reason == expected_reason
|
||||
assert res.reason_category == expected_category
|
||||
|
||||
|
||||
def test_batch_allows_permitted_user_and_assistant_units():
|
||||
entries = []
|
||||
for idx, (role, meta_key) in enumerate(
|
||||
[("user", "compress_user"), ("assistant", "compress_assistant")]
|
||||
):
|
||||
unit = CompressionUnit(
|
||||
text="x" * 300,
|
||||
provider="openai",
|
||||
endpoint="responses",
|
||||
role=role,
|
||||
item_type="message",
|
||||
cache_zone="live",
|
||||
mutable=True,
|
||||
min_bytes=512,
|
||||
metadata={meta_key: "true"},
|
||||
)
|
||||
entries.append(
|
||||
CompressionBatchEntry(
|
||||
entry_id=f"u{idx}",
|
||||
routed=RoutedCompressionUnit(unit=unit, slot=(idx, ("output", None))),
|
||||
)
|
||||
)
|
||||
|
||||
# Test permitted user units
|
||||
user_entries = [entries[0], entries[0]]
|
||||
user_batches, user_skipped = build_compression_batches(user_entries, min_batch_bytes=512)
|
||||
assert len(user_batches) == 1
|
||||
assert user_skipped == []
|
||||
|
||||
router = _ShorteningRouter()
|
||||
user_results = compress_batch_with_router(
|
||||
user_batches[0],
|
||||
router=router,
|
||||
tokenizer=_CharacterCounter(),
|
||||
)
|
||||
assert router.calls == 1
|
||||
assert all(res.modified for _, res in user_results)
|
||||
|
||||
# Test permitted assistant units
|
||||
assistant_entries = [entries[1], entries[1]]
|
||||
asst_batches, asst_skipped = build_compression_batches(assistant_entries, min_batch_bytes=512)
|
||||
assert len(asst_batches) == 1
|
||||
assert asst_skipped == []
|
||||
|
||||
asst_results = compress_batch_with_router(
|
||||
asst_batches[0],
|
||||
router=router,
|
||||
tokenizer=_CharacterCounter(),
|
||||
)
|
||||
assert router.calls == 2
|
||||
assert all(res.modified for _, res in asst_results)
|
||||
|
||||
Reference in New Issue
Block a user