Files
headroom/tests/test_cache_aligner_detector_only.py
Tejas ChopraandClaude Opus 5.5 e4a1ec19be refactor(transforms): deprecate unused compress_with_stats, extract_batch and align_for_cache (#4098)
## Description

`DiffCompressor.compress_with_stats`, `HTMLExtractor.extract_batch` and
`cache_aligner.align_for_cache` are public. The first two are methods on
classes exported from `headroom.transforms`, and the third's docstring
called it a stable public API. None is called anywhere in Headroom, the
plugins or the consumer repos; only tests call them. Each now emits a
`DeprecationWarning` that names the replacement.

Refs #4136

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `DiffCompressor.compress_with_stats` warns and points to
`DiffCompressor.compress`. Unlike `compress()`, it never wrote its CCR
entry to the compression store, so a retrieval marker in its output
could dangle. The docstring now says so.
- `HTMLExtractor.extract_batch` warns and points to calling
`HTMLExtractor.extract` per page.
- `align_for_cache` warns and points to
`CacheAligner(config).apply(messages, tokenizer)` and the
`stable_prefix_hash:` marker.
- The existing tests now assert the warnings. A new test checks that
`align_for_cache` warns at the caller and still returns the messages and
the hash.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ pytest tests/test_cache_aligner_detector_only.py tests/test_cache_aligner_prefix_stability.py tests/test_transforms/test_html_extractor.py tests/test_transforms/test_diff_compressor_rust_parity.py tests/test_transforms/test_diff_compressor.py
118 passed in 3.87s
$ ruff check / format: All checks passed;  mypy: Success
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, repo `.venv`, this branch's worktree
with the prebuilt `headroom._core`. Temp `HOME`,
`HEADROOM_WORKSPACE_DIR` and `CODEX_HOME`; nothing touched the real
home.
- Exact command / steps: (1) shared smoke harness: a real proxy from
this checkout against a local fake upstream, with one Anthropic and one
OpenAI request carrying a compressible tool result (`smoke.sh
<worktree>`); (2) `python -W always -c 'align_for_cache(...);
DiffCompressor().compress_with_stats(...)'`.
- Observed result: The proxy serves both requests (200) with the same
savings as main. Each deprecated call emits one DeprecationWarning
attributed to the caller and still returns its result. Output below.
- Not tested: Third-party callers; none is known.

```text
branch: cleanup/transforms-deprecate-unused-methods @ a3e57303
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines ($SMOKE/ws/logs/proxy-51205.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: $SMOKE)
```

```text
$ python -W always -c "align_for_cache([...]); DiffCompressor().compress_with_stats(...)"
<string>:4: DeprecationWarning: align_for_cache is deprecated and will be removed in a future release; use CacheAligner(config).apply(messages, tokenizer) and read the stable_prefix_hash: marker from markers_inserted.
<string>:5: DeprecationWarning: DiffCompressor.compress_with_stats is deprecated and will be removed in a future release; use DiffCompressor.compress instead.
ok
```

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Not applicable; no rollout-managed behaviour
is touched.
- Stable/default behavior changed: Only a DeprecationWarning when the
deprecated methods are called.
- Kill switch / disable path: Not applicable; the warning is a standard
DeprecationWarning and can be filtered with `warnings`.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this single commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.

Documentation changes are not in this PR. They are recorded in the
cleanup docs ledger for the single follow-up docs PR.

Deprecated (not removed): `DiffCompressor.compress_with_stats`,
`HTMLExtractor.extract_batch`, `cache_aligner.align_for_cache`. The
warnings fire on every call, which is standard. Python's default warning
filters show each one once per call site.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:03:03 -07:00

352 lines
13 KiB
Python

"""Detector-only tests for ``CacheAligner`` (PR-A2 / P2-23).
After PR-A2, ``CacheAligner`` is a detector-only transform: it never
mutates messages, only emits warnings. These tests pin that contract.
"""
from __future__ import annotations
import logging
from collections.abc import Iterator
from contextlib import contextmanager
import pytest
from headroom import OpenAIProvider, Tokenizer
from headroom.config import CacheAlignerConfig
from headroom.transforms.cache_aligner import (
CacheAligner,
detect_volatile_content,
)
_provider = OpenAIProvider()
def _tokenizer() -> Tokenizer:
counter = _provider.get_token_counter("gpt-4o")
return Tokenizer(counter, "gpt-4o")
@pytest.fixture
def tokenizer() -> Tokenizer:
return _tokenizer()
def _system_user_messages(system_text: str) -> list[dict[str, object]]:
return [
{"role": "system", "content": system_text},
{"role": "user", "content": "hello"},
]
@contextmanager
def _capture_warnings() -> Iterator[list[str]]:
"""Capture warning messages from the cache_aligner logger.
Bypasses pytest's ``caplog`` so logger-propagation tweaks elsewhere in
the test suite don't break detection assertions.
"""
captured: list[str] = []
class _ListHandler(logging.Handler):
def emit(self, record: logging.LogRecord) -> None:
captured.append(record.getMessage())
target_logger = logging.getLogger("headroom.transforms.cache_aligner")
prev_level = target_logger.level
target_logger.setLevel(logging.WARNING)
handler = _ListHandler(level=logging.WARNING)
target_logger.addHandler(handler)
try:
yield captured
finally:
target_logger.removeHandler(handler)
target_logger.setLevel(prev_level)
# ---------------------------------------------------------------------------
# Detector tests on the ``apply`` surface
# ---------------------------------------------------------------------------
def test_volatile_uuid_detected_warned_not_rewritten(tokenizer: Tokenizer) -> None:
system_text = (
"You are a helpful assistant.\n"
"Session ID: 550e8400-e29b-41d4-a716-446655440000\n"
"Help the user."
)
messages = _system_user_messages(system_text)
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
with _capture_warnings() as records:
result = aligner.apply(messages, tokenizer)
# Bytes preserved — never rewritten.
assert result.messages[0]["content"] == system_text
assert result.messages[1]["content"] == "hello"
# Warning surfaced (in result + via the logger).
assert result.warnings, "expected a warning for detected UUID"
assert any("uuid" in w.lower() for w in result.warnings)
assert any("uuid" in rec.lower() for rec in records)
def test_volatile_iso8601_detected_warned(tokenizer: Tokenizer) -> None:
system_text = "You are a logging assistant.\nCurrent time: 2024-01-15T10:30:00\nHelp the user."
messages = _system_user_messages(system_text)
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
with _capture_warnings() as records:
result = aligner.apply(messages, tokenizer)
assert result.messages[0]["content"] == system_text
assert any("iso8601" in w.lower() for w in result.warnings)
assert any("iso8601" in rec.lower() for rec in records)
def test_volatile_jwt_detected_warned(tokenizer: Tokenizer) -> None:
jwt = (
"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9."
"eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ."
"SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c"
)
system_text = f"You are an assistant.\nAuth Token: {jwt}\nHelp the user."
messages = _system_user_messages(system_text)
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
with _capture_warnings() as records:
result = aligner.apply(messages, tokenizer)
assert result.messages[0]["content"] == system_text
assert any("jwt" in w.lower() for w in result.warnings)
assert any("jwt" in rec.lower() for rec in records)
def test_volatile_hex_hash_detected_warned(tokenizer: Tokenizer) -> None:
sha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
system_text = f"You are a security assistant.\nHash: {sha256}\nHelp."
messages = _system_user_messages(system_text)
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
with _capture_warnings() as records:
result = aligner.apply(messages, tokenizer)
assert result.messages[0]["content"] == system_text
assert any("hex_hash" in w.lower() for w in result.warnings)
assert any("hex_hash" in rec.lower() for rec in records)
def test_no_false_positives_on_normal_prose(tokenizer: Tokenizer) -> None:
system_text = (
"You are a helpful assistant. Answer the user's questions clearly. "
"Be polite, concise, and accurate. If you do not know the answer, "
"say so plainly."
)
messages = _system_user_messages(system_text)
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
with _capture_warnings() as records:
result = aligner.apply(messages, tokenizer)
assert result.messages[0]["content"] == system_text
assert result.warnings == [], f"unexpected warnings: {result.warnings}"
assert records == [], f"unexpected log records: {records}"
def test_apply_never_mutates_input(tokenizer: Tokenizer) -> None:
"""The detector never mutates the caller's input list."""
system_text = "Session: 550e8400-e29b-41d4-a716-446655440000"
messages = _system_user_messages(system_text)
snapshot = [dict(m) for m in messages]
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
aligner.apply(messages, tokenizer)
for original, current in zip(snapshot, messages, strict=True):
assert original == current
def test_apply_does_not_attempt_rewrite_on_dynamic_content(
tokenizer: Tokenizer,
) -> None:
"""``transforms_applied`` must be empty even when warnings fire."""
system_text = (
"Session: 550e8400-e29b-41d4-a716-446655440000\n"
"Hash: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
)
messages = _system_user_messages(system_text)
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
result = aligner.apply(messages, tokenizer)
assert result.transforms_applied == []
# System bytes must equal input bytes (no rewrite).
assert result.messages[0]["content"] == system_text
def test_should_apply_false_when_disabled(tokenizer: Tokenizer) -> None:
messages = _system_user_messages("Session: 550e8400-e29b-41d4-a716-446655440000")
aligner = CacheAligner(CacheAlignerConfig(enabled=False))
assert not aligner.should_apply(messages, tokenizer)
def test_should_apply_false_without_system_message(tokenizer: Tokenizer) -> None:
messages = [{"role": "user", "content": "hello"}]
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
assert not aligner.should_apply(messages, tokenizer)
def test_should_apply_false_when_policy_disables_aligner(tokenizer: Tokenizer) -> None:
"""F2.1 c4/5: ``compression_policy.cache_aligner_enabled=False``
must disable the detector even when content + config would
otherwise opt in.
This is the canary that the per-auth-mode plumbing actually
reaches CacheAligner. A regression here means subscription
requests would silently keep updating
``self._previous_prefix_hash`` and emitting volatility warnings,
which are the exact log lines #327/#388 reporters complained
about.
F2.2 added three per-mode tuning fields to CompressionPolicy
(``volatile_token_threshold``, ``max_lossy_ratio``,
``toin_read_only``); the policy here uses the Subscription
defaults from ``policy_for_mode(AuthMode.SUBSCRIPTION)`` so the
fixture mirrors a real subscription request.
"""
from headroom.transforms.compression_policy import CompressionPolicy
messages = _system_user_messages("Session: 550e8400-e29b-41d4-a716-446655440000")
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
# Sanity: without a policy, the detector opts in.
assert aligner.should_apply(messages, tokenizer)
# F2.1 gate: with the subscription policy, the detector opts out.
sub_policy = CompressionPolicy(
live_zone_only=True,
cache_aligner_enabled=False,
volatile_token_threshold=32,
max_lossy_ratio=0.25,
toin_read_only=True,
)
assert not aligner.should_apply(messages, tokenizer, compression_policy=sub_policy)
def test_should_apply_true_when_policy_enables_aligner(tokenizer: Tokenizer) -> None:
"""F2.1 c4/5: ``compression_policy.cache_aligner_enabled=True``
must NOT short-circuit. PAYG/OAuth keep current behaviour.
F2.2 added three per-mode tuning fields; the policy here uses the
PAYG defaults from ``policy_for_mode(AuthMode.PAYG)`` so the
fixture mirrors a real PAYG request.
"""
from headroom.transforms.compression_policy import CompressionPolicy
messages = _system_user_messages("Session: 550e8400-e29b-41d4-a716-446655440000")
aligner = CacheAligner(CacheAlignerConfig(enabled=True))
payg_policy = CompressionPolicy(
live_zone_only=False,
cache_aligner_enabled=True,
volatile_token_threshold=128,
max_lossy_ratio=0.45,
toin_read_only=False,
)
assert aligner.should_apply(messages, tokenizer, compression_policy=payg_policy)
# ---------------------------------------------------------------------------
# Pure-function detection tests
# ---------------------------------------------------------------------------
def test_detect_volatile_content_uuid() -> None:
findings = detect_volatile_content("Session: 550e8400-e29b-41d4-a716-446655440000")
labels = [f.label for f in findings]
assert "uuid" in labels
def test_detect_volatile_content_iso_date() -> None:
findings = detect_volatile_content("Now: 2024-01-15T10:30:00")
labels = [f.label for f in findings]
assert "iso8601" in labels
def test_detect_volatile_content_jwt() -> None:
# ggignore: canonical fake JWT, payload {"sub":"1"}, no privilege.
jwt = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c" # noqa: S105
findings = detect_volatile_content(f"Token: {jwt}")
labels = [f.label for f in findings]
assert "jwt" in labels
def test_detect_volatile_content_md5() -> None:
findings = detect_volatile_content("Hash: d41d8cd98f00b204e9800998ecf8427e")
labels = [f.label for f in findings]
assert "hex_hash" in labels
def test_detect_volatile_content_sha1() -> None:
findings = detect_volatile_content("Hash: da39a3ee5e6b4b0d3255bfef95601890afd80709")
labels = [f.label for f in findings]
assert "hex_hash" in labels
def test_detect_volatile_content_returns_truncated_sample() -> None:
long_token = "0123456789abcdef0123456789abcdef" # 32-char hex (MD5-shaped)
findings = detect_volatile_content(f"hash: {long_token}")
assert findings
# Sample is truncated for any token longer than 16 chars.
assert "..." in findings[0].sample
def test_detect_volatile_content_empty_string_no_findings() -> None:
assert detect_volatile_content("") == []
def test_detect_volatile_content_normal_text_no_findings() -> None:
findings = detect_volatile_content("You are a helpful assistant. Be polite.")
assert findings == []
def test_alignment_score_perfect_when_no_dynamic_content() -> None:
aligner = CacheAligner()
score = aligner.get_alignment_score([{"role": "system", "content": "You are helpful."}])
assert score == 100.0
def test_alignment_score_decreases_with_dynamic_content() -> None:
aligner = CacheAligner()
score = aligner.get_alignment_score(
[
{
"role": "system",
"content": (
"Session: 550e8400-e29b-41d4-a716-446655440000\n"
"Hash: d41d8cd98f00b204e9800998ecf8427e"
),
}
]
)
assert score < 100.0
def test_no_regex_imported() -> None:
"""The cache_aligner module must not import ``re`` (build constraint)."""
import headroom.transforms.cache_aligner as mod
# ``re`` should not be in the module's globals.
assert "re" not in mod.__dict__, (
"headroom.transforms.cache_aligner must not depend on the regex module"
)
def test_align_for_cache_is_deprecated_and_still_works() -> None:
from headroom.transforms.cache_aligner import align_for_cache
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "hi"},
]
with pytest.warns(DeprecationWarning, match="align_for_cache is deprecated") as record:
out, stable_hash = align_for_cache(messages)
assert record[0].filename == __file__
assert out == messages
assert isinstance(stable_hash, str)