mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(transforms): judge the code_aware Kompress fallback in tokens (#3881)
## Description `ContentRouter._apply_strategy_to_content` compares two different units in the `code_aware` path: - `code_aware`'s result is stored as a **word** count: `compressed_tokens = len(output.content.split())` (introduced with the registry dispatch in #2400, after #1857 had moved every other count to `_estimate_tokens`). - The no-savings Kompress fallback is then kept only `if fallback_tokens < compressed_tokens:`, where `fallback_tokens` is a **token** count (cl100k payload tokens for a real Kompress result). Code runs about 2 tokens per whitespace word, so an unchanged code block looks like a ~55% compression and the fallback can essentially never win. The common case is a Claude Code `Read` of source: the `N\t` line-number column stops `code_aware` from parsing, it hands the text back unchanged with `compressed=True`, the fallback runs Kompress over the whole block (up to ~20 s of ONNX inference on a 1,800-line file, holding the Kompress slot), and the result is always discarded. The phantom ~0.45 ratio is then refused by the lossy-unrecoverable guard and logged as `lossy_unrecoverable_skipped`, 0 tokens saved. It is the same class as the CONFIG word-count bug fixed in #2759. Fix: 1. Measure `code_aware`'s output with `_estimate_tokens`, the unit of `original_tokens` and of every other branch. 2. Measure the Kompress fallback's result with the same estimator instead of the count Kompress returns (a Kompress passthrough reports **words**, which would otherwise let an unchanged block "win"). 3. The lossless-then-lossy inline Kompress attempt in the same branch is measured the same way and now recorded in `strategy_chain` even when it loses, so the no-savings fallback below does not run the identical inference a second time. **This is a compression behaviour change, stated plainly.** The fallback can now win, so a `code_aware` no-op block (typically a large line-numbered code `Read`) can now be Kompress-compressed with a CCR retrieval marker, where before it went out verbatim after the inference was thrown away. Savings go up; the Kompress inference that already ran now pays for itself. Real `code_aware` compressions are now scored in honest tokens rather than an optimistic word ratio, so a marginal one could in principle land above `min_ratio` where it passed before (none did on the corpus below). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/content_router.py`: `code_aware` result counted with `_estimate_tokens`; the no-savings Kompress fallback and the lossless-then-lossy inline attempt measured with `_estimate_tokens`; the inline attempt recorded in the chain before it runs. - `tests/test_transforms_content_router.py` (new, no tree-sitter needed): - `test_code_aware_noop_keeps_a_smaller_kompress_fallback`: a no-op `code_aware` on 120 line-numbered Rust lines, with a Kompress result smaller in tokens but larger than the word count, keeps the Kompress result. - `test_code_aware_noop_reports_tokens_when_kompress_passes_through`: when Kompress passes through, the block is reported at its token count, not its word count. - `test_code_aware_noop_runs_kompress_once_when_it_passes_through`: with lossless-then-lossy on, a losing inline attempt is not retried by the fallback. - `test_code_aware_noop_keeps_a_smaller_inline_kompress`: with lossless-then-lossy on, a winning inline attempt is kept, with one Kompress call. - `tests/test_code_aware_regressions.py`, `tests/test_router_registry_dispatch.py`: two existing assertions pinned the word-count metric (`len(shrunk.split())`); they now assert `_estimate_tokens`, the unit `original_tokens` uses. Their intent (the compressor's own estimate must not leak in, a real compression must not trigger a fallback) is unchanged. - `tests/test_router_registry_smartcrusher.py`, `tests/test_openai_responses_context_compaction.py`: two SMART_CRUSHER no-op tests go through the same shared fallback and asserted the count a mocked `_try_ml_compressor` reported (one mock "shrank" by returning longer text than it was given). The mocks now return shorter text and the assertions use `_estimate_tokens` of it. This also means a Kompress *passthrough* after a SMART_CRUSHER or CONFIG no-op, which Kompress reports in words, can no longer be adopted as a phantom saving. The first two new tests fail on `main`ffc6edb4(the Kompress result is discarded; the no-op is reported at 720 "tokens", its word count, against 2,136 estimator tokens). The third fails when the chain entry is removed (Kompress runs twice). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev --extra code pytest tests/test_transforms_content_router.py -q 72 passed, 2 skipped $ uv run --frozen --extra dev --extra code pytest tests/test_code_aware_regressions.py -q 5 passed $ uv run --frozen --extra dev --extra code pytest tests/test_router_registry_dispatch.py -q 11 passed $ uv run --frozen --extra dev --extra code pytest tests/test_transforms/test_content_router.py -q 63 passed $ uv run --frozen --extra dev --extra code pytest tests/test_lossless_mode.py -q 26 passed $ uv run --frozen --extra dev --extra code pytest tests/test_builtin_compressor_adapters.py -q 17 passed, 1 skipped $ uv run --frozen --extra dev --extra code pytest tests/test_content_router_tool_role_reversibility.py tests/test_agent_savings.py -q 10 passed / 39 passed $ uv run --frozen --extra dev --extra code pytest tests/test_router_registry_smartcrusher.py -q 7 passed $ uv run --frozen --extra dev --extra code pytest tests/test_openai_responses_context_compaction.py -q 10 passed # plus every other test file that mocks _try_ml_compressor (kompress_size_gate, lossless_then_lossy, ccr_tag_placeholder_regression, # relevance_split, kompress_request_nonblocking, bash_search_lossless_fold, text_crusher_routing, transforms_tabular): all pass # new tests with the headroom/ change stashed (mainffc6edb4): FAILED tests/test_transforms_content_router.py::test_code_aware_noop_keeps_a_smaller_kompress_fallback FAILED tests/test_transforms_content_router.py::test_code_aware_noop_reports_tokens_when_kompress_passes_through $ uvx ruff@0.16.8 check <6 touched files> All checks passed! $ uvx ruff@0.16.8 format --check <6 touched files> 6 files already formatted $ uv run --frozen --extra dev mypy headroom/transforms/content_router.py --ignore-missing-imports Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS 15 (16 GB, under load from a concurrent Rust release build), Python 3.11 worktree venv with `--extra dev --extra proxy --extra code` (tree-sitter + ONNX Kompress, `chopratejas/kompress-v2-base` from the local HF cache, warm), upstream mainffc6edb4with and without this commit. A real `HeadroomProxy` after `seed_proxy_env_defaults()`, with `savings_profile="coding"` and `code_aware_enabled=True` (what the `headroom proxy` CLI resolves by default), driven through `anthropic_pipeline.apply(...)` with `proxy_pipeline_kwargs(config)`. - Exact command / steps: 23 tool results built from a pristine `git archive ffc6edb4` of this repo, each sent as `[user, assistant tool_use, user tool_result]`: 10 Claude Code `Read`s in the `N\t` format (Rust, Python, TypeScript, Markdown, TOML, YAML), 10 `Bash` outputs (`cat -n` / `nl -ba` / `sed -n` / `cat` reads, `rg`, `ls -la`, JSON, `git log --stat`, `git diff`) and 3 raw-code results from a non-excluded MCP tool. Arms run fix, main, main, fix; `_try_ml_compressor` wrapped only to count calls and time them. - Observed result: exactly 4 of 23 items change, all large code `Read`s where `code_aware` was a no-op and Kompress had already run on main: | item | tokens | main saved | this PR saved | |---|---:|---:|---:| | `Read crates/headroom-proxy/src/proxy.rs` (1,709 lines) | 21,040 | 0 (`lossy_unrecoverable_skipped`, Kompress ~15-20 s discarded) | 4,847 (CCR marker) | | `Read crates/headroom-proxy/src/config.rs` | 10,623 | 0 (`lossy_unrecoverable_skipped`) | 2,072 | | `Read crates/headroom-proxy/src/responses_items.rs` | 6,614 | 0 (`lossy_unrecoverable_skipped`) | 1,422 | | `Read sdk/typescript/src/client.ts` | 5,876 | 0 (`lossy_unrecoverable_skipped`) | 799 | Corpus total 169,870 tokens: saved 13,674 on main vs 22,814 with this change (+9,140). Kompress calls are the same in both arms (12 per round; one main round made 13), so this does not reduce inference time; it keeps the result of inference main already paid for. The other 19 items are byte-identical between arms. A first full run of the same corpus was deterministic in both arms with the identical totals. In the second run the machine's concurrent build pushed a few Kompress calls past the 20 s request deadline (27.6 s on `proxy.rs`), so in one fix round `proxy.rs` and `config.rs` came back `ratio_too_high`, and in one main round `server.py` flipped; that is load, not the change. With `ProxyConfig` defaults (`code_aware` off, no savings profile) a 20-item run was byte-identical between arms. - Not tested: the model-facing effect of a Kompress-compressed code `Read` in a live Claude Code session (whether it is retrieved via CCR or re-read); Kompress on GPU backends; the latency effect of skipping a fallback that provably cannot win (not implemented, see Additional Notes). ### Update: measured on top of #3885 The four gains above were measured with each `Read` as the NEWEST message. Under `mode=token` with `savings_profile="coding"` (or `"general"`), a newest `Read` is only compressible because of a separate bug: the profile's `protect_recent=0` reaches the router as `read_protection_window=0` and replaces token mode's `max(4, 0.3*n)` window, so a `Read` has no protection at any position. #3885 fixes that. Without #3885, this PR turns the newest file read into a lossy one-line rewrite (`analyzer.py` 13,461 -> 11,139 tokens, newlines 1,082 -> 1); with it, that read stays byte-exact. Re-measured on the installed 0.39.0 wheel with the hunks exec'd in-process: a 23-item corpus of 147,527 tokens built from `git archive ffc6edb4` (10 `Read`s including the four above, 10 Bash outputs, 3 MCP code results), a fresh `HeadroomProxy` per arm, real ONNX Kompress with `HEADROOM_KOMPRESS_TIME_BUDGET_SECONDS=0` so load cannot flip a result. "aged" = the item followed by 40 messages. Both arms with this PR were run twice; 0 of 92 rows differed. | #3885 | this PR | newest: saved | aged: saved | |---|---|---:|---:| | no | no (main) | 19,972 (13.5%) | 19,972 (13.5%) | | no | yes | 38,240 (25.9%) | 38,240 (25.9%) | | yes | no | 9,696 (6.6%) | 19,972 (13.5%) | | yes | yes | 9,696 (6.6%) | 38,240 (25.9%) | With #3885 in place this PR adds nothing on a newest `Read` (routed `excluded_tool`) and +18,268 tokens once the same four code `Read`s age out of the window (proxy.rs 10,131, config.rs 4,487, responses_items.rs 2,145, client.ts 1,505; the other 19 items are identical). Per-item savings are about 2x the table above because this ran on the 0.39.0 wheel rather than main. Two limits: in live traffic an aged `Read` already inside the frozen prefix is never recompressed, so this is an upper bound; and under the default 20 s Kompress budget, proxy.rs (14-65 s of inference on a loaded 16 GB laptop) often comes back unchanged on slower machines. Recommend merging #3885 first or together. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Stable. - Stable/default behavior changed: Yes, when `code_aware` is enabled (the CLI default): a `code_aware` no-op block whose Kompress fallback is smaller in tokens is now Kompress-compressed with a CCR marker instead of forwarded verbatim; `code_aware` results are reported in estimator tokens instead of words. With `code_aware` disabled, nothing changes. - Kill switch / disable path: `--no-code-aware` / `HEADROOM_CODE_AWARE_ENABLED=0` (no-op blocks then take the plain Kompress route), or `HEADROOM_DISABLE_KOMPRESS=1` to turn Kompress off entirely. No dedicated switch for this fix. - Unsafe override required: None. - Qualification impact: Savings rise on large code reads (+9,140 tokens, +67% relative, on the corpus above). Those blocks now carry a lossy Kompress body plus a CCR retrieval marker, so it is worth watching retrieval and re-read rates on the release that carries it. - Rollback path: Revert this PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Documentation: N/A, internal accounting. - For reviewers: under the coding profile (`protect_recent=0`), a Claude Code `Read` result is already compressible on `main` (`codex.py` and `server.py` above are compressed by `code_aware` on both arms). If the intent is that `Read` results are never lossy-compressed, the word-count bug was masking a gap for no-op code reads, and that deserves an explicit protection rather than this accident; this PR does not change which blocks reach the router. #3885 is that protection for recent reads in token mode (see the update above). - Deliberately left out: skipping the Kompress call when it provably cannot win. In tokens, the only hard bound (the words Kompress always keeps, plus its marker) rarely reaches the block's token count, and on this corpus the fallback won 4 of the 5 times it ran. A heuristic skip is a latency optimisation with its own savings trade and belongs in its own PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
ffc6edb4ba
commit
6326965046
@@ -3704,7 +3704,11 @@ class ContentRouter(Transform):
|
||||
)
|
||||
if output is not None and output.compressed:
|
||||
compressed = output.content
|
||||
compressed_tokens = len(output.content.split())
|
||||
# Tokens, like ``original_tokens`` and every other
|
||||
# branch: a word count here (code runs ~2.2
|
||||
# tokens/word) made an unchanged block look like a
|
||||
# 55% compression and no fallback could beat it.
|
||||
compressed_tokens = _estimate_tokens(output.content)
|
||||
decision_reason = "code_aware"
|
||||
if compressed is None:
|
||||
# Fallback to Kompress
|
||||
@@ -3728,19 +3732,17 @@ class ContentRouter(Transform):
|
||||
# lossless has no savings. Reads are protected upstream, so
|
||||
# only NON-read code reaches here. Keep Kompress ONLY if it
|
||||
# actually shrinks (never inflate).
|
||||
_k, _kt = self._try_ml_compressor(content, context, question)
|
||||
if (
|
||||
_k is not None
|
||||
and _kt is not None
|
||||
and _kt < original_tokens
|
||||
and len(_k) < len(content)
|
||||
):
|
||||
# Recorded as tried even if it loses, so the no-savings
|
||||
# fallback below does not run the same inference again.
|
||||
strategy_chain.append(CompressionStrategy.KOMPRESS.value)
|
||||
_k, _ = self._try_ml_compressor(content, context, question)
|
||||
_kt = _estimate_tokens(_k)
|
||||
if _kt < original_tokens and len(_k) < len(content):
|
||||
compressed, compressed_tokens = _k, _kt
|
||||
strategy = CompressionStrategy.KOMPRESS
|
||||
actual_strategy = strategy
|
||||
compressor_name = "KompressCompressor"
|
||||
decision_reason = "code_aware_no_shrink_fallback_kompress"
|
||||
strategy_chain.append(CompressionStrategy.KOMPRESS.value)
|
||||
|
||||
elif strategy == CompressionStrategy.SMART_CRUSHER:
|
||||
# SmartCrusher handles its own TOIN recording
|
||||
@@ -3965,9 +3967,11 @@ class ContentRouter(Transform):
|
||||
already_tried_kompress = CompressionStrategy.KOMPRESS.value in strategy_chain
|
||||
if not already_tried_kompress:
|
||||
strategy_chain.append(CompressionStrategy.KOMPRESS.value)
|
||||
fallback_compressed, fallback_tokens = self._try_ml_compressor(
|
||||
content, context, question
|
||||
)
|
||||
fallback_compressed, _ = self._try_ml_compressor(content, context, question)
|
||||
# Measure with the router's estimator, the unit of
|
||||
# ``compressed_tokens``. Kompress reports a passthrough in
|
||||
# WORDS, which would let an unchanged block "win".
|
||||
fallback_tokens = _estimate_tokens(fallback_compressed)
|
||||
else:
|
||||
fallback_compressed = compressed
|
||||
fallback_tokens = compressed_tokens
|
||||
|
||||
@@ -7,8 +7,9 @@ default flip).
|
||||
2. Multi-line Go signature (`) error {`) — the brace-bearing signature line
|
||||
survives truncation instead of being silently dropped.
|
||||
3. ContentRouter CODE_AWARE token accounting — compressed_tokens must come
|
||||
from the same word-count metric as original_tokens, not the compressor's
|
||||
own (differently-scaled) estimator, or real savings get misread as none.
|
||||
from the same metric as original_tokens (the router's token estimator),
|
||||
not the compressor's own (differently-scaled) estimator, or real savings
|
||||
get misread as none.
|
||||
4. prefer_code_aware_for_code defaults to True, both on the dataclass and via
|
||||
the HEADROOM_PREFER_CODE_AWARE_FOR_CODE env var.
|
||||
"""
|
||||
@@ -27,6 +28,7 @@ from headroom.transforms.content_router import (
|
||||
CompressionStrategy,
|
||||
ContentRouter,
|
||||
ContentRouterConfig,
|
||||
_estimate_tokens,
|
||||
)
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
@@ -96,13 +98,13 @@ class _FakeCodeCompressor:
|
||||
r.compressed = self._compressed
|
||||
# Deliberately inflated/differently-scaled "own" token estimate —
|
||||
# simulates a compressor whose internal counter isn't comparable to
|
||||
# the router's len(text.split()) word count.
|
||||
# the router's own token estimate.
|
||||
r.compressed_tokens = 10_000_000
|
||||
return r
|
||||
|
||||
|
||||
def test_content_router_code_aware_token_accounting_matches_word_count() -> None:
|
||||
"""compressed_tokens must be derived from len(result.compressed.split()),
|
||||
def test_content_router_code_aware_token_accounting_matches_router_estimate() -> None:
|
||||
"""compressed_tokens must be derived from _estimate_tokens(result.compressed),
|
||||
the same metric as original_tokens, not the compressor's own estimator.
|
||||
Using the mismatched estimator made a real compression look like "no
|
||||
savings" and forced a needless fallback to Kompress."""
|
||||
@@ -117,7 +119,7 @@ def test_content_router_code_aware_token_accounting_matches_word_count() -> None
|
||||
)
|
||||
|
||||
assert compressed == compressed_text
|
||||
assert compressed_tokens == len(compressed_text.split())
|
||||
assert compressed_tokens == _estimate_tokens(compressed_text)
|
||||
assert strategy_chain == ["code_aware"], (
|
||||
f"real compression must not trigger a Kompress fallback, got {strategy_chain}"
|
||||
)
|
||||
|
||||
@@ -12,6 +12,7 @@ from headroom.transforms.content_router import (
|
||||
CompressionStrategy,
|
||||
ContentRouter,
|
||||
ContentRouterConfig,
|
||||
_estimate_tokens,
|
||||
)
|
||||
|
||||
|
||||
@@ -432,7 +433,8 @@ def test_content_router_retries_kompress_when_structured_strategy_noops(monkeypa
|
||||
)
|
||||
|
||||
assert compressed == "short summary"
|
||||
assert compressed_tokens == 2
|
||||
# Measured with the router's estimator, the unit of the strategy result.
|
||||
assert compressed_tokens == _estimate_tokens("short summary")
|
||||
# The fallback chain must record both strategies it tried.
|
||||
assert strategy_chain == ["smart_crusher", "kompress"]
|
||||
|
||||
|
||||
@@ -244,8 +244,8 @@ def test_code_aware_router_dispatch_matches_direct(monkeypatch: pytest.MonkeyPat
|
||||
content, CompressionStrategy.CODE_AWARE, "q", language="python", bias=1.0
|
||||
)
|
||||
assert out == shrunk
|
||||
# CODE_AWARE's historical token metric is len(compressed.split()), NOT _estimate_tokens.
|
||||
assert tokens == len(shrunk.split())
|
||||
# Measured in the router's token estimate, the unit of original_tokens.
|
||||
assert tokens == _estimate_tokens(shrunk)
|
||||
assert chain == [CompressionStrategy.CODE_AWARE.value]
|
||||
# The flip forwarded content, language, and context through the registry adapter.
|
||||
assert seen == {"content": content, "language": "python", "context": "q"}
|
||||
|
||||
@@ -144,12 +144,15 @@ def test_smart_crusher_kompress_fallback_matches_direct(monkeypatch: pytest.Monk
|
||||
lambda: SimpleNamespace(crush=lambda c, query="", bias=1.0: SimpleNamespace(compressed=c)),
|
||||
)
|
||||
# Kompress shrinks (fallback_tokens < compressed_tokens) so it is adopted.
|
||||
monkeypatch.setattr(router, "_try_ml_compressor", lambda c, ctx, q: ("KOMPRESSED::" + c, 3))
|
||||
# The fallback is measured with the router's estimator, not the count the
|
||||
# ML compressor reports, so the mock must actually return less text.
|
||||
kompressed = "KOMPRESSED::" + _JSON[:40]
|
||||
monkeypatch.setattr(router, "_try_ml_compressor", lambda c, ctx, q: (kompressed, 3))
|
||||
out, tokens, chain = router._apply_strategy_to_content(
|
||||
_JSON, CompressionStrategy.SMART_CRUSHER, "ctx", bias=1.0
|
||||
)
|
||||
assert out == "KOMPRESSED::" + _JSON
|
||||
assert tokens == 3
|
||||
assert out == kompressed
|
||||
assert tokens == _estimate_tokens(kompressed)
|
||||
assert chain == [
|
||||
CompressionStrategy.SMART_CRUSHER.value,
|
||||
CompressionStrategy.KOMPRESS.value,
|
||||
|
||||
@@ -2251,3 +2251,123 @@ def test_html_extraction_miss_still_gets_deferred_json_splice() -> None:
|
||||
assert len(text) < len(shell)
|
||||
json_marker = '"id":'
|
||||
assert json_marker not in text # the array was compressed away
|
||||
|
||||
|
||||
# A Claude Code Read of Rust: line-numbered code that code_aware hands back
|
||||
# unchanged (it cannot parse the number column) while reporting compressed=True.
|
||||
# Code runs ~2 tokens per whitespace word, so a word count badly undercounts it.
|
||||
_NUMBERED_RUST = "\n".join(
|
||||
f"{i}\tlet value_{i} = compute(&items[{i}..], Some(opts.clone()))?;" for i in range(1, 121)
|
||||
)
|
||||
|
||||
|
||||
def _noop_code_aware_router(monkeypatch: pytest.MonkeyPatch, kompress_out: str) -> ContentRouter:
|
||||
monkeypatch.delenv("HEADROOM_LOSSLESS_THEN_LOSSY", raising=False)
|
||||
router = ContentRouter(ContentRouterConfig(enable_code_aware=True))
|
||||
|
||||
class NoopCodeCompressor:
|
||||
def compress(self, content: str, language=None, context: str = "") -> SimpleNamespace:
|
||||
return SimpleNamespace(compressed=content, compressed_tokens=0)
|
||||
|
||||
monkeypatch.setattr(router, "_get_code_compressor", lambda: NoopCodeCompressor())
|
||||
|
||||
def kompress(content: str, *_args: object, **_kwargs: object) -> tuple[str, int]:
|
||||
# What Kompress reports: payload tokens for a real compression, the
|
||||
# WORD count for a passthrough.
|
||||
if kompress_out == content:
|
||||
return content, len(content.split())
|
||||
return kompress_out, _estimate_tokens(kompress_out)
|
||||
|
||||
monkeypatch.setattr(router, "_try_ml_compressor", kompress)
|
||||
return router
|
||||
|
||||
|
||||
def test_code_aware_noop_keeps_a_smaller_kompress_fallback(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The fallback is judged in tokens, the unit of the code_aware result.
|
||||
|
||||
With code_aware's result counted in words, a Kompress result at ~60% of the
|
||||
tokens still "lost" to the unchanged block, so the inference always ran and
|
||||
was always thrown away.
|
||||
"""
|
||||
kompressed = "\n".join(f"{i}\tvalue_{i} compute(&items[{i}..])" for i in range(1, 121))
|
||||
assert _estimate_tokens(kompressed) < _estimate_tokens(_NUMBERED_RUST)
|
||||
assert _estimate_tokens(kompressed) > len(_NUMBERED_RUST.split())
|
||||
router = _noop_code_aware_router(monkeypatch, kompressed)
|
||||
|
||||
compressed, compressed_tokens, strategy_chain = router._apply_strategy_to_content(
|
||||
_NUMBERED_RUST, CompressionStrategy.CODE_AWARE, context=""
|
||||
)
|
||||
|
||||
assert compressed == kompressed
|
||||
assert compressed_tokens == _estimate_tokens(kompressed)
|
||||
assert strategy_chain == ["code_aware", "kompress"]
|
||||
|
||||
|
||||
def test_code_aware_noop_reports_tokens_when_kompress_passes_through(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""An unchanged block is reported at its own token count, never its word count."""
|
||||
router = _noop_code_aware_router(monkeypatch, _NUMBERED_RUST)
|
||||
|
||||
compressed, compressed_tokens, _chain = router._apply_strategy_to_content(
|
||||
_NUMBERED_RUST, CompressionStrategy.CODE_AWARE, context=""
|
||||
)
|
||||
|
||||
assert compressed == _NUMBERED_RUST
|
||||
assert compressed_tokens == _estimate_tokens(_NUMBERED_RUST)
|
||||
|
||||
|
||||
def test_code_aware_noop_runs_kompress_once_when_it_passes_through(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Lossless-then-lossy tries Kompress inline; a passthrough must not be retried."""
|
||||
router = _noop_code_aware_router(monkeypatch, _NUMBERED_RUST)
|
||||
router._lossless_then_lossy = True
|
||||
calls: list[str] = []
|
||||
kompress = router._try_ml_compressor
|
||||
|
||||
def counting(content: str, *args: object, **kwargs: object) -> tuple[str, int]:
|
||||
calls.append(content)
|
||||
return kompress(content, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(router, "_try_ml_compressor", counting)
|
||||
|
||||
compressed, compressed_tokens, _chain = router._apply_strategy_to_content(
|
||||
_NUMBERED_RUST, CompressionStrategy.CODE_AWARE, context=""
|
||||
)
|
||||
|
||||
assert compressed == _NUMBERED_RUST
|
||||
assert compressed_tokens == _estimate_tokens(_NUMBERED_RUST)
|
||||
assert len(calls) == 1
|
||||
|
||||
|
||||
def test_code_aware_noop_keeps_a_smaller_inline_kompress(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Lossless-then-lossy's inline attempt keeps a Kompress result that wins.
|
||||
|
||||
Measured in tokens, a code_aware no-op no longer looks like a shrink, so the
|
||||
inline attempt fires and its smaller result is kept without a second call.
|
||||
"""
|
||||
kompressed = "\n".join(f"{i}\tvalue_{i} compute(&items[{i}..])" for i in range(1, 121))
|
||||
router = _noop_code_aware_router(monkeypatch, kompressed)
|
||||
router._lossless_then_lossy = True
|
||||
calls: list[str] = []
|
||||
kompress = router._try_ml_compressor
|
||||
|
||||
def counting(content: str, *args: object, **kwargs: object) -> tuple[str, int]:
|
||||
calls.append(content)
|
||||
return kompress(content, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(router, "_try_ml_compressor", counting)
|
||||
|
||||
compressed, compressed_tokens, strategy_chain = router._apply_strategy_to_content(
|
||||
_NUMBERED_RUST, CompressionStrategy.CODE_AWARE, context=""
|
||||
)
|
||||
|
||||
assert compressed == kompressed
|
||||
assert compressed_tokens == _estimate_tokens(kompressed)
|
||||
assert strategy_chain == ["code_aware", "kompress"]
|
||||
assert len(calls) == 1
|
||||
|
||||
Reference in New Issue
Block a user