mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description
The process-wide reasoning compaction memo is currently keyed solely by
the text hash. If two distinct compressor instances receive identical
reasoning, the second call reuses the cached output from the first
compressor rather than invoking the newly selected compressor. Both
Anthropic thinking-to-text conversion and OpenAI plain-text reasoning
rely on this memoization layer.
This PR scopes memo entries to the compressor instance alongside the
text hash. Each entry maintains a weak reference to verify identity in
case Python recycles an object ID, preventing unused compressor/model
instances from being kept alive in memory. Compressors that do not
support weak references continue to function cleanly without
memoization. The existing shared global LRU capacity limit remains
intact.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
## Changes Made
- Include compressor identity in reasoning memo lookups.
- Validate weak references before accepting a cached result to guard
against recycled object IDs.
- Preserve the existing global entry limit and avoid leaking
compressor/model instances.
- Add test coverage for both message formats, repeated calls, object
lifecycle, slotted compressors, and cross-instance LRU eviction.
## Testing
- [x] Related suites (compactor, version-gate, cold-prefix, and
thinking-fidelity): **48 passed**.
- [x] Added regressions: **3 failed, 3 passed** on unchanged main; all
six pass cleanly with this patch.
- [x] Code quality: Ruff checks, formatting, and repository pre-commit /
commit-message hooks pass.
- [x] Pre-push verification (`make ci-precheck`): passes cleanly
(including native extension rebuild/import, Rust tests, Python tests,
and commitlint).
```text
Before: 3 failed, 3 passed
After: 48 passed
```
## Real Behavior Proof
- Environment: Windows x86-64, Python 3.13.9, `main` at
`a2d4fe72381e186651466ee94e92ce9d8d1396cc`; two
`RemoteKompressCompressor` instances targeting distinct paths on a local
HTTP server.
- Exact command / steps: Run the script below on `main` versus this
branch. Both clients submit the same reasoning text; the mock server
returns a distinct summary for each path (`min_input_words=10` ensures
the 60-word input is processed).
- Observed result: On `main`, both outputs resolve to `first backend
summary`, and only `/first/compress` receives a request. With this
patch, outputs match their respective endpoints, and each path receives
one request.
- Not tested: Trained Kompress model inference, paid provider requests,
model task quality, or mutations to a compressor's configuration after
construction.
```python
from collections import Counter
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import threading
from headroom.transforms.kompress_compressor import KompressConfig
from headroom.transforms.kompress_remote import RemoteKompressCompressor
from headroom.transforms.thinking_compactor import compact_reasoning_openai_chat
calls = Counter()
class Endpoint(BaseHTTPRequestHandler):
def log_message(self, *args):
pass
def do_POST(self):
self.rfile.read(int(self.headers["Content-Length"]))
name = self.path.split("/")[1]
calls[name] += 1
data = json.dumps({"compressed": f"{name} backend summary"}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
server = ThreadingHTTPServer(("127.0.0.1", 0), Endpoint)
threading.Thread(target=server.serve_forever, daemon=True).start()
try:
engines = [RemoteKompressCompressor(
endpoint=f"http://127.0.0.1:{server.server_port}/{name}",
config=KompressConfig(enable_ccr=False, min_input_words=10),
) for name in ("first", "second")]
messages = [{"role": "assistant", "content": "answer",
"reasoning_content": "reasoning " * 60}]
for engine in engines:
result, _ = compact_reasoning_openai_chat(
messages, kompress=engine, keep_last_turns=0,
)
print(result[0]["reasoning_content"])
print(dict(calls))
finally:
server.shutdown()
server.server_close()
```
```text
Before:
first backend summary
first backend summary
{'first': 1}
After:
first backend summary
second backend summary
{'first': 1, 'second': 1}
```
## Runtime Rollout Safety
- Rollout-managed feature(s): none.
- Minimum rollout channel: stable.
- Stable/default behavior changed: reasoning compaction returns results
from the selected compressor instance rather than reusing an entry from
another instance. The transform remains opt-in via
`HEADROOM_THINKING_COMPACT` at proxy call sites.
- Kill switch / disable path: unset `HEADROOM_THINKING_COMPACT` to
disable the reasoning compaction pass.
- Unsafe override required: no.
- Qualification impact: configuring a new compressor instance incurs its
initial compression call; repeated calls to the same instance remain
memoized. Total cache entries remain bounded.
- Rollback path: revert this patch; memo resides entirely in memory and
requires no migration.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Additional Notes
I implemented and verified this with Codex assistance for drafting and
local tests.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4569"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
89 lines
2.7 KiB
Python
89 lines
2.7 KiB
Python
"""Compaction memoization must respect the compressor selected by the caller."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import gc
|
|
import weakref
|
|
from types import SimpleNamespace
|
|
|
|
import pytest
|
|
|
|
from headroom.transforms import thinking_compactor as tc
|
|
|
|
|
|
class Compressor:
|
|
def __init__(self, output):
|
|
self.output = output
|
|
self.calls = 0
|
|
|
|
def compress(self, text, **kwargs):
|
|
self.calls += 1
|
|
return SimpleNamespace(compressed=self.output)
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def clear_cache():
|
|
tc._COMPACT_CACHE.clear()
|
|
yield
|
|
tc._COMPACT_CACHE.clear()
|
|
|
|
|
|
def compact(compressor, shape="field"):
|
|
reasoning = "reasoning " * 60
|
|
if shape == "field":
|
|
messages = [{"role": "assistant", "content": "answer", "reasoning_content": reasoning}]
|
|
result, _ = tc.compact_reasoning_openai_chat(
|
|
messages, kompress=compressor, keep_last_turns=0
|
|
)
|
|
return result[0]["reasoning_content"]
|
|
messages = [{"role": "assistant", "content": [{"type": "thinking", "thinking": reasoning}]}]
|
|
result, _ = tc.compact_thinking_to_text(messages, kompress=compressor, keep_last_turns=0)
|
|
return result[0]["content"][0]["text"]
|
|
|
|
|
|
@pytest.mark.parametrize("shape", ["field", "thinking"])
|
|
def test_different_compressors_do_not_share_compacted_output(shape):
|
|
first = Compressor("first backend summary")
|
|
second = Compressor("second backend summary")
|
|
|
|
assert "first backend summary" in compact(first, shape)
|
|
assert "second backend summary" in compact(second, shape)
|
|
assert first.calls == second.calls == 1
|
|
|
|
|
|
def test_repeated_calls_keep_the_selected_compressors_memo():
|
|
compressor = Compressor("stable summary")
|
|
assert compact(compressor) == compact(compressor)
|
|
assert compressor.calls == 1
|
|
|
|
|
|
def test_cache_does_not_keep_unused_compressors_alive():
|
|
compressor = Compressor("summary")
|
|
compact(compressor)
|
|
reference = weakref.ref(compressor)
|
|
del compressor
|
|
gc.collect()
|
|
assert reference() is None
|
|
|
|
|
|
def test_compressor_without_weakref_support_still_works():
|
|
class SlottedCompressor:
|
|
__slots__ = ()
|
|
|
|
def compress(self, text, **kwargs):
|
|
return SimpleNamespace(compressed="slotted summary")
|
|
|
|
compressor = SlottedCompressor()
|
|
assert compact(compressor) == "slotted summary"
|
|
assert compact(compressor) == "slotted summary"
|
|
|
|
|
|
def test_cache_limit_is_shared_across_compressors(monkeypatch):
|
|
monkeypatch.setattr(tc, "_CACHE_CAP", 1)
|
|
first, second = Compressor("first summary"), Compressor("second summary")
|
|
compact(first)
|
|
compact(second)
|
|
assert len(tc._COMPACT_CACHE) == 1
|
|
assert compact(first) == "first summary"
|
|
assert first.calls == 2
|