mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-23 02:49:17 +08:00
feat/https-compose-caddy
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c8168a0aa0 |
fix(cache): invalidate cached CCR markers after store eviction (#4872)
## Description When `ContentRouter` caches a compressed output that carries a CCR retrieval marker (`Retrieve more: hash=...`, `Retrieve original: hash=...`, or `<<ccr:...>>`), a subsequent `compress()` / `ContentRouter.apply()` call could hit Tier 2 `CompressionCache` and return that marker even after the underlying `CompressionStore` entry was evicted by `max_entries`, cleared, or expired. Any follow-up retrieval for that marker then returned `None`. This change: 1. Adds `CompressionCache.invalidate(key)` to evict a stale cached entry without marking it skipped, and tracks verified CCR marker hashes per cached entry (`_marker_hashes`) cleaned up alongside `_results`. 2. Records only marker hashes that actually existed in `CompressionStore` at `put()` time via `ContentRouter._collect_verified_marker_hashes`, preserving compatibility with synthetic/mocked router tests that do not write to `CompressionStore`. 3. Validates tracked marker hashes via `ContentRouter._cached_markers_retrievable` on both Tier 2 cache hit paths (`ContentRouter.apply` and `ContentRouter._compress_block_content`), invalidating stale entries and falling through to fresh compression so the original content is re-stored and retrievable. Closes #4796 *Disclosure: This fix and PR description were written by an AI agent (沈寻 / Shen Xun, `@xunWhale`) on behalf of `@e0_7`, and verified locally with pre-fix failure / post-fix pass pytest outputs, repo-wide ruff/mypy, and the issue reproduction.* ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/content_router.py`: added `_CCR_MARKER_HASH_RE`, `CompressionCache.invalidate()`, `CompressionCache.get_marker_hashes()`, `_marker_hashes` tracking inside `CompressionCache`, and `ContentRouter._collect_verified_marker_hashes()` / `_cached_markers_retrievable()` checks on both string and content-block Tier 2 cache hit paths. - `tests/test_router_result_cache.py`: added 4 regression tests covering `CompressionCache.invalidate()`, capacity eviction under `compress()` for both `role="tool"` string messages and Anthropic `tool_result` content blocks, and custom `CompressionStore.clear()` invalidation. ## Testing - [x] Unit tests pass (`pytest` on router/cache/compression suites; see scope note below) - [x] Linting passes (`ruff check .` & `ruff format --check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output Pre-fix baseline (`0368a87`, without `content_router.py` fix): ```text $ PYTHONPATH=. pytest tests/test_router_result_cache.py -k "capacity_eviction or store_cleared or invalidate_drops" -v FAILED tests/test_router_result_cache.py::test_invalidate_drops_cached_result_without_marking_skipped - AttributeError: 'CompressionCache' object has no attribute 'invalidate' FAILED tests/test_router_result_cache.py::test_compress_repopulates_ccr_store_after_capacity_eviction - assert None is not None FAILED tests/test_router_result_cache.py::test_compress_block_content_repopulates_ccr_store_after_capacity_eviction - assert None is not None FAILED tests/test_router_result_cache.py::test_router_invalidates_cached_marker_when_store_cleared - TypeError: CompressionCache.put() got an unexpected keyword argument 'marker_hashes' ======================= 4 failed, 28 deselected in 0.39s ======================= ``` Post-fix checks: ```text $ ruff check . All checks passed! $ ruff format --check . 1940 files already formatted $ mypy headroom Success: no issues found in 583 source files $ PYTHONPATH=. pytest tests/test_router_result_cache.py -v ... tests/test_router_result_cache.py::test_invalidate_drops_cached_result_without_marking_skipped PASSED [ 90%] tests/test_router_result_cache.py::test_compress_repopulates_ccr_store_after_capacity_eviction PASSED [ 93%] tests/test_router_result_cache.py::test_compress_block_content_repopulates_ccr_store_after_capacity_eviction PASSED [ 96%] tests/test_router_result_cache.py::test_router_invalidates_cached_marker_when_store_cleared PASSED [100%] ============================== 32 passed in 1.93s ============================== $ PYTHONPATH=. pytest tests/test_router_result_cache.py tests/test_router_cache_policy.py tests/test_transforms_content_router.py tests/test_compression_store.py tests/test_compression_safety_rails.py tests/test_compression_determinism.py tests/test_compression_decision.py tests/test_compression_units.py tests/test_compression_batches.py tests/test_compression_cache.py tests/test_ccr_batch_store.py -q ======================= 423 passed, 2 skipped in 23.02s ======================== ``` *(Scope note: The full repository test suite was not run locally because optional extras such as `[proxy]`/`fastapi` are not installed in this environment; `ruff check .`, `ruff format --check .`, and `mypy headroom` ran across the entire repository.)* ## Real behavior proof - Environment: Debian Linux (Python 3.11.2), Headroom `main` (`0368a87`) with `headroom._core` from `headroom-ai==0.44.0`, `InMemoryBackend(max_entries=2)`, `kompress_model="disabled"`. - Exact command / steps: Ran the issue #4796 reproduction (`compress` on 3 distinct 251-line tool log payloads so entry 0 is capacity-evicted from `CompressionStore`, followed by a repeat `compress(originals[0])` and `store.retrieve(repeat_key)`). - Observed result: - Before patch: `before_repeat_retrieve: None`, `repeat_key == keys[0]: True`, `store.retrieve(repeat_key): None` - After patch: `before_repeat_retrieve: None`, `after_repeat_key_match: True retrieved_present: True exact_match: True` (`store.retrieve(repeat_key).original_content` equals `originals[0][-1]["content"]`). - Not tested: Distributed Redis `CompressionStore` backend, `[proxy]` extra tests (`fastapi` not installed), or live upstream LLM API calls; `kompress_model="disabled"` was used as in the issue reproduction. ## Runtime Rollout Safety - Rollout-managed feature(s): None (bug fix in `ContentRouter` Tier 2 cache hit validation). - Minimum rollout channel: N/A - Stable/default behavior changed: Only invalidates cached CCR marker hits when their store-backed CCR entry is no longer present in `CompressionStore`. - Kill switch / disable path: Standard `HEADROOM_COMPRESSION_CACHE` disable / `invalidate`. - Unsafe override required: No. - Qualification impact: Prevents unrecoverable CCR markers on repeat compression after store capacity eviction. - Rollback path: Revert this single commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A — internal cache bug fix, no public API or doc change) - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4872" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
d58b3ef19b |
fix(cache): clear older skips when storing successful results (#4819)
## Description
Concurrent compression requests can finish with different verdicts for
the same content key. If a no-op marks the key as skipped before another
request stores a successful result, the old put leaves that skip in
place. The next request returns the original even though the successful
compression is present in memory and SQLite.
Remove the older skip under the existing memory lock when admitting a
successful result. First-sight mark_skip and explicit move_to_skip
retain their original behavior. The initial PR also routed mark_skip
through disk invalidation; review identified the unnecessary DELETEs
that added to new non-compressible blocks, so that part has been
removed.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- Clear an older skip when a successful result is admitted.
- Cover both existing skip APIs with memory-only and SQLite-backed
caches.
- Trace real SQLite statements to preserve the memory-only first-sight
skip path.
## Testing
- [x] Relevant unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Formatting passes (`ruff format --check .`)
- [x] Type checking passes (original full-package mypy hook in WSL)
- [x] Regression tests added
- [x] Standalone runtime validation completed
### Test Output
```text
Four successful-result regressions failed on original main.
The added first-sight SQL regression failed on the initial PR and passes after review revision.
pytest tests/test_router_result_cache.py tests/test_router_cache_policy.py tests/test_transforms/test_content_router.py tests/test_content_router_concurrency.py tests/test_compression_determinism.py tests/test_cache tests/test_cold_prefix.py:
561 passed, 3 skipped
Original pre-commit and commit-msg hooks: passed
Whole-tree Ruff check and format --check: passed
Original pre-push make ci-precheck: passed
Rust workspace: 1749 passed, 3 ignored
Native extension: rebuilt, installed and import-verified
Critical Python: 177 passed, 8 skipped
commitlint: passed
```
Optional-model skips and the existing token-count approximation warning
remain visible in the gate logs.
## Real Behavior Proof
- Environment: Windows 11, CPython 3.13.9, existing dev venv; clean
baseline `c05fbdddc873078dd97cb9ede841e75ef7ed0bef`, patch
`b7f8872ac59c41abd12e2274364bcfecba67ceb7`. Explicit checkout on
PYTHONPATH; no provider credentials. The proof enables loopback HTTP;
model downloads remain disabled.
- Exact command / steps: Save the following script as `proof.py` and run
`python proof.py` from either checkout with `PYTHONPATH=.`. Identical
before/after scripts ran outside pytest.
```python
"""Two actual router requests with real remote HTTP, then SQLite replay."""
from concurrent.futures import ThreadPoolExecutor
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import tempfile
import threading
from headroom.cache.result_store import ResultStore
from headroom.tokenizers.estimator import CharacterCounter
from headroom.transforms.content_router import CompressionCache, ContentRouter, ContentRouterConfig
os.environ['HEADROOM_CCR_BACKEND']='memory'
os.environ['HEADROOM_OFFLINE']='0' # This script binds and calls loopback only.
os.environ['HEADROOM_FREEZE_BLOCK_DECISION']='0'
first_seen, release_first = threading.Event(), threading.Event()
lock=threading.Lock();calls=[]
short='The deployment is healthy and the requested records are present.'
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
payload=json.loads(self.rfile.read(int(self.headers['Content-Length'])))
with lock:
index=len(calls);calls.append(payload)
if index==0:
first_seen.set()
if not release_first.wait(15): raise RuntimeError('controller did not release response')
text=short
else:
text=payload['content']
data=json.dumps({'compressed':text,'original_tokens':len(payload['content'].split()),'compressed_tokens':len(text.split()),'compression_ratio':0.1 if index==0 else 1.0}).encode()
self.send_response(200);self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(data)));self.end_headers();self.wfile.write(data)
def log_message(self,*args):pass
server=ThreadingHTTPServer(('127.0.0.1',0),Handler)
thread=threading.Thread(target=server.serve_forever,daemon=True);thread.start()
os.environ['HEADROOM_KOMPRESS_ENDPOINT']=f'http://127.0.0.1:{server.server_port}'
payload=[{'role':'tool','tool_call_id':'t1','content':'The service contains useful deployment observations for the current investigation. '*80}]
tokenizer=CharacterCounter()
try:
with tempfile.TemporaryDirectory(prefix='headroom-tier-proof-') as d:
router=ContentRouter(ContentRouterConfig(ccr_inject_marker=False, result_cache_disk=False))
disk=ResultStore(Path(d)/'cache.db',1800)
router._cache=CompressionCache(store=disk,namespace='proof:')
def apply():return router.apply(payload,tokenizer,force_kompress=True).messages[0]['content']
# The real HTTP client is built once before sharing the router.
router._get_kompress()
with ThreadPoolExecutor(max_workers=2) as pool:
earlier=pool.submit(apply)
assert first_seen.wait(10),'first router call did not reach HTTP'
later=pool.submit(apply)
noop=later.result(timeout=10)
release_first.set()
compressed=earlier.result(timeout=10)
replay=apply()
reopened=ContentRouter(ContentRouterConfig(ccr_inject_marker=False,result_cache_disk=False))
reopened._cache=CompressionCache(store=ResultStore(Path(d)/'cache.db',1800),namespace='proof:')
disk_replay=reopened.apply(payload,tokenizer,force_kompress=True).messages[0]['content']
print(json.dumps({'noop_is_original':noop==payload[0]['content'],'successful_request_is_compressed':compressed==short,'next_request_reuses_success':replay==short,'reopened_sqlite_reuses_success':disk_replay==short,'skip_entries':router._cache.skip_size,'http_calls':len(calls)},indent=2))
reopened._cache._store._db.close()
disk._db.close()
router._get_kompress()._client.close()
finally:
release_first.set();server.shutdown();server.server_close();thread.join(5)
```
The first-sight SQLite trace was also run outside pytest with this
script:
```python
"""Trace actual SQLite statements for first-sight negative cache entries."""
import json
from pathlib import Path
import tempfile
from headroom.cache.result_store import ResultStore
from headroom.transforms.content_router import CompressionCache
with tempfile.TemporaryDirectory(prefix='headroom-skip-io-') as directory:
store=ResultStore(Path(directory)/'cache.db',1800)
cache=CompressionCache(store=store)
statements=[]
store._db.set_trace_callback(statements.append)
for i in range(100):cache.mark_skip(f'new-{i}')
print(json.dumps({'new_skip_entries':cache.skip_size,'sqlite_statements':len(statements),'deletes':sum(s.startswith('DELETE ') for s in statements)}))
store._db.close()
```
- Observed result: Two real ContentRouter.apply calls reach a bound
loopback RemoteKompress endpoint. The controller lets the no-op finish
first, then releases the successful compression. On clean main,
next_request_reuses_success=false and skip_entries=1, while reopening
SQLite reads the successful result. With the patch both the next request
and reopened cache reuse that result, skip_entries=0, and the endpoint
still receives only the original two calls. The trace script records 100
DELETEs for 100 new skips on the initial PR, versus zero SQL statements
after this revision.
- Not tested: Trained-model quality, paid endpoints, production
throughput, cross-process in-memory coherence, network filesystems, or
the complete Python suite. HTTP responses are deterministic synthetic
fixtures; actual router, HTTP client and SQLite operations are used.
## Runtime Rollout Safety
- Rollout-managed feature(s): Existing cache behavior; no new feature
flag or channel expansion.
- Minimum rollout channel: Normal release after maintainer review and CI
qualification.
- Stable/default behavior changed: A successful store supersedes an
older skip. First-sight negative classification keeps its existing
memory-only path.
- Kill switch / disable path: Revert this commit to restore the prior
successful-store behavior; no new configuration is introduced.
- Unsafe override required: No.
- Qualification impact: Related tests and original local gates passed;
hosted CI is tracked separately.
- Rollback path: Revert this commit; no dependency or storage-schema
change.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have added tests that prove the fix
- [x] Relevant new and existing tests pass locally
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Developed and validated with Codex assistance. I reviewed the initial
changes and runtime evidence; the review follow-up was checked with
Codex.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4819"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
90cb69ed4e |
fix(cache): order disk writes with result invalidation (#4766)
## Description
A router cache `put()` admitted its value before taking the disk-write
lock, while `clear()` and `move_to_skip()` did not share that lock. A
delayed write could run after invalidation and restore the old SQLite
row; a fresh cache instance then read a result that had been cleared.
The text-only check for a newer put also let an older write replace
newer ratio/strategy metadata when both texts matched.
Order disk invalidations with writes, and identify each in-flight write
with a revocable token. Clear/skip transitions and newer puts revoke
older tokens even after memory-budget eviction. Tokens are removed when
the write finishes, so tracking is bounded by in-flight work. This
preserves the existing memory admission order and disk-read race
protections.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- Serialize clear/skip disk mutations with writes using the existing
disk lock.
- Check write identity instead of comparing only compressed text.
- Exercise delayed writes before clear/skip, subsequent replacements,
matching-text metadata, and reopening the actual SQLite file; retain
existing concurrency tests.
- Keep skip deletion inside the memory critical section so a newly
starting disk read cannot re-admit the row before deletion.
## Testing
- [x] Relevant unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Formatting passes (`ruff format --check .`)
- [x] Type checking passes (original full-package mypy hook)
- [x] Regression tests added
- [x] Standalone runtime validation completed
### Test Output
```text
Initial new regressions against original production code: 3 failed, 2 passed
pytest tests/test_router_result_cache.py tests/test_transforms/test_content_router.py tests/test_cache tests/test_cold_prefix.py:
485 passed, 3 skipped
Review follow-up on the initial PR head: 1 failed
Original pre-commit and commit-msg hooks (including follow-up): passed
Whole-tree Ruff check / format --check: passed (1932 files)
Original pre-push make ci-precheck: passed
Rust workspace: 1749 passed, 3 ignored
Native extension: built, installed and import-verified
Critical Python: 177 passed, 8 skipped
commitlint: passed
```
The three cache skips require optional spaCy/embedding models.
Critical-suite skips require optional relevance models. The existing
token-count approximation warning remains visible in the gate log.
Initial broader runs failed because the new worktrees lacked the native
extension; these were rerun successfully after building it.
## Real Behavior Proof
- Environment: Windows 11, CPython 3.13.9, existing development venv;
clean baseline `200c838c7393d20ae56a0c338f832ed27133c398`, patch
`d016ccc32ab38d2df01e133154ada22e8eef8c56`. `HEADROOM_OFFLINE=1`,
`PYTHONPATH=.` for the selected checkout. No paid/provider credentials.
- Exact command / steps: Save the following identical script as
`proof.py` and run `python proof.py` from either checkout. Before and
after runs were outside pytest.
```python
"""Real SQLite rows after deterministic write/invalidation interleavings."""
import json
from pathlib import Path
from tempfile import TemporaryDirectory
import threading
from headroom.cache.result_store import ResultStore
from headroom.transforms.content_router import CompressionCache
reports = []
with TemporaryDirectory() as directory:
for operation in ("clear", "move_to_skip"):
path = Path(directory) / (operation + ".db")
store = ResultStore(path, 1800)
cache = CompressionCache(store=store, namespace="proof:")
waiting, release = threading.Event(), threading.Event()
real = cache._disk_lock
class PauseFirstWrite:
pending = True
def __enter__(self):
if self.pending:
self.pending = False
waiting.set()
if not release.wait(10): raise RuntimeError("Writer release missing")
return real.__enter__()
def __exit__(self, *exc): return real.__exit__(*exc)
cache._disk_lock = PauseFirstWrite()
writer = threading.Thread(target=cache.put, args=("k", "old response", 0.5, "text"))
writer.start()
try:
assert waiting.wait(10)
if operation == "clear": cache.clear()
else: cache.move_to_skip("k")
finally:
release.set()
writer.join(10)
assert not writer.is_alive()
reopened_store = ResultStore(path, 1800)
reopened = CompressionCache(store=reopened_store, namespace="proof:")
reports.append({"operation": operation, "disk_row": reopened_store.get("proof:k"),
"reopened_cache_result": reopened.get("k"), "memory_entries": cache.size})
# Close real SQLite handles before TemporaryDirectory removes files on Windows.
store._db.close()
reopened_store._db.close()
store = ResultStore(Path(directory) / "read-race.db", 1800)
cache = CompressionCache(store=store)
cache.put("k", "old response", 0.5, "text")
real = cache._lock
reads = []
class ReadAfterUnlock:
pending = True
def __enter__(self): return real.__enter__()
def __exit__(self, *exc):
real.__exit__(*exc)
if self.pending:
self.pending = False
reads.append(cache.get("k"))
cache._lock = ReadAfterUnlock()
cache.move_to_skip("k")
reports.append({"operation": "get-during-move-to-skip", "interleaved_reads": reads,
"memory_result_after_skip": cache.get("k"), "disk_row": store.get(cache._namespace + "k")})
store._db.close()
print(json.dumps(reports, indent=2))
```
- Observed result: A controlled lock boundary pauses a real put after
memory admission. On clean main, clear and move_to_skip both finish, but
releasing the writer recreates the old SQLite row and reopening returns
old response. With this patch, both the persisted row and reopened
result are None for both operations. No cache/store return values are
mocked. The review follow-up also starts a real get at the first
memory-unlock boundary of move_to_skip. On the initial PR head that read
restores old response in memory even though the disk row is deleted;
with the follow-up the read, final memory result and disk row are all
None.
- Not tested: Cross-process in-memory cache coherence, network
filesystems, production throughput and the complete Python suite. The
ordering guarantee is for mutators on one cache instance; SQLite still
provides its existing shared-file behavior.
## Runtime Rollout Safety
- Rollout-managed feature(s): Existing cache behavior; no new feature
flag or channel expansion.
- Minimum rollout channel: Normal release after maintainer review and CI
qualification.
- Stable/default behavior changed: A delayed write cannot resurrect an
invalidated result; newer same-text writes retain their metadata. Memory
reads and byte-budget eviction keep their existing behavior.
- Kill switch / disable path: HEADROOM_COMPRESSION_CACHE_DISK=0 disables
the existing disk tier.
- Unsafe override required: No.
- Qualification impact: Related cache tests and original local gates
passed; hosted CI is tracked separately.
- Rollback path: Revert this commit. No dependency or storage-schema
change.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have added tests that prove the fix
- [x] Relevant new and existing tests pass locally
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Implementation and local validation were completed with Codex
assistance. The author reviewed the initial code and validation record;
the review follow-up was checked with Codex.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4766"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
|
||
|
|
b196424e35 |
fix(cache): bound the router's compression cache, keep the rest in SQLite (#4357)
## Why The content router's result cache (`CompressionCache`, original → compressed) had no size limit, and its 30-min TTL was only checked when an entry was read again. Content that never came back stayed in memory for the life of the proxy. A 300,000-result soak grew past 550 MB. ## Change - **Memory budget:** results kept least-recently-used within `HEADROOM_COMPRESSION_CACHE_MB` (default 64, results only, sized with `sys.getsizeof`); skip set capped to match. The same soak now levels off (~+170 MB process growth, cache at 67 MB). - **Idle expiry:** a result expires 75 min after its *last use*. Nothing signals a session's end; once a session has been idle past the provider's longest prompt-cache TTL (1 h), there's no cached prefix left to protect, so recompressing on resume costs only the compression. Results with a CCR marker keep the CCR TTL from creation. - **Disk tier** (`headroom/cache/result_store.py`, proxy only): `workspace_dir()/compression_cache.db`, owner-only, WAL. An entry the budget evicted, or a restart lost, is read back instead of recompressed (a recompression could change the bytes of a cached prefix). - Namespaced by a digest of the router config + headroom version, so differently configured routers never share output. - Marker-free results only. - A SQLite error is a miss, never a failed transform. - Expired rows are deleted on open. - Off in stateless mode or with `HEADROOM_COMPRESSION_CACHE_DISK=0`; `HEADROOM_COMPRESSION_CACHE_PATH` moves it. - **Stable keys:** a blake2b digest (`content_cache_key`) instead of per-process `hash()`, so workers share the file and a shared store for replicas (Redis, for `/v1/compress` behind gateways) can follow. ## Platforms SQLite, blake2b and pathlib behave the same on macOS, Linux and Windows. File permissions go through `connect_private_sqlite` (owner-only on POSIX, inherited ACL on Windows, as for `ccr_store.db`). As with the CCR store, WAL needs a local filesystem. ## Tests `tests/test_router_result_cache.py`: - budget and LRU order - skip cap - evicted result comes back from disk unchanged - idle result dropped while a live one is kept - cross-config isolation - namespace and keys stable across processes - DB error is a miss - CCR-marker results stay off disk - stateless writes no file 731 router/proxy tests pass. ruff 0.17.0 and mypy clean on the changed files. `conftest` points the file at `tmp_path` so tests never write `~/.headroom`. ## Real behavior Environment: macOS 26, Python 3.12, this branch, default budget (64 MB). Steps: one process puts 300,000 distinct results of ~1.8 KB each (about 550 MB of text) through `CompressionCache`, measuring RSS after each 50,000; once memory-only, once with the SQLite tier. | results put | 50K | 100K | 150K | 200K | 250K | 300K | |---|---|---|---|---|---|---| | RSS growth, memory only (MB) | 89 | 91 | 168 | 168 | 168 | 168 | | RSS growth, with disk tier (MB) | 92 | 94 | 171 | 171 | 171 | 171 | The cache holds 67 MB at its budget. With the disk tier, the first result put (long evicted from memory) still reads back unchanged. Before this PR the same run grows without bound (no cap; TTL checked only on read). Not run: Windows (relies on CI's Windows job); a long-lived production proxy. ## Review fixes |