mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
`/v1/compress` returns a body that someone else's client forwards
unchanged, so anything Headroom adds there reaches the provider. Two
things got added that the caller never asked for.
## Stale-Read retrieval hashes in marker-free modes
Read lifecycle replaces a Read that a later edit made stale with a short
notice. That notice always ended in `Retrieve original: hash=…` and
wrote the original to the CCR store, even in the default and
`lossy_inline` modes. The docs call those modes marker-free, and their
callers have no retrieval tool, so the hash pointed nowhere.
- Marker-free modes keep the notice ("re-read the file for current
content") but drop the hash and skip the store write.
- Lossless routers (proxy `--lossless`,
`ContentRouterConfig(lossless=True)`) skip read lifecycle entirely,
because the notice drops the file's bytes.
- Read maturation now recognises the hashless notice. Otherwise it would
replace the notice with a marker whose hash points at the notice itself.
## Cache breakpoints the client didn't send
On the session path, `normalize_message_cache_control` fell back to
placing `{"type": "ephemeral"}` on the last block message when the
client marked nothing. A caller that chose not to cache got billed cache
writes. `/v1/compress` now passes `place_unmarked=False`: if the client
marked no message, markers are stripped and none are placed. Proxy
handlers keep today's behaviour, and client markers in a misaligned list
still go through the old path.
## Tests
Each new test fails on main.
- `test_marker_free_modes_mark_a_stale_read_without_a_retrieval_hash`,
for the default mode and `lossy_inline`
- `test_lossless_router_keeps_a_stale_read`
- `test_session_turns_add_no_marker_the_caller_did_not_send`
- `test_respects_lifecycle_markers`, extended to the hashless notice
The related suites (gateway, transforms, read maturation, prefix
tracker, cache_control, compress endpoint and session) pass: 1371 passed
and 9 skipped before the review fixes, then 222 of 222 on the touched
files after them. Lint is clean on CI's ruff 0.17.0, and mypy is clean
on the touched modules.
## Not in this PR
- Read maturation (`handlers/anthropic.py`) is not gated on `--lossless`
or `--no-ccr` either. It's the same kind of bug on the proxy path, so it
belongs in a separate fix.
- The "lossless densify only" pipeline in `compress.py` sets
`ccr_enabled=False` without `lossless=True`, so read lifecycle still
runs there.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4678"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
## Real Behavior Proof
Maintainer qualification on Windows, Python 3.12.13, using an actual
Uvicorn server bound to an owned loopback socket and HTTPX over real
HTTP. The normal `/v1/compress` endpoint and compression pipeline ran
with offline mode and Kompress disabled; no handler, compression stage,
HTTP response or store was mocked. Workspace, configuration, savings and
TOIN paths were isolated. All 243 native inputs and the reused artifact
SHA-256
`5be63a4f31d3e38e7427d28380ecb977616e3f6a3dd6a1c59af5ff6361a05978` were
verified; this was not a fresh native build.
- Exact command / steps: the committed probe is `python -m pytest -q -s
tests/test_compress_runtime_http.py`. The qualifying run used the
maintainer loopback-only launcher with an explicit verified native path
and SHA-256, followed by the four focused
session/API/maturation/lifecycle suites. All 138 tests passed. The probe
starts the actual server, GETs `/health`, and sends nine POSTs to
`/v1/compress`: a Read followed by Edit in default, lossy-inline and
lossless modes, then three session turns without caller cache markers
and three turns with a caller marker. It stops and joins the server in
`finally`.
- Observed result: health and all nine compression requests returned
HTTP 200. Default and lossy-inline emitted stale notices with no
retrieval hash or CCR hashes, saving 3,576 tokens per request. Lossless
preserved the original Read bytes. All three unmarked session turns
returned zero cache markers; all three marked controls retained exactly
one. Session compression saved 7,350 / 15,901 / 25,958 tokens. Server
shutdown joined successfully.
- Before-fix control: the same real-server requests with all six
affected production modules replaced by current main
`200c838c7393d20ae56a0c338f832ed27133c398` failed five assertions:
default and lossy-inline included retrieval hashes, and each unmarked
session turn added a cache marker. Lossless and the three marked
controls passed on main and are not claimed as reproduced regressions.
Candidate source was restored byte-for-byte.
- Not tested: provider forwarding or billing, a live model, deployment
behavior, or the separate proxy read-maturation scope already listed
above. External startup connection attempts were blocked by the
qualifying launcher; this is not a zero-attempt network claim. No author
readiness or self-review declaration is added by this maintainer
evidence.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
150 lines
6.1 KiB
Python
150 lines
6.1 KiB
Python
"""Actual local uvicorn/HTTP exercise of the published compress endpoint."""
|
|
|
|
import hashlib
|
|
import json
|
|
import os
|
|
import socket
|
|
import threading
|
|
import time
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
|
|
def test_actual_server_compress_contract(tmp_path, monkeypatch):
|
|
for key, path in {
|
|
"HEADROOM_WORKSPACE_DIR": "workspace",
|
|
"HEADROOM_CONFIG_DIR": "config",
|
|
"HEADROOM_SAVINGS_PATH": "savings.json",
|
|
"HEADROOM_SAVINGS_EVENTS_PATH": "events.jsonl",
|
|
"HEADROOM_TOIN_PATH": "toin.json",
|
|
}.items():
|
|
monkeypatch.setenv(key, str(tmp_path / path))
|
|
for key, value in {
|
|
"HEADROOM_OFFLINE": "1",
|
|
"HEADROOM_TELEMETRY": "off",
|
|
"HEADROOM_BEACON": "off",
|
|
"HEADROOM_CC_SWITCH_RECONCILE": "0",
|
|
"HEADROOM_KOMPRESS_BACKGROUND_WARMUP": "0",
|
|
}.items():
|
|
monkeypatch.setenv(key, value)
|
|
httpx = pytest.importorskip("httpx")
|
|
uvicorn = pytest.importorskip("uvicorn")
|
|
native = pytest.importorskip("headroom._core")
|
|
from headroom.proxy.server import ProxyConfig, create_app
|
|
from tests.test_compress_session_mode import _anthropic_turns, _count_message_markers
|
|
from tests.test_proxy_compress_endpoint import TestCompressEndpointLossyInlineMode
|
|
|
|
native_path = Path(native.__file__).resolve()
|
|
expected = Path(os.environ.get("HEADROOM_REVIEW_NATIVE_PATH", str(native_path))).resolve()
|
|
assert native_path == expected
|
|
digest = hashlib.sha256(native_path.read_bytes()).hexdigest()
|
|
if expected_digest := os.environ.get("HEADROOM_REVIEW_NATIVE_SHA256"):
|
|
assert digest == expected_digest
|
|
app = create_app(
|
|
ProxyConfig(
|
|
host="127.0.0.1",
|
|
optimize=True,
|
|
offline=True,
|
|
memory_enabled=False,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
image_optimize=False,
|
|
disable_kompress=True,
|
|
subscription_tracking_enabled=False,
|
|
periodic_toin_stats_enabled=False,
|
|
periodic_malloc_trim_enabled=False,
|
|
)
|
|
)
|
|
listener = socket.socket()
|
|
listener.bind(("127.0.0.1", 0))
|
|
port = listener.getsockname()[1]
|
|
server = uvicorn.Server(uvicorn.Config(app, log_level="warning", loop="asyncio"))
|
|
worker = threading.Thread(target=server.run, kwargs={"sockets": [listener]}, daemon=True)
|
|
cases = []
|
|
health = None
|
|
worker.start()
|
|
try:
|
|
deadline = time.monotonic() + 20
|
|
while not server.started and worker.is_alive() and time.monotonic() < deadline:
|
|
time.sleep(0.05)
|
|
assert server.started and app.state.ready
|
|
with httpx.Client(
|
|
base_url=f"http://127.0.0.1:{port}", trust_env=False, timeout=60
|
|
) as client:
|
|
health = client.get("/health").status_code
|
|
assert health == 200
|
|
history = TestCompressEndpointLossyInlineMode._read_then_edit()
|
|
for mode in [None, "lossy_inline", "lossless"]:
|
|
payload = {"messages": history, "model": "gpt-4"}
|
|
if mode is not None:
|
|
payload["config"] = {"mode": mode}
|
|
response = client.post("/v1/compress", json=payload)
|
|
assert response.status_code == 200, response.text
|
|
data = response.json()
|
|
content = data["messages"][2]["content"]
|
|
ok = (
|
|
content == history[2]["content"]
|
|
if mode == "lossless"
|
|
else content.startswith("[Read content stale: /x/a.py")
|
|
and "hash=" not in content
|
|
and not data["ccr_hashes"]
|
|
)
|
|
cases.append(
|
|
{
|
|
"case": mode or "default",
|
|
"status": response.status_code,
|
|
"ok": ok,
|
|
"hashes": data["ccr_hashes"],
|
|
"read_bytes_preserved": content == history[2]["content"],
|
|
"contains_retrieval_hash": "hash=" in content,
|
|
"tokens_saved": data["tokens_saved"],
|
|
}
|
|
)
|
|
for marked in (False, True):
|
|
for turn in range(1, 4):
|
|
messages = _anthropic_turns(turn)
|
|
if not marked:
|
|
del messages[-1]["content"][-1]["cache_control"]
|
|
response = client.post(
|
|
"/v1/compress",
|
|
json={
|
|
"messages": messages,
|
|
"model": "claude-sonnet-5",
|
|
"config": {"session_id": f"actual-http-{marked}"},
|
|
},
|
|
)
|
|
assert response.status_code == 200, response.text
|
|
data = response.json()
|
|
count = _count_message_markers(data["messages"])
|
|
cases.append(
|
|
{
|
|
"case": f"session-marked-{marked}-turn-{turn}",
|
|
"status": response.status_code,
|
|
"markers": count,
|
|
"ok": count == (1 if marked else 0),
|
|
"tokens_saved": data["tokens_saved"],
|
|
}
|
|
)
|
|
finally:
|
|
server.should_exit = True
|
|
worker.join(timeout=15)
|
|
listener.close()
|
|
report = {
|
|
"native_path": str(native_path),
|
|
"native_sha256": digest,
|
|
"port": port,
|
|
"health": health,
|
|
"cases": cases,
|
|
"shutdown_joined": not worker.is_alive(),
|
|
"scope": "Actual uvicorn and loopback HTTP with normal endpoint/pipeline;no provider upstream,billing,MLmodel or productiondeployment claim",
|
|
}
|
|
Path(
|
|
os.environ.get("REVIEW4678_RUNTIME_REPORT", str(tmp_path / "runtime-report.json"))
|
|
).write_text(json.dumps(report, indent=2))
|
|
print("ACTUAL_HTTP_REPORT " + json.dumps(report))
|
|
assert not worker.is_alive()
|
|
assert any(x["tokens_saved"] > 0 for x in cases), "compression did not execute"
|
|
assert all(x["ok"] for x in cases), report
|