Commit Graph
23 Commits
Author SHA1 Message Date
Mansurshakh JaparovandJD Davis 0f3b0d8a21 fix(memory): resolve pi cwd before injecting project tools (#4618)
## Description

Closes #4442.

pi's system `<cwd>` block now resolves the memory project. If project
scope is still unresolved with the `empty` fallback, the Anthropic
handler no longer advertises unusable memory tools, including tools
recorded earlier in the session.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Parse pi cwd only in system text. Keep main's line parser, header/CLI
precedence and limited user-message fallback.
- Reuse `is_project_unresolved` for context and sticky-tool replay.
Client tools and global/user storage behavior stay intact.
- Add initialized-handler scope regressions, replace
source-inspection/forwarding-only checks, and document project scope in
the published memory page.

## Testing

- [x] Unit tests pass (`pytest`) — related suite and the normal critical
gate, not the entire Python suite locally.
- [x] Linting passes (`ruff check .`) — hosted lint passed on 5ace63e2;
local filesystem limitation below.
- [x] Type checking passes (`mypy headroom`) — normal full hook with its
existing `--ignore-missing-imports` setting.
- [x] New tests added for the scope regression
- [x] Manual testing performed

### Test Output

```text
# 16 related test files
329 passed, 12 skipped, 1 warning in 43.07s

# ordinary pre-push: make ci-precheck
178 passed, 5 skipped, 1 warning in 10.87s
✅ ci-precheck PASSED — safe to push.

uv run ruff format --check .
1860 files already formatted
```

Normal commit hooks (Ruff, formatter, full mypy, commitlint) and the
complete pre-push path passed: Rust fmt/clippy/workspace tests, native
build/import verification, critical Python tests and commitlint. The
existing Starlette test-client warning was not filtered.

Local whole-tree Ruff reports EXE002 on five **unchanged** OAuth plugin
files: my `fuseblk` checkout exposes them as executable despite Git
modes `100644`; `chmod` returns zero without changing the mode. The
uncached check agrees. I did not change those files, add shebangs or
ignore the rule. Hosted Ruff, formatting and full mypy passed on
5ace63e2; the local filesystem limitation remains.

The first full CI caught a missing scope predicate on the existing
lifecycle test double. Fixed in
[5ace63e2](https://github.com/headroomlabs-ai/headroom/commit/5ace63e2cbd2295f9594d1e0442a679aad99979c);
its complete file passes (`5 passed, 1 warning in 13.01s`). This
follow-up changes no production code or assertions and adds no runtime
fallback. The behavior proof below ran on 7aeb83cf with the same
production source; that qualification is historical; current-head checks
are recorded below.

### Author follow-up: 80453a03

Moved the project-scope guidance out of the unpublished wiki and into
`docs/content/docs/memory.mdx`; this commit changes no production code
or tests. Viewed the actual Next.js `/docs/memory#proxy-project-scope`
page and its table-of-contents entry; no browser errors.

The existing router/scope tests pass: 65 passed, with four SQLite
ResourceWarnings. All seven pi-parser lines and all six branch edges are
covered, so I did not add duplicate tests for the old Codecov
percentage.

Normal commit and pre-push hooks passed, including Rust checks, native
build/import, 180 critical Python tests (5 skipped, 1
tokenizer-approximation warning), and commitlint. My first push timed
out with the wrong local ONNX path; the next found missing pip in the
fresh uv environment. Both setup issues were fixed without bypassing
checks. Current-head hosted CI is not yet qualified.

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, freshly built native core,
real TCP/Uvicorn proxy and SQLite/ONNX memory backend. Only the upstream
is a local Anthropic-shaped HTTP server; no provider charges.
- Exact command / steps: after `uv sync --locked --extra dev --extra
proxy --extra memory` and `make verify-rust-core`, run this from the
checkout. It prints the actual loaded native module/hash and leaves a
disposable `.headroom-4442-*` directory for inspecting SQLite. Both HTTP
listeners shut down before exit.

<details>
<summary>Standalone HTTP/SQLite reproduction</summary>

```bash
./.venv/bin/python - <<'PY'
import hashlib,http.client,json,os,socket,tempfile,threading,time
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from pathlib import Path
import uvicorn
import headroom._core as core
assert Path(core.__file__).resolve().parent == Path.cwd().resolve()/"headroom"
native_sha=hashlib.sha256(Path(core.__file__).read_bytes()).hexdigest()
from headroom.memory.storage_router import ProjectResolver,RequestContext,extract_system_prompt
from headroom.proxy.server import ProxyConfig,create_app
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES
from headroom.proxy.helpers import get_session_tool_tracker
root=Path(tempfile.mkdtemp(prefix=".headroom-4442-",dir=Path.cwd()))
project_a=root/"pi%2Fproject-a";project_b=root/"project-b"
project_a.mkdir(exist_ok=True);project_b.mkdir(exist_ok=True)
os.environ["HEADROOM_WORKSPACE_DIR"]=str(root/"workspace")
os.environ["HEADROOM_BEACON"]="off"
os.environ["HEADROOM_TELEMETRY"]="off"
os.environ["NO_PROXY"]="127.0.0.1,localhost"
forwarded=[]
class Gateway(BaseHTTPRequestHandler):
    def do_POST(self):
        body=json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        forwarded.append({"path":self.path,"body":body})
        payload=json.dumps({"id":"msg_local_scope","type":"message","role":"assistant","model":"claude-haiku-4-5","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","stop_sequence":None,"usage":{"input_tokens":10,"output_tokens":1}}).encode()
        self.send_response(200);self.send_header("Content-Type","application/json");self.send_header("Content-Length",str(len(payload)));self.end_headers();self.wfile.write(payload)
    def log_message(self,*args):pass
upstream=ThreadingHTTPServer(("127.0.0.1",0),Gateway)
ut=threading.Thread(target=upstream.serve_forever,daemon=True);ut.start()
config=ProxyConfig(mode="cache",anthropic_api_url=f"http://127.0.0.1:{upstream.server_port}",memory_enabled=True,memory_db_path=str(root/"legacy.db"),traffic_learning_enabled=True,subscription_tracking_enabled=False,cost_tracking_enabled=False,cache_enabled=False,rate_limit_enabled=False,log_requests=False,image_optimize=False,ccr_inject_tool=False,ccr_handle_responses=False,ccr_context_tracking=False,anthropic_pre_upstream_memory_context_timeout_seconds=30.0)
app=create_app(config)
sock=socket.socket();sock.bind(("127.0.0.1",0));sock.listen(128)
port=sock.getsockname()[1]
server=uvicorn.Server(uvicorn.Config(app,log_level="warning",lifespan="on",timeout_graceful_shutdown=3))
st=threading.Thread(target=server.run,kwargs={"sockets":[sock]},daemon=True);st.start()
try:
    deadline=time.monotonic()+180
    while not server.started:
        if not st.is_alive() or time.monotonic()>deadline:raise RuntimeError("real proxy startup failed")
        time.sleep(.02)
    session="headroom4442-native-project-scope"
    common={"Content-Type":"application/json","x-api-key":"local-unbilled-key","anthropic-version":"2023-06-01","x-headroom-user-id":"native-scope-user","x-headroom-session-id":session}
    client_tools=[{"name":"get_weather","description":"Weather","input_schema":{"type":"object","properties":{}}}]
    rows=[]
    cases=[("fresh_unresolved","You are helpful.",{},False),("pi_cwd",f"<cwd>\n{project_a}\n</cwd>",{},True),("warm_unresolved","You are helpful.",{},False),("pi_cwd_restored",f"<cwd>\n{project_a}\n</cwd>",{},True),("explicit_header_wins",f"<cwd>\n{project_b}\n</cwd>",{"x-headroom-project-id":"explicit-project"},True),("ordinary_user_cwd_ignored","You are helpful.",{},False)]
    for label,system,extra,expected in cases:
        body={"model":"claude-haiku-4-5","max_tokens":16,"system":system,"messages":[{"role":"user","content":f"cwd:\n<cwd>\n{project_b}\n</cwd>" if label=="ordinary_user_cwd_ignored" else "hello scope"}],"tools":client_tools}
        before=len(forwarded)
        conn=http.client.HTTPConnection("127.0.0.1",port,timeout=90)
        conn.request("POST","/v1/messages",body=json.dumps(body).encode(),headers={**common,**extra})
        response=conn.getresponse();payload=response.read();conn.close()
        assert response.status==200,(label,response.status,payload.decode())
        assert len(forwarded)==before+1,(label,"unexpected upstream request count")
        actual=forwarded[-1]["body"]
        names={tool.get("name") for tool in actual.get("tools",[])}
        assert names==({"get_weather"}|MEMORY_TOOL_NAMES if expected else {"get_weather"}),(label,names)
        scope=ProjectResolver().resolve(RequestContext(headers={**common,**extra},system_prompt=extract_system_prompt(body),base_user_id="native-scope-user"))
        rows.append({"case":label,"status":response.status,"memoryTools":sorted(names & MEMORY_TOOL_NAMES),"project":scope,"gatewayPath":forwarded[-1]["path"]})
    assert forwarded[1]["body"]["tools"]==forwarded[3]["body"]["tools"],"restored definitions changed"
    tracker=get_session_tool_tracker()
    assert tracker.should_inject("anthropic",session),"probe did not exercise an actual recorded sticky session"
    from urllib.parse import quote
    key_a=ProjectResolver().resolve(RequestContext(headers={"x-headroom-cwd":quote(str(project_a),safe="/")},system_prompt="",base_user_id="native-scope-user"))[0]
    assert rows[1]["project"][0]==key_a
    assert rows[4]["project"][1]=="explicit-project"
    router=app.state.proxy.memory_handler._router
    assert router is not None,"real memory backend never initialized"
    opened=router.open_backends()
    project_db=root/"memories"/"projects"/key_a/"memory.db"
    assert project_db in opened and project_db.is_file(),("real pi project database not opened",[str(x) for x in opened])
    print(json.dumps({"mode":config.mode,"realHttpListener":f"http://127.0.0.1:{port}","localGateway":f"http://127.0.0.1:{upstream.server_port}","rows":rows,"realStickyGoldenSession":True,"realPiProjectDatabase":str(project_db),"openedDatabases":[str(x) for x in opened],"nativeModule":core.__file__,"nativeSha256":native_sha,"noLivePiOrPaidProvider":True},indent=2))
finally:
    server.should_exit=True;st.join(20)
    upstream.shutdown();upstream.server_close();ut.join(5);sock.close()
    assert not st.is_alive() and not ut.is_alive(),"native proof leaked a server"


PY
```

</details>

- Observed result: unchanged `main` fc1ba891 fails at the first
unresolved request: five proxy memory tools reach upstream. With the
fix, all six requests return HTTP 200:

| Same-session request | Proxy memory tools |
| --- | ---: |
| Fresh, no cwd | 0 |
| pi system cwd | 5 |
| Cwd absent again | 0 |
| Same pi cwd restored | 5 |
| Explicit project header, different system cwd | 5 |
| Cwd block only in ordinary user text | 0 |

The client weather tool stays present. Restored definitions are
structurally equal to the first resolved turn; the explicit header wins.
The literal-percent cwd agrees with its encoded-header form, and the
actual project SQLite database opens at the expected key.

- Not tested: live pi/Caveman, paid-provider cache accounting/billing,
Windows, or other provider protocols. This does not claim to fix Caveman
header forwarding.

## Runtime Rollout Safety

- Rollout-managed feature(s): none added; existing Anthropic memory
injection.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: pi cwd resolves; unresolved
project/empty scope no longer adds unusable proxy memory tools.
- Kill switch / disable path: existing `--no-memory`; explicit
project/cwd headers still work.
- Unsafe override required: no.
- Qualification impact: no storage migration or new dependency; scoped
HTTP/SQLite behavior checked.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes — related
suite and critical gate, not a whole-suite claim
- [x] I did **not** edit `CHANGELOG.md` — release-please owns it

## Screenshots (if applicable)

N/A; the observable surface is the HTTP payload and project database.

## Additional Notes

AI-assisted; no subagents. OpenAI injection policy is unchanged. Keeping
this draft until the current hosted checks are qualified.

One earlier post-build probe completed the requests and closed both
listeners but hit its whole-process shutdown deadline. The same program
and native artifact later exited normally with faulthandler enabled; its
diagnostic abort timer did not fire. I have not identified the
intermittent cause and am not claiming a shutdown fix. No production
timeout or feature flag was changed to get that result.


The two npm audits now pass on 5ace63e2, along with the other Security
jobs. The earlier [unchanged-base
failure](https://github.com/headroomlabs-ai/headroom/actions/runs/41399323584)
is historical evidence, not a current blocker. This PR changes no
dependency manifests, lockfiles, workflow or audit exceptions.


### Maintainer current-main requalification

Exact head `60597623e00967153d947a46bf197950250f310e`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Clean mechanical update from current upstream main, preserving the pi
system-cwd resolver, request-scoped project guard, main's cache-mode
search skip/logging, and client-tool preservation. Ordinary user cwd
blocks still do not enable the pi fallback; unresolved project scope
still blocks both context injection and replayed memory tools. No
contributor production logic was rewritten and no new production finding
remains in this reviewed scope.

The initial combined tree dropped main's memory-invariant file and two
stream/timeout regression cases. Maintainer test-only repairs restore
those tests rather than reducing coverage: the invariant file is
byte-identical to main, every main assertion in the four
restored/updated test files is preserved by AST comparison, and three
fixtures now supply the handler protocol methods needed for resolved
scope and beta headers. The lifecycle main control passes without that
fixture extension; no claim is made that this particular fixture failure
independently fails main. All 222 affected
scope/invariant/stream/timeout/lifecycle/backpressure/cache-stability/system-prompt
tests pass with one skip. Ten shared Gemini request-fixture cases also
pass after the separate test-only repair for independently verified main
failures. All 12 main-relative changed Python files pass Ruff/format;
whitespace checks pass.

The author's existing real TCP/SQLite/ONNX reproduction and explicit
limits were read and remain in the body. These fresh Windows checks
qualify the combined request-scope regressions; they are not a claim of
rerunning that Linux proof, live pi/client/provider behavior, or a fresh
native build. An existing binary was supplied solely for test startup.
Draft status is preserved; readiness remains the author's decision and
nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `60e01cf7c25e90abb9f5cb66fa221da83e7c5177`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.

Clean integration with upstream/main
ea59ac3eef. Re-reviewed project-scoping
delta against intervening memory/backend/stream changes: pi cwd remains
limited to system context; ordinary user cwd blocks do not enable
fallback; unresolved projects skip context/tool injection and sticky
tool replay while retaining client tools and main cache-mode no-search
behavior.232
scope/invariant/stream/timeout/lifecycle/backpressure/cache/system-prompt/Gemini
tests pass,1optional skip; touched-source Ruff and whitespace pass. No
contributor production policy rewritten. Earlier author TCP/SQLite/ONNX
evidence remains scoped, with no fresh native build or live-pi/provider
claim. Existing draft preserved; final approval awaits exact-head hosted
qualification. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `e93ceb35df7f78bfa2dcc0611609131cf2b08b0a`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 232 passed, 1 skipped, 1 warning in
47.62s. Selected real dependency overlays and native DLL paths are
recorded in final4618-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

No new source finding in the reviewed scope. Final formal approval
awaits exact-head hosted qualification. Project resolution and
unresolved-scope injection/replay gates remain unchanged. Existing draft
status is preserved.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `d693dd2b6e33fd47ff853f7e28d938fc1dbc30ac`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378. Clean source
integration preserves pi systemcwd resolution and unresolved-scope
context/tool/sticky-replay gates.70integration patches exactly matchmain
delta;Anthropic exception patch differs only byclosing-parenthesis diff
alignment and its exception AST equalsmain. Broad
memory/router/projectscope/continuation/backpressure/cache/lifecycle/doctor/nativeops
run:405passed,1skipped,one newmain fake-handler fixture failed
missingis_project_unresolved. Maintainer adds only that fake method
returningFalse,respecting resolvedfixture scope;all39backpressure tests
nowpass,including unchanged
injectedSQLiteexception,HTTP200andsemaphorerelease assertions. No
production logic changed.33changedPythonlint/format,offline
lock/whitespacepass;fixedtestlint/formatrechecked.
Existingpr3982nativeDLLverified fullinputhashes/checksum;no
freshbuildclaimed. Currentauthor/Devin discussion checked,inline
endpoint empty;prior scoped realbehavior evidence remains applicable to
unchanged contributor inputs. No new sourcefinding;no
realprovider/agentlaunch claim. Draftpreserved.
Exactnewheadhostedqualification pending;maintainers
handlechecks,nothingmerged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `0da0b5a01958eaaac1d625227cb4bdb9477c5d3b`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main delta after diff-location
normalization. Contributor pi system-cwd resolution and
unresolved-project memory context/tool/sticky-replay gates are
unchanged. Fresh
memory-router/project-scope/streaming/continuation/backpressure/cache/lifecycle
and incoming Vertex/backend/CLI tests: 381 passed, 12 skipped, 1
warning. Twelve changed Python files pass Ruff lint/format; offline lock
and whitespace pass.

Static production/test scope and security review considered user versus
system cwd inputs, malformed/empty cwd, missing scope and stale tool
replay: tests keep pi cwd tags in ordinary user content from resolving
project scope, and unresolved scope blocks replay. No unrelated
executable, workflow, dependency or setup changes. Flagged inherited
workflow-validation job is excluded by unchanged workflow paths. Author
discussion checked; collected inline endpoint has no findings. Prior
scoped behavior evidence applies to unchanged contributor inputs; no new
real agent/provider launch claimed. Native DLL reused only after
complete tracked input-set/hash and artifact checksum verification, no
fresh build claimed. Draft preserved. Formal approval awaits exact
published-head hosted qualification. Maintainers handle checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-10 22:28:01 -06:00
Mansurshakh JaparovandJD Davis 9553d4292f fix(cache): retain prefix trackers for the provider cache lifetime (#4023)
## Description

Session cleanup currently drops prefix trackers after the default
600-second session lifetime even when Anthropic's request selects a
one-hour cache TTL. Losing the tracker also loses the frozen forwarded
prefix and causes the cache-preserving overlay to replay different
bytes.

Closes #4017.

## Type of Change

- [x] Bug fix
- [x] Documentation update

## Changes Made

- Resolve the request's existing Anthropic cache TTL before selecting
the prefix lineage and pass that TTL through tracker selection/creation.
- Retain each lineage for the greater of the configured session floor
and its observed/current provider TTL plus the existing 60-second safety
margin.
- Keep the lifetime per lineage rather than changing global
configuration: a five-minute session does not inherit another session's
one-hour lifetime.
- Preserve normal expiry beyond the provider lifetime and margin.
- Honor configured cache TTL before a confirmed cache write; discard
expired state before accepting a later request's longer TTL.
- Re-register the live lineage family after lookup/cleanup so recreated
sessions cannot share a tracker's frozen state across sibling
conversations or tool profiles.
- Update existing fake callbacks to the changed tracker contract,
preserve all consumer byte-stability assertions, and remove one
handler-to-store wiring-only test rather than re-pinning its internal
identity assertions.
- Document local tracker retention separately from provider cache
availability.

## Testing

- [x] Unit tests pass (`pytest`), including frozen-prefix byte stability
and independent TTL expiry.
- [x] Changed-file lint, configured pre-commit mypy and manual
production timestamp/overlay checks pass.

### Test Output

```text
Review correction: 273 passed, 2 dependency warnings in 83.79s
{"expires_beyond_provider_ttl_and_margin": true, "five_minute_session_evicted": true, "forwarded_prefix_byte_exact": true, "initial_frozen": 1, "one_hour_session_retained": true, "resolved_ttl_seconds": 3600, "resumed_frozen": 1, "same_tracker_after_660s": true}
```

The mandatory pre-push `make ci-precheck` passed: workspace Rust
fmt/clippy/tests, native extension build/import verification, the
complete configured Python gate, and Commitlint. The native gate
explicitly used the installed CPU `libonnxruntime.so.1.26.0` via
`ORT_DYLIB_PATH`, without skipping cached-model assertions.

```text
[build_rust_extension] headroom._core build + install + verify: OK
================== 177 passed, 4 skipped, 1 warning in 12.87s ==================
✅ ci-precheck PASSED — safe to push.
```

```sh
python -m pytest -q tests/test_cache/test_prefix_tracker.py
python -m pytest -q tests/test_proxy_anthropic_cache_stability.py tests/test_cache_prefix_overlay.py tests/test_cache_ttl_preserved.py tests/test_cold_prefix.py
```

**74 passed** in tracker tests; **54 passed** in request/cache tests
(one existing Starlette TestClient deprecation warning). Two new
regressions fail on baseline because the request TTL is not accepted; a
separate production runtime comparison demonstrates the actual pre-fix
loss, not just the missing parameter.

After rebasing onto current main
`1ad92f0cbc3dea27676e253588100f64a2af6994`, ran those suites together
with `tests/test_prefix_tracker_client_rewrote_reply.py`,
`tests/test_cache_mode_delta_marker.py`, and
`tests/test_content_router_mcp_small_result.py`: **150 passed**. The
actual production timestamp/byte-retention smoke was repeated
successfully on the rebased code, covering compatibility with main's
newly merged client-reply normalization.

Review regression and affected HTTP-consumer suite: **273 passed**,
including tracker expiry, Anthropic/OpenAI cache stability,
byte-faithful forwarding, header isolation, buffered timeouts, CCR
injection, and immutable system prompts. All affected two-argument test
callbacks now accept the TTL contract. The obsolete walker-call-order
test was removed; wire-byte assertions remain. The new
configured-TTL/expiry cases fail against the published baseline; the
post-cleanup regression also verifies that an unrelated conversation
gets a different tracker after recreation.

Changed-file Ruff, formatting, installed pre-commit `mypy headroom`, and
Commitlint hooks passed. Full-tree Ruff is locally blocked by executable
filesystem permissions on five unchanged OAuth2 plugin files (tracked
100644). A separate full dev-environment mypy run found an unchanged
optional LangChain `truthy-function` error; the configured isolated
pre-commit hook passed. No unrelated source/permissions changed.

## Real Behavior Proof

- Environment: Linux x86_64, CPython 3.13.14; unchanged
`dea9092e19385f286b7f67a64f4d85de2708d756` and this branch; real
`SessionTrackerStore`, `PrefixCacheTracker`, request TTL parser, token
estimator and `overlay_cached_prefix`, no mocked store or mocked clock.
- Exact command / steps: parse `cache_control: {type: ephemeral, ttl:
1h}` for `claude-sonnet-4-5`; establish one real cache-write-confirmed
frozen message with different original/forwarded prefix bytes; age
`_last_activity` to `time.time()-660` and make periodic cleanup due;
resume the same lineage and compare the tracker, frozen count, and
forwarded-prefix bytes. Also age beyond 3901 seconds and compare
independent 300/3600-second sessions.
- Before: resolved TTL 3600; after 660 seconds, tracker identity
changes, frozen count drops **1→0**, and forwarded-prefix bytes differ.
- Observed result: after the fix, after 660 seconds the same tracker
remains, frozen count stays **1**, and forwarded-prefix bytes are
identical. The one-hour session survives independently, the five-minute
session expires, and the one-hour lineage still expires beyond TTL plus
margin.
- Review runtime proof against published head `e1a107df`: actual stores
and prefix overlays confirm configured TTL retention before the first
cache write, no resurrection of expired state by a later one-hour
request, eventual expiry, and separate trackers after the maintainer's
3901-second cleanup/recreation sequence. On that exact baseline the
unrelated lookup reuses the wrong tracker, although the overlay guard
preserves the particular unrelated input bytes; after the fix both
identity isolation and byte preservation hold. Identical messages under
different tool profiles also receive a cold, separate tracker after the
expired bare-key transition.
- Not tested: Anthropic cloud cache availability, billed savings, or an
actual one-hour wait. The local production state and byte invariants
were exercised by aging actual timestamps, not by a paid upstream
request. Preserving a tracker cannot guarantee that the provider retains
its cache.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; repair of existing TTL-aware
cache-prefix behavior.
- Minimum rollout channel: stable; no rollout-managed feature is
enabled.
- Stable/default behavior changed: eligible local prefix trackers
survive for their provider cache lifetime. Short-TTL/default sessions
remain bounded by their own session floor.
- Kill switch / disable path: the existing prefix-freeze disable path
remains available; no new flag.
- Unsafe override required: no.
- Qualification impact: local Anthropic prefix lineage
selection/retention, not cache pricing or subscription quota policy.
- Rollback path: revert the commit to restore previous retention. No
persistent schema change.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] Existing cache TTL parser and safety margin reused; cache docs
updated.
- [x] Regressions cover frozen-byte retention, expiry, mixed lifetimes,
and existing request byte-stability behavior.
- [x] `CHANGELOG.md` not edited; Conventional Commit title supplied.

Remote CI results are not inferred from local verification.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-07 17:46:21 -05:00
613ae92150 fix(proxy): keep the newest user message verbatim in cache-mode delta compression (#3923)
## Description

**What users see:** with the default settings (`cache` mode plus the
`coding` savings profile), the proxy can rewrite the newest user message
of a conversation from the second request onward. #3619 made the rule
that the user's own prompt reaches the model verbatim, but one code path
missed it. For a Claude Code agent team this is the message a teammate
receives from the lead (`<teammate-message>…`), which is one way a
teammate can end up with a message it calls garbage (#1174).

**Why it happens:** in cache mode the first request of a conversation is
compressed in full. After that, only the new messages (the "stable
delta") are compressed and spliced onto the prefix that was already
forwarded. Every Anthropic `pipeline.apply(...)` call passes
`prefix_replay_guaranteed=True` except this delta call. Without that
flag the router does not protect the newest user message, and the coding
profile turns user-message compression on.

**Fix:** pass `prefix_replay_guaranteed=True` on the delta call as well.
The flag means "the caller replays last turn's forwarded bytes over this
turn's output", which is exactly what the delta path does. Tool results
in the newest message still compress; only the user's own text is kept
as written.

Refs #1174. This is one of two independent causes found while
investigating that issue; the other is in a separate PR (linked in the
issue). I am not using `Closes` because the reporter has not confirmed
which cause they hit.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/anthropic.py`: the cache-mode stable-delta
`anthropic_pipeline.apply(...)` call now passes
`prefix_replay_guaranteed=True`, matching the other five Anthropic
calls.
- `tests/test_proxy_anthropic_cache_stability.py`: new
`test_cache_mode_delta_keeps_newest_user_prompt_verbatim` drives a
second-turn request through the real handler and asserts that the delta
call carries the flag. It fails on `main` (`assert None is True`). What
the router does with the flag is already covered by
`test_follow_up_prompt_string_after_assistant_turn_is_verbatim_under_replay`
and its siblings in `tests/test_transforms_content_router.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# New test on unmodified main
$ pytest tests/test_proxy_anthropic_cache_stability.py -k delta_keeps_newest -q
E   AssertionError: assert None is True
1 failed

# With the fix, related suites
$ pytest tests/test_proxy_anthropic_cache_stability.py tests/test_transforms_content_router.py \
    tests/test_cache_mode_cold_start.py tests/test_cold_start_fast_pass.py \
    tests/test_anthropic_stage_timings.py tests/test_anthropic_compaction_transforms.py \
    tests/test_claude_session_mode_benchmark.py tests/test_proxy_handler_helpers.py -q -n 6
207 passed, 2 warnings in 24.80s

$ uvx ruff@0.16.8 check . && uvx ruff@0.16.8 format --check .
All checks passed!
1705 files already formatted

$ uvx mypy@1.20.2 headroom --ignore-missing-imports --no-site-packages
headroom\fileperms.py:71: error: Module has no attribute "fchmod"  [attr-defined]
Found 1 error in 1 file (checked 554 source files)
# Windows-only: os.fchmod does not exist on Windows. The file is not touched
# by this PR; CI runs mypy on Linux.
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12, this branch vs `upstream/main`
at d5318ac2. Real proxy app (`create_app`) with `mode="cache"`,
`savings_profile="coding"`, the real compression pipeline and a warm
Kompress ONNX model; only the upstream HTTP call is stubbed.
- Exact command / steps: two requests of one conversation. Request 1: a
short spawn prompt. Request 2: the same history plus an assistant reply
plus a new user message, "here is the current task list, pick the first
pending task owned by nobody and claim it", followed by a 60-item
task-list JSON (one item pending). Compare the newest user message the
proxy forwards with the one the client sent.
- Observed result: on `main` the second request goes through the delta
path with `prefix_replay_guaranteed=None`, the router applies
`router:text_block:mixed`, and the user message is forwarded as 2,920
characters instead of 6,563. The JSON is re-encoded into a compact table
(`[60]{blockedBy:json,id:string,...}` followed by CSV-like rows). On
this branch the same request shows `prefix_replay_guaranteed=True`, no
transform on that message, and the message is forwarded byte-for-byte
(6,563 → 6,563). With plain-prose messages `main` also hands the text to
a compressor (`route_counts` shows `cache_miss` + `ratio_too_high`) and
only keeps the original because the result was not small enough; on this
branch the text is never handed to a compressor.
- Not tested: a live Claude Code agent-team run against the real
Anthropic API. Effect on savings for sessions whose follow-up messages
are large pasted documents (those are now kept verbatim on turn 2+, the
same as on turn 1 and in token mode).

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This path is not gated in
`headroom/rollout.py`.
- Minimum rollout channel: Not applicable.
- Stable/default behavior changed: Yes, in cache mode with user-message
compression on (the default coding profile): from the second request on,
the text of the newest user message is no longer compressed. This
matches what the first request and token mode already do.
- Kill switch / disable path: None needed; reverting the one-line change
restores the old behaviour.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this PR. No state or persisted format changes.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- No documentation change: #3619 already documents the "prompt stays
verbatim on replaying callers" behaviour; this applies it to the path it
missed.
- The newest user message is still compressed when it answers a fenced
shell command in the previous assistant turn (a text harness's tool
output). That exception comes from #3619 and is unchanged.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3923"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 10:02:30 -05:00
Ashish Patel 41dab2d099 fix(ccr): verify a scanned marker's hash before advertising it (#2908)
## Description

`CCRToolInjector.scan_for_markers()` decides whether a compression
marker is Headroom's own by *shape* alone — any bracket marker carrying
a 24-hex hash counts, per the generic fallback pattern
(`\[.*?compressed.*?hash=([a-f0-9]{24})\]`). Other context tools emit
exactly that shape. Once a foreign hash is scanned,
`has_compressed_content` flips true and the retrieve tool + "Available
hashes" system instruction get injected for a hash this proxy never
stored — the model calls `headroom_retrieve`, gets a guaranteed miss,
and re-does work it already had. Two wasted turns per adopted foreign
hash.

Closes #2836

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/ccr/tool_injection.py`: added
`CCRToolInjector.verify_ownership()` — filters `detected_hashes` down to
hashes the compression store actually recognizes, via the same
`store.exists()` check the retrieve endpoint itself performs. Added a
small `_HashOwnershipStore` Protocol (structural typing, not a hard
dependency on the concrete `CompressionStore` class) and a
`compression_store` constructor field for dependency injection/testing.
`scan_for_markers()` itself is untouched — kept store-independent (pure
regex) rather than baking the check into the scan loop, since that
approach broke 24 existing tests that correctly test "does this shape
match" in isolation.
- `headroom/proxy/handlers/anthropic.py`,
`headroom/proxy/handlers/openai.py`: call `injector.verify_ownership()`
right after `scan_for_markers()` — the two real per-request call sites.
- `verify_ownership()` is also called inside `process_request()` (the
convenience wrapper `batch.py`'s Google path uses), so that path is
covered without a separate call site edit.
- `tests/test_ccr_tool_injection.py`: new `TestVerifyOwnership` class (6
tests) — the exact issue repro, a real-hash-survives case, mixed
own/foreign hashes, explicit store override, store-exception safety
(must not raise), and no-op-on-empty-hashes.
- `tests/test_proxy_anthropic_cache_stability.py`: 3 pre-existing tests
needed updating for the new (correct) behavior — two `_FakeInjector`
test doubles needed a `verify_ownership()` stub added, and one real
end-to-end test needed a genuine store entry seeded (via
`explicit_hash`) for the hash its hand-typed marker references, instead
of asserting on an unverified shape-only match.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — not run locally, will
confirm via CI
- [x] New tests added for new functionality
- [x] Manual testing performed (see Real Behavior Proof)

### Test Output

```text
$ .venv/Scripts/python -m pytest tests/test_ccr_marker_policy.py tests/test_ccr_tool_always_on.py tests/test_ccr_tool_injection.py tests/test_proxy/test_ccr_frozen_prefix_coupling.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_handlers_batch.py tests/test_proxy_handler_helpers.py -q
151 passed in 15.87s

$ .venv/Scripts/python -m pytest tests/test_proxy_ccr.py tests/test_proxy_openai_responses_stream_ccr.py tests/test_anthropic_ccr_workspace_unbound.py tests/test_compression_store.py tests/test_no_ccr_lossy.py tests/test_proxy_handlers_batch.py -q
121 passed in 20.73s

$ .venv/Scripts/ruff check . && .venv/Scripts/ruff format --check .   # touched files only
All checks passed / already formatted
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.14.5, local venv
- Exact command / steps: ran the issue's exact 3-line repro
(`CCRToolInjector.scan_for_markers()` on the foreign marker text, then
`verify_ownership()`) before and after the fix; separately verified a
genuinely-Headroom-stored hash (via `store.store(...,
explicit_hash=...)`) still survives verification and still drives
injection
- Observed result: before the fix (scan only, no verify step exists yet)
`has_compressed_content` is `True` for the foreign marker — matches the
bug report exactly. After adding `verify_ownership()`: foreign marker →
`detected_hashes == []`, `has_compressed_content is False`; real stored
hash → `detected_hashes == [real_hash]`, `has_compressed_content is
True`.
- Not tested: have not driven this through a live two-context-tool proxy
session (e.g. Headroom alongside another CCR-shaped tool in the same
conversation) — verified at the unit/integration level (the exact repro
plus the real proxy handler call sites via
`test_proxy_anthropic_cache_stability.py`'s end-to-end `TestClient`
tests), not via a live multi-tool session.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A —
internal CCR safety behavior, no user-facing docs reference the
marker-adoption mechanism)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable (release-please
generates this automatically from commit messages)

## Additional Notes

Design note on why `verify_ownership()` is a separate step rather than
baked into `scan_for_markers()`: my first attempt did exactly that and
broke 24 tests across `test_ccr_tool_injection.py`,
`test_ccr_marker_policy.py`, `test_proxy_anthropic_cache_stability.py`,
and `test_proxy_handler_helpers.py` — all of them legitimately testing
"does the regex detect this marker shape" independent of any store
state. Keeping the scan pure and adding an explicit, separately-testable
verification step kept that test surface intact while still closing the
real gap at the three places that actually decide whether to advertise
the retrieve tool.
2026-08-13 11:46:21 -05:00
aaeba0a319 fix(proxy): compress cache-mode cold starts and tag prefix-mismatch passthrough (#2365)
## Description

Since v0.31.0 shipped cache mode as the default (68676daa), users report
the dashboard showing "Optimization ENABLED" while every request
forwards with 0 tokens saved — Before Compression == After Compression
even on 100k-token requests (#2357).

Root cause: in the cache-mode branch of `handle_anthropic_messages`,
when `_extract_cache_stable_delta` returns `None` the handler silently
sets `optimized_messages = messages`. That single fall-through covers
two very different cases:

1. **Session cold start** — no previous turn recorded for the session.
Every fresh proxy session, including a resumed 100k-token Claude Code
transcript, was forwarded raw. Until session identity stabilized (#2193
helped), this could be *every* turn, i.e. compression literally never
ran.
2. **Mid-session prefix mismatch** — the client rewrote history. This
passthrough is intentional (replaying a rewritten transcript risks
per-turn cache busts) but was invisible: no tag, no log, so
`optimize:true` + 0 savings looked like a broken product.

This PR: (1) cold starts now run the same full-message compression as
non-cache modes — there is no provider cache prefix to protect yet, and
the compressed output is recorded as the forwarded messages so later
turns replay it byte-identically through the existing stable-delta path
(append-only cache safety preserved); (2) the mismatch passthrough is
kept but tagged `passthrough_reason=cache_mode_prefix_mismatch` and
logged, mirroring the existing `pre_upstream_backpressure` inline-tag
pattern.

Fixes #2357

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [ ] Refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/anthropic.py` (cache-mode branch only): split
the `delta is None` fall-through. When
`prefix_tracker.get_last_original_messages()` is empty (cold start), run
the full pipeline via `_run_compression_in_executor` (same call shape as
the non-cache branch) and append a `cache_mode:cold_start_full`
transform marker. When a previous turn exists but the delta is `None`
(prefix mismatch), keep the conservative passthrough but set
`tags["passthrough_reason"] = "cache_mode_prefix_mismatch"` and log it.
`CompressionDecision` untouched — it is the frozen pre-pipeline gate;
inline tags are the established mechanism for mid-pipeline passthrough
reasons.
- `tests/test_cache_mode_cold_start.py` (new): handler-level tests using
the same dummy-handler harness as `tests/test_cold_start_fast_pass.py`,
with `mode="cache"`. Cold start → pipeline invoked once, compressed form
forwarded upstream, no passthrough tag. Prefix mismatch → pipeline not
invoked, original bytes forwarded unmodified, outcome tags carry
`passthrough_reason=cache_mode_prefix_mismatch`.

## Testing

- [x] Unit tests pass locally
- [x] Lint/format/type checks pass locally

```
$ python -m pytest tests/test_cache_mode_cold_start.py tests/test_cache_mode_delta_marker.py tests/test_cache_prefix_overlay.py tests/test_cache/test_prefix_tracker.py tests/test_token_headroom_mode.py tests/test_cold_start_fast_pass.py tests/test_anthropic_pre_upstream_backpressure.py tests/test_handler_outcome_tag_invariant.py -q
146 passed

$ ruff check headroom/proxy/handlers/anthropic.py tests/test_cache_mode_cold_start.py
All checks passed!
$ ruff format --check headroom/proxy/handlers/anthropic.py tests/test_cache_mode_cold_start.py
2 files already formatted
$ mypy headroom --ignore-missing-imports
(no error lines)
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, this branch; real
`AnthropicHandlerMixin.handle_anthropic_messages` driven end-to-end
through a FastAPI `Request` with upstream stubbed at `_retry_request`
(harness identical to the existing
`tests/test_cold_start_fast_pass.py`), `mode="cache"`.
- Exact command / steps: `python -m pytest
tests/test_cache_mode_cold_start.py -q`, plus a manual run of the
mismatch scenario with `logging.basicConfig(level=INFO)`.
- Observed result: Cold start: `anthropic_pipeline.apply` invoked once
and the forwarded upstream body contains the compressed tool_result
content (previously: forwarded raw with zero pipeline invocations).
Mismatch: bytes forwarded unmodified, and the proxy log now emits
`[req-...] Compression skipped: reason=cache_mode_prefix_mismatch` with
the same reason present in `RequestOutcome.tags["passthrough_reason"]`
(previously: nothing).
- Not tested: a live multi-turn session against the real Anthropic API
measuring `cache_read_input_tokens` across turns (the byte-identical
replay contract the cold-start path relies on is the same one exercised
by the existing stable-delta tests in
`tests/test_cache_mode_delta_marker.py` and
`tests/test_cache_prefix_overlay.py`, all green).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
2026-08-11 23:55:11 -05:00
Tejas ChopraandTejas Chopra 1a04c957f5 fix(cache): stabilize Anthropic block-growing lineages (#2917)
## Description

Fixes the remaining Anthropic prompt-cache failure in #2671 and the
newly reported parallel-tool-profile variant.

The production failure has three connected parts:

1. `SessionTrackerStore.resolve_tracker` only recognized whole-message
prefixes. A caller that grows or regenerates blocks inside one message
therefore received a fresh tracker every turn, so previous forwarded
state was always empty and breakpoint relocation could never run.
2. `normalize_message_cache_control` always moved the message breakpoint
to the newest block. That is correct for a pure block append, but a
message that rewrites its tail can never match the prior newest-block
write and repeatedly rewrites the full message prefix.
3. Parallel Anthropic sub-calls can carry identical messages but
different tools. Because tools precede messages in the provider cache
key, sharing one frozen-prefix tracker across those calls
cross-contaminates cache state even when message lineage is identical.

This PR deliberately combines the valid parts of #2699 and #2702, fixes
the discriminator between their two shapes, and adds cache-key affinity
for the second pattern reported on #2671. In particular, a pure append
is identified by `stable_prefix_blocks == previous_block_count`;
rewritten-tail relocation is only possible when `stable_prefix_blocks <
previous_block_count`. This prevents a pure append from being pinned to
an old boundary.

Closes #2671.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added one canonical history classifier with distinct exact,
whole-message append, pure block append, rewritten-tail, and diverged
outcomes.
- Kept pure block appends on newest-block breakpoint placement so each
request reads the old prefix and writes only its appended blocks.
- Added block-level replay of the prior forwarded bytes for pure
appends; the whole-message delta path explicitly refuses this shape so
it cannot silently discard appended blocks.
- Kept a rewritten-tail request on its existing tracker and anchored its
breakpoint to the end of the byte-stable leading run.
- Made rewritten-tail matching conservative: one changed message,
unchanged message count, no shrink, at least 8 stable leading blocks
covering at least half of old and new content, and a fixed suffix of at
least 2 blocks.
- Required a unique best rewritten-tail lineage match. Ambiguity creates
a fresh lineage instead of making sibling sub-calls ping-pong one
tracker.
- Added a stable affinity fingerprint over model, deterministically
forwarded tools, tool choice, thinking, and output configuration.
Different provider cache-key profiles cannot share frozen-prefix state.
- Snapshotted previous original/forwarded messages once in the Anthropic
handler and reused that exact state for delta extraction, replay, and
breakpoint placement.
- Added `HEADROOM_STABLE_BOUNDARY_BREAKPOINT=0` as a rollback switch for
rewritten-tail relocation.

The canonical projection is used only for comparison. Replayed content
always comes from the exact previously forwarded bytes or the current
raw/optimized tail; canonicalized data is never reconstructed into an
upstream request.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` on changed files)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_issue_2671_block_growth_cache.py -q
12 passed in 0.10s

$ python -m pytest tests/test_cache -q
253 passed, 3 skipped in 1.94s

$ python -m pytest <Anthropic handler/proxy regression set> -q
134 passed, 1 warning in 9.42s

$ ruff format --check <changed files>
4 files already formatted

$ ruff check <changed files>
All checks passed!

$ git diff --check
# clean
```

The Anthropic regression set covers beta stickiness, CCR injection,
compaction transforms, pre-upstream backpressure, streaming
reconstruction, upstream headers, model sanitization, diagnostics, and
cache stability.

## Real Behavior Proof

- Environment: macOS, Python 3.12.13, real FastAPI Anthropic handler
with a local upstream stub plus a deterministic provider-cache oracle.
- Exact steps: send a cold 35-block aggregate message, then three
requests that preserve a 30-block prefix and fixed two-block suffix
while regenerating a growing middle tail. Resolve the real session
tracker, normalize the real handler body, record the response, and
repeat.
- Observed result: handler breakpoint indices are `34 -> 29 -> 29`; the
cache oracle transitions from a cold 35-block write to establishing the
30-block stable boundary, then produces `(read=30, write=0)` on
subsequent rewritten-tail turns. A separate pure-append sequence
produces `(0,30) -> (30,4) -> (34,4) -> (38,5)`, proving its breakpoint
continues to advance.
- Also observed: identical message histories with different tool schemas
resolve to distinct trackers in the real handler path.
- Not tested: a live Anthropic billing soak, the complete repository
test suite, or mypy. #2702 contains earlier live production measurements
for the rewritten-tail mechanism; this PR adds the pure-append
correction, affinity isolation, and broader regression model.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding documentation updates where applicable
(internal behavior is documented in code; no user-facing surface
changed)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing relevant unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from the Conventional Commit PR title

## Additional Notes

- Consolidates the complementary approaches in #2699 and #2702. Credit
to @axisrow and @nangsontay for the traces, root-cause work, and live
validation that made the two production shapes distinguishable.
- The 20-block minimum for relocation mirrors the provider lookup-window
risk boundary and keeps short ordinary messages on the established
newest-block behavior.
- Disabling stable-boundary relocation does not disable improved lineage
resolution or tool-profile isolation; it restores only the previous
breakpoint placement.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
2026-08-10 22:38:51 -07:00
nangsontay 08fce29b47 fix(proxy): stop toggling headroom_retrieve in the Anthropic tools array (#2672)
## Description

`should_inject_ccr_tool` deferred CCR tool injection whenever
`frozen_message_count > 0`. Because `tools` is the head of Anthropic's
cache key, that dropped a tool which was already inside the
provider-cached prefix and invalidated the whole prefix — in both
directions (`0 → >0` removes it; `>0 → 0` on proxy restart, `/model`
switch, lineage eviction or TTL lapse adds it back).

On three days of local proxy logs the turns that flipped injection state
carried **44.7% of all cache-write tokens at a 52.0% hit rate**, against
98.1% for non-flipping turns. The log signature is `cache_read`
alternating between two values exactly 172 tokens apart — the 464-byte
tool definition.

This deletes the gate and calls `apply_session_sticky_ccr_tool`
directly, which is **what `openai.py` already does** — the two handlers
now have the same shape. Net −61 production lines, no new state, no new
config flag.

Fixes defect 1 of #2671.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement

## Changes Made

- `headroom/proxy/ccr_marker_policy.py` — deleted the
`should_inject_ccr_tool` gate; `apply_session_sticky_ccr_tool` is now
the single decision point.
- `headroom/proxy/handlers/anthropic.py` — calls
`apply_session_sticky_ccr_tool` directly, matching `openai.py`.
- `headroom/proxy/helpers.py` — dropped the now-unused gate plumbing.
- `tests/test_proxy_anthropic_cache_stability.py` — new test asserting
the forwarded `tools` array is byte-identical across a `frozen 0 → >0`
transition.
- `tests/test_ccr_marker_policy.py` — removed the three unit tests that
pinned the deleted decision (they encoded the defect).
- `tests/test_proxy/test_ccr_frozen_prefix_coupling.py` — same
unredeemable-marker intent, re-pinned at the sticky helper.
- `tests/test_proxy/test_anthropic_ccr_deferred_injection.py` — autouse
reset fixture for the process-global `SessionCcrTracker` (separate
commit).
- Formatting-only follow-up commit applying `ruff format` (pinned
0.15.17) to the two test files above.

### Why deleting the gate is safe

`apply_session_sticky_ccr_tool` already holds the correct rule. Its four
branches, in order:

| # | condition | action |
|---|---|---|
| 1 | tool already in the incoming tool list (client/MCP pre-registered)
| skip; the client's bytes win |
| 2 | `session_id is None` (WS / pre-session) | per-turn flag drives it
verbatim |
| 3 | session has done CCR | always inject the recorded golden bytes |
| 4 | fresh session, no compression this turn | **skip** |

Branch 4 is the safety property: a session that has never compressed
still gets no tool, so removing the gate cannot start injecting into
non-CCR conversations. Branch 3 is what the gate was starving.
`has_new_ccr_markers` still gates first-time injection, so markers
replayed from the previously-forwarded prefix cannot trigger one.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

The three deleted unit tests encoded the defect. Coverage moves to the
property that actually matters and was previously untested: **the
forwarded `tools` array must be byte-identical across a `frozen 0 → >0`
transition.** That test asserts on the forwarded request body rather
than on a policy function's return value; unit-testing the old policy in
isolation is exactly what let a wrong-but-self-consistent decision pass.
Verified failing on `upstream/main` with an assertion on the missing
tool (not an `ImportError`, so it fails for the right reason).

Full suite: same pre-existing unrelated failures as `upstream/main`,
**zero new** (verified by running the whole suite on both revisions and
diffing the failure sets).

### Test Output

```text
$ uv run pytest tests/test_ccr_marker_policy.py \
    tests/test_proxy/test_anthropic_ccr_deferred_injection.py \
    tests/test_proxy/test_ccr_frozen_prefix_coupling.py \
    tests/test_proxy_anthropic_cache_stability.py -q
collected 48 items

tests/test_ccr_marker_policy.py .....                                    [ 10%]
tests/test_proxy/test_anthropic_ccr_deferred_injection.py .............. [ 39%]
.                                                                        [ 41%]
tests/test_proxy/test_ccr_frozen_prefix_coupling.py ..                   [ 45%]
tests/test_proxy_anthropic_cache_stability.py .......................... [100%]

======================= 48 passed, 2 warnings in 13.59s ========================

$ ruff check .
All checks passed!

$ ruff format --check .
1349 files already formatted
```

## Real Behavior Proof

- Environment: local macOS proxy serving live Claude Code traffic to the
Anthropic API; baseline = 3 days of proxy logs on `upstream/main`, after
= 5.5 hours with this change live.
- Exact command / steps: ran the proxy with this branch built in, drove
normal Claude Code sessions through it (including `/model` switches and
proxy restarts, the two events that used to flip injection state), then
parsed 235 real turns from the proxy logs with the same parser used for
the baseline in #2671.
- Observed result: flip turns fell from 177 (44.7% of all cache write)
to 2 (1.7%); steady-state write share 1.192% → 0.867%; aggregate hit
rate 86.75% → 89.21%; main conversation warm hit rate 98.1% → 97.70%
(n=149). The 2 remaining "flips" have `cache_read == 0` — cold starts
that the bucketing counts as a state change, not real flips.

| metric | baseline | after |
|---|---|---|
| flip turns | 177, carrying 44.7% of all cache write | **2**, carrying
**1.7%** |
| main conv, warm | 98.1% | **97.70%** (n=149) |
| steady-state write share | 1.192% | **0.867%** |
| aggregate | 86.75% | **89.21%** |

- Not tested: `mypy headroom` was not run locally for this body; the
OpenAI handler path (unchanged by this PR); tracker state loss
mid-session (see note below); and defect 2 of #2671 (the sub-call
breakpoint), which is untouched and is now 54.9% of remaining cache
write — that is why aggregate stays just under 90%.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

**Pre-existing and unchanged here:** if the tracker loses state
mid-session while the transcript still carries markers, branch 4 returns
no tool and those markers are unredeemable. `upstream/main` has no
recovery for that; this PR neither creates nor fixes it. See my comment
on #2500, which adds a recovery path for the related dangling-reference
case.

**N/A checklist items:** no documentation changes — this removes an
internal policy function with no user-facing surface. `mypy headroom`
left unchecked because it was not run for this body; CI covers it.

**Merge-order conflict with #2500 (please read before landing either):**
this PR *deletes* `should_inject_ccr_tool`, which is the exact function
#2500 extends with `transcript_requires_tool`. Whichever lands second
needs a semantic rebase, not just a textual one — git will not flag it.
If this PR lands first, #2500's recovery path should re-target
`apply_session_sticky_ccr_tool` (the sticky helper now owns the decision
alone) or the handler call site in `handlers/anthropic.py`. If #2500
lands first, the gate deletion here still applies but the
`transcript_requires_tool` override needs to move with it. Happy to do
the rebase either way — say which order you prefer.
2026-08-03 16:18:11 -07:00
Tejas Chopra 7c2f0ea079 feat(cache): provider-agnostic cache-mode delta + cc-agnostic prefix comparison (#1868)
## Description

<!-- Briefly explain the change and why it is needed. -->

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- 

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
# Paste relevant command output or artifact links here
```

## Real Behavior Proof

- Environment:
- Exact command / steps:
- Observed result:
- Not tested:

## Review Readiness

- [ ] I have performed a self-review
- [ ] This PR is ready for human review

## Checklist

- [ ] My code follows the project's style guidelines
- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

Add screenshots to help explain your changes.

## Additional Notes

<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->
2026-07-08 13:29:35 -07:00
Vinay Gupta a9322477e3 fix: preserve anthropic passthrough tool order (#1427)
## Description

Preserves Anthropic `tools` order when Headroom is forwarding a
passthrough/no-optimize request. This fixes a Claude Code style
`tool_result` continuation failure against stricter Anthropic-compatible
upstreams that treat the client's original tool ordering as part of the
conversation state.

Closes #1417

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Preserve client-provided Anthropic `tools` order when `optimize=False`
or the request is explicitly in Headroom passthrough/bypass mode.
- Keep deterministic tool sorting for optimized requests where Headroom
may rewrite the body for cache stability.
- Avoid sorting batch-request tools before the no-optimize passthrough
branch.
- Add regression coverage for the Anthropic HTTP path to prove
no-optimize forwarding keeps `Read`, then `Bash` tool order.
- Update existing cache-stability and byte-faithful forwarding tests so
no-optimize/passthrough expects preserved client order while optimized
mode still proves deterministic sorting.

## Testing

- [x] Focused unit tests pass (`pytest` on touched proxy test files)
- [x] Linting passes (`ruff check` and `ruff format --check` on touched
files)
- [x] Type checking passes (`mypy headroom`)
- [x] New regression tests added
- [x] Manual testing performed

### Test Output

```text
$ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with pytest --with pytest-asyncio --with anyio --with 'httpx[http2]' --with fastapi --with pydantic --with tiktoken --with click --with rich --with opentelemetry-api --with opentelemetry-sdk --with zstandard --with openai --with mcp --with uvicorn pytest tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py -q
============================= test session starts ==============================
platform darwin -- Python 3.13.11, pytest-9.1.1, pluggy-1.6.0
rootdir: /Users/vinaygupta/Desktop/git/headroom-fix-anthropic-tool-order
configfile: pyproject.toml
plugins: anyio-4.14.1, asyncio-1.4.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 87 items

tests/test_proxy_handler_helpers.py ..........................           [ 29%]
tests/test_anthropic_stage_timings.py ....                               [ 34%]
tests/test_proxy_anthropic_cache_stability.py .........................  [ 63%]
tests/test_proxy_byte_faithful_forwarding.py ........................... [ 94%]
.....                                                                    [100%]

=============================== warnings summary ===============================
.../fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.

======================== 87 passed, 1 warning in 5.13s =========================

$ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py
5 files already formatted

$ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py
All checks passed!
```

## Real Behavior Proof

- Environment: macOS, Python 3.13.11, local fake Anthropic-compatible
upstream, local Headroom proxy launched with `--no-optimize --no-cache
--no-rate-limit --stateless`.
- Exact command / steps: ran a local reproduction harness that starts a
fake `/v1/messages` upstream and Headroom proxy, then sends a Claude
Code style two-turn flow: first assistant `Bash` `tool_use`, then user
`tool_result`.
- Observed result: after this patch, both direct and proxied flows
returned `200` for `first_tool_use` and `second_tool_result`. The fake
upstream log showed the proxied `tools` array remained `["Read",
"Bash"]` on both turns.

```text
DIRECT
  first_tool_use: 200
  second_tool_result: 200

PROXIED
  first_tool_use: 200
  second_tool_result: 200

UPSTREAM REQUEST LOG
  proxied first turn tools: ["Read", "Bash"]
  proxied tool_result turn tools: ["Read", "Bash"]
```

- Not tested: full `pytest`, full-repo `ruff check .`, `mypy headroom`,
or a live third-party Anthropic-compatible provider.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- This PR intentionally does not add documentation because it fixes
passthrough behavior rather than introducing a new user-facing option.
- The code-comment checklist item is left unchecked because the change
is covered by a small helper docstring and regression tests; no extra
inline comments seemed necessary.
- `CHANGELOG.md` is left unchanged because this is a narrowly scoped bug
fix.
- Local pytest collection for these proxy tests required a local
`headroom._core` extension symlink, which was removed before committing.
2026-06-30 08:38:51 -05:00
weijie_chen b4682d6f91 fix(proxy): honor force_kompress routing profile (#996)
## Description

Honor the proxy savings profile's `force_kompress` setting all the way
through the Anthropic proxy path.

`HEADROOM_SAVINGS_PROFILE=agent-90` already resolves to
`force_kompress=True`, but `ContentRouter` still paid for the full
auto-detection path before selecting Kompress. On long Claude Code /
tool-output conversations this can hang inside the detection/router path
before any `Transform content_router` line is emitted. This change makes
the forced-Kompress path skip unused strategy detection during
compression, while still preserving recent-code protection via the
lightweight regex detector.

This also passes `proxy_pipeline_kwargs(self.config)` through Anthropic
batch requests so batch traffic receives the same savings-profile knobs
as normal Anthropic messages.

Refs #946

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Skip `is_mixed_content()` / `_detect_content()` when runtime
`force_kompress` is set and route directly to
`CompressionStrategy.KOMPRESS`.
- Keep forced-Kompress recent-code protection, but use
`_regex_detect_content_type()` instead of the full router detection
chain.
- Read `_runtime_force_kompress` defensively in `ContentRouter.apply()`
so regular `ContentRouter()` instances keep the normal content-detection
path.
- Pass proxy savings-profile kwargs into Anthropic batch compression.
- Add regression tests for forced-Kompress routing, normal routing,
recent-code protection, and Anthropic batch profile propagation.
- Update `CHANGELOG.md`.

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/transforms/content_router.py tests/test_transforms_content_router.py
All checks passed!

$ ruff format --check headroom/transforms/content_router.py tests/test_transforms_content_router.py
2 files already formatted

$ pytest tests/test_transforms_content_router.py::test_force_kompress_bypasses_content_detection \
    tests/test_transforms_content_router.py::test_normal_compress_path_still_uses_content_detection \
    tests/test_transforms_content_router.py::test_force_kompress_apply_uses_lightweight_detection \
    tests/test_transforms_content_router.py::test_force_kompress_apply_lightweight_detection_protects_recent_code \
    tests/test_proxy_anthropic_cache_stability.py::test_batch_optimization_passes_savings_profile_kwargs \
    tests/test_bundled_tools_savings.py -q
============================= test session starts =============================
platform win32 -- Python 3.13.1, pytest-9.0.3, pluggy-1.6.0
rootdir: E:\work\code\third-party\headroom
configfile: pyproject.toml
plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0, timeout-2.4.0
collected 11 items

tests\test_transforms_content_router.py ....                             [ 36%]
tests\test_proxy_anthropic_cache_stability.py .                          [ 45%]
tests\test_bundled_tools_savings.py ....ss                               [100%]

======================== 9 passed, 2 skipped in 9.77s =========================
```

Full-suite attempt status on Windows / Python 3.13 after installing
missing local test dependencies and bundled tools (`fastembed`,
`socksio`, `pytest-timeout`, `difft`, `scc`, cached HF models with
offline env vars):

```text
tests/test_adapter_hooks.py: 29 passed, 2 failed
  - sqlite:///C:\... and jsonl:///C:\... URLs are parsed into invalid \C:\... paths on Windows.

tests/test_cache/test_client_integration.py: 16 failed
  - Same Windows URL path parsing issue.

tests/test_cli/test_wrap_helpers.py: 29 passed, then KeyboardInterrupt during read/cleanup.

tests/test_memory tests/test_storage:
  - Collection/run receives KeyboardInterrupt in this Windows environment.
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.1, `headroom-ai` v0.25.0,
Anthropic proxy on `127.0.0.1:8787`, Kompress ONNX backend,
`HEADROOM_SAVINGS_PROFILE=agent-90`,
`HEADROOM_COMPRESS_USER_MESSAGES=1`, `HEADROOM_MIN_TOKENS=120`.
- Exact command / steps: started the proxy with the local launcher, sent
a long `/v1/messages` request with a fake upstream token, and inspected
`/livez`, `/stats?include_config=true`, and
`~/.headroom/logs/proxy.log`.
- Observed result: request returned promptly with the expected upstream
auth failure after local compression, and logs showed the compression
ran before forwarding:

```text
/livez healthy
/v1/messages completed in ~3005ms with expected upstream 401
Transform content_router: 1900 -> 193 tokens (saved 1707) [1328.4ms]
Pipeline complete: 1907 -> 200 tokens (saved 1707, 89.5% reduction)
UPSTREAM_ERROR ... compressed=yes transforms=['router:kompress:0.06'] original_tokens=1886 optimized_tokens=119
PERF ... tok_before=1886 tok_after=119 tok_saved=1767 transforms=router:kompress:0.06
/stats tokens.saved = 1767
/stats compressions_by_strategy = {"kompress": 1}
```

- Not tested: full upstream CI matrix, full `uv run pytest`, full `ruff
check .`, `mypy headroom`, real Anthropic success response with a valid
upstream token, and Anthropic batch against the live upstream. The
Anthropic batch change is covered by a local handler regression test.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

This PR is ready for human review. The patch is scoped to the
forced-Kompress profile path and does not change the default
auto-routing behavior when `force_kompress` is false.

The latest `PR Governance / template` check passes after the readiness
checkbox update. A later `PR Governance / label` run currently fails
while trying to execute `.github/scripts/pr-health-labels.py` from the
base checkout; that file is missing on the checked-out base ref, so this
appears to be a governance workflow issue rather than a
PR-template/content failure in this branch.
2026-06-22 18:44:32 -05:00
chopratejas 2fb905fdb0 fix: integrate B6+B7 — fix cross-test contamination + injector mock parity
Two follow-ups surfaced when B6 and B7 were merged onto the megamerge
branch and the full suite ran:

1. tests/test_proxy_anthropic_cache_stability.py
   PR-B7 added `injector.scan_for_markers(optimized_messages)` to the
   Anthropic handler so the always-on tool-registration logic can see
   detected hashes for the current request. The two pre-existing
   `_FakeInjector` mocks (`test_ccr_system_instruction_injection_disabled_*`
   and `test_ccr_tool_injection_disabled_*`) didn't implement that method.
   Added a no-op `scan_for_markers` returning [] to both mocks — matches
   the real injector's contract for the not-yet-compressed request shape
   these tests exercise.

2. tests/test_memory_tool_mode.py::test_tool_mode_skip_emits_structured_log
   The B6 caplog assertion passed in isolation but failed in the full
   suite. Root cause: when an earlier test triggers proxy startup,
   `_setup_file_logging` flips `headroom.propagate=False` and attaches a
   RotatingFileHandler to the headroom logger. caplog captures via
   propagation to root, so log records stop reaching it. The conftest
   autouse fixture that resets `propagate=True` before every test gets
   shadowed by fixture-ordering edge cases.

   Principled fix: attach `caplog.handler` directly to
   `headroom.proxy.memory_handler` for the duration of the test so the
   capture is independent of propagation state. Restore the original
   level + remove the handler in `finally` to keep the test hermetic.

Both B6 and B7 cherry-picks themselves are unmodified. This commit only
adjusts test harness code so the pre-existing mocks/capture stay
consistent with the new code paths.
2026-05-02 17:05:58 -07:00
chopratejas f0dcc02775 fix: A3 — byte-faithful Python forwarders; serialize canonical only when mutated
Eliminates P0-2 universally. Every Python forwarder (server.py
`_retry_request`, handlers/streaming.py `_stream_response`,
handlers/openai.py `_ws_http_fallback`, handlers/batch.py `_batch_passthrough`
+ batch-create + Google batch passthrough, handlers/anthropic.py CCR
continuation + batch endpoint) now switches from `httpx ... json=body` to
`httpx ... content=raw_bytes`. The default httpx JSON encoder was
re-serializing every request with `, `/`: ` separators and `\\uXXXX` ASCII
escapes — collapsing Anthropic prompt-cache hit-rate.

Forwarder strategy:
  - unmutated body → forward `await request.body()` verbatim;
  - mutated body  → re-serialize once via the new
    `serialize_body_canonical(body) -> bytes` helper (compact separators,
    `ensure_ascii=False`, dict insertion order preserved).

`HEADROOM_PROXY_PYTHON_FORWARDER_MODE` env var configures the mode:
  - `byte_faithful` (default) — the new behavior;
  - `legacy_json_kwarg` — explicit operator opt-in for emergency rollback.
Documented in `docs/content/docs/configuration.mdx`. NOT a fallback —
unknown values raise loudly per build constraint #4.

`BodyMutationTracker` accompanies each request through the handler so
transform sites mark the tracker (`memory_injection`,
`image_compression`, `compression_*`, `batch_compression`,
`ccr_continuation`, etc.). At forwarder dispatch we additionally compare
the final body dict against the parsed original bytes as a structural
safety net — any silent mutation we missed still triggers canonical
re-serialization.

A2 follow-up: `handlers/openai.py:534-540` (Chat Completions memory
injection) was prepending a system message; replaced with
`append_text_to_latest_user_chat_message`, the OpenAI Chat Completions
analog of `_append_context_to_latest_non_frozen_user_turn`. The cache
hot zone (system messages) is now sacrosanct on /v1/chat/completions
too. Honors `HEADROOM_MEMORY_INJECTION_MODE=disabled`.

Structured logging: every forwarder emits an `event=outbound_request`
log line with `forwarder`, `path`, `body_bytes`, `body_mutated`,
`mutation_reasons`, `source` (passthrough|canonical|legacy),
`request_id`. Never logs Authorization or full body.

`_read_request_json` factored to share `_read_request_body_bytes` with
new `read_request_json_with_bytes` so the anthropic handler can capture
both the parsed dict and the original (decompressed) bytes.

Tests:
  - `tests/test_proxy_byte_faithful_forwarding.py` (28 tests):
    SHA-256 byte-equality on /v1/messages and streaming, unicode
    preservation, numeric precision, mutation-tracker invariants,
    canonical-serializer properties, legacy-mode rollback, OpenAI
    Chat memory routing.
  - Existing test mocks updated to accept the new `**kwargs` on
    `_retry_request` (no behavior change).
  - `tests/test_proxy_handlers_batch.py` updated to read the captured
    `content=` bytes (formerly `json=`).
  - One A2 test corrected (`test_anthropic_tool_sort_and_context_append_helpers`)
    to match the live-zone-tail semantics introduced by A2.

Constraints satisfied: configurable env var; no new regex / hardcodes;
no silent fallback (`legacy_json_kwarg` is operator opt-in);
performant (`prepare_outbound_body_bytes` is O(1) for passthrough);
elegant single-responsibility helpers; structured tracing logs.
2026-05-02 09:02:10 -07:00
chopratejas 521fbbeabd style: apply ruff format to test_proxy_anthropic_cache_stability lambdas 2026-05-01 13:52:46 -07:00
chopratejas 35eaf8de7f fix(proxy): remove content-keyed TTL walker that conflated content with positional cache (#327)
The Anthropic token-mode handler walked past prefix_tracker.frozen_message_count
whenever an upcoming tool_result's content-hash matched comp_cache._stable_hashes
or should_defer_compression returned True. That conflated content equality with
positional cache membership.

Anthropic's prefix cache is POSITIONAL: bytes 0..K cached, anything past K is
fresh. _stable_hashes is content-keyed and grows unbounded. In long Claude Code
sessions where tool_result content rhymes across turns (repeated system prompts,
repeated file reads, repeated tool descriptions), the walker advanced
frozen_message_count to len(messages) on every turn and the pipeline produced
transforms_applied=[] on 73% of requests in user SvenMeyer's reported session
(headroom-stats-2026-05-01.json: 74 of 101 eligible requests "prefix_frozen") —
even after the prior fix in 44944fb. The 15 requests that did compress averaged
21%, proving compression itself works when reached.

Fix: delete the walker. The freeze boundary is now

    frozen_message_count = min(
        prefix_tracker.frozen_message_count,    # positional ground truth
        comp_cache.compute_frozen_count(messages),  # local cache lower bound
    )

compute_frozen_count's use of _stable_hashes can only LOWER the freeze via the
min clamp, never raise it past prefix_tracker's value. For any position in the
gap [compute_frozen_count, prefix_tracker.frozen_count], recompressing produces
byte-stable output (compression is deterministic on input content), so
Anthropic's prefix cache stays valid.

Cross-handler verification:
* OpenAI handler (proxy/handlers/openai.py:358-382) does not have this walker
  — uses only compute_frozen_count. Codex routes through OpenAI handler. Both
  unaffected.
* Streaming and non-streaming both invoke anthropic_pipeline.apply() before the
  upstream call. One fix covers both paths.
* Cache mode (is_cache_mode) takes the _extract_cache_stable_delta path and is
  independent of the walker. Unaffected.

Tests: six new regression tests lock down the post-fix invariants — clamp to
min(prefix_tracker, compute_frozen_count); fresh tool_result whose hash matches
old _stable_hashes entry is not frozen; frozen prefix byte-stable across the
pipeline; 10-turn session produces non-empty compression suffix every turn;
streaming and non-streaming compute identical frozen_message_count; OpenAI
handler never calls the walker functions. Plus scripts/smoke_issue_327.py
(gated by RUN_LIVE_API=1) drives a 10-turn conversation against
api.anthropic.com in both shapes (string + list-of-blocks) and both modes
(streaming + non-streaming).

ci-precheck clean. 191 tests pass.

Follow-ups (separate PRs):
* Fix _cache perpetually empty (anthropic.py result.messages != working_messages
  comparison rarely fires in token mode).
* Cap _stable_hashes with bounded LRU + 1h TTL — hygiene only after the freeze
  gate is removed.
* List-shape tool_result content gates at content_router.py:1975 and
  intelligent_context.py:657 (cluster A from the audit).
2026-05-01 12:04:28 -07:00
SwiftWing21andClaude Sonnet 4.6 429ae0095b fix(proxy): guard CCR tool injection against frozen prefix to preserve cache
The Anthropic handler's CCR injector path applied a frozen_message_count
guard to system instruction injection but not to tool injection. When
Kompress fired for the first time in a session, the tools array was
mutated unconditionally, invalidating Anthropic's prefix cache and
dropping cache_read_input_tokens to zero on calls where ~48K tokens
were previously being cached.

Mirror the existing inject_system_instructions guard for inject_tool:
when frozen_message_count > 0, defer tool injection so the warm prefix
stays intact.

Adds test_ccr_tool_injection_disabled_when_prefix_frozen as a direct
companion to the existing test_ccr_system_instruction_injection_
disabled_when_prefix_frozen.

Fixes #294

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 10:01:11 -07:00
chopratejas a9aa66e4c6 Fix 19 test failures: missing security attr, nosec inside f-strings, stale test mock
- Initialize self.security = None on HeadroomProxy (enterprise plugin hook)
- Move '# nosec B608' comments outside f-strings in sqlite.py and fts5.py
  (SQLite doesn't understand # as comment, causing OperationalError)
- Add mark_stable_from_messages to _FakeCompressionCache test mocks
- Update token mode freeze assertion to match cache-aware behavior
2026-04-07 18:30:29 -07:00
JerrettDavisandCopilot 6f701033a1 test: align anthropic cache stability fixtures
Sync the Anthropic cache stability test double with the prefix tracker contract used by the handler.

Format the benchmark scripts that were failing ruff format --check in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-06 23:26:51 -05:00
JerrettDavis 5ffa77f4a3 Harden anthropic cache-mode replay stability 2026-04-05 16:11:39 -05:00
JerrettDavis 96fd3d9652 Add Anthropic cache-mode delta replay 2026-04-04 22:23:10 -05:00
JerrettDavis f2a32f9721 Harden Anthropic cache mode stability 2026-04-04 22:06:52 -05:00
JerrettDavis 83b730f2b9 Fix CI lint/format failures after proxy mode hardening 2026-04-04 14:42:32 -05:00
JerrettDavis 54419ad8b8 Rebrand proxy modes to token/cache and harden cache-mode stability 2026-04-04 14:32:07 -05:00
JerrettDavis 7d02829f02 Harden Anthropic prefix cache stability across proxy and batch paths 2026-04-04 13:45:37 -05:00