Files
headroom/tests/test_proxy/test_anthropic_buffered_timeout.py
Mansurshakh JaparovandJD Davis 0f3b0d8a21 fix(memory): resolve pi cwd before injecting project tools (#4618)
## Description

Closes #4442.

pi's system `<cwd>` block now resolves the memory project. If project
scope is still unresolved with the `empty` fallback, the Anthropic
handler no longer advertises unusable memory tools, including tools
recorded earlier in the session.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Parse pi cwd only in system text. Keep main's line parser, header/CLI
precedence and limited user-message fallback.
- Reuse `is_project_unresolved` for context and sticky-tool replay.
Client tools and global/user storage behavior stay intact.
- Add initialized-handler scope regressions, replace
source-inspection/forwarding-only checks, and document project scope in
the published memory page.

## Testing

- [x] Unit tests pass (`pytest`) — related suite and the normal critical
gate, not the entire Python suite locally.
- [x] Linting passes (`ruff check .`) — hosted lint passed on 5ace63e2;
local filesystem limitation below.
- [x] Type checking passes (`mypy headroom`) — normal full hook with its
existing `--ignore-missing-imports` setting.
- [x] New tests added for the scope regression
- [x] Manual testing performed

### Test Output

```text
# 16 related test files
329 passed, 12 skipped, 1 warning in 43.07s

# ordinary pre-push: make ci-precheck
178 passed, 5 skipped, 1 warning in 10.87s
✅ ci-precheck PASSED — safe to push.

uv run ruff format --check .
1860 files already formatted
```

Normal commit hooks (Ruff, formatter, full mypy, commitlint) and the
complete pre-push path passed: Rust fmt/clippy/workspace tests, native
build/import verification, critical Python tests and commitlint. The
existing Starlette test-client warning was not filtered.

Local whole-tree Ruff reports EXE002 on five **unchanged** OAuth plugin
files: my `fuseblk` checkout exposes them as executable despite Git
modes `100644`; `chmod` returns zero without changing the mode. The
uncached check agrees. I did not change those files, add shebangs or
ignore the rule. Hosted Ruff, formatting and full mypy passed on
5ace63e2; the local filesystem limitation remains.

The first full CI caught a missing scope predicate on the existing
lifecycle test double. Fixed in
[5ace63e2](https://github.com/headroomlabs-ai/headroom/commit/5ace63e2cbd2295f9594d1e0442a679aad99979c);
its complete file passes (`5 passed, 1 warning in 13.01s`). This
follow-up changes no production code or assertions and adds no runtime
fallback. The behavior proof below ran on 7aeb83cf with the same
production source; that qualification is historical; current-head checks
are recorded below.

### Author follow-up: 80453a03

Moved the project-scope guidance out of the unpublished wiki and into
`docs/content/docs/memory.mdx`; this commit changes no production code
or tests. Viewed the actual Next.js `/docs/memory#proxy-project-scope`
page and its table-of-contents entry; no browser errors.

The existing router/scope tests pass: 65 passed, with four SQLite
ResourceWarnings. All seven pi-parser lines and all six branch edges are
covered, so I did not add duplicate tests for the old Codecov
percentage.

Normal commit and pre-push hooks passed, including Rust checks, native
build/import, 180 critical Python tests (5 skipped, 1
tokenizer-approximation warning), and commitlint. My first push timed
out with the wrong local ONNX path; the next found missing pip in the
fresh uv environment. Both setup issues were fixed without bypassing
checks. Current-head hosted CI is not yet qualified.

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, freshly built native core,
real TCP/Uvicorn proxy and SQLite/ONNX memory backend. Only the upstream
is a local Anthropic-shaped HTTP server; no provider charges.
- Exact command / steps: after `uv sync --locked --extra dev --extra
proxy --extra memory` and `make verify-rust-core`, run this from the
checkout. It prints the actual loaded native module/hash and leaves a
disposable `.headroom-4442-*` directory for inspecting SQLite. Both HTTP
listeners shut down before exit.

<details>
<summary>Standalone HTTP/SQLite reproduction</summary>

```bash
./.venv/bin/python - <<'PY'
import hashlib,http.client,json,os,socket,tempfile,threading,time
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from pathlib import Path
import uvicorn
import headroom._core as core
assert Path(core.__file__).resolve().parent == Path.cwd().resolve()/"headroom"
native_sha=hashlib.sha256(Path(core.__file__).read_bytes()).hexdigest()
from headroom.memory.storage_router import ProjectResolver,RequestContext,extract_system_prompt
from headroom.proxy.server import ProxyConfig,create_app
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES
from headroom.proxy.helpers import get_session_tool_tracker
root=Path(tempfile.mkdtemp(prefix=".headroom-4442-",dir=Path.cwd()))
project_a=root/"pi%2Fproject-a";project_b=root/"project-b"
project_a.mkdir(exist_ok=True);project_b.mkdir(exist_ok=True)
os.environ["HEADROOM_WORKSPACE_DIR"]=str(root/"workspace")
os.environ["HEADROOM_BEACON"]="off"
os.environ["HEADROOM_TELEMETRY"]="off"
os.environ["NO_PROXY"]="127.0.0.1,localhost"
forwarded=[]
class Gateway(BaseHTTPRequestHandler):
    def do_POST(self):
        body=json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        forwarded.append({"path":self.path,"body":body})
        payload=json.dumps({"id":"msg_local_scope","type":"message","role":"assistant","model":"claude-haiku-4-5","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","stop_sequence":None,"usage":{"input_tokens":10,"output_tokens":1}}).encode()
        self.send_response(200);self.send_header("Content-Type","application/json");self.send_header("Content-Length",str(len(payload)));self.end_headers();self.wfile.write(payload)
    def log_message(self,*args):pass
upstream=ThreadingHTTPServer(("127.0.0.1",0),Gateway)
ut=threading.Thread(target=upstream.serve_forever,daemon=True);ut.start()
config=ProxyConfig(mode="cache",anthropic_api_url=f"http://127.0.0.1:{upstream.server_port}",memory_enabled=True,memory_db_path=str(root/"legacy.db"),traffic_learning_enabled=True,subscription_tracking_enabled=False,cost_tracking_enabled=False,cache_enabled=False,rate_limit_enabled=False,log_requests=False,image_optimize=False,ccr_inject_tool=False,ccr_handle_responses=False,ccr_context_tracking=False,anthropic_pre_upstream_memory_context_timeout_seconds=30.0)
app=create_app(config)
sock=socket.socket();sock.bind(("127.0.0.1",0));sock.listen(128)
port=sock.getsockname()[1]
server=uvicorn.Server(uvicorn.Config(app,log_level="warning",lifespan="on",timeout_graceful_shutdown=3))
st=threading.Thread(target=server.run,kwargs={"sockets":[sock]},daemon=True);st.start()
try:
    deadline=time.monotonic()+180
    while not server.started:
        if not st.is_alive() or time.monotonic()>deadline:raise RuntimeError("real proxy startup failed")
        time.sleep(.02)
    session="headroom4442-native-project-scope"
    common={"Content-Type":"application/json","x-api-key":"local-unbilled-key","anthropic-version":"2023-06-01","x-headroom-user-id":"native-scope-user","x-headroom-session-id":session}
    client_tools=[{"name":"get_weather","description":"Weather","input_schema":{"type":"object","properties":{}}}]
    rows=[]
    cases=[("fresh_unresolved","You are helpful.",{},False),("pi_cwd",f"<cwd>\n{project_a}\n</cwd>",{},True),("warm_unresolved","You are helpful.",{},False),("pi_cwd_restored",f"<cwd>\n{project_a}\n</cwd>",{},True),("explicit_header_wins",f"<cwd>\n{project_b}\n</cwd>",{"x-headroom-project-id":"explicit-project"},True),("ordinary_user_cwd_ignored","You are helpful.",{},False)]
    for label,system,extra,expected in cases:
        body={"model":"claude-haiku-4-5","max_tokens":16,"system":system,"messages":[{"role":"user","content":f"cwd:\n<cwd>\n{project_b}\n</cwd>" if label=="ordinary_user_cwd_ignored" else "hello scope"}],"tools":client_tools}
        before=len(forwarded)
        conn=http.client.HTTPConnection("127.0.0.1",port,timeout=90)
        conn.request("POST","/v1/messages",body=json.dumps(body).encode(),headers={**common,**extra})
        response=conn.getresponse();payload=response.read();conn.close()
        assert response.status==200,(label,response.status,payload.decode())
        assert len(forwarded)==before+1,(label,"unexpected upstream request count")
        actual=forwarded[-1]["body"]
        names={tool.get("name") for tool in actual.get("tools",[])}
        assert names==({"get_weather"}|MEMORY_TOOL_NAMES if expected else {"get_weather"}),(label,names)
        scope=ProjectResolver().resolve(RequestContext(headers={**common,**extra},system_prompt=extract_system_prompt(body),base_user_id="native-scope-user"))
        rows.append({"case":label,"status":response.status,"memoryTools":sorted(names & MEMORY_TOOL_NAMES),"project":scope,"gatewayPath":forwarded[-1]["path"]})
    assert forwarded[1]["body"]["tools"]==forwarded[3]["body"]["tools"],"restored definitions changed"
    tracker=get_session_tool_tracker()
    assert tracker.should_inject("anthropic",session),"probe did not exercise an actual recorded sticky session"
    from urllib.parse import quote
    key_a=ProjectResolver().resolve(RequestContext(headers={"x-headroom-cwd":quote(str(project_a),safe="/")},system_prompt="",base_user_id="native-scope-user"))[0]
    assert rows[1]["project"][0]==key_a
    assert rows[4]["project"][1]=="explicit-project"
    router=app.state.proxy.memory_handler._router
    assert router is not None,"real memory backend never initialized"
    opened=router.open_backends()
    project_db=root/"memories"/"projects"/key_a/"memory.db"
    assert project_db in opened and project_db.is_file(),("real pi project database not opened",[str(x) for x in opened])
    print(json.dumps({"mode":config.mode,"realHttpListener":f"http://127.0.0.1:{port}","localGateway":f"http://127.0.0.1:{upstream.server_port}","rows":rows,"realStickyGoldenSession":True,"realPiProjectDatabase":str(project_db),"openedDatabases":[str(x) for x in opened],"nativeModule":core.__file__,"nativeSha256":native_sha,"noLivePiOrPaidProvider":True},indent=2))
finally:
    server.should_exit=True;st.join(20)
    upstream.shutdown();upstream.server_close();ut.join(5);sock.close()
    assert not st.is_alive() and not ut.is_alive(),"native proof leaked a server"


PY
```

</details>

- Observed result: unchanged `main` fc1ba891 fails at the first
unresolved request: five proxy memory tools reach upstream. With the
fix, all six requests return HTTP 200:

| Same-session request | Proxy memory tools |
| --- | ---: |
| Fresh, no cwd | 0 |
| pi system cwd | 5 |
| Cwd absent again | 0 |
| Same pi cwd restored | 5 |
| Explicit project header, different system cwd | 5 |
| Cwd block only in ordinary user text | 0 |

The client weather tool stays present. Restored definitions are
structurally equal to the first resolved turn; the explicit header wins.
The literal-percent cwd agrees with its encoded-header form, and the
actual project SQLite database opens at the expected key.

- Not tested: live pi/Caveman, paid-provider cache accounting/billing,
Windows, or other provider protocols. This does not claim to fix Caveman
header forwarding.

## Runtime Rollout Safety

- Rollout-managed feature(s): none added; existing Anthropic memory
injection.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: pi cwd resolves; unresolved
project/empty scope no longer adds unusable proxy memory tools.
- Kill switch / disable path: existing `--no-memory`; explicit
project/cwd headers still work.
- Unsafe override required: no.
- Qualification impact: no storage migration or new dependency; scoped
HTTP/SQLite behavior checked.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes — related
suite and critical gate, not a whole-suite claim
- [x] I did **not** edit `CHANGELOG.md` — release-please owns it

## Screenshots (if applicable)

N/A; the observable surface is the HTTP payload and project database.

## Additional Notes

AI-assisted; no subagents. OpenAI injection policy is unchanged. Keeping
this draft until the current hosted checks are qualified.

One earlier post-build probe completed the requests and closed both
listeners but hit its whole-process shutdown deadline. The same program
and native artifact later exited normally with faulthandler enabled; its
diagnostic abort timer did not fire. I have not identified the
intermittent cause and am not claiming a shutdown fix. No production
timeout or feature flag was changed to get that result.


The two npm audits now pass on 5ace63e2, along with the other Security
jobs. The earlier [unchanged-base
failure](https://github.com/headroomlabs-ai/headroom/actions/runs/41399323584)
is historical evidence, not a current blocker. This PR changes no
dependency manifests, lockfiles, workflow or audit exceptions.


### Maintainer current-main requalification

Exact head `60597623e00967153d947a46bf197950250f310e`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Clean mechanical update from current upstream main, preserving the pi
system-cwd resolver, request-scoped project guard, main's cache-mode
search skip/logging, and client-tool preservation. Ordinary user cwd
blocks still do not enable the pi fallback; unresolved project scope
still blocks both context injection and replayed memory tools. No
contributor production logic was rewritten and no new production finding
remains in this reviewed scope.

The initial combined tree dropped main's memory-invariant file and two
stream/timeout regression cases. Maintainer test-only repairs restore
those tests rather than reducing coverage: the invariant file is
byte-identical to main, every main assertion in the four
restored/updated test files is preserved by AST comparison, and three
fixtures now supply the handler protocol methods needed for resolved
scope and beta headers. The lifecycle main control passes without that
fixture extension; no claim is made that this particular fixture failure
independently fails main. All 222 affected
scope/invariant/stream/timeout/lifecycle/backpressure/cache-stability/system-prompt
tests pass with one skip. Ten shared Gemini request-fixture cases also
pass after the separate test-only repair for independently verified main
failures. All 12 main-relative changed Python files pass Ruff/format;
whitespace checks pass.

The author's existing real TCP/SQLite/ONNX reproduction and explicit
limits were read and remain in the body. These fresh Windows checks
qualify the combined request-scope regressions; they are not a claim of
rerunning that Linux proof, live pi/client/provider behavior, or a fresh
native build. An existing binary was supplied solely for test startup.
Draft status is preserved; readiness remains the author's decision and
nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `60e01cf7c25e90abb9f5cb66fa221da83e7c5177`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.

Clean integration with upstream/main
ea59ac3eef. Re-reviewed project-scoping
delta against intervening memory/backend/stream changes: pi cwd remains
limited to system context; ordinary user cwd blocks do not enable
fallback; unresolved projects skip context/tool injection and sticky
tool replay while retaining client tools and main cache-mode no-search
behavior.232
scope/invariant/stream/timeout/lifecycle/backpressure/cache/system-prompt/Gemini
tests pass,1optional skip; touched-source Ruff and whitespace pass. No
contributor production policy rewritten. Earlier author TCP/SQLite/ONNX
evidence remains scoped, with no fresh native build or live-pi/provider
claim. Existing draft preserved; final approval awaits exact-head hosted
qualification. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `e93ceb35df7f78bfa2dcc0611609131cf2b08b0a`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 232 passed, 1 skipped, 1 warning in
47.62s. Selected real dependency overlays and native DLL paths are
recorded in final4618-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

No new source finding in the reviewed scope. Final formal approval
awaits exact-head hosted qualification. Project resolution and
unresolved-scope injection/replay gates remain unchanged. Existing draft
status is preserved.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `d693dd2b6e33fd47ff853f7e28d938fc1dbc30ac`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378. Clean source
integration preserves pi systemcwd resolution and unresolved-scope
context/tool/sticky-replay gates.70integration patches exactly matchmain
delta;Anthropic exception patch differs only byclosing-parenthesis diff
alignment and its exception AST equalsmain. Broad
memory/router/projectscope/continuation/backpressure/cache/lifecycle/doctor/nativeops
run:405passed,1skipped,one newmain fake-handler fixture failed
missingis_project_unresolved. Maintainer adds only that fake method
returningFalse,respecting resolvedfixture scope;all39backpressure tests
nowpass,including unchanged
injectedSQLiteexception,HTTP200andsemaphorerelease assertions. No
production logic changed.33changedPythonlint/format,offline
lock/whitespacepass;fixedtestlint/formatrechecked.
Existingpr3982nativeDLLverified fullinputhashes/checksum;no
freshbuildclaimed. Currentauthor/Devin discussion checked,inline
endpoint empty;prior scoped realbehavior evidence remains applicable to
unchanged contributor inputs. No new sourcefinding;no
realprovider/agentlaunch claim. Draftpreserved.
Exactnewheadhostedqualification pending;maintainers
handlechecks,nothingmerged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `0da0b5a01958eaaac1d625227cb4bdb9477c5d3b`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main delta after diff-location
normalization. Contributor pi system-cwd resolution and
unresolved-project memory context/tool/sticky-replay gates are
unchanged. Fresh
memory-router/project-scope/streaming/continuation/backpressure/cache/lifecycle
and incoming Vertex/backend/CLI tests: 381 passed, 12 skipped, 1
warning. Twelve changed Python files pass Ruff lint/format; offline lock
and whitespace pass.

Static production/test scope and security review considered user versus
system cwd inputs, malformed/empty cwd, missing scope and stale tool
replay: tests keep pi cwd tags in ordinary user content from resolving
project scope, and unresolved scope blocks replay. No unrelated
executable, workflow, dependency or setup changes. Flagged inherited
workflow-validation job is excluded by unchanged workflow paths. Author
discussion checked; collected inline endpoint has no findings. Prior
scoped behavior evidence applies to unchanged contributor inputs; no new
real agent/provider launch claimed. Native DLL reused only after
complete tracked input-set/hash and artifact checksum verification, no
fresh build claimed. Draft preserved. Formal approval awaits exact
published-head hosted qualification. Maintainers handle checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-10 22:28:01 -06:00

479 lines
16 KiB
Python

from __future__ import annotations
import pytest
fastapi = pytest.importorskip("fastapi")
httpx = pytest.importorskip("httpx")
from fastapi.testclient import TestClient # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
class _FakePrefixTracker:
def get_frozen_message_count(self) -> int:
return 0
def get_last_original_messages(self) -> list[dict]:
return []
def get_last_forwarded_messages(self) -> list[dict]:
return []
def update_from_response(self, **kwargs): # noqa: ANN003
return None
class _BufferedPassthroughClient:
def __init__(self, response: httpx.Response) -> None:
self.response = response
self.calls: list[dict[str, object]] = []
async def request(self, method, url, headers=None, content=None, timeout=None): # noqa: ANN001
self.calls.append(
{
"method": method,
"url": url,
"headers": headers,
"content": content,
"timeout": timeout,
}
)
_assert_buffered_timeout(timeout)
return self.response
async def get(self, url, headers=None, timeout=None): # noqa: ANN001
return await self.request("GET", url, headers=headers, timeout=timeout)
async def post(self, url, headers=None, content=None, timeout=None): # noqa: ANN001
return await self.request("POST", url, headers=headers, content=content, timeout=timeout)
async def aclose(self) -> None:
return None
def _assert_buffered_timeout(timeout: httpx.Timeout | None) -> None:
assert isinstance(timeout, httpx.Timeout)
assert timeout.connect == 3.0
assert timeout.read == 19.0
# `write` is no longer request_timeout_seconds (7): sending the request is a
# separate operation from waiting for the answer, so it has its own bound --
# the fixture's write_timeout_seconds, not the read budget (#3259).
assert timeout.write == 11.0
assert timeout.pool == 3.0
def _make_config() -> ProxyConfig:
return ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
connect_timeout_seconds=3,
request_timeout_seconds=7,
write_timeout_seconds=11,
anthropic_buffered_request_timeout_seconds=19,
)
def _install_prefix_tracker(proxy) -> None:
tracker = _FakePrefixTracker()
proxy.session_tracker_store.compute_session_id = lambda request, model, messages: "s1"
proxy.session_tracker_store.get_or_create = (
lambda session_id, provider, cache_ttl_seconds=None: tracker
)
def _anthropic_message_response() -> dict[str, object]:
return {
"id": "msg_test_1",
"type": "message",
"role": "assistant",
"content": [{"type": "text", "text": "ok"}],
"usage": {
"input_tokens": 12,
"output_tokens": 3,
"cache_read_input_tokens": 0,
"cache_creation_input_tokens": 0,
},
}
def _anthropic_batch_response() -> dict[str, object]:
return {"id": "batch_test_1", "object": "batch", "status": "in_progress"}
def _anthropic_list_response() -> httpx.Response:
return httpx.Response(
200,
json={"object": "list", "data": [], "first_id": None, "last_id": None},
headers={"content-type": "application/json"},
)
def _anthropic_null_usage_response() -> dict[str, object]:
"""An Anthropic Messages response whose usage counts are present but null.
An Anthropic-compatible gateway (custom ANTHROPIC_TARGET_API_URL) can emit
this shape on a stopped or empty turn — the same class that crashed the
Gemini path in #2347.
"""
return {
"id": "msg_test_null",
"type": "message",
"role": "assistant",
"content": [{"type": "text", "text": "ok"}],
"usage": {
"input_tokens": None,
"output_tokens": None,
"cache_read_input_tokens": None,
"cache_creation_input_tokens": None,
},
}
def test_anthropic_messages_buffered_survives_null_usage_counts():
config = _make_config()
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
# Use the real prefix tracker (its `_cached_token_count` is read on this
# path); only pin the session id so the tracker resolves deterministically.
proxy.session_tracker_store.compute_session_id = lambda request, model, messages: "s1"
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
return httpx.Response(200, json=_anthropic_null_usage_response())
proxy._retry_request = _fake_retry # type: ignore[assignment]
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"content-type": "application/json",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"messages": [{"role": "user", "content": "hello"}],
},
)
# Null usage counts must not crash outcome recording / cache-bust math.
assert response.status_code == 200, response.text
def test_anthropic_messages_buffered_timeout_override_reaches_retry_request():
config = _make_config()
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
_install_prefix_tracker(proxy)
captured: dict[str, object] = {}
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
timeout = kwargs.get("timeout")
captured["timeout"] = timeout
_assert_buffered_timeout(timeout)
return httpx.Response(200, json=_anthropic_message_response())
proxy._retry_request = _fake_retry # type: ignore[assignment]
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"content-type": "application/json",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"messages": [{"role": "user", "content": "hello"}],
},
)
assert "timeout" in captured, response.text
assert isinstance(captured["timeout"], httpx.Timeout)
def test_anthropic_batch_create_buffered_timeout_override_reaches_retry_request():
config = _make_config()
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
captured: dict[str, object] = {}
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
timeout = kwargs.get("timeout")
captured["timeout"] = timeout
_assert_buffered_timeout(timeout)
return httpx.Response(200, json=_anthropic_batch_response())
proxy._retry_request = _fake_retry # type: ignore[assignment]
response = client.post(
"/v1/messages/batches",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"content-type": "application/json",
},
json={
"requests": [
{
"custom_id": "req-1",
"params": {
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"messages": [{"role": "user", "content": "hello"}],
},
}
]
},
)
assert response.status_code == 200, response.text
assert isinstance(captured["timeout"], httpx.Timeout)
def test_anthropic_batch_passthrough_buffered_timeout_override_reaches_http_client():
config = _make_config()
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
http_client = _BufferedPassthroughClient(_anthropic_list_response())
proxy.http_client = http_client
response = client.get(
"/v1/messages/batches",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
},
)
assert response.status_code == 200, response.text
assert len(http_client.calls) == 1
assert isinstance(http_client.calls[0]["timeout"], httpx.Timeout)
def test_anthropic_batch_results_buffered_timeout_override_reaches_http_client_get():
config = _make_config()
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
http_client = _BufferedPassthroughClient(
httpx.Response(
200,
content=b'{"custom_id":"req-1","result":{"type":"succeeded"}}\n',
headers={"content-type": "application/jsonl"},
)
)
proxy.http_client = http_client
response = client.get(
"/v1/messages/batches/batch_test_1/results",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
},
)
assert response.status_code == 200, response.text
assert len(http_client.calls) == 1
assert http_client.calls[0]["method"] == "GET"
assert isinstance(http_client.calls[0]["timeout"], httpx.Timeout)
def test_anthropic_ccr_continuation_uses_buffered_timeout() -> None:
config = _make_config()
config.ccr_inject_tool = True
config.ccr_handle_responses = True
app = create_app(config)
class _CCRHandler:
def has_ccr_tool_calls(self, response, provider): # noqa: ANN001
return True
async def handle_response( # noqa: ANN001
self,
response,
optimized_messages,
tools,
api_call_fn,
provider,
):
return await api_call_fn(
optimized_messages
+ [{"role": "assistant", "content": response.get("content", [])}],
tools,
)
with TestClient(app) as client:
proxy = client.app.state.proxy
_install_prefix_tracker(proxy)
proxy.ccr_response_handler = _CCRHandler()
http_client = _BufferedPassthroughClient(
httpx.Response(200, json=_anthropic_message_response())
)
proxy.http_client = http_client
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
return httpx.Response(200, json=_anthropic_message_response())
proxy._retry_request = _fake_retry # type: ignore[assignment]
client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"content-type": "application/json",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"messages": [{"role": "user", "content": "hello"}],
},
)
assert len(http_client.calls) == 1
assert http_client.calls[0]["method"] == "POST"
assert isinstance(http_client.calls[0]["timeout"], httpx.Timeout)
def test_anthropic_memory_continuation_uses_buffered_timeout() -> None:
config = _make_config()
config.memory_enabled = True
app = create_app(config)
class _MemoryHandler:
def __init__(self) -> None:
self.config = type(
"MemoryConfig",
(),
{
"inject_context": False,
"inject_tools": False,
"project_root_override": "",
},
)()
self.initialized = False
self.backend = None
def is_project_unresolved(self, _request_ctx: object) -> bool:
return False
def get_beta_headers(self) -> dict[str, str]:
return {}
def has_memory_tool_calls(self, response, provider): # noqa: ANN001
return True
async def handle_memory_tool_calls( # noqa: ANN001
self,
response,
user_id,
provider,
**kwargs,
):
return [{"type": "tool_result", "tool_use_id": "mem_1", "content": "memory"}]
with TestClient(app) as client:
proxy = client.app.state.proxy
_install_prefix_tracker(proxy)
proxy.memory_handler = _MemoryHandler()
captured_timeouts: list[httpx.Timeout | None] = []
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
timeout = kwargs.get("timeout")
captured_timeouts.append(timeout)
if len(body["messages"]) > 1:
_assert_buffered_timeout(timeout)
return httpx.Response(200, json=_anthropic_message_response())
proxy._retry_request = _fake_retry # type: ignore[assignment]
client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"content-type": "application/json",
"x-headroom-user-id": "user-1",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"messages": [{"role": "user", "content": "hello"}],
},
)
assert len(captured_timeouts) == 2
assert all(isinstance(timeout, httpx.Timeout) for timeout in captured_timeouts)
def test_retry_request_without_override_uses_client_default_timeout():
config = _make_config()
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
captured: list[dict[str, object]] = []
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured.append(kwargs)
return httpx.Response(
200,
json={
"id": "chatcmpl-1",
"object": "chat.completion",
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": "ok"},
"finish_reason": "stop",
}
],
"usage": {"prompt_tokens": 5, "completion_tokens": 1, "total_tokens": 6},
},
)
proxy._retry_request = _fake_retry # type: ignore[assignment]
response = client.post(
"/v1/chat/completions",
headers={"authorization": "Bearer test-key", "content-type": "application/json"},
json={
"model": "gpt-4o",
"messages": [{"role": "user", "content": "hello"}],
},
)
assert response.status_code == 200, response.text
assert len(captured) == 1
assert "timeout" not in captured[0], (
"_retry_request without an override must not pass timeout=None to httpx"
)
def test_generic_proxy_timeout_defaults_stay_unchanged():
app = create_app(ProxyConfig())
with TestClient(app) as client:
timeout = client.app.state.proxy.http_client.timeout
assert timeout.connect == 10.0
assert timeout.read == 300.0
assert timeout.pool == 10.0
# `write` deliberately no longer tracks read. Inheriting 300s put the send
# bound above the ~180-220s point where the OS abandons retransmission, so
# it could never fire against a dead peer (#3259).
assert timeout.write == 150.0
assert timeout.write < timeout.read