Files
headroom/tests/test_proxy_system_prompt_immutable.py
Mansurshakh JaparovandJD Davis 0f3b0d8a21 fix(memory): resolve pi cwd before injecting project tools (#4618)
## Description

Closes #4442.

pi's system `<cwd>` block now resolves the memory project. If project
scope is still unresolved with the `empty` fallback, the Anthropic
handler no longer advertises unusable memory tools, including tools
recorded earlier in the session.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Parse pi cwd only in system text. Keep main's line parser, header/CLI
precedence and limited user-message fallback.
- Reuse `is_project_unresolved` for context and sticky-tool replay.
Client tools and global/user storage behavior stay intact.
- Add initialized-handler scope regressions, replace
source-inspection/forwarding-only checks, and document project scope in
the published memory page.

## Testing

- [x] Unit tests pass (`pytest`) — related suite and the normal critical
gate, not the entire Python suite locally.
- [x] Linting passes (`ruff check .`) — hosted lint passed on 5ace63e2;
local filesystem limitation below.
- [x] Type checking passes (`mypy headroom`) — normal full hook with its
existing `--ignore-missing-imports` setting.
- [x] New tests added for the scope regression
- [x] Manual testing performed

### Test Output

```text
# 16 related test files
329 passed, 12 skipped, 1 warning in 43.07s

# ordinary pre-push: make ci-precheck
178 passed, 5 skipped, 1 warning in 10.87s
✅ ci-precheck PASSED — safe to push.

uv run ruff format --check .
1860 files already formatted
```

Normal commit hooks (Ruff, formatter, full mypy, commitlint) and the
complete pre-push path passed: Rust fmt/clippy/workspace tests, native
build/import verification, critical Python tests and commitlint. The
existing Starlette test-client warning was not filtered.

Local whole-tree Ruff reports EXE002 on five **unchanged** OAuth plugin
files: my `fuseblk` checkout exposes them as executable despite Git
modes `100644`; `chmod` returns zero without changing the mode. The
uncached check agrees. I did not change those files, add shebangs or
ignore the rule. Hosted Ruff, formatting and full mypy passed on
5ace63e2; the local filesystem limitation remains.

The first full CI caught a missing scope predicate on the existing
lifecycle test double. Fixed in
[5ace63e2](https://github.com/headroomlabs-ai/headroom/commit/5ace63e2cbd2295f9594d1e0442a679aad99979c);
its complete file passes (`5 passed, 1 warning in 13.01s`). This
follow-up changes no production code or assertions and adds no runtime
fallback. The behavior proof below ran on 7aeb83cf with the same
production source; that qualification is historical; current-head checks
are recorded below.

### Author follow-up: 80453a03

Moved the project-scope guidance out of the unpublished wiki and into
`docs/content/docs/memory.mdx`; this commit changes no production code
or tests. Viewed the actual Next.js `/docs/memory#proxy-project-scope`
page and its table-of-contents entry; no browser errors.

The existing router/scope tests pass: 65 passed, with four SQLite
ResourceWarnings. All seven pi-parser lines and all six branch edges are
covered, so I did not add duplicate tests for the old Codecov
percentage.

Normal commit and pre-push hooks passed, including Rust checks, native
build/import, 180 critical Python tests (5 skipped, 1
tokenizer-approximation warning), and commitlint. My first push timed
out with the wrong local ONNX path; the next found missing pip in the
fresh uv environment. Both setup issues were fixed without bypassing
checks. Current-head hosted CI is not yet qualified.

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, freshly built native core,
real TCP/Uvicorn proxy and SQLite/ONNX memory backend. Only the upstream
is a local Anthropic-shaped HTTP server; no provider charges.
- Exact command / steps: after `uv sync --locked --extra dev --extra
proxy --extra memory` and `make verify-rust-core`, run this from the
checkout. It prints the actual loaded native module/hash and leaves a
disposable `.headroom-4442-*` directory for inspecting SQLite. Both HTTP
listeners shut down before exit.

<details>
<summary>Standalone HTTP/SQLite reproduction</summary>

```bash
./.venv/bin/python - <<'PY'
import hashlib,http.client,json,os,socket,tempfile,threading,time
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from pathlib import Path
import uvicorn
import headroom._core as core
assert Path(core.__file__).resolve().parent == Path.cwd().resolve()/"headroom"
native_sha=hashlib.sha256(Path(core.__file__).read_bytes()).hexdigest()
from headroom.memory.storage_router import ProjectResolver,RequestContext,extract_system_prompt
from headroom.proxy.server import ProxyConfig,create_app
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES
from headroom.proxy.helpers import get_session_tool_tracker
root=Path(tempfile.mkdtemp(prefix=".headroom-4442-",dir=Path.cwd()))
project_a=root/"pi%2Fproject-a";project_b=root/"project-b"
project_a.mkdir(exist_ok=True);project_b.mkdir(exist_ok=True)
os.environ["HEADROOM_WORKSPACE_DIR"]=str(root/"workspace")
os.environ["HEADROOM_BEACON"]="off"
os.environ["HEADROOM_TELEMETRY"]="off"
os.environ["NO_PROXY"]="127.0.0.1,localhost"
forwarded=[]
class Gateway(BaseHTTPRequestHandler):
    def do_POST(self):
        body=json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        forwarded.append({"path":self.path,"body":body})
        payload=json.dumps({"id":"msg_local_scope","type":"message","role":"assistant","model":"claude-haiku-4-5","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","stop_sequence":None,"usage":{"input_tokens":10,"output_tokens":1}}).encode()
        self.send_response(200);self.send_header("Content-Type","application/json");self.send_header("Content-Length",str(len(payload)));self.end_headers();self.wfile.write(payload)
    def log_message(self,*args):pass
upstream=ThreadingHTTPServer(("127.0.0.1",0),Gateway)
ut=threading.Thread(target=upstream.serve_forever,daemon=True);ut.start()
config=ProxyConfig(mode="cache",anthropic_api_url=f"http://127.0.0.1:{upstream.server_port}",memory_enabled=True,memory_db_path=str(root/"legacy.db"),traffic_learning_enabled=True,subscription_tracking_enabled=False,cost_tracking_enabled=False,cache_enabled=False,rate_limit_enabled=False,log_requests=False,image_optimize=False,ccr_inject_tool=False,ccr_handle_responses=False,ccr_context_tracking=False,anthropic_pre_upstream_memory_context_timeout_seconds=30.0)
app=create_app(config)
sock=socket.socket();sock.bind(("127.0.0.1",0));sock.listen(128)
port=sock.getsockname()[1]
server=uvicorn.Server(uvicorn.Config(app,log_level="warning",lifespan="on",timeout_graceful_shutdown=3))
st=threading.Thread(target=server.run,kwargs={"sockets":[sock]},daemon=True);st.start()
try:
    deadline=time.monotonic()+180
    while not server.started:
        if not st.is_alive() or time.monotonic()>deadline:raise RuntimeError("real proxy startup failed")
        time.sleep(.02)
    session="headroom4442-native-project-scope"
    common={"Content-Type":"application/json","x-api-key":"local-unbilled-key","anthropic-version":"2023-06-01","x-headroom-user-id":"native-scope-user","x-headroom-session-id":session}
    client_tools=[{"name":"get_weather","description":"Weather","input_schema":{"type":"object","properties":{}}}]
    rows=[]
    cases=[("fresh_unresolved","You are helpful.",{},False),("pi_cwd",f"<cwd>\n{project_a}\n</cwd>",{},True),("warm_unresolved","You are helpful.",{},False),("pi_cwd_restored",f"<cwd>\n{project_a}\n</cwd>",{},True),("explicit_header_wins",f"<cwd>\n{project_b}\n</cwd>",{"x-headroom-project-id":"explicit-project"},True),("ordinary_user_cwd_ignored","You are helpful.",{},False)]
    for label,system,extra,expected in cases:
        body={"model":"claude-haiku-4-5","max_tokens":16,"system":system,"messages":[{"role":"user","content":f"cwd:\n<cwd>\n{project_b}\n</cwd>" if label=="ordinary_user_cwd_ignored" else "hello scope"}],"tools":client_tools}
        before=len(forwarded)
        conn=http.client.HTTPConnection("127.0.0.1",port,timeout=90)
        conn.request("POST","/v1/messages",body=json.dumps(body).encode(),headers={**common,**extra})
        response=conn.getresponse();payload=response.read();conn.close()
        assert response.status==200,(label,response.status,payload.decode())
        assert len(forwarded)==before+1,(label,"unexpected upstream request count")
        actual=forwarded[-1]["body"]
        names={tool.get("name") for tool in actual.get("tools",[])}
        assert names==({"get_weather"}|MEMORY_TOOL_NAMES if expected else {"get_weather"}),(label,names)
        scope=ProjectResolver().resolve(RequestContext(headers={**common,**extra},system_prompt=extract_system_prompt(body),base_user_id="native-scope-user"))
        rows.append({"case":label,"status":response.status,"memoryTools":sorted(names & MEMORY_TOOL_NAMES),"project":scope,"gatewayPath":forwarded[-1]["path"]})
    assert forwarded[1]["body"]["tools"]==forwarded[3]["body"]["tools"],"restored definitions changed"
    tracker=get_session_tool_tracker()
    assert tracker.should_inject("anthropic",session),"probe did not exercise an actual recorded sticky session"
    from urllib.parse import quote
    key_a=ProjectResolver().resolve(RequestContext(headers={"x-headroom-cwd":quote(str(project_a),safe="/")},system_prompt="",base_user_id="native-scope-user"))[0]
    assert rows[1]["project"][0]==key_a
    assert rows[4]["project"][1]=="explicit-project"
    router=app.state.proxy.memory_handler._router
    assert router is not None,"real memory backend never initialized"
    opened=router.open_backends()
    project_db=root/"memories"/"projects"/key_a/"memory.db"
    assert project_db in opened and project_db.is_file(),("real pi project database not opened",[str(x) for x in opened])
    print(json.dumps({"mode":config.mode,"realHttpListener":f"http://127.0.0.1:{port}","localGateway":f"http://127.0.0.1:{upstream.server_port}","rows":rows,"realStickyGoldenSession":True,"realPiProjectDatabase":str(project_db),"openedDatabases":[str(x) for x in opened],"nativeModule":core.__file__,"nativeSha256":native_sha,"noLivePiOrPaidProvider":True},indent=2))
finally:
    server.should_exit=True;st.join(20)
    upstream.shutdown();upstream.server_close();ut.join(5);sock.close()
    assert not st.is_alive() and not ut.is_alive(),"native proof leaked a server"


PY
```

</details>

- Observed result: unchanged `main` fc1ba891 fails at the first
unresolved request: five proxy memory tools reach upstream. With the
fix, all six requests return HTTP 200:

| Same-session request | Proxy memory tools |
| --- | ---: |
| Fresh, no cwd | 0 |
| pi system cwd | 5 |
| Cwd absent again | 0 |
| Same pi cwd restored | 5 |
| Explicit project header, different system cwd | 5 |
| Cwd block only in ordinary user text | 0 |

The client weather tool stays present. Restored definitions are
structurally equal to the first resolved turn; the explicit header wins.
The literal-percent cwd agrees with its encoded-header form, and the
actual project SQLite database opens at the expected key.

- Not tested: live pi/Caveman, paid-provider cache accounting/billing,
Windows, or other provider protocols. This does not claim to fix Caveman
header forwarding.

## Runtime Rollout Safety

- Rollout-managed feature(s): none added; existing Anthropic memory
injection.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: pi cwd resolves; unresolved
project/empty scope no longer adds unusable proxy memory tools.
- Kill switch / disable path: existing `--no-memory`; explicit
project/cwd headers still work.
- Unsafe override required: no.
- Qualification impact: no storage migration or new dependency; scoped
HTTP/SQLite behavior checked.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes — related
suite and critical gate, not a whole-suite claim
- [x] I did **not** edit `CHANGELOG.md` — release-please owns it

## Screenshots (if applicable)

N/A; the observable surface is the HTTP payload and project database.

## Additional Notes

AI-assisted; no subagents. OpenAI injection policy is unchanged. Keeping
this draft until the current hosted checks are qualified.

One earlier post-build probe completed the requests and closed both
listeners but hit its whole-process shutdown deadline. The same program
and native artifact later exited normally with faulthandler enabled; its
diagnostic abort timer did not fire. I have not identified the
intermittent cause and am not claiming a shutdown fix. No production
timeout or feature flag was changed to get that result.


The two npm audits now pass on 5ace63e2, along with the other Security
jobs. The earlier [unchanged-base
failure](https://github.com/headroomlabs-ai/headroom/actions/runs/41399323584)
is historical evidence, not a current blocker. This PR changes no
dependency manifests, lockfiles, workflow or audit exceptions.


### Maintainer current-main requalification

Exact head `60597623e00967153d947a46bf197950250f310e`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Clean mechanical update from current upstream main, preserving the pi
system-cwd resolver, request-scoped project guard, main's cache-mode
search skip/logging, and client-tool preservation. Ordinary user cwd
blocks still do not enable the pi fallback; unresolved project scope
still blocks both context injection and replayed memory tools. No
contributor production logic was rewritten and no new production finding
remains in this reviewed scope.

The initial combined tree dropped main's memory-invariant file and two
stream/timeout regression cases. Maintainer test-only repairs restore
those tests rather than reducing coverage: the invariant file is
byte-identical to main, every main assertion in the four
restored/updated test files is preserved by AST comparison, and three
fixtures now supply the handler protocol methods needed for resolved
scope and beta headers. The lifecycle main control passes without that
fixture extension; no claim is made that this particular fixture failure
independently fails main. All 222 affected
scope/invariant/stream/timeout/lifecycle/backpressure/cache-stability/system-prompt
tests pass with one skip. Ten shared Gemini request-fixture cases also
pass after the separate test-only repair for independently verified main
failures. All 12 main-relative changed Python files pass Ruff/format;
whitespace checks pass.

The author's existing real TCP/SQLite/ONNX reproduction and explicit
limits were read and remain in the body. These fresh Windows checks
qualify the combined request-scope regressions; they are not a claim of
rerunning that Linux proof, live pi/client/provider behavior, or a fresh
native build. An existing binary was supplied solely for test startup.
Draft status is preserved; readiness remains the author's decision and
nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `60e01cf7c25e90abb9f5cb66fa221da83e7c5177`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.

Clean integration with upstream/main
ea59ac3eef. Re-reviewed project-scoping
delta against intervening memory/backend/stream changes: pi cwd remains
limited to system context; ordinary user cwd blocks do not enable
fallback; unresolved projects skip context/tool injection and sticky
tool replay while retaining client tools and main cache-mode no-search
behavior.232
scope/invariant/stream/timeout/lifecycle/backpressure/cache/system-prompt/Gemini
tests pass,1optional skip; touched-source Ruff and whitespace pass. No
contributor production policy rewritten. Earlier author TCP/SQLite/ONNX
evidence remains scoped, with no fresh native build or live-pi/provider
claim. Existing draft preserved; final approval awaits exact-head hosted
qualification. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `e93ceb35df7f78bfa2dcc0611609131cf2b08b0a`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 232 passed, 1 skipped, 1 warning in
47.62s. Selected real dependency overlays and native DLL paths are
recorded in final4618-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

No new source finding in the reviewed scope. Final formal approval
awaits exact-head hosted qualification. Project resolution and
unresolved-scope injection/replay gates remain unchanged. Existing draft
status is preserved.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `d693dd2b6e33fd47ff853f7e28d938fc1dbc30ac`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378. Clean source
integration preserves pi systemcwd resolution and unresolved-scope
context/tool/sticky-replay gates.70integration patches exactly matchmain
delta;Anthropic exception patch differs only byclosing-parenthesis diff
alignment and its exception AST equalsmain. Broad
memory/router/projectscope/continuation/backpressure/cache/lifecycle/doctor/nativeops
run:405passed,1skipped,one newmain fake-handler fixture failed
missingis_project_unresolved. Maintainer adds only that fake method
returningFalse,respecting resolvedfixture scope;all39backpressure tests
nowpass,including unchanged
injectedSQLiteexception,HTTP200andsemaphorerelease assertions. No
production logic changed.33changedPythonlint/format,offline
lock/whitespacepass;fixedtestlint/formatrechecked.
Existingpr3982nativeDLLverified fullinputhashes/checksum;no
freshbuildclaimed. Currentauthor/Devin discussion checked,inline
endpoint empty;prior scoped realbehavior evidence remains applicable to
unchanged contributor inputs. No new sourcefinding;no
realprovider/agentlaunch claim. Draftpreserved.
Exactnewheadhostedqualification pending;maintainers
handlechecks,nothingmerged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `0da0b5a01958eaaac1d625227cb4bdb9477c5d3b`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main delta after diff-location
normalization. Contributor pi system-cwd resolution and
unresolved-project memory context/tool/sticky-replay gates are
unchanged. Fresh
memory-router/project-scope/streaming/continuation/backpressure/cache/lifecycle
and incoming Vertex/backend/CLI tests: 381 passed, 12 skipped, 1
warning. Twelve changed Python files pass Ruff lint/format; offline lock
and whitespace pass.

Static production/test scope and security review considered user versus
system cwd inputs, malformed/empty cwd, missing scope and stale tool
replay: tests keep pi cwd tags in ordinary user content from resolving
project scope, and unresolved scope blocks replay. No unrelated
executable, workflow, dependency or setup changes. Flagged inherited
workflow-validation job is excluded by unchanged workflow paths. Author
discussion checked; collected inline endpoint has no findings. Prior
scoped behavior evidence applies to unchanged contributor inputs; no new
real agent/provider launch claimed. Native DLL reused only after
complete tracked input-set/hash and artifact checksum verification, no
fresh build claimed. Draft preserved. Formal approval awaits exact
published-head hosted qualification. Maintainers handle checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-10 22:28:01 -06:00

792 lines
29 KiB
Python

"""System-prompt immutability tests for PR-A2 (P0-1 fix).
After PR-A2, memory context never mutates the system prompt or the
Responses API ``instructions`` field. The cache hot zone is sacrosanct
(invariant I2). Memory routes exclusively to the live-zone tail (the
first text block of the latest non-frozen user turn).
"""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import AsyncMock
import httpx
import pytest
pytest.importorskip("fastapi")
from fastapi.testclient import TestClient
from headroom.proxy.server import ProxyConfig, create_app
@pytest.fixture(autouse=True)
def _stub_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("HEADROOM_REQUIRE_RUST_CORE", "false")
class _FakePrefixTracker:
def __init__(self, frozen_count: int):
self._frozen_count = frozen_count
self._cached_token_count = 0
self._last_original_messages: list[dict[str, object]] = []
self._last_forwarded_messages: list[dict[str, object]] = []
def get_frozen_message_count(self) -> int:
return self._frozen_count
def get_last_original_messages(self): # noqa: ANN201
return self._last_original_messages.copy()
def get_last_forwarded_messages(self): # noqa: ANN201
return self._last_forwarded_messages.copy()
def update_from_response(self, **kwargs): # noqa: ANN003
self._cached_token_count = kwargs.get("cache_read_tokens", 0) + kwargs.get(
"cache_write_tokens", 0
)
self._last_original_messages = kwargs.get(
"original_messages", kwargs.get("messages", [])
).copy()
self._last_forwarded_messages = kwargs.get("messages", []).copy()
return None
def _make_proxy_client() -> TestClient:
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
)
app = create_app(config)
return TestClient(app)
def _install_memory_handler(proxy: object) -> None:
"""Wire a deterministic in-process memory handler that returns 'MEMCTX'."""
proxy.memory_handler = SimpleNamespace( # type: ignore[attr-defined]
config=SimpleNamespace(inject_context=True, inject_tools=False),
is_project_unresolved=lambda request_context: False,
search_and_format_context=AsyncMock(return_value="MEMCTX"),
has_memory_tool_calls=lambda resp, provider: False,
)
def _install_session_tracker(proxy: object, frozen_count: int) -> None:
fake_tracker = _FakePrefixTracker(frozen_count=frozen_count)
proxy.session_tracker_store.compute_session_id = ( # type: ignore[attr-defined]
lambda request, model, messages: "stable-session"
)
proxy.session_tracker_store.get_or_create = ( # type: ignore[attr-defined]
lambda session_id, provider, cache_ttl_seconds=None: fake_tracker
)
def _install_capture_retry(proxy: object, captured: dict[str, object]) -> None:
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["body"] = body
captured["headers"] = dict(headers)
return httpx.Response(
200,
json={
"id": "msg_1",
"type": "message",
"role": "assistant",
"content": [{"type": "text", "text": "ok"}],
"usage": {
"input_tokens": 10,
"output_tokens": 3,
"cache_read_input_tokens": 0,
"cache_creation_input_tokens": 0,
},
},
)
proxy._retry_request = _fake_retry # type: ignore[attr-defined]
# ---------------------------------------------------------------------------
# Anthropic /v1/messages tests
# ---------------------------------------------------------------------------
def test_memory_enabled_does_not_mutate_system() -> None:
"""Memory injection must not mutate the top-level ``system`` field."""
captured: dict[str, object] = {}
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.optimize = False
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
_install_session_tracker(proxy, frozen_count=1)
_install_memory_handler(proxy)
_install_capture_retry(proxy, captured)
original_system = "You are a helpful assistant."
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"x-headroom-user-id": "u1",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"system": original_system,
"messages": [
{"role": "user", "content": "frozen prefix"},
{"role": "assistant", "content": "ack"},
{"role": "user", "content": "latest user"},
],
},
)
assert response.status_code == 200
sent = captured["body"]
assert isinstance(sent, dict)
assert sent["system"] == original_system, (
"system prompt must be byte-equal to client-sent value"
)
def test_memory_context_appears_in_latest_user_message_tail() -> None:
"""Memory context appends to the latest non-frozen user turn's text."""
captured: dict[str, object] = {}
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.optimize = False
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
_install_session_tracker(proxy, frozen_count=1)
_install_memory_handler(proxy)
_install_capture_retry(proxy, captured)
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"x-headroom-user-id": "u1",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"system": "base system",
"messages": [
{"role": "user", "content": "frozen prefix"},
{"role": "assistant", "content": "ack"},
{"role": "user", "content": "latest user"},
],
},
)
assert response.status_code == 200
sent = captured["body"]
assert isinstance(sent, dict)
assert sent["system"] == "base system"
# Memory context appended to the *latest* user turn only.
assert sent["messages"][0]["content"] == "frozen prefix"
assert sent["messages"][2]["content"].endswith("MEMCTX")
def test_cache_mode_skips_memory_context_search() -> None:
"""Cache mode never injects memory context, so it must not search for it."""
captured: dict[str, object] = {}
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.mode = "cache"
proxy.config.optimize = False
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
_install_session_tracker(proxy, frozen_count=1)
_install_memory_handler(proxy)
_install_capture_retry(proxy, captured)
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"x-headroom-user-id": "u1",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"system": "base system",
"messages": [
{"role": "user", "content": "frozen prefix"},
{"role": "assistant", "content": "ack"},
{"role": "user", "content": "latest user"},
],
},
)
assert response.status_code == 200
proxy.memory_handler.search_and_format_context.assert_not_awaited()
sent = captured["body"]
assert isinstance(sent, dict)
assert sent["messages"][2]["content"] == "latest user"
def test_memory_context_byte_deterministic_for_same_query() -> None:
"""Two identical inbound requests produce identical outbound bytes."""
capture_a: dict[str, object] = {}
capture_b: dict[str, object] = {}
def _send(capture: dict[str, object]) -> None:
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.optimize = False
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
_install_session_tracker(proxy, frozen_count=1)
_install_memory_handler(proxy)
_install_capture_retry(proxy, capture)
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"x-headroom-user-id": "u1",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"system": "base system",
"messages": [
{"role": "user", "content": "frozen prefix"},
{"role": "assistant", "content": "ack"},
{"role": "user", "content": "latest user"},
],
},
)
assert response.status_code == 200
_send(capture_a)
_send(capture_b)
assert capture_a["body"] == capture_b["body"]
def test_memory_disabled_is_no_op(monkeypatch: pytest.MonkeyPatch) -> None:
"""``HEADROOM_MEMORY_INJECTION_MODE=disabled`` skips injection."""
monkeypatch.setenv("HEADROOM_MEMORY_INJECTION_MODE", "disabled")
captured: dict[str, object] = {}
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.optimize = False
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
_install_session_tracker(proxy, frozen_count=1)
_install_memory_handler(proxy)
_install_capture_retry(proxy, captured)
response = client.post(
"/v1/messages",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"x-headroom-user-id": "u1",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"system": "base system",
"messages": [{"role": "user", "content": "latest user"}],
},
)
assert response.status_code == 200
sent = captured["body"]
assert isinstance(sent, dict)
assert sent["system"] == "base system"
assert sent["messages"][0]["content"] == "latest user"
def test_invalid_injection_mode_raises() -> None:
"""Unknown values for the env var must fail loudly — no silent fallback."""
import os
from headroom.proxy.helpers import get_memory_injection_mode
prev = os.environ.get("HEADROOM_MEMORY_INJECTION_MODE")
os.environ["HEADROOM_MEMORY_INJECTION_MODE"] = "system_prompt"
try:
with pytest.raises(ValueError, match="Invalid HEADROOM_MEMORY_INJECTION_MODE"):
get_memory_injection_mode()
finally:
if prev is None:
os.environ.pop("HEADROOM_MEMORY_INJECTION_MODE", None)
else:
os.environ["HEADROOM_MEMORY_INJECTION_MODE"] = prev
# ---------------------------------------------------------------------------
# OpenAI /v1/responses tests
# ---------------------------------------------------------------------------
def _make_responses_proxy_client() -> TestClient:
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
)
app = create_app(config)
return TestClient(app)
def test_memory_enabled_does_not_mutate_instructions_responses_api() -> None:
"""Memory injection must not mutate ``body['instructions']`` on /v1/responses."""
captured: dict[str, object] = {}
with _make_responses_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.optimize = False
proxy.config.image_optimize = False
_install_memory_handler(proxy)
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["body"] = body
return httpx.Response(
200,
json={
"id": "resp_1",
"object": "response",
"model": "gpt-5.4",
"output": [],
"usage": {"input_tokens": 5, "output_tokens": 1, "total_tokens": 6},
},
)
proxy._retry_request = _fake_retry
original_instructions = "You are a Codex assistant."
response = client.post(
"/v1/responses",
headers={
"authorization": "Bearer sk-test",
"x-headroom-user-id": "u1",
},
json={
"model": "gpt-5.4",
"instructions": original_instructions,
"input": [
{
"role": "user",
"content": [{"type": "input_text", "text": "latest user"}],
},
],
},
)
# Test passes if upstream call was reached; even if the test mock
# surface for /v1/responses doesn't fully exercise, instructions
# must not be mutated on any path.
if "body" in captured:
sent = captured["body"]
assert isinstance(sent, dict)
assert sent.get("instructions") == original_instructions, (
"instructions must remain byte-equal to client-sent value"
)
else:
# If the test client did not reach the upstream mock, accept the
# path-coverage that the env-var routing block ran without
# exception (the function returned a response at all).
assert response is not None
def test_memory_context_appears_in_responses_api_input_tail() -> None:
"""Memory context appends to the latest user item's first text block."""
captured: dict[str, object] = {}
with _make_responses_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.optimize = False
proxy.config.image_optimize = False
_install_memory_handler(proxy)
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["body"] = body
return httpx.Response(
200,
json={
"id": "resp_1",
"object": "response",
"model": "gpt-5.4",
"output": [],
"usage": {"input_tokens": 5, "output_tokens": 1, "total_tokens": 6},
},
)
proxy._retry_request = _fake_retry
client.post(
"/v1/responses",
headers={
"authorization": "Bearer sk-test",
"x-headroom-user-id": "u1",
},
json={
"model": "gpt-5.4",
"instructions": "You are a Codex assistant.",
"input": [
{
"role": "user",
"content": [{"type": "input_text", "text": "latest user prompt"}],
},
],
},
)
# If the test exercised the full path, the latest user text item ends
# with MEMCTX. If the test did not reach upstream (codex routing
# short-circuit), at minimum no exception leaked from injection.
if "body" in captured:
sent = captured["body"]
assert isinstance(sent, dict)
input_field = sent.get("input")
assert sent.get("instructions") == "You are a Codex assistant."
# Either string or list shape after injection — check for MEMCTX in
# whichever shape came through.
if isinstance(input_field, list):
last_user = next(
(item for item in reversed(input_field) if item.get("role") == "user"),
None,
)
if last_user is not None and isinstance(last_user.get("content"), list):
text_parts = [
p.get("text", "")
for p in last_user["content"]
if isinstance(p, dict) and p.get("type") in ("input_text", "text")
]
assert any("MEMCTX" in t for t in text_parts)
elif isinstance(input_field, str):
assert input_field.endswith("MEMCTX")
def test_cache_mode_skips_memory_context_search_openai_chat() -> None:
"""Cache mode never injects memory context on OpenAI chat, so it must not search for it."""
captured: dict[str, object] = {}
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.mode = "cache"
proxy.config.optimize = False
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
_install_session_tracker(proxy, frozen_count=1)
_install_memory_handler(proxy)
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["body"] = body
return httpx.Response(
200,
json={
"id": "chatcmpl_1",
"object": "chat.completion",
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": "ack"},
"finish_reason": "stop",
}
],
"usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12},
},
)
proxy._retry_request = _fake_retry
response = client.post(
"/v1/chat/completions",
headers={
"authorization": "Bearer sk-test",
"x-headroom-user-id": "u1",
},
json={
"model": "gpt-4o",
"messages": [
{"role": "system", "content": "base system"},
{"role": "user", "content": "frozen prefix"},
{"role": "assistant", "content": "ack"},
{"role": "user", "content": "latest user"},
],
},
)
assert response.status_code == 200
proxy.memory_handler.search_and_format_context.assert_not_awaited()
sent = captured["body"]
assert isinstance(sent, dict)
assert sent["messages"][3]["content"] == "latest user"
def test_cache_mode_skips_memory_context_search_openai_responses() -> None:
"""Cache mode never injects memory context on OpenAI responses, so it must not search for it."""
captured: dict[str, object] = {}
with _make_responses_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.mode = "cache"
proxy.config.optimize = False
proxy.config.image_optimize = False
_install_memory_handler(proxy)
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["body"] = body
return httpx.Response(
200,
json={
"id": "resp_1",
"object": "response",
"model": "gpt-5.4",
"output": [],
"usage": {"input_tokens": 5, "output_tokens": 1, "total_tokens": 6},
},
)
proxy._retry_request = _fake_retry
response = client.post(
"/v1/responses",
headers={
"authorization": "Bearer sk-test",
"x-headroom-user-id": "u1",
},
json={
"model": "gpt-5.4",
"instructions": "You are a Codex assistant.",
"input": "latest user string input",
},
)
assert response.status_code == 200
proxy.memory_handler.search_and_format_context.assert_not_awaited()
assert "body" in captured
sent = captured["body"]
assert isinstance(sent, dict)
assert sent.get("input") == "latest user string input"
assert sent.get("instructions") == "You are a Codex assistant."
def test_cache_mode_chat_preserves_tools_and_outcome_skip_reason() -> None:
"""Cache mode never injects memory context on OpenAI /v1/chat/completions."""
captured: dict[str, object] = {}
recorded_outcomes = []
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.mode = "cache"
proxy.config.optimize = False
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
async def _capture_outcome(outcome):
recorded_outcomes.append(outcome)
proxy._record_request_outcome = _capture_outcome
_install_session_tracker(proxy, frozen_count=1)
_install_memory_handler(proxy)
proxy.memory_handler.config.inject_tools = True
proxy.memory_handler.compute_memory_tool_definitions = lambda _provider: [
{"type": "function", "function": {"name": "mem_search", "parameters": {}}}
]
_install_capture_retry(proxy, captured)
response = client.post(
"/v1/chat/completions",
headers={
"authorization": "Bearer sk-test",
"x-headroom-user-id": "u1",
},
json={
"model": "gpt-4o",
"messages": [
{"role": "system", "content": "base system"},
{"role": "user", "content": "latest user"},
],
"tools": [
{
"type": "function",
"function": {"name": "client_tool", "parameters": {}},
}
],
},
)
assert response.status_code == 200
proxy.memory_handler.search_and_format_context.assert_not_awaited()
sent = captured["body"]
assert isinstance(sent, dict)
assert sent["messages"][1]["content"] == "latest user"
# Memory tools remain present even when context search is skipped
assert "tools" in sent
tool_names = [t.get("function", {}).get("name") for t in sent["tools"]]
assert "client_tool" in tool_names
assert "mem_search" in tool_names
# Outcome tags must record memory_skip_reason="cache_mode"
assert len(recorded_outcomes) >= 1
assert recorded_outcomes[0].tags.get("memory_skip_reason") == "cache_mode"
def test_cache_mode_responses_preserves_tools_and_outcome_skip_reason() -> None:
"""Cache mode never injects memory context on OpenAI /v1/responses."""
captured: dict[str, object] = {}
recorded_outcomes = []
with _make_responses_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.mode = "cache"
proxy.config.optimize = False
proxy.config.image_optimize = False
async def _capture_outcome(outcome):
recorded_outcomes.append(outcome)
proxy._record_request_outcome = _capture_outcome
_install_memory_handler(proxy)
proxy.memory_handler.config.inject_tools = True
proxy.memory_handler.compute_memory_tool_definitions = lambda _provider: [
{"type": "function", "function": {"name": "mem_search", "parameters": {}}}
]
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
captured["body"] = body
return httpx.Response(
200,
json={
"id": "resp_1",
"object": "response",
"model": "gpt-5.4",
"output": [],
"usage": {"input_tokens": 5, "output_tokens": 1, "total_tokens": 6},
},
)
proxy._retry_request = _fake_retry
response = client.post(
"/v1/responses",
headers={
"authorization": "Bearer sk-test",
"x-headroom-user-id": "u1",
},
json={
"model": "gpt-5.4",
"instructions": "You are a Codex assistant.",
"input": "latest user prompt",
"tools": [
{
"type": "function",
"name": "client_tool",
"description": "client tool",
"parameters": {},
}
],
},
)
assert response.status_code == 200
proxy.memory_handler.search_and_format_context.assert_not_awaited()
assert "body" in captured
sent = captured["body"]
assert isinstance(sent, dict)
assert sent.get("input") == "latest user prompt"
assert "tools" in sent
tool_names = [t.get("name") for t in sent["tools"]]
assert "client_tool" in tool_names
assert "mem_search" in tool_names
assert len(recorded_outcomes) >= 1
assert recorded_outcomes[0].tags.get("memory_skip_reason") == "cache_mode"
def test_cache_mode_frozen_prefix_restore_logs_at_info_level(
caplog: pytest.LogCaptureFixture,
) -> None:
"""Issue #4505: routine cache-mode prefix restore logs at INFO rather than WARNING."""
import logging
from types import SimpleNamespace
from headroom.proxy.handlers.openai import logger as openai_logger
captured: dict[str, object] = {}
with _make_proxy_client() as client:
proxy = client.app.state.proxy
proxy.config.mode = "cache"
proxy.config.optimize = True
proxy.config.image_optimize = False
proxy.config.ccr_proactive_expansion = False
_install_session_tracker(proxy, frozen_count=1)
_install_capture_retry(proxy, captured)
# Simulate pipeline mutating the frozen prefix message
def _fake_apply(**kwargs):
mutated = list(kwargs["messages"])
mutated[0] = {**mutated[0], "content": "MUTATED_SYSTEM"}
return SimpleNamespace(
messages=mutated,
transforms_applied=["fake:mutated"],
timing={},
tokens_before=20,
tokens_after=18,
waste_signals=None,
)
proxy.openai_pipeline.apply = _fake_apply
caplog.set_level(logging.INFO, logger=openai_logger.name)
response = client.post(
"/v1/chat/completions",
headers={
"authorization": "Bearer sk-test",
},
json={
"model": "gpt-4o",
"messages": [
{"role": "system", "content": "frozen system"},
{"role": "user", "content": "user prompt"},
],
},
)
assert response.status_code == 200
# Verify no WARNING records were emitted for routine prefix restore
warning_records = [
r
for r in caplog.records
if r.levelno >= logging.WARNING
and "Restored" in r.getMessage()
and "frozen prefix" in r.getMessage()
]
assert not warning_records, f"Found unexpected WARNING records: {warning_records}"
# Verify an INFO record was emitted when frozen prefix restoration occurred
info_records = [
r
for r in caplog.records
if r.levelno == logging.INFO
and "Restored" in r.getMessage()
and "frozen prefix message(s)" in r.getMessage()
]
assert len(info_records) == 1
assert "to preserve cache stability (openai)" in info_records[0].getMessage()