mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat/https-compose-caddy
37
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
854a696544 |
fix(proxy): retry a TLS error only while the request is being written (#4776)
## Description A corrupt TLS record after the handshake surfaces as a raw `ssl.SSLError` (for example `[SSL: RECORD_LAYER_FAILURE] record layer failure`, or the `SSLV3_ALERT_BAD_RECORD_MAC` bursts in #1615). httpcore maps `ssl.SSLError` to `ConnectError` only in `start_tls`; its `read`/`write` maps cover only anyio's resource errors and timeouts, and anyio's `TLSStream` re-raises a non-EOF `SSLError` as it is. So the error is not an `httpx.TransportError`, and the #1639 retry in `_retry_request` and `_stream_response` lets it through. That error alone does not say whether upstream received the request. This PR retries it only when httpcore reports that writing the request failed: its documented `trace` request extension emits `send_request_headers.failed` or `send_request_body.failed`. Upstream then never had a complete request to run. A TLS error raised after the request was written propagates, as on main, so a POST upstream may have accepted is never sent twice. In the streaming path it still ends as the SSE 502, now with `upstream_tls_error` instead of a generic `api_error`. What this no longer fixes: the case the first version of this PR targeted, a corrupt record in the reply on a reused HTTP/1.1 keep-alive connection, happens after the request was written. It is ambiguous, so it still fails the request (502), as on main. What it does fix: a TLS failure while the request is still being written, for example on a shared HTTP/2 connection where a large body waits for flow-control window and the read for that window hits the corrupt record. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `helpers.py`: `RequestWriteTrace`, an httpcore `trace` callback that records whether `send_request_headers` or `send_request_body` failed. `forbids_retry(exc)` is true for a raw `ssl.SSLError` raised after a complete write. - `server.py` (`_retry_request`) and `handlers/streaming.py` (`_stream_response`): each attempt attaches a fresh trace, and a raw `ssl.SSLError` is retried only when that trace shows the write failed. Every other retry is unchanged. The streaming exhaustion handler still catches `ssl.SSLError`, for the 502 `upstream_tls_error` SSE body. - `tests/test_h2_stream_reset_retry.py`: the SSL cases now run through real httpcore over a fake socket. An accepted POST sends one POST and propagates, in the buffered and the streaming path. A failed write is retried, in both paths. The review's harness (an `httpx.MockTransport` that records the POST, then raises `ssl.SSLError`) sends one POST. - Three test fakes of `http_client.post` accept the new `extensions` kwarg. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_h2_stream_reset_retry.py -q 11 passed in 0.39s # source reverted to the previous head's retry ( |
||
|
|
e1becdf732 |
fix(proxy): use HTTP/1.1 client for ChatGPT streaming and model metadata (#4006)
## Description Ensures that ChatGPT endpoints (`chatgpt.com`, including `/v1/responses` forwarded to `https://chatgpt.com/backend-api/codex/responses`, `/v1/models`, Codex subpaths, and retried requests) consistently utilize `http_client_h1` instead of multiplexing over the default HTTP/2 client. `server.py` already provisions `self.http_client_h1` specifically because ChatGPT's Cloudflare edge challenges HTTP/2 fingerprints on sensitive endpoints. When large context requests (e.g. 500KB - 800KB payload turns) are forwarded over HTTP/2 streaming, HTTP/2 connection/stream flow-control window stalls block the single multiplexed TCP connection in `httpcore`. When this occurs, all subsequent requests to ChatGPT hosts (including `/v1/models` and subsequent turn completions) become blocked waiting behind the stalled connection, eventually leading to proxy unresponsiveness and accumulated asyncio task queues. While `handle_passthrough` in `openai.py` and `openai_images.py` already selected `http_client_h1`, `StreamingMixin._stream_response_generator`, `handle_model_metadata_endpoint`, `_retry_request` in `server.py`, and Codex subpath routes in `proxy_routes.py` previously hardcoded `self.http_client`. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Expose `prefers_http1_passthrough` in `headroom/proxy/handlers/openai.py` (aliasing `_prefers_http1_passthrough`). - Add `_select_streaming_client` to `StreamingMixin` in `headroom/proxy/handlers/streaming.py` that selects `http_client_h1` when the target URL matches ChatGPT endpoints. - Update `_stream_response_generator` and `_compact_sse_stream` to use `_select_streaming_client`. - Update `_retry_request` in `headroom/proxy/server.py` to route ChatGPT target URLs through `http_client_h1`. - Update `_register_openai_responses_subpath_route` in `headroom/providers/proxy_routes.py` to forward through `http_client_h1` for `handle_chatgpt_codex_responses_subpath`. - In `headroom/providers/model_metadata.py`, pass `getattr(proxy, "http_client_h1", None) or proxy.http_client` to `handle_chatgpt_model_metadata`. - Add unit tests in `tests/test_proxy_handler_helpers.py` and `tests/test_provider_model_metadata.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py ======================== 76 passed, 1 warning in 8.24s ======================== uv run ruff check headroom/ tests/test_proxy_handler_helpers.py tests/test_provider_model_metadata.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS 27 (Darwin arm64), Python 3.13, Headroom Proxy v0.39.1 / v0.40.0, Codex CLI and OpenClaw Desktop client. - Exact command / steps: Sent 800 KB streaming turns to /v1/responses targeting https://chatgpt.com/backend-api/codex/responses while monitoring TCP socket states via lsof/netstat and triggering concurrent /v1/models calls. - Observed result: Streaming responses and model metadata resolve over HTTP/1.1 without multiplexed stream stall; Send-Q drains cleanly, turns finish in ~2.1s, and all unit tests pass. - Not tested: Third-party proprietary non-ChatGPT OAuth endpoints (e.g., enterprise SSO providers). ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: stable - Stable/default behavior changed: No (aligns streaming, retries, and subpath routes with existing `http_client_h1` passthrough policy for ChatGPT hosts) - Kill switch / disable path: Fallback to `self.http_client` when `http_client_h1` is unset - Unsafe override required: No - Qualification impact: None - Rollback path: Revert commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ### Maintainer current-main requalification Exact head `d6cee507ff0aae7f09705b2a5b9a8bd8264b137f`, upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. Updated against upstream/main |
||
|
|
5ad2363798 |
fix(proxy): keep mid-conversation system sections for third-party models (#4591)
## Description Follow-up to #3552 and #2991. The #765 wire-contract guard still hoists every **text** `role="system"` section out of `messages[]` when the model is not on the Claude 5 allow-list, and on a third-party Anthropic-compatible upstream (vLLM, GLM, …) it never is. #3553 fixed the non-text half of #3552; its point 4 ("it also re-introduces #2991's cache and semantics costs for every custom-upstream user") was left open. It fires on both custom-upstream paths: - a configured target (`--anthropic-api-url` / `ANTHROPIC_TARGET_API_URL`): the guard gets the real model name, which is not on the allow-list; - a per-request `x-headroom-base-url`: the guard gets `model=None`. The client in this setup appends a mid-conversation system section (a `<system-reminder>`) on most turns, so on every request the hoisted text lengthens the top-level `system`. The rendered prompt then changes at its very start, and the upstream's prefix cache misses the whole conversation every turn. That happens in cache mode too, whose point is a byte-identical frozen prefix. Seen with that client → `headroom proxy --anthropic-api-url http://<vllm>:8000` → vLLM serving Qwen3.8-Flash-Next: on a ~345K-token session, two consecutive turns each waited 245 s for the first token (a full re-prefill), and 0 tokens were compressed. Sent straight to vLLM, the same session keeps its sections in place and vLLM accepts them. **Fix.** For a non-Claude model on a third-party upstream (`upstream_base_url`, or the handler's existing `_third_party_anthropic_upstream` for a configured target), valid-position sections are treated as supported. They are forwarded where the client put them, which is where the client sends them without Headroom. The following do not change: - invalid placements, including the #765 leading `messages[0]` case, are still relocated; - Claude-named models on a third-party upstream (`claude-…`, `anthropic/claude-…`, `global.anthropic.claude-…`) keep today's conservative hoist, because a gateway may reach a provider without the feature; - Anthropic, Vertex, Bedrock and other backends keep the existing allow-list path. Those only serve Claude-named models, so `is_third_party_model` is false for them. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/helpers.py`: adds `is_third_party_model(model, *, third_party_upstream)`. `relocate_system_messages_to_top_level(..., *, third_party_model=False)` treats such a model as supporting mid-conversation sections. - `headroom/proxy/handlers/anthropic.py`: the guard passes `third_party_model=is_third_party_model(model, third_party_upstream=bool(upstream_base_url or _third_party_anthropic_upstream))`. `_third_party_anthropic_upstream` is assigned unconditionally in the same `try` block, before the guard. - `tests/test_anthropic_system_relocation_third_party.py` (new): sends a request through the real handler on both third-party paths (a configured target and a per-request `x-headroom-base-url`) and checks the forwarded body. A non-Claude model's section stays in place. A Claude-named model on those paths, and a non-Claude model on Anthropic itself, are still relocated. - `tests/test_proxy_handler_helpers.py`: - a decision table for `is_third_party_model`; - a check that valid sections stay in place for a third-party model; - a check that invalid placements (leading, and after an assistant turn) are still moved. ## Testing - [x] Unit tests pass (`pytest`): the touched test files, plus the related files listed below - [x] Linting passes (`ruff check .`): ruff 0.17.0 on the changed files, plus `ruff format --check` - [x] Type checking passes (`mypy headroom`): the two changed modules - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_anthropic_system_relocation_third_party.py tests/test_proxy_handler_helpers.py 89 passed in 4.89s # the source change reverted, tests kept: $ pytest tests/test_anthropic_system_relocation_third_party.py FAILED tests/test_anthropic_system_relocation_third_party.py::test_third_party_model_keeps_mid_conversation_system_sections[upstream0-https://vllm.example] FAILED tests/test_anthropic_system_relocation_third_party.py::test_third_party_model_keeps_mid_conversation_system_sections[upstream1-https://override.example] 2 failed, 3 passed in 3.95s $ pytest tests/test_anthropic_system_relocation_third_party.py tests/test_proxy_handler_helpers.py \ tests/test_custom_base_passthrough_telemetry.py tests/test_system_compaction_handler.py \ tests/test_session_affinity.py tests/test_proxy_anthropic_cache_stability.py \ tests/test_proxy_system_prompt_immutable.py tests/test_observed_wire_shapes.py \ tests/test_anthropic_auto_mode_passthrough.py tests/test_cache_prefix_safety.py \ tests/test_ccr_tool_injection_cache_stability.py tests/test_header_isolation.py \ tests/test_anthropic_deferred_tool_lineage.py 276 passed in 20.81s $ ruff check <changed files> && ruff format --check <changed files> # ruff 0.17.0 All checks passed! 4 files already formatted $ mypy headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py --follow-imports=silent Success: no issues found in 2 source files # same on the base commit ``` These ran with this branch on `PYTHONPATH` over the headroom-ai 0.42.0 wheel's compiled `_core`; no local Rust build. The full suite was not run. ## Real Behavior Proof - Environment: macOS (Darwin 25.4.0, arm64), Python 3.13. Patched is this branch; stock is headroom-ai 0.42.0, whose `relocate_system_messages_to_top_level` and guard call site are identical to `main` before this change. Both run as `headroom proxy --anthropic-api-url <upstream> --mode cache --no-cache --no-rate-limit --no-subscription-tracking` with `HEADROOM_OFFLINE=1`. - Exact command / steps: (1) Wire capture: a stub Anthropic-compatible upstream records every forwarded body. Two agent-shaped streaming turns for model `qwen3.8-flash-next` go through each proxy, and turn 2 adds a trailing `<system-reminder>` section. A script prints what reached the upstream (output below). (2) Live run: the same client in print mode, with a Bash tool, sends a three-request tool-using task through each proxy to vLLM serving Qwen3.8-Flash-Next on an NVIDIA GB10, and the guard's warnings in the proxy log are counted. - Observed result: (1) stock 0.42.0 hoists both sections, and the top-level `system` changes between turns; this branch forwards both requests unchanged, with the same top-level `system` on both turns. (2) On the live run, stock 0.42.0 logged `Relocated role=system` 4 times for the task and this branch 0 times. The task was answered correctly, and on this branch the last request (~15.2K tokens) returned in ~5 s, so its prefix was cached. - Not tested: the Anthropic API itself, Bedrock, Vertex and Foundry (their paths are unchanged for Claude-named models); the OpenAI and Gemini handlers (untouched); the full test suite; a local Rust build of `_core`. ```text == stock (headroom, version 0.42.0) turn 1: forwarded roles=['user', 'assistant', 'user'] system blocks=2 sha=85b84b30e488 turn 2: forwarded roles=['user', 'assistant', 'user', 'assistant', 'user'] system blocks=3 sha=b6cc1c0d07e7 top-level system identical across turns: False WARNING - Relocated role=system message(s) out of messages[] into the top-level system parameter (Anthropic wire-contract guard, issue #765) INFO - event=outbound_request forwarder=streaming method=POST body_bytes=403 body_mutated=true mutation_reasons=system_role_relocated source=canonical WARNING - Relocated role=system message(s) out of messages[] into the top-level system parameter (Anthropic wire-contract guard, issue #765) INFO - event=outbound_request forwarder=streaming method=POST body_bytes=582 body_mutated=true mutation_reasons=system_role_relocated source=canonical == patched (this branch) turn 1: forwarded roles=['user', 'system', 'assistant', 'user'] system blocks=1 sha=d8329387e7a7 turn 2: forwarded roles=['user', 'system', 'assistant', 'user', 'assistant', 'user', 'system'] system blocks=1 sha=d8329387e7a7 top-level system identical across turns: True INFO - event=outbound_request forwarder=streaming method=POST body_bytes=435 body_mutated=false mutation_reasons= source=passthrough INFO - event=outbound_request forwarder=streaming method=POST body_bytes=631 body_mutated=false mutation_reasons= source=passthrough ``` ## Runtime Rollout Safety - Rollout-managed feature(s): none. The wire-contract guard is unconditional. - Minimum rollout channel: n/a - Stable/default behavior changed: yes, only for a non-Claude model on a third-party Anthropic upstream. Its valid mid-conversation `role="system"` sections are forwarded unchanged instead of hoisted. - Kill switch / disable path: none added - Unsafe override required: no - Qualification impact: none expected - Rollback path: revert this PR ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (n/a, see below) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - Documentation: no page describes the guard's model allow-list, so none changes. - If you would rather keep the hoist and give operators a switch (#3552's suggestion 3), I can rework this into that. Keeping the client's sections seemed the smaller change, and it matches how they reach the upstream without Headroom. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
b1f340e0b3 |
fix(ccr): keep proactive expansion out of the cached prefix (#4300)
## Description CCR proactive expansion used to append retrieved originals into the newest user turn. That turn sits inside the cached prefix when the client's trailing system message holds the cache breakpoint. The extra text existed only in the forwarded body and in the prefix-tracker snapshot; the client never resent it. On the next turn, cache continuity depended on overlay replaying that injection. When the tracker was gone (proxy restart) or `confirmed_frozen_count` was 0, overlay declined the larger replay and rewrote the conversation from the expanded message, from message 0 if that was the first turn. This change places the expansion in a new text block immediately after the last message-level `cache_control` block and omits that block from the prefix-tracker snapshot. Existing prefix bytes stay identical, role alternation is unchanged, and a restart or declined replay costs only that turn's uncached injection. Closes #4287 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Inject CCR expansion after the last cache breakpoint instead of into the latest user turn. - Strip expansion blocks before `PrefixCacheTracker.update_from_response` on the Anthropic HTTP, Bedrock, and streaming paths. - Keep cache-mode skip of the append as-is. - When the breakpoint block is one of several parallel `tool_result` blocks, insert the expansion after the whole consecutive `tool_result` group, so the turn stays `[tool_result, tool_result, text]` instead of `[tool_result, text, tool_result]`. - Mark the block the proxy inserts and strip only that exact block from the tracker snapshot. Client-authored text that happens to use the expansion tags is no longer removed. - Raise the `aiohttp` constraint to `>=3.14.4` (CVE-2026-111063, CVE-2026-111064, GHSA-2g87-pp6c-29x5) and relock; uv resolves aiohttp 3.14.5 and yarl 1.25.1. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy`) on the changed handler - [x] New tests added for new functionality - [x] Tests fail on current `main` and pass on this branch ### Test Output ```text pytest tests/test_proxy_handler_helpers.py tests/test_ccr_expansion_conversation_scope.py \ tests/test_cache_prefix_overlay.py tests/test_ccr_context_tracker.py \ tests/test_bedrock_prefix_tracker_wiring.py tests/test_backend_streaming_cache_metrics.py \ tests/test_proxy_streaming_request_logger.py tests/test_proxy_anthropic_cache_stability.py \ tests/test_cache_breakpoint_diagnostics.py tests/test_cross_turn_cache_safety.py \ tests/test_anthropic_ccr_workspace_unbound.py -q 225 passed, 1 skipped pytest tests scripts/tests -q (local macOS, offline) 41 failed, 16689 passed, 627 skipped, 2 errors The same failing set fails identically on unmodified main in this environment (missing langchain_anthropic / PIL, live-API and UDS/network tests); none are in the CCR or Anthropic handler suites. ruff check . All checks passed! ruff format --check . 1802 files already formatted mypy headroom/proxy/handlers/anthropic.py --ignore-missing-imports Success uv lock --check Resolved 251 packages pip-audit (uv export --extra all): aiohttp findings cleared; newer advisories on certifi 2026.1.4 (fix 2026.10.19) and pillow 12.3.0 (fix 13.0.0) also fail on main and are not addressed here ``` ## Real Behavior Proof - Environment: Linux, Python 3.12, token mode, CCR proactive expansion on, FastAPI TestClient against `/v1/messages`. - Exact command / steps: `pytest tests/test_ccr_expansion_conversation_scope.py::test_expansion_stays_out_of_the_cached_prefix_and_tracker_snapshot tests/test_cache_prefix_overlay.py` (TestClient request to `/v1/messages` with the trailing system message holding `cache_control`, then overlay with `confirmed_frozen_count=0` as a stand-in for the restart path). - Observed result: the client user turn is forwarded unexpanded; expansion sits after the trailing system's `cache_control`; the tracker snapshot has no expansion; overlay with `confirmed_frozen_count=0` replays the unexpanded prefix. - Not tested: live Anthropic cache_read/cache_creation after a real proxy restart. Redacted real-restart runtime evidence is still to come from the author and will be added to this PR; nothing above is from a live upstream. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: none (not rollout-managed) - Stable/default behavior changed: expansion placement and tracker snapshot - Kill switch / disable path: `HEADROOM_NO_CCR_PROACTIVE_EXPANSION=1` / `--no-ccr-proactive-expansion` - Unsafe override required: no - Qualification impact: none expected; the cached prefix bytes are unchanged and only the uncached tail gains the expansion block - Rollback path: revert the commits ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4300" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ## Demo  <video src="https://raw.githubusercontent.com/mvanhorn/pr-demo-assets/main/headroom/ccr-transient-expansion/demo.mp4" controls width="720"></video> --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
75cfbba91d |
fix(proxy): record outcome when a passthrough stream ends mid-character (#4429)
## Description
When a streaming passthrough response ends inside a multi-byte UTF-8
character, the `finally` block of `_handle_streaming_passthrough`
appends a `\n\n` terminator to the leftover bytes and the strict SSE
decoder raises `UnicodeDecodeError`. The request outcome is then never
recorded and the ASGI app logs a traceback. This PR drops the unusable
tail, keeps the usage already received, and records the outcome with the
shared stream-failure status (502, `_stream_outcome_status`) because a
stream cut mid-character did not complete.
Fixes #4427
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/openai.py`: in
`_handle_streaming_passthrough`'s `finally`, the final
`_parse_sse_usage_from_buffer` call is wrapped in
`contextlib.suppress(UnicodeDecodeError)`. A final
complete-but-unterminated event is still parsed as before; only an
undecodable tail is discarded. When that happens the outcome is built
with `status_code=_stream_outcome_status(upstream_response.status_code,
completed_normally)`, the same helper the other streaming paths use in
`streaming.py`, so `emit_request_outcome` books it as failed instead of
as a 200 success. The observed token fields are unchanged.
- `tests/test_proxy_handler_helpers.py`: new test
`test_vertex_stream_passthrough_records_outcome_when_stream_ends_mid_character`
asserts one outcome, `status_code == 502`, 13 input / 5 output tokens
kept, the upstream response closed exactly once, and that
`emit_request_outcome` calls `record_failed` once and `record_request`
never.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
With only the round-2 source change reverted (new assertions kept),
`tests/test_proxy_handler_helpers.py -k mid_character`:
```text
tests/test_proxy_handler_helpers.py:908: in test_vertex_stream_passthrough_records_outcome_when_stream_ends_mid_character
assert outcomes[0].status_code == 502
E AssertionError: assert 200 == 502
1 failed, 74 deselected
```
With the fix, plus `tests/test_route_advice.py
tests/test_5xx_accounting_all_providers.py
tests/test_outcome_records_5xx_as_failed.py`:
```text
============================= 152 passed in 6.00s ==============================
```
```text
$ ruff check headroom/proxy/handlers/openai.py tests/test_proxy_handler_helpers.py
All checks passed!
$ ruff format --check headroom/proxy/handlers/openai.py tests/test_proxy_handler_helpers.py
2 files already formatted
```
`mypy` was not run.
## Real Behavior Proof
- Environment: WSL2 Ubuntu on Windows 11, Python 3.12.15, branch based
on upstream/main `e995eb9f`; `headroom/_core.abi3.so` taken from the
official headroom-ai 0.41.0 wheel (untracked, not committed). Real proxy
on 127.0.0.1:18940, local stub upstream on 127.0.0.1:18941
(`VERTEX_TARGET_API_URL`); the stub log shows the proxied request, so no
provider API was contacted.
- Exact command / steps: start the stub (chunked `text/event-stream`:
one complete usage event `data:
{"type":"message_delta","usage":{"output_tokens":3}}`, then `data:
{"t":"\xe2\x82`, then a clean end), start `python -m
headroom.proxy.server --host 127.0.0.1 --port 18940 --no-cache
--no-rate-limit`, read `GET /stats`, `POST
/v1/projects/p/locations/l/publishers/mistral/models/m:streamRawPredict`
with `{"x":1}`, read `GET /stats` again. Same steps on the base and on
this branch.
- Observed result: base: proxy log shows `Exception in ASGI application`
ending in `UnicodeDecodeError: 'utf-8' codec can't decode bytes in
position 12-13: unexpected end of data`, and `requests.total
before/after: 0 0`. This branch: no traceback in the proxy log, and
`requests.total before/after: 0 1` (this was the first version of the
fix, before the status change). Re-run for the failure status on this
head (
|
||
|
|
a9eb2a7b6f |
fix(proxy): keep Claude Haiku 5.5 mid-conversation system messages in place (#4153)
## Description
Claude Haiku 5.5 accepts `role: "system"` entries inside `messages[]`
(mid-conversation system messages, no beta header).
`relocate_system_messages_to_top_level` (`headroom/proxy/helpers.py`)
doesn't list the Haiku 5 family, so on every Haiku 5.5 request it moves
each valid mid-conversation section into the top-level `system` field.
That has two effects:
- The operator instruction moves from the point in the conversation
where it became true to the top of the prompt, which changes its
meaning.
- The top-level `system` that the cached prefix starts with changes from
conversation to conversation.
Seen on a local proxy over ~28 hours (2026-10-10 01:14 → 2026-10-11
05:27): all 599 `Relocated role=system` warnings were `claude-haiku-5-5`
requests, which is Claude Code's subagent traffic. The Opus 5.5 (9,360)
and Sonnet 5.5 (539) requests in the same window were never relocated.
This adds `"claude-haiku-5"` to the supported families. Haiku 4.5, which
doesn't support mid-conversation system messages, is still relocated.
No linked issue.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/helpers.py`: add `"claude-haiku-5"` to the model
families `relocate_system_messages_to_top_level` treats as supporting
mid-conversation system messages. The substring matching works for every
model-ID form the list already handles, including `claude-haiku-5-5` and
Bedrock-style `…anthropic.claude-haiku-5-5…`. It doesn't match
`claude-haiku-4-5`.
- `tests/test_proxy_handler_helpers.py`: parametrize the "preserves a
valid mid-conversation section" test over `claude-opus-5` and
`claude-haiku-5-5`, and the "moves a valid shape for an unsupported
model" test over `claude-sonnet-4-6` and `claude-haiku-4-5`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ HOME=$(mktemp -d) pytest -q tests/test_proxy_handler_helpers.py
74 passed # origin/main: 72 passed; +2 new parametrized cases
$ ruff check headroom/proxy/helpers.py tests/test_proxy_handler_helpers.py
All checks passed!
$ ruff format --check headroom/proxy/helpers.py tests/test_proxy_handler_helpers.py
2 files already formatted
$ mypy headroom/proxy/helpers.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS arm64, repo `.venv` (Python 3.10.18). The live
proxy traffic came from `headroom proxy` on 127.0.0.1:8787, forwarding
to `api.anthropic.com`.
- Exact command / steps: called `relocate_system_messages_to_top_level`
with a valid `user → system → assistant` request on `origin/main`
(
|
||
|
|
d9e29bb260 |
fix(proxy): keep leading Anthropic system payloads valid (#3671)
## Description Leading Anthropic system-role sections containing image/document or malformed text-shaped blocks could produce an invalid upstream request. Normalize representable text into the top-level `system` field using Anthropic TextBlockParam fields only, preserve cache/citation metadata, and remove unrepresentable leading blocks with an explicit warning. Mid-conversation non-text retention remains covered by the existing contract. Closes #3667 ## Type of Change - [x] Bug fix - [ ] New feature - [ ] Breaking change - [ ] Documentation update ## Changes Made - Keep only `type`, `text`, `cache_control`, and `citations` when converting leading text-shaped blocks. Image sources and tool IDs/names/inputs cannot leak into top-level text blocks, including blocks already labeled `text`. - Normalize every contiguous leading system section, so a second image-only section cannot become an invalid first message after the first section is removed. - Preserve valid text/cache/citation metadata without mutating the input messages. - Retain the PR's explicit warning/drop policy for leading content with no text representation; mid-conversation non-text blocks retain their existing behavior. ## Testing Current head: `cc05220696735f4162a986e24e5e6ab47ab43e71`. - Six new regressions fail before correction and pass afterward: five text-block type/field variants and consecutive leading system sections. - 182 helper, Anthropic cache-stability, system-prompt immutability, and byte-faithful forwarding tests pass (one existing Starlette warning). - Ruff lint/format, whitespace, and Linux-target mypy over 566 files pass. - Allowed text-block fields were checked against the installed Anthropic SDK TextBlockParam contract. ## Real Behavior Proof - Setup: Windows Python 3.12, a real uvicorn proxy and a separate real loopback HTTP upstream enforcing Anthropic's top-level text-block field contract. Optimization, caching, cost tracking, and subscription polling disabled. - Steps: run `pr3671-resume17-runtime.py`, send five real TCP Messages requests, and inspect the upstream-received bodies and proxy responses. - Observed: image-caption, tool-text, text-with-extra-fields, consecutive leading sections, and valid-text control all return HTTP 200. The upstream receives text-only allowed fields and messages beginning with a user turn; cache-control TTL survives. Consecutive sections preserve both text instructions. - Before correction, the same replay returns HTTP 400 for the first four cases; the valid-text control remains HTTP 200. Evidence: `pr3671-resume17-runtime-red.log` and `pr3671-resume17-runtime.log`; tests: `pr3671-resume17-tests.log`. - Not tested: live Anthropic account/billing traffic. Only network delivery is redirected to the loopback upstream. The intentional removal of unrepresentable leading system content remains a warning-producing normalization, not a lossless image conversion. ## Runtime Rollout Safety - Rollout-managed features: none; normalization uses the existing Anthropic wire-contract guard. - Default behavior changed: leading text-shaped blocks emit valid Anthropic text fields, all contiguous leading sections normalize, and unrepresentable leading system blocks are removed with warnings. - Valid text/cache/citation metadata and mid-conversation retention controls are covered. No new unsafe override or network behavior. - Rollback: revert the normalization commits. ## Review Readiness - [x] Self-review completed - [x] Regression coverage and local checks pass - [x] Current real proxy behavior supplied - [x] Did not edit CHANGELOG.md Hosted checks and review must qualify this new head before approval. No merge performed. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
7f608ea932 |
fix(proxy): take a streamed Anthropic turn's uncached input from usage (#3946)
## Description
`StreamingMixin._finalize_stream_response` derives a streamed Anthropic
turn's uncached input as `optimized_tokens - cache_read - cache_write`.
`optimized_tokens` is Headroom's own tokenizer count, so the derivation
turns the estimate's error into uncached input. Anthropic already
reports the uncached count as `usage.input_tokens` on `message_start`
(tokens after the last cache breakpoint), and the non-streaming path
records exactly that field (`anthropic.py`, `uncached_input_tokens =
int(usage.get("input_tokens", 0) or 0)`). Only OpenAI and Gemini take
the provider's count in the streaming finalizer today.
The error goes both ways:
- Estimate over the provider's count: phantom uncached tokens on every
turn. On long, screenshot-heavy Claude Code sessions (~1M context) the
local count ran 9-13% over Anthropic's. One turn Anthropic reported as
`input_tokens=2, cache_read=360949, cache_creation=840` was recorded
with 50,663 uncached tokens. Over one day, three such sessions logged
20.1M uncached tokens where Anthropic reported 3.3M.
- Estimate under the provider's count (most ordinary sessions): the
`max(..., 0)` clamp records 0 and real uncached input goes missing.
Uncached input feeds `new_input_tokens` (uncached + cache write), so the
new-input savings rate moved with the estimate's bias, not with
compression. On the day above it read 14.8% when the rate on Anthropic's
own numbers was 24.7%.
The finalizer now takes `stream_state["input_tokens"]` for `provider ==
"anthropic"` when message_start reported it. Without a usage frame (for
example an error before the stream started), it keeps the old
derivation. `tokens_saved`, `optimized_tokens` and forwarded bytes are
unchanged.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py`: `_finalize_stream_response`
sets `uncached_input_tokens` from the provider's `input_tokens` for
Anthropic when it is an int, else falls back to the tokenizer
derivation.
- `tests/test_proxy_handler_helpers.py`:
`test_stream_finalizer_takes_anthropic_uncached_input_from_usage` covers
a local count over the provider's (the production turn above) and under
it (the old clamp to 0).
`test_stream_finalizer_derives_anthropic_uncached_input_without_usage`
pins the fallback.
- `headroom/proxy/handlers/streaming.py`: `_parse_sse_usage_from_buffer`
no longer defaults an absent Anthropic `message_start` `input_tokens` to
0. It used to, so a usage frame carrying only cache counts read as
"provider reported 0" and the finalizer booked 0 uncached instead of
falling back (Devin's finding). The key now stays absent,
`stream_state["input_tokens"]` stays `None`, and the fallback runs. This
parser feeds both `_finalize_stream_response` call sites; the native
event path (`_stream_response_bedrock`) and the buffered CCR path in
`anthropic.py` already kept absent as `None`.
- `tests/test_proxy_handler_helpers.py`:
`test_sse_parser_leaves_absent_anthropic_input_tokens_absent` covers the
parser half.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
Before the fix (upstream/main
|
||
|
|
3948dbe409 |
fix(proxy): inject memory context past a trailing system message (#3948)
## Description
In `--mode token`, memory context never reaches the model under Claude
Code 2.1.x. Claude Code sends a trailing `role: "system"` message
(environment context that carries the cache breakpoint) after the user
turn:
```text
messages[0] user text <system-reminder>…</system-reminder> (no cache_control)
user text <system-reminder>…</system-reminder> (no cache_control)
user text <the prompt> (no cache_control)
messages[1] system text # Environment … (cache_control)
```
`_append_context_to_latest_non_frozen_user_turn` only looked at
`messages[-1]`, saw a non-user role and returned unchanged. Memory
search found the context, but every request logged
`decision=no_eligible_user_turn` and nothing was injected. The same
helper serves the CCR proactive-expansion path, which had the same
problem.
This PR skips trailing system-role messages when locating the latest
user turn. The system message stays byte-identical and the frozen-prefix
check still applies to the user turn, so prompt-cache behaviour is
unchanged. Follow-up to #3947, which fixed memory *tools* on streaming
turns; this one fixes memory *context injection*.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`:
`_append_context_to_latest_non_frozen_user_turn` walks back past
trailing `role: "system"` messages before the frozen-prefix and
user-role checks; docstring explains why.
- `tests/test_proxy_handler_helpers.py`:
`test_append_context_skips_trailing_system_message` covers injection
into the Claude Code 2.1.x shape (system message untouched), the frozen
prefix still blocking, never reaching past an assistant turn,
tool_result-only turns, and a system-only list.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ HOME=$(mktemp -d) .venv/bin/python -m pytest -q tests/test_proxy_handler_helpers.py tests/test_proxy_anthropic_cache_stability.py tests/test_cache_breakpoint_diagnostics.py tests/test_ccr_context_tracker.py
origin/main: 156 passed, 1 skipped
this branch: 157 passed, 1 skipped
$ HOME=$(mktemp -d) .venv/bin/python -m pytest -q $(grep -rlE 'injected_live_zone_tail|no_eligible_user_turn|_append_context_to_latest|memory_injection|MemoryDecision' tests)
317 passed, 1 skipped
$ uvx ruff@0.16.8 check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py
All checks passed!
$ uvx ruff@0.16.8 format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py
2 files already formatted
```
## Real Behavior Proof
- Environment: macOS 27, real Claude Code 2.1.287 (`claude -p`, Opus
5.5) → Headroom proxy (`--memory --mode token`) → real Anthropic
upstream. Before = `origin/main` (
|
||
|
|
b1b005a9a8 |
fix(memory): align project routing with wrap headers (#3603)
## Description When `headroom wrap claude` runs from two directories named `api`, both requests currently send `X-Headroom-Project: api`. Using that label as a memory or CCR identity can make the two workspaces share history. This keeps `X-Headroom-Project` as the human-readable savings label and adds `X-Headroom-Cwd`, derived from the canonical launch directory, for routing. The resolver now ignores the label when it is the only project signal. Closes #3597 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Keep `X-Headroom-Project` for savings attribution only. - Add a percent-encoded `X-Headroom-Cwd` header containing the canonical launch directory. - Hash the decoded cwd for memory and CCR routing, while retaining the basename as the display label. - Preserve user-supplied project and cwd headers without duplicating or overwriting them. - Add regressions for same-basename directories, label-only fail-closed behavior, and an explicit user-supplied label collision. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text 216 passed, 1 warning in 4.80s 7 files already formatted All checks passed! git diff --check: clean ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 VPS, Python 3.12.3, Rust extension built in the test environment. - Exact command / steps: Wrapped two temporary directories with the same basename (`api`), using the same explicit `X-Headroom-Project` value, then resolved both memory and CCR workspaces. - Observed result: Both requests retained the same display label but produced different cwd headers and different memory and CCR keys. A request containing only the display label returned the fail-closed result. - Not tested: A live request to an external Claude provider. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Stable. - Stable/default behavior changed: `headroom wrap claude` now sends a second `X-Headroom-Cwd` routing header; `X-Headroom-Project` remains a savings label. - Kill switch / disable path: Remove the generated routing header or disable project-scoped memory/CCR using the existing configuration. - Unsafe override required: No. - Qualification impact: Requests with only the savings label remain unscoped and fail closed; requests with a cwd identity retain project-scoped memory and CCR. - Rollback path: Revert commit `101b332`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) --------- Co-authored-by: Tejas Chopra <chopratejas@gmail.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
690fb251dc |
harden: fix security issue in server.py (#3479)
## Summary
Harden input handling in `headroom/proxy/server.py` (flagged by
multi_agent_ai).
## Vulnerability
| Field | Value |
|-------|-------|
| **ID** | V-001 |
| **Severity** | HIGH |
| **Scanner** | multi_agent_ai |
| **Rule** | `V-001` |
| **File** | `headroom/proxy/server.py:3803` |
| **Assessment** | Defensive hardening |
**Description**: The proxy server parses JSON request bodies using
Starlette's request.json() at four locations without visible enforcement
of the MAX_REQUEST_BODY_SIZE constant defined in helpers.py. While the
configuration defines size limits, the actual enforcement in the request
handling path requires verification. Without guaranteed size limits,
attackers can send oversized JSON payloads causing memory exhaustion.
## Threat Model Context
This server appears to be publicly accessible. This is a web service -
vulnerabilities in request handlers are directly exploitable by remote
attackers.
## Changes
- `headroom/proxy/server.py`
## Behavior Preservation
The change is scoped to 1 file on the vulnerable path.
## Security Invariant
> **Property**: The security boundary is maintained under adversarial
input
<details>
<summary>Regression test</summary>
```python
import pytest
import asyncio
from unittest.mock import AsyncMock, MagicMock
import sys
import os
# Ensure the module can be imported
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from headroom.proxy.server import MAX_REQUEST_BODY_SIZE
@pytest.mark.parametrize("payload_size", [
MAX_REQUEST_BODY_SIZE - 1, # valid: just under limit
MAX_REQUEST_BODY_SIZE, # boundary: exactly at limit
MAX_REQUEST_BODY_SIZE + 1, # exploit: exceeds limit
])
def test_request_body_size_enforced(payload_size):
"""Invariant: Request body size MUST NOT exceed MAX_REQUEST_BODY_SIZE."""
# Verify the constant exists and is reasonable
assert isinstance(MAX_REQUEST_BODY_SIZE, int)
assert MAX_REQUEST_BODY_SIZE > 0
# The security property: any payload at or below limit is acceptable,
# anything above MUST be rejected. This test documents the boundary.
is_acceptable = payload_size <= MAX_REQUEST_BODY_SIZE
# For the exploit case, this MUST be False (would be rejected by proper enforcement)
if payload_size > MAX_REQUEST_BODY_SIZE:
assert not is_acceptable, f"Payload size {payload_size} exceeds limit {MAX_REQUEST_BODY_SIZE}"
else:
assert is_acceptable, f"Payload size {payload_size} should be within limit {MAX_REQUEST_BODY_SIZE}"
```
</details>
This test guards against regressions — it's useful independent of the
code change above.
---
*This patch removes an exploit primitive — a code pattern that, while
not independently exploitable today, could be chained with other
weaknesses by automated exploit-development tooling. Proactive removal
of such primitives raises the bar against increasingly capable automated
attack tools.*
---
*Automated security fix by [OrbisAI Security](https://orbisappsec.com)*
<!-- orbis-meta: rule=V-001 scanner=multi_agent_ai -->
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
|
964671d883 |
fix(proxy): keep non text blocks in place when relocating system sections (#3553)
## Description Closes #3552 when a payload carries a mid conversation system message holding non text blocks, `relocate_system_messages_to_top_level` hoisted the whole thing into the top level `system` parameter, image and document blocks included the top level `system` parameter only takes text, so anthropic compatible upstreams that type `system` as a string reject the request, the reporter hit `Input should be a valid string` with `loc body system str` on a z.ai style endpoint the fix keeps the hoist text only: text blocks and bare strings move up, non text blocks stay in a system message at the original position, nothing is dropped and the message order is untouched ### Steps to reproduce 1. run the new tests on untouched main: `python -m pytest -q tests/test_proxy_handler_helpers.py::test_relocate_system_messages_keeps_image_blocks_out_of_top_level_system` 2. Expected (after this fix): text moves to top level `system`, the image block stays in a mid conversation system message 3. Actual (raw output on untouched main |
||
|
|
91fd58d1df |
fix(memory): honor project root override for Anthropic CCR (#3606)
## Description
The documented `--memory-project-root` override is ignored by Anthropic
CCR workspace resolution, so clients without cwd metadata remain
unresolved. This change makes the configured project root available to
that resolver.
Closes #3596
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Read `memory_project_root_override` from the proxy configuration when
resolving an Anthropic CCR workspace.
- Add regression coverage for a configured project root and update
resolver tests for the instance method.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
UV_CACHE_DIR=$PWD/.uv-cache uv run --no-sync pytest tests/test_proxy_handler_helpers.py -q
54 passed in 6.70s
UV_CACHE_DIR=$PWD/.uv-cache uv run --no-sync pytest tests/test_memory_storage_router.py -q
28 passed in 0.30s
UV_CACHE_DIR=$PWD/.uv-cache uv run --no-sync ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py
2 files already formatted
UV_CACHE_DIR=$PWD/.uv-cache uv run --no-sync ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py
All checks passed!
```
## Real Behavior Proof
- Environment: Linux, CPython 3.11.16, checkout commit
|
||
|
|
b3f443636d |
fix(proxy): align signed-thinking wire accounting (#3015)
## Description Signed-thinking histories force byte-faithful passthrough because re-serializing signed Anthropic blocks can invalidate their signatures. Headroom correctly forwarded the original client bytes, but continued reporting mutations, transforms, savings, response headers, and prefix state from a different body that never reached the provider. Separately, the final Anthropic guard hoisted every `role: system` message into the top-level prompt, including valid mid-conversation system sections, changing their semantics and destroying the cached prefix if that mutation ever shipped. This coupled fix makes downstream accounting use the actual wire body whenever the signed-thinking lock discards edits, and narrows system relocation to the current Anthropic model and placement contract. Closes #2990 Closes #2991 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Detects signed thinking in the original request as well as the mutated body, so a transform cannot remove the block and accidentally bypass the byte lock. - Keeps the original-body signature probe best-effort under malformed, recursive, and `MemoryError` conditions. - Carries discarded mutation reasons through the streaming forwarder and emits the existing structured warning on HTTP streaming paths too. - When signed passthrough wins, resets message savings, tool-schema savings, attribution ledgers, transform labels, response headers, and prefix tracking to the original client wire body. - Adds bounded public diagnostic tags naming/counting discarded mutation reasons without exposing body content. - Preserves valid mid-conversation system sections on currently supported Claude models and official Anthropic, Bedrock, and parsed `*.googleapis.com` routes; hostname-boundary validation rejects lookalike and userinfo URLs. - Preserves consecutive system sections and enforces documented predecessor/successor placement rules. - Continues relocating initial, invalidly placed, unsupported-model, and conservative third-party-gateway system messages to avoid upstream 400s. - Includes current `main`, including #2996, #2997, #2971, #3009, #3012, and the MCP dependency cap. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest -q <wire/cache/savings/system focused suite> 379 passed uv run pytest -q tests/test_proxy/test_anthropic_recount_and_reparse_safety.py tests/test_proxy_byte_faithful_forwarding.py tests/test_proxy_handler_helpers.py 99 passed pytest tests scripts/tests --splits 4 --group N --tb=short -q All four CI-shaped fresh-process groups passed locally after correcting the MemoryError regression; each completed in roughly 75-83 seconds. Post-CodeQL correction: 91 focused tests passed; all four exact-head CI-shaped shards passed in roughly 82-95 seconds. uv run ruff format --check . 1411 files already formatted uv run ruff check . All checks passed uv run mypy headroom Success: no issues found in 520 source files ``` ## Real Behavior Proof - Environment: macOS arm64, Python 3.13, branch rebased onto current `main`. - Exact command / steps: sent a signed-thinking request whose tool schema is measurably compacted inside the handler, captured the exact upstream bytes, wrapped the real outcome funnel, and inspected response headers, aggregate metrics, attribution tags, transforms, and prefix-tracker state. Exercised valid, consecutive, invalid, initial, supported-model, and unsupported-model system placements. - Observed result: upstream bytes remain byte-identical to the client; discarded edits contribute zero tokens, zero tool savings, no transform header, and no attribution while the prefix tracker stores the actual wire messages. Valid mid-conversation system sections remain in place; only out-of-contract sections relocate. - Not tested: live paid Anthropic traffic with production credentials. The placement/model contract was verified against the current official documentation and wire behavior is covered with a byte-capturing transport. ## Runtime Rollout Safety - Rollout-managed feature(s): signed-thinking wire-truth accounting and Anthropic mid-conversation system preservation. - Minimum rollout channel: normal patch release after exact-head CI is entirely green. - Stable/default behavior changed: discarded mutations no longer inflate savings; supported valid system sections are no longer hoisted into the top-level prompt. - Kill switch / disable path: no unsafe runtime override; human revert restores the previous conservative relocation/accounting behavior. - Unsafe override required: none. - Qualification impact: all Python shards, byte-forwarding, cache-prefix, outcome/savings, signed-thinking, Anthropic handler, static, Docker, and security checks must remain green. - Rollback path: fix forward through a human-reviewed corrective PR; no persisted data or configuration migration is involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation — inline wire-contract documentation; no separate guide is required - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable; proxy wire behavior and accounting only. ## Additional Notes Human review only. No merge or auto-merge is configured. Current provider contract reference: https://platform.claude.com/docs/en/build-with-claude/mid-conversation-system-messages |
||
|
|
f9807fd69e |
feat(proxy): let extensions report cost savings and their own latency (#3051)
## What
Two changes that let a proxy extension report **what it saved** and
**what it cost**, so both show up under `/stats`, the dashboard, and
Prometheus.
`record_scope_savings` already existed and already accepted `usd` — the
one channel in the proxy that can express savings *without* tokens. Two
things stopped it working end to end.
### 1. Savings were silently dropped on Gemini traffic (bug)
`bind_scope` shares one attribution ledger between ASGI middleware and
the request handler. Anthropic and OpenAI call it; **Gemini never did**,
so anything an extension recorded into the request scope was discarded
for Gemini traffic only — silently, because an empty ledger and an
unbound one are indistinguishable at the outcome funnel. Now bound at
all four Gemini tag sites.
### 2. An extension's own latency was invisible (gap)
`overhead_ms` is measured *inside* the handler, and an ASGI extension
**wraps** that handler — so every millisecond it spends reaches the
client while every timing surface stays flat. An extension that halves
the bill and adds 200 ms per request is a trade the operator has to see
both halves of, and only one half was reaching the dashboard.
`record_scope_timing(scope, stage, ms)` is the symmetric counterpart to
`record_scope_savings`, carried on the same bound ledger and merged into
`RequestOutcome.pipeline_timing` at the outcome funnel — one place, so
every provider picks it up at once.
## API surface
```python
from headroom.proxy.savings_attribution import record_scope_savings, record_scope_timing
record_scope_savings(scope, "my_extension", tokens=0, usd=0.004) # money without tokens
record_scope_timing(scope, "my_extension", elapsed_ms)
```
Both take the ASGI `scope`, because middleware has no other way in.
Documented in `extensions.py` — the module extension authors actually
read, and the stability contract for this interface.
- Savings → `/stats` `savings.by_source`, dashboard card,
`headroom_savings_attributed_usd_total{source=...}`
- Timing → `/stats` `pipeline_timing`, dashboard Performance panel,
`headroom_transform_timing_ms_*`
**Attribution only.** These rows explain the headline total; they are
never added to it.
## Changes to existing behavior
- `public_tags` now strips `_headroom_stage_timing` as well as
`_headroom_savings_attribution`. Both ride on `tags` because that is the
one dict reaching the outcome funnel from every handler, and a list and
a dict must not land in a string-keyed label store.
- `pipeline_timing` passed to `metrics.record_request` is merged rather
than passed through **only when an extension contributed timings**; with
no extension the handler's own dict is passed through unchanged
(asserted by identity in the tests).
- Stage names are extension-supplied, so they are capped at 16 and
namespaced `ext:` — `deep_copy` reported by a plugin must never
accumulate into the same series as `deep_copy` measured by the pipeline.
A handler's own timing wins a collision (unreachable while the prefix
stands; the safe way round if it ever goes).
## Failure modes
Both calls are bounded (32 sources, 16 stages), never raise, and never
change a response — telemetry from a plugin must not be able to break
the request it is describing. Non-positive and non-numeric durations are
ignored: a zero is a clock artifact, not an observation, and averaging
it in would drag the mean down exactly where the stage is cheapest to
skip. `timings_from_tags` tolerates junk on the tag.
## Test-double fix
Three Gemini test fakes (`FakeRequest`, `_FakeRequest`,
`_VertexGeminiImageRequest`) had no `.scope`, which every real Starlette
`Request` has. They now do. This is a double that had drifted from the
type it stands in for; the alternative was weakening the handler to
tolerate a request shape that cannot occur in production.
---
## Real behavior proof
**Setup:** macOS 15.4 (darwin 25.4.0), Python 3.12.13, this branch at
`c814b950`, real `create_app` proxy with `respx`-mocked Anthropic
upstream, a demo ASGI extension added via `app.add_middleware`.
**The extension** — written as a third party would, reporting `tokens=0`
because it re-routed `claude-opus-5` → `claude-haiku-4-5`: same tokens,
cheaper model. That is precisely the case no existing Headroom savings
channel can express, since all of them compute `saved = before - after`.
```python
class DemoRouter:
def __init__(self, app): self.app = app
async def __call__(self, scope, receive, send):
if scope.get("type") != "http":
return await self.app(scope, receive, send)
started = time.perf_counter()
record_scope_savings(scope, "routemegood", tokens=0, usd=0.173)
record_scope_timing(scope, "routemegood", (time.perf_counter() - started) * 1000)
await self.app(scope, receive, send)
```
**Ran:** three POSTs to `/v1/messages`, then `GET /stats` and `GET
/metrics`.
**Observed:**
```
upstream call -> 200
upstream call -> 200
upstream call -> 200
=== /stats savings.by_source (what the dashboard renders) ===
[
{
"source": "routemegood",
"realized": true,
"events": 3,
"tokens": 0,
"usd": 0.519
}
]
=== /stats pipeline_timing (dashboard Performance panel) ===
{
"ext:routemegood": {
"average_ms": 0.01,
"max_ms": 0.02,
"count": 3
}
}
=== /metrics ===
# HELP headroom_savings_attributed_tokens_total Tokens attributed to a savings source
# TYPE headroom_savings_attributed_tokens_total counter
headroom_savings_attributed_tokens_total{realized="true",source="routemegood"} 0
# HELP headroom_savings_attributed_usd_total Cost savings attributed to a source; may be negative
# TYPE headroom_savings_attributed_usd_total gauge
headroom_savings_attributed_usd_total{realized="true",source="routemegood"} 0.519
headroom_transform_timing_ms_sum{transform="ext:routemegood"} 0.03
```
`$0.519 = 3 × $0.173` — three requests, correctly accumulated, with
`tokens: 0` throughout.
**Also have (not a substitute for the above):** 22 new unit tests in
`tests/test_extension_attribution.py`, including four that drive the
real `_record_request_outcome` funnel via the same descriptor-binding
harness `test_request_outcome.py` uses.
Full suite on this branch: **10,989 passed, 578 skipped**. Three
failures —
`test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter`
(full-suite ordering; passes in isolation),
`test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline`,
and `test_release_workflows.py::test_no_native_tls_in_wheel_build_tree`
(needs `cargo`) — **reproduce identically on clean `main`** (`2f4d001c`,
10,967 passed, same 3 failed). Verified by stashing this branch and
re-running the full suite on main in the same tree.
**What I did not test:** a live provider (upstream is `respx`-mocked);
the Gemini `bind_scope` fix against real Google traffic (covered by the
existing 114 Gemini tests, which all pass); the dashboard rendered in a
browser — I verified the JSON shape its templates bind to
(`stats.savings?.by_source`, `stats.pipeline_timing`) rather than the
pixels.
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
9fde127534 |
fix(proxy): relocate stray system-role messages to the top-level system param (#765) (#1357)
## Description On requests large enough to trigger compression, the proxy emitted an upstream Anthropic request whose `messages[0]` had `role: "system"`. Anthropic's Messages API rejects any `system` role inside `messages[]`: ``` 400 invalid_request_error: "messages.0: use the top-level 'system' parameter for the initial system prompt" ``` The original request correctly carries its system prompt in the top-level `system` parameter; a compression/transform/pipeline step relocates the harness system block into `messages[0]`, so the request fails outright (intermittent only because it requires a context large enough to compress). This adds a wire-contract guard in the Anthropic forwarder: as the **last** step before sending upstream (after every transform, memory injection, tool sort, and pipeline extension, covering both the Bedrock and direct paths), any stray `role="system"` message is relocated out of `messages[]` and merged back into the top-level `system` parameter. Content order is preserved (existing system first, relocated content after) and block-level `cache_control` survives. The guard is a no-op on the common path (no system-role entry → inputs pass through unchanged). Closes #765 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/helpers.py`: new pure helper `relocate_system_messages_to_top_level(messages, system) -> (clean_messages, new_system, changed)` plus `_system_message_to_blocks`. Handles `system` being `None`/`str`/`list`, never drops content, preserves order and content blocks. - `headroom/proxy/handlers/anthropic.py`: invoke the guard just before the byte-faithful forward block; on relocation, update `body["messages"]`/`body["system"]`, mark the body mutated (`system_role_relocated`) so the byte-faithful forwarder re-serializes, and log a warning. - `tests/test_proxy_handler_helpers.py`: 3 unit tests (relocate stray system into top-level, append-to-existing-system order, no-op without a system entry). - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --extra dev python -m pytest tests/test_proxy_handler_helpers.py -q 29 passed in 4.95s # Regression on the forward path (byte-faithful forwarding, system-prompt immutability, cache stability): $ uv run --extra dev python -m pytest tests/test_proxy_handler_helpers.py tests/test_proxy_byte_faithful_forwarding.py tests/test_proxy_system_prompt_immutable.py tests/test_proxy_anthropic_cache_stability.py -q 90 passed, 15 warnings in 29.72s $ uv run ruff check headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py All checks passed! $ uv run ruff format --check ... # 3 files already formatted $ uv run mypy headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py Success: no issues found in 2 source files ``` ## Test verification (RED → GREEN) The new tests exercise the guard directly and import the new helper at module top, so reverting the production fix makes them fail at collection. **RED — production fix reverted (helper removed):** ```text ImportError while importing test module 'tests/test_proxy_handler_helpers.py'. E ImportError: cannot import name 'relocate_system_messages_to_top_level' from 'headroom.proxy.helpers' =========================== 1 error in 0.41s =============================== ``` **GREEN — production fix applied:** ```text tests/test_proxy_handler_helpers.py ... [100%] ======================= 3 passed, 26 deselected in 1.50s ======================= ``` ## Real Behavior Proof - Environment: Python 3.13, `uv run` in this repo, branch `fix/issue-765`. - Exact command / steps: ran the guard on a body in the exact #765 failure shape — `system: None` and a `role="system"` harness block at `messages[0]`: - Observed result: ```text BEFORE: messages[0].role = system (Anthropic 400 trigger) changed = True AFTER roles = ['user', 'assistant'] system param = [{"type": "text", "text": "You are Claude Code. <system-reminder>...</system-reminder>"}] OK: no role=system in messages[]; system content preserved in top-level param ``` The illegal `role="system"` entry is removed from `messages[]` and its content lands in the top-level `system` parameter — exactly the body Anthropic accepts. - Not tested: a full live 250k+-token Claude Code session against the real Anthropic API (needs a large live context + API key); the fix is validated at the request-shaping boundary the 400 is raised on. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The guard intentionally fires at the forwarder boundary rather than in any single transform: the issue's captures show the relocation can originate from the compression path, and pipeline extensions / hooks can also mutate `messages` late. Enforcing Anthropic's wire contract once, at the point the body is serialized upstream, fixes the 400 regardless of which step introduced the stray entry and matches the architecture invariant "never produce a `system`-role entry within `messages[]`". --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
f840d5f2fe |
fix(memory): make explicit-project and user store keys collision-resistant (#2231)
## Description
Two of the memory storage router's key-derivation paths can pool
distinct identities into one store.
`ProjectResolver._identity_from_cwd` builds a collision-resistant key by
appending a `sha256` digest to the sanitized basename:
```python
safe_basename = cls._sanitize_basename(basename) or "project"
digest = hashlib.sha256(normalised.encode("utf-8")).hexdigest()[:16]
key = f"{safe_basename}-{digest}"
```
But the two non-cwd paths use the bare sanitized basename as the key:
```python
# Tier 1 — explicit x-headroom-project-id
safe = self._sanitize_basename(explicit)
if safe:
return safe, explicit # <-- no digest
# USER mode
user_safe = ProjectResolver._sanitize_basename(ctx.base_user_id) or "default"
db_path = self._config.root_dir / "users" / user_safe / "memory.db" # <-- no digest
```
`_sanitize_basename` maps every disallowed character to a single dash,
so distinct inputs collapse to the same basename:
- `acme/api` and `acme api` (and `acme@api`) all → `acme-api`
- user ids `alice/qa` and `alice qa` → `alice-qa`
Both the project key (`root/projects/<key>/memory.db`) and the USER key
(`root/users/<key>/memory.db`) are derived directly from that basename,
so two distinct project ids — or, in USER mode, two distinct **users** —
resolve to the same `memory.db` and share each other's memories. USER
mode exists specifically to isolate users, so this is a cross-user
data-isolation leak; the explicit-project-id path is the same leak
across projects. Both are client-controlled (`x-headroom-project-id` /
`x-headroom-user-id` headers), so the collision is easy to hit and could
even be provoked deliberately.
## Fix
Append the same digest of the raw id to both keys, exactly as
`_identity_from_cwd` does, keeping the sanitized basename as a
human-readable prefix:
```python
digest = hashlib.sha256(explicit.encode("utf-8")).hexdigest()[:16]
return f"{safe}-{digest}", explicit
```
```python
digest = hashlib.sha256(ctx.base_user_id.encode("utf-8")).hexdigest()[:16]
user_key = f"{user_safe}-{digest}"
db_path = self._config.root_dir / "users" / user_key / "memory.db"
```
Distinct ids now always land on distinct stores; the same id remains
stable across calls.
**Migration note:** this changes the on-disk key format for the
explicit-project and USER stores (`<basename>` → `<basename>-<digest>`).
Memories written under the old bare-basename paths are not migrated; the
router will start a fresh store at the new path. GLOBAL and cwd-derived
PROJECT stores (which already carried the digest) are unaffected.
Flagging this explicitly so you can decide whether a migration shim is
wanted before merge.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/memory/storage_router.py`: append a `sha256` digest to the
explicit-project-id key (Tier 1) and the USER-mode key, matching
`_identity_from_cwd`.
- `tests/test_memory_storage_router.py`: update the Tier-1 key assertion
to the prefix+digest form; add collision regression tests for the
explicit-project and USER paths.
- `CHANGELOG.md`: Bug Fixes entry (including the migration note).
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/memory/storage_router.py tests/test_memory_storage_router.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/memory/storage_router.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the key derivation with a dependency-free script mirroring
`_sanitize_basename` + the digest, and left the full pytest to CI.
- Exact command / steps: derived keys for `alice/qa` and `alice qa`
under the OLD bare-basename scheme and the NEW digest scheme.
- Observed result: OLD → both `alice-qa` (identical → shared store); NEW
→ `alice-qa-7e02fc2dfbc447b4` vs `alice-qa-4c9241514a374ba3` (distinct),
stable per input, with the `alice-qa-` prefix retained.
- Not tested: a live proxy with two colliding tenants; full local
`pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The changed/added tests
use the existing `tests/test_memory_storage_router.py` harness so they
run under the normal CI pytest job; behaviour is additionally verified
by the standalone proof above. I updated
`test_resolver_tier1_explicit_project_id_wins` to assert the new
prefix+digest key. Happy to add a migration shim (read the old path if
the new one is empty) if you'd prefer that over the fresh-store
behavior.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
22b707fd31 |
fix(proxy/cost): count Gemini thinking tokens in output usage (#2639)
## Description
The Gemini handlers take the response's output-token count straight from
`candidatesTokenCount`:
```python
output_tokens = _usage_int(usage.get("candidatesTokenCount"))
```
For Gemini 2.5 thinking models that undercounts. Gemini reports
`candidatesTokenCount` **sometimes inclusive** of the reasoning tokens
(`thoughtsTokenCount`) and **sometimes exclusive** of them. When it is
exclusive, the thinking tokens are a separate bucket that is still
billed at the output rate, so dropping them makes `output_tokens` (and
therefore the output cost that flows through `record_tokens` ->
`estimate_cost`) too low. The gap grows with reasoning effort.
litellm handles exactly this: it adds `thoughtsTokenCount` to completion
tokens unless `promptTokenCount + candidatesTokenCount ==
totalTokenCount` (its `is_candidate_token_count_inclusive` check). The
Headroom handlers had no equivalent.
## Fix
Add `gemini_output_tokens(usage_meta)` in
`headroom/proxy/token_counting.py`:
- No `thoughtsTokenCount` (the common non-2.5 case): return
`candidatesTokenCount` unchanged.
- `promptTokenCount + candidatesTokenCount == totalTokenCount`:
candidates already include thoughts, return `candidatesTokenCount`.
- Otherwise: return `candidatesTokenCount + thoughtsTokenCount`.
This mirrors litellm's rule and is robust to missing or null fields.
Wire it into the native Gemini handler (both the generate and count
paths), the streaming usage extractors, and the OpenAI-compatible
passthrough usage normalizer, so every Gemini usage path counts output
the same way.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- `headroom/proxy/token_counting.py`: add `gemini_output_tokens()`.
- `headroom/proxy/handlers/gemini.py`: use it for `output_tokens` on
both response paths.
- `headroom/proxy/handlers/streaming.py`: use it in the two Gemini
streaming usage extractors.
- `headroom/proxy/handlers/openai.py`: use it in
`_passthrough_usage_from_json` (Gemini-shaped usage).
- `tests/test_proxy_handler_helpers.py`: unit test for
`gemini_output_tokens` (inclusive / exclusive / no-thinking / empty) and
a `_passthrough_usage_from_json` test that thinking tokens land in
`output_tokens`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` / `ruff format --check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for the fix
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_proxy_handler_helpers.py -k "gemini_output_tokens or thinking or vertex_usage_metadata" -q
3 passed
$ python -m pytest tests/test_proxy_gemini_native_integration.py tests/test_proxy/test_gemini_savings_profile.py tests/test_proxy_handler_helpers.py -q
38 passed, 18 skipped
# with the wiring reverted, the passthrough test fails (output_tokens is 200, not 700):
$ git stash push headroom/proxy/handlers/openai.py && \
python -m pytest tests/test_proxy_handler_helpers.py -k passthrough_usage_counts_gemini_thinking -q
1 failed
$ uvx ruff@0.15.17 check headroom/proxy/token_counting.py headroom/proxy/handlers/gemini.py headroom/proxy/handlers/streaming.py headroom/proxy/handlers/openai.py tests/test_proxy_handler_helpers.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/token_counting.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called `gemini_output_tokens` on an exclusive
usage (`prompt=1000, candidates=200, thoughts=500, total=1700`), an
inclusive usage (`candidates=700, total=1700`), a no-thinking usage, and
`{}`; drove `_passthrough_usage_from_json` with a thinking usage; then
reverted the handler wiring and re-ran the passthrough test.
- Observed result: exclusive returns 700 (200 visible plus 500
thinking), inclusive returns 700, no-thinking returns the candidates
count, empty returns 0; `_passthrough_usage_from_json` reports
`output_tokens=700`. With the wiring reverted it reports 200 (the
undercount). Verified against litellm's documented rule.
- Not tested: a live Gemini 2.5 request end to end (the accounting is
verified at the usage-extraction boundary against litellm's reference
logic).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
|
2a63ec70b6 |
fix(image): reuse image models instead of rebuilding them per request (#2513) (#2536)
## Description Fixes #2513. Image compression rebuilt its heavyweight models on every request: - `_compress_messages_worker` (`proxy/image_isolation.py`) created a new `ImageCompressor()` per call, and - `ImageCompressor.compress` (`image/compressor.py`) created a new `OnnxTechniqueRouter(use_siglip=...)` per image. Each `OnnxTechniqueRouter` loads native `ort.InferenceSession` models, and ONNX Runtime holds C++ memory that Python's GC does not eagerly reclaim. The image pool is a **persistent** single-worker `ProcessPoolExecutor`, so those sessions accumulated in the worker and RSS grew ~70 KB/request, reaching ~1.1 GB after ~15k image requests in a day (the log shows one `[RapidOCR] Using engine_name: onnxruntime` line per request, confirming reloads). ## Fix Load the models once and reuse them: - `ImageCompressor` caches the ONNX router on `self._onnx_router` (built lazily via `_get_onnx_router`) instead of building one per `compress()` call. - The isolation worker keeps a per-process `ImageCompressor` singleton (`_get_worker_compressor`) and reuses it across calls. - `_get_image_compressor()` (main process, used for the `has_images()` gate) returns a shared instance too. - Shared instances are marked `_is_singleton`, and `close()` is a no-op on them, so a caller's per-request `close()` no longer unloads the models the next request reuses. A non-singleton `close()` still releases the torch router and drops the cached ONNX router. RSS is now flat after the initial model load; behavior is otherwise unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/image/compressor.py`: add `_onnx_router` cache + `_get_onnx_router`, use it in `compress()`, add the `_is_singleton` flag, and make `close()` a no-op on a singleton (drop the cached ONNX router on a real close). - `headroom/proxy/image_isolation.py`: reuse a per-worker `ImageCompressor` singleton in `_compress_messages_worker` instead of building/closing one per call. - `headroom/proxy/helpers.py`: `_get_image_compressor()` returns a shared singleton instance. - `tests/test_image_compressor_singleton_reuse.py` (new): the ONNX router is built once and cached, singleton `close()` is a no-op while non-singleton `close()` releases, and both `_get_image_compressor` and the worker helper return a shared singleton. - `tests/test_proxy_handler_helpers.py`: updated the two existing `_get_image_compressor` tests that pinned the old fresh-per-call behavior to assert the singleton reuse instead (and reset the new module global so they stay isolated). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_image_compressor_singleton_reuse.py -q 5 passed # with the fix reverted, all five fail (router rebuilt per call, close() # unloads the shared models, helpers return fresh instances) $ uvx ruff@0.15.17 check headroom/image/compressor.py headroom/proxy/image_isolation.py headroom/proxy/helpers.py tests/test_image_compressor_singleton_reuse.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/image/compressor.py headroom/proxy/image_isolation.py headroom/proxy/helpers.py Success: no issues found in 3 source files ``` The pre-existing async tests in `tests/test_image_compression_isolation.py` (4 `@pytest.mark.asyncio` cases) fail identically on clean `main` in this environment because pytest-asyncio is not configured here; they are unrelated to this change and pass in CI. ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: with `OnnxTechniqueRouter` construction mocked, called `ImageCompressor._get_onnx_router()` twice and asserted a single construction; exercised `close()` on singleton vs non-singleton instances; and called `_get_image_compressor()` / `_get_worker_compressor()` twice each. Then reverted the three source files and re-ran. - Observed result: with the fix the ONNX router is constructed once and reused, singleton `close()` leaves `_router`/`_onnx_router` intact (no `release_models`), non-singleton `close()` releases and nulls them, and both helper accessors return the same `_is_singleton` instance; with the fix reverted every one of these fails (fresh construction / unconditional release / new instances). Ran against the actual modules. - Not tested: a live multi-hour image workload measuring RSS (the leak is inferred from the removed per-request model construction; the ONNX/torch model load itself is mocked here). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
|
09d1ef45be |
fix(proxy): compress Hermes scoped coding-agent passthrough (#1815)
## Description Compress Hermes Studio scoped coding-agent passthrough requests in the generic OpenAI passthrough handler. Hermes can route scoped Claude Code and Codex traffic through Headroom while preserving its own proxy paths; this PR keeps Hermes responsible for scoped proxy authentication/provider adaptation while still applying Headroom compression to supported chat payloads before forwarding. The compression remains narrow-scoped: - Only chat messages with `user` or `assistant` roles are compressed. - Tool, function, reasoning, and system items are preserved byte-stable. - Non-dict items in the Responses `input` array are preserved and spliced back. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Detect `/api/codex-proxy/.../v1/responses` paths and compress supported Responses `input` chat items before forwarding. - Detect `/api/claude-code-proxy/.../v1/messages` paths and compress supported Anthropic `messages` payloads before forwarding. - Preserve bypass, malformed payload, missing-model, tool/function, reasoning/system, and non-dict passthrough behavior. - Add regression coverage in `tests/test_hermes_passthrough_compression.py`. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_hermes_passthrough_compression.py -v test_codex_proxy_preserves_tool_and_function_items PASSED test_codex_proxy_preserves_nondict_items PASSED test_codex_proxy_bypass_header_skips_compression PASSED test_codex_proxy_malformed_input_preserved PASSED test_codex_proxy_compression_applies_to_chat_messages PASSED test_claude_proxy_preserves_tool_use_items PASSED test_claude_proxy_bypass_header_skips_compression PASSED test_claude_proxy_no_model_forwarded_unchanged PASSED test_claude_proxy_compression_applies_to_chat_messages PASSED test_non_hermes_routes_not_affected PASSED ``` ## Real Behavior Proof - Environment: Author-reported local test environment for `headroom/proxy/handlers/openai.py` and `tests/test_hermes_passthrough_compression.py`. - Exact command / steps: `python -m pytest tests/test_hermes_passthrough_compression.py -v`. - Observed result: The 10 Hermes passthrough regression tests passed, covering Codex and Claude scoped proxy routes plus preservation/bypass cases. - Not tested: End-to-end Hermes Studio traffic against a live upstream service is not covered by this PR body evidence. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes Generated with Claude Code. The unchecked checklist items are not required for this narrow proxy-handler test change. --------- Co-authored-by: x1051445024 <你的GitHub注册邮箱> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
9db8a6bbf6 |
fix(proxy): handle ClientDisconnect in passthrough body reads (#2033)
## Description
Catch `starlette.requests.ClientDisconnect` when reading request bodies
in passthrough/forwarding handlers. Closes #2019
Without this, a client that disconnects mid-request causes an unhandled
`ClientDisconnect` to propagate through the entire middleware stack,
crashing the ASGI TaskGroup and contributing to proxy instability over
long sessions (memory growth, freeze, unresponsive to SIGTERM).
**Adversarial review uncovered 3 additional unprotected sites** in
`proxy_routes.py` — same pattern (body read before try/except). Now
fixed.
## Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
## Changes Made
**Proxy handlers** (6 sites, first commit):
- openai `handle_passthrough`: wrap `await request.body()` in try/except
ClientDisconnect (main crash site)
- openai `_handle_streaming_passthrough`: same protection
- anthropic batch passthrough: same protection
- batch `_google_batch_passthrough`: same protection
- batch `handle_google_batch_passthrough`: same protection
- bedrock fallback-forward path: early-return on ClientDisconnect
instead of attempting verbatim forward
**Proxy routes** (3 sites, second commit — found by adversarial design
scan):
- `_handle_chatgpt_model_metadata` (proxy_routes.py:398)
- `_handle_chatgpt_codex_images` (proxy_routes.py:438)
- `openai_responses_sub` nested handler (proxy_routes.py:597)
All nine sites return HTTP 204 on disconnect to allow the request to
terminate cleanly.
## Testing
- [x] **Existing tests**: 34/34 pass in `test_proxy_handler_helpers.py`
- [x] **Unit tests**: 2 new tests — passthrough + streaming passthrough
disconnect
- [x] **Adversarial concurrency**: 50 threads × 10 iterations = 500
concurrent disconnect requests — zero crashes, all return 204
- [x] **Adversarial edge cases**: minimal request state, regression
check (normal request path unaffected)
- [x] **PBT (Hypothesis)**: 250 random method/path combinations, 3
properties verified:
- All disconnect requests return 204
- ClientDisconnect never leaks out of handler
- Response is always valid HTTP 2xx
```text
# Unit tests
tests/test_proxy_handler_helpers.py::test_handle_passthrough_client_disconnect PASSED
tests/test_proxy_handler_helpers.py::test_handle_streaming_passthrough_client_disconnect PASSED
# PBT (3 properties × 100-250 examples each)
/tmp/pbt_client_disconnect.py::test_disconnect_always_returns_204 PASSED
/tmp/pbt_client_disconnect.py::test_disconnect_does_not_crash_asgi PASSED
/tmp/pbt_client_disconnect.py::test_response_is_valid_http PASSED
# Adversarial
/tmp/adversarial_client_disconnect.py → 500 concurrent requests: 0 errors, all 204
```
- [x] `ruff check` and `ruff format --check` pass on all changed files
## Real Behavior Proof
- Environment: Linux, Python 3.12, headroom main @
|
||
|
|
55efb1c77d |
fix(proxy): keep OpenAI tool observations mutable in cache mode (#1884)
## Description Diagnoses and fixes the low-savings OpenAI-compatible cache-mode path reported in #1696. OpenAI-compatible tool-calling clients can end a turn with `role: "tool"` (or legacy `role: "function"`) rather than `role: "user"`. The OpenAI chat handler's cache-mode freeze boundary treated those tails as non-mutable, and because `HeadroomProxy` resolves `_strict_previous_turn_frozen_count` from the Anthropic mixin first, the OpenAI-specific helper was not used in production. That froze the entire conversation before `ContentRouter` ran, leaving no live tool observation to compress and producing near-pass-through savings on long coding sessions. This PR keeps final OpenAI tool/function observations mutable in cache mode, explicitly calls the OpenAI helper to avoid the mixin-name collision, and clamps negative token-savings artifacts at the metrics/cost aggregation boundary so stats cannot under-report actual forwarded savings. Closes #1696 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Treat final OpenAI `user`, `tool`, and `function` messages as the mutable cache-mode live zone. - Route OpenAI cache-boundary calls through `OpenAIHandlerMixin._strict_previous_turn_frozen_count` explicitly so the Anthropic mixin method cannot shadow it in `HeadroomProxy`'s MRO. - Preserve cache-mode live-tail boundaries even when compression-cache state would otherwise freeze the whole request. - Clamp negative `tokens_saved` artifacts in `CostTracker.record_tokens` and `PrometheusMetrics.record_request`. - Add regression coverage for OpenAI final `tool`/`function` tails, over-frozen tracker state, and non-negative savings aggregation. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ maturin build --profile ci --out dist --interpreter python Built wheel for abi3 Python >= 3.10 to dist\headroom_ai-0.29.0-cp310-abi3-win_amd64.whl $ python -m pytest tests\test_proxy_handler_helpers.py tests\test_proxy_openai_cache_stability.py tests\test_observability_metrics.py tests\test_cost_tracker_counterfactual.py 49 passed in 10.27s $ python -m ruff check . All checks passed! $ python -m mypy headroom Success: no issues found in 407 source files $ python -m pytest 53 failed, 7703 passed, 488 skipped, 5893 warnings, 131 errors in 595.18s (0:09:55) ``` Full-suite note: the full local `pytest` run was attempted on Windows/Python 3.13 after building `headroom._core`. It did not complete green due to broad pre-existing/local-environment failures outside this change area, dominated by SQLite/memory persistence permission/path errors plus unrelated adapter/cache/tool tests. The focused regression suite for this PR passes, and repo-level lint/type gates pass. ## Real Behavior Proof - Environment: Windows, Python 3.13.13, Rust/Cargo available, local `headroom._core` wheel built with `maturin build --profile ci`. - Exact command / steps: ran the OpenAI cache-stability tests with final `role: "tool"` and `role: "function"` chat tails. - Observed result: `test_openai_cache_mode_keeps_final_tool_observation_mutable[tool]` and `[function]` pass, proving the pipeline receives `frozen_message_count == 2` for a 3-message request instead of freezing all 3 messages. - Not tested: live Lemonade/KiloCode upstream session; no local Lemonade Server was available. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes Docs and CHANGELOG are N/A for this narrow proxy bug fix. The broad local `pytest` checkbox is intentionally left unchecked because the full suite had unrelated local-environment failures; see the test output above. Focused regression tests, `ruff check .`, and `mypy headroom` are green. |
||
|
|
7c2f0ea079 |
feat(cache): provider-agnostic cache-mode delta + cc-agnostic prefix comparison (#1868)
## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|
|
3076e32172 |
fix(proxy): serve /favicon.ico locally instead of tunneling upstream (#1787) (#1847)
## Description The Headroom dashboard tunnels `GET /favicon.ico` requests to the wrapped upstream provider instead of serving its own. No route matched `/favicon.ico` in `headroom/proxy/server.py`, so the request fell through to the catch-all passthrough route (`headroom/providers/proxy_routes.py:994-1026`) registered by `register_provider_routes(app, proxy)`, and got forwarded to whichever LLM backend the proxy is wrapping — burning a real upstream request (and possibly failing auth) for a browser's automatic favicon fetch while viewing `/dashboard`. Closes #1787 ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/server.py`: added a `GET /favicon.ico` route returning `Response(status_code=204)`, registered next to the existing `/dashboard` route — i.e. before `register_provider_routes(app, proxy)` (line ~4184) registers the passthrough catch-all, so it takes priority. - `tests/test_proxy_handler_helpers.py`: `_PassthroughRequest.url.path` was hardcoded to `/favicon.ico` as a generic "goes to passthrough" example, which encoded the bug as expected behavior. Changed to `/some/other/path` so the passthrough-helper test no longer depends on favicon requests going upstream. - `tests/test_proxy_favicon_route.py` (new): regression test spinning up the real FastAPI app via `create_app`/`TestClient`, asserting `GET /favicon.ico` returns 204 and `proxy.handle_passthrough` is never called. - `CHANGELOG.md`: added an entry under `## Unreleased` / `### Fixed`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_proxy_favicon_route.py tests/test_proxy_handler_helpers.py -q 28 passed $ python -m pytest tests/test_proxy_healthchecks.py tests/test_proxy_passthrough_integration.py tests/test_proxy_favicon_route.py tests/test_proxy_handler_helpers.py -q 41 passed, 19 skipped $ python -m ruff check headroom/proxy/server.py tests/test_proxy_favicon_route.py tests/test_proxy_handler_helpers.py All checks passed! $ python -m ruff format --check headroom/proxy/server.py tests/test_proxy_favicon_route.py tests/test_proxy_handler_helpers.py 3 files already formatted $ python -m mypy headroom/proxy/server.py (no errors) ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, local checkout, `python -m pytest`/`ruff`/`mypy` run directly (no `uv` available in this shell). - Exact command / steps: `python -m pytest tests/test_proxy_favicon_route.py -v` — this test builds the real proxy app with `create_app(ProxyConfig(...))`, wraps `client.app.state.proxy.handle_passthrough` with a mock, then issues `client.get("/favicon.ico")` via a real `TestClient` request through the full FastAPI routing stack (not a unit-level call of the handler function directly). - Observed result: response status is `204`, and `handle_passthrough` (the function that forwards to the upstream provider) is asserted `not_called()` — confirming the request is now intercepted before reaching the catch-all passthrough route, and does not tunnel to the wrapped provider. - Not tested: did not manually run `headroom wrap <provider>` end-to-end and open a real browser tab to `/dashboard` to visually confirm the favicon icon in the tab (the fix returns 204/no-icon rather than a real bundled `.ico` — browsers handle this fine, but the visual "no more broken/upstream favicon request" experience wasn't screenshotted). The FastAPI-level test above exercises the actual routing/dispatch path this bug lived in. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A — no user-facing docs describe dashboard route internals beyond CHANGELOG) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated CHANGELOG.md where applicable ## Screenshots (if applicable) N/A — server-side route change, no UI change. ## Additional Notes Deliberately kept the fix minimal: no `StaticFiles` mount or general static-asset serving system was added, since a single favicon route doesn't warrant that abstraction. No real `.ico` binary asset was bundled either — a `204 No Content` response is sufficient for browsers and avoids maintaining a binary asset in the repo; this can be upgraded to serve a real branded icon later if desired. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
248ae0f3e0 |
fix(proxy): freeze must forward cached (compressed) prefix byte-identical — stop token-mode cache busting (#1850)
The freeze path (both providers) emits the agent's ORIGINAL bytes for a frozen message, but the provider cached whatever we FORWARDED last turn (the compressed form). Forwarding original then mismatches the cached prefix and busts it from that point — re-creating the whole suffix. Measured on a real SWE-bench run: 100% of attributed misses were prefix_change, ~56% of ALL cache-writes were bust-induced (2.8M tokens), driving cache_create +150% and cost +41% vs baseline. Cache mode already avoided this via _extract_cache_stable_delta (replay the previously-forwarded prefix, compress only the delta). Token mode called apply(frozen_count) directly, which forwards original for the frozen region. Fix: add a shared, provider-agnostic overlay_cached_prefix() that replays the previously-forwarded (cached, compressed) prefix byte-identical, append-only guarded and idempotent, and apply it in BOTH the Anthropic and OpenAI handlers right before forwarding. This makes freezing byte-identical in every mode, so the only remaining difference between "token" and "cache" mode is how large a mutable (still-compressible) tail each leaves — not whether the frozen prefix busts the cache. Tests: - test_cache_prefix_overlay.py: the helper (replay, append-only guard, idempotence). - test_cross_turn_cache_safety.py: the invariant that was missing — drive the REAL tracker + freeze + overlay over multiple append-only turns against a simulated provider prefix cache and assert the forwarded prefix stays byte-identical turn-over-turn. Load-bearing: it fails (detects the bust) without the overlay. ## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|
|
da2d8dc9db |
fix(proxy): cancel retry backoff on shutdown (#1834)
## Description During proxy shutdown, an in-flight retrying request can currently stay asleep inside `_retry_request()` and keep the client socket hanging until the retry timer expires or an external supervisor kills the process. This wires retry backoff to a proxy-scoped shutdown event so shutdown interrupts those waits immediately and returns a clear `503` response instead of leaving the request stalled. Closes #1821. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a proxy-scoped shutdown event in `headroom/proxy/server.py`. - Cleared that event at startup and set it at shutdown before teardown proceeds. - Replaced both retry-backoff sleeps with a helper that wakes on either timeout or shutdown. - Returned a shutdown `503` with `retry-after: 0` when shutdown interrupts retry backoff. - Stopped the shutdown interruption logs from falling back to the raw upstream URL when no safe path string is available. - Added focused regressions for retry-backoff interruption and shutdown event signaling. - Updated the existing Retry-After tests to observe the new shutdown-aware wait helper instead of the old raw sleep hook. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy_handler_helpers.py tests/test_proxy_pipeline_lifecycle.py -q`) - [x] Unit tests pass (`uv run pytest tests/test_proxy_retry_429.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/server.py tests/test_proxy_handler_helpers.py tests/test_proxy_retry_429.py tests/test_proxy_pipeline_lifecycle.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_handler_helpers.py tests/test_proxy_pipeline_lifecycle.py -q 32 passed, 1 warning in 13.05s uv run pytest tests/test_proxy_retry_429.py -q 10 passed, 1 warning in 1.12s uv run ruff check headroom/proxy/server.py tests/test_proxy_handler_helpers.py tests/test_proxy_retry_429.py tests/test_proxy_pipeline_lifecycle.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, project `uv` environment, focused proxy retry and shutdown regressions. - Exact command / steps: copy the updated shutdown regression files into a detached `origin/main` worktree and run `tests/test_proxy_handler_helpers.py` plus `tests/test_proxy_pipeline_lifecycle.py`, then rerun those files on this branch and separately rerun `tests/test_proxy_retry_429.py` after updating the existing Retry-After tests to patch the shutdown-aware wait helper. - Observed result: base fails because retry backoff still returns the original `429` and `shutdown()` leaves the retry event unset; head passes the focused file, preserves the existing Retry-After assertions, and returns a shutdown `503` with `retry-after: 0` while signaling retry waiters during shutdown. - Not tested: live systemd-managed shutdown on Linux or a full VS Code / Claude Code session. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes This is intentionally scoped to retry backoff during shutdown. It does not try to cancel unrelated in-flight request work or change the broader retry policy outside shutdown. |
||
|
|
a9322477e3 |
fix: preserve anthropic passthrough tool order (#1427)
## Description Preserves Anthropic `tools` order when Headroom is forwarding a passthrough/no-optimize request. This fixes a Claude Code style `tool_result` continuation failure against stricter Anthropic-compatible upstreams that treat the client's original tool ordering as part of the conversation state. Closes #1417 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Preserve client-provided Anthropic `tools` order when `optimize=False` or the request is explicitly in Headroom passthrough/bypass mode. - Keep deterministic tool sorting for optimized requests where Headroom may rewrite the body for cache stability. - Avoid sorting batch-request tools before the no-optimize passthrough branch. - Add regression coverage for the Anthropic HTTP path to prove no-optimize forwarding keeps `Read`, then `Bash` tool order. - Update existing cache-stability and byte-faithful forwarding tests so no-optimize/passthrough expects preserved client order while optimized mode still proves deterministic sorting. ## Testing - [x] Focused unit tests pass (`pytest` on touched proxy test files) - [x] Linting passes (`ruff check` and `ruff format --check` on touched files) - [x] Type checking passes (`mypy headroom`) - [x] New regression tests added - [x] Manual testing performed ### Test Output ```text $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with pytest --with pytest-asyncio --with anyio --with 'httpx[http2]' --with fastapi --with pydantic --with tiktoken --with click --with rich --with opentelemetry-api --with opentelemetry-sdk --with zstandard --with openai --with mcp --with uvicorn pytest tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py -q ============================= test session starts ============================== platform darwin -- Python 3.13.11, pytest-9.1.1, pluggy-1.6.0 rootdir: /Users/vinaygupta/Desktop/git/headroom-fix-anthropic-tool-order configfile: pyproject.toml plugins: anyio-4.14.1, asyncio-1.4.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 87 items tests/test_proxy_handler_helpers.py .......................... [ 29%] tests/test_anthropic_stage_timings.py .... [ 34%] tests/test_proxy_anthropic_cache_stability.py ......................... [ 63%] tests/test_proxy_byte_faithful_forwarding.py ........................... [ 94%] ..... [100%] =============================== warnings summary =============================== .../fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. ======================== 87 passed, 1 warning in 5.13s ========================= $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py 5 files already formatted $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS, Python 3.13.11, local fake Anthropic-compatible upstream, local Headroom proxy launched with `--no-optimize --no-cache --no-rate-limit --stateless`. - Exact command / steps: ran a local reproduction harness that starts a fake `/v1/messages` upstream and Headroom proxy, then sends a Claude Code style two-turn flow: first assistant `Bash` `tool_use`, then user `tool_result`. - Observed result: after this patch, both direct and proxied flows returned `200` for `first_tool_use` and `second_tool_result`. The fake upstream log showed the proxied `tools` array remained `["Read", "Bash"]` on both turns. ```text DIRECT first_tool_use: 200 second_tool_result: 200 PROXIED first_tool_use: 200 second_tool_result: 200 UPSTREAM REQUEST LOG proxied first turn tools: ["Read", "Bash"] proxied tool_result turn tools: ["Read", "Bash"] ``` - Not tested: full `pytest`, full-repo `ruff check .`, `mypy headroom`, or a live third-party Anthropic-compatible provider. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - This PR intentionally does not add documentation because it fixes passthrough behavior rather than introducing a new user-facing option. - The code-comment checklist item is left unchecked because the change is covered by a small helper docstring and regression tests; no extra inline comments seemed necessary. - `CHANGELOG.md` is left unchanged because this is a narrowly scoped bug fix. - Local pytest collection for these proxy tests required a local `headroom._core` extension symlink, which was removed before committing. |
||
|
|
3be2526b76 |
fix(proxy): add an Anthropic buffered read-timeout override (#1331)
## Description Buffered Anthropic `/v1/messages` requests still use Headroom's generic 300-second read timeout, which can produce proxy-generated `502 ReadTimeout` errors on long turns. This adds a dedicated buffered Anthropic timeout, keeps it applied across CCR and memory continuations plus batch paths, and makes the direct server entrypoint enforce the same positive-integer contract as the Click CLI. Closes #1261. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `anthropic_buffered_request_timeout_seconds` for buffered Anthropic reads. - Routed `/v1/messages`, CCR continuation, memory continuation, batch create, batch passthrough, and batch results through that timeout. - Enforced the same positive-integer validation for `HEADROOM_ANTHROPIC_BUFFERED_REQUEST_TIMEOUT_SECONDS` and `--anthropic-buffered-request-timeout-seconds` in both startup paths. - Added focused regressions and updated `CHANGELOG.md`. ## Testing - [x] `uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring` - [x] `uv run ruff check .` - [x] `uv run ruff format . --check` ### Test Output ```text $ uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring 17 passed in 3.42s $ uv run ruff check . All checks passed! $ uv run ruff format . --check 966 files already formatted ``` ## Real Behavior Proof - Environment: local FastAPI `TestClient` with stubbed retry and HTTP client seams - Exact command / steps: run `uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring`; the tests build `ProxyConfig(request_timeout_seconds=7, connect_timeout_seconds=3, anthropic_buffered_request_timeout_seconds=19)`, drive `/v1/messages`, `/v1/messages/batches`, `/v1/messages/batches/{batch_id}/results`, a CCR continuation, and a memory continuation through `TestClient`, then verify `HEADROOM_ANTHROPIC_BUFFERED_REQUEST_TIMEOUT_SECONDS=0` falls back to `600`, `--anthropic-buffered-request-timeout-seconds 0` is rejected, and default proxy timeouts stay `read=300` and `write=300` - Observed result: buffered Anthropic paths use `httpx.Timeout(connect=3, read=19, write=7, pool=3)`, continuation requests stay on that same budget, invalid zero-valued startup config is rejected or ignored back to the default, and unrelated proxy timeout defaults stay unchanged - Not tested: live upstream Anthropic latency beyond the focused stubbed-timeout regression ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have added tests that prove the fix - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable |
||
|
|
8c00f7103c |
fix(codex): poll /wham/usage for subscription limits (handshake no longer sends x-codex-* headers) (#924)
## Description Codex's subscription usage window (primary/secondary rate-limit gauges) stopped populating for ChatGPT-OAuth sessions. This PR restores it by polling Codex's dedicated usage endpoint instead of relying on response headers that are no longer sent. ### Why the previous approach no longer works The existing code populates `CodexRateLimitState` from `x-codex-*` rate-limit headers captured on the `/v1/responses` WebSocket handshake (`update_from_headers` at WS accept). That worked when OpenAI returned `x-codex-primary-used-percent`, `x-codex-primary-window-minutes`, etc. on the handshake response. OpenAI has since stopped sending those headers on the ChatGPT WebSocket handshake. I confirmed this by faithfully replaying a real Plus-account handshake (both `prewarm` and regular `request_kind`): no `x-codex-*` headers come back on either. This matches OpenAI's own move to a dedicated usage endpoint (`GET /backend-api/codex/usage` in CodexApi mode) and reports such as openai/codex#14728. So `update_from_headers` now runs on every accept but finds nothing to parse, and the window silently stays empty. The headers aren't coming back, so there is nothing to fix in the parsing path. The data now lives behind a request we have to make ourselves. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `subscription/codex_rate_limits.py`: - `parse_codex_usage_payload()` / `update_from_usage_payload()` — map the `GET /backend-api/wham/usage` JSON (`rate_limit.primary_window` / `secondary_window` with `used_percent`, `limit_window_seconds`, `reset_at`; `credits`; `rate_limit_reached_type`) into the existing `CodexRateLimitState`. `limit_window_seconds` is converted to window-minutes with the same round-up codex-rs uses (`(secs + 59) // 60`). - `maybe_schedule_usage_poll()` — fire-and-forget, throttled to one request per 60s, scoped to ChatGPT sessions (requires both a Bearer token and `ChatGPT-Account-Id`; API-key traffic is skipped). Uses an in-flight guard so concurrent accepts don't stack polls. Endpoint is overridable via `HEADROOM_CODEX_USAGE_URL`. - `proxy/handlers/openai.py`: - At the Codex WS accept site, after the now-usually-empty `update_from_headers` block, schedule the usage poll. Wrapped in `contextlib.suppress` and fully non-blocking so it can never delay or fail the WebSocket accept. The old header-capture path is intentionally left in place as a no-cost fallback in case OpenAI restores the headers. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (live `/wham/usage` replay against a Plus account returned HTTP 200 with the expected schema; payload fixture in tests mirrors that real shape) ## Test Output ``` $ uv run pytest tests/test_codex_rate_limits.py -q ........................................ [100%] 41 passed in 0.16s $ uv run ruff check headroom/subscription/codex_rate_limits.py headroom/proxy/handlers/openai.py tests/test_codex_rate_limits.py All checks passed! $ uv run mypy headroom/subscription/codex_rate_limits.py Success: no issues found in 1 source file ``` ## Additional Notes - New tests cover: full-payload mapping, window-minutes round-up, credits balance kept only when `has_credits`, promo object vs string, empty payload returns `None`, missing `used_percent` skipped, header-gating (requires Bearer + account-id), poll throttling, and no-event-loop safety. - Scoping to `ChatGPT-Account-Id` keeps the poll off API-key traffic, and the 60s throttle plus in-flight guard bound it to at most one lightweight GET per minute per running proxy. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
3c77e52ce4 |
feat: add Vertex AI proxy routing (#793)
## Description Adds first-class GCP Vertex AI proxy routing for publisher REST endpoints so Vertex requests are forwarded to a configurable regional Vertex host instead of falling through to the generic OpenAI/Anthropic/Gemini passthrough selection. Fixes #792 ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Documentation update ## Changes Made - Added a `vertex` provider target with `VERTEX_TARGET_API_URL` and `--vertex-api-url` support. - Registered explicit Vertex publisher routes for Google `generateContent`, `streamGenerateContent`, `countTokens` and Anthropic publisher `rawPredict`, `streamRawPredict` passthrough. - Added startup banner/routing output for Vertex AI. - Added focused tests for provider target resolution, CLI/env config, banner output, and route delegation. - Added `wiki/vertex.md` with usage examples and Google Cloud source links. ## Sources - Vertex AI Gemini inference reference: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference - Google Cloud REST authentication: https://docs.cloud.google.com/docs/authentication/rest - Google Application Default Credentials: https://docs.cloud.google.com/docs/authentication/application-default-credentials ## Testing - [x] Linting passes (`python -m ruff check .`) - [x] New tests added for new functionality - [x] Focused unit tests pass - [ ] Full unit suite completed locally - [ ] Rust tests completed locally - [ ] Type checking passes locally ## Test Output ```text $ python -m ruff check . All checks passed! $ python -m pytest tests/test_provider_registry.py tests/test_provider_proxy_routes.py tests/test_cli_proxy_env.py tests/test_banner_upstream_targets.py -q 57 passed, 1 warning in 13.75s ``` Local limitations: - `python -m pytest tests scripts/tests -q` timed out after 1 hour on this Windows machine before completing. - `cargo test -p headroom-proxy --test integration_vertex_raw_predict` could not run because `cargo` is not installed on PATH in this environment. - The commit hook's `mypy` step fails locally on an existing Windows `fcntl` typing issue in `headroom/subscription/tracker.py`; `ruff`, `ruff-format`, and plugin-version hooks passed, and the commit was made with only `mypy` skipped. ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have made corresponding changes to the documentation - [x] I have added tests that prove the feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
|
1bc163f5bc |
fix(ccr): scope proactive expansion by workspace (cross-project leak)
Closes the cross-project context leak Jocelyn reported 2026-05-26:
working on a Ruby/Rails project (daphni-rails), an unrelated Python
file (an Ollama inference provider from project `tamag0`) was being
injected into context as "Proactive Context Expansion - relevant to
your query". Two completely different projects, two different
languages, two different working directories — but the same proxy
process was serving both, and the in-memory ContextTracker had no
workspace identity to filter on.
Root cause
----------
`self.ccr_context_tracker` is one instance per proxy process. Every
session, every project, every user shared the same `_contexts` dict.
`track_compression()` stored sample content with no provenance key;
`analyze_query()` ran lexical keyword overlap across the full dict
without filtering. Within the 5-minute age window, surface-level
token matches ("provider", "session", "oauth", generic code/test
structure) scored above the 0.3 relevance threshold, recommendations
came back, and execute_expansions() injected the full original
content into a foreign session.
Refuted: this is NOT a race condition (joce's hypothesis). It
reproduces single-threaded, one-request-at-a-time. Plain shared
mutable state.
Fix
---
Add a required `workspace_key` to the tracker API and filter on it
inside `analyze_query`:
1. `CompressedContext` gets a `workspace_key: str` field.
2. `track_compression(..., workspace_key=...)` is now keyword-only,
no default — fail-loud on missing.
3. `analyze_query(..., workspace_key=...)` is also keyword-only; an
empty workspace_key short-circuits to `[]` (fail-closed per
`feedback_no_silent_fallbacks`).
4. The loop at `analyze_query` skips any entry whose workspace_key
differs from the request's.
In the Anthropic proxy handler:
5. New `_resolve_ccr_workspace(request, body)` static helper uses the
memory subsystem's `ProjectResolver` so CCR and memory agree on
project identity. Tier order: x-headroom-project-id →
x-headroom-cwd → CLI override → cwd: line in system prompt.
6. Both track and analyze sites gate on `ccr_workspace_key` being
non-empty — turning off proactive expansion entirely when project
identity can't be resolved is the safest default (it's an
optimization, not correctness).
7. `format_expansions_for_context(expansions, workspace_label=...)`
was already wired (GH #462 Fix C); the call site now passes the
label so the injected block declares its provenance, symmetric
with the memory injection header.
Affected population
-------------------
- Default mode (no `--cache`): bug fixed.
- Cache mode: was never affected — proactive expansion short-
circuits in cache mode to preserve prefix stability.
Tests
-----
- 6 new workspace-scoping tests in `test_ccr_context_tracker.py`:
same-workspace match still works, cross-workspace silently
filtered, empty workspace_key fail-closes, two workspaces each
see only their own, workspace_label propagates to formatter, LRU
cross-workspace doesn't leak even with full tracker.
- 6 new `_resolve_ccr_workspace` resolver tests in
`test_proxy_handler_helpers.py`: explicit project-id wins, cwd
header → key+label, two cwds get distinct keys, no-signal
fail-closed, system-prompt cwd: fallback, malformed request
fail-closed.
- 32 existing tracker tests updated to pass `workspace_key="ws-test"`.
- 55/55 tests pass; ci-precheck green.
Defense-in-depth follow-up
--------------------------
The compression_store itself (`headroom/cache/compression_store.py`)
also lacks workspace scoping — a CCR `headroom_retrieve` call from
Project B for a hash created by Project A would succeed. The
practical attack surface is closed by this PR (hashes only reach
Project B's model via proactive expansion, now gated), but
defense-in-depth hardening of the store is worth a separate PR.
Filed as task #44.
|
||
|
|
160989c43e | fix(proxy): bound Codex Responses compression work | ||
|
|
eaf5980b4a | fix: stabilize codex compression, stats, and proxy lifecycle | ||
|
|
f0dcc02775 |
fix: A3 — byte-faithful Python forwarders; serialize canonical only when mutated
Eliminates P0-2 universally. Every Python forwarder (server.py
`_retry_request`, handlers/streaming.py `_stream_response`,
handlers/openai.py `_ws_http_fallback`, handlers/batch.py `_batch_passthrough`
+ batch-create + Google batch passthrough, handlers/anthropic.py CCR
continuation + batch endpoint) now switches from `httpx ... json=body` to
`httpx ... content=raw_bytes`. The default httpx JSON encoder was
re-serializing every request with `, `/`: ` separators and `\\uXXXX` ASCII
escapes — collapsing Anthropic prompt-cache hit-rate.
Forwarder strategy:
- unmutated body → forward `await request.body()` verbatim;
- mutated body → re-serialize once via the new
`serialize_body_canonical(body) -> bytes` helper (compact separators,
`ensure_ascii=False`, dict insertion order preserved).
`HEADROOM_PROXY_PYTHON_FORWARDER_MODE` env var configures the mode:
- `byte_faithful` (default) — the new behavior;
- `legacy_json_kwarg` — explicit operator opt-in for emergency rollback.
Documented in `docs/content/docs/configuration.mdx`. NOT a fallback —
unknown values raise loudly per build constraint #4.
`BodyMutationTracker` accompanies each request through the handler so
transform sites mark the tracker (`memory_injection`,
`image_compression`, `compression_*`, `batch_compression`,
`ccr_continuation`, etc.). At forwarder dispatch we additionally compare
the final body dict against the parsed original bytes as a structural
safety net — any silent mutation we missed still triggers canonical
re-serialization.
A2 follow-up: `handlers/openai.py:534-540` (Chat Completions memory
injection) was prepending a system message; replaced with
`append_text_to_latest_user_chat_message`, the OpenAI Chat Completions
analog of `_append_context_to_latest_non_frozen_user_turn`. The cache
hot zone (system messages) is now sacrosanct on /v1/chat/completions
too. Honors `HEADROOM_MEMORY_INJECTION_MODE=disabled`.
Structured logging: every forwarder emits an `event=outbound_request`
log line with `forwarder`, `path`, `body_bytes`, `body_mutated`,
`mutation_reasons`, `source` (passthrough|canonical|legacy),
`request_id`. Never logs Authorization or full body.
`_read_request_json` factored to share `_read_request_body_bytes` with
new `read_request_json_with_bytes` so the anthropic handler can capture
both the parsed dict and the original (decompressed) bytes.
Tests:
- `tests/test_proxy_byte_faithful_forwarding.py` (28 tests):
SHA-256 byte-equality on /v1/messages and streaming, unicode
preservation, numeric precision, mutation-tracker invariants,
canonical-serializer properties, legacy-mode rollback, OpenAI
Chat memory routing.
- Existing test mocks updated to accept the new `**kwargs` on
`_retry_request` (no behavior change).
- `tests/test_proxy_handlers_batch.py` updated to read the captured
`content=` bytes (formerly `json=`).
- One A2 test corrected (`test_anthropic_tool_sort_and_context_append_helpers`)
to match the live-zone-tail semantics introduced by A2.
Constraints satisfied: configurable env var; no new regex / hardcodes;
no silent fallback (`legacy_json_kwarg` is operator opt-in);
performant (`prepare_outbound_body_bytes` is O(1) for passthrough);
elegant single-responsibility helpers; structured tracing logs.
|
||
|
|
cf60882949 | fix: release image router models after compression | ||
|
|
efd2ac1ca4 |
chore: renormalize line endings to LF
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but 74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings, violating that contract. Every macOS/Linux clone reports these files as "modified" on fresh checkout because git's diff engine sees the stored bytes don't match the attribute contract, even though the working tree and index match byte-for-byte. Running `git add --renormalize .` rewrites each affected blob so the stored form matches the attribute declaration. No semantic changes — every affected file's diff is "N insertions, N deletions" with inserts and deletes being the same lines modulo line endings. Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub blame skip this mechanical commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
1b377d8c43 |
test: add focused pipeline coverage
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |