mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description The buffered CCR path flips a streaming Claude Code turn to `stream: false` upstream so the proxy holds the whole reply before serving `headroom_retrieve`. With `stream: false`, Anthropic sends no byte until the generation is complete. A middlebox that drops HTTP connections idle for ~60s (TLS-inspecting antivirus, VPN, corporate proxy; TCP keepalive does not reset an HTTP-layer idle timer) closes the connection first: each attempt fails at ~60s with `Server disconnected without sending a response`, the client gets a 502 after ~186s, and Claude Code resends the same body forever. The CCR continuation that serves the retrieval (`anthropic_ccr_continuation`, its own `http_client.post`) is a second `stream: false` call and is cut the same way. Without Headroom the client streams and the same turns complete. This PR keeps the buffered design and makes only its upstream leg stream. The buffered request and every continuation built from it (CCR, memory, turn hook) go to first-party Anthropic as `stream: true`, and the event stream is folded back into the Message JSON the buffered path already reads, so retrieval, memory, turn hooks, usage accounting and SSE resynthesis see exactly what they saw before. Bytes (deltas and pings) now flow for the whole generation, so an idle timer never fires. The fold reuses `AnthropicSSEEnvelope`, the reconstruction #3130 and #3266 already use on this path. Closes #4205 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/anthropic.py`: new `_send_buffered_as_stream(send, headers, body, *, url, request_id)`. It sends `{**body, "stream": true}` with `Accept: text/event-stream` (one Accept value, any casing replaced) and folds the reply: - not a 200 event stream (an HTTP error, or an upstream that answered JSON anyway): returned unchanged; - a whole stream: a 200 JSON response holding the folded Message; - an in-band `error` event: the event as the body, with its own HTTP status (529 `overloaded_error`, 429 `rate_limit_error`, other documented types to their status, else 500); - a stream that ended before `message_stop` or with a block open: the existing strict 502 `upstream_protocol_error` of `_adapt_event_stream_to_json` (#3266); - a complete stream holding a known event the fold cannot read (an unknown delta type): logged, and the original `stream: false` body is sent instead. One more generation, but never a 502 the client would resend forever. A connection dropped mid-stream raises inside `send`, exactly as for a `stream: false` reply. An in-band error arrives after upstream already answered 200, so `_retry_request` does not retry it; the client gets the status and retries as it would on a direct stream. - It is used at the four places Headroom sends a body it made non-streaming: the buffered CCR request (only when `buffered_stream_ccr`; a client's own `stream: false` request still goes through `_retry_request` with its original bytes), the CCR continuation (the `prepare_outbound_body_bytes` + `log_outbound_request` + `http_client.post` sequence moved into a local `_post_continuation`, unchanged otherwise, so a fallback attempt logs its own outbound line), the memory continuation and the turn-hook continuation. - Gates: the body is not already streaming, the upstream is first-party Anthropic (`is_custom_anthropic_base_url(url)` is false; the host-equality check tool-search gating already uses), and the kill switch `HEADROOM_BUFFERED_UPSTREAM_STREAM` is not `0`/`false`/`no`/`off`. Copilot and other gateways keep the `stream: false` request: the fold is checked against Anthropic's own stream, and a gateway dialect it cannot read would cost every turn a second generation. - `headroom/proxy/runtime_env.py`: `HEADROOM_BUFFERED_UPSTREAM_STREAM` is a registered knob in `RUNTIME_ENV_KNOBS`, read per request through `runtime_env.getenv`, so a reusing `headroom wrap` pushes it to the running proxy and `POST /admin/runtime-env` sets it with no restart. Listed in the `security-model.mdx` allowlist. - `headroom/proxy/anthropic_wire.py`: `fold_anthropic_stream(raw)` parses with `AnthropicSSEEnvelope.parse` and trims each block to the non-streaming Message shape: a null member leaves every block, `thinking` keeps `type`/`thinking`/`signature`, `redacted_thinking` keeps `type`/`data`. Claude Opus 5.5 streams a thinking `content_block_start` with `estimated_tokens: null`; the CCR continuation resends the folded turn as history, and Anthropic answers any block carrying that member with 400 `Extra inputs are not permitted`. - `headroom/proxy/anthropic_wire.py` `_response_from_events`: `message_delta.usage` is merged only where non-null, as the Anthropic SDK's `MessageStream` does. Before, a null count there overwrote the real `message_start` count (input and cache tokens) in the reconstructed message. - `headroom/proxy/buffered_ccr_response.py`: the status-to-error-type table moves out of `_anthropic_error_type` into `ANTHROPIC_ERROR_TYPES` so the handler can invert it; `_anthropic_error_type` returns the same values. - Unchanged on purpose: `body["stream"] = False` and its mutation reason `ccr_streaming_retrieve_buffered_non_stream` stay the buffered path's internal representation (every later step, the response-cache guard and the #3587 log counts key on them), and the `using buffered stream:false upstream request` log line keeps its text. - Tests: new `tests/test_buffered_ccr_upstream_stream.py` (16 tests): the fold equals the Anthropic Python SDK's `accumulate_event` result on a stream with thinking, signature, redacted thinking, text, citations, split `input_json_delta`, pings and a null usage count; stream-only members are dropped; truncated, in-band error and unknown-delta outcomes; the send helper's gates (kill switch, gateway URL, already streaming), passthrough of non-stream replies, 529 for an in-band overload, 502 for a truncated stream, the `stream: false` fallback; and through the handler, the buffered turn and its CCR continuation both go upstream `stream: true` with `Accept: text/event-stream` and the continuation's resent thinking block carries only its schema, while a client's own `stream: false` request goes out `stream: false` with `Accept: application/json` and its original bytes. - Existing tests that pinned `stream: false` on the wire for a buffered turn now pin `stream: true` (`test_anthropic_streaming_ccr_retrieve.py`, `test_ccr_buffered_stream_signed_thinking.py`, `test_proxy_response_cache_replay.py`, `test_no_ccr_disables_response_handling.py`). `test_anthropic_buffered_ccr_usage.py` now answers the buffered turn with an event stream, so usage booking is checked through the fold. `test_buffered_ccr_accept_header.py` (#3078) pins the `stream: false` leg it is about with the kill switch; custom upstreams such as Copilot always take that leg. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - ran on the 3 changed source files only, see below - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_buffered_ccr_upstream_stream.py -q 16 passed, 1 warning in 2.89s # every test file that touches the buffered CCR path, the CCR continuation or SSE reconstruction $ pytest tests/test_buffered_ccr_upstream_stream.py tests/test_sse_safeguards_passthrough.py \ tests/test_buffered_ccr_salvage.py tests/test_anthropic_buffered_ccr_usage.py \ tests/test_buffered_ccr_grace_window.py tests/test_anthropic_buffered_sse.py \ tests/test_ccr_buffered_stream_signed_thinking.py tests/test_toin_feedback.py \ tests/test_proxy_response_cache_replay.py tests/test_proxy/test_anthropic_buffered_timeout.py \ tests/test_anthropic_auto_mode_passthrough.py tests/test_proxy/test_openai_responses_ccr.py \ tests/test_sse_thinking_blocks.py tests/test_streaming_sse_malformed_events.py \ tests/test_route_advice.py tests/test_memory_tool_stream.py \ tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_proxy_byte_faithful_forwarding.py \ tests/test_buffered_ccr_accept_header.py tests/test_anthropic_buffered_memory_continuation.py \ tests/test_anthropic_ccr_workspace_unbound.py tests/test_no_ccr_disables_response_handling.py \ tests/test_proxy_ccr.py tests/test_ccr_inline_resolve_handlers.py \ tests/test_proxy/test_anthropic_ccr_raise.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py \ tests/test_nonstream_sse_policy.py tests/test_ccr_response_handler_extra.py \ tests/test_compress_route_tokenizer_by_model.py -q 855 passed, 6 warnings in 57.94s # the updated assertions fail on main's source (they pin stream:true on the wire) $ git checkout upstream/main -- headroom/ && pytest tests/test_anthropic_buffered_ccr_usage.py \ tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_ccr_buffered_stream_signed_thinking.py \ tests/test_no_ccr_disables_response_handling.py tests/test_proxy_response_cache_replay.py -q --tb=no 12 failed, 33 passed, 1 warning in 13.64s # whole suite $ pytest tests -q 18 failed, 16435 passed, 583 skipped, 119 warnings in 1226.31s (0:20:26) # 16 of the 18 fail the same way on unmodified main in the same environment # (tests/test_proxy_uds.py x11, tests/test_cli/test_wrap_persistent.py x4, # tests/test_pricing_from_litellm.py groq/llama-guard-3-8b x1); the other 2 # (test_offline_egress_chokepoint.py::TestUpdateCheckOffline::test_explicit_update_check_never_constructs_request, # test_stateless_no_disk.py::test_cli_stateless_flag_exports_env_for_children) pass when run alone on this branch. $ ruff check . All checks passed! $ ruff format --check . 1790 files already formatted $ mypy headroom/proxy/anthropic_wire.py headroom/proxy/buffered_ccr_response.py headroom/proxy/handlers/anthropic.py Success: no issues found in 3 source files ``` ## Real Behavior Proof - Environment: macOS 27.0.1 arm64, Python 3.11.13. Upstream `main`f1bc2609versus this branch, same venv. - Exact command / steps: `repro_buffered_idle_cut.py` (full script in #4205) runs the real proxy app (`create_app(ProxyConfig(...))` + `TestClient`, real `_retry_request`, real `http_client`) against a fake Anthropic upstream that takes 5s per reply (stream: a delta or ping every 0.5s; non-stream: silent for 5s, then JSON), reached through a TCP relay that closes any connection silent for 2s, a scaled-down 60s middlebox. Scenario A: a streaming turn with a redeemable marker (buffered path), slow reply. Scenario B: the model calls `headroom_retrieve` in 0.3s and the continuation is slow. The script points the first-party host check at the fake upstream and isolates `HOME` and the savings ledger. `PYTHONPATH=<checkout> python repro_buffered_idle_cut.py 2>/dev/null` - Observed result: on main both scenarios fail and every upstream request goes out `stream: false` and is cut; on the branch both complete in about the generation time with no cut; with `HEADROOM_BUFFERED_UPSTREAM_STREAM=0` on the branch the main behaviour returns. ```text # mainf1bc2609Scenario A (slow buffered turn) client: HTTP 200 after 10.1s -> FAIL: 20 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "An error occurred while processing your request. Please try again."}} upstream requests: ['turn stream=false', 'turn stream=false', 'turn stream=false'] connections the relay cut for idling: 3 Scenario B (slow CCR continuation) client: HTTP 502 after 2.3s -> FAIL: 0 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "Unable to safely complete streamed CCR retrieval."}} upstream requests: ['retrieve call stream=false', 'continuation stream=false'] connections the relay cut for idling: 1 # proxy log on main WARNING - Request failed (attempt 1), retrying in 1322ms: Server disconnected without sending a response. WARNING - Request failed (attempt 2), retrying in 2536ms: Server disconnected without sending a response. ERROR - [hr_1791973393_000001] Request failed: RemoteProtocolError: Server disconnected without sending a response. ERROR - CCR: Continuation API call failed: RemoteProtocolError: RemoteProtocolError('Server disconnected without sending a response.') WARNING - [hr_1791973403_000002] CCR: Buffered streaming response still contains an unresolved headroom_retrieve after handling; failing closed # this branch Scenario A (slow buffered turn) client: HTTP 200 after 5.2s -> OK: the client got the final answer upstream requests: ['turn stream=true'] connections the relay cut for idling: 0 Scenario B (slow CCR continuation) client: HTTP 200 after 5.3s -> OK: the client got the final answer upstream requests: ['retrieve call stream=true', 'continuation stream=true'] connections the relay cut for idling: 0 # this branch, HEADROOM_BUFFERED_UPSTREAM_STREAM=0 Scenario A (slow buffered turn) client: HTTP 200 after 9.0s -> FAIL: 16 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "An error occurred while processing your request. Please try again."}} upstream requests: ['turn stream=false', 'turn stream=false', 'turn stream=false'] connections the relay cut for idling: 3 Scenario B (slow CCR continuation) client: HTTP 502 after 2.4s -> FAIL: 0 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "Unable to safely complete streamed CCR retrieval."}} upstream requests: ['retrieve call stream=false', 'continuation stream=false'] connections the relay cut for idling: 1 ``` - Not tested: this branch has not sent traffic to api.anthropic.com. The fold is checked against the Anthropic Python SDK's own accumulator and against hand-built streams, not against live replies. For context, the same transport change (a different fold implementation with the same contract) has run as a downstream patch on the 0.39.0 wheel and was checked live: 56 turns across Opus, Sonnet, Haiku and Fable models (thinking, parallel `tool_use`, `web_search` with citations, `max_tokens` cutoffs) matched the non-streaming reply's top-level, block and usage shapes; 28 folded replies resent as history all returned 200; behind a relay that cuts a connection after 20s of silence, a 409s thinking turn completed (longest silence 2.1s) while the `stream: false` path looped on 502s. The OpenAI Responses buffered path has the same flip and is not changed here. Gateways (Copilot, custom base URLs) are not changed. ## Runtime Rollout Safety - Rollout-managed feature(s): None. This is a transport fix, not a `rollout.py` feature. - Minimum rollout channel: N/A; it applies on every channel. - Stable/default behavior changed: Yes, for first-party Anthropic only. A buffered CCR turn and its CCR, memory and turn-hook continuations now go upstream `stream: true` with `Accept: text/event-stream`, and the proxy folds the reply into the same Message JSON. Request content is otherwise identical, so there is no compression, cache or token-count effect. The client still gets the buffered response it got before (keepalive pings, then the resynthesized stream), so time to first byte (#3587) is unchanged. A client's own `stream: false` request, every custom upstream, and the OpenAI paths are unchanged. One shared change: `message_delta` usage nulls no longer overwrite `message_start` counts in `AnthropicSSEEnvelope`, so a re-rendered stream on the streaming path now carries the real count where it carried `null`. - Kill switch / disable path: `HEADROOM_BUFFERED_UPSTREAM_STREAM=0` (also `false`, `no`, `off`), a runtime-env knob read on every request: set it in the proxy environment, let a reusing `headroom wrap` push it, or `POST /admin/runtime-env` (no restart). Every buffered request then goes out `stream: false` exactly as on main (`test_a_propagated_kill_switch_keeps_the_request_non_streaming`). - Unsafe override required: No. - Qualification impact: transport only. A complete stream the fold cannot read costs one extra generation (logged as `Could not fold the upstream event stream ...; sending the request again as stream:false`); that should not happen with the first-party API today and is the safe answer to a future delta type. - Rollback path: set `HEADROOM_BUFFERED_UPSTREAM_STREAM=0` (no restart needed), or revert these commits. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A, proxy transport change. ## Additional Notes - Documentation: no user-facing doc describes the buffered request's wire shape, so none changed. The kill switch is documented in the helper's docstring. - Relation to #3587: that issue is the latency side of the same flip (time to first byte becomes the whole generation, and the flip fires on turns where retrieval never redeems). This PR does not change when the buffered path is taken or what the client sees first; it fixes the availability side, where a turn longer than a middlebox's idle limit never arrives. #3587's asks (a stricter redeemability gate, a switch that disables buffering) are independent of this change. - Relation to #3013: #3013 replaces the buffered path with event-level splicing of the client stream behind `ccr_event_streaming` (default on) and keeps the buffered `stream: false` request as the qualified fallback when that flag is off. This PR is the small transport fix for the buffered design as it is on main: it does not touch the `buffered_stream_ccr` gating or the log line #3013 rewrites, and it adds no new conflict with #3013 (a `git merge-tree` of #3013's head against main and against this branch shows the same 3 conflict hunks in `anthropic.py` and 2 in `test_anthropic_streaming_ccr_retrieve.py`). If #3013 lands, its buffered fallback gets a streamed upstream leg from this change for free. - OpenAI Responses: that handler has the same flip (`openai.py`, `using buffered stream:false`) and the same exposure; it is left for a follow-up because its fold is a different event format. - `compaction_delta` and other delta types `AnthropicSSEEnvelope` does not reconstruct take the `stream: false` fallback rather than a guess; teaching the envelope a new delta type removes that extra generation. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4206" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> - Review fix61b42465: the kill switch is a registered runtime-env knob (see Changes Made), so the rollback reaches a proxy that `headroom wrap` reuses. Useful here because a fold failure can cost a second generation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
250 lines
9.3 KiB
Python
250 lines
9.3 KiB
Python
"""A buffered CCR turn must not ask for SSE it no longer wants (#3078).
|
|
|
|
Server-side retrieval flips a ``stream: true`` turn to ``stream: false`` so the
|
|
whole reply is in hand before answering. The body was rewritten; the client's
|
|
``Accept: text/event-stream`` was not, so the request that went on the wire
|
|
contradicted itself — "answer as JSON" in the body, "I only accept SSE" in the
|
|
headers.
|
|
|
|
Anthropic tolerates that, which is why it never showed up against the first-party
|
|
API. GitHub Copilot's Anthropic-compatible gateway does not, and answers with a
|
|
generic ``api_error``. That produced the reported shape exactly: the first call
|
|
of an OpenCode session succeeds (no marker yet, so no buffering), and the next
|
|
one — the first to carry a redeemable marker, and so the first to be flipped —
|
|
fails.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
|
|
import pytest
|
|
|
|
fastapi = pytest.importorskip("fastapi")
|
|
httpx = pytest.importorskip("httpx")
|
|
|
|
from fastapi.testclient import TestClient # noqa: E402
|
|
|
|
from headroom.cache.backends import InMemoryBackend # noqa: E402
|
|
from headroom.cache.compression_store import ( # noqa: E402
|
|
get_compression_store,
|
|
reset_compression_store,
|
|
)
|
|
from headroom.ccr.tool_injection import create_ccr_tool_definition # noqa: E402
|
|
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _stream_false_leg(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""Pin the ``stream: false`` request #3078 is about.
|
|
|
|
A first-party buffered turn now asks upstream for a stream and folds it
|
|
(``test_buffered_ccr_upstream_stream.py``). This request is still what a
|
|
custom upstream such as Copilot gets, and what the kill switch restores.
|
|
"""
|
|
monkeypatch.setenv("HEADROOM_BUFFERED_UPSTREAM_STREAM", "0")
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _store():
|
|
reset_compression_store()
|
|
get_compression_store(backend=InMemoryBackend())
|
|
try:
|
|
yield
|
|
finally:
|
|
reset_compression_store()
|
|
|
|
|
|
def _drive(*, with_marker: bool, accept: str | None) -> dict[str, object]:
|
|
"""Run one streaming turn; report the body `stream` and headers sent upstream."""
|
|
marker = get_compression_store().store(
|
|
original=json.dumps({"earlier": "tool output"}),
|
|
compressed="{}",
|
|
original_item_count=400,
|
|
)
|
|
config = ProxyConfig(
|
|
optimize=False,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
memory_enabled=False,
|
|
ccr_inject_tool=True,
|
|
ccr_handle_responses=True,
|
|
ccr_context_tracking=False,
|
|
image_optimize=False,
|
|
)
|
|
seen: dict[str, object] = {}
|
|
app = create_app(config)
|
|
with TestClient(app) as client:
|
|
proxy = client.app.state.proxy
|
|
|
|
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
|
|
sent = json.loads(body) if isinstance(body, (str, bytes)) else body
|
|
seen["stream"] = sent.get("stream")
|
|
seen["headers"] = dict(headers or {})
|
|
return httpx.Response(
|
|
200,
|
|
json={
|
|
"id": "msg_1",
|
|
"type": "message",
|
|
"role": "assistant",
|
|
"model": "claude-sonnet-4-6",
|
|
"content": [{"type": "text", "text": "ok"}],
|
|
"stop_reason": "end_turn",
|
|
"usage": {
|
|
"input_tokens": 10,
|
|
"output_tokens": 5,
|
|
"cache_read_input_tokens": 0,
|
|
"cache_creation_input_tokens": 0,
|
|
},
|
|
},
|
|
)
|
|
|
|
proxy._retry_request = _fake_retry # type: ignore[assignment]
|
|
|
|
# A turn that is *not* flipped never reaches `_retry_request` — the plain
|
|
# streaming path has its own upstream call — so capture that one too.
|
|
async def _fake_stream(url, headers, body, *args, **kwargs): # noqa: ANN001
|
|
from fastapi.responses import StreamingResponse
|
|
|
|
sent = json.loads(body) if isinstance(body, (str, bytes)) else body
|
|
seen["stream"] = sent.get("stream")
|
|
seen["headers"] = dict(headers or {})
|
|
|
|
async def _gen():
|
|
yield b'event: message_stop\ndata: {"type":"message_stop"}\n\n'
|
|
|
|
return StreamingResponse(_gen(), media_type="text/event-stream")
|
|
|
|
proxy._stream_response = _fake_stream # type: ignore[assignment]
|
|
headers = {"x-api-key": "test-key", "anthropic-version": "2023-06-01"}
|
|
if accept is not None:
|
|
headers["accept"] = accept
|
|
content = f"go <<ccr:{marker}>>" if with_marker else "go"
|
|
client.post(
|
|
"/v1/messages",
|
|
json={
|
|
"model": "claude-sonnet-4-6",
|
|
"max_tokens": 64,
|
|
"stream": True,
|
|
"tools": [create_ccr_tool_definition("anthropic")],
|
|
"messages": [{"role": "user", "content": content}],
|
|
},
|
|
headers=headers,
|
|
)
|
|
return seen
|
|
|
|
|
|
def _accepts(headers: dict) -> list[str]:
|
|
return [v for k, v in headers.items() if k.lower() == "accept"]
|
|
|
|
|
|
def test_buffered_turn_asks_for_json() -> None:
|
|
seen = _drive(with_marker=True, accept="text/event-stream")
|
|
|
|
# Precondition: this turn really was flipped to buffered.
|
|
assert seen["stream"] is False
|
|
assert _accepts(seen["headers"]) == ["application/json"] # type: ignore[arg-type]
|
|
|
|
|
|
def test_buffered_turn_leaves_exactly_one_accept_header() -> None:
|
|
"""Replaced, never appended — two Accept values is its own bug."""
|
|
seen = _drive(with_marker=True, accept="text/event-stream")
|
|
|
|
assert len(_accepts(seen["headers"])) == 1 # type: ignore[arg-type]
|
|
|
|
|
|
def test_accept_header_is_replaced_regardless_of_casing() -> None:
|
|
"""Header names are case-insensitive; the SSE value must not survive."""
|
|
seen = _drive(with_marker=True, accept="TEXT/EVENT-STREAM")
|
|
|
|
values = _accepts(seen["headers"]) # type: ignore[arg-type]
|
|
assert values == ["application/json"]
|
|
assert not any("event-stream" in v.lower() for v in values)
|
|
|
|
|
|
def test_buffered_turn_without_a_client_accept_still_asks_for_json() -> None:
|
|
seen = _drive(with_marker=True, accept=None)
|
|
|
|
assert seen["stream"] is False
|
|
assert _accepts(seen["headers"]) == ["application/json"] # type: ignore[arg-type]
|
|
|
|
|
|
def test_a_streaming_turn_keeps_its_sse_accept() -> None:
|
|
"""No marker means no flip, so nothing about the request should change."""
|
|
seen = _drive(with_marker=False, accept="text/event-stream")
|
|
|
|
assert seen["stream"] is not False
|
|
assert _accepts(seen["headers"]) == ["text/event-stream"] # type: ignore[arg-type]
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# The same flip exists on the OpenAI Responses path, which serves Copilot
|
|
# --------------------------------------------------------------------------- #
|
|
def _drive_responses(*, accept: str) -> dict[str, object]:
|
|
"""Run one streaming /v1/responses turn and report what went upstream."""
|
|
from headroom.ccr import CCR_TOOL_NAME
|
|
|
|
config = ProxyConfig(
|
|
optimize=False,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
memory_enabled=False,
|
|
ccr_inject_tool=True,
|
|
ccr_handle_responses=True,
|
|
ccr_context_tracking=False,
|
|
image_optimize=False,
|
|
)
|
|
seen: dict[str, object] = {}
|
|
app = create_app(config)
|
|
with TestClient(app) as client:
|
|
proxy = client.app.state.proxy
|
|
|
|
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
|
|
sent = json.loads(body) if isinstance(body, (str, bytes)) else body
|
|
seen["stream"] = sent.get("stream")
|
|
seen["headers"] = dict(headers or {})
|
|
return httpx.Response(
|
|
200,
|
|
json={
|
|
"id": "resp_1",
|
|
"object": "response",
|
|
"model": "gpt-4o",
|
|
"status": "completed",
|
|
"output": [
|
|
{
|
|
"type": "message",
|
|
"role": "assistant",
|
|
"content": [{"type": "output_text", "text": "ok"}],
|
|
}
|
|
],
|
|
"usage": {"input_tokens": 5, "output_tokens": 2, "total_tokens": 7},
|
|
},
|
|
)
|
|
|
|
proxy._retry_request = _fake_retry # type: ignore[assignment]
|
|
client.post(
|
|
"/v1/responses",
|
|
json={
|
|
"model": "gpt-4o",
|
|
"stream": True,
|
|
# Responses tool defs are flat, not nested under "function".
|
|
"tools": [{"type": "function", "name": CCR_TOOL_NAME}],
|
|
"input": "go",
|
|
},
|
|
headers={
|
|
"authorization": "Bearer test-key",
|
|
"accept": accept,
|
|
"content-type": "application/json",
|
|
},
|
|
)
|
|
return seen
|
|
|
|
|
|
def test_responses_buffered_turn_asks_for_json() -> None:
|
|
"""This handler serves GitHub Copilot, the gateway that rejects the mismatch."""
|
|
seen = _drive_responses(accept="text/event-stream")
|
|
|
|
assert seen.get("stream") is False, "precondition: the turn must be buffered"
|
|
assert _accepts(seen["headers"]) == ["application/json"] # type: ignore[arg-type]
|
|
assert len(_accepts(seen["headers"])) == 1 # type: ignore[arg-type]
|