Files
headroom/tests/test_buffered_ccr_accept_header.py
ca6ba3486c fix(ccr): stream the buffered CCR upstream request so idle-killing networks cannot cut it (#4206)
## Description

The buffered CCR path flips a streaming Claude Code turn to `stream:
false` upstream so the proxy holds the whole reply before serving
`headroom_retrieve`. With `stream: false`, Anthropic sends no byte until
the generation is complete. A middlebox that drops HTTP connections idle
for ~60s (TLS-inspecting antivirus, VPN, corporate proxy; TCP keepalive
does not reset an HTTP-layer idle timer) closes the connection first:
each attempt fails at ~60s with `Server disconnected without sending a
response`, the client gets a 502 after ~186s, and Claude Code resends
the same body forever. The CCR continuation that serves the retrieval
(`anthropic_ccr_continuation`, its own `http_client.post`) is a second
`stream: false` call and is cut the same way. Without Headroom the
client streams and the same turns complete.

This PR keeps the buffered design and makes only its upstream leg
stream. The buffered request and every continuation built from it (CCR,
memory, turn hook) go to first-party Anthropic as `stream: true`, and
the event stream is folded back into the Message JSON the buffered path
already reads, so retrieval, memory, turn hooks, usage accounting and
SSE resynthesis see exactly what they saw before. Bytes (deltas and
pings) now flow for the whole generation, so an idle timer never fires.
The fold reuses `AnthropicSSEEnvelope`, the reconstruction #3130 and
#3266 already use on this path.

Closes #4205

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/anthropic.py`: new
`_send_buffered_as_stream(send, headers, body, *, url, request_id)`. It
sends `{**body, "stream": true}` with `Accept: text/event-stream` (one
Accept value, any casing replaced) and folds the reply:
- not a 200 event stream (an HTTP error, or an upstream that answered
JSON anyway): returned unchanged;
  - a whole stream: a 200 JSON response holding the folded Message;
- an in-band `error` event: the event as the body, with its own HTTP
status (529 `overloaded_error`, 429 `rate_limit_error`, other documented
types to their status, else 500);
- a stream that ended before `message_stop` or with a block open: the
existing strict 502 `upstream_protocol_error` of
`_adapt_event_stream_to_json` (#3266);
- a complete stream holding a known event the fold cannot read (an
unknown delta type): logged, and the original `stream: false` body is
sent instead. One more generation, but never a 502 the client would
resend forever.
A connection dropped mid-stream raises inside `send`, exactly as for a
`stream: false` reply. An in-band error arrives after upstream already
answered 200, so `_retry_request` does not retry it; the client gets the
status and retries as it would on a direct stream.
- It is used at the four places Headroom sends a body it made
non-streaming: the buffered CCR request (only when
`buffered_stream_ccr`; a client's own `stream: false` request still goes
through `_retry_request` with its original bytes), the CCR continuation
(the `prepare_outbound_body_bytes` + `log_outbound_request` +
`http_client.post` sequence moved into a local `_post_continuation`,
unchanged otherwise, so a fallback attempt logs its own outbound line),
the memory continuation and the turn-hook continuation.
- Gates: the body is not already streaming, the upstream is first-party
Anthropic (`is_custom_anthropic_base_url(url)` is false; the
host-equality check tool-search gating already uses), and the kill
switch `HEADROOM_BUFFERED_UPSTREAM_STREAM` is not
`0`/`false`/`no`/`off`. Copilot and other gateways keep the `stream:
false` request: the fold is checked against Anthropic's own stream, and
a gateway dialect it cannot read would cost every turn a second
generation.
- `headroom/proxy/runtime_env.py`: `HEADROOM_BUFFERED_UPSTREAM_STREAM`
is a registered knob in `RUNTIME_ENV_KNOBS`, read per request through
`runtime_env.getenv`, so a reusing `headroom wrap` pushes it to the
running proxy and `POST /admin/runtime-env` sets it with no restart.
Listed in the `security-model.mdx` allowlist.
- `headroom/proxy/anthropic_wire.py`: `fold_anthropic_stream(raw)`
parses with `AnthropicSSEEnvelope.parse` and trims each block to the
non-streaming Message shape: a null member leaves every block,
`thinking` keeps `type`/`thinking`/`signature`, `redacted_thinking`
keeps `type`/`data`. Claude Opus 5.5 streams a thinking
`content_block_start` with `estimated_tokens: null`; the CCR
continuation resends the folded turn as history, and Anthropic answers
any block carrying that member with 400 `Extra inputs are not
permitted`.
- `headroom/proxy/anthropic_wire.py` `_response_from_events`:
`message_delta.usage` is merged only where non-null, as the Anthropic
SDK's `MessageStream` does. Before, a null count there overwrote the
real `message_start` count (input and cache tokens) in the reconstructed
message.
- `headroom/proxy/buffered_ccr_response.py`: the status-to-error-type
table moves out of `_anthropic_error_type` into `ANTHROPIC_ERROR_TYPES`
so the handler can invert it; `_anthropic_error_type` returns the same
values.
- Unchanged on purpose: `body["stream"] = False` and its mutation reason
`ccr_streaming_retrieve_buffered_non_stream` stay the buffered path's
internal representation (every later step, the response-cache guard and
the #3587 log counts key on them), and the `using buffered stream:false
upstream request` log line keeps its text.
- Tests: new `tests/test_buffered_ccr_upstream_stream.py` (16 tests):
the fold equals the Anthropic Python SDK's `accumulate_event` result on
a stream with thinking, signature, redacted thinking, text, citations,
split `input_json_delta`, pings and a null usage count; stream-only
members are dropped; truncated, in-band error and unknown-delta
outcomes; the send helper's gates (kill switch, gateway URL, already
streaming), passthrough of non-stream replies, 529 for an in-band
overload, 502 for a truncated stream, the `stream: false` fallback; and
through the handler, the buffered turn and its CCR continuation both go
upstream `stream: true` with `Accept: text/event-stream` and the
continuation's resent thinking block carries only its schema, while a
client's own `stream: false` request goes out `stream: false` with
`Accept: application/json` and its original bytes.
- Existing tests that pinned `stream: false` on the wire for a buffered
turn now pin `stream: true` (`test_anthropic_streaming_ccr_retrieve.py`,
`test_ccr_buffered_stream_signed_thinking.py`,
`test_proxy_response_cache_replay.py`,
`test_no_ccr_disables_response_handling.py`).
`test_anthropic_buffered_ccr_usage.py` now answers the buffered turn
with an event stream, so usage booking is checked through the fold.
`test_buffered_ccr_accept_header.py` (#3078) pins the `stream: false`
leg it is about with the kill switch; custom upstreams such as Copilot
always take that leg.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) - ran on the 3 changed
source files only, see below
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_buffered_ccr_upstream_stream.py -q
16 passed, 1 warning in 2.89s

# every test file that touches the buffered CCR path, the CCR continuation or SSE reconstruction
$ pytest tests/test_buffered_ccr_upstream_stream.py tests/test_sse_safeguards_passthrough.py \
    tests/test_buffered_ccr_salvage.py tests/test_anthropic_buffered_ccr_usage.py \
    tests/test_buffered_ccr_grace_window.py tests/test_anthropic_buffered_sse.py \
    tests/test_ccr_buffered_stream_signed_thinking.py tests/test_toin_feedback.py \
    tests/test_proxy_response_cache_replay.py tests/test_proxy/test_anthropic_buffered_timeout.py \
    tests/test_anthropic_auto_mode_passthrough.py tests/test_proxy/test_openai_responses_ccr.py \
    tests/test_sse_thinking_blocks.py tests/test_streaming_sse_malformed_events.py \
    tests/test_route_advice.py tests/test_memory_tool_stream.py \
    tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_proxy_byte_faithful_forwarding.py \
    tests/test_buffered_ccr_accept_header.py tests/test_anthropic_buffered_memory_continuation.py \
    tests/test_anthropic_ccr_workspace_unbound.py tests/test_no_ccr_disables_response_handling.py \
    tests/test_proxy_ccr.py tests/test_ccr_inline_resolve_handlers.py \
    tests/test_proxy/test_anthropic_ccr_raise.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py \
    tests/test_nonstream_sse_policy.py tests/test_ccr_response_handler_extra.py \
    tests/test_compress_route_tokenizer_by_model.py -q
855 passed, 6 warnings in 57.94s

# the updated assertions fail on main's source (they pin stream:true on the wire)
$ git checkout upstream/main -- headroom/ && pytest tests/test_anthropic_buffered_ccr_usage.py \
    tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_ccr_buffered_stream_signed_thinking.py \
    tests/test_no_ccr_disables_response_handling.py tests/test_proxy_response_cache_replay.py -q --tb=no
12 failed, 33 passed, 1 warning in 13.64s

# whole suite
$ pytest tests -q
18 failed, 16435 passed, 583 skipped, 119 warnings in 1226.31s (0:20:26)
# 16 of the 18 fail the same way on unmodified main in the same environment
# (tests/test_proxy_uds.py x11, tests/test_cli/test_wrap_persistent.py x4,
#  tests/test_pricing_from_litellm.py groq/llama-guard-3-8b x1); the other 2
# (test_offline_egress_chokepoint.py::TestUpdateCheckOffline::test_explicit_update_check_never_constructs_request,
#  test_stateless_no_disk.py::test_cli_stateless_flag_exports_env_for_children) pass when run alone on this branch.

$ ruff check .
All checks passed!
$ ruff format --check .
1790 files already formatted
$ mypy headroom/proxy/anthropic_wire.py headroom/proxy/buffered_ccr_response.py headroom/proxy/handlers/anthropic.py
Success: no issues found in 3 source files
```

## Real Behavior Proof

- Environment: macOS 27.0.1 arm64, Python 3.11.13. Upstream `main`
f1bc2609 versus this branch, same venv.
- Exact command / steps: `repro_buffered_idle_cut.py` (full script in
#4205) runs the real proxy app (`create_app(ProxyConfig(...))` +
`TestClient`, real `_retry_request`, real `http_client`) against a fake
Anthropic upstream that takes 5s per reply (stream: a delta or ping
every 0.5s; non-stream: silent for 5s, then JSON), reached through a TCP
relay that closes any connection silent for 2s, a scaled-down 60s
middlebox. Scenario A: a streaming turn with a redeemable marker
(buffered path), slow reply. Scenario B: the model calls
`headroom_retrieve` in 0.3s and the continuation is slow. The script
points the first-party host check at the fake upstream and isolates
`HOME` and the savings ledger. `PYTHONPATH=<checkout> python
repro_buffered_idle_cut.py 2>/dev/null`
- Observed result: on main both scenarios fail and every upstream
request goes out `stream: false` and is cut; on the branch both complete
in about the generation time with no cut; with
`HEADROOM_BUFFERED_UPSTREAM_STREAM=0` on the branch the main behaviour
returns.

```text
# main f1bc2609
Scenario A (slow buffered turn)
  client: HTTP 200 after 10.1s -> FAIL: 20 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "An error occurred while processing your request. Please try again."}}
  upstream requests: ['turn stream=false', 'turn stream=false', 'turn stream=false']
  connections the relay cut for idling: 3

Scenario B (slow CCR continuation)
  client: HTTP 502 after 2.3s -> FAIL: 0 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "Unable to safely complete streamed CCR retrieval."}}
  upstream requests: ['retrieve call stream=false', 'continuation stream=false']
  connections the relay cut for idling: 1

# proxy log on main
WARNING - Request failed (attempt 1), retrying in 1322ms: Server disconnected without sending a response.
WARNING - Request failed (attempt 2), retrying in 2536ms: Server disconnected without sending a response.
ERROR - [hr_1791973393_000001] Request failed: RemoteProtocolError: Server disconnected without sending a response.
ERROR - CCR: Continuation API call failed: RemoteProtocolError: RemoteProtocolError('Server disconnected without sending a response.')
WARNING - [hr_1791973403_000002] CCR: Buffered streaming response still contains an unresolved headroom_retrieve after handling; failing closed

# this branch
Scenario A (slow buffered turn)
  client: HTTP 200 after 5.2s -> OK: the client got the final answer
  upstream requests: ['turn stream=true']
  connections the relay cut for idling: 0

Scenario B (slow CCR continuation)
  client: HTTP 200 after 5.3s -> OK: the client got the final answer
  upstream requests: ['retrieve call stream=true', 'continuation stream=true']
  connections the relay cut for idling: 0

# this branch, HEADROOM_BUFFERED_UPSTREAM_STREAM=0
Scenario A (slow buffered turn)
  client: HTTP 200 after 9.0s -> FAIL: 16 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "An error occurred while processing your request. Please try again."}}
  upstream requests: ['turn stream=false', 'turn stream=false', 'turn stream=false']
  connections the relay cut for idling: 3

Scenario B (slow CCR continuation)
  client: HTTP 502 after 2.4s -> FAIL: 0 keepalive pings, then {"type": "error", "error": {"type": "api_error", "message": "Unable to safely complete streamed CCR retrieval."}}
  upstream requests: ['retrieve call stream=false', 'continuation stream=false']
  connections the relay cut for idling: 1
```

- Not tested: this branch has not sent traffic to api.anthropic.com. The
fold is checked against the Anthropic Python SDK's own accumulator and
against hand-built streams, not against live replies. For context, the
same transport change (a different fold implementation with the same
contract) has run as a downstream patch on the 0.39.0 wheel and was
checked live: 56 turns across Opus, Sonnet, Haiku and Fable models
(thinking, parallel `tool_use`, `web_search` with citations,
`max_tokens` cutoffs) matched the non-streaming reply's top-level, block
and usage shapes; 28 folded replies resent as history all returned 200;
behind a relay that cuts a connection after 20s of silence, a 409s
thinking turn completed (longest silence 2.1s) while the `stream: false`
path looped on 502s. The OpenAI Responses buffered path has the same
flip and is not changed here. Gateways (Copilot, custom base URLs) are
not changed.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This is a transport fix, not a
`rollout.py` feature.
- Minimum rollout channel: N/A; it applies on every channel.
- Stable/default behavior changed: Yes, for first-party Anthropic only.
A buffered CCR turn and its CCR, memory and turn-hook continuations now
go upstream `stream: true` with `Accept: text/event-stream`, and the
proxy folds the reply into the same Message JSON. Request content is
otherwise identical, so there is no compression, cache or token-count
effect. The client still gets the buffered response it got before
(keepalive pings, then the resynthesized stream), so time to first byte
(#3587) is unchanged. A client's own `stream: false` request, every
custom upstream, and the OpenAI paths are unchanged. One shared change:
`message_delta` usage nulls no longer overwrite `message_start` counts
in `AnthropicSSEEnvelope`, so a re-rendered stream on the streaming path
now carries the real count where it carried `null`.
- Kill switch / disable path: `HEADROOM_BUFFERED_UPSTREAM_STREAM=0`
(also `false`, `no`, `off`), a runtime-env knob read on every request:
set it in the proxy environment, let a reusing `headroom wrap` push it,
or `POST /admin/runtime-env` (no restart). Every buffered request then
goes out `stream: false` exactly as on main
(`test_a_propagated_kill_switch_keeps_the_request_non_streaming`).
- Unsafe override required: No.
- Qualification impact: transport only. A complete stream the fold
cannot read costs one extra generation (logged as `Could not fold the
upstream event stream ...; sending the request again as stream:false`);
that should not happen with the first-party API today and is the safe
answer to a future delta type.
- Rollback path: set `HEADROOM_BUFFERED_UPSTREAM_STREAM=0` (no restart
needed), or revert these commits.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` - it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A, proxy transport change.

## Additional Notes

- Documentation: no user-facing doc describes the buffered request's
wire shape, so none changed. The kill switch is documented in the
helper's docstring.
- Relation to #3587: that issue is the latency side of the same flip
(time to first byte becomes the whole generation, and the flip fires on
turns where retrieval never redeems). This PR does not change when the
buffered path is taken or what the client sees first; it fixes the
availability side, where a turn longer than a middlebox's idle limit
never arrives. #3587's asks (a stricter redeemability gate, a switch
that disables buffering) are independent of this change.
- Relation to #3013: #3013 replaces the buffered path with event-level
splicing of the client stream behind `ccr_event_streaming` (default on)
and keeps the buffered `stream: false` request as the qualified fallback
when that flag is off. This PR is the small transport fix for the
buffered design as it is on main: it does not touch the
`buffered_stream_ccr` gating or the log line #3013 rewrites, and it adds
no new conflict with #3013 (a `git merge-tree` of #3013's head against
main and against this branch shows the same 3 conflict hunks in
`anthropic.py` and 2 in `test_anthropic_streaming_ccr_retrieve.py`). If
#3013 lands, its buffered fallback gets a streamed upstream leg from
this change for free.
- OpenAI Responses: that handler has the same flip (`openai.py`, `using
buffered stream:false`) and the same exposure; it is left for a
follow-up because its fold is a different event format.
- `compaction_delta` and other delta types `AnthropicSSEEnvelope` does
not reconstruct take the `stream: false` fallback rather than a guess;
teaching the envelope a new delta type removes that extra generation.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4206"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
- Review fix 61b42465: the kill switch is a registered runtime-env knob
(see Changes Made), so the rollback reaches a proxy that `headroom wrap`
reuses. Useful here because a fold failure can cost a second generation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-15 16:29:46 -05:00

250 lines
9.3 KiB
Python

"""A buffered CCR turn must not ask for SSE it no longer wants (#3078).
Server-side retrieval flips a ``stream: true`` turn to ``stream: false`` so the
whole reply is in hand before answering. The body was rewritten; the client's
``Accept: text/event-stream`` was not, so the request that went on the wire
contradicted itself — "answer as JSON" in the body, "I only accept SSE" in the
headers.
Anthropic tolerates that, which is why it never showed up against the first-party
API. GitHub Copilot's Anthropic-compatible gateway does not, and answers with a
generic ``api_error``. That produced the reported shape exactly: the first call
of an OpenCode session succeeds (no marker yet, so no buffering), and the next
one — the first to carry a redeemable marker, and so the first to be flipped —
fails.
"""
from __future__ import annotations
import json
import pytest
fastapi = pytest.importorskip("fastapi")
httpx = pytest.importorskip("httpx")
from fastapi.testclient import TestClient # noqa: E402
from headroom.cache.backends import InMemoryBackend # noqa: E402
from headroom.cache.compression_store import ( # noqa: E402
get_compression_store,
reset_compression_store,
)
from headroom.ccr.tool_injection import create_ccr_tool_definition # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
@pytest.fixture(autouse=True)
def _stream_false_leg(monkeypatch: pytest.MonkeyPatch) -> None:
"""Pin the ``stream: false`` request #3078 is about.
A first-party buffered turn now asks upstream for a stream and folds it
(``test_buffered_ccr_upstream_stream.py``). This request is still what a
custom upstream such as Copilot gets, and what the kill switch restores.
"""
monkeypatch.setenv("HEADROOM_BUFFERED_UPSTREAM_STREAM", "0")
@pytest.fixture(autouse=True)
def _store():
reset_compression_store()
get_compression_store(backend=InMemoryBackend())
try:
yield
finally:
reset_compression_store()
def _drive(*, with_marker: bool, accept: str | None) -> dict[str, object]:
"""Run one streaming turn; report the body `stream` and headers sent upstream."""
marker = get_compression_store().store(
original=json.dumps({"earlier": "tool output"}),
compressed="{}",
original_item_count=400,
)
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
memory_enabled=False,
ccr_inject_tool=True,
ccr_handle_responses=True,
ccr_context_tracking=False,
image_optimize=False,
)
seen: dict[str, object] = {}
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
sent = json.loads(body) if isinstance(body, (str, bytes)) else body
seen["stream"] = sent.get("stream")
seen["headers"] = dict(headers or {})
return httpx.Response(
200,
json={
"id": "msg_1",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-6",
"content": [{"type": "text", "text": "ok"}],
"stop_reason": "end_turn",
"usage": {
"input_tokens": 10,
"output_tokens": 5,
"cache_read_input_tokens": 0,
"cache_creation_input_tokens": 0,
},
},
)
proxy._retry_request = _fake_retry # type: ignore[assignment]
# A turn that is *not* flipped never reaches `_retry_request` — the plain
# streaming path has its own upstream call — so capture that one too.
async def _fake_stream(url, headers, body, *args, **kwargs): # noqa: ANN001
from fastapi.responses import StreamingResponse
sent = json.loads(body) if isinstance(body, (str, bytes)) else body
seen["stream"] = sent.get("stream")
seen["headers"] = dict(headers or {})
async def _gen():
yield b'event: message_stop\ndata: {"type":"message_stop"}\n\n'
return StreamingResponse(_gen(), media_type="text/event-stream")
proxy._stream_response = _fake_stream # type: ignore[assignment]
headers = {"x-api-key": "test-key", "anthropic-version": "2023-06-01"}
if accept is not None:
headers["accept"] = accept
content = f"go <<ccr:{marker}>>" if with_marker else "go"
client.post(
"/v1/messages",
json={
"model": "claude-sonnet-4-6",
"max_tokens": 64,
"stream": True,
"tools": [create_ccr_tool_definition("anthropic")],
"messages": [{"role": "user", "content": content}],
},
headers=headers,
)
return seen
def _accepts(headers: dict) -> list[str]:
return [v for k, v in headers.items() if k.lower() == "accept"]
def test_buffered_turn_asks_for_json() -> None:
seen = _drive(with_marker=True, accept="text/event-stream")
# Precondition: this turn really was flipped to buffered.
assert seen["stream"] is False
assert _accepts(seen["headers"]) == ["application/json"] # type: ignore[arg-type]
def test_buffered_turn_leaves_exactly_one_accept_header() -> None:
"""Replaced, never appended — two Accept values is its own bug."""
seen = _drive(with_marker=True, accept="text/event-stream")
assert len(_accepts(seen["headers"])) == 1 # type: ignore[arg-type]
def test_accept_header_is_replaced_regardless_of_casing() -> None:
"""Header names are case-insensitive; the SSE value must not survive."""
seen = _drive(with_marker=True, accept="TEXT/EVENT-STREAM")
values = _accepts(seen["headers"]) # type: ignore[arg-type]
assert values == ["application/json"]
assert not any("event-stream" in v.lower() for v in values)
def test_buffered_turn_without_a_client_accept_still_asks_for_json() -> None:
seen = _drive(with_marker=True, accept=None)
assert seen["stream"] is False
assert _accepts(seen["headers"]) == ["application/json"] # type: ignore[arg-type]
def test_a_streaming_turn_keeps_its_sse_accept() -> None:
"""No marker means no flip, so nothing about the request should change."""
seen = _drive(with_marker=False, accept="text/event-stream")
assert seen["stream"] is not False
assert _accepts(seen["headers"]) == ["text/event-stream"] # type: ignore[arg-type]
# --------------------------------------------------------------------------- #
# The same flip exists on the OpenAI Responses path, which serves Copilot
# --------------------------------------------------------------------------- #
def _drive_responses(*, accept: str) -> dict[str, object]:
"""Run one streaming /v1/responses turn and report what went upstream."""
from headroom.ccr import CCR_TOOL_NAME
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
memory_enabled=False,
ccr_inject_tool=True,
ccr_handle_responses=True,
ccr_context_tracking=False,
image_optimize=False,
)
seen: dict[str, object] = {}
app = create_app(config)
with TestClient(app) as client:
proxy = client.app.state.proxy
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
sent = json.loads(body) if isinstance(body, (str, bytes)) else body
seen["stream"] = sent.get("stream")
seen["headers"] = dict(headers or {})
return httpx.Response(
200,
json={
"id": "resp_1",
"object": "response",
"model": "gpt-4o",
"status": "completed",
"output": [
{
"type": "message",
"role": "assistant",
"content": [{"type": "output_text", "text": "ok"}],
}
],
"usage": {"input_tokens": 5, "output_tokens": 2, "total_tokens": 7},
},
)
proxy._retry_request = _fake_retry # type: ignore[assignment]
client.post(
"/v1/responses",
json={
"model": "gpt-4o",
"stream": True,
# Responses tool defs are flat, not nested under "function".
"tools": [{"type": "function", "name": CCR_TOOL_NAME}],
"input": "go",
},
headers={
"authorization": "Bearer test-key",
"accept": accept,
"content-type": "application/json",
},
)
return seen
def test_responses_buffered_turn_asks_for_json() -> None:
"""This handler serves GitHub Copilot, the gateway that rejects the mismatch."""
seen = _drive_responses(accept="text/event-stream")
assert seen.get("stream") is False, "precondition: the turn must be buffered"
assert _accepts(seen["headers"]) == ["application/json"] # type: ignore[arg-type]
assert len(_accepts(seen["headers"])) == 1 # type: ignore[arg-type]