Files
headroom/tests/test_ccr_response_handler_extra.py
Devanshi Vyas 12c157966f fix(proxy): preserve Claude Code auto-mode protocol (#3784)
## Description

Fix Claude Code server-side auto-mode compatibility when requests pass
through
Headroom’s native Anthropic Messages endpoint.

Previously, auto-mode capability headers, safeguard payloads, query
parameters,
unknown response fields, or opaque SSE frames could be rewritten or lost
while
proxying and handling CCR continuations. This could trigger Claude
Code’s
client-side classifier fallback and its associated warning.

This change adds a request-local Anthropic wire envelope, preserves
client-owned
capability data and opaque response attachments, prevents classifier
payloads
from entering Headroom-owned observations, and documents that
schema-translating
Bedrock, Vertex, and LiteLLM routes do not currently support the
server-side
auto-mode protocol.


Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Preserve native Anthropic request paths and raw query parameters.
- Preserve client-owned `anthropic-version`, `anthropic-beta`, and
safeguard data.
- Avoid sticky-beta, memory, or turn-hook rewrites on classifier-bearing
turns.
- Preserve unknown Anthropic response fields across CCR continuations.
- Preserve opaque SSE frames through request-local parse/render
envelopes.
- Reject unknown or malformed deltas during strict buffered
reconstruction.
- Redact safeguard payloads from diagnostics and pipeline observations.
- Consolidate Anthropic SSE parsing and rendering into the wire module.
- Document auto-mode limitations for Bedrock, Vertex, and LiteLLM
translation paths.
- Add request, response, streaming, CCR, malformed-byte, and passthrough
regression tests.

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output


```text
$ python -m pytest -p no:cacheprovider \
    tests/test_anthropic_auto_mode_passthrough.py \
    tests/test_anthropic_buffered_sse.py \
    tests/test_ccr_response_handler_extra.py \
    tests/test_ccr_buffered_stream_signed_thinking.py \
    tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py \
    tests/test_buffered_ccr_salvage.py \
    tests/test_buffered_ccr_accept_header.py \
    tests/test_proxy/test_anthropic_copilot_upstream_auth.py \
    tests/test_proxy_streaming_request_logger.py \
    tests/test_proxy_streaming_resilience.py -q

125 passed, 1 warning in 13.70s

$ ruff check --no-cache .
All checks passed!

$ ruff format --check --no-cache .
1559 files already formatted

$ python -m mypy \
    headroom/proxy/anthropic_wire.py \
    headroom/ccr/response_handler.py \
    headroom/proxy/handlers/anthropic.py \
    headroom/proxy/handlers/streaming.py

Success: no issues found in 4 source files

$ npx --no-install commitlint \
    --from origin/main --to HEAD \
    --config .commitlintrc.json

Exited successfully.
```

## Real Behavior Proof

- Environment: Pending. Record OS, Python version, Claude Code version,
provider/model, and Headroom revision.
- Exact command / steps: Pending. Start Headroom with `headroom proxy
--port 8787`, then run Claude Code with
`ANTHROPIC_BASE_URL=http://127.0.0.1:8787` and exercise server-side auto
mode.
- Observed result: Pending. Attach redacted output or logs showing the
request completed through Headroom and whether Claude Code used its
client-side fallback.
- Not tested: Live Claude Code auto mode against Anthropic; Bedrock,
Vertex, and LiteLLM translation routes.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Native Anthropic safeguard-capable
requests.
- Kill switch / disable path: Set `CLAUDE_CODE_AUTO_MODE_SERVER=0` to
use Claude Code’s client-side fallback.
- Unsafe override required: No.
- Qualification impact: Live qualification is pending. Translation
routes remain documented as unsupported for server-side auto mode.
- Rollback path: Revert this PR or deploy the previous Headroom
revision.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code where needed
- [x] I have made corresponding documentation changes
- [x] No new warnings were observed in focused tests
- [x] I have added tests for the fix
- [ ] Full existing and new unit test suite passes locally
- [x] I did **not** edit `CHANGELOG.md`

## Screenshots (if applicable)

N/A. This change affects proxy wire behavior.
2026-09-25 09:53:27 -07:00

888 lines
33 KiB
Python

from __future__ import annotations
import json
from typing import Any
import pytest
from headroom.ccr.response_handler import (
CCRResponseHandler,
CCRToolCall,
CCRToolResult,
StreamingCCRBuffer,
StreamingCCRHandler,
)
from headroom.ccr.tool_injection import CCR_TOOL_NAME
class FakeStore:
def __init__(self, *, retrieve_error: Exception | None = None) -> None:
self.retrieve_error = retrieve_error
def retrieve(self, hash_key: str):
if self.retrieve_error:
raise self.retrieve_error
return {"unexpected": True}
async def _async_iter(items: list[bytes]):
for item in items:
yield item
def _sse_json_events(chunks: list[bytes]) -> list[dict[str, Any]]:
events = []
for chunk in chunks:
for line in chunk.decode("utf-8").splitlines():
if not line.startswith("data: "):
continue
payload = line[len("data: ") :]
if payload == "[DONE]":
continue
events.append(json.loads(payload))
return events
def test_extract_tool_calls_google_and_invalid_shapes() -> None:
handler = CCRResponseHandler()
google_response = {
"candidates": [
{
"content": {
"parts": [
{"text": "hello"},
{"functionCall": {"name": CCR_TOOL_NAME, "args": {"hash": "abc"}}},
]
}
}
]
}
assert handler._extract_tool_calls(google_response, "google") == [
{"functionCall": {"name": CCR_TOOL_NAME, "args": {"hash": "abc"}}}
]
assert handler._extract_tool_calls({"content": "bad"}, "anthropic") == []
with pytest.raises(IndexError):
handler._extract_tool_calls({"choices": []}, "openai")
assert handler._extract_tool_calls({"candidates": []}, "google") == []
def test_parse_ccr_tool_calls_google_and_other_calls() -> None:
handler = CCRResponseHandler()
response = {
"candidates": [
{
"content": {
"parts": [
{
"functionCall": {
"name": CCR_TOOL_NAME,
"args": {
"hash": "aaaaaaaaaaaaaaaaaaaaaaaa",
"query": "pizza",
},
}
},
{"functionCall": {"name": "other_tool", "args": {}}},
]
}
}
]
}
ccr_calls, other_calls = handler._parse_ccr_tool_calls(response, "google")
assert ccr_calls == [
CCRToolCall(
tool_call_id=CCR_TOOL_NAME,
hash_key="aaaaaaaaaaaaaaaaaaaaaaaa",
)
]
assert other_calls == [{"functionCall": {"name": "other_tool", "args": {}}}]
def test_execute_retrieval_error_paths(monkeypatch: pytest.MonkeyPatch) -> None:
handler = CCRResponseHandler()
# Retrieval is by hash only; a store error surfaces as a failed result.
monkeypatch.setattr(
"headroom.ccr.response_handler.get_compression_store",
lambda: FakeStore(retrieve_error=RuntimeError("retrieve boom")),
)
retrieve_result = handler._execute_retrieval(CCRToolCall(tool_call_id="t2", hash_key="abc"))
assert retrieve_result.success is False
assert "Retrieval failed: retrieve boom" in retrieve_result.content
def test_create_tool_result_message_google_and_generic_formats() -> None:
handler = CCRResponseHandler()
results = [
CCRToolResult(tool_call_id="headroom_retrieve", content='{"count": 1}', success=True)
]
google_message = handler._create_tool_result_message(results, "google")
assert google_message == {
"role": "user",
"parts": [{"functionResponse": {"name": "headroom_retrieve", "response": {"count": 1}}}],
}
generic_message = handler._create_tool_result_message(
[CCRToolResult(tool_call_id="tool-1", content="not-json", success=False)],
"other",
)
assert generic_message["role"] == "tool"
assert json.loads(generic_message["content"]) == [
{"tool_call_id": "tool-1", "result": "not-json"}
]
invalid_google = handler._create_tool_result_message(
[CCRToolResult(tool_call_id="headroom_retrieve", content="not-json", success=True)],
"google",
)
assert invalid_google["parts"][0]["functionResponse"]["response"] == {"content": "not-json"}
def test_create_tool_result_message_google_preserves_call_id() -> None:
handler = CCRResponseHandler()
message = handler._create_tool_result_message(
[
CCRToolResult(
tool_call_id="call-1",
tool_name="headroom_retrieve",
content='{"count": 1}',
success=True,
)
],
"google",
)
assert message["parts"][0]["functionResponse"] == {
"name": "headroom_retrieve",
"id": "call-1",
"response": {"count": 1},
}
def test_extract_assistant_message_google_and_generic() -> None:
handler = CCRResponseHandler()
google_message = handler._extract_assistant_message(
{"candidates": [{"content": {"parts": [{"text": "hello"}]}}]},
"google",
)
assert google_message == {"role": "model", "parts": [{"text": "hello"}]}
assert handler._extract_assistant_message({}, "google") == {"role": "model", "parts": []}
assert handler._extract_assistant_message({"content": "plain"}, "other") == {
"role": "assistant",
"content": "plain",
}
@pytest.mark.asyncio
async def test_handle_response_openai_success_and_failure(monkeypatch: pytest.MonkeyPatch) -> None:
handler = CCRResponseHandler()
initial_response = {
"choices": [
{
"message": {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {
"name": CCR_TOOL_NAME,
"arguments": '{"hash":"aaaaaaaaaaaaaaaaaaaaaaaa"}',
},
}
],
}
}
]
}
monkeypatch.setattr(
handler,
"_execute_retrieval",
lambda call: CCRToolResult(
tool_call_id=call.tool_call_id,
content='{"hash":"aaaaaaaaaaaaaaaaaaaaaaaa"}',
success=True,
),
)
captured_messages: list[list[dict[str, Any]]] = []
async def success_api_call(messages, tools):
captured_messages.append(messages)
return {"choices": [{"message": {"role": "assistant", "content": "done"}}]}
result = await handler.handle_response(
initial_response, [{"role": "user", "content": "hi"}], [], success_api_call, "openai"
)
assert result == {"choices": [{"message": {"role": "assistant", "content": "done"}}]}
assert captured_messages[0][1]["role"] == "assistant"
assert captured_messages[0][2]["role"] == "tool"
assert handler.get_stats()["total_retrievals"] == 1
async def failing_api_call(messages, tools):
raise RuntimeError("continuation failed")
failed = await handler.handle_response(initial_response, [], [], failing_api_call, "openai")
assert failed == initial_response
def test_streaming_buffer_and_parse_sse_helpers() -> None:
buffer = StreamingCCRBuffer()
assert buffer.add_chunk(b"plain") is False
assert buffer.get_accumulated() == b"plain"
handler = StreamingCCRHandler(CCRResponseHandler(), provider="anthropic")
# Per SSE spec each event is terminated by `\n\n`. The byte-buffer
# parser introduced in PR-A8 requires the spec terminator so partial
# multi-byte UTF-8 reads don't corrupt event boundaries.
stop_details = {"type": "refusal", "message": "policy refusal"}
anthropic_data = b"\n\n".join(
[
b'data: {"type":"message_start","message":{"id":"msg_1","model":"claude-fable-5","role":"assistant","usage":{"input_tokens":7}}}',
b'data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}}',
b'data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":""}}',
b'data: {"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"sig_fable"}}',
b'data: {"type":"content_block_stop","index":0}',
b'data: {"type":"content_block_start","index":1,"content_block":{"type":"redacted_thinking","data":"ENC:abc"}}',
b'data: {"type":"content_block_stop","index":1}',
b'data: {"type":"content_block_start","content_block":{"type":"text","text":"Hel"}}',
b'data: {"type":"content_block_delta","delta":{"type":"text_delta","text":"lo"}}',
b'data: {"type":"content_block_stop"}',
b'data: {"type":"content_block_start","content_block":{"type":"tool_use","id":"tool_1","name":"headroom_retrieve"}}',
b'data: {"type":"content_block_delta","delta":{"type":"input_json_delta","partial_json":"{\\"hash\\":\\"abc\\"}"}}',
b'data: {"type":"content_block_stop"}',
(
b'data: {"type":"message_delta","delta":{"stop_reason":"refusal",'
b'"stop_details":{"type":"refusal","message":"policy refusal"}},'
b'"usage":{"output_tokens":3}}'
),
b"data: [DONE]\n\n",
]
)
parsed = handler._parse_sse_stream(anthropic_data)
assert parsed["content"][0] == {
"type": "thinking",
"signature": "sig_fable",
"thinking": "",
}
assert parsed["content"][1] == {"type": "redacted_thinking", "data": "ENC:abc"}
assert parsed["content"][2] == {"type": "text", "text": "Hello"}
assert parsed["content"][3]["name"] == "headroom_retrieve"
assert parsed["content"][3]["input"] == {"hash": "abc"}
assert parsed["stop_reason"] == "refusal"
assert parsed["stop_details"] == stop_details
assert parsed["usage"]["output_tokens"] == 3
openai_handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
parsed_openai = openai_handler._reconstruct_openai_response(
[
{"choices": [{"delta": {"content": "Hi"}}]},
{
"choices": [
{
"delta": {
"tool_calls": [
{
"index": 0,
"id": "call_1",
"function": {
"name": "headroom_retrieve",
"arguments": '{"hash":"aaaaaaaaaaaa',
},
}
]
}
}
]
},
{
"choices": [
{
"delta": {
"tool_calls": [
{
"index": 0,
"function": {"arguments": 'aaaaaaaaaaaa"}'},
}
]
}
}
]
},
]
)
message = parsed_openai["choices"][0]["message"]
assert message["content"] == "Hi"
assert message["tool_calls"][0]["id"] == "call_1"
assert message["tool_calls"][0]["function"]["arguments"] == (
'{"hash":"aaaaaaaaaaaaaaaaaaaaaaaa"}'
)
def test_reconstruct_openai_response_tolerates_null_tool_calls_and_function() -> None:
# Some OpenAI-compatible providers put ``"tool_calls": null`` (and
# ``"function": null``) in a streaming delta instead of omitting the key.
# Only checking key presence made the reconstruction iterate ``None`` and
# raise ``TypeError``, aborting the whole CCR round.
handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
parsed = handler._reconstruct_openai_response(
[
{"choices": [{"delta": {"content": "Hi", "tool_calls": None}}]},
{"choices": [{"delta": {"tool_calls": [{"index": 0, "function": None}]}}]},
{
"choices": [
{
"delta": {
"tool_calls": [
{
"index": 0,
"id": "call_1",
"function": {"name": "f", "arguments": "{}"},
}
]
}
}
]
},
]
)
message = parsed["choices"][0]["message"]
# The null frames did not crash, and the real tool call still reconstructs.
assert message["content"] == "Hi"
assert message["tool_calls"][0]["id"] == "call_1"
assert message["tool_calls"][0]["function"] == {"name": "f", "arguments": "{}"}
def test_extract_assistant_message_responses_output_null_coerces_to_list() -> None:
# A Responses turn with a present-but-null `output` (some gateways send this
# on an empty/filtered turn) must not become None: handle_response later
# does `current_messages.extend(...)` on it, which would raise TypeError.
handler = CCRResponseHandler()
assert handler._extract_assistant_message({"output": None}, "openai_responses") == {
"_openai_responses_output_items": []
}
# An absent output is also an empty list, and a real output passes through.
assert handler._extract_assistant_message({}, "openai_responses") == {
"_openai_responses_output_items": []
}
assert handler._extract_assistant_message(
{"output": [{"type": "message"}]}, "openai_responses"
) == {"_openai_responses_output_items": [{"type": "message"}]}
@pytest.mark.asyncio
async def test_streaming_handler_process_stream_pass_through_and_ccr(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response_handler = CCRResponseHandler()
handler = StreamingCCRHandler(response_handler, provider="anthropic")
passthrough_chunks = [
b'data: {"type":"content_block_delta","delta":{"text":"hello"}}',
b'data: {"stop_reason":"end_turn"}',
]
yielded = [
chunk
async for chunk in handler.process_stream(
_async_iter(passthrough_chunks), [], None, lambda m, t: None
)
]
assert yielded == passthrough_chunks
ccr_handler = StreamingCCRHandler(response_handler, provider="anthropic")
monkeypatch.setattr(
ccr_handler,
"_parse_sse_stream",
lambda data: {
"content": [
{
"type": "tool_use",
"id": "tool_1",
"name": CCR_TOOL_NAME,
"input": {"hash": "abc"},
}
]
},
)
async def fake_handle_response(response, messages, tools, api_call_fn, provider): # noqa: ANN001
return {"content": [{"type": "text", "text": "done"}]}
async def fake_response_to_sse(response): # noqa: ANN001
yield b"event: message_start\n"
yield b"event: message_stop\n"
monkeypatch.setattr(response_handler, "handle_response", fake_handle_response)
monkeypatch.setattr(ccr_handler, "_response_to_sse", fake_response_to_sse)
ccr_chunks = [
b'{"type":"tool_use","name":"headroom_retrieve"',
b',"stop_reason":"tool_use"}',
b"tail",
]
streamed = [
chunk
async for chunk in ccr_handler.process_stream(
_async_iter(ccr_chunks), [], None, lambda m, t: None
)
]
assert streamed == [b"event: message_start\n", b"event: message_stop\n"]
@pytest.mark.asyncio
async def test_streaming_ccr_keeps_classifier_frames_through_continuation(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response_handler = CCRResponseHandler()
handler = StreamingCCRHandler(response_handler, provider="anthropic")
async def fake_handle_response(response, messages, tools, api_call_fn, provider): # noqa: ANN001
return {
"id": "msg_continuation",
"type": "message",
"role": "assistant",
"model": "claude-test",
"content": [{"type": "text", "text": "retrieval complete"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 20, "output_tokens": 4},
}
monkeypatch.setattr(response_handler, "handle_response", fake_handle_response)
initial = b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_initial","type":"message","role":"assistant","model":"claude-test","content":[],"safeguard_results":{"decision":"allow","tool_use_id":"toolu_auto_001"}}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"headroom_retrieve","input":{}}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\\"hash\\":\\"abc\\"}"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: safeguard_results
data: {"type":"safeguard_results","result":{"decision":"allow","tool_use_id":"toolu_auto_001"}}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"}}
event: message_stop
data: {"type":"message_stop"}
"""
streamed = [
chunk
async for chunk in handler.process_stream(
_async_iter([initial]), [], None, lambda m, t: None
)
]
rendered = b"".join(streamed)
assert b"safeguard_results" in rendered
assert b"toolu_auto_001" in rendered
assert b"msg_continuation" in rendered
@pytest.mark.asyncio
async def test_streaming_ccr_replays_malformed_known_frame_through_continuation(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response_handler = CCRResponseHandler()
handler = StreamingCCRHandler(response_handler, provider="anthropic")
async def fake_handle_response(response, messages, tools, api_call_fn, provider): # noqa: ANN001
return {
"id": "msg_continuation",
"type": "message",
"role": "assistant",
"model": "claude-test",
"content": [{"type": "text", "text": "retrieval complete"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 20, "output_tokens": 4},
}
monkeypatch.setattr(response_handler, "handle_response", fake_handle_response)
malformed = b"event: message_delta\ndata: {not-json\n\n"
initial = (
b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_initial","type":"message","role":"assistant","model":"claude-test","content":[]}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"headroom_retrieve","input":{}}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\\"hash\\":\\"abc\\"}"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
"""
+ malformed
+ b"""event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"}}
event: message_stop
data: {"type":"message_stop"}
"""
)
streamed = [
chunk
async for chunk in handler.process_stream(
_async_iter([initial]), [], None, lambda m, t: None
)
]
rendered = b"".join(streamed)
assert b"msg_continuation" in rendered
assert rendered.count(malformed) == 1
@pytest.mark.asyncio
async def test_streaming_handler_falls_back_to_buffer_on_processing_error(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response_handler = CCRResponseHandler()
handler = StreamingCCRHandler(response_handler, provider="openai")
monkeypatch.setattr(
handler,
"_parse_sse_stream",
lambda data: (_ for _ in ()).throw(RuntimeError("parse failed")),
)
# Real OpenAI wire shape: a `tool_calls` delta naming the CCR tool, then
# the `[DONE]` sentinel. This test previously fed Anthropic-shaped bytes to
# an ``openai`` handler, so it never reached the OpenAI detection path.
chunks = [
b'data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,'
b'"id":"call_1","function":{"name":"headroom_retrieve",'
b'"arguments":"{}"}}]}}]}\n\n',
b"data: [DONE]\n\n",
]
streamed = [
chunk
async for chunk in handler.process_stream(_async_iter(chunks), [], None, lambda m, t: None)
]
assert streamed == [b"".join(chunks)]
@pytest.mark.asyncio
async def test_response_to_sse_formats() -> None:
anthropic = StreamingCCRHandler(CCRResponseHandler(), provider="anthropic")
anthropic_chunks = [chunk async for chunk in anthropic._response_to_sse({"content": []})]
assert anthropic_chunks[0].startswith(b"event: message_start\n")
assert anthropic_chunks[-1] == b'event: message_stop\ndata: {"type": "message_stop"}\n\n'
openai = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
openai_chunks = [chunk async for chunk in openai._response_to_sse({"choices": []})]
# An empty body still produces well-formed chunk frames (role, then a
# terminal frame carrying finish_reason) rather than a single non-streaming
# body a streaming client cannot read.
assert openai_chunks[-1] == b"data: [DONE]\n\n"
frames = [json.loads(chunk.decode()[len("data: ") :]) for chunk in openai_chunks[:-1]]
assert [frame["object"] for frame in frames] == ["chat.completion.chunk"] * 2
assert frames[0]["choices"][0]["delta"] == {"role": "assistant"}
assert frames[-1]["choices"][0]["finish_reason"] == "stop"
@pytest.mark.asyncio
async def test_response_to_sse_preserves_anthropic_shape() -> None:
handler = StreamingCCRHandler(CCRResponseHandler(), provider="anthropic")
stop_details = {"type": "refusal", "message": "policy refusal"}
response = {
"id": "msg_1",
"model": "claude-fable-5",
"role": "assistant",
"content": [
{"type": "thinking", "thinking": "", "signature": "sig_fable"},
{"type": "redacted_thinking", "data": "ENC:abc"},
{"type": "text", "text": "done"},
],
"stop_reason": "refusal",
"stop_details": stop_details,
"usage": {"input_tokens": 7, "output_tokens": 3},
}
chunks = [chunk async for chunk in handler._response_to_sse(response)]
events = _sse_json_events(chunks)
message_delta = next(event for event in events if event["type"] == "message_delta")
assert message_delta["delta"]["stop_reason"] == "refusal"
assert message_delta["delta"]["stop_details"] == stop_details
assert any(event.get("delta", {}).get("type") == "signature_delta" for event in events)
assert any(
event.get("content_block", {}).get("type") == "redacted_thinking" for event in events
)
parsed = handler._parse_sse_stream(b"".join(chunks))
assert parsed["content"][0]["signature"] == "sig_fable"
assert parsed["content"][0]["thinking"] == ""
assert parsed["content"][1]["data"] == "ENC:abc"
assert parsed["stop_reason"] == "refusal"
assert parsed["stop_details"] == stop_details
def test_reconstruct_server_tool_use_input_from_partial_json() -> None:
# StreamingCCRHandler._reconstruct_anthropic_response must parse streamed
# input_json_delta into `input` for server_tool_use, not only tool_use.
# The narrow type gate left server_tool_use.input malformed and leaked the
# `_partial_json` scratch key into replayed assistant history → Anthropic
# 400 `server_tool_use.input: Input should be an object` (#2438).
handler = StreamingCCRHandler(CCRResponseHandler(), provider="anthropic")
events = [
{"type": "message_start", "message": {"id": "msg_1", "model": "claude-opus-4"}},
{
"type": "content_block_start",
"index": 0,
"content_block": {
"type": "server_tool_use",
"id": "srvtoolu_1",
"name": "web_search",
"input": {},
},
},
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "input_json_delta", "partial_json": '{"query": "x"}'},
},
{"type": "content_block_stop", "index": 0},
]
response = handler._reconstruct_anthropic_response(events)
block = response["content"][0]
assert block["type"] == "server_tool_use"
assert block["input"] == {"query": "x"}
assert "_partial_json" not in block
def _openai_chunk(delta: dict[str, Any], finish_reason: str | None = None) -> bytes:
"""One `chat.completion.chunk` SSE frame in the shape a real backend sends."""
payload = {
"id": "chatcmpl_1",
"object": "chat.completion.chunk",
"created": 1700000000,
"model": "gpt-4o-mini",
"choices": [{"index": 0, "delta": delta, "finish_reason": finish_reason}],
}
return f"data: {json.dumps(payload)}\n\n".encode()
def test_streaming_buffer_detects_ccr_in_openai_tool_calls_delta() -> None:
# An OpenAI-compatible stream never emits Anthropic's `"type":"tool_use"`
# marker; its tool calls arrive as a `tool_calls` array inside
# `choices[].delta`. Scanning only for the Anthropic marker meant CCR was
# never detected on this provider.
buffer = StreamingCCRBuffer(provider="openai")
assert buffer.add_chunk(_openai_chunk({"role": "assistant"})) is False
detected = buffer.add_chunk(
_openai_chunk(
{
"tool_calls": [
{
"index": 0,
"id": "call_1",
"function": {"name": CCR_TOOL_NAME, "arguments": ""},
}
]
}
)
)
assert detected is True
assert buffer.detected_ccr is True
# A non-CCR tool call on the same provider must not trip detection.
other = StreamingCCRBuffer(provider="openai")
assert (
other.add_chunk(
_openai_chunk(
{"tool_calls": [{"index": 0, "id": "c", "function": {"name": "other_tool"}}]}
)
)
is False
)
assert other.detected_ccr is False
@pytest.mark.asyncio
async def test_openai_stream_without_ccr_yields_every_chunk() -> None:
# A short OpenAI stream with no CCR call must pass through byte for byte.
# End-of-stream was detected by scanning for Anthropic's `stop_reason`,
# which an OpenAI stream never contains, so nothing was ever flushed and
# the client received an empty response.
handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
chunks = [
_openai_chunk({"role": "assistant"}),
_openai_chunk({"content": "hello "}),
_openai_chunk({"content": "world"}),
_openai_chunk({}, finish_reason="stop"),
b"data: [DONE]\n\n",
]
streamed = [
chunk
async for chunk in handler.process_stream(_async_iter(chunks), [], None, lambda m, t: None)
]
assert streamed == chunks
@pytest.mark.asyncio
async def test_openai_stream_past_flush_threshold_keeps_the_tail() -> None:
# Past 10 000 buffered bytes the handler flushes in batches. Without an
# end-of-stream match the final sub-threshold batch was never flushed, so
# a long response visibly stopped mid-sentence.
handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
chunks = [
_openai_chunk({"role": "assistant"}),
_openai_chunk({"content": "x" * 11000}),
_openai_chunk({"content": "the tail that used to be dropped"}),
_openai_chunk({}, finish_reason="stop"),
b"data: [DONE]\n\n",
]
streamed = [
chunk
async for chunk in handler.process_stream(_async_iter(chunks), [], None, lambda m, t: None)
]
assert streamed == chunks
assert b"the tail that used to be dropped" in b"".join(streamed)
@pytest.mark.asyncio
async def test_openai_stream_without_done_sentinel_still_flushes() -> None:
# Upstream can truncate before `[DONE]`, and some gateways omit it. Bytes
# left in the buffer when the source iterator is exhausted are real
# response data, so they are flushed rather than discarded.
handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
chunks = [_openai_chunk({"role": "assistant"}), _openai_chunk({"content": "partial answer"})]
streamed = [
chunk
async for chunk in handler.process_stream(_async_iter(chunks), [], None, lambda m, t: None)
]
assert streamed == chunks
def test_reconstruct_openai_response_marks_tool_calls_finish_reason() -> None:
# OpenAI requires `finish_reason: "tool_calls"` when the message carries
# tool calls. It was hardcoded to "stop", so a client driving its agent
# loop off `finish_reason` ended the turn instead of running the tools.
handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
parsed = handler._reconstruct_openai_response(
[
{"id": "chatcmpl_1", "model": "gpt-4o-mini", "created": 1700000000},
{
"choices": [
{
"delta": {
"tool_calls": [
{
"index": 0,
"id": "call_1",
"function": {
"name": CCR_TOOL_NAME,
"arguments": '{"hash":"abc"}',
},
}
]
},
"finish_reason": None,
}
]
},
{
"choices": [{"delta": None, "finish_reason": "tool_calls"}],
"usage": {"prompt_tokens": 12, "completion_tokens": 3},
},
]
)
assert parsed["choices"][0]["finish_reason"] == "tool_calls"
# The chunk envelope is carried through so the reconstructed body is a
# valid `chat.completion` rather than a bare `choices` list.
assert parsed["object"] == "chat.completion"
assert parsed["id"] == "chatcmpl_1"
assert parsed["model"] == "gpt-4o-mini"
assert parsed["created"] == 1700000000
assert parsed["usage"] == {"prompt_tokens": 12, "completion_tokens": 3}
def test_reconstruct_openai_response_keeps_upstream_finish_reason() -> None:
# With no tool calls, the upstream reason is preserved instead of being
# rewritten to "stop": a truncated turn must stay reported as truncated.
handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
parsed = handler._reconstruct_openai_response(
[
{"choices": [{"delta": {"content": "half an ans"}, "finish_reason": None}]},
{"choices": [{"delta": {}, "finish_reason": "length"}]},
]
)
assert parsed["choices"][0]["finish_reason"] == "length"
assert parsed["choices"][0]["message"]["content"] == "half an ans"
# And an absent reason still defaults to "stop".
defaulted = handler._reconstruct_openai_response(
[{"choices": [{"delta": {"content": "hi"}}]}],
)
assert defaulted["choices"][0]["finish_reason"] == "stop"
@pytest.mark.asyncio
async def test_response_to_sse_emits_openai_chunk_frames() -> None:
# A streaming client reads `choices[].delta`. Re-serialising the
# reconstructed non-streaming body (`choices[].message`) into one SSE frame
# made both the content and the tool calls invisible to it.
handler = StreamingCCRHandler(CCRResponseHandler(), provider="openai")
response = {
"id": "chatcmpl_2",
"object": "chat.completion",
"created": 1700000001,
"model": "gpt-4o-mini",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "done",
"tool_calls": [
{
"id": "call_9",
"type": "function",
"function": {"name": "do_thing", "arguments": '{"a":1}'},
}
],
},
"finish_reason": "tool_calls",
}
],
}
chunks = [chunk async for chunk in handler._response_to_sse(response)]
assert chunks[-1] == b"data: [DONE]\n\n"
frames = [json.loads(chunk.decode()[len("data: ") :]) for chunk in chunks[:-1]]
assert all(frame["object"] == "chat.completion.chunk" for frame in frames)
assert all("delta" in frame["choices"][0] for frame in frames)
assert all(frame["id"] == "chatcmpl_2" for frame in frames)
deltas = [frame["choices"][0]["delta"] for frame in frames]
assert deltas[0] == {"role": "assistant"}
assert deltas[1] == {"content": "done"}
assert deltas[2]["tool_calls"][0]["id"] == "call_9"
assert deltas[2]["tool_calls"][0]["index"] == 0
assert deltas[2]["tool_calls"][0]["function"] == {"name": "do_thing", "arguments": '{"a":1}'}
assert frames[-1]["choices"][0]["finish_reason"] == "tool_calls"