fix(proxy): preserve Claude Code auto-mode protocol (#3784)

## Description

Fix Claude Code server-side auto-mode compatibility when requests pass
through
Headroom’s native Anthropic Messages endpoint.

Previously, auto-mode capability headers, safeguard payloads, query
parameters,
unknown response fields, or opaque SSE frames could be rewritten or lost
while
proxying and handling CCR continuations. This could trigger Claude
Code’s
client-side classifier fallback and its associated warning.

This change adds a request-local Anthropic wire envelope, preserves
client-owned
capability data and opaque response attachments, prevents classifier
payloads
from entering Headroom-owned observations, and documents that
schema-translating
Bedrock, Vertex, and LiteLLM routes do not currently support the
server-side
auto-mode protocol.


Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Preserve native Anthropic request paths and raw query parameters.
- Preserve client-owned `anthropic-version`, `anthropic-beta`, and
safeguard data.
- Avoid sticky-beta, memory, or turn-hook rewrites on classifier-bearing
turns.
- Preserve unknown Anthropic response fields across CCR continuations.
- Preserve opaque SSE frames through request-local parse/render
envelopes.
- Reject unknown or malformed deltas during strict buffered
reconstruction.
- Redact safeguard payloads from diagnostics and pipeline observations.
- Consolidate Anthropic SSE parsing and rendering into the wire module.
- Document auto-mode limitations for Bedrock, Vertex, and LiteLLM
translation paths.
- Add request, response, streaming, CCR, malformed-byte, and passthrough
regression tests.

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output


```text
$ python -m pytest -p no:cacheprovider \
    tests/test_anthropic_auto_mode_passthrough.py \
    tests/test_anthropic_buffered_sse.py \
    tests/test_ccr_response_handler_extra.py \
    tests/test_ccr_buffered_stream_signed_thinking.py \
    tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py \
    tests/test_buffered_ccr_salvage.py \
    tests/test_buffered_ccr_accept_header.py \
    tests/test_proxy/test_anthropic_copilot_upstream_auth.py \
    tests/test_proxy_streaming_request_logger.py \
    tests/test_proxy_streaming_resilience.py -q

125 passed, 1 warning in 13.70s

$ ruff check --no-cache .
All checks passed!

$ ruff format --check --no-cache .
1559 files already formatted

$ python -m mypy \
    headroom/proxy/anthropic_wire.py \
    headroom/ccr/response_handler.py \
    headroom/proxy/handlers/anthropic.py \
    headroom/proxy/handlers/streaming.py

Success: no issues found in 4 source files

$ npx --no-install commitlint \
    --from origin/main --to HEAD \
    --config .commitlintrc.json

Exited successfully.
```

## Real Behavior Proof

- Environment: Pending. Record OS, Python version, Claude Code version,
provider/model, and Headroom revision.
- Exact command / steps: Pending. Start Headroom with `headroom proxy
--port 8787`, then run Claude Code with
`ANTHROPIC_BASE_URL=http://127.0.0.1:8787` and exercise server-side auto
mode.
- Observed result: Pending. Attach redacted output or logs showing the
request completed through Headroom and whether Claude Code used its
client-side fallback.
- Not tested: Live Claude Code auto mode against Anthropic; Bedrock,
Vertex, and LiteLLM translation routes.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Native Anthropic safeguard-capable
requests.
- Kill switch / disable path: Set `CLAUDE_CODE_AUTO_MODE_SERVER=0` to
use Claude Code’s client-side fallback.
- Unsafe override required: No.
- Qualification impact: Live qualification is pending. Translation
routes remain documented as unsupported for server-side auto mode.
- Rollback path: Revert this PR or deploy the previous Headroom
revision.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code where needed
- [x] I have made corresponding documentation changes
- [x] No new warnings were observed in focused tests
- [x] I have added tests for the fix
- [ ] Full existing and new unit test suite passes locally
- [x] I did **not** edit `CHANGELOG.md`

## Screenshots (if applicable)

N/A. This change affects proxy wire behavior.
This commit is contained in:
Devanshi Vyas
2026-09-25 09:53:27 -07:00
committed by GitHub
parent 7ce2580ff9
commit 12c157966f
13 changed files with 1523 additions and 528 deletions
+6
View File
@@ -112,6 +112,12 @@ Or via `~/.claude/settings.json`:
Claude Code now talks plain Anthropic `/v1/messages` to Headroom; Headroom compresses
and forwards to Bedrock via LiteLLM, then translates the answer back.
This translating route is not currently compatible with Claude Code's server-side
auto-mode classifier protocol. Ordinary model traffic remains fail-open, but
classifier-bearing turns may use Claude Code's billable client-side fallback.
Use native Anthropic transport for server-side auto mode, or set
`CLAUDE_CODE_AUTO_MODE_SERVER=0` when the fallback is intentional.
## Application inference profiles (account-specific ARNs)
If your IAM policy only permits **application inference profiles** (account-specific
+4
View File
@@ -114,5 +114,9 @@ and has the smallest moving parts. Use this alternative only if you can't set
- Pick a Claude model that is enabled in your Vertex project/region
(e.g. `claude-sonnet-4-6`, `claude-haiku-4-5`).
- Streaming, tool use, and prompt caching all work through Headroom.
- Claude Code's server-side auto-mode classifier is supported only by the
native Anthropic-compatible wire path. The Vertex translation path remains
fail-open but may use Claude Code's billable client-side classifier fallback
until a provider-native safeguard fixture is verified.
- Want to point at a private Vertex gateway instead of Google's host? Start the
proxy with `--vertex-api-url https://your-gateway` and Headroom will forward there.
+8
View File
@@ -43,6 +43,14 @@ The callback compresses messages in LiteLLM's `async_pre_call_hook` before they
This works with every provider LiteLLM supports: OpenAI, Anthropic, Bedrock, Azure, Vertex AI, Cohere, Groq, Mistral, Together, Ollama, and more.
<Callout type="warning" title="Claude Code auto mode">
LiteLLM and other schema-translating routes are not claimed to preserve Claude
Code's server-side auto-mode classifier protocol. Requests continue to fail
open as ordinary model traffic, but Claude Code may use its billable
client-side classifier fallback. Native Anthropic transport is the supported
path until an adapter has an end-to-end safeguard fixture.
</Callout>
## With LiteLLM Proxy
If you run LiteLLM as a proxy server, use the ASGI middleware:
+38 -124
View File
@@ -19,6 +19,8 @@ from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from typing import Any
from headroom.proxy.anthropic_wire import AnthropicSSEEnvelope
from ..cache.compression_store import format_retrieval_miss_detail, get_compression_store
from .tool_calls import (
CCRToolCall,
@@ -725,8 +727,19 @@ class StreamingCCRHandler:
# Parse the complete response
try:
# For SSE streams, we need to parse the accumulated data
complete_data = self._parse_sse_stream(self.buffer.get_accumulated())
# The envelope is request-local. It carries opaque Anthropic
# frames over a CCR continuation without coupling concurrent
# streams through this long-lived handler instance.
stream_bytes = self.buffer.get_accumulated()
complete_data = self._parse_sse_stream(stream_bytes)
envelope = None
if self.provider == "anthropic":
candidate = self._parse_anthropic_sse_envelope(stream_bytes)
# Keep the long-standing parser seam usable by callers and
# tests that replace it. Only attach the wire envelope when
# that parser returned the corresponding native message.
if complete_data == candidate.message:
envelope = candidate
# Handle CCR
final_response = await self.response_handler.handle_response(
@@ -739,7 +752,11 @@ class StreamingCCRHandler:
# Re-stream the final response
# Convert back to SSE format
async for chunk in self._response_to_sse(final_response):
if envelope is None:
response_stream = self._response_to_sse(final_response)
else:
response_stream = self._response_to_sse(final_response, envelope=envelope)
async for chunk in response_stream:
yield chunk
except Exception as e:
@@ -759,6 +776,9 @@ class StreamingCCRHandler:
event is an upstream protocol bug — surfaced loudly, not
silently corrupted.
"""
if self.provider == "anthropic":
return self._parse_anthropic_sse_envelope(data).message
from headroom.proxy.helpers import parse_sse_events_from_byte_buffer
# Accumulate all event data via the canonical bytes-buffer
@@ -785,132 +805,20 @@ class StreamingCCRHandler:
len(buf),
)
# Reconstruct response from events
# This is provider-specific
if self.provider == "anthropic":
return self._reconstruct_anthropic_response(events)
else:
return self._reconstruct_openai_response(events)
return self._reconstruct_openai_response(events)
def _reconstruct_anthropic_response(
self,
events: list[dict[str, Any]],
) -> dict[str, Any]:
"""Reconstruct Anthropic response from stream events."""
response: dict[str, Any] = {
"content": [],
"stop_reason": None,
"usage": {},
}
return AnthropicSSEEnvelope.from_events(events).message
blocks_by_index: dict[int, dict[str, Any]] = {}
current_block: dict[str, Any] | None = None
@staticmethod
def _parse_anthropic_sse_envelope(data: bytes) -> AnthropicSSEEnvelope:
"""Parse a complete Anthropic stream into its request-local envelope."""
for event in events:
event_type = event.get("type", "")
if event_type == "content_block_start":
block = event.get("content_block", {})
block_index = event.get("index", len(blocks_by_index))
btype = block.get("type")
current_block = {"type": btype}
if btype == "text":
current_block["text"] = block.get("text", "")
elif btype == "tool_use":
current_block.update(
{
"id": block.get("id", ""),
"name": block.get("name", ""),
"input": {},
}
)
elif btype == "thinking":
current_block["thinking_buffer"] = block.get("thinking", "")
if "signature" in block:
current_block["signature"] = block["signature"]
elif btype == "redacted_thinking":
if "data" in block:
current_block["data"] = block["data"]
elif btype:
current_block = dict(block)
blocks_by_index[block_index] = current_block
elif event_type == "content_block_delta":
idx = event.get("index")
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
if target is None:
continue
delta = event.get("delta", {})
dtype = delta.get("type")
if dtype == "text_delta":
target["text"] = target.get("text", "") + delta.get("text", "")
elif dtype == "input_json_delta":
# Accumulate for any block streaming input (tool_use AND
# server_tool_use); the stop handler parses it into `input`
# (#2438).
partial = delta.get("partial_json", "")
target["_partial_json"] = target.get("_partial_json", "") + partial
elif dtype == "thinking_delta":
target["thinking_buffer"] = target.get("thinking_buffer", "") + delta.get(
"thinking", ""
)
elif dtype == "signature_delta":
if "signature" in delta:
target["signature"] = delta["signature"]
elif dtype == "citations_delta":
citation = delta.get("citation")
if citation is not None:
target.setdefault("citations", []).append(citation)
elif event_type == "content_block_stop":
idx = event.get("index")
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
if target is not None:
# Parse streamed `_partial_json` into `input` for any block
# that carried input_json_delta — tool_use AND
# server_tool_use — not just tool_use. The narrow type gate
# left server_tool_use.input malformed and leaked the scratch
# key into replayed history (#2438). Always strip the key.
if "_partial_json" in target:
partial = target.pop("_partial_json")
try:
target["input"] = json.loads(partial) if partial else {}
except json.JSONDecodeError:
target["input"] = {}
if target.get("type") == "thinking" and "thinking_buffer" in target:
target["thinking"] = target.pop("thinking_buffer")
if target not in response["content"]:
response["content"].append(target)
current_block = None
elif event_type == "message_start":
msg = event.get("message", {})
if "id" in msg:
response["id"] = msg["id"]
if "model" in msg:
response["model"] = msg["model"]
if "role" in msg:
response["role"] = msg["role"]
if "stop_reason" in msg:
response["stop_reason"] = msg["stop_reason"]
if "stop_details" in msg:
response["stop_details"] = msg["stop_details"]
if msg.get("usage"):
response["usage"].update(msg["usage"])
elif event_type == "message_delta":
delta = event.get("delta", {})
if "stop_reason" in delta:
response["stop_reason"] = delta["stop_reason"]
if "stop_details" in delta:
response["stop_details"] = delta["stop_details"]
if event.get("usage"):
response["usage"].update(event["usage"])
elif event_type == "message_stop":
pass
return response
return AnthropicSSEEnvelope.parse(data)
def _reconstruct_openai_response(
self,
@@ -1082,6 +990,8 @@ class StreamingCCRHandler:
async def _response_to_sse(
self,
response: dict[str, Any],
*,
envelope: Any | None = None,
) -> Any: # AsyncGenerator[bytes, None]
"""Convert a response back to SSE format for streaming.
@@ -1089,10 +999,14 @@ class StreamingCCRHandler:
to chunk the response more granularly.
"""
if self.provider == "anthropic":
from headroom.proxy.handlers.streaming import StreamingMixin
if envelope is not None:
for chunk in envelope.render(response):
yield chunk
else:
from headroom.proxy.handlers.streaming import StreamingMixin
for chunk in StreamingMixin()._response_to_sse(response, "anthropic"):
yield chunk
for chunk in StreamingMixin()._response_to_sse(response, "anthropic"):
yield chunk
else:
# OpenAI SSE format: `chat.completion.chunk` frames, then [DONE].
for chunk in self._openai_response_to_chunks(response):
+682
View File
@@ -0,0 +1,682 @@
"""Loss-minimizing helpers for the native Anthropic Messages wire envelope.
The proxy is allowed to understand the message content it optimizes, but it
must not need to understand every field Anthropic adds to the surrounding
protocol. This module keeps that distinction explicit: known message events
are reconstructed into a normal response dictionary, while every SSE frame the
reconstruction cannot use (unknown events, and known events whose bytes or
payload are malformed) is retained as opaque bytes and replayed by
:meth:`AnthropicSSEEnvelope.render`.
The opaque data is deliberately kept out of the provider JSON. It is an
internal rendering detail and can therefore never accidentally be serialized
upstream as a synthetic Headroom field.
"""
from __future__ import annotations
import copy
import json
import re
from collections.abc import Collection
from dataclasses import dataclass, field, replace
from typing import Any
from headroom.copilot_auth import build_copilot_upstream_url
_DANGEROUS_TOOL_USE_RE = re.compile(r"(?:^|,)\s*dangerous-tool-use-[^,\s]+", re.IGNORECASE)
# Members of each known event (and delta type) that reconstruction interprets.
# Anything else on a known frame is an extension carried by ``_FrameExtensions``.
_EVENT_FIELDS: dict[str, frozenset[str]] = {
"message_start": frozenset({"type", "message"}),
"content_block_start": frozenset({"type", "index", "content_block"}),
"content_block_delta": frozenset({"type", "index", "delta"}),
"content_block_stop": frozenset({"type", "index"}),
"message_delta": frozenset({"type", "delta", "usage"}),
"message_stop": frozenset({"type"}),
}
_DELTA_FIELDS: dict[str, frozenset[str]] = {
"text_delta": frozenset({"type", "text"}),
"input_json_delta": frozenset({"type", "partial_json"}),
"thinking_delta": frozenset({"type", "thinking"}),
"signature_delta": frozenset({"type", "signature"}),
"citations_delta": frozenset({"type", "citation"}),
}
_MESSAGE_DELTA_FIELDS = ("stop_reason", "stop_sequence", "stop_details")
# The payload member each known delta type must carry, and its JSON type.
_DELTA_PAYLOADS: dict[str, tuple[str, type]] = {
"text_delta": ("text", str),
"input_json_delta": ("partial_json", str),
"thinking_delta": ("thinking", str),
"signature_delta": ("signature", str),
"citations_delta": ("citation", dict),
}
# The block field each string delta is concatenated onto.
_ACCUMULATED_FIELDS = {
"text_delta": "text",
"input_json_delta": "_partial_json",
"thinking_delta": "thinking",
}
_KNOWN_MESSAGE_FIELDS = frozenset(
{
"id",
"type",
"role",
"model",
"content",
"stop_reason",
"stop_sequence",
"stop_details",
"usage",
}
)
@dataclass(frozen=True)
class _SSEFrame:
"""One raw SSE frame and its decoded payload, when it has one."""
raw: bytes
event_type: str
payload: dict[str, Any] | None
def build_anthropic_upstream_url(base_url: str, path: str, raw_query: str = "") -> str:
"""Build an Anthropic upstream URL without dropping the incoming query.
``raw_query`` is intentionally not parsed and re-encoded. Beta query
parameters are protocol data, so preserving their spelling and ordering is
safer than normalizing them through a mapping.
"""
url = build_copilot_upstream_url(base_url, path)
if raw_query:
return f"{url}?{raw_query}"
return url
def has_dangerous_tool_use_beta(value: str | None) -> bool:
"""Return whether a beta header contains an auto-mode capability token."""
return bool(value and _DANGEROUS_TOOL_USE_RE.search(value))
def is_safeguard_capable_request(body: Any, anthropic_beta: str | None) -> bool:
"""Classify a request without inspecting or retaining the safeguard value."""
return isinstance(body, dict) and (
"safeguards" in body or has_dangerous_tool_use_beta(anthropic_beta)
)
def strip_safeguard_payload(value: Any) -> Any:
"""Copy an observation while removing classifier payload fields.
This is only for logs, pipeline events, and diagnostics. The actual
upstream response is never passed through this helper, so wire fidelity is
unaffected.
"""
if isinstance(value, dict):
return {
key: strip_safeguard_payload(item)
for key, item in value.items()
if key not in {"safeguards", "safeguard_results"}
}
if isinstance(value, list):
return [strip_safeguard_payload(item) for item in value]
return value
def preserve_opaque_response_fields(
original: dict[str, Any], replacement: dict[str, Any]
) -> dict[str, Any]:
"""Retain unknown Anthropic response attachments across a transform.
Response transforms such as CCR may replace the message produced by the
first upstream call with a continuation message. Capability attachments
belong to the surrounding protocol rather than to Headroom's mutable
message view, so keep them unless the replacement explicitly supplies its
own value.
"""
merged = copy.deepcopy(replacement)
for key, value in original.items():
if key not in _KNOWN_MESSAGE_FIELDS:
merged.setdefault(key, copy.deepcopy(value))
return merged
def _split_frames(raw: bytes) -> list[bytes]:
"""Split SSE bytes while retaining each frame's original delimiters."""
frames: list[bytes] = []
start = 0
for match in re.finditer(rb"\r\n\r\n|\n\n|\r\r", raw):
end = match.end()
if raw[start : match.start()].strip():
frames.append(raw[start:end])
start = end
if raw[start:].strip():
frames.append(raw[start:])
return frames
def _frame_parts(raw: bytes) -> tuple[str, str | None]:
event_name = ""
data_lines: list[bytes] = []
for line in raw.splitlines():
if line.startswith(b"event:"):
try:
event_name = line[6:].strip().decode("utf-8")
except UnicodeDecodeError:
return "", None
elif line.startswith(b"data:"):
data_lines.append(line[5:].lstrip())
try:
return event_name, b"\n".join(data_lines).decode("utf-8")
except UnicodeDecodeError:
# Leave the complete frame opaque so rendering can replay its exact
# bytes. Replacing malformed input would violate the proxy's fail-open
# wire-fidelity contract.
return event_name, None
def _json_payload(raw: bytes) -> tuple[str, dict[str, Any] | None]:
event_name, data = _frame_parts(raw)
if data is None:
return event_name, None
if not data or data == "[DONE]":
return event_name or "[DONE]", None
try:
payload = json.loads(data)
except (TypeError, json.JSONDecodeError):
return event_name, None
if not isinstance(payload, dict):
return event_name, None
return event_name or str(payload.get("type", "")), payload
def _sse_event(payload: dict[str, Any], *, event_name: str | None = None) -> bytes:
name = event_name or str(payload.get("type", "message"))
return f"event: {name}\ndata: {json.dumps(payload, ensure_ascii=False)}\n\n".encode()
def _wire_index(value: Any) -> int | None:
"""Return a content block index, or None when it is not a valid one.
``index`` is provider JSON and may be any JSON value. Only a non-negative
integer identifies a block; anything else (including an unhashable object
or array) must leave its frame unreconstructable instead of reaching a
dictionary lookup.
"""
if isinstance(value, int) and not isinstance(value, bool) and value >= 0:
return int(value)
return None
def _extensions(payload: dict[str, Any], known: Collection[str]) -> dict[str, Any]:
"""Return a copy of the members of ``payload`` this module does not interpret."""
return {key: copy.deepcopy(value) for key, value in payload.items() if key not in known}
@dataclass
class _BlockExtensions:
"""Unknown members carried by one content block's known frames."""
start: dict[str, Any] = field(default_factory=dict)
stop: dict[str, Any] = field(default_factory=dict)
# delta type -> (members on the event, members inside its ``delta``)
deltas: dict[str, tuple[dict[str, Any], dict[str, Any]]] = field(default_factory=dict)
@dataclass
class _FrameExtensions:
"""Unknown members of known frames, keyed by the canonical frame they rode on.
Rendering regenerates known frames from the message dictionary, which has
no place for members added to the events themselves. They are kept here
and re-attached to the corresponding regenerated frame instead.
"""
message_start: dict[str, Any] = field(default_factory=dict)
message_delta: dict[str, Any] = field(default_factory=dict)
message_delta_delta: dict[str, Any] = field(default_factory=dict)
# ``message_delta.usage`` keys upstream sent; values come from the
# rendered response so a CCR continuation reports its own usage.
message_delta_usage_keys: list[str] = field(default_factory=list)
message_stop: dict[str, Any] = field(default_factory=dict)
# Keyed by position in ``message["content"]``, not by wire index.
blocks: dict[int, _BlockExtensions] = field(default_factory=dict)
def for_content(self, original: Any, rendered: Any) -> _FrameExtensions:
"""Keep block members only where the rendered block is the original one.
A response transform such as CCR can replace content blocks; members
observed on the original block's frames do not describe its
replacement. Message-level members belong to the surrounding protocol
and are always kept.
"""
before = original if isinstance(original, list) else []
after = rendered if isinstance(rendered, list) else []
blocks = {
position: extensions
for position, extensions in self.blocks.items()
if position < len(before)
and position < len(after)
and after[position] == before[position]
}
return replace(self, blocks=blocks)
@dataclass
class _Reconstruction:
message: dict[str, Any]
extensions: _FrameExtensions
saw_start: bool
saw_stop: bool
saw_error: bool
open_blocks: set[int]
# Positions of the frames the message (plus extensions) was built from.
consumed: set[int]
def _response_from_events(frames: list[_SSEFrame]) -> _Reconstruction:
"""Reconstruct the message and report which frames it was built from.
The ``consumed`` positions are the only frames that
:func:`_render_known_response` can regenerate. Every other frame,
including a known event whose payload is malformed or does not fit the
stream's structure, must be replayed verbatim.
"""
response: dict[str, Any] = {"content": [], "usage": {}}
extensions = _FrameExtensions()
blocks_by_index: dict[int, dict[str, Any]] = {}
# Keyed by ``id()`` of the block dictionary; the dictionaries stay alive
# in ``blocks_by_index`` or ``response["content"]`` for the whole parse.
block_extensions: dict[int, _BlockExtensions] = {}
current_block: dict[str, Any] | None = None
appended: set[int] = set()
open_blocks: set[int] = set()
consumed: set[int] = set()
saw_start = saw_stop = saw_error = False
for position, frame in enumerate(frames):
data = frame.payload
event_type = frame.event_type
if data is None:
continue
if event_type == "message_start":
saw_start = True
message = data.get("message")
usage = message.get("usage") if isinstance(message, dict) else None
# ``usage`` is merged into by later message_delta frames, so a
# non-object value would make the message unreconstructable.
if isinstance(message, dict) and (usage is None or isinstance(usage, dict)):
consumed.add(position)
extensions.message_start.update(_extensions(data, _EVENT_FIELDS[event_type]))
# Start with the complete upstream message object. This is
# what preserves future top-level attachments.
response.update(copy.deepcopy(message))
# The stream's content blocks below are authoritative. A
# defensive non-empty content array in message_start must not
# be duplicated when those blocks are replayed.
response["content"] = []
if response.get("usage") is None:
response["usage"] = {}
elif event_type == "content_block_start":
block = data.get("content_block")
if not isinstance(block, dict):
continue
raw_index = data.get("index")
if raw_index is None:
index = len(response["content"])
else:
parsed_index = _wire_index(raw_index)
if parsed_index is None:
continue
index = parsed_index
current_block = copy.deepcopy(block)
blocks_by_index[index] = current_block
block_extensions[id(current_block)] = _BlockExtensions(
start=_extensions(data, _EVENT_FIELDS[event_type])
)
open_blocks.add(index)
consumed.add(position)
elif event_type == "content_block_delta":
delta = data.get("delta")
if not isinstance(delta, dict):
continue
raw_index = data.get("index")
block_index = _wire_index(raw_index)
if raw_index is not None and block_index is None:
continue
target = blocks_by_index.get(block_index) if block_index is not None else current_block
dtype = delta.get("type")
if target is None or not isinstance(dtype, str) or dtype not in _DELTA_FIELDS:
continue
# Validate the raw member: a missing, null or falsy non-string value
# is not an empty delta, and must not be consumed as one.
member, member_type = _DELTA_PAYLOADS[dtype]
payload = delta.get(member)
if not isinstance(payload, member_type):
continue
accumulated = _ACCUMULATED_FIELDS.get(dtype)
if accumulated is not None and not isinstance(target.get(accumulated, ""), str):
continue
if dtype == "citations_delta" and not isinstance(target.get("citations", []), list):
continue
consumed.add(position)
event_extensions, delta_extensions = block_extensions[id(target)].deltas.setdefault(
dtype, ({}, {})
)
event_extensions.update(_extensions(data, _EVENT_FIELDS[event_type]))
delta_extensions.update(_extensions(delta, _DELTA_FIELDS[dtype]))
if accumulated is not None:
target[accumulated] = target.get(accumulated, "") + payload
elif dtype == "signature_delta":
target["signature"] = payload
elif dtype == "citations_delta":
target.setdefault("citations", []).append(payload)
elif event_type == "content_block_stop":
raw_index = data.get("index")
block_index = _wire_index(raw_index)
if raw_index is not None and block_index is None:
continue
target = blocks_by_index.get(block_index) if block_index is not None else current_block
if target is None:
continue
consumed.add(position)
block_extensions[id(target)].stop.update(_extensions(data, _EVENT_FIELDS[event_type]))
partial = target.pop("_partial_json", None)
if partial is not None:
try:
target["input"] = json.loads(partial) if partial else {}
except (TypeError, json.JSONDecodeError):
target["input"] = {}
appended_key = block_index if block_index is not None else id(target)
if appended_key not in appended:
response["content"].append(target)
appended.add(appended_key)
if block_index is not None:
open_blocks.discard(block_index)
current_block = None
elif event_type == "message_delta":
delta = data.get("delta")
usage = data.get("usage")
if (delta is not None and not isinstance(delta, dict)) or (
usage is not None and not isinstance(usage, dict)
):
continue
consumed.add(position)
if isinstance(delta, dict):
for key in _MESSAGE_DELTA_FIELDS:
if key in delta:
response[key] = copy.deepcopy(delta[key])
extensions.message_delta_delta.update(_extensions(delta, _MESSAGE_DELTA_FIELDS))
if isinstance(usage, dict):
response.setdefault("usage", {}).update(copy.deepcopy(usage))
for key in usage:
if key not in extensions.message_delta_usage_keys:
extensions.message_delta_usage_keys.append(key)
# Some Anthropic additions are attached directly to the delta
# event rather than nested below ``delta``. Preserve them as
# response fields without interpreting their schemas.
event_extensions = _extensions(data, _EVENT_FIELDS[event_type])
response.update(copy.deepcopy(event_extensions))
extensions.message_delta.update(event_extensions)
elif event_type == "message_stop":
saw_stop = True
consumed.add(position)
extensions.message_stop.update(_extensions(data, _EVENT_FIELDS[event_type]))
elif event_type == "error":
saw_error = True
extensions.blocks = {
index: block_extensions[id(block)]
for index, block in enumerate(response["content"])
if id(block) in block_extensions
}
return _Reconstruction(
response, extensions, saw_start, saw_stop, saw_error, open_blocks, consumed
)
@dataclass
class AnthropicSSEEnvelope:
"""Parsed Anthropic response plus opaque frames retained for replay."""
message: dict[str, Any]
# (number of reconstructed frames before it, raw bytes) for every frame
# that the reconstructed message cannot regenerate.
_opaque_frames: list[tuple[int, bytes]]
_extensions: _FrameExtensions
saw_message_start: bool
saw_message_stop: bool
saw_error: bool
saw_unreconstructable_frame: bool
open_block_indices: set[int]
@classmethod
def parse(cls, raw_sse_bytes: bytes) -> AnthropicSSEEnvelope:
"""Parse Anthropic SSE bytes while retaining opaque frames verbatim.
Args:
raw_sse_bytes: Complete or partial Anthropic SSE wire bytes.
Returns:
A request-local envelope containing the reconstructed message and
every original frame needed for loss-minimizing replay.
"""
frames = [_SSEFrame(raw, *_json_payload(raw)) for raw in _split_frames(raw_sse_bytes)]
reconstruction = _response_from_events(frames)
# Anything the reconstruction did not consume is replayed verbatim.
# A recognized event name is not enough: a known frame with malformed
# bytes or an unusable payload would otherwise be neither rendered
# nor replayed, and would silently disappear from the stream.
opaque_frames: list[tuple[int, bytes]] = []
known_count = 0
saw_unreconstructable_frame = False
for position, frame in enumerate(frames):
if position in reconstruction.consumed:
known_count += 1
continue
if frame.event_type in _EVENT_FIELDS:
saw_unreconstructable_frame = True
opaque_frames.append((known_count, frame.raw))
return cls(
reconstruction.message,
opaque_frames,
reconstruction.extensions,
reconstruction.saw_start,
reconstruction.saw_stop,
reconstruction.saw_error,
saw_unreconstructable_frame,
reconstruction.open_blocks,
)
@classmethod
def from_events(cls, events: list[dict[str, Any]]) -> AnthropicSSEEnvelope:
"""Build an envelope for callers that already decoded event JSON."""
raw = b"".join(_sse_event(event) for event in events if isinstance(event, dict))
return cls.parse(raw)
def is_complete(self) -> bool:
"""Return whether the stream has complete structural framing."""
return (
self.saw_message_start
and self.saw_message_stop
and not self.saw_error
and not self.open_block_indices
)
def is_message_reconstructable(self) -> bool:
"""Return whether strict callers can safely use the message dictionary."""
return self.is_complete() and not self.saw_unreconstructable_frame
def render(self, message: dict[str, Any] | None = None) -> list[bytes]:
"""Render a changed message and replay opaque frames at their anchors."""
response = preserve_opaque_response_fields(
self.message,
message if isinstance(message, dict) else self.message,
)
extensions = self._extensions.for_content(
self.message.get("content"), response.get("content")
)
standard = _render_known_response(response, extensions)
if self._opaque_frames:
by_anchor: dict[int, list[bytes]] = {}
for known_before, raw in self._opaque_frames:
by_anchor.setdefault(min(known_before, len(standard)), []).append(raw)
out: list[bytes] = []
for index in range(len(standard) + 1):
out.extend(by_anchor.get(index, []))
if index < len(standard):
out.append(standard[index])
return out
return standard
def _block_delta(index: int, delta: dict[str, Any], extensions: _BlockExtensions) -> bytes:
event_extensions, delta_extensions = extensions.deltas.get(delta["type"], ({}, {}))
return _sse_event(
{
"type": "content_block_delta",
"index": index,
"delta": {**delta, **copy.deepcopy(delta_extensions)},
**copy.deepcopy(event_extensions),
}
)
def _render_known_response(
response: dict[str, Any], extensions: _FrameExtensions | None = None
) -> list[bytes]:
ext = extensions if extensions is not None else _FrameExtensions()
message = copy.deepcopy(response)
content = message.pop("content", [])
raw_usage = message.get("usage")
usage: dict[str, Any] = raw_usage if isinstance(raw_usage, dict) else {}
msg_start_message = copy.deepcopy(message)
msg_start_message.setdefault("type", "message")
msg_start_message.setdefault("role", "assistant")
msg_start_message.setdefault("content", [])
msg_start_message.setdefault("stop_reason", None)
msg_start_message["usage"] = usage
events = [
_sse_event(
{
"type": "message_start",
"message": msg_start_message,
**copy.deepcopy(ext.message_start),
}
)
]
if isinstance(content, list):
for index, block in enumerate(content):
if not isinstance(block, dict):
continue
block_ext = ext.blocks.get(index, _BlockExtensions())
block_type = block.get("type")
start_block = copy.deepcopy(block)
if block_type == "text":
start_block["text"] = ""
start_block.pop("citations", None)
elif block_type == "tool_use":
start_block["input"] = {}
elif block_type == "thinking":
start_block["thinking"] = ""
# Any other block, including ``server_tool_use``, is emitted
# complete in its start frame: its input is not streamed.
events.append(
_sse_event(
{
"type": "content_block_start",
"index": index,
"content_block": start_block,
**copy.deepcopy(block_ext.start),
}
)
)
if block_type == "text" and block.get("text"):
events.append(
_block_delta(index, {"type": "text_delta", "text": block["text"]}, block_ext)
)
for citation in block.get("citations", []) or []:
events.append(
_block_delta(
index, {"type": "citations_delta", "citation": citation}, block_ext
)
)
elif block_type == "tool_use" and "input" in block:
partial_json = json.dumps(block.get("input") or {}, ensure_ascii=False)
events.append(
_block_delta(
index, {"type": "input_json_delta", "partial_json": partial_json}, block_ext
)
)
elif block_type == "thinking":
if block.get("thinking"):
events.append(
_block_delta(
index,
{"type": "thinking_delta", "thinking": block["thinking"]},
block_ext,
)
)
if block.get("signature"):
events.append(
_block_delta(
index,
{"type": "signature_delta", "signature": block["signature"]},
block_ext,
)
)
events.append(
_sse_event(
{
"type": "content_block_stop",
"index": index,
**copy.deepcopy(block_ext.stop),
}
)
)
delta_usage = {"output_tokens": usage.get("output_tokens", 0)}
for key in ext.message_delta_usage_keys:
if key in usage:
delta_usage[key] = copy.deepcopy(usage[key])
delta: dict[str, Any] = {"type": "message_delta", "delta": {}, "usage": delta_usage}
for key in _MESSAGE_DELTA_FIELDS:
if key in response:
delta["delta"][key] = response[key]
delta["delta"].update(copy.deepcopy(ext.message_delta_delta))
delta.update(copy.deepcopy(ext.message_delta))
events.append(_sse_event(delta))
events.append(_sse_event({"type": "message_stop", **copy.deepcopy(ext.message_stop)}))
return events
def render_anthropic_sse_response(response: dict[str, Any]) -> list[bytes]:
"""Render an Anthropic response dictionary as canonical SSE frames.
Args:
response: Anthropic Messages API response dictionary.
Returns:
Canonically encoded Anthropic SSE frames.
"""
return _render_known_response(response)
+75 -25
View File
@@ -31,12 +31,14 @@ import httpx
from headroom.agent_savings import proxy_pipeline_kwargs
from headroom.ccr.context_tracker import looks_like_claude_code_compact_summary
from headroom.ccr.marker_resolution import resolve_markers_in_response
from headroom.copilot_auth import (
apply_copilot_api_auth,
build_copilot_upstream_url,
is_copilot_upstream_url,
)
from headroom.copilot_auth import apply_copilot_api_auth, is_copilot_upstream_url
from headroom.pipeline import PipelineStage, summarize_routing_markers
from headroom.proxy.anthropic_wire import (
build_anthropic_upstream_url,
is_safeguard_capable_request,
preserve_opaque_response_fields,
strip_safeguard_payload,
)
from headroom.proxy.auth_mode import (
classify_auth_mode,
classify_client,
@@ -1069,6 +1071,12 @@ class AnthropicHandlerMixin:
},
},
)
# This is a boolean capability classification only. The
# safeguards value itself is never copied into logs, metrics, or
# any Headroom-owned state.
client_anthropic_version = request.headers.get("anthropic-version")
client_anthropic_beta = request.headers.get("anthropic-beta")
safeguard_capable_request = is_safeguard_capable_request(body, client_anthropic_beta)
raw_model = body.get("model") or model_override or "unknown"
model = (
sanitize_anthropic_model_id(raw_model) if isinstance(raw_model, str) else raw_model
@@ -1636,23 +1644,33 @@ class AnthropicHandlerMixin:
log_beta_header_merge,
)
_client_beta_value = headers.get("anthropic-beta")
_client_beta_value = client_anthropic_beta
_client_beta_count = (
len([t for t in (_client_beta_value or "").split(",") if t.strip()])
if _client_beta_value
else 0
)
_sticky_beta_value = get_session_beta_tracker().record_and_get_sticky_betas(
provider="anthropic",
session_id=session_id,
client_value=_client_beta_value,
)
if safeguard_capable_request:
# Claude Code's classifier capability is negotiated by the
# client-owned header. Do not let prior turns' sticky beta
# state rewrite this turn.
_sticky_beta_value = _client_beta_value or ""
else:
_sticky_beta_value = get_session_beta_tracker().record_and_get_sticky_betas(
provider="anthropic",
session_id=session_id,
client_value=_client_beta_value,
)
_sticky_beta_count = (
len([t for t in _sticky_beta_value.split(",") if t.strip()])
if _sticky_beta_value
else 0
)
if _sticky_beta_value and _sticky_beta_value != (_client_beta_value or ""):
if (
not safeguard_capable_request
and _sticky_beta_value
and _sticky_beta_value != (_client_beta_value or "")
):
headers["anthropic-beta"] = _sticky_beta_value
elif not _sticky_beta_value and "anthropic-beta" in headers:
# Sticky value can only equal "" when both client and
@@ -2809,6 +2827,11 @@ class AnthropicHandlerMixin:
# need its own merge helper.
headers[key] = value
continue
if safeguard_capable_request:
# Memory may still inject its tools, but it
# must not negotiate a different capability on
# a classifier-bearing turn.
continue
existing_value = headers.get(key, "")
required_tokens = [t.strip() for t in value.split(",") if t.strip()]
_headroom_beta_added = True
@@ -3187,7 +3210,11 @@ class AnthropicHandlerMixin:
# client's own tokens — the same reduction the gateway contract
# applies before handing ``headers`` to the gateway.
_hook_headers = getattr(_req_ctx, "provider_headers", None)
if isinstance(_hook_headers, dict) and _hook_headers:
if (
isinstance(_hook_headers, dict)
and _hook_headers
and not safeguard_capable_request
):
from headroom.proxy.turn_hooks import merge_provider_headers
for _hh_key, _hh_value in merge_provider_headers(
@@ -3439,6 +3466,18 @@ class AnthropicHandlerMixin:
except (json.JSONDecodeError, ValueError, MemoryError, RecursionError):
body_mutation_tracker.mark_mutated("original_unparseable")
if safeguard_capable_request:
# Restore the exact client-owned capability headers after all
# pipeline, memory, and hook stages. Header names are
# case-insensitive; values are not normalized or merged.
for key in list(headers):
if key.lower() in {"anthropic-version", "anthropic-beta"}:
headers.pop(key, None)
if client_anthropic_version is not None:
headers["anthropic-version"] = client_anthropic_version
if client_anthropic_beta is not None:
headers["anthropic-beta"] = client_anthropic_beta
if (
(upstream_base_url or self.ANTHROPIC_API_URL != "https://api.anthropic.com")
and stream
@@ -3505,7 +3544,7 @@ class AnthropicHandlerMixin:
model=model,
messages=body["messages"],
tools=tools,
response=backend_response.body,
response=(None if safeguard_capable_request else backend_response.body),
metadata={
"path": pipeline_path,
"stream": False,
@@ -3518,7 +3557,7 @@ class AnthropicHandlerMixin:
request_id=request_id,
provider=pipeline_provider,
model=model,
response=backend_response.body,
response=(None if safeguard_capable_request else backend_response.body),
metadata={
"path": pipeline_path,
"stream": False,
@@ -3719,13 +3758,11 @@ class AnthropicHandlerMixin:
# every one of those turns to "anthropic" on the dashboard.
# For a non-Copilot base the builder only joins base + path, so the
# URL itself is unchanged.
url = (
build_copilot_upstream_url(upstream_base_url, request.url.path)
if upstream_base_url
else build_copilot_upstream_url(self.ANTHROPIC_API_URL, "/v1/messages")
url = build_anthropic_upstream_url(
upstream_base_url or self.ANTHROPIC_API_URL,
request.url.path,
request.url.query,
)
if upstream_base_url and request.url.query:
url = f"{url}?{request.url.query}"
try:
ccr_handler_config = getattr(self.ccr_response_handler, "config", None)
@@ -4028,7 +4065,7 @@ class AnthropicHandlerMixin:
model=model,
messages=body["messages"],
tools=tools,
response=response,
response=(None if safeguard_capable_request else response),
metadata={
"path": pipeline_path,
"stream": False,
@@ -4043,7 +4080,7 @@ class AnthropicHandlerMixin:
request_id=request_id,
provider=pipeline_provider,
model=model,
response=response,
response=(None if safeguard_capable_request else response),
metadata={
"path": pipeline_path,
"stream": False,
@@ -4067,11 +4104,21 @@ class AnthropicHandlerMixin:
if response.status_code >= 400:
try:
err_body = response.json()
if safeguard_capable_request:
err_body = strip_safeguard_payload(err_body)
err_msg = err_body.get("error", {}).get("message", "")
err_type = err_body.get("error", {}).get("type", "")
except Exception:
err_body = {"raw": response.text[:2000]}
err_msg = str(response.text[:500])
err_body = (
{"raw": "<redacted classifier-bearing upstream error>"}
if safeguard_capable_request
else {"raw": response.text[:2000]}
)
err_msg = (
"<redacted classifier-bearing upstream error>"
if safeguard_capable_request
else str(response.text[:500])
)
err_type = "parse_error"
logger.warning(
@@ -4342,6 +4389,9 @@ class AnthropicHandlerMixin:
api_call_fn,
provider="anthropic",
)
final_resp_json = preserve_opaque_response_fields(
resp_json, final_resp_json
)
# Update response content with final response
resp_json = final_resp_json
# Remove encoding headers since content is now uncompressed JSON
+24 -379
View File
@@ -409,393 +409,31 @@ class StreamingMixin:
cannot replay; anything else returns None so the caller can fail
loudly instead.
"""
if provider != "anthropic":
return None # Only implemented for Anthropic
if provider == "anthropic":
# Keep the complete wire envelope alongside the mutable message
# view. Callers that need to render a changed response own this
# envelope in their request-local flow; never attach it to this
# shared handler instance.
from headroom.proxy.anthropic_wire import AnthropicSSEEnvelope
# Event framing is CRLF in some intermediaries (and mixed after a
# retry through one). Normalize before the line split so a
# correctly framed stream is never read as zero events.
sse_data = sse_data.replace("\r\n", "\n").replace("\r", "\n")
response: dict[str, Any] = {"content": [], "usage": {}}
saw_message_start = False
saw_message_stop = False
saw_error = False
saw_unreplayable_delta = False
open_block_indices: set[int] = set()
# Track blocks by their `index` field so out-of-order events
# don't corrupt the reconstruction. The current block pointer
# remains for backward-compat with code that walks this dict
# sequentially, but the index map is the source of truth.
blocks_by_index: dict[int, dict[str, Any]] = {}
current_block: dict[str, Any] | None = None
# Track which block indices have already been appended to
# `response["content"]`. Dedup used to be `target not in
# response["content"]` — plain dict-equality. Two distinct blocks
# that happen to accumulate identical values (most commonly two
# separate empty `thinking` blocks, e.g. from a retried HTTP/2
# stream reset redelivering a truncated segment) either got
# wrongly collapsed into one, or — when their partial content
# happened to differ (same index, unequal dict) — both slipped
# through as duplicates. Indexing by `index` (falling back to
# object identity for the legacy no-index path) makes dedup exact
# regardless of what the accumulated content looks like: one
# entry per block index, first `content_block_stop` wins.
appended_block_keys: set[int] = set()
for line in sse_data.split("\n"):
if not line.startswith("data:"):
continue
data_str = line[5:].strip()
if not data_str or data_str == "[DONE]":
continue
try:
data = json.loads(data_str)
except json.JSONDecodeError:
continue
event_type = data.get("type", "")
if event_type == "message_start":
saw_message_start = True
msg = data.get("message", {})
response["id"] = msg.get("id")
response["type"] = msg.get("type", "message")
if "stop_sequence" in msg:
response["stop_sequence"] = msg["stop_sequence"]
response["model"] = msg.get("model")
response["role"] = msg.get("role", "assistant")
response["stop_reason"] = msg.get("stop_reason")
if "stop_details" in msg:
response["stop_details"] = msg["stop_details"]
if msg.get("usage"):
response["usage"].update(msg["usage"])
elif event_type == "content_block_start":
block = data.get("content_block", {})
block_index = data.get("index", len(response["content"]))
btype = block.get("type")
current_block = {
"type": btype,
"index": block_index,
}
if btype == "text":
current_block["text"] = block.get("text", "")
elif btype == "tool_use":
current_block["id"] = block.get("id")
current_block["name"] = block.get("name")
current_block["input"] = {}
elif btype == "thinking":
# Thinking block — accumulate text via
# `thinking_delta`; signature arrives via
# `signature_delta` (single value, not accumulated).
current_block["thinking_buffer"] = block.get("thinking", "")
if "signature" in block:
current_block["signature"] = block["signature"]
elif btype == "redacted_thinking":
# Per Anthropic spec §2.7: opaque encrypted reasoning
# block. The `data` field is preserved as-is and
# MUST be replayed unchanged on the next turn for
# signature validation to pass.
if "data" in block:
current_block["data"] = block["data"]
elif btype:
# Non-standard block (server_tool_use, web_search_tool_result,
# ...): copy through all of the block's fields so the
# reconstruction doesn't silently drop them to a bare
# {type, index}. Mirrors the sibling reconstructor
# `_reconstruct_anthropic_response`, which does `dict(block)`.
for _k, _v in block.items():
if _k != "type":
current_block[_k] = _v
blocks_by_index[block_index] = current_block
open_block_indices.add(block_index)
elif event_type == "content_block_delta":
# Resolve the target block by index (preferred) or fall
# back to current_block for legacy linear streams.
idx = data.get("index")
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
if target is not None:
delta = data.get("delta", {})
dtype = delta.get("type")
if dtype == "text_delta":
target["text"] = target.get("text", "") + delta.get("text", "")
elif dtype == "input_json_delta":
# Accumulate partial JSON for tool input.
partial = delta.get("partial_json", "")
target["_partial_json"] = target.get("_partial_json", "") + partial
elif dtype == "thinking_delta":
# Accumulate thinking text into the dedicated
# buffer so it never collides with `text` on
# text blocks (separate field per guide §2.7).
target["thinking_buffer"] = target.get("thinking_buffer", "") + delta.get(
"thinking", ""
)
elif dtype == "signature_delta":
# Single value, not accumulated. Last-write
# wins per Anthropic spec.
if "signature" in delta:
target["signature"] = delta["signature"]
elif dtype == "citations_delta":
# Append the citation object to the citations
# list so multi-citation blocks reconstruct
# correctly. Per guide §2.5: each delta carries
# one full citation object under `citation`.
citations = target.setdefault("citations", [])
citation = delta.get("citation")
if citation is not None:
citations.append(citation)
else:
# A delta this reconstructor has no rule for: the
# accumulated block is missing whatever it carried.
saw_unreplayable_delta = True
elif event_type == "content_block_stop":
idx = data.get("index")
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
if target is not None:
# Parse accumulated JSON into `input` for any block that
# streamed `input_json_delta` — tool_use AND server_tool_use
# (and future tool-ish blocks). Gating on the block type
# missed server_tool_use, leaving its `input` at the empty
# start-event value and leaking the `_partial_json` scratch
# key into replayed assistant history, which Anthropic then
# rejects with `server_tool_use.input: Input should be an
# object` (#2438). Always strip the scratch key.
if "_partial_json" in target:
raw = target.pop("_partial_json")
try:
target["input"] = json.loads(raw) if raw else {}
except json.JSONDecodeError:
target["input"] = {}
# Materialize the thinking buffer into the
# canonical `thinking` field expected by the
# Anthropic API.
if target.get("type") == "thinking" and "thinking_buffer" in target:
target["thinking"] = target.pop("thinking_buffer")
# Append the block exactly once, keyed by its block
# index (or object identity when no index was ever
# assigned). `current_block` may not match the
# indexed target if the stream interleaved multiple
# blocks; index-keyed map is authoritative.
block_key = idx if idx is not None else id(target)
if block_key not in appended_block_keys:
response["content"].append(target)
appended_block_keys.add(block_key)
if idx is not None:
open_block_indices.discard(idx)
current_block = None
elif event_type == "message_delta":
delta = data.get("delta", {})
if "stop_reason" in delta:
response["stop_reason"] = delta["stop_reason"]
if "stop_details" in delta:
response["stop_details"] = delta["stop_details"]
if "stop_sequence" in delta:
response["stop_sequence"] = delta["stop_sequence"]
if data.get("usage"):
response["usage"].update(data["usage"])
elif event_type == "message_stop":
saw_message_stop = True
elif event_type == "error":
# An in-band failure. Permissive callers keep salvaging
# what arrived before it; a strict caller must not dress
# the remains up as a successful turn.
saw_error = True
if require_complete:
if (
not saw_message_start
or not saw_message_stop
or saw_error
or saw_unreplayable_delta
or open_block_indices
):
envelope = AnthropicSSEEnvelope.parse(sse_data.encode("utf-8"))
if require_complete and not envelope.is_message_reconstructable():
return None
# ``index`` is a response-delta field. Anthropic rejects it on
# the next request ("content.0.text.index: Extra inputs are not
# permitted"), so it must not survive into a body the client
# will persist and echo back.
for block in response["content"]:
if isinstance(block, dict):
block.pop("index", None)
return response
return response if response.get("content") else None
return (
envelope.message
if envelope.message.get("content") or envelope.saw_message_start
else None
)
def _response_to_sse(self, response: dict[str, Any], provider: str) -> list[bytes]:
"""Convert a response dict back to SSE format.
"""Convert a response dictionary back to provider SSE frames."""
Args:
response: API response dict.
provider: Provider type for formatting.
Returns:
List of SSE event bytes.
"""
if provider != "anthropic":
return []
events: list[bytes] = []
from headroom.proxy.anthropic_wire import render_anthropic_sse_response
# message_start
msg_start = {
"type": "message_start",
"message": {
"id": response.get("id", "msg_generated"),
"type": "message",
"role": response.get("role", "assistant"),
"model": response.get("model", "unknown"),
"content": [],
"stop_reason": None,
"usage": response.get("usage", {}),
},
}
events.append(f"event: message_start\ndata: {json.dumps(msg_start)}\n\n".encode())
# Content blocks. `content` is provider/reconstruction-controlled, so a
# present-but-null value or a non-list would crash `enumerate`, and a
# non-dict element would crash `block.get(...)`. Guard both, matching the
# sibling `_record_ccr_feedback_from_response` below. This is reached from
# a call site (anthropic.py buffered CCR path) that only catches
# ValueError, so an unguarded TypeError/AttributeError would 500 the
# streamed request.
content = response.get("content")
for idx, block in enumerate(content if isinstance(content, list) else []):
if not isinstance(block, dict):
continue
# content_block_start
if block.get("type") == "text":
block_start = {
"type": "content_block_start",
"index": idx,
"content_block": {"type": "text", "text": ""},
}
elif block.get("type") == "tool_use":
block_start = {
"type": "content_block_start",
"index": idx,
"content_block": {
"type": "tool_use",
"id": block.get("id", f"toolu_{idx}"),
"name": block.get("name", ""),
"input": {},
},
}
elif block.get("type") == "thinking":
content_block = {
"type": "thinking",
"thinking": "",
}
if "signature" in block:
content_block["signature"] = block["signature"]
block_start = {
"type": "content_block_start",
"index": idx,
"content_block": content_block,
}
elif block.get("type") == "redacted_thinking":
block_start = {
"type": "content_block_start",
"index": idx,
"content_block": {
"type": "redacted_thinking",
"data": block.get("data", ""),
},
}
elif block.get("type") == "server_tool_use":
block_start = {
"type": "content_block_start",
"index": idx,
"content_block": block,
}
else:
block_start = {
"type": "content_block_start",
"index": idx,
"content_block": block,
}
events.append(
f"event: content_block_start\ndata: {json.dumps(block_start)}\n\n".encode()
)
# content_block_delta(s)
if block.get("type") == "text" and block.get("text"):
delta = {
"type": "content_block_delta",
"index": idx,
"delta": {"type": "text_delta", "text": block["text"]},
}
events.append(f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode())
for citation in block.get("citations", []) or []:
citation_delta = {
"type": "content_block_delta",
"index": idx,
"delta": {"type": "citations_delta", "citation": citation},
}
events.append(
f"event: content_block_delta\ndata: {json.dumps(citation_delta)}\n\n".encode()
)
elif block.get("type") == "tool_use" and block.get("input"):
delta = {
"type": "content_block_delta",
"index": idx,
"delta": {
"type": "input_json_delta",
"partial_json": json.dumps(block["input"]),
},
}
events.append(f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode())
elif block.get("type") == "thinking":
if block.get("thinking"):
delta = {
"type": "content_block_delta",
"index": idx,
"delta": {"type": "thinking_delta", "thinking": block["thinking"]},
}
events.append(
f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode()
)
if block.get("signature"):
delta = {
"type": "content_block_delta",
"index": idx,
"delta": {"type": "signature_delta", "signature": block["signature"]},
}
events.append(
f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode()
)
# content_block_stop
block_stop = {"type": "content_block_stop", "index": idx}
events.append(f"event: content_block_stop\ndata: {json.dumps(block_stop)}\n\n".encode())
# message_delta
msg_delta_payload: dict[str, Any] = {}
if "stop_reason" in response:
msg_delta_payload["stop_reason"] = response["stop_reason"]
if "stop_details" in response:
msg_delta_payload["stop_details"] = response["stop_details"]
usage = response.get("usage")
if not isinstance(usage, dict):
usage = {}
msg_delta = {
"type": "message_delta",
"delta": msg_delta_payload,
"usage": {"output_tokens": usage.get("output_tokens", 0)},
}
events.append(f"event: message_delta\ndata: {json.dumps(msg_delta)}\n\n".encode())
# message_stop
events.append(b'event: message_stop\ndata: {"type": "message_stop"}\n\n')
return events
return render_anthropic_sse_response(response)
def _record_ccr_feedback_from_response(
self, response: dict, provider: str, request_id: str
@@ -1248,6 +886,7 @@ class StreamingMixin:
# bytes once before entering the connection-retry loop. When a
# transform mutated the body we re-serialize canonically; otherwise
# we forward the original client bytes verbatim.
from headroom.proxy.anthropic_wire import is_safeguard_capable_request
from headroom.proxy.body_forwarding import select_outbound_body
from headroom.proxy.helpers import (
capture_codex_wire_debug,
@@ -1274,8 +913,14 @@ class StreamingMixin:
source=outbound_source,
dropped_mutation_reasons=outbound.dropped_mutation_reasons,
)
safeguard_capable = provider == "anthropic" and is_safeguard_capable_request(
body, headers.get("anthropic-beta")
)
_codex_wire_debug = (
codex_wire_debug_enabled() and provider == "openai" and "/responses" in url
codex_wire_debug_enabled()
and not safeguard_capable
and provider == "openai"
and "/responses" in url
)
if _codex_wire_debug:
capture_codex_wire_debug(
@@ -0,0 +1,18 @@
event: message_start
data: {"type":"message_start","message":{"id":"msg_nested_001","type":"message","role":"assistant","model":"claude-test-20260924","content":[],"stop_reason":null,"usage":{"input_tokens":12,"output_tokens":1}},"x_message_start":{"sentinel":"message_start"}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""},"x_block_start":{"sentinel":"block_start"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Checking.","x_text_delta":{"sentinel":"text_delta"}},"x_delta_event":{"sentinel":"delta_event"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0,"x_block_stop":{"sentinel":"block_stop"}}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null,"x_message_delta_delta":{"sentinel":"message_delta_delta"}},"usage":{"output_tokens":3,"cache_read_input_tokens":7,"x_usage":{"sentinel":"usage"}}}
event: message_stop
data: {"type":"message_stop","x_message_stop":{"sentinel":"message_stop"}}
@@ -0,0 +1,33 @@
{
"model": "claude-test-20260924",
"max_tokens": 128,
"stream": true,
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Run the proposed action."
}
]
}
],
"tools": [
{
"name": "run_action",
"description": "Run a sanitized test action.",
"input_schema": {
"type": "object",
"properties": {
"target": {"type": "string"}
},
"required": ["target"]
}
}
],
"safeguards": {
"mode": "auto",
"tool_use_ids": ["toolu_auto_001"]
}
}
@@ -0,0 +1,20 @@
{
"id": "msg_auto_001",
"type": "message",
"role": "assistant",
"model": "claude-test-20260924",
"content": [
{
"type": "tool_use",
"id": "toolu_auto_001",
"name": "run_action",
"input": {"target": "sanitized-target"}
}
],
"stop_reason": "tool_use",
"usage": {"input_tokens": 12, "output_tokens": 8},
"safeguard_results": {
"decision": "allow",
"tool_use_id": "toolu_auto_001"
}
}
@@ -0,0 +1,20 @@
event: message_start
data: {"type":"message_start","message":{"id":"msg_auto_001","type":"message","role":"assistant","model":"claude-test-20260924","content":[],"usage":{"input_tokens":12},"safeguard_results":{"decision":"allow","tool_use_id":"toolu_auto_001"}}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"run_action","input":{}}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"target\":\"sanitized-target\"}"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: safeguard_results
data: {"type":"safeguard_results","result":{"decision":"allow","tool_use_id":"toolu_auto_001"}}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"},"usage":{"output_tokens":8}}
event: message_stop
data: {"type":"message_stop"}
@@ -0,0 +1,484 @@
from __future__ import annotations
import json
from pathlib import Path
import httpx
import pytest
from fastapi.testclient import TestClient
from headroom.proxy.anthropic_wire import (
AnthropicSSEEnvelope,
build_anthropic_upstream_url,
has_dangerous_tool_use_beta,
is_safeguard_capable_request,
strip_safeguard_payload,
)
from headroom.proxy.handlers.streaming import StreamingMixin
from headroom.proxy.server import ProxyConfig, create_app
FIXTURES = Path(__file__).parent / "fixtures" / "anthropic"
def _fixture(name: str) -> bytes:
return (FIXTURES / name).read_bytes()
def test_native_url_preserves_path_and_raw_query() -> None:
assert (
build_anthropic_upstream_url(
"https://api.anthropic.com", "/v1/messages", "beta=true&beta=false"
)
== "https://api.anthropic.com/v1/messages?beta=true&beta=false"
)
def test_classifier_request_detection_does_not_need_safeguard_schema() -> None:
request = json.loads(_fixture("claude_code_auto_mode_request.json"))
assert is_safeguard_capable_request(request, "prompt-caching-2024-07-31")
assert has_dangerous_tool_use_beta("prompt-caching-2024-07-31,dangerous-tool-use-2026-09")
assert not is_safeguard_capable_request({}, "prompt-caching-2024-07-31")
def test_sse_round_trip_preserves_opaque_result_order_and_ids() -> None:
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_stream.sse"))
assert envelope.message["safeguard_results"]["decision"] == "allow"
assert envelope.message["content"][0]["id"] == "toolu_auto_001"
rendered = b"".join(envelope.render(envelope.message))
safeguard_offset = rendered.index(b"event: safeguard_results")
delta_offset = rendered.index(b"event: message_delta")
stop_offset = rendered.index(b"event: message_stop")
assert safeguard_offset < delta_offset < stop_offset
assert b'"id": "toolu_auto_001"' in rendered
def test_sse_round_trip_keeps_results_across_a_ccr_continuation() -> None:
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_stream.sse"))
continuation = {
"id": "msg_continuation_001",
"type": "message",
"role": "assistant",
"model": "claude-test-20260924",
"content": [{"type": "text", "text": "retrieval complete"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 20, "output_tokens": 4},
}
rendered = b"".join(envelope.render(continuation))
assert b"safeguard_results" in rendered
assert b"toolu_auto_001" in rendered
assert b"msg_continuation_001" in rendered
def _rendered_events(chunks: list[bytes]) -> list[dict]:
return [
json.loads(line[len("data: ") :])
for line in b"".join(chunks).decode("utf-8").splitlines()
if line.startswith("data: ")
]
def test_nested_extensions_on_known_frames_survive_round_trip() -> None:
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_nested_extensions.sse"))
events = {event["type"]: event for event in _rendered_events(envelope.render())}
assert envelope.is_message_reconstructable()
assert events["message_start"]["x_message_start"] == {"sentinel": "message_start"}
assert events["content_block_start"]["x_block_start"] == {"sentinel": "block_start"}
assert events["content_block_delta"]["x_delta_event"] == {"sentinel": "delta_event"}
assert events["content_block_delta"]["delta"] == {
"type": "text_delta",
"text": "Checking.",
"x_text_delta": {"sentinel": "text_delta"},
}
assert events["content_block_stop"]["x_block_stop"] == {"sentinel": "block_stop"}
assert events["message_delta"]["delta"]["x_message_delta_delta"] == {
"sentinel": "message_delta_delta"
}
assert events["message_delta"]["usage"]["cache_read_input_tokens"] == 7
assert events["message_delta"]["usage"]["x_usage"] == {"sentinel": "usage"}
assert events["message_stop"]["x_message_stop"] == {"sentinel": "message_stop"}
def test_block_extensions_do_not_attach_to_replacement_blocks() -> None:
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_nested_extensions.sse"))
continuation = {
"id": "msg_continuation_002",
"type": "message",
"role": "assistant",
"model": "claude-test-20260924",
"content": [{"type": "text", "text": "retrieval complete"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 20, "output_tokens": 4},
}
rendered = b"".join(envelope.render(continuation))
events = {event["type"]: event for event in _rendered_events([rendered])}
# Message-level members belong to the protocol and survive the swap.
assert events["message_start"]["x_message_start"] == {"sentinel": "message_start"}
assert "x_message_delta_delta" in events["message_delta"]["delta"]
assert events["message_stop"]["x_message_stop"] == {"sentinel": "message_stop"}
# Members of the replaced block's frames do not describe the new block.
for sentinel in (b"x_block_start", b"x_delta_event", b"x_text_delta", b"x_block_stop"):
assert sentinel not in rendered
assert b"retrieval complete" in rendered
def test_unknown_content_delta_is_complete_and_replayed_verbatim() -> None:
raw = b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_future","type":"message","role":"assistant","content":[]}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"future_delta","value":"opaque"}}
event: message_stop
data: {"type":"message_stop"}
"""
envelope = AnthropicSSEEnvelope.parse(raw)
assert envelope.is_complete()
assert not envelope.is_message_reconstructable()
assert b'"type":"future_delta"' in b"".join(envelope.render())
def test_malformed_utf8_delta_is_replayed_without_replacement() -> None:
raw = b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_bad_utf8","type":"message","role":"assistant","content":[]}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\xff"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: message_stop
data: {"type":"message_stop"}
"""
envelope = AnthropicSSEEnvelope.parse(raw)
rendered = b"".join(envelope.render())
assert envelope.is_complete()
assert not envelope.is_message_reconstructable()
assert b"\xff" in rendered
assert b"\xef\xbf\xbd" not in rendered
_MALFORMED_KNOWN_FRAMES = {
"invalid-json-message-delta": b"event: message_delta\ndata: {not-json\n\n",
"invalid-utf8-block-start": (
b"event: content_block_start\n"
b'data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":"\xff"}}\n\n'
),
"non-object-content-block": (
b"event: content_block_start\n"
b'data: {"type":"content_block_start","index":1,"content_block":"future"}\n\n'
),
"delta-for-unopened-block": (
b"event: content_block_delta\n"
b'data: {"type":"content_block_delta","index":9,"delta":{"type":"text_delta","text":"x"}}\n\n'
),
"stop-for-unopened-block": (
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":9}\n\n'
),
# ``index`` and ``delta.type`` are provider JSON and may be any JSON value;
# a non-scalar one must not reach a dictionary lookup.
"object-index-block-start": (
b"event: content_block_start\n"
b'data: {"type":"content_block_start","index":{"i":1},"content_block":{"type":"text","text":""}}\n\n'
),
"object-index-delta": (
b"event: content_block_delta\n"
b'data: {"type":"content_block_delta","index":{"i":0},"delta":{"type":"text_delta","text":"x"}}\n\n'
),
"array-index-delta": (
b"event: content_block_delta\n"
b'data: {"type":"content_block_delta","index":[0],"delta":{"type":"text_delta","text":"x"}}\n\n'
),
"array-delta-type": (
b"event: content_block_delta\n"
b'data: {"type":"content_block_delta","index":0,"delta":{"type":["text_delta"],"text":"x"}}\n\n'
),
"non-string-delta-text": (
b"event: content_block_delta\n"
b'data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":{"t":"x"}}}\n\n'
),
"object-index-stop": (
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":{"i":0}}\n\n'
),
"array-index-stop": (
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":[0]}\n\n'
),
"array-usage-message-delta": (
b'event: message_delta\ndata: {"type":"message_delta","delta":{},"usage":[1]}\n\n'
),
}
@pytest.mark.parametrize("bad_frame", _MALFORMED_KNOWN_FRAMES.values(), ids=_MALFORMED_KNOWN_FRAMES)
def test_malformed_known_frame_is_replayed_verbatim(bad_frame: bytes) -> None:
raw = (
b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_bad_known","type":"message","role":"assistant","content":[]}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hi"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
"""
+ bad_frame
+ b"""event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":1}}
event: message_stop
data: {"type":"message_stop"}
"""
)
envelope = AnthropicSSEEnvelope.parse(raw)
rendered = b"".join(envelope.render())
assert envelope.is_complete()
assert not envelope.is_message_reconstructable()
assert rendered.count(bad_frame) == 1
assert rendered.index(b"event: content_block_stop") < rendered.index(bad_frame)
assert rendered.index(bad_frame) < rendered.rindex(b"event: message_delta")
@pytest.mark.parametrize(
"bad_frames",
[
pytest.param(
b"event: content_block_delta\n"
b'data: {"type":"content_block_delta","index":{"i":0},"delta":{"type":"text_delta","text":"x"}}\n\n'
b"event: content_block_delta\n"
b'data: {"type":"content_block_delta","index":0,"delta":{"type":["text_delta"],"text":"x"}}\n\n',
id="delta",
),
pytest.param(
b"event: content_block_stop\n"
b'data: {"type":"content_block_stop","index":{"i":0}}\n\n'
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":[0]}\n\n',
id="stop",
),
],
)
def test_invalid_discriminators_on_an_open_block_do_not_break_the_stream(
bad_frames: bytes,
) -> None:
raw = (
b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_open","type":"message","role":"assistant","content":[]}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hel"}}
"""
+ bad_frames
+ b"""event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"lo"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: message_stop
data: {"type":"message_stop"}
"""
)
envelope = AnthropicSSEEnvelope.parse(raw)
rendered = b"".join(envelope.render())
# The valid frames around the invalid ones still reconstruct the block.
assert envelope.message["content"] == [{"type": "text", "text": "hello"}]
assert envelope.is_complete()
assert not envelope.is_message_reconstructable()
assert rendered.count(bad_frames) == 1
_DELTA_PAYLOAD_CASES = {
"text_delta": ("text", {"type": "text", "text": ""}),
"thinking_delta": ("thinking", {"type": "thinking", "thinking": ""}),
"input_json_delta": (
"partial_json",
{"type": "tool_use", "id": "toolu_1", "name": "run", "input": {}},
),
"signature_delta": ("signature", {"type": "thinking", "thinking": ""}),
"citations_delta": ("citation", {"type": "text", "text": ""}),
}
_MISSING = object()
def _single_delta_stream(block: dict, delta: dict) -> tuple[bytes, bytes]:
def event(payload: dict) -> bytes:
return f"event: {payload['type']}\ndata: {json.dumps(payload)}\n\n".encode()
frame = event({"type": "content_block_delta", "index": 0, "delta": delta})
raw = (
event(
{
"type": "message_start",
"message": {"id": "msg_d", "type": "message", "role": "assistant", "content": []},
}
)
+ event({"type": "content_block_start", "index": 0, "content_block": block})
+ frame
+ event({"type": "content_block_stop", "index": 0})
+ event({"type": "message_stop"})
)
return raw, frame
@pytest.mark.parametrize(
"value", [0, False, None, _MISSING], ids=["zero", "false", "null", "missing"]
)
@pytest.mark.parametrize("delta_type", _DELTA_PAYLOAD_CASES)
def test_falsy_non_string_delta_payload_is_replayed_not_consumed(
delta_type: str, value: object
) -> None:
member, block = _DELTA_PAYLOAD_CASES[delta_type]
delta = {"type": delta_type} if value is _MISSING else {"type": delta_type, member: value}
raw, frame = _single_delta_stream(block, delta)
envelope = AnthropicSSEEnvelope.parse(raw)
assert envelope.is_complete()
assert not envelope.is_message_reconstructable()
assert b"".join(envelope.render()).count(frame) == 1
@pytest.mark.parametrize("delta_type", ["text_delta", "thinking_delta", "input_json_delta"])
def test_empty_string_delta_payload_is_still_reconstructed(delta_type: str) -> None:
member, block = _DELTA_PAYLOAD_CASES[delta_type]
raw, frame = _single_delta_stream(block, {"type": delta_type, member: ""})
envelope = AnthropicSSEEnvelope.parse(raw)
assert envelope.is_message_reconstructable()
assert frame not in b"".join(envelope.render())
def test_non_object_message_usage_is_replayed_not_merged() -> None:
bad_start = (
b"event: message_start\n"
b'data: {"type":"message_start","message":{"id":"msg_usage","type":"message",'
b'"role":"assistant","content":[],"usage":[12]}}\n\n'
)
raw = (
bad_start
+ b"""event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":1}}
event: message_stop
data: {"type":"message_stop"}
"""
)
envelope = AnthropicSSEEnvelope.parse(raw)
rendered = b"".join(envelope.render())
assert not envelope.is_message_reconstructable()
assert rendered.count(bad_start) == 1
assert envelope.message["usage"] == {"output_tokens": 1}
def test_non_stream_response_keeps_unknown_top_level_fields() -> None:
response = json.loads(_fixture("claude_code_auto_mode_response.json"))
rendered = b"".join(StreamingMixin()._response_to_sse(response, "anthropic"))
assert b"safeguard_results" in rendered
assert b"toolu_auto_001" in rendered
def test_observation_copy_removes_classifier_payloads() -> None:
value = {
"safeguards": {"sentinel": "do-not-log"},
"nested": [{"safeguard_results": {"sentinel": "do-not-log"}}],
"safe": "kept",
}
observed = strip_safeguard_payload(value)
assert observed == {"nested": [{}], "safe": "kept"}
assert value["safeguards"]["sentinel"] == "do-not-log"
class _AutoModeFixtureStream(httpx.AsyncByteStream):
async def __aiter__(self):
yield _fixture("claude_code_auto_mode_stream.sse")
class _CapturingAnthropicTransport(httpx.AsyncBaseTransport):
def __init__(self) -> None:
self.url: str | None = None
self.headers: dict[str, str] | None = None
self.body: bytes | None = None
async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
self.url = str(request.url)
self.headers = dict(request.headers.items())
self.body = b"".join([chunk async for chunk in request.stream])
return httpx.Response(
200,
headers={"content-type": "text/event-stream"},
stream=_AutoModeFixtureStream(),
)
def test_native_handler_preserves_classifier_request_envelope() -> None:
config = ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
)
app = create_app(config)
proxy = app.state.proxy
transport = _CapturingAnthropicTransport()
proxy.http_client = httpx.AsyncClient(transport=transport)
request = json.loads(_fixture("claude_code_auto_mode_request.json"))
request["model"] = "claude-sonnet-4-6"
with TestClient(app).stream(
"POST",
"/v1/messages?beta=true",
headers={
"x-api-key": "test-key",
"anthropic-version": "2023-06-01",
"anthropic-beta": "claude-code-20250219,dangerous-tool-use-2026-09-03",
"content-type": "application/json",
},
content=json.dumps(request).encode(),
) as response:
assert response.status_code == 200
assert b"safeguard_results" in b"".join(response.iter_bytes())
assert transport.url == "https://api.anthropic.com/v1/messages?beta=true"
assert transport.headers is not None
assert (
transport.headers["anthropic-beta"] == "claude-code-20250219,dangerous-tool-use-2026-09-03"
)
assert transport.body is not None
assert json.loads(transport.body)["safeguards"] == request["safeguards"]
+111
View File
@@ -433,6 +433,117 @@ async def test_streaming_handler_process_stream_pass_through_and_ccr(
assert streamed == [b"event: message_start\n", b"event: message_stop\n"]
@pytest.mark.asyncio
async def test_streaming_ccr_keeps_classifier_frames_through_continuation(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response_handler = CCRResponseHandler()
handler = StreamingCCRHandler(response_handler, provider="anthropic")
async def fake_handle_response(response, messages, tools, api_call_fn, provider): # noqa: ANN001
return {
"id": "msg_continuation",
"type": "message",
"role": "assistant",
"model": "claude-test",
"content": [{"type": "text", "text": "retrieval complete"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 20, "output_tokens": 4},
}
monkeypatch.setattr(response_handler, "handle_response", fake_handle_response)
initial = b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_initial","type":"message","role":"assistant","model":"claude-test","content":[],"safeguard_results":{"decision":"allow","tool_use_id":"toolu_auto_001"}}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"headroom_retrieve","input":{}}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\\"hash\\":\\"abc\\"}"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: safeguard_results
data: {"type":"safeguard_results","result":{"decision":"allow","tool_use_id":"toolu_auto_001"}}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"}}
event: message_stop
data: {"type":"message_stop"}
"""
streamed = [
chunk
async for chunk in handler.process_stream(
_async_iter([initial]), [], None, lambda m, t: None
)
]
rendered = b"".join(streamed)
assert b"safeguard_results" in rendered
assert b"toolu_auto_001" in rendered
assert b"msg_continuation" in rendered
@pytest.mark.asyncio
async def test_streaming_ccr_replays_malformed_known_frame_through_continuation(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response_handler = CCRResponseHandler()
handler = StreamingCCRHandler(response_handler, provider="anthropic")
async def fake_handle_response(response, messages, tools, api_call_fn, provider): # noqa: ANN001
return {
"id": "msg_continuation",
"type": "message",
"role": "assistant",
"model": "claude-test",
"content": [{"type": "text", "text": "retrieval complete"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 20, "output_tokens": 4},
}
monkeypatch.setattr(response_handler, "handle_response", fake_handle_response)
malformed = b"event: message_delta\ndata: {not-json\n\n"
initial = (
b"""event: message_start
data: {"type":"message_start","message":{"id":"msg_initial","type":"message","role":"assistant","model":"claude-test","content":[]}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"headroom_retrieve","input":{}}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\\"hash\\":\\"abc\\"}"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
"""
+ malformed
+ b"""event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"}}
event: message_stop
data: {"type":"message_stop"}
"""
)
streamed = [
chunk
async for chunk in handler.process_stream(
_async_iter([initial]), [], None, lambda m, t: None
)
]
rendered = b"".join(streamed)
assert b"msg_continuation" in rendered
assert rendered.count(malformed) == 1
@pytest.mark.asyncio
async def test_streaming_handler_falls_back_to_buffer_on_processing_error(
monkeypatch: pytest.MonkeyPatch,