mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): preserve Claude Code auto-mode protocol (#3784)
## Description
Fix Claude Code server-side auto-mode compatibility when requests pass
through
Headroom’s native Anthropic Messages endpoint.
Previously, auto-mode capability headers, safeguard payloads, query
parameters,
unknown response fields, or opaque SSE frames could be rewritten or lost
while
proxying and handling CCR continuations. This could trigger Claude
Code’s
client-side classifier fallback and its associated warning.
This change adds a request-local Anthropic wire envelope, preserves
client-owned
capability data and opaque response attachments, prevents classifier
payloads
from entering Headroom-owned observations, and documents that
schema-translating
Bedrock, Vertex, and LiteLLM routes do not currently support the
server-side
auto-mode protocol.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Preserve native Anthropic request paths and raw query parameters.
- Preserve client-owned `anthropic-version`, `anthropic-beta`, and
safeguard data.
- Avoid sticky-beta, memory, or turn-hook rewrites on classifier-bearing
turns.
- Preserve unknown Anthropic response fields across CCR continuations.
- Preserve opaque SSE frames through request-local parse/render
envelopes.
- Reject unknown or malformed deltas during strict buffered
reconstruction.
- Redact safeguard payloads from diagnostics and pipeline observations.
- Consolidate Anthropic SSE parsing and rendering into the wire module.
- Document auto-mode limitations for Bedrock, Vertex, and LiteLLM
translation paths.
- Add request, response, streaming, CCR, malformed-byte, and passthrough
regression tests.
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest -p no:cacheprovider \
tests/test_anthropic_auto_mode_passthrough.py \
tests/test_anthropic_buffered_sse.py \
tests/test_ccr_response_handler_extra.py \
tests/test_ccr_buffered_stream_signed_thinking.py \
tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py \
tests/test_buffered_ccr_salvage.py \
tests/test_buffered_ccr_accept_header.py \
tests/test_proxy/test_anthropic_copilot_upstream_auth.py \
tests/test_proxy_streaming_request_logger.py \
tests/test_proxy_streaming_resilience.py -q
125 passed, 1 warning in 13.70s
$ ruff check --no-cache .
All checks passed!
$ ruff format --check --no-cache .
1559 files already formatted
$ python -m mypy \
headroom/proxy/anthropic_wire.py \
headroom/ccr/response_handler.py \
headroom/proxy/handlers/anthropic.py \
headroom/proxy/handlers/streaming.py
Success: no issues found in 4 source files
$ npx --no-install commitlint \
--from origin/main --to HEAD \
--config .commitlintrc.json
Exited successfully.
```
## Real Behavior Proof
- Environment: Pending. Record OS, Python version, Claude Code version,
provider/model, and Headroom revision.
- Exact command / steps: Pending. Start Headroom with `headroom proxy
--port 8787`, then run Claude Code with
`ANTHROPIC_BASE_URL=http://127.0.0.1:8787` and exercise server-side auto
mode.
- Observed result: Pending. Attach redacted output or logs showing the
request completed through Headroom and whether Claude Code used its
client-side fallback.
- Not tested: Live Claude Code auto mode against Anthropic; Bedrock,
Vertex, and LiteLLM translation routes.
## Runtime Rollout Safety
- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Native Anthropic safeguard-capable
requests.
- Kill switch / disable path: Set `CLAUDE_CODE_AUTO_MODE_SERVER=0` to
use Claude Code’s client-side fallback.
- Unsafe override required: No.
- Qualification impact: Live qualification is pending. Translation
routes remain documented as unsupported for server-side auto mode.
- Rollback path: Revert this PR or deploy the previous Headroom
revision.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code where needed
- [x] I have made corresponding documentation changes
- [x] No new warnings were observed in focused tests
- [x] I have added tests for the fix
- [ ] Full existing and new unit test suite passes locally
- [x] I did **not** edit `CHANGELOG.md`
## Screenshots (if applicable)
N/A. This change affects proxy wire behavior.
This commit is contained in:
@@ -112,6 +112,12 @@ Or via `~/.claude/settings.json`:
|
||||
Claude Code now talks plain Anthropic `/v1/messages` to Headroom; Headroom compresses
|
||||
and forwards to Bedrock via LiteLLM, then translates the answer back.
|
||||
|
||||
This translating route is not currently compatible with Claude Code's server-side
|
||||
auto-mode classifier protocol. Ordinary model traffic remains fail-open, but
|
||||
classifier-bearing turns may use Claude Code's billable client-side fallback.
|
||||
Use native Anthropic transport for server-side auto mode, or set
|
||||
`CLAUDE_CODE_AUTO_MODE_SERVER=0` when the fallback is intentional.
|
||||
|
||||
## Application inference profiles (account-specific ARNs)
|
||||
|
||||
If your IAM policy only permits **application inference profiles** (account-specific
|
||||
|
||||
@@ -114,5 +114,9 @@ and has the smallest moving parts. Use this alternative only if you can't set
|
||||
- Pick a Claude model that is enabled in your Vertex project/region
|
||||
(e.g. `claude-sonnet-4-6`, `claude-haiku-4-5`).
|
||||
- Streaming, tool use, and prompt caching all work through Headroom.
|
||||
- Claude Code's server-side auto-mode classifier is supported only by the
|
||||
native Anthropic-compatible wire path. The Vertex translation path remains
|
||||
fail-open but may use Claude Code's billable client-side classifier fallback
|
||||
until a provider-native safeguard fixture is verified.
|
||||
- Want to point at a private Vertex gateway instead of Google's host? Start the
|
||||
proxy with `--vertex-api-url https://your-gateway` and Headroom will forward there.
|
||||
|
||||
@@ -43,6 +43,14 @@ The callback compresses messages in LiteLLM's `async_pre_call_hook` before they
|
||||
|
||||
This works with every provider LiteLLM supports: OpenAI, Anthropic, Bedrock, Azure, Vertex AI, Cohere, Groq, Mistral, Together, Ollama, and more.
|
||||
|
||||
<Callout type="warning" title="Claude Code auto mode">
|
||||
LiteLLM and other schema-translating routes are not claimed to preserve Claude
|
||||
Code's server-side auto-mode classifier protocol. Requests continue to fail
|
||||
open as ordinary model traffic, but Claude Code may use its billable
|
||||
client-side classifier fallback. Native Anthropic transport is the supported
|
||||
path until an adapter has an end-to-end safeguard fixture.
|
||||
</Callout>
|
||||
|
||||
## With LiteLLM Proxy
|
||||
|
||||
If you run LiteLLM as a proxy server, use the ASGI middleware:
|
||||
|
||||
@@ -19,6 +19,8 @@ from collections.abc import Awaitable, Callable
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from headroom.proxy.anthropic_wire import AnthropicSSEEnvelope
|
||||
|
||||
from ..cache.compression_store import format_retrieval_miss_detail, get_compression_store
|
||||
from .tool_calls import (
|
||||
CCRToolCall,
|
||||
@@ -725,8 +727,19 @@ class StreamingCCRHandler:
|
||||
|
||||
# Parse the complete response
|
||||
try:
|
||||
# For SSE streams, we need to parse the accumulated data
|
||||
complete_data = self._parse_sse_stream(self.buffer.get_accumulated())
|
||||
# The envelope is request-local. It carries opaque Anthropic
|
||||
# frames over a CCR continuation without coupling concurrent
|
||||
# streams through this long-lived handler instance.
|
||||
stream_bytes = self.buffer.get_accumulated()
|
||||
complete_data = self._parse_sse_stream(stream_bytes)
|
||||
envelope = None
|
||||
if self.provider == "anthropic":
|
||||
candidate = self._parse_anthropic_sse_envelope(stream_bytes)
|
||||
# Keep the long-standing parser seam usable by callers and
|
||||
# tests that replace it. Only attach the wire envelope when
|
||||
# that parser returned the corresponding native message.
|
||||
if complete_data == candidate.message:
|
||||
envelope = candidate
|
||||
|
||||
# Handle CCR
|
||||
final_response = await self.response_handler.handle_response(
|
||||
@@ -739,7 +752,11 @@ class StreamingCCRHandler:
|
||||
|
||||
# Re-stream the final response
|
||||
# Convert back to SSE format
|
||||
async for chunk in self._response_to_sse(final_response):
|
||||
if envelope is None:
|
||||
response_stream = self._response_to_sse(final_response)
|
||||
else:
|
||||
response_stream = self._response_to_sse(final_response, envelope=envelope)
|
||||
async for chunk in response_stream:
|
||||
yield chunk
|
||||
|
||||
except Exception as e:
|
||||
@@ -759,6 +776,9 @@ class StreamingCCRHandler:
|
||||
event is an upstream protocol bug — surfaced loudly, not
|
||||
silently corrupted.
|
||||
"""
|
||||
if self.provider == "anthropic":
|
||||
return self._parse_anthropic_sse_envelope(data).message
|
||||
|
||||
from headroom.proxy.helpers import parse_sse_events_from_byte_buffer
|
||||
|
||||
# Accumulate all event data via the canonical bytes-buffer
|
||||
@@ -785,132 +805,20 @@ class StreamingCCRHandler:
|
||||
len(buf),
|
||||
)
|
||||
|
||||
# Reconstruct response from events
|
||||
# This is provider-specific
|
||||
if self.provider == "anthropic":
|
||||
return self._reconstruct_anthropic_response(events)
|
||||
else:
|
||||
return self._reconstruct_openai_response(events)
|
||||
return self._reconstruct_openai_response(events)
|
||||
|
||||
def _reconstruct_anthropic_response(
|
||||
self,
|
||||
events: list[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
"""Reconstruct Anthropic response from stream events."""
|
||||
response: dict[str, Any] = {
|
||||
"content": [],
|
||||
"stop_reason": None,
|
||||
"usage": {},
|
||||
}
|
||||
return AnthropicSSEEnvelope.from_events(events).message
|
||||
|
||||
blocks_by_index: dict[int, dict[str, Any]] = {}
|
||||
current_block: dict[str, Any] | None = None
|
||||
@staticmethod
|
||||
def _parse_anthropic_sse_envelope(data: bytes) -> AnthropicSSEEnvelope:
|
||||
"""Parse a complete Anthropic stream into its request-local envelope."""
|
||||
|
||||
for event in events:
|
||||
event_type = event.get("type", "")
|
||||
|
||||
if event_type == "content_block_start":
|
||||
block = event.get("content_block", {})
|
||||
block_index = event.get("index", len(blocks_by_index))
|
||||
btype = block.get("type")
|
||||
current_block = {"type": btype}
|
||||
if btype == "text":
|
||||
current_block["text"] = block.get("text", "")
|
||||
elif btype == "tool_use":
|
||||
current_block.update(
|
||||
{
|
||||
"id": block.get("id", ""),
|
||||
"name": block.get("name", ""),
|
||||
"input": {},
|
||||
}
|
||||
)
|
||||
elif btype == "thinking":
|
||||
current_block["thinking_buffer"] = block.get("thinking", "")
|
||||
if "signature" in block:
|
||||
current_block["signature"] = block["signature"]
|
||||
elif btype == "redacted_thinking":
|
||||
if "data" in block:
|
||||
current_block["data"] = block["data"]
|
||||
elif btype:
|
||||
current_block = dict(block)
|
||||
blocks_by_index[block_index] = current_block
|
||||
|
||||
elif event_type == "content_block_delta":
|
||||
idx = event.get("index")
|
||||
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
|
||||
if target is None:
|
||||
continue
|
||||
delta = event.get("delta", {})
|
||||
dtype = delta.get("type")
|
||||
if dtype == "text_delta":
|
||||
target["text"] = target.get("text", "") + delta.get("text", "")
|
||||
elif dtype == "input_json_delta":
|
||||
# Accumulate for any block streaming input (tool_use AND
|
||||
# server_tool_use); the stop handler parses it into `input`
|
||||
# (#2438).
|
||||
partial = delta.get("partial_json", "")
|
||||
target["_partial_json"] = target.get("_partial_json", "") + partial
|
||||
elif dtype == "thinking_delta":
|
||||
target["thinking_buffer"] = target.get("thinking_buffer", "") + delta.get(
|
||||
"thinking", ""
|
||||
)
|
||||
elif dtype == "signature_delta":
|
||||
if "signature" in delta:
|
||||
target["signature"] = delta["signature"]
|
||||
elif dtype == "citations_delta":
|
||||
citation = delta.get("citation")
|
||||
if citation is not None:
|
||||
target.setdefault("citations", []).append(citation)
|
||||
|
||||
elif event_type == "content_block_stop":
|
||||
idx = event.get("index")
|
||||
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
|
||||
if target is not None:
|
||||
# Parse streamed `_partial_json` into `input` for any block
|
||||
# that carried input_json_delta — tool_use AND
|
||||
# server_tool_use — not just tool_use. The narrow type gate
|
||||
# left server_tool_use.input malformed and leaked the scratch
|
||||
# key into replayed history (#2438). Always strip the key.
|
||||
if "_partial_json" in target:
|
||||
partial = target.pop("_partial_json")
|
||||
try:
|
||||
target["input"] = json.loads(partial) if partial else {}
|
||||
except json.JSONDecodeError:
|
||||
target["input"] = {}
|
||||
if target.get("type") == "thinking" and "thinking_buffer" in target:
|
||||
target["thinking"] = target.pop("thinking_buffer")
|
||||
if target not in response["content"]:
|
||||
response["content"].append(target)
|
||||
current_block = None
|
||||
|
||||
elif event_type == "message_start":
|
||||
msg = event.get("message", {})
|
||||
if "id" in msg:
|
||||
response["id"] = msg["id"]
|
||||
if "model" in msg:
|
||||
response["model"] = msg["model"]
|
||||
if "role" in msg:
|
||||
response["role"] = msg["role"]
|
||||
if "stop_reason" in msg:
|
||||
response["stop_reason"] = msg["stop_reason"]
|
||||
if "stop_details" in msg:
|
||||
response["stop_details"] = msg["stop_details"]
|
||||
if msg.get("usage"):
|
||||
response["usage"].update(msg["usage"])
|
||||
|
||||
elif event_type == "message_delta":
|
||||
delta = event.get("delta", {})
|
||||
if "stop_reason" in delta:
|
||||
response["stop_reason"] = delta["stop_reason"]
|
||||
if "stop_details" in delta:
|
||||
response["stop_details"] = delta["stop_details"]
|
||||
if event.get("usage"):
|
||||
response["usage"].update(event["usage"])
|
||||
|
||||
elif event_type == "message_stop":
|
||||
pass
|
||||
|
||||
return response
|
||||
return AnthropicSSEEnvelope.parse(data)
|
||||
|
||||
def _reconstruct_openai_response(
|
||||
self,
|
||||
@@ -1082,6 +990,8 @@ class StreamingCCRHandler:
|
||||
async def _response_to_sse(
|
||||
self,
|
||||
response: dict[str, Any],
|
||||
*,
|
||||
envelope: Any | None = None,
|
||||
) -> Any: # AsyncGenerator[bytes, None]
|
||||
"""Convert a response back to SSE format for streaming.
|
||||
|
||||
@@ -1089,10 +999,14 @@ class StreamingCCRHandler:
|
||||
to chunk the response more granularly.
|
||||
"""
|
||||
if self.provider == "anthropic":
|
||||
from headroom.proxy.handlers.streaming import StreamingMixin
|
||||
if envelope is not None:
|
||||
for chunk in envelope.render(response):
|
||||
yield chunk
|
||||
else:
|
||||
from headroom.proxy.handlers.streaming import StreamingMixin
|
||||
|
||||
for chunk in StreamingMixin()._response_to_sse(response, "anthropic"):
|
||||
yield chunk
|
||||
for chunk in StreamingMixin()._response_to_sse(response, "anthropic"):
|
||||
yield chunk
|
||||
else:
|
||||
# OpenAI SSE format: `chat.completion.chunk` frames, then [DONE].
|
||||
for chunk in self._openai_response_to_chunks(response):
|
||||
|
||||
@@ -0,0 +1,682 @@
|
||||
"""Loss-minimizing helpers for the native Anthropic Messages wire envelope.
|
||||
|
||||
The proxy is allowed to understand the message content it optimizes, but it
|
||||
must not need to understand every field Anthropic adds to the surrounding
|
||||
protocol. This module keeps that distinction explicit: known message events
|
||||
are reconstructed into a normal response dictionary, while every SSE frame the
|
||||
reconstruction cannot use (unknown events, and known events whose bytes or
|
||||
payload are malformed) is retained as opaque bytes and replayed by
|
||||
:meth:`AnthropicSSEEnvelope.render`.
|
||||
|
||||
The opaque data is deliberately kept out of the provider JSON. It is an
|
||||
internal rendering detail and can therefore never accidentally be serialized
|
||||
upstream as a synthetic Headroom field.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Collection
|
||||
from dataclasses import dataclass, field, replace
|
||||
from typing import Any
|
||||
|
||||
from headroom.copilot_auth import build_copilot_upstream_url
|
||||
|
||||
_DANGEROUS_TOOL_USE_RE = re.compile(r"(?:^|,)\s*dangerous-tool-use-[^,\s]+", re.IGNORECASE)
|
||||
# Members of each known event (and delta type) that reconstruction interprets.
|
||||
# Anything else on a known frame is an extension carried by ``_FrameExtensions``.
|
||||
_EVENT_FIELDS: dict[str, frozenset[str]] = {
|
||||
"message_start": frozenset({"type", "message"}),
|
||||
"content_block_start": frozenset({"type", "index", "content_block"}),
|
||||
"content_block_delta": frozenset({"type", "index", "delta"}),
|
||||
"content_block_stop": frozenset({"type", "index"}),
|
||||
"message_delta": frozenset({"type", "delta", "usage"}),
|
||||
"message_stop": frozenset({"type"}),
|
||||
}
|
||||
_DELTA_FIELDS: dict[str, frozenset[str]] = {
|
||||
"text_delta": frozenset({"type", "text"}),
|
||||
"input_json_delta": frozenset({"type", "partial_json"}),
|
||||
"thinking_delta": frozenset({"type", "thinking"}),
|
||||
"signature_delta": frozenset({"type", "signature"}),
|
||||
"citations_delta": frozenset({"type", "citation"}),
|
||||
}
|
||||
_MESSAGE_DELTA_FIELDS = ("stop_reason", "stop_sequence", "stop_details")
|
||||
# The payload member each known delta type must carry, and its JSON type.
|
||||
_DELTA_PAYLOADS: dict[str, tuple[str, type]] = {
|
||||
"text_delta": ("text", str),
|
||||
"input_json_delta": ("partial_json", str),
|
||||
"thinking_delta": ("thinking", str),
|
||||
"signature_delta": ("signature", str),
|
||||
"citations_delta": ("citation", dict),
|
||||
}
|
||||
# The block field each string delta is concatenated onto.
|
||||
_ACCUMULATED_FIELDS = {
|
||||
"text_delta": "text",
|
||||
"input_json_delta": "_partial_json",
|
||||
"thinking_delta": "thinking",
|
||||
}
|
||||
_KNOWN_MESSAGE_FIELDS = frozenset(
|
||||
{
|
||||
"id",
|
||||
"type",
|
||||
"role",
|
||||
"model",
|
||||
"content",
|
||||
"stop_reason",
|
||||
"stop_sequence",
|
||||
"stop_details",
|
||||
"usage",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _SSEFrame:
|
||||
"""One raw SSE frame and its decoded payload, when it has one."""
|
||||
|
||||
raw: bytes
|
||||
event_type: str
|
||||
payload: dict[str, Any] | None
|
||||
|
||||
|
||||
def build_anthropic_upstream_url(base_url: str, path: str, raw_query: str = "") -> str:
|
||||
"""Build an Anthropic upstream URL without dropping the incoming query.
|
||||
|
||||
``raw_query`` is intentionally not parsed and re-encoded. Beta query
|
||||
parameters are protocol data, so preserving their spelling and ordering is
|
||||
safer than normalizing them through a mapping.
|
||||
"""
|
||||
|
||||
url = build_copilot_upstream_url(base_url, path)
|
||||
if raw_query:
|
||||
return f"{url}?{raw_query}"
|
||||
return url
|
||||
|
||||
|
||||
def has_dangerous_tool_use_beta(value: str | None) -> bool:
|
||||
"""Return whether a beta header contains an auto-mode capability token."""
|
||||
|
||||
return bool(value and _DANGEROUS_TOOL_USE_RE.search(value))
|
||||
|
||||
|
||||
def is_safeguard_capable_request(body: Any, anthropic_beta: str | None) -> bool:
|
||||
"""Classify a request without inspecting or retaining the safeguard value."""
|
||||
|
||||
return isinstance(body, dict) and (
|
||||
"safeguards" in body or has_dangerous_tool_use_beta(anthropic_beta)
|
||||
)
|
||||
|
||||
|
||||
def strip_safeguard_payload(value: Any) -> Any:
|
||||
"""Copy an observation while removing classifier payload fields.
|
||||
|
||||
This is only for logs, pipeline events, and diagnostics. The actual
|
||||
upstream response is never passed through this helper, so wire fidelity is
|
||||
unaffected.
|
||||
"""
|
||||
|
||||
if isinstance(value, dict):
|
||||
return {
|
||||
key: strip_safeguard_payload(item)
|
||||
for key, item in value.items()
|
||||
if key not in {"safeguards", "safeguard_results"}
|
||||
}
|
||||
if isinstance(value, list):
|
||||
return [strip_safeguard_payload(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def preserve_opaque_response_fields(
|
||||
original: dict[str, Any], replacement: dict[str, Any]
|
||||
) -> dict[str, Any]:
|
||||
"""Retain unknown Anthropic response attachments across a transform.
|
||||
|
||||
Response transforms such as CCR may replace the message produced by the
|
||||
first upstream call with a continuation message. Capability attachments
|
||||
belong to the surrounding protocol rather than to Headroom's mutable
|
||||
message view, so keep them unless the replacement explicitly supplies its
|
||||
own value.
|
||||
"""
|
||||
|
||||
merged = copy.deepcopy(replacement)
|
||||
for key, value in original.items():
|
||||
if key not in _KNOWN_MESSAGE_FIELDS:
|
||||
merged.setdefault(key, copy.deepcopy(value))
|
||||
return merged
|
||||
|
||||
|
||||
def _split_frames(raw: bytes) -> list[bytes]:
|
||||
"""Split SSE bytes while retaining each frame's original delimiters."""
|
||||
|
||||
frames: list[bytes] = []
|
||||
start = 0
|
||||
for match in re.finditer(rb"\r\n\r\n|\n\n|\r\r", raw):
|
||||
end = match.end()
|
||||
if raw[start : match.start()].strip():
|
||||
frames.append(raw[start:end])
|
||||
start = end
|
||||
if raw[start:].strip():
|
||||
frames.append(raw[start:])
|
||||
return frames
|
||||
|
||||
|
||||
def _frame_parts(raw: bytes) -> tuple[str, str | None]:
|
||||
event_name = ""
|
||||
data_lines: list[bytes] = []
|
||||
for line in raw.splitlines():
|
||||
if line.startswith(b"event:"):
|
||||
try:
|
||||
event_name = line[6:].strip().decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return "", None
|
||||
elif line.startswith(b"data:"):
|
||||
data_lines.append(line[5:].lstrip())
|
||||
try:
|
||||
return event_name, b"\n".join(data_lines).decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
# Leave the complete frame opaque so rendering can replay its exact
|
||||
# bytes. Replacing malformed input would violate the proxy's fail-open
|
||||
# wire-fidelity contract.
|
||||
return event_name, None
|
||||
|
||||
|
||||
def _json_payload(raw: bytes) -> tuple[str, dict[str, Any] | None]:
|
||||
event_name, data = _frame_parts(raw)
|
||||
if data is None:
|
||||
return event_name, None
|
||||
if not data or data == "[DONE]":
|
||||
return event_name or "[DONE]", None
|
||||
try:
|
||||
payload = json.loads(data)
|
||||
except (TypeError, json.JSONDecodeError):
|
||||
return event_name, None
|
||||
if not isinstance(payload, dict):
|
||||
return event_name, None
|
||||
return event_name or str(payload.get("type", "")), payload
|
||||
|
||||
|
||||
def _sse_event(payload: dict[str, Any], *, event_name: str | None = None) -> bytes:
|
||||
name = event_name or str(payload.get("type", "message"))
|
||||
return f"event: {name}\ndata: {json.dumps(payload, ensure_ascii=False)}\n\n".encode()
|
||||
|
||||
|
||||
def _wire_index(value: Any) -> int | None:
|
||||
"""Return a content block index, or None when it is not a valid one.
|
||||
|
||||
``index`` is provider JSON and may be any JSON value. Only a non-negative
|
||||
integer identifies a block; anything else (including an unhashable object
|
||||
or array) must leave its frame unreconstructable instead of reaching a
|
||||
dictionary lookup.
|
||||
"""
|
||||
|
||||
if isinstance(value, int) and not isinstance(value, bool) and value >= 0:
|
||||
return int(value)
|
||||
return None
|
||||
|
||||
|
||||
def _extensions(payload: dict[str, Any], known: Collection[str]) -> dict[str, Any]:
|
||||
"""Return a copy of the members of ``payload`` this module does not interpret."""
|
||||
|
||||
return {key: copy.deepcopy(value) for key, value in payload.items() if key not in known}
|
||||
|
||||
|
||||
@dataclass
|
||||
class _BlockExtensions:
|
||||
"""Unknown members carried by one content block's known frames."""
|
||||
|
||||
start: dict[str, Any] = field(default_factory=dict)
|
||||
stop: dict[str, Any] = field(default_factory=dict)
|
||||
# delta type -> (members on the event, members inside its ``delta``)
|
||||
deltas: dict[str, tuple[dict[str, Any], dict[str, Any]]] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _FrameExtensions:
|
||||
"""Unknown members of known frames, keyed by the canonical frame they rode on.
|
||||
|
||||
Rendering regenerates known frames from the message dictionary, which has
|
||||
no place for members added to the events themselves. They are kept here
|
||||
and re-attached to the corresponding regenerated frame instead.
|
||||
"""
|
||||
|
||||
message_start: dict[str, Any] = field(default_factory=dict)
|
||||
message_delta: dict[str, Any] = field(default_factory=dict)
|
||||
message_delta_delta: dict[str, Any] = field(default_factory=dict)
|
||||
# ``message_delta.usage`` keys upstream sent; values come from the
|
||||
# rendered response so a CCR continuation reports its own usage.
|
||||
message_delta_usage_keys: list[str] = field(default_factory=list)
|
||||
message_stop: dict[str, Any] = field(default_factory=dict)
|
||||
# Keyed by position in ``message["content"]``, not by wire index.
|
||||
blocks: dict[int, _BlockExtensions] = field(default_factory=dict)
|
||||
|
||||
def for_content(self, original: Any, rendered: Any) -> _FrameExtensions:
|
||||
"""Keep block members only where the rendered block is the original one.
|
||||
|
||||
A response transform such as CCR can replace content blocks; members
|
||||
observed on the original block's frames do not describe its
|
||||
replacement. Message-level members belong to the surrounding protocol
|
||||
and are always kept.
|
||||
"""
|
||||
|
||||
before = original if isinstance(original, list) else []
|
||||
after = rendered if isinstance(rendered, list) else []
|
||||
blocks = {
|
||||
position: extensions
|
||||
for position, extensions in self.blocks.items()
|
||||
if position < len(before)
|
||||
and position < len(after)
|
||||
and after[position] == before[position]
|
||||
}
|
||||
return replace(self, blocks=blocks)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Reconstruction:
|
||||
message: dict[str, Any]
|
||||
extensions: _FrameExtensions
|
||||
saw_start: bool
|
||||
saw_stop: bool
|
||||
saw_error: bool
|
||||
open_blocks: set[int]
|
||||
# Positions of the frames the message (plus extensions) was built from.
|
||||
consumed: set[int]
|
||||
|
||||
|
||||
def _response_from_events(frames: list[_SSEFrame]) -> _Reconstruction:
|
||||
"""Reconstruct the message and report which frames it was built from.
|
||||
|
||||
The ``consumed`` positions are the only frames that
|
||||
:func:`_render_known_response` can regenerate. Every other frame,
|
||||
including a known event whose payload is malformed or does not fit the
|
||||
stream's structure, must be replayed verbatim.
|
||||
"""
|
||||
|
||||
response: dict[str, Any] = {"content": [], "usage": {}}
|
||||
extensions = _FrameExtensions()
|
||||
blocks_by_index: dict[int, dict[str, Any]] = {}
|
||||
# Keyed by ``id()`` of the block dictionary; the dictionaries stay alive
|
||||
# in ``blocks_by_index`` or ``response["content"]`` for the whole parse.
|
||||
block_extensions: dict[int, _BlockExtensions] = {}
|
||||
current_block: dict[str, Any] | None = None
|
||||
appended: set[int] = set()
|
||||
open_blocks: set[int] = set()
|
||||
consumed: set[int] = set()
|
||||
saw_start = saw_stop = saw_error = False
|
||||
|
||||
for position, frame in enumerate(frames):
|
||||
data = frame.payload
|
||||
event_type = frame.event_type
|
||||
if data is None:
|
||||
continue
|
||||
if event_type == "message_start":
|
||||
saw_start = True
|
||||
message = data.get("message")
|
||||
usage = message.get("usage") if isinstance(message, dict) else None
|
||||
# ``usage`` is merged into by later message_delta frames, so a
|
||||
# non-object value would make the message unreconstructable.
|
||||
if isinstance(message, dict) and (usage is None or isinstance(usage, dict)):
|
||||
consumed.add(position)
|
||||
extensions.message_start.update(_extensions(data, _EVENT_FIELDS[event_type]))
|
||||
# Start with the complete upstream message object. This is
|
||||
# what preserves future top-level attachments.
|
||||
response.update(copy.deepcopy(message))
|
||||
# The stream's content blocks below are authoritative. A
|
||||
# defensive non-empty content array in message_start must not
|
||||
# be duplicated when those blocks are replayed.
|
||||
response["content"] = []
|
||||
if response.get("usage") is None:
|
||||
response["usage"] = {}
|
||||
elif event_type == "content_block_start":
|
||||
block = data.get("content_block")
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
raw_index = data.get("index")
|
||||
if raw_index is None:
|
||||
index = len(response["content"])
|
||||
else:
|
||||
parsed_index = _wire_index(raw_index)
|
||||
if parsed_index is None:
|
||||
continue
|
||||
index = parsed_index
|
||||
current_block = copy.deepcopy(block)
|
||||
blocks_by_index[index] = current_block
|
||||
block_extensions[id(current_block)] = _BlockExtensions(
|
||||
start=_extensions(data, _EVENT_FIELDS[event_type])
|
||||
)
|
||||
open_blocks.add(index)
|
||||
consumed.add(position)
|
||||
elif event_type == "content_block_delta":
|
||||
delta = data.get("delta")
|
||||
if not isinstance(delta, dict):
|
||||
continue
|
||||
raw_index = data.get("index")
|
||||
block_index = _wire_index(raw_index)
|
||||
if raw_index is not None and block_index is None:
|
||||
continue
|
||||
target = blocks_by_index.get(block_index) if block_index is not None else current_block
|
||||
dtype = delta.get("type")
|
||||
if target is None or not isinstance(dtype, str) or dtype not in _DELTA_FIELDS:
|
||||
continue
|
||||
# Validate the raw member: a missing, null or falsy non-string value
|
||||
# is not an empty delta, and must not be consumed as one.
|
||||
member, member_type = _DELTA_PAYLOADS[dtype]
|
||||
payload = delta.get(member)
|
||||
if not isinstance(payload, member_type):
|
||||
continue
|
||||
accumulated = _ACCUMULATED_FIELDS.get(dtype)
|
||||
if accumulated is not None and not isinstance(target.get(accumulated, ""), str):
|
||||
continue
|
||||
if dtype == "citations_delta" and not isinstance(target.get("citations", []), list):
|
||||
continue
|
||||
consumed.add(position)
|
||||
event_extensions, delta_extensions = block_extensions[id(target)].deltas.setdefault(
|
||||
dtype, ({}, {})
|
||||
)
|
||||
event_extensions.update(_extensions(data, _EVENT_FIELDS[event_type]))
|
||||
delta_extensions.update(_extensions(delta, _DELTA_FIELDS[dtype]))
|
||||
if accumulated is not None:
|
||||
target[accumulated] = target.get(accumulated, "") + payload
|
||||
elif dtype == "signature_delta":
|
||||
target["signature"] = payload
|
||||
elif dtype == "citations_delta":
|
||||
target.setdefault("citations", []).append(payload)
|
||||
elif event_type == "content_block_stop":
|
||||
raw_index = data.get("index")
|
||||
block_index = _wire_index(raw_index)
|
||||
if raw_index is not None and block_index is None:
|
||||
continue
|
||||
target = blocks_by_index.get(block_index) if block_index is not None else current_block
|
||||
if target is None:
|
||||
continue
|
||||
consumed.add(position)
|
||||
block_extensions[id(target)].stop.update(_extensions(data, _EVENT_FIELDS[event_type]))
|
||||
partial = target.pop("_partial_json", None)
|
||||
if partial is not None:
|
||||
try:
|
||||
target["input"] = json.loads(partial) if partial else {}
|
||||
except (TypeError, json.JSONDecodeError):
|
||||
target["input"] = {}
|
||||
appended_key = block_index if block_index is not None else id(target)
|
||||
if appended_key not in appended:
|
||||
response["content"].append(target)
|
||||
appended.add(appended_key)
|
||||
if block_index is not None:
|
||||
open_blocks.discard(block_index)
|
||||
current_block = None
|
||||
elif event_type == "message_delta":
|
||||
delta = data.get("delta")
|
||||
usage = data.get("usage")
|
||||
if (delta is not None and not isinstance(delta, dict)) or (
|
||||
usage is not None and not isinstance(usage, dict)
|
||||
):
|
||||
continue
|
||||
consumed.add(position)
|
||||
if isinstance(delta, dict):
|
||||
for key in _MESSAGE_DELTA_FIELDS:
|
||||
if key in delta:
|
||||
response[key] = copy.deepcopy(delta[key])
|
||||
extensions.message_delta_delta.update(_extensions(delta, _MESSAGE_DELTA_FIELDS))
|
||||
if isinstance(usage, dict):
|
||||
response.setdefault("usage", {}).update(copy.deepcopy(usage))
|
||||
for key in usage:
|
||||
if key not in extensions.message_delta_usage_keys:
|
||||
extensions.message_delta_usage_keys.append(key)
|
||||
# Some Anthropic additions are attached directly to the delta
|
||||
# event rather than nested below ``delta``. Preserve them as
|
||||
# response fields without interpreting their schemas.
|
||||
event_extensions = _extensions(data, _EVENT_FIELDS[event_type])
|
||||
response.update(copy.deepcopy(event_extensions))
|
||||
extensions.message_delta.update(event_extensions)
|
||||
elif event_type == "message_stop":
|
||||
saw_stop = True
|
||||
consumed.add(position)
|
||||
extensions.message_stop.update(_extensions(data, _EVENT_FIELDS[event_type]))
|
||||
elif event_type == "error":
|
||||
saw_error = True
|
||||
|
||||
extensions.blocks = {
|
||||
index: block_extensions[id(block)]
|
||||
for index, block in enumerate(response["content"])
|
||||
if id(block) in block_extensions
|
||||
}
|
||||
return _Reconstruction(
|
||||
response, extensions, saw_start, saw_stop, saw_error, open_blocks, consumed
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class AnthropicSSEEnvelope:
|
||||
"""Parsed Anthropic response plus opaque frames retained for replay."""
|
||||
|
||||
message: dict[str, Any]
|
||||
# (number of reconstructed frames before it, raw bytes) for every frame
|
||||
# that the reconstructed message cannot regenerate.
|
||||
_opaque_frames: list[tuple[int, bytes]]
|
||||
_extensions: _FrameExtensions
|
||||
saw_message_start: bool
|
||||
saw_message_stop: bool
|
||||
saw_error: bool
|
||||
saw_unreconstructable_frame: bool
|
||||
open_block_indices: set[int]
|
||||
|
||||
@classmethod
|
||||
def parse(cls, raw_sse_bytes: bytes) -> AnthropicSSEEnvelope:
|
||||
"""Parse Anthropic SSE bytes while retaining opaque frames verbatim.
|
||||
|
||||
Args:
|
||||
raw_sse_bytes: Complete or partial Anthropic SSE wire bytes.
|
||||
|
||||
Returns:
|
||||
A request-local envelope containing the reconstructed message and
|
||||
every original frame needed for loss-minimizing replay.
|
||||
"""
|
||||
|
||||
frames = [_SSEFrame(raw, *_json_payload(raw)) for raw in _split_frames(raw_sse_bytes)]
|
||||
reconstruction = _response_from_events(frames)
|
||||
|
||||
# Anything the reconstruction did not consume is replayed verbatim.
|
||||
# A recognized event name is not enough: a known frame with malformed
|
||||
# bytes or an unusable payload would otherwise be neither rendered
|
||||
# nor replayed, and would silently disappear from the stream.
|
||||
opaque_frames: list[tuple[int, bytes]] = []
|
||||
known_count = 0
|
||||
saw_unreconstructable_frame = False
|
||||
for position, frame in enumerate(frames):
|
||||
if position in reconstruction.consumed:
|
||||
known_count += 1
|
||||
continue
|
||||
if frame.event_type in _EVENT_FIELDS:
|
||||
saw_unreconstructable_frame = True
|
||||
opaque_frames.append((known_count, frame.raw))
|
||||
|
||||
return cls(
|
||||
reconstruction.message,
|
||||
opaque_frames,
|
||||
reconstruction.extensions,
|
||||
reconstruction.saw_start,
|
||||
reconstruction.saw_stop,
|
||||
reconstruction.saw_error,
|
||||
saw_unreconstructable_frame,
|
||||
reconstruction.open_blocks,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_events(cls, events: list[dict[str, Any]]) -> AnthropicSSEEnvelope:
|
||||
"""Build an envelope for callers that already decoded event JSON."""
|
||||
|
||||
raw = b"".join(_sse_event(event) for event in events if isinstance(event, dict))
|
||||
return cls.parse(raw)
|
||||
|
||||
def is_complete(self) -> bool:
|
||||
"""Return whether the stream has complete structural framing."""
|
||||
|
||||
return (
|
||||
self.saw_message_start
|
||||
and self.saw_message_stop
|
||||
and not self.saw_error
|
||||
and not self.open_block_indices
|
||||
)
|
||||
|
||||
def is_message_reconstructable(self) -> bool:
|
||||
"""Return whether strict callers can safely use the message dictionary."""
|
||||
|
||||
return self.is_complete() and not self.saw_unreconstructable_frame
|
||||
|
||||
def render(self, message: dict[str, Any] | None = None) -> list[bytes]:
|
||||
"""Render a changed message and replay opaque frames at their anchors."""
|
||||
|
||||
response = preserve_opaque_response_fields(
|
||||
self.message,
|
||||
message if isinstance(message, dict) else self.message,
|
||||
)
|
||||
extensions = self._extensions.for_content(
|
||||
self.message.get("content"), response.get("content")
|
||||
)
|
||||
standard = _render_known_response(response, extensions)
|
||||
if self._opaque_frames:
|
||||
by_anchor: dict[int, list[bytes]] = {}
|
||||
for known_before, raw in self._opaque_frames:
|
||||
by_anchor.setdefault(min(known_before, len(standard)), []).append(raw)
|
||||
out: list[bytes] = []
|
||||
for index in range(len(standard) + 1):
|
||||
out.extend(by_anchor.get(index, []))
|
||||
if index < len(standard):
|
||||
out.append(standard[index])
|
||||
return out
|
||||
return standard
|
||||
|
||||
|
||||
def _block_delta(index: int, delta: dict[str, Any], extensions: _BlockExtensions) -> bytes:
|
||||
event_extensions, delta_extensions = extensions.deltas.get(delta["type"], ({}, {}))
|
||||
return _sse_event(
|
||||
{
|
||||
"type": "content_block_delta",
|
||||
"index": index,
|
||||
"delta": {**delta, **copy.deepcopy(delta_extensions)},
|
||||
**copy.deepcopy(event_extensions),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _render_known_response(
|
||||
response: dict[str, Any], extensions: _FrameExtensions | None = None
|
||||
) -> list[bytes]:
|
||||
ext = extensions if extensions is not None else _FrameExtensions()
|
||||
message = copy.deepcopy(response)
|
||||
content = message.pop("content", [])
|
||||
raw_usage = message.get("usage")
|
||||
usage: dict[str, Any] = raw_usage if isinstance(raw_usage, dict) else {}
|
||||
msg_start_message = copy.deepcopy(message)
|
||||
msg_start_message.setdefault("type", "message")
|
||||
msg_start_message.setdefault("role", "assistant")
|
||||
msg_start_message.setdefault("content", [])
|
||||
msg_start_message.setdefault("stop_reason", None)
|
||||
msg_start_message["usage"] = usage
|
||||
events = [
|
||||
_sse_event(
|
||||
{
|
||||
"type": "message_start",
|
||||
"message": msg_start_message,
|
||||
**copy.deepcopy(ext.message_start),
|
||||
}
|
||||
)
|
||||
]
|
||||
|
||||
if isinstance(content, list):
|
||||
for index, block in enumerate(content):
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
block_ext = ext.blocks.get(index, _BlockExtensions())
|
||||
block_type = block.get("type")
|
||||
start_block = copy.deepcopy(block)
|
||||
if block_type == "text":
|
||||
start_block["text"] = ""
|
||||
start_block.pop("citations", None)
|
||||
elif block_type == "tool_use":
|
||||
start_block["input"] = {}
|
||||
elif block_type == "thinking":
|
||||
start_block["thinking"] = ""
|
||||
# Any other block, including ``server_tool_use``, is emitted
|
||||
# complete in its start frame: its input is not streamed.
|
||||
events.append(
|
||||
_sse_event(
|
||||
{
|
||||
"type": "content_block_start",
|
||||
"index": index,
|
||||
"content_block": start_block,
|
||||
**copy.deepcopy(block_ext.start),
|
||||
}
|
||||
)
|
||||
)
|
||||
if block_type == "text" and block.get("text"):
|
||||
events.append(
|
||||
_block_delta(index, {"type": "text_delta", "text": block["text"]}, block_ext)
|
||||
)
|
||||
for citation in block.get("citations", []) or []:
|
||||
events.append(
|
||||
_block_delta(
|
||||
index, {"type": "citations_delta", "citation": citation}, block_ext
|
||||
)
|
||||
)
|
||||
elif block_type == "tool_use" and "input" in block:
|
||||
partial_json = json.dumps(block.get("input") or {}, ensure_ascii=False)
|
||||
events.append(
|
||||
_block_delta(
|
||||
index, {"type": "input_json_delta", "partial_json": partial_json}, block_ext
|
||||
)
|
||||
)
|
||||
elif block_type == "thinking":
|
||||
if block.get("thinking"):
|
||||
events.append(
|
||||
_block_delta(
|
||||
index,
|
||||
{"type": "thinking_delta", "thinking": block["thinking"]},
|
||||
block_ext,
|
||||
)
|
||||
)
|
||||
if block.get("signature"):
|
||||
events.append(
|
||||
_block_delta(
|
||||
index,
|
||||
{"type": "signature_delta", "signature": block["signature"]},
|
||||
block_ext,
|
||||
)
|
||||
)
|
||||
events.append(
|
||||
_sse_event(
|
||||
{
|
||||
"type": "content_block_stop",
|
||||
"index": index,
|
||||
**copy.deepcopy(block_ext.stop),
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
delta_usage = {"output_tokens": usage.get("output_tokens", 0)}
|
||||
for key in ext.message_delta_usage_keys:
|
||||
if key in usage:
|
||||
delta_usage[key] = copy.deepcopy(usage[key])
|
||||
delta: dict[str, Any] = {"type": "message_delta", "delta": {}, "usage": delta_usage}
|
||||
for key in _MESSAGE_DELTA_FIELDS:
|
||||
if key in response:
|
||||
delta["delta"][key] = response[key]
|
||||
delta["delta"].update(copy.deepcopy(ext.message_delta_delta))
|
||||
delta.update(copy.deepcopy(ext.message_delta))
|
||||
events.append(_sse_event(delta))
|
||||
events.append(_sse_event({"type": "message_stop", **copy.deepcopy(ext.message_stop)}))
|
||||
return events
|
||||
|
||||
|
||||
def render_anthropic_sse_response(response: dict[str, Any]) -> list[bytes]:
|
||||
"""Render an Anthropic response dictionary as canonical SSE frames.
|
||||
|
||||
Args:
|
||||
response: Anthropic Messages API response dictionary.
|
||||
|
||||
Returns:
|
||||
Canonically encoded Anthropic SSE frames.
|
||||
"""
|
||||
|
||||
return _render_known_response(response)
|
||||
@@ -31,12 +31,14 @@ import httpx
|
||||
from headroom.agent_savings import proxy_pipeline_kwargs
|
||||
from headroom.ccr.context_tracker import looks_like_claude_code_compact_summary
|
||||
from headroom.ccr.marker_resolution import resolve_markers_in_response
|
||||
from headroom.copilot_auth import (
|
||||
apply_copilot_api_auth,
|
||||
build_copilot_upstream_url,
|
||||
is_copilot_upstream_url,
|
||||
)
|
||||
from headroom.copilot_auth import apply_copilot_api_auth, is_copilot_upstream_url
|
||||
from headroom.pipeline import PipelineStage, summarize_routing_markers
|
||||
from headroom.proxy.anthropic_wire import (
|
||||
build_anthropic_upstream_url,
|
||||
is_safeguard_capable_request,
|
||||
preserve_opaque_response_fields,
|
||||
strip_safeguard_payload,
|
||||
)
|
||||
from headroom.proxy.auth_mode import (
|
||||
classify_auth_mode,
|
||||
classify_client,
|
||||
@@ -1069,6 +1071,12 @@ class AnthropicHandlerMixin:
|
||||
},
|
||||
},
|
||||
)
|
||||
# This is a boolean capability classification only. The
|
||||
# safeguards value itself is never copied into logs, metrics, or
|
||||
# any Headroom-owned state.
|
||||
client_anthropic_version = request.headers.get("anthropic-version")
|
||||
client_anthropic_beta = request.headers.get("anthropic-beta")
|
||||
safeguard_capable_request = is_safeguard_capable_request(body, client_anthropic_beta)
|
||||
raw_model = body.get("model") or model_override or "unknown"
|
||||
model = (
|
||||
sanitize_anthropic_model_id(raw_model) if isinstance(raw_model, str) else raw_model
|
||||
@@ -1636,23 +1644,33 @@ class AnthropicHandlerMixin:
|
||||
log_beta_header_merge,
|
||||
)
|
||||
|
||||
_client_beta_value = headers.get("anthropic-beta")
|
||||
_client_beta_value = client_anthropic_beta
|
||||
_client_beta_count = (
|
||||
len([t for t in (_client_beta_value or "").split(",") if t.strip()])
|
||||
if _client_beta_value
|
||||
else 0
|
||||
)
|
||||
_sticky_beta_value = get_session_beta_tracker().record_and_get_sticky_betas(
|
||||
provider="anthropic",
|
||||
session_id=session_id,
|
||||
client_value=_client_beta_value,
|
||||
)
|
||||
if safeguard_capable_request:
|
||||
# Claude Code's classifier capability is negotiated by the
|
||||
# client-owned header. Do not let prior turns' sticky beta
|
||||
# state rewrite this turn.
|
||||
_sticky_beta_value = _client_beta_value or ""
|
||||
else:
|
||||
_sticky_beta_value = get_session_beta_tracker().record_and_get_sticky_betas(
|
||||
provider="anthropic",
|
||||
session_id=session_id,
|
||||
client_value=_client_beta_value,
|
||||
)
|
||||
_sticky_beta_count = (
|
||||
len([t for t in _sticky_beta_value.split(",") if t.strip()])
|
||||
if _sticky_beta_value
|
||||
else 0
|
||||
)
|
||||
if _sticky_beta_value and _sticky_beta_value != (_client_beta_value or ""):
|
||||
if (
|
||||
not safeguard_capable_request
|
||||
and _sticky_beta_value
|
||||
and _sticky_beta_value != (_client_beta_value or "")
|
||||
):
|
||||
headers["anthropic-beta"] = _sticky_beta_value
|
||||
elif not _sticky_beta_value and "anthropic-beta" in headers:
|
||||
# Sticky value can only equal "" when both client and
|
||||
@@ -2809,6 +2827,11 @@ class AnthropicHandlerMixin:
|
||||
# need its own merge helper.
|
||||
headers[key] = value
|
||||
continue
|
||||
if safeguard_capable_request:
|
||||
# Memory may still inject its tools, but it
|
||||
# must not negotiate a different capability on
|
||||
# a classifier-bearing turn.
|
||||
continue
|
||||
existing_value = headers.get(key, "")
|
||||
required_tokens = [t.strip() for t in value.split(",") if t.strip()]
|
||||
_headroom_beta_added = True
|
||||
@@ -3187,7 +3210,11 @@ class AnthropicHandlerMixin:
|
||||
# client's own tokens — the same reduction the gateway contract
|
||||
# applies before handing ``headers`` to the gateway.
|
||||
_hook_headers = getattr(_req_ctx, "provider_headers", None)
|
||||
if isinstance(_hook_headers, dict) and _hook_headers:
|
||||
if (
|
||||
isinstance(_hook_headers, dict)
|
||||
and _hook_headers
|
||||
and not safeguard_capable_request
|
||||
):
|
||||
from headroom.proxy.turn_hooks import merge_provider_headers
|
||||
|
||||
for _hh_key, _hh_value in merge_provider_headers(
|
||||
@@ -3439,6 +3466,18 @@ class AnthropicHandlerMixin:
|
||||
except (json.JSONDecodeError, ValueError, MemoryError, RecursionError):
|
||||
body_mutation_tracker.mark_mutated("original_unparseable")
|
||||
|
||||
if safeguard_capable_request:
|
||||
# Restore the exact client-owned capability headers after all
|
||||
# pipeline, memory, and hook stages. Header names are
|
||||
# case-insensitive; values are not normalized or merged.
|
||||
for key in list(headers):
|
||||
if key.lower() in {"anthropic-version", "anthropic-beta"}:
|
||||
headers.pop(key, None)
|
||||
if client_anthropic_version is not None:
|
||||
headers["anthropic-version"] = client_anthropic_version
|
||||
if client_anthropic_beta is not None:
|
||||
headers["anthropic-beta"] = client_anthropic_beta
|
||||
|
||||
if (
|
||||
(upstream_base_url or self.ANTHROPIC_API_URL != "https://api.anthropic.com")
|
||||
and stream
|
||||
@@ -3505,7 +3544,7 @@ class AnthropicHandlerMixin:
|
||||
model=model,
|
||||
messages=body["messages"],
|
||||
tools=tools,
|
||||
response=backend_response.body,
|
||||
response=(None if safeguard_capable_request else backend_response.body),
|
||||
metadata={
|
||||
"path": pipeline_path,
|
||||
"stream": False,
|
||||
@@ -3518,7 +3557,7 @@ class AnthropicHandlerMixin:
|
||||
request_id=request_id,
|
||||
provider=pipeline_provider,
|
||||
model=model,
|
||||
response=backend_response.body,
|
||||
response=(None if safeguard_capable_request else backend_response.body),
|
||||
metadata={
|
||||
"path": pipeline_path,
|
||||
"stream": False,
|
||||
@@ -3719,13 +3758,11 @@ class AnthropicHandlerMixin:
|
||||
# every one of those turns to "anthropic" on the dashboard.
|
||||
# For a non-Copilot base the builder only joins base + path, so the
|
||||
# URL itself is unchanged.
|
||||
url = (
|
||||
build_copilot_upstream_url(upstream_base_url, request.url.path)
|
||||
if upstream_base_url
|
||||
else build_copilot_upstream_url(self.ANTHROPIC_API_URL, "/v1/messages")
|
||||
url = build_anthropic_upstream_url(
|
||||
upstream_base_url or self.ANTHROPIC_API_URL,
|
||||
request.url.path,
|
||||
request.url.query,
|
||||
)
|
||||
if upstream_base_url and request.url.query:
|
||||
url = f"{url}?{request.url.query}"
|
||||
|
||||
try:
|
||||
ccr_handler_config = getattr(self.ccr_response_handler, "config", None)
|
||||
@@ -4028,7 +4065,7 @@ class AnthropicHandlerMixin:
|
||||
model=model,
|
||||
messages=body["messages"],
|
||||
tools=tools,
|
||||
response=response,
|
||||
response=(None if safeguard_capable_request else response),
|
||||
metadata={
|
||||
"path": pipeline_path,
|
||||
"stream": False,
|
||||
@@ -4043,7 +4080,7 @@ class AnthropicHandlerMixin:
|
||||
request_id=request_id,
|
||||
provider=pipeline_provider,
|
||||
model=model,
|
||||
response=response,
|
||||
response=(None if safeguard_capable_request else response),
|
||||
metadata={
|
||||
"path": pipeline_path,
|
||||
"stream": False,
|
||||
@@ -4067,11 +4104,21 @@ class AnthropicHandlerMixin:
|
||||
if response.status_code >= 400:
|
||||
try:
|
||||
err_body = response.json()
|
||||
if safeguard_capable_request:
|
||||
err_body = strip_safeguard_payload(err_body)
|
||||
err_msg = err_body.get("error", {}).get("message", "")
|
||||
err_type = err_body.get("error", {}).get("type", "")
|
||||
except Exception:
|
||||
err_body = {"raw": response.text[:2000]}
|
||||
err_msg = str(response.text[:500])
|
||||
err_body = (
|
||||
{"raw": "<redacted classifier-bearing upstream error>"}
|
||||
if safeguard_capable_request
|
||||
else {"raw": response.text[:2000]}
|
||||
)
|
||||
err_msg = (
|
||||
"<redacted classifier-bearing upstream error>"
|
||||
if safeguard_capable_request
|
||||
else str(response.text[:500])
|
||||
)
|
||||
err_type = "parse_error"
|
||||
|
||||
logger.warning(
|
||||
@@ -4342,6 +4389,9 @@ class AnthropicHandlerMixin:
|
||||
api_call_fn,
|
||||
provider="anthropic",
|
||||
)
|
||||
final_resp_json = preserve_opaque_response_fields(
|
||||
resp_json, final_resp_json
|
||||
)
|
||||
# Update response content with final response
|
||||
resp_json = final_resp_json
|
||||
# Remove encoding headers since content is now uncompressed JSON
|
||||
|
||||
@@ -409,393 +409,31 @@ class StreamingMixin:
|
||||
cannot replay; anything else returns None so the caller can fail
|
||||
loudly instead.
|
||||
"""
|
||||
if provider != "anthropic":
|
||||
return None # Only implemented for Anthropic
|
||||
if provider == "anthropic":
|
||||
# Keep the complete wire envelope alongside the mutable message
|
||||
# view. Callers that need to render a changed response own this
|
||||
# envelope in their request-local flow; never attach it to this
|
||||
# shared handler instance.
|
||||
from headroom.proxy.anthropic_wire import AnthropicSSEEnvelope
|
||||
|
||||
# Event framing is CRLF in some intermediaries (and mixed after a
|
||||
# retry through one). Normalize before the line split so a
|
||||
# correctly framed stream is never read as zero events.
|
||||
sse_data = sse_data.replace("\r\n", "\n").replace("\r", "\n")
|
||||
|
||||
response: dict[str, Any] = {"content": [], "usage": {}}
|
||||
saw_message_start = False
|
||||
saw_message_stop = False
|
||||
saw_error = False
|
||||
saw_unreplayable_delta = False
|
||||
open_block_indices: set[int] = set()
|
||||
# Track blocks by their `index` field so out-of-order events
|
||||
# don't corrupt the reconstruction. The current block pointer
|
||||
# remains for backward-compat with code that walks this dict
|
||||
# sequentially, but the index map is the source of truth.
|
||||
blocks_by_index: dict[int, dict[str, Any]] = {}
|
||||
current_block: dict[str, Any] | None = None
|
||||
# Track which block indices have already been appended to
|
||||
# `response["content"]`. Dedup used to be `target not in
|
||||
# response["content"]` — plain dict-equality. Two distinct blocks
|
||||
# that happen to accumulate identical values (most commonly two
|
||||
# separate empty `thinking` blocks, e.g. from a retried HTTP/2
|
||||
# stream reset redelivering a truncated segment) either got
|
||||
# wrongly collapsed into one, or — when their partial content
|
||||
# happened to differ (same index, unequal dict) — both slipped
|
||||
# through as duplicates. Indexing by `index` (falling back to
|
||||
# object identity for the legacy no-index path) makes dedup exact
|
||||
# regardless of what the accumulated content looks like: one
|
||||
# entry per block index, first `content_block_stop` wins.
|
||||
appended_block_keys: set[int] = set()
|
||||
|
||||
for line in sse_data.split("\n"):
|
||||
if not line.startswith("data:"):
|
||||
continue
|
||||
data_str = line[5:].strip()
|
||||
if not data_str or data_str == "[DONE]":
|
||||
continue
|
||||
|
||||
try:
|
||||
data = json.loads(data_str)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
event_type = data.get("type", "")
|
||||
|
||||
if event_type == "message_start":
|
||||
saw_message_start = True
|
||||
msg = data.get("message", {})
|
||||
response["id"] = msg.get("id")
|
||||
response["type"] = msg.get("type", "message")
|
||||
if "stop_sequence" in msg:
|
||||
response["stop_sequence"] = msg["stop_sequence"]
|
||||
response["model"] = msg.get("model")
|
||||
response["role"] = msg.get("role", "assistant")
|
||||
response["stop_reason"] = msg.get("stop_reason")
|
||||
if "stop_details" in msg:
|
||||
response["stop_details"] = msg["stop_details"]
|
||||
if msg.get("usage"):
|
||||
response["usage"].update(msg["usage"])
|
||||
|
||||
elif event_type == "content_block_start":
|
||||
block = data.get("content_block", {})
|
||||
block_index = data.get("index", len(response["content"]))
|
||||
btype = block.get("type")
|
||||
current_block = {
|
||||
"type": btype,
|
||||
"index": block_index,
|
||||
}
|
||||
if btype == "text":
|
||||
current_block["text"] = block.get("text", "")
|
||||
elif btype == "tool_use":
|
||||
current_block["id"] = block.get("id")
|
||||
current_block["name"] = block.get("name")
|
||||
current_block["input"] = {}
|
||||
elif btype == "thinking":
|
||||
# Thinking block — accumulate text via
|
||||
# `thinking_delta`; signature arrives via
|
||||
# `signature_delta` (single value, not accumulated).
|
||||
current_block["thinking_buffer"] = block.get("thinking", "")
|
||||
if "signature" in block:
|
||||
current_block["signature"] = block["signature"]
|
||||
elif btype == "redacted_thinking":
|
||||
# Per Anthropic spec §2.7: opaque encrypted reasoning
|
||||
# block. The `data` field is preserved as-is and
|
||||
# MUST be replayed unchanged on the next turn for
|
||||
# signature validation to pass.
|
||||
if "data" in block:
|
||||
current_block["data"] = block["data"]
|
||||
elif btype:
|
||||
# Non-standard block (server_tool_use, web_search_tool_result,
|
||||
# ...): copy through all of the block's fields so the
|
||||
# reconstruction doesn't silently drop them to a bare
|
||||
# {type, index}. Mirrors the sibling reconstructor
|
||||
# `_reconstruct_anthropic_response`, which does `dict(block)`.
|
||||
for _k, _v in block.items():
|
||||
if _k != "type":
|
||||
current_block[_k] = _v
|
||||
blocks_by_index[block_index] = current_block
|
||||
open_block_indices.add(block_index)
|
||||
|
||||
elif event_type == "content_block_delta":
|
||||
# Resolve the target block by index (preferred) or fall
|
||||
# back to current_block for legacy linear streams.
|
||||
idx = data.get("index")
|
||||
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
|
||||
if target is not None:
|
||||
delta = data.get("delta", {})
|
||||
dtype = delta.get("type")
|
||||
if dtype == "text_delta":
|
||||
target["text"] = target.get("text", "") + delta.get("text", "")
|
||||
elif dtype == "input_json_delta":
|
||||
# Accumulate partial JSON for tool input.
|
||||
partial = delta.get("partial_json", "")
|
||||
target["_partial_json"] = target.get("_partial_json", "") + partial
|
||||
elif dtype == "thinking_delta":
|
||||
# Accumulate thinking text into the dedicated
|
||||
# buffer so it never collides with `text` on
|
||||
# text blocks (separate field per guide §2.7).
|
||||
target["thinking_buffer"] = target.get("thinking_buffer", "") + delta.get(
|
||||
"thinking", ""
|
||||
)
|
||||
elif dtype == "signature_delta":
|
||||
# Single value, not accumulated. Last-write
|
||||
# wins per Anthropic spec.
|
||||
if "signature" in delta:
|
||||
target["signature"] = delta["signature"]
|
||||
elif dtype == "citations_delta":
|
||||
# Append the citation object to the citations
|
||||
# list so multi-citation blocks reconstruct
|
||||
# correctly. Per guide §2.5: each delta carries
|
||||
# one full citation object under `citation`.
|
||||
citations = target.setdefault("citations", [])
|
||||
citation = delta.get("citation")
|
||||
if citation is not None:
|
||||
citations.append(citation)
|
||||
else:
|
||||
# A delta this reconstructor has no rule for: the
|
||||
# accumulated block is missing whatever it carried.
|
||||
saw_unreplayable_delta = True
|
||||
|
||||
elif event_type == "content_block_stop":
|
||||
idx = data.get("index")
|
||||
target = (blocks_by_index.get(idx) if idx is not None else None) or current_block
|
||||
if target is not None:
|
||||
# Parse accumulated JSON into `input` for any block that
|
||||
# streamed `input_json_delta` — tool_use AND server_tool_use
|
||||
# (and future tool-ish blocks). Gating on the block type
|
||||
# missed server_tool_use, leaving its `input` at the empty
|
||||
# start-event value and leaking the `_partial_json` scratch
|
||||
# key into replayed assistant history, which Anthropic then
|
||||
# rejects with `server_tool_use.input: Input should be an
|
||||
# object` (#2438). Always strip the scratch key.
|
||||
if "_partial_json" in target:
|
||||
raw = target.pop("_partial_json")
|
||||
try:
|
||||
target["input"] = json.loads(raw) if raw else {}
|
||||
except json.JSONDecodeError:
|
||||
target["input"] = {}
|
||||
# Materialize the thinking buffer into the
|
||||
# canonical `thinking` field expected by the
|
||||
# Anthropic API.
|
||||
if target.get("type") == "thinking" and "thinking_buffer" in target:
|
||||
target["thinking"] = target.pop("thinking_buffer")
|
||||
# Append the block exactly once, keyed by its block
|
||||
# index (or object identity when no index was ever
|
||||
# assigned). `current_block` may not match the
|
||||
# indexed target if the stream interleaved multiple
|
||||
# blocks; index-keyed map is authoritative.
|
||||
block_key = idx if idx is not None else id(target)
|
||||
if block_key not in appended_block_keys:
|
||||
response["content"].append(target)
|
||||
appended_block_keys.add(block_key)
|
||||
if idx is not None:
|
||||
open_block_indices.discard(idx)
|
||||
current_block = None
|
||||
|
||||
elif event_type == "message_delta":
|
||||
delta = data.get("delta", {})
|
||||
if "stop_reason" in delta:
|
||||
response["stop_reason"] = delta["stop_reason"]
|
||||
if "stop_details" in delta:
|
||||
response["stop_details"] = delta["stop_details"]
|
||||
if "stop_sequence" in delta:
|
||||
response["stop_sequence"] = delta["stop_sequence"]
|
||||
if data.get("usage"):
|
||||
response["usage"].update(data["usage"])
|
||||
|
||||
elif event_type == "message_stop":
|
||||
saw_message_stop = True
|
||||
|
||||
elif event_type == "error":
|
||||
# An in-band failure. Permissive callers keep salvaging
|
||||
# what arrived before it; a strict caller must not dress
|
||||
# the remains up as a successful turn.
|
||||
saw_error = True
|
||||
|
||||
if require_complete:
|
||||
if (
|
||||
not saw_message_start
|
||||
or not saw_message_stop
|
||||
or saw_error
|
||||
or saw_unreplayable_delta
|
||||
or open_block_indices
|
||||
):
|
||||
envelope = AnthropicSSEEnvelope.parse(sse_data.encode("utf-8"))
|
||||
if require_complete and not envelope.is_message_reconstructable():
|
||||
return None
|
||||
# ``index`` is a response-delta field. Anthropic rejects it on
|
||||
# the next request ("content.0.text.index: Extra inputs are not
|
||||
# permitted"), so it must not survive into a body the client
|
||||
# will persist and echo back.
|
||||
for block in response["content"]:
|
||||
if isinstance(block, dict):
|
||||
block.pop("index", None)
|
||||
return response
|
||||
|
||||
return response if response.get("content") else None
|
||||
return (
|
||||
envelope.message
|
||||
if envelope.message.get("content") or envelope.saw_message_start
|
||||
else None
|
||||
)
|
||||
|
||||
def _response_to_sse(self, response: dict[str, Any], provider: str) -> list[bytes]:
|
||||
"""Convert a response dict back to SSE format.
|
||||
"""Convert a response dictionary back to provider SSE frames."""
|
||||
|
||||
Args:
|
||||
response: API response dict.
|
||||
provider: Provider type for formatting.
|
||||
|
||||
Returns:
|
||||
List of SSE event bytes.
|
||||
"""
|
||||
if provider != "anthropic":
|
||||
return []
|
||||
|
||||
events: list[bytes] = []
|
||||
from headroom.proxy.anthropic_wire import render_anthropic_sse_response
|
||||
|
||||
# message_start
|
||||
msg_start = {
|
||||
"type": "message_start",
|
||||
"message": {
|
||||
"id": response.get("id", "msg_generated"),
|
||||
"type": "message",
|
||||
"role": response.get("role", "assistant"),
|
||||
"model": response.get("model", "unknown"),
|
||||
"content": [],
|
||||
"stop_reason": None,
|
||||
"usage": response.get("usage", {}),
|
||||
},
|
||||
}
|
||||
events.append(f"event: message_start\ndata: {json.dumps(msg_start)}\n\n".encode())
|
||||
|
||||
# Content blocks. `content` is provider/reconstruction-controlled, so a
|
||||
# present-but-null value or a non-list would crash `enumerate`, and a
|
||||
# non-dict element would crash `block.get(...)`. Guard both, matching the
|
||||
# sibling `_record_ccr_feedback_from_response` below. This is reached from
|
||||
# a call site (anthropic.py buffered CCR path) that only catches
|
||||
# ValueError, so an unguarded TypeError/AttributeError would 500 the
|
||||
# streamed request.
|
||||
content = response.get("content")
|
||||
for idx, block in enumerate(content if isinstance(content, list) else []):
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
# content_block_start
|
||||
if block.get("type") == "text":
|
||||
block_start = {
|
||||
"type": "content_block_start",
|
||||
"index": idx,
|
||||
"content_block": {"type": "text", "text": ""},
|
||||
}
|
||||
elif block.get("type") == "tool_use":
|
||||
block_start = {
|
||||
"type": "content_block_start",
|
||||
"index": idx,
|
||||
"content_block": {
|
||||
"type": "tool_use",
|
||||
"id": block.get("id", f"toolu_{idx}"),
|
||||
"name": block.get("name", ""),
|
||||
"input": {},
|
||||
},
|
||||
}
|
||||
elif block.get("type") == "thinking":
|
||||
content_block = {
|
||||
"type": "thinking",
|
||||
"thinking": "",
|
||||
}
|
||||
if "signature" in block:
|
||||
content_block["signature"] = block["signature"]
|
||||
block_start = {
|
||||
"type": "content_block_start",
|
||||
"index": idx,
|
||||
"content_block": content_block,
|
||||
}
|
||||
elif block.get("type") == "redacted_thinking":
|
||||
block_start = {
|
||||
"type": "content_block_start",
|
||||
"index": idx,
|
||||
"content_block": {
|
||||
"type": "redacted_thinking",
|
||||
"data": block.get("data", ""),
|
||||
},
|
||||
}
|
||||
elif block.get("type") == "server_tool_use":
|
||||
block_start = {
|
||||
"type": "content_block_start",
|
||||
"index": idx,
|
||||
"content_block": block,
|
||||
}
|
||||
else:
|
||||
block_start = {
|
||||
"type": "content_block_start",
|
||||
"index": idx,
|
||||
"content_block": block,
|
||||
}
|
||||
|
||||
events.append(
|
||||
f"event: content_block_start\ndata: {json.dumps(block_start)}\n\n".encode()
|
||||
)
|
||||
|
||||
# content_block_delta(s)
|
||||
if block.get("type") == "text" and block.get("text"):
|
||||
delta = {
|
||||
"type": "content_block_delta",
|
||||
"index": idx,
|
||||
"delta": {"type": "text_delta", "text": block["text"]},
|
||||
}
|
||||
events.append(f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode())
|
||||
for citation in block.get("citations", []) or []:
|
||||
citation_delta = {
|
||||
"type": "content_block_delta",
|
||||
"index": idx,
|
||||
"delta": {"type": "citations_delta", "citation": citation},
|
||||
}
|
||||
events.append(
|
||||
f"event: content_block_delta\ndata: {json.dumps(citation_delta)}\n\n".encode()
|
||||
)
|
||||
elif block.get("type") == "tool_use" and block.get("input"):
|
||||
delta = {
|
||||
"type": "content_block_delta",
|
||||
"index": idx,
|
||||
"delta": {
|
||||
"type": "input_json_delta",
|
||||
"partial_json": json.dumps(block["input"]),
|
||||
},
|
||||
}
|
||||
events.append(f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode())
|
||||
elif block.get("type") == "thinking":
|
||||
if block.get("thinking"):
|
||||
delta = {
|
||||
"type": "content_block_delta",
|
||||
"index": idx,
|
||||
"delta": {"type": "thinking_delta", "thinking": block["thinking"]},
|
||||
}
|
||||
events.append(
|
||||
f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode()
|
||||
)
|
||||
if block.get("signature"):
|
||||
delta = {
|
||||
"type": "content_block_delta",
|
||||
"index": idx,
|
||||
"delta": {"type": "signature_delta", "signature": block["signature"]},
|
||||
}
|
||||
events.append(
|
||||
f"event: content_block_delta\ndata: {json.dumps(delta)}\n\n".encode()
|
||||
)
|
||||
|
||||
# content_block_stop
|
||||
block_stop = {"type": "content_block_stop", "index": idx}
|
||||
events.append(f"event: content_block_stop\ndata: {json.dumps(block_stop)}\n\n".encode())
|
||||
|
||||
# message_delta
|
||||
msg_delta_payload: dict[str, Any] = {}
|
||||
if "stop_reason" in response:
|
||||
msg_delta_payload["stop_reason"] = response["stop_reason"]
|
||||
if "stop_details" in response:
|
||||
msg_delta_payload["stop_details"] = response["stop_details"]
|
||||
usage = response.get("usage")
|
||||
if not isinstance(usage, dict):
|
||||
usage = {}
|
||||
msg_delta = {
|
||||
"type": "message_delta",
|
||||
"delta": msg_delta_payload,
|
||||
"usage": {"output_tokens": usage.get("output_tokens", 0)},
|
||||
}
|
||||
events.append(f"event: message_delta\ndata: {json.dumps(msg_delta)}\n\n".encode())
|
||||
|
||||
# message_stop
|
||||
events.append(b'event: message_stop\ndata: {"type": "message_stop"}\n\n')
|
||||
|
||||
return events
|
||||
return render_anthropic_sse_response(response)
|
||||
|
||||
def _record_ccr_feedback_from_response(
|
||||
self, response: dict, provider: str, request_id: str
|
||||
@@ -1248,6 +886,7 @@ class StreamingMixin:
|
||||
# bytes once before entering the connection-retry loop. When a
|
||||
# transform mutated the body we re-serialize canonically; otherwise
|
||||
# we forward the original client bytes verbatim.
|
||||
from headroom.proxy.anthropic_wire import is_safeguard_capable_request
|
||||
from headroom.proxy.body_forwarding import select_outbound_body
|
||||
from headroom.proxy.helpers import (
|
||||
capture_codex_wire_debug,
|
||||
@@ -1274,8 +913,14 @@ class StreamingMixin:
|
||||
source=outbound_source,
|
||||
dropped_mutation_reasons=outbound.dropped_mutation_reasons,
|
||||
)
|
||||
safeguard_capable = provider == "anthropic" and is_safeguard_capable_request(
|
||||
body, headers.get("anthropic-beta")
|
||||
)
|
||||
_codex_wire_debug = (
|
||||
codex_wire_debug_enabled() and provider == "openai" and "/responses" in url
|
||||
codex_wire_debug_enabled()
|
||||
and not safeguard_capable
|
||||
and provider == "openai"
|
||||
and "/responses" in url
|
||||
)
|
||||
if _codex_wire_debug:
|
||||
capture_codex_wire_debug(
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_nested_001","type":"message","role":"assistant","model":"claude-test-20260924","content":[],"stop_reason":null,"usage":{"input_tokens":12,"output_tokens":1}},"x_message_start":{"sentinel":"message_start"}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""},"x_block_start":{"sentinel":"block_start"}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Checking.","x_text_delta":{"sentinel":"text_delta"}},"x_delta_event":{"sentinel":"delta_event"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0,"x_block_stop":{"sentinel":"block_stop"}}
|
||||
|
||||
event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null,"x_message_delta_delta":{"sentinel":"message_delta_delta"}},"usage":{"output_tokens":3,"cache_read_input_tokens":7,"x_usage":{"sentinel":"usage"}}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop","x_message_stop":{"sentinel":"message_stop"}}
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"model": "claude-test-20260924",
|
||||
"max_tokens": 128,
|
||||
"stream": true,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Run the proposed action."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"name": "run_action",
|
||||
"description": "Run a sanitized test action.",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"target": {"type": "string"}
|
||||
},
|
||||
"required": ["target"]
|
||||
}
|
||||
}
|
||||
],
|
||||
"safeguards": {
|
||||
"mode": "auto",
|
||||
"tool_use_ids": ["toolu_auto_001"]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"id": "msg_auto_001",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-test-20260924",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": "toolu_auto_001",
|
||||
"name": "run_action",
|
||||
"input": {"target": "sanitized-target"}
|
||||
}
|
||||
],
|
||||
"stop_reason": "tool_use",
|
||||
"usage": {"input_tokens": 12, "output_tokens": 8},
|
||||
"safeguard_results": {
|
||||
"decision": "allow",
|
||||
"tool_use_id": "toolu_auto_001"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_auto_001","type":"message","role":"assistant","model":"claude-test-20260924","content":[],"usage":{"input_tokens":12},"safeguard_results":{"decision":"allow","tool_use_id":"toolu_auto_001"}}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"run_action","input":{}}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"target\":\"sanitized-target\"}"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: safeguard_results
|
||||
data: {"type":"safeguard_results","result":{"decision":"allow","tool_use_id":"toolu_auto_001"}}
|
||||
|
||||
event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"},"usage":{"output_tokens":8}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
@@ -0,0 +1,484 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from headroom.proxy.anthropic_wire import (
|
||||
AnthropicSSEEnvelope,
|
||||
build_anthropic_upstream_url,
|
||||
has_dangerous_tool_use_beta,
|
||||
is_safeguard_capable_request,
|
||||
strip_safeguard_payload,
|
||||
)
|
||||
from headroom.proxy.handlers.streaming import StreamingMixin
|
||||
from headroom.proxy.server import ProxyConfig, create_app
|
||||
|
||||
FIXTURES = Path(__file__).parent / "fixtures" / "anthropic"
|
||||
|
||||
|
||||
def _fixture(name: str) -> bytes:
|
||||
return (FIXTURES / name).read_bytes()
|
||||
|
||||
|
||||
def test_native_url_preserves_path_and_raw_query() -> None:
|
||||
assert (
|
||||
build_anthropic_upstream_url(
|
||||
"https://api.anthropic.com", "/v1/messages", "beta=true&beta=false"
|
||||
)
|
||||
== "https://api.anthropic.com/v1/messages?beta=true&beta=false"
|
||||
)
|
||||
|
||||
|
||||
def test_classifier_request_detection_does_not_need_safeguard_schema() -> None:
|
||||
request = json.loads(_fixture("claude_code_auto_mode_request.json"))
|
||||
assert is_safeguard_capable_request(request, "prompt-caching-2024-07-31")
|
||||
assert has_dangerous_tool_use_beta("prompt-caching-2024-07-31,dangerous-tool-use-2026-09")
|
||||
assert not is_safeguard_capable_request({}, "prompt-caching-2024-07-31")
|
||||
|
||||
|
||||
def test_sse_round_trip_preserves_opaque_result_order_and_ids() -> None:
|
||||
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_stream.sse"))
|
||||
assert envelope.message["safeguard_results"]["decision"] == "allow"
|
||||
assert envelope.message["content"][0]["id"] == "toolu_auto_001"
|
||||
|
||||
rendered = b"".join(envelope.render(envelope.message))
|
||||
safeguard_offset = rendered.index(b"event: safeguard_results")
|
||||
delta_offset = rendered.index(b"event: message_delta")
|
||||
stop_offset = rendered.index(b"event: message_stop")
|
||||
assert safeguard_offset < delta_offset < stop_offset
|
||||
assert b'"id": "toolu_auto_001"' in rendered
|
||||
|
||||
|
||||
def test_sse_round_trip_keeps_results_across_a_ccr_continuation() -> None:
|
||||
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_stream.sse"))
|
||||
continuation = {
|
||||
"id": "msg_continuation_001",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-test-20260924",
|
||||
"content": [{"type": "text", "text": "retrieval complete"}],
|
||||
"stop_reason": "end_turn",
|
||||
"usage": {"input_tokens": 20, "output_tokens": 4},
|
||||
}
|
||||
|
||||
rendered = b"".join(envelope.render(continuation))
|
||||
|
||||
assert b"safeguard_results" in rendered
|
||||
assert b"toolu_auto_001" in rendered
|
||||
assert b"msg_continuation_001" in rendered
|
||||
|
||||
|
||||
def _rendered_events(chunks: list[bytes]) -> list[dict]:
|
||||
return [
|
||||
json.loads(line[len("data: ") :])
|
||||
for line in b"".join(chunks).decode("utf-8").splitlines()
|
||||
if line.startswith("data: ")
|
||||
]
|
||||
|
||||
|
||||
def test_nested_extensions_on_known_frames_survive_round_trip() -> None:
|
||||
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_nested_extensions.sse"))
|
||||
events = {event["type"]: event for event in _rendered_events(envelope.render())}
|
||||
|
||||
assert envelope.is_message_reconstructable()
|
||||
assert events["message_start"]["x_message_start"] == {"sentinel": "message_start"}
|
||||
assert events["content_block_start"]["x_block_start"] == {"sentinel": "block_start"}
|
||||
assert events["content_block_delta"]["x_delta_event"] == {"sentinel": "delta_event"}
|
||||
assert events["content_block_delta"]["delta"] == {
|
||||
"type": "text_delta",
|
||||
"text": "Checking.",
|
||||
"x_text_delta": {"sentinel": "text_delta"},
|
||||
}
|
||||
assert events["content_block_stop"]["x_block_stop"] == {"sentinel": "block_stop"}
|
||||
assert events["message_delta"]["delta"]["x_message_delta_delta"] == {
|
||||
"sentinel": "message_delta_delta"
|
||||
}
|
||||
assert events["message_delta"]["usage"]["cache_read_input_tokens"] == 7
|
||||
assert events["message_delta"]["usage"]["x_usage"] == {"sentinel": "usage"}
|
||||
assert events["message_stop"]["x_message_stop"] == {"sentinel": "message_stop"}
|
||||
|
||||
|
||||
def test_block_extensions_do_not_attach_to_replacement_blocks() -> None:
|
||||
envelope = AnthropicSSEEnvelope.parse(_fixture("claude_code_auto_mode_nested_extensions.sse"))
|
||||
continuation = {
|
||||
"id": "msg_continuation_002",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-test-20260924",
|
||||
"content": [{"type": "text", "text": "retrieval complete"}],
|
||||
"stop_reason": "end_turn",
|
||||
"usage": {"input_tokens": 20, "output_tokens": 4},
|
||||
}
|
||||
|
||||
rendered = b"".join(envelope.render(continuation))
|
||||
events = {event["type"]: event for event in _rendered_events([rendered])}
|
||||
|
||||
# Message-level members belong to the protocol and survive the swap.
|
||||
assert events["message_start"]["x_message_start"] == {"sentinel": "message_start"}
|
||||
assert "x_message_delta_delta" in events["message_delta"]["delta"]
|
||||
assert events["message_stop"]["x_message_stop"] == {"sentinel": "message_stop"}
|
||||
# Members of the replaced block's frames do not describe the new block.
|
||||
for sentinel in (b"x_block_start", b"x_delta_event", b"x_text_delta", b"x_block_stop"):
|
||||
assert sentinel not in rendered
|
||||
assert b"retrieval complete" in rendered
|
||||
|
||||
|
||||
def test_unknown_content_delta_is_complete_and_replayed_verbatim() -> None:
|
||||
raw = b"""event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_future","type":"message","role":"assistant","content":[]}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"future_delta","value":"opaque"}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
"""
|
||||
|
||||
envelope = AnthropicSSEEnvelope.parse(raw)
|
||||
|
||||
assert envelope.is_complete()
|
||||
assert not envelope.is_message_reconstructable()
|
||||
assert b'"type":"future_delta"' in b"".join(envelope.render())
|
||||
|
||||
|
||||
def test_malformed_utf8_delta_is_replayed_without_replacement() -> None:
|
||||
raw = b"""event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_bad_utf8","type":"message","role":"assistant","content":[]}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\xff"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
"""
|
||||
|
||||
envelope = AnthropicSSEEnvelope.parse(raw)
|
||||
rendered = b"".join(envelope.render())
|
||||
|
||||
assert envelope.is_complete()
|
||||
assert not envelope.is_message_reconstructable()
|
||||
assert b"\xff" in rendered
|
||||
assert b"\xef\xbf\xbd" not in rendered
|
||||
|
||||
|
||||
_MALFORMED_KNOWN_FRAMES = {
|
||||
"invalid-json-message-delta": b"event: message_delta\ndata: {not-json\n\n",
|
||||
"invalid-utf8-block-start": (
|
||||
b"event: content_block_start\n"
|
||||
b'data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":"\xff"}}\n\n'
|
||||
),
|
||||
"non-object-content-block": (
|
||||
b"event: content_block_start\n"
|
||||
b'data: {"type":"content_block_start","index":1,"content_block":"future"}\n\n'
|
||||
),
|
||||
"delta-for-unopened-block": (
|
||||
b"event: content_block_delta\n"
|
||||
b'data: {"type":"content_block_delta","index":9,"delta":{"type":"text_delta","text":"x"}}\n\n'
|
||||
),
|
||||
"stop-for-unopened-block": (
|
||||
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":9}\n\n'
|
||||
),
|
||||
# ``index`` and ``delta.type`` are provider JSON and may be any JSON value;
|
||||
# a non-scalar one must not reach a dictionary lookup.
|
||||
"object-index-block-start": (
|
||||
b"event: content_block_start\n"
|
||||
b'data: {"type":"content_block_start","index":{"i":1},"content_block":{"type":"text","text":""}}\n\n'
|
||||
),
|
||||
"object-index-delta": (
|
||||
b"event: content_block_delta\n"
|
||||
b'data: {"type":"content_block_delta","index":{"i":0},"delta":{"type":"text_delta","text":"x"}}\n\n'
|
||||
),
|
||||
"array-index-delta": (
|
||||
b"event: content_block_delta\n"
|
||||
b'data: {"type":"content_block_delta","index":[0],"delta":{"type":"text_delta","text":"x"}}\n\n'
|
||||
),
|
||||
"array-delta-type": (
|
||||
b"event: content_block_delta\n"
|
||||
b'data: {"type":"content_block_delta","index":0,"delta":{"type":["text_delta"],"text":"x"}}\n\n'
|
||||
),
|
||||
"non-string-delta-text": (
|
||||
b"event: content_block_delta\n"
|
||||
b'data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":{"t":"x"}}}\n\n'
|
||||
),
|
||||
"object-index-stop": (
|
||||
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":{"i":0}}\n\n'
|
||||
),
|
||||
"array-index-stop": (
|
||||
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":[0]}\n\n'
|
||||
),
|
||||
"array-usage-message-delta": (
|
||||
b'event: message_delta\ndata: {"type":"message_delta","delta":{},"usage":[1]}\n\n'
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_frame", _MALFORMED_KNOWN_FRAMES.values(), ids=_MALFORMED_KNOWN_FRAMES)
|
||||
def test_malformed_known_frame_is_replayed_verbatim(bad_frame: bytes) -> None:
|
||||
raw = (
|
||||
b"""event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_bad_known","type":"message","role":"assistant","content":[]}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hi"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
"""
|
||||
+ bad_frame
|
||||
+ b"""event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":1}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
"""
|
||||
)
|
||||
|
||||
envelope = AnthropicSSEEnvelope.parse(raw)
|
||||
rendered = b"".join(envelope.render())
|
||||
|
||||
assert envelope.is_complete()
|
||||
assert not envelope.is_message_reconstructable()
|
||||
assert rendered.count(bad_frame) == 1
|
||||
assert rendered.index(b"event: content_block_stop") < rendered.index(bad_frame)
|
||||
assert rendered.index(bad_frame) < rendered.rindex(b"event: message_delta")
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad_frames",
|
||||
[
|
||||
pytest.param(
|
||||
b"event: content_block_delta\n"
|
||||
b'data: {"type":"content_block_delta","index":{"i":0},"delta":{"type":"text_delta","text":"x"}}\n\n'
|
||||
b"event: content_block_delta\n"
|
||||
b'data: {"type":"content_block_delta","index":0,"delta":{"type":["text_delta"],"text":"x"}}\n\n',
|
||||
id="delta",
|
||||
),
|
||||
pytest.param(
|
||||
b"event: content_block_stop\n"
|
||||
b'data: {"type":"content_block_stop","index":{"i":0}}\n\n'
|
||||
b'event: content_block_stop\ndata: {"type":"content_block_stop","index":[0]}\n\n',
|
||||
id="stop",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_invalid_discriminators_on_an_open_block_do_not_break_the_stream(
|
||||
bad_frames: bytes,
|
||||
) -> None:
|
||||
raw = (
|
||||
b"""event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_open","type":"message","role":"assistant","content":[]}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hel"}}
|
||||
|
||||
"""
|
||||
+ bad_frames
|
||||
+ b"""event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"lo"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
"""
|
||||
)
|
||||
|
||||
envelope = AnthropicSSEEnvelope.parse(raw)
|
||||
rendered = b"".join(envelope.render())
|
||||
|
||||
# The valid frames around the invalid ones still reconstruct the block.
|
||||
assert envelope.message["content"] == [{"type": "text", "text": "hello"}]
|
||||
assert envelope.is_complete()
|
||||
assert not envelope.is_message_reconstructable()
|
||||
assert rendered.count(bad_frames) == 1
|
||||
|
||||
|
||||
_DELTA_PAYLOAD_CASES = {
|
||||
"text_delta": ("text", {"type": "text", "text": ""}),
|
||||
"thinking_delta": ("thinking", {"type": "thinking", "thinking": ""}),
|
||||
"input_json_delta": (
|
||||
"partial_json",
|
||||
{"type": "tool_use", "id": "toolu_1", "name": "run", "input": {}},
|
||||
),
|
||||
"signature_delta": ("signature", {"type": "thinking", "thinking": ""}),
|
||||
"citations_delta": ("citation", {"type": "text", "text": ""}),
|
||||
}
|
||||
_MISSING = object()
|
||||
|
||||
|
||||
def _single_delta_stream(block: dict, delta: dict) -> tuple[bytes, bytes]:
|
||||
def event(payload: dict) -> bytes:
|
||||
return f"event: {payload['type']}\ndata: {json.dumps(payload)}\n\n".encode()
|
||||
|
||||
frame = event({"type": "content_block_delta", "index": 0, "delta": delta})
|
||||
raw = (
|
||||
event(
|
||||
{
|
||||
"type": "message_start",
|
||||
"message": {"id": "msg_d", "type": "message", "role": "assistant", "content": []},
|
||||
}
|
||||
)
|
||||
+ event({"type": "content_block_start", "index": 0, "content_block": block})
|
||||
+ frame
|
||||
+ event({"type": "content_block_stop", "index": 0})
|
||||
+ event({"type": "message_stop"})
|
||||
)
|
||||
return raw, frame
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value", [0, False, None, _MISSING], ids=["zero", "false", "null", "missing"]
|
||||
)
|
||||
@pytest.mark.parametrize("delta_type", _DELTA_PAYLOAD_CASES)
|
||||
def test_falsy_non_string_delta_payload_is_replayed_not_consumed(
|
||||
delta_type: str, value: object
|
||||
) -> None:
|
||||
member, block = _DELTA_PAYLOAD_CASES[delta_type]
|
||||
delta = {"type": delta_type} if value is _MISSING else {"type": delta_type, member: value}
|
||||
raw, frame = _single_delta_stream(block, delta)
|
||||
|
||||
envelope = AnthropicSSEEnvelope.parse(raw)
|
||||
|
||||
assert envelope.is_complete()
|
||||
assert not envelope.is_message_reconstructable()
|
||||
assert b"".join(envelope.render()).count(frame) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("delta_type", ["text_delta", "thinking_delta", "input_json_delta"])
|
||||
def test_empty_string_delta_payload_is_still_reconstructed(delta_type: str) -> None:
|
||||
member, block = _DELTA_PAYLOAD_CASES[delta_type]
|
||||
raw, frame = _single_delta_stream(block, {"type": delta_type, member: ""})
|
||||
|
||||
envelope = AnthropicSSEEnvelope.parse(raw)
|
||||
|
||||
assert envelope.is_message_reconstructable()
|
||||
assert frame not in b"".join(envelope.render())
|
||||
|
||||
|
||||
def test_non_object_message_usage_is_replayed_not_merged() -> None:
|
||||
bad_start = (
|
||||
b"event: message_start\n"
|
||||
b'data: {"type":"message_start","message":{"id":"msg_usage","type":"message",'
|
||||
b'"role":"assistant","content":[],"usage":[12]}}\n\n'
|
||||
)
|
||||
raw = (
|
||||
bad_start
|
||||
+ b"""event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":1}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
"""
|
||||
)
|
||||
|
||||
envelope = AnthropicSSEEnvelope.parse(raw)
|
||||
rendered = b"".join(envelope.render())
|
||||
|
||||
assert not envelope.is_message_reconstructable()
|
||||
assert rendered.count(bad_start) == 1
|
||||
assert envelope.message["usage"] == {"output_tokens": 1}
|
||||
|
||||
|
||||
def test_non_stream_response_keeps_unknown_top_level_fields() -> None:
|
||||
response = json.loads(_fixture("claude_code_auto_mode_response.json"))
|
||||
rendered = b"".join(StreamingMixin()._response_to_sse(response, "anthropic"))
|
||||
assert b"safeguard_results" in rendered
|
||||
assert b"toolu_auto_001" in rendered
|
||||
|
||||
|
||||
def test_observation_copy_removes_classifier_payloads() -> None:
|
||||
value = {
|
||||
"safeguards": {"sentinel": "do-not-log"},
|
||||
"nested": [{"safeguard_results": {"sentinel": "do-not-log"}}],
|
||||
"safe": "kept",
|
||||
}
|
||||
observed = strip_safeguard_payload(value)
|
||||
assert observed == {"nested": [{}], "safe": "kept"}
|
||||
assert value["safeguards"]["sentinel"] == "do-not-log"
|
||||
|
||||
|
||||
class _AutoModeFixtureStream(httpx.AsyncByteStream):
|
||||
async def __aiter__(self):
|
||||
yield _fixture("claude_code_auto_mode_stream.sse")
|
||||
|
||||
|
||||
class _CapturingAnthropicTransport(httpx.AsyncBaseTransport):
|
||||
def __init__(self) -> None:
|
||||
self.url: str | None = None
|
||||
self.headers: dict[str, str] | None = None
|
||||
self.body: bytes | None = None
|
||||
|
||||
async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
|
||||
self.url = str(request.url)
|
||||
self.headers = dict(request.headers.items())
|
||||
self.body = b"".join([chunk async for chunk in request.stream])
|
||||
return httpx.Response(
|
||||
200,
|
||||
headers={"content-type": "text/event-stream"},
|
||||
stream=_AutoModeFixtureStream(),
|
||||
)
|
||||
|
||||
|
||||
def test_native_handler_preserves_classifier_request_envelope() -> None:
|
||||
config = ProxyConfig(
|
||||
optimize=False,
|
||||
cache_enabled=False,
|
||||
rate_limit_enabled=False,
|
||||
cost_tracking_enabled=False,
|
||||
log_requests=False,
|
||||
ccr_inject_tool=False,
|
||||
ccr_handle_responses=False,
|
||||
ccr_context_tracking=False,
|
||||
image_optimize=False,
|
||||
)
|
||||
app = create_app(config)
|
||||
proxy = app.state.proxy
|
||||
transport = _CapturingAnthropicTransport()
|
||||
proxy.http_client = httpx.AsyncClient(transport=transport)
|
||||
request = json.loads(_fixture("claude_code_auto_mode_request.json"))
|
||||
request["model"] = "claude-sonnet-4-6"
|
||||
|
||||
with TestClient(app).stream(
|
||||
"POST",
|
||||
"/v1/messages?beta=true",
|
||||
headers={
|
||||
"x-api-key": "test-key",
|
||||
"anthropic-version": "2023-06-01",
|
||||
"anthropic-beta": "claude-code-20250219,dangerous-tool-use-2026-09-03",
|
||||
"content-type": "application/json",
|
||||
},
|
||||
content=json.dumps(request).encode(),
|
||||
) as response:
|
||||
assert response.status_code == 200
|
||||
assert b"safeguard_results" in b"".join(response.iter_bytes())
|
||||
|
||||
assert transport.url == "https://api.anthropic.com/v1/messages?beta=true"
|
||||
assert transport.headers is not None
|
||||
assert (
|
||||
transport.headers["anthropic-beta"] == "claude-code-20250219,dangerous-tool-use-2026-09-03"
|
||||
)
|
||||
assert transport.body is not None
|
||||
assert json.loads(transport.body)["safeguards"] == request["safeguards"]
|
||||
@@ -433,6 +433,117 @@ async def test_streaming_handler_process_stream_pass_through_and_ccr(
|
||||
assert streamed == [b"event: message_start\n", b"event: message_stop\n"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_streaming_ccr_keeps_classifier_frames_through_continuation(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
response_handler = CCRResponseHandler()
|
||||
handler = StreamingCCRHandler(response_handler, provider="anthropic")
|
||||
|
||||
async def fake_handle_response(response, messages, tools, api_call_fn, provider): # noqa: ANN001
|
||||
return {
|
||||
"id": "msg_continuation",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-test",
|
||||
"content": [{"type": "text", "text": "retrieval complete"}],
|
||||
"stop_reason": "end_turn",
|
||||
"usage": {"input_tokens": 20, "output_tokens": 4},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(response_handler, "handle_response", fake_handle_response)
|
||||
initial = b"""event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_initial","type":"message","role":"assistant","model":"claude-test","content":[],"safeguard_results":{"decision":"allow","tool_use_id":"toolu_auto_001"}}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"headroom_retrieve","input":{}}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\\"hash\\":\\"abc\\"}"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: safeguard_results
|
||||
data: {"type":"safeguard_results","result":{"decision":"allow","tool_use_id":"toolu_auto_001"}}
|
||||
|
||||
event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
"""
|
||||
|
||||
streamed = [
|
||||
chunk
|
||||
async for chunk in handler.process_stream(
|
||||
_async_iter([initial]), [], None, lambda m, t: None
|
||||
)
|
||||
]
|
||||
rendered = b"".join(streamed)
|
||||
|
||||
assert b"safeguard_results" in rendered
|
||||
assert b"toolu_auto_001" in rendered
|
||||
assert b"msg_continuation" in rendered
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_streaming_ccr_replays_malformed_known_frame_through_continuation(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
response_handler = CCRResponseHandler()
|
||||
handler = StreamingCCRHandler(response_handler, provider="anthropic")
|
||||
|
||||
async def fake_handle_response(response, messages, tools, api_call_fn, provider): # noqa: ANN001
|
||||
return {
|
||||
"id": "msg_continuation",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-test",
|
||||
"content": [{"type": "text", "text": "retrieval complete"}],
|
||||
"stop_reason": "end_turn",
|
||||
"usage": {"input_tokens": 20, "output_tokens": 4},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(response_handler, "handle_response", fake_handle_response)
|
||||
malformed = b"event: message_delta\ndata: {not-json\n\n"
|
||||
initial = (
|
||||
b"""event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_initial","type":"message","role":"assistant","model":"claude-test","content":[]}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_auto_001","name":"headroom_retrieve","input":{}}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\\"hash\\":\\"abc\\"}"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
"""
|
||||
+ malformed
|
||||
+ b"""event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"tool_use"}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
|
||||
"""
|
||||
)
|
||||
|
||||
streamed = [
|
||||
chunk
|
||||
async for chunk in handler.process_stream(
|
||||
_async_iter([initial]), [], None, lambda m, t: None
|
||||
)
|
||||
]
|
||||
rendered = b"".join(streamed)
|
||||
|
||||
assert b"msg_continuation" in rendered
|
||||
assert rendered.count(malformed) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_streaming_handler_falls_back_to_buffer_on_processing_error(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
|
||||
Reference in New Issue
Block a user