mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): carry safeguard_results through SSE resynthesis (#3958)
## Description When the buffered streaming-CCR path flips a `stream: true` request to `stream: false` and re-renders the JSON reply as SSE, `safeguard_results` is emitted only inside `message_start.message`. Claude Code reads the server-side auto-mode classifier verdict from `message_delta.delta.safeguard_results`. When it is absent there on a completed turn, Claude Code concludes the server did not review the action and falls back to client-side classifier requests (separately billed) for the rest of the session. This adds `safeguard_results` to `_MESSAGE_DELTA_FIELDS` in `anthropic_wire.py`, so the canonical renderer also places it in `message_delta.delta`, and the SSE parser lifts it from there. Byte-faithful passthrough streams already preserved the field through `message_delta_delta` extensions; only the JSON-to-SSE resynthesis path dropped it. Follow-up to #3784. Its fixture (`claude_code_auto_mode_stream.sse`) carries the verdict in `message_start.message` and a standalone `safeguard_results` event. In real Claude Code auto-mode traffic captured against `api.anthropic.com`, the verdict arrived in `message_delta.delta` on streams and at top level on non-streaming JSON, and stripping it from `message_delta.delta` reproduced the client-side fallback (4 of 4 attempts). Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/anthropic_wire.py`: add `safeguard_results` to `_MESSAGE_DELTA_FIELDS`. - `tests/test_sse_safeguards_passthrough.py`: 8 tests covering rendering (list, `null`, absent), `stop_reason`/`stop_sequence` alongside the verdict, parsing from both `message_start` and `message_delta`, and a parse-render round trip. ## Testing - [x] Unit tests pass (`pytest`) — targeted suites below; full suite not run - [x] Linting passes (`ruff check .`) — on changed files - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_sse_safeguards_passthrough.py tests/test_anthropic_auto_mode_passthrough.py \ tests/test_anthropic_buffered_sse.py tests/test_sse_thinking_blocks.py tests/test_observed_wire_shapes.py 110 passed # Without the fix, on main: $ pytest tests/test_sse_safeguards_passthrough.py 5 failed, 3 passed $ ruff check headroom/proxy/anthropic_wire.py tests/test_sse_safeguards_passthrough.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS 26 (arm64), Python 3.12, this branch installed with `pip install -e '.[proxy]'` (Rust core built from this tree). `headroom proxy --mode token --no-cache --anthropic-api-url http://127.0.0.1:18901`, all other settings default. Upstream is a local scripted Anthropic endpoint returning the response shape captured from `api.anthropic.com` for auto-mode turns (non-streaming JSON with top-level `safeguard_results` as a list). - Exact command / steps: send a Claude Code-shaped `stream: true` request carrying `safeguards`, `anthropic-beta: dangerous-tool-use-2026-09-03`, and a 400-row JSON tool result. The proxy compresses the tool result with a CCR marker, injects `headroom_retrieve`, and takes the buffered path. Ran once on this branch and once with `anthropic_wire.py` reverted to `main`: ```text curl -sN http://127.0.0.1:18902/v1/messages -H 'content-type: application/json' \ -H 'x-api-key: test-key' -H 'anthropic-version: 2023-06-01' \ -H 'anthropic-beta: dangerous-tool-use-2026-09-03' -d @request.json ``` - Observed result: Both runs took the buffered path (client sent `stream: true`, upstream received `stream: false` with a CCR marker); main omitted the verdict from message_delta and this patch preserved it, as captured below: ```text proxy: CCR: stream:true request has headroom_retrieve available; using buffered stream:false upstream request for server-side retrieval handling upstream received: {"stream": false, "has_safeguards": true, "tools": ["Bash", "headroom_retrieve"], "ccr_marker_in_body": true} # main — verdict missing from message_delta data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence": null}, "usage": {"output_tokens": 14}} # this branch — verdict present where Claude Code reads it data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence": null, "safeguard_results": [{"type": "dangerous_tool_use", "decision": "allow", "tool_use_id": "toolu_1"}]}, "usage": {"output_tokens": 14}} ``` On both runs `message_start.message` already carried `safeguard_results`; only the `message_delta` frame differs. - Prior real-traffic evidence (motivation): with Claude Code headless auto mode (`CLAUDE_CODE_AUTO_MODE_SERVER=1`) through a Headroom proxy to `api.anthropic.com`, verdicts arrive in `message_delta.delta.safeguard_results` on streams, and removing them from that frame triggers Claude Code's client-side classifier fallback (4/4). - Not tested: a live `api.anthropic.com` turn that enters the buffered CCR path with this patch, and Claude Code's client-side reaction on that path; the upstream above is scripted. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: re-rendered Anthropic SSE now includes `safeguard_results` in `message_delta.delta` when the response carries it; no change for responses without the field - Kill switch / disable path: none needed - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes Documentation: no user-facing behavior or configuration changes. Type checking was not run; the change is a tuple constant. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Sandeep Thuthike <288501+sunboy@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Sandeep Thuthike
Claude Opus 5.5
JD Davis
parent
fee53b4daa
commit
f37ef59189
@@ -42,7 +42,9 @@ _DELTA_FIELDS: dict[str, frozenset[str]] = {
|
||||
"signature_delta": frozenset({"type", "signature"}),
|
||||
"citations_delta": frozenset({"type", "citation"}),
|
||||
}
|
||||
_MESSAGE_DELTA_FIELDS = ("stop_reason", "stop_sequence", "stop_details")
|
||||
# Claude Code reads auto-mode classifier verdicts from message_delta.delta.safeguard_results;
|
||||
# a resynthesized stream without them falls back to billed client-side classification.
|
||||
_MESSAGE_DELTA_FIELDS = ("stop_reason", "stop_sequence", "stop_details", "safeguard_results")
|
||||
# The payload member each known delta type must carry, and its JSON type.
|
||||
_DELTA_PAYLOADS: dict[str, tuple[str, type]] = {
|
||||
"text_delta": ("text", str),
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
# tests/test_sse_safeguards_passthrough.py
|
||||
"""Claude Code reads auto-mode verdicts from message_delta.delta.safeguard_results
|
||||
(streaming) or top-level safeguard_results (JSON). Headroom's buffered-CCR path
|
||||
rebuilds SSE from JSON; the rebuild must carry the field or Claude Code falls
|
||||
back to a billed client-side classifier for the rest of the session."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.proxy.handlers.streaming import StreamingMixin
|
||||
|
||||
|
||||
class _Mixin(StreamingMixin):
|
||||
pass
|
||||
|
||||
|
||||
def _events(raw: list[bytes]) -> list[dict]:
|
||||
out = []
|
||||
for chunk in raw:
|
||||
for block in chunk.decode().split("\n\n"):
|
||||
for line in block.splitlines():
|
||||
if line.startswith("data: "):
|
||||
out.append(json.loads(line[6:]))
|
||||
return out
|
||||
|
||||
|
||||
def _by_type(events: list[dict], t: str) -> list[dict]:
|
||||
return [e for e in events if e.get("type") == t]
|
||||
|
||||
|
||||
_RESULTS = [{"type": "dangerous_tool_use", "verdict": "allow", "confidence": 0.99}]
|
||||
|
||||
|
||||
def _json_response(**extra):
|
||||
base = {
|
||||
"id": "msg_1",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-x",
|
||||
"content": [{"type": "text", "text": "ok"}],
|
||||
"stop_reason": "end_turn",
|
||||
"stop_sequence": None,
|
||||
"usage": {"input_tokens": 10, "output_tokens": 2},
|
||||
}
|
||||
base.update(extra)
|
||||
return base
|
||||
|
||||
|
||||
def test_response_to_sse_carries_safeguard_results_in_message_delta():
|
||||
ev = _events(_Mixin()._response_to_sse(_json_response(safeguard_results=_RESULTS), "anthropic"))
|
||||
(delta,) = _by_type(ev, "message_delta")
|
||||
assert delta["delta"]["safeguard_results"] == _RESULTS
|
||||
|
||||
|
||||
def test_response_to_sse_keeps_stop_reason_alongside_safeguard_results():
|
||||
ev = _events(_Mixin()._response_to_sse(_json_response(safeguard_results=_RESULTS), "anthropic"))
|
||||
(delta,) = _by_type(ev, "message_delta")
|
||||
assert delta["delta"]["stop_reason"] == "end_turn"
|
||||
assert delta["delta"]["safeguard_results"] == _RESULTS
|
||||
|
||||
|
||||
def test_response_to_sse_emits_null_safeguard_results():
|
||||
ev = _events(_Mixin()._response_to_sse(_json_response(safeguard_results=None), "anthropic"))
|
||||
(delta,) = _by_type(ev, "message_delta")
|
||||
assert "safeguard_results" in delta["delta"]
|
||||
assert delta["delta"]["safeguard_results"] is None
|
||||
|
||||
|
||||
def test_response_to_sse_omits_key_when_source_lacks_it():
|
||||
ev = _events(_Mixin()._response_to_sse(_json_response(), "anthropic"))
|
||||
(delta,) = _by_type(ev, "message_delta")
|
||||
assert "safeguard_results" not in delta["delta"]
|
||||
|
||||
|
||||
def test_response_to_sse_carries_stop_sequence():
|
||||
ev = _events(
|
||||
_Mixin()._response_to_sse(
|
||||
_json_response(stop_reason="stop_sequence", stop_sequence="END"), "anthropic"
|
||||
)
|
||||
)
|
||||
(delta,) = _by_type(ev, "message_delta")
|
||||
assert delta["delta"]["stop_sequence"] == "END"
|
||||
|
||||
|
||||
def _sse(*events: dict) -> bytes:
|
||||
return b"".join(f"event: {e['type']}\ndata: {json.dumps(e)}\n\n".encode() for e in events)
|
||||
|
||||
|
||||
def _stream_with_results_in(where: str) -> bytes:
|
||||
start = {
|
||||
"type": "message_start",
|
||||
"message": {
|
||||
"id": "msg_1",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-x",
|
||||
"content": [],
|
||||
"stop_reason": None,
|
||||
"usage": {"input_tokens": 10, "output_tokens": 0},
|
||||
},
|
||||
}
|
||||
delta = {
|
||||
"type": "message_delta",
|
||||
"delta": {"stop_reason": "end_turn"},
|
||||
"usage": {"output_tokens": 2},
|
||||
}
|
||||
if where == "delta":
|
||||
delta["delta"]["safeguard_results"] = _RESULTS
|
||||
else:
|
||||
start["message"]["safeguard_results"] = _RESULTS
|
||||
return _sse(
|
||||
start,
|
||||
{"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}},
|
||||
{"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "ok"}},
|
||||
{"type": "content_block_stop", "index": 0},
|
||||
delta,
|
||||
{"type": "message_stop"},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("where", ["delta", "start"])
|
||||
def test_parse_sse_to_response_lifts_safeguard_results(where):
|
||||
parsed = _Mixin()._parse_sse_to_response(_stream_with_results_in(where).decode(), "anthropic")
|
||||
assert parsed is not None
|
||||
assert parsed["safeguard_results"] == _RESULTS
|
||||
|
||||
|
||||
def test_round_trip_preserves_safeguard_results():
|
||||
m = _Mixin()
|
||||
parsed = m._parse_sse_to_response(_stream_with_results_in("delta").decode(), "anthropic")
|
||||
ev = _events(m._response_to_sse(parsed, "anthropic"))
|
||||
(delta,) = _by_type(ev, "message_delta")
|
||||
assert delta["delta"]["safeguard_results"] == _RESULTS
|
||||
Reference in New Issue
Block a user