fix(proxy): keep leading Anthropic system payloads valid (#3671)

## Description

Leading Anthropic system-role sections containing image/document or
malformed text-shaped blocks could produce an invalid upstream request.
Normalize representable text into the top-level `system` field using
Anthropic TextBlockParam fields only, preserve cache/citation metadata,
and remove unrepresentable leading blocks with an explicit warning.
Mid-conversation non-text retention remains covered by the existing
contract.

Closes #3667

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update

## Changes Made

- Keep only `type`, `text`, `cache_control`, and `citations` when
converting leading text-shaped blocks. Image sources and tool
IDs/names/inputs cannot leak into top-level text blocks, including
blocks already labeled `text`.
- Normalize every contiguous leading system section, so a second
image-only section cannot become an invalid first message after the
first section is removed.
- Preserve valid text/cache/citation metadata without mutating the input
messages.
- Retain the PR's explicit warning/drop policy for leading content with
no text representation; mid-conversation non-text blocks retain their
existing behavior.

## Testing

Current head: `cc05220696735f4162a986e24e5e6ab47ab43e71`.

- Six new regressions fail before correction and pass afterward: five
text-block type/field variants and consecutive leading system sections.
- 182 helper, Anthropic cache-stability, system-prompt immutability, and
byte-faithful forwarding tests pass (one existing Starlette warning).
- Ruff lint/format, whitespace, and Linux-target mypy over 566 files
pass.
- Allowed text-block fields were checked against the installed Anthropic
SDK TextBlockParam contract.

## Real Behavior Proof

- Setup: Windows Python 3.12, a real uvicorn proxy and a separate real
loopback HTTP upstream enforcing Anthropic's top-level text-block field
contract. Optimization, caching, cost tracking, and subscription polling
disabled.
- Steps: run `pr3671-resume17-runtime.py`, send five real TCP Messages
requests, and inspect the upstream-received bodies and proxy responses.
- Observed: image-caption, tool-text, text-with-extra-fields,
consecutive leading sections, and valid-text control all return HTTP
200. The upstream receives text-only allowed fields and messages
beginning with a user turn; cache-control TTL survives. Consecutive
sections preserve both text instructions.
- Before correction, the same replay returns HTTP 400 for the first four
cases; the valid-text control remains HTTP 200. Evidence:
`pr3671-resume17-runtime-red.log` and `pr3671-resume17-runtime.log`;
tests: `pr3671-resume17-tests.log`.
- Not tested: live Anthropic account/billing traffic. Only network
delivery is redirected to the loopback upstream. The intentional removal
of unrepresentable leading system content remains a warning-producing
normalization, not a lossless image conversion.

## Runtime Rollout Safety

- Rollout-managed features: none; normalization uses the existing
Anthropic wire-contract guard.
- Default behavior changed: leading text-shaped blocks emit valid
Anthropic text fields, all contiguous leading sections normalize, and
unrepresentable leading system blocks are removed with warnings.
- Valid text/cache/citation metadata and mid-conversation retention
controls are covered. No new unsafe override or network behavior.
- Rollback: revert the normalization commits.

## Review Readiness

- [x] Self-review completed
- [x] Regression coverage and local checks pass
- [x] Current real proxy behavior supplied
- [x] Did not edit CHANGELOG.md

Hosted checks and review must qualify this new head before approval. No
merge performed.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
lorenzozane
2026-10-08 10:41:39 -05:00
committed by GitHub
co-authored by JerrettDavis
parent fd067033da
commit d9e29bb260
2 changed files with 172 additions and 11 deletions
+55 -11
View File
@@ -1048,6 +1048,22 @@ _ROLE_SYSTEM = "system"
_TEXT_BLOCK_TYPE = "text"
def _coerce_system_block(block: dict[str, Any]) -> dict[str, Any] | None:
"""Return a block representable by Anthropic's top-level ``system`` field."""
text = block.get("text")
if isinstance(text, str) and text:
# Anthropic TextBlockParam accepts these fields only. Image sources
# and tool IDs/inputs remain invalid even after changing ``type``.
coerced = {
key: value
for key, value in block.items()
if key in {"type", "text", "cache_control", "citations"}
}
coerced["type"] = _TEXT_BLOCK_TYPE
return coerced
return None
def _system_message_to_blocks(message: dict[str, Any]) -> list[Any]:
"""Convert a ``role="system"`` message into Anthropic system content blocks."""
content = message.get("content")
@@ -1078,10 +1094,12 @@ def relocate_system_messages_to_top_level(
field as the issue-765 last-line wire-contract guard.
The relocated content is appended after any existing top-level ``system``
so wire order (system prompt, then conversation) is preserved and no content
is dropped. Only text-shaped content moves: non-text blocks (images,
documents) stay in a mid-conversation system section at their original
position, because top-level `system` accepts text blocks only (issue #3552).
so wire order (system prompt, then conversation) is preserved. Only
text-shaped content moves: non-text blocks (images, documents) stay in a
mid-conversation system section at their original position, because
top-level `system` accepts text blocks only (issue #3552). A leading system
section cannot stay in ``messages[0]``; non-text blocks there are dropped
with a warning because they cannot be represented in either valid location.
Returns ``(clean_messages, new_system, changed)``. When no system-role
message is present the inputs pass through unchanged (``changed=False``) so
@@ -1144,26 +1162,52 @@ def relocate_system_messages_to_top_level(
relocated_blocks: list[Any] = []
retained: dict[int, dict[str, Any]] = {}
leading_system_indices: set[int] = set()
for index, message in enumerate(messages):
if not isinstance(message, dict) or message.get("role") != _ROLE_SYSTEM:
break
leading_system_indices.add(index)
for i in sorted(system_indices):
message = messages[i]
content = message.get("content") if isinstance(message, dict) else None
if isinstance(content, list):
# Only text-shaped content may move into the top-level ``system``
# parameter (text blocks and bare strings). Non-text blocks such as
# images or documents stay in place so nothing is dropped and
# upstreams that reject non-text system blocks keep working
# (issue #3552).
# images or documents stay in mid-conversation sections so nothing
# is dropped and upstreams that reject non-text system blocks keep
# working (issue #3552). A leading section cannot remain there
# without violating Anthropic's message contract.
hoisted_from_list: list[Any] = []
leftovers: list[Any] = []
for block in content:
if isinstance(block, dict) and block.get("type") == _TEXT_BLOCK_TYPE:
hoisted_from_list.append(block)
if isinstance(block, dict):
if i in leading_system_indices:
coerced = _coerce_system_block(block)
if coerced is not None:
hoisted_from_list.append(coerced)
else:
logger.warning(
"event=system_relocation_block_dropped block_type=%s "
"reason=not_representable_as_text",
block.get("type", "missing"),
)
elif block.get("type") == _TEXT_BLOCK_TYPE:
hoisted_from_list.append(block)
else:
leftovers.append(block)
elif isinstance(block, str) and block:
hoisted_from_list.append({"type": _TEXT_BLOCK_TYPE, "text": block})
else:
leftovers.append(block)
if i in leading_system_indices:
logger.warning(
"event=system_relocation_block_dropped block_type=%s "
"reason=not_representable_as_text",
"string" if isinstance(block, str) else type(block).__name__,
)
else:
leftovers.append(block)
relocated_blocks.extend(hoisted_from_list)
if leftovers:
if leftovers and i not in leading_system_indices:
retained[i] = {**message, "content": leftovers}
else:
# String (and other) content converts losslessly to text blocks.
+117
View File
@@ -492,6 +492,123 @@ def test_relocate_system_messages_image_only_sections_pass_through_unchanged() -
assert new_system == system
def test_relocate_system_messages_drops_non_text_from_leading_section(
caplog: pytest.LogCaptureFixture,
) -> None:
image_block = {
"type": "image",
"source": {"type": "base64", "media_type": "image/png", "data": "aGk="},
}
cached_text = {
"type": "text",
"text": "leading instruction",
"cache_control": {"type": "ephemeral"},
}
messages: list[dict] = [
{"role": "system", "content": [cached_text, image_block]},
{"role": "user", "content": "hi"},
]
caplog.set_level("WARNING", logger="headroom.proxy")
clean, new_system, changed = relocate_system_messages_to_top_level(
messages, [{"type": "text", "text": "base"}], "claude-opus-5"
)
assert changed is True
assert clean == [{"role": "user", "content": "hi"}]
assert new_system == [
{"type": "text", "text": "base"},
cached_text,
]
assert "event=system_relocation_block_dropped" in caplog.text
assert "block_type=image" in caplog.text
def test_relocate_system_messages_drops_all_non_text_leading_section() -> None:
image_block = {
"type": "image",
"source": {"type": "base64", "media_type": "image/png", "data": "aGk="},
}
messages: list[dict] = [
{"role": "system", "content": [image_block]},
{"role": "user", "content": "hi"},
]
clean, new_system, changed = relocate_system_messages_to_top_level(
messages, [{"type": "text", "text": "base"}], "claude-opus-5"
)
assert changed is True
assert clean == [{"role": "user", "content": "hi"}]
assert new_system == [{"type": "text", "text": "base"}]
@pytest.mark.parametrize("block_type", ["text", "image", "tool_use", "tool_result", None])
def test_leading_system_text_uses_only_anthropic_text_block_fields(block_type) -> None:
from copy import deepcopy
cache_control = {"type": "ephemeral", "ttl": "1h"}
citations = [
{
"type": "char_location",
"cited_text": "instruction",
"document_index": 0,
"document_title": "source",
"start_char_index": 0,
"end_char_index": 11,
}
]
block = {
"type": block_type,
"text": "instruction",
"cache_control": cache_control,
"citations": citations,
"source": {"type": "url", "url": "https://example.invalid/image.png"},
"id": "tool1",
"name": "read",
"input": {"path": "file.txt"},
"tool_use_id": "tool1",
"is_error": False,
}
messages = [
{"role": "system", "content": [block]},
{"role": "user", "content": "hi"},
]
original = deepcopy(messages)
clean, system, changed = relocate_system_messages_to_top_level(messages, None, None)
assert changed
assert clean == [{"role": "user", "content": "hi"}]
assert system == [
{
"type": "text",
"text": "instruction",
"cache_control": cache_control,
"citations": citations,
}
]
assert messages == original
def test_all_contiguous_leading_system_sections_are_normalized() -> None:
image = {"type": "image", "source": {"type": "url", "url": "https://example.invalid/a"}}
clean, system, changed = relocate_system_messages_to_top_level(
[
{"role": "system", "content": "first instruction"},
{"role": "system", "content": [image]},
{"role": "system", "content": [{"type": "text", "text": "second instruction"}]},
{"role": "user", "content": "hi"},
],
None,
None,
)
assert changed
assert clean == [{"role": "user", "content": "hi"}]
assert system == [
{"type": "text", "text": "first instruction"},
{"type": "text", "text": "second instruction"},
]
def test_headroom_bypass_helper_is_transport_neutral() -> None:
assert _headroom_bypass_enabled({"x-headroom-bypass": "true"}) is True
assert _headroom_bypass_enabled({"x-headroom-bypass": " TRUE "}) is True