mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): keep leading Anthropic system payloads valid (#3671)
## Description Leading Anthropic system-role sections containing image/document or malformed text-shaped blocks could produce an invalid upstream request. Normalize representable text into the top-level `system` field using Anthropic TextBlockParam fields only, preserve cache/citation metadata, and remove unrepresentable leading blocks with an explicit warning. Mid-conversation non-text retention remains covered by the existing contract. Closes #3667 ## Type of Change - [x] Bug fix - [ ] New feature - [ ] Breaking change - [ ] Documentation update ## Changes Made - Keep only `type`, `text`, `cache_control`, and `citations` when converting leading text-shaped blocks. Image sources and tool IDs/names/inputs cannot leak into top-level text blocks, including blocks already labeled `text`. - Normalize every contiguous leading system section, so a second image-only section cannot become an invalid first message after the first section is removed. - Preserve valid text/cache/citation metadata without mutating the input messages. - Retain the PR's explicit warning/drop policy for leading content with no text representation; mid-conversation non-text blocks retain their existing behavior. ## Testing Current head: `cc05220696735f4162a986e24e5e6ab47ab43e71`. - Six new regressions fail before correction and pass afterward: five text-block type/field variants and consecutive leading system sections. - 182 helper, Anthropic cache-stability, system-prompt immutability, and byte-faithful forwarding tests pass (one existing Starlette warning). - Ruff lint/format, whitespace, and Linux-target mypy over 566 files pass. - Allowed text-block fields were checked against the installed Anthropic SDK TextBlockParam contract. ## Real Behavior Proof - Setup: Windows Python 3.12, a real uvicorn proxy and a separate real loopback HTTP upstream enforcing Anthropic's top-level text-block field contract. Optimization, caching, cost tracking, and subscription polling disabled. - Steps: run `pr3671-resume17-runtime.py`, send five real TCP Messages requests, and inspect the upstream-received bodies and proxy responses. - Observed: image-caption, tool-text, text-with-extra-fields, consecutive leading sections, and valid-text control all return HTTP 200. The upstream receives text-only allowed fields and messages beginning with a user turn; cache-control TTL survives. Consecutive sections preserve both text instructions. - Before correction, the same replay returns HTTP 400 for the first four cases; the valid-text control remains HTTP 200. Evidence: `pr3671-resume17-runtime-red.log` and `pr3671-resume17-runtime.log`; tests: `pr3671-resume17-tests.log`. - Not tested: live Anthropic account/billing traffic. Only network delivery is redirected to the loopback upstream. The intentional removal of unrepresentable leading system content remains a warning-producing normalization, not a lossless image conversion. ## Runtime Rollout Safety - Rollout-managed features: none; normalization uses the existing Anthropic wire-contract guard. - Default behavior changed: leading text-shaped blocks emit valid Anthropic text fields, all contiguous leading sections normalize, and unrepresentable leading system blocks are removed with warnings. - Valid text/cache/citation metadata and mid-conversation retention controls are covered. No new unsafe override or network behavior. - Rollback: revert the normalization commits. ## Review Readiness - [x] Self-review completed - [x] Regression coverage and local checks pass - [x] Current real proxy behavior supplied - [x] Did not edit CHANGELOG.md Hosted checks and review must qualify this new head before approval. No merge performed. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JerrettDavis
parent
fd067033da
commit
d9e29bb260
+55
-11
@@ -1048,6 +1048,22 @@ _ROLE_SYSTEM = "system"
|
||||
_TEXT_BLOCK_TYPE = "text"
|
||||
|
||||
|
||||
def _coerce_system_block(block: dict[str, Any]) -> dict[str, Any] | None:
|
||||
"""Return a block representable by Anthropic's top-level ``system`` field."""
|
||||
text = block.get("text")
|
||||
if isinstance(text, str) and text:
|
||||
# Anthropic TextBlockParam accepts these fields only. Image sources
|
||||
# and tool IDs/inputs remain invalid even after changing ``type``.
|
||||
coerced = {
|
||||
key: value
|
||||
for key, value in block.items()
|
||||
if key in {"type", "text", "cache_control", "citations"}
|
||||
}
|
||||
coerced["type"] = _TEXT_BLOCK_TYPE
|
||||
return coerced
|
||||
return None
|
||||
|
||||
|
||||
def _system_message_to_blocks(message: dict[str, Any]) -> list[Any]:
|
||||
"""Convert a ``role="system"`` message into Anthropic system content blocks."""
|
||||
content = message.get("content")
|
||||
@@ -1078,10 +1094,12 @@ def relocate_system_messages_to_top_level(
|
||||
field as the issue-765 last-line wire-contract guard.
|
||||
|
||||
The relocated content is appended after any existing top-level ``system``
|
||||
so wire order (system prompt, then conversation) is preserved and no content
|
||||
is dropped. Only text-shaped content moves: non-text blocks (images,
|
||||
documents) stay in a mid-conversation system section at their original
|
||||
position, because top-level `system` accepts text blocks only (issue #3552).
|
||||
so wire order (system prompt, then conversation) is preserved. Only
|
||||
text-shaped content moves: non-text blocks (images, documents) stay in a
|
||||
mid-conversation system section at their original position, because
|
||||
top-level `system` accepts text blocks only (issue #3552). A leading system
|
||||
section cannot stay in ``messages[0]``; non-text blocks there are dropped
|
||||
with a warning because they cannot be represented in either valid location.
|
||||
|
||||
Returns ``(clean_messages, new_system, changed)``. When no system-role
|
||||
message is present the inputs pass through unchanged (``changed=False``) so
|
||||
@@ -1144,26 +1162,52 @@ def relocate_system_messages_to_top_level(
|
||||
|
||||
relocated_blocks: list[Any] = []
|
||||
retained: dict[int, dict[str, Any]] = {}
|
||||
leading_system_indices: set[int] = set()
|
||||
for index, message in enumerate(messages):
|
||||
if not isinstance(message, dict) or message.get("role") != _ROLE_SYSTEM:
|
||||
break
|
||||
leading_system_indices.add(index)
|
||||
for i in sorted(system_indices):
|
||||
message = messages[i]
|
||||
content = message.get("content") if isinstance(message, dict) else None
|
||||
if isinstance(content, list):
|
||||
# Only text-shaped content may move into the top-level ``system``
|
||||
# parameter (text blocks and bare strings). Non-text blocks such as
|
||||
# images or documents stay in place so nothing is dropped and
|
||||
# upstreams that reject non-text system blocks keep working
|
||||
# (issue #3552).
|
||||
# images or documents stay in mid-conversation sections so nothing
|
||||
# is dropped and upstreams that reject non-text system blocks keep
|
||||
# working (issue #3552). A leading section cannot remain there
|
||||
# without violating Anthropic's message contract.
|
||||
hoisted_from_list: list[Any] = []
|
||||
leftovers: list[Any] = []
|
||||
for block in content:
|
||||
if isinstance(block, dict) and block.get("type") == _TEXT_BLOCK_TYPE:
|
||||
hoisted_from_list.append(block)
|
||||
if isinstance(block, dict):
|
||||
if i in leading_system_indices:
|
||||
coerced = _coerce_system_block(block)
|
||||
if coerced is not None:
|
||||
hoisted_from_list.append(coerced)
|
||||
else:
|
||||
logger.warning(
|
||||
"event=system_relocation_block_dropped block_type=%s "
|
||||
"reason=not_representable_as_text",
|
||||
block.get("type", "missing"),
|
||||
)
|
||||
elif block.get("type") == _TEXT_BLOCK_TYPE:
|
||||
hoisted_from_list.append(block)
|
||||
else:
|
||||
leftovers.append(block)
|
||||
elif isinstance(block, str) and block:
|
||||
hoisted_from_list.append({"type": _TEXT_BLOCK_TYPE, "text": block})
|
||||
else:
|
||||
leftovers.append(block)
|
||||
if i in leading_system_indices:
|
||||
logger.warning(
|
||||
"event=system_relocation_block_dropped block_type=%s "
|
||||
"reason=not_representable_as_text",
|
||||
"string" if isinstance(block, str) else type(block).__name__,
|
||||
)
|
||||
else:
|
||||
leftovers.append(block)
|
||||
relocated_blocks.extend(hoisted_from_list)
|
||||
if leftovers:
|
||||
if leftovers and i not in leading_system_indices:
|
||||
retained[i] = {**message, "content": leftovers}
|
||||
else:
|
||||
# String (and other) content converts losslessly to text blocks.
|
||||
|
||||
@@ -492,6 +492,123 @@ def test_relocate_system_messages_image_only_sections_pass_through_unchanged() -
|
||||
assert new_system == system
|
||||
|
||||
|
||||
def test_relocate_system_messages_drops_non_text_from_leading_section(
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
image_block = {
|
||||
"type": "image",
|
||||
"source": {"type": "base64", "media_type": "image/png", "data": "aGk="},
|
||||
}
|
||||
cached_text = {
|
||||
"type": "text",
|
||||
"text": "leading instruction",
|
||||
"cache_control": {"type": "ephemeral"},
|
||||
}
|
||||
messages: list[dict] = [
|
||||
{"role": "system", "content": [cached_text, image_block]},
|
||||
{"role": "user", "content": "hi"},
|
||||
]
|
||||
|
||||
caplog.set_level("WARNING", logger="headroom.proxy")
|
||||
clean, new_system, changed = relocate_system_messages_to_top_level(
|
||||
messages, [{"type": "text", "text": "base"}], "claude-opus-5"
|
||||
)
|
||||
|
||||
assert changed is True
|
||||
assert clean == [{"role": "user", "content": "hi"}]
|
||||
assert new_system == [
|
||||
{"type": "text", "text": "base"},
|
||||
cached_text,
|
||||
]
|
||||
assert "event=system_relocation_block_dropped" in caplog.text
|
||||
assert "block_type=image" in caplog.text
|
||||
|
||||
|
||||
def test_relocate_system_messages_drops_all_non_text_leading_section() -> None:
|
||||
image_block = {
|
||||
"type": "image",
|
||||
"source": {"type": "base64", "media_type": "image/png", "data": "aGk="},
|
||||
}
|
||||
messages: list[dict] = [
|
||||
{"role": "system", "content": [image_block]},
|
||||
{"role": "user", "content": "hi"},
|
||||
]
|
||||
|
||||
clean, new_system, changed = relocate_system_messages_to_top_level(
|
||||
messages, [{"type": "text", "text": "base"}], "claude-opus-5"
|
||||
)
|
||||
|
||||
assert changed is True
|
||||
assert clean == [{"role": "user", "content": "hi"}]
|
||||
assert new_system == [{"type": "text", "text": "base"}]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("block_type", ["text", "image", "tool_use", "tool_result", None])
|
||||
def test_leading_system_text_uses_only_anthropic_text_block_fields(block_type) -> None:
|
||||
from copy import deepcopy
|
||||
|
||||
cache_control = {"type": "ephemeral", "ttl": "1h"}
|
||||
citations = [
|
||||
{
|
||||
"type": "char_location",
|
||||
"cited_text": "instruction",
|
||||
"document_index": 0,
|
||||
"document_title": "source",
|
||||
"start_char_index": 0,
|
||||
"end_char_index": 11,
|
||||
}
|
||||
]
|
||||
block = {
|
||||
"type": block_type,
|
||||
"text": "instruction",
|
||||
"cache_control": cache_control,
|
||||
"citations": citations,
|
||||
"source": {"type": "url", "url": "https://example.invalid/image.png"},
|
||||
"id": "tool1",
|
||||
"name": "read",
|
||||
"input": {"path": "file.txt"},
|
||||
"tool_use_id": "tool1",
|
||||
"is_error": False,
|
||||
}
|
||||
messages = [
|
||||
{"role": "system", "content": [block]},
|
||||
{"role": "user", "content": "hi"},
|
||||
]
|
||||
original = deepcopy(messages)
|
||||
clean, system, changed = relocate_system_messages_to_top_level(messages, None, None)
|
||||
assert changed
|
||||
assert clean == [{"role": "user", "content": "hi"}]
|
||||
assert system == [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "instruction",
|
||||
"cache_control": cache_control,
|
||||
"citations": citations,
|
||||
}
|
||||
]
|
||||
assert messages == original
|
||||
|
||||
|
||||
def test_all_contiguous_leading_system_sections_are_normalized() -> None:
|
||||
image = {"type": "image", "source": {"type": "url", "url": "https://example.invalid/a"}}
|
||||
clean, system, changed = relocate_system_messages_to_top_level(
|
||||
[
|
||||
{"role": "system", "content": "first instruction"},
|
||||
{"role": "system", "content": [image]},
|
||||
{"role": "system", "content": [{"type": "text", "text": "second instruction"}]},
|
||||
{"role": "user", "content": "hi"},
|
||||
],
|
||||
None,
|
||||
None,
|
||||
)
|
||||
assert changed
|
||||
assert clean == [{"role": "user", "content": "hi"}]
|
||||
assert system == [
|
||||
{"type": "text", "text": "first instruction"},
|
||||
{"type": "text", "text": "second instruction"},
|
||||
]
|
||||
|
||||
|
||||
def test_headroom_bypass_helper_is_transport_neutral() -> None:
|
||||
assert _headroom_bypass_enabled({"x-headroom-bypass": "true"}) is True
|
||||
assert _headroom_bypass_enabled({"x-headroom-bypass": " TRUE "}) is True
|
||||
|
||||
Reference in New Issue
Block a user