fix(proxy): keep Claude Haiku 5.5 mid-conversation system messages in place (#4153)

## Description

Claude Haiku 5.5 accepts `role: "system"` entries inside `messages[]`
(mid-conversation system messages, no beta header).
`relocate_system_messages_to_top_level` (`headroom/proxy/helpers.py`)
doesn't list the Haiku 5 family, so on every Haiku 5.5 request it moves
each valid mid-conversation section into the top-level `system` field.
That has two effects:

- The operator instruction moves from the point in the conversation
where it became true to the top of the prompt, which changes its
meaning.
- The top-level `system` that the cached prefix starts with changes from
conversation to conversation.

Seen on a local proxy over ~28 hours (2026-10-10 01:14 → 2026-10-11
05:27): all 599 `Relocated role=system` warnings were `claude-haiku-5-5`
requests, which is Claude Code's subagent traffic. The Opus 5.5 (9,360)
and Sonnet 5.5 (539) requests in the same window were never relocated.

This adds `"claude-haiku-5"` to the supported families. Haiku 4.5, which
doesn't support mid-conversation system messages, is still relocated.

No linked issue.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/helpers.py`: add `"claude-haiku-5"` to the model
families `relocate_system_messages_to_top_level` treats as supporting
mid-conversation system messages. The substring matching works for every
model-ID form the list already handles, including `claude-haiku-5-5` and
Bedrock-style `…anthropic.claude-haiku-5-5…`. It doesn't match
`claude-haiku-4-5`.
- `tests/test_proxy_handler_helpers.py`: parametrize the "preserves a
valid mid-conversation section" test over `claude-opus-5` and
`claude-haiku-5-5`, and the "moves a valid shape for an unsupported
model" test over `claude-sonnet-4-6` and `claude-haiku-4-5`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ HOME=$(mktemp -d) pytest -q tests/test_proxy_handler_helpers.py
74 passed                       # origin/main: 72 passed; +2 new parametrized cases

$ ruff check headroom/proxy/helpers.py tests/test_proxy_handler_helpers.py
All checks passed!
$ ruff format --check headroom/proxy/helpers.py tests/test_proxy_handler_helpers.py
2 files already formatted
$ mypy headroom/proxy/helpers.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS arm64, repo `.venv` (Python 3.10.18). The live
proxy traffic came from `headroom proxy` on 127.0.0.1:8787, forwarding
to `api.anthropic.com`.
- Exact command / steps: called `relocate_system_messages_to_top_level`
with a valid `user → system → assistant` request on `origin/main`
(976aa714) and on this branch, for `claude-haiku-5-5` and
`claude-haiku-4-5`. Separately, joined every `Relocated role=system`
warning in the proxy log to that request's `PERF model=` line by request
id.
- Observed result: on `origin/main`, `claude-haiku-5-5` gives
`changed=True roles=['user', 'assistant'] system=[{'type': 'text',
'text': 'base'}, {'type': 'text', 'text': 'The user added: update the
changelog too.'}]`. On this branch it gives `changed=False
roles=['user', 'system', 'assistant'] system='base'`. `claude-haiku-4-5`
is relocated on both. In the proxy log, 599 of 599 relocated requests
were `claude-haiku-5-5`, and Haiku 5.5's in-conversation turns had a
cache hit under 50% 7.1% of the time, against 0.3% for Opus 5.5 and 0.2%
for Sonnet 5.5. That's a correlation; the log can't key turns to
conversations. At Haiku 5.5's $0.10/MTok the cache cost is small. The
main issue is where the operator instruction ends up.
- Not tested: a live Haiku 5.5 request with the section left in place.
On the proxy I could test through, the released build relocates Haiku
requests before they leave. Haiku 5.5 support comes from Anthropic's API
reference, which lists it with no beta header. For the existing
`claude-sonnet-5` entry, a live first-party request with a trailing
`role: "system"` message on `claude-sonnet-5` returned HTTP 200 and
followed the instruction, so that entry is correct even though some docs
omit Sonnet 5. Bedrock and Vertex weren't tested.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A. This is the default wire-contract guard,
with no flag.
- Stable/default behavior changed: yes. Valid mid-conversation system
sections on Claude Haiku 5.x requests are now forwarded in place instead
of being moved to the top-level `system`. Invalid placements (leading,
or followed by a `user` turn) are still moved for every model.
- Kill switch / disable path: none needed for a model-list entry. If an
upstream rejected Haiku 5.5 mid-conversation system messages, removing
the entry restores the old behavior.
- Unsafe override required: no.
- Qualification impact: none expected. Requests that already relocate
keep the same output; only Haiku 5.x requests with valid sections
change.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A.

## Additional Notes

- Documentation: N/A. No user-facing docs list the supported models for
this guard.
- Type checking was run on the changed module
(`headroom/proxy/helpers.py`), not the whole package.
- Sources disagree on Claude Sonnet 5 (some docs omit it). The live
check above supports keeping it, so this PR leaves it alone.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4153"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
This commit is contained in:
Kenneth Wong
2026-10-13 16:21:13 -07:00
committed by GitHub
co-authored by Kenneth Wong
parent 22a1296cfa
commit a9eb2a7b6f
2 changed files with 16 additions and 8 deletions
+1
View File
@@ -1095,6 +1095,7 @@ def relocate_system_messages_to_top_level(
family in model_id
for family in (
"claude-fable-5",
"claude-haiku-5",
"claude-mythos-5",
"claude-opus-4-8",
"claude-opus-5",
+15 -8
View File
@@ -333,7 +333,17 @@ def test_relocate_system_messages_noop_without_system_entry() -> None:
assert system == "A"
def test_relocate_system_messages_preserves_valid_mid_conversation_section() -> None:
@pytest.mark.parametrize(
"model",
[
"claude-opus-5",
# Claude Haiku 5.5 supports mid-conversation system messages too; it was
# missing here, so Claude Code's Haiku subagent traffic had every valid
# section hoisted into the top-level system prompt.
"claude-haiku-5-5",
],
)
def test_relocate_system_messages_preserves_valid_mid_conversation_section(model: str) -> None:
messages = [
{"role": "user", "content": "Run the tests."},
{
@@ -343,9 +353,7 @@ def test_relocate_system_messages_preserves_valid_mid_conversation_section() ->
{"role": "assistant", "content": "I will do both."},
]
clean, system, changed = relocate_system_messages_to_top_level(
messages, "base", "claude-opus-5"
)
clean, system, changed = relocate_system_messages_to_top_level(messages, "base", model)
assert changed is False
assert clean is messages
@@ -392,15 +400,14 @@ def test_relocate_system_messages_still_moves_invalid_mid_conversation_placement
assert system == [{"type": "text", "text": messages[1]["content"]}]
def test_relocate_system_messages_moves_valid_shape_for_unsupported_model() -> None:
@pytest.mark.parametrize("model", ["claude-sonnet-4-6", "claude-haiku-4-5"])
def test_relocate_system_messages_moves_valid_shape_for_unsupported_model(model: str) -> None:
messages = [
{"role": "user", "content": "question"},
{"role": "system", "content": "mid-turn instruction"},
]
clean, system, changed = relocate_system_messages_to_top_level(
messages, None, "claude-sonnet-4-6"
)
clean, system, changed = relocate_system_messages_to_top_level(messages, None, model)
assert changed is True
assert clean == [{"role": "user", "content": "question"}]