fix(proxy): inject memory context past a trailing system message (#3948)

## Description

In `--mode token`, memory context never reaches the model under Claude
Code 2.1.x. Claude Code sends a trailing `role: "system"` message
(environment context that carries the cache breakpoint) after the user
turn:

```text
messages[0] user   text <system-reminder>…</system-reminder>   (no cache_control)
            user   text <system-reminder>…</system-reminder>   (no cache_control)
            user   text <the prompt>                            (no cache_control)
messages[1] system text # Environment …                         (cache_control)
```

`_append_context_to_latest_non_frozen_user_turn` only looked at
`messages[-1]`, saw a non-user role and returned unchanged. Memory
search found the context, but every request logged
`decision=no_eligible_user_turn` and nothing was injected. The same
helper serves the CCR proactive-expansion path, which had the same
problem.

This PR skips trailing system-role messages when locating the latest
user turn. The system message stays byte-identical and the frozen-prefix
check still applies to the user turn, so prompt-cache behaviour is
unchanged. Follow-up to #3947, which fixed memory *tools* on streaming
turns; this one fixes memory *context injection*.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/anthropic.py`:
`_append_context_to_latest_non_frozen_user_turn` walks back past
trailing `role: "system"` messages before the frozen-prefix and
user-role checks; docstring explains why.
- `tests/test_proxy_handler_helpers.py`:
`test_append_context_skips_trailing_system_message` covers injection
into the Claude Code 2.1.x shape (system message untouched), the frozen
prefix still blocking, never reaching past an assistant turn,
tool_result-only turns, and a system-only list.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ HOME=$(mktemp -d) .venv/bin/python -m pytest -q tests/test_proxy_handler_helpers.py tests/test_proxy_anthropic_cache_stability.py tests/test_cache_breakpoint_diagnostics.py tests/test_ccr_context_tracker.py
origin/main: 156 passed, 1 skipped
this branch: 157 passed, 1 skipped

$ HOME=$(mktemp -d) .venv/bin/python -m pytest -q $(grep -rlE 'injected_live_zone_tail|no_eligible_user_turn|_append_context_to_latest|memory_injection|MemoryDecision' tests)
317 passed, 1 skipped

$ uvx ruff@0.16.8 check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py
All checks passed!
$ uvx ruff@0.16.8 format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS 27, real Claude Code 2.1.287 (`claude -p`, Opus
5.5) → Headroom proxy (`--memory --mode token`) → real Anthropic
upstream. Before = `origin/main` (540f4da1) in a worktree; after = this
branch in a worktree; same interpreter, same copy of a memory store
seeded by a session-1 run ("knowledge-base changes go on kb/ branches;
PR titles start with the entity id; squash-merge; ADR reviewer must not
be in decided_by").
- Exact command / steps: session 2 with the same
`x-headroom-project-id`: `claude -p "I've drafted a new ADR called
ADR-20260930-s3-lifecycle-rules (decided_by: kenneth). I'm ready to
push. What should the branch be called, how should I title the PR and
merge it, and who can review it?"`, then count `memory_injection`
decisions in the proxy log.
- Observed result: before, no injection and the answer applies none of
the saved conventions; after, memory is injected on the first request
and the answer applies them. The remaining `no_eligible_user_turn` after
the fix is the follow-up tool_result turn, which is expected. Excerpts:

  ```text
origin/main proxy log: 2 x decision=no_eligible_user_turn
bytes_injected=0
answer: "this repo has no branch or PR process … If you want to invent a
convention anyway,
adr/20260930-s3-lifecycle-rules with a squash merge would match …"
this branch proxy log: 1 x decision=injected_live_zone_tail
bytes_injected=1199
1 x decision=no_eligible_user_turn bytes_injected=0 (tool_result turn)
answer: "PR title must start with the entity id (from a team convention
saved in an earlier session)
… Merge with squash … A PR that adds or changes an ADR needs a reviewer
who isn't in its decided_by"
  ```
- Not tested: `mypy headroom` (not run locally); interactive Claude Code
sessions (the request shape was captured from `claude -p`); the OpenAI
and Gemini handlers, which have their own injection code and are not
changed.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This changes the existing
memory-context injection path for Anthropic requests.
- Minimum rollout channel: stable.
- Stable/default behavior changed: only for requests whose messages end
in one or more `role: "system"` messages; injection now targets the user
turn before them instead of being skipped. Default `--mode cache` still
skips injection by design, so default behaviour is unchanged.
- Kill switch / disable path: `--no-memory-context` (or
`HEADROOM_MEMORY_INJECTION_MODE=disabled`).
- Unsafe override required: no.
- Qualification impact: none expected; the cache hot zone (system
prompt, frozen prefix, trailing system message) is never mutated.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Documentation: no doc change needed; behaviour now matches what the
injector's docstring already described.
- Possibly related, not changed here:
`_strict_previous_turn_frozen_count` (cache mode) also treats a final
non-user message as "freeze everything", so with Claude Code's trailing
system message the current turn may be frozen in cache mode too. That
affects compression rather than memory and needs its own cache-stability
review.

https://claude.ai/code/session_01B6gHB8BWnYShVzrkgcBKWf

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3948"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Kenneth Wong
2026-10-03 16:13:05 -05:00
committed by GitHub
co-authored by JD Davis
parent 5eaf24ec97
commit 3948dbe409
2 changed files with 52 additions and 1 deletions
+8 -1
View File
@@ -636,6 +636,11 @@ class AnthropicHandlerMixin:
first text block of the latest user message is mutated, which is by
definition the live zone.
Trailing ``role: "system"`` messages are skipped when locating that
turn: Claude Code appends one after the user message (environment
context carrying the cache breakpoint), and it is not a conversational
turn. It is left byte-identical.
Returns the input list unchanged if no eligible user text block
exists (e.g., the last message is an assistant turn or a tool
result, or the user message has no text block).
@@ -644,7 +649,9 @@ class AnthropicHandlerMixin:
return messages
i = len(messages) - 1
if i < frozen_message_count:
while i >= 0 and messages[i].get("role") == "system":
i -= 1
if i < 0 or i < frozen_message_count:
return messages
msg = messages[i]
if msg.get("role") != "user":
+44
View File
@@ -945,6 +945,50 @@ def test_anthropic_tool_sort_and_context_append_helpers() -> None:
) == [{"role": "user", "content": [{"type": "text", "text": "hello\n\nctx"}]}]
def test_append_context_skips_trailing_system_message() -> None:
# Claude Code 2.1.x request shape: the user turn is followed by a
# role="system" message carrying the environment and the cache breakpoint.
trailing_system = {
"role": "system",
"content": [
{"type": "text", "text": "# Environment", "cache_control": {"type": "ephemeral"}}
],
}
user_turn = {
"role": "user",
"content": [
{"type": "text", "text": "<system-reminder>ctx</system-reminder>"},
{"type": "text", "text": "question"},
],
}
inject = AnthropicHandlerMixin._append_context_to_latest_non_frozen_user_turn
assert inject([user_turn, trailing_system], "memory", frozen_message_count=0) == [
{
"role": "user",
"content": [
{"type": "text", "text": "<system-reminder>ctx</system-reminder>\n\nmemory"},
{"type": "text", "text": "question"},
],
},
trailing_system,
]
# The user turn is still subject to the frozen prefix.
messages = [user_turn, trailing_system]
assert inject(messages, "memory", frozen_message_count=1) is messages
# Skipping system messages never reaches past a non-user turn.
messages = [user_turn, {"role": "assistant", "content": "ok"}, trailing_system]
assert inject(messages, "memory", frozen_message_count=0) is messages
# Tool-result-only turns have no text block to extend.
messages = [
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "t1", "content": "x"}]},
trailing_system,
]
assert inject(messages, "memory", frozen_message_count=0) is messages
messages = [trailing_system]
assert inject(messages, "memory", frozen_message_count=0) is messages
def test_anthropic_image_compression_helper_only_rewrites_latest_eligible_turn() -> None:
image_message = {
"role": "user",