mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
refactor(ccr): read the expansion query through extract_user_query (#2707)
## Description
The CCR proactive-expansion path hand-rolled its own "most recent user
question" scan instead of calling `headroom.utils.extract_user_query`,
which every other consumer on this handler already uses. Two copies of
the same read meant any fix to how the query is identified had to be
made twice — and only one copy would get it.
No behaviour change intended for the normal case. This is groundwork:
there is a real defect in how that query is identified (the helper does
not strip injected `<system-reminder>` boilerplate, so under harnesses
that inject context the "query" is the boilerplate rather than the
user's question). Fixing that belongs in its own PR because it moves
SmartCrusher's keep/drop scoring on every provider path and needs a
before/after quality measurement. Collapsing to one call site first
means that fix lands in one place.
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: replace the inline 12-line
reversed-scan in the proactive-expansion block with
`extract_user_query(messages, latest_user_turn_only=True)`.
- `headroom/utils.py`: add `latest_user_turn_only` to
`extract_user_query`.
The flag exists to preserve behaviour exactly. `extract_user_query`
walks back to *earlier* user turns when the newest one yields no text;
the expansion path must not. On a `tool_result` continuation turn the
newest user turn carries no text at all, and scoring relevance against
an older turn's question would expand context for a stale intent — i.e.
without the flag this refactor would make expansion fire on turns where
it previously stayed silent.
One deliberate difference remains: within the chosen turn the helper
skips empty text blocks rather than committing to the first block it
sees, so an empty leading text block no longer suppresses expansion
outright. That is a no-op whenever the first text block has content,
which is the normal case.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_utils.py tests/test_proxy_ccr.py -q
======================== 25 passed, 3 warnings in 5.08s ========================
$ pytest tests/ -q -k "ccr or proactive or expansion or utils or memory_query or extract"
=== 2 failed, 782 passed, 22 skipped, 9409 deselected, 43 warnings in 43.49s ===
$ ruff check headroom/utils.py headroom/proxy/handlers/anthropic.py tests/test_utils.py
All checks passed!
$ ruff format --check headroom/utils.py headroom/proxy/handlers/anthropic.py tests/test_utils.py
3 files already formatted
$ mypy headroom/utils.py --ignore-missing-imports
Success: no issues found in 1 source file
```
The 2 failures are
`tests/test_cli_proxy_env.py::TestCLICompressionOnlyFlags::{test_ccr_defaults_on,test_no_ccr_flag}`
and are **pre-existing and unrelated** — they fail identically with this
change stashed, and pass under a clean `HOME`. The real
`~/.headroom/settings.json` leaks into those config tests.
Two new tests pin the flag: that it returns `""` on a `tool_result`
continuation turn where the default walks back to `"the original
question"`, and that it agrees with the default whenever the newest user
turn does carry text.
## Real Behavior Proof
The contract for a refactor like this is "the proxy makes the same
decisions". So it was verified by running the real proxy on both `main`
and this branch and diffing every observable.
- Environment: macOS 15 (arm64), Python 3.13.14. `headroom proxy --mode
token --backend anthropic --anthropic-api-url http://127.0.0.1:<stub>`,
isolated `HOME` per run so neither could touch a developer's live
logs/store, `HF_HOME` pointed at cached kompress weights so the
lossy+CCR-marker path is exercised and expansion can actually arm.
- Exact command / steps: three requests over one conversation through
the real HTTP path (`x-headroom-cwd: /tmp/proof`, `user-agent:
claude-code/1.4.2`): a user turn carrying the real
`~/.claude/rules/*.md` text (~8.2k tokens); then `assistant` + a short
user turn so that block becomes compressible and gets tracked as a CCR
entry; then a follow-up whose leading text block shares vocabulary with
it so `analyze_query` scores above threshold. A stub upstream recorded
every forwarded body. Identical request sequence against both builds.
- Observed result: identical on every axis — this branch resolves the
same query and therefore reaches the same expansion decision.
| observable | `main` | this branch |
|---|---|---|
| expanded hash | `f0cf4efb42373ec225f57725` |
`f0cf4efb42373ec225f57725` |
| items expanded | 1417 | 1417 |
| expansion firings in run | 2 | 2 |
| `tok_before` → `tok_after` per turn | 8170→9553, 8184→9567, 8245→11880
| 8170→9553, 8184→9567, 8245→11880 |
| forwarded body bytes upstream | 43,507 / 43,637 / 54,130 | 43,507 /
43,637 / 54,130 |
(The growth visible in those numbers is the pre-existing expansion
behaviour on `main`, unchanged here by design — it is what #2706
addresses. This PR only proves it does not *move*.)
- Not tested: the `tool_result` continuation turn — the case
`latest_user_turn_only` exists for — is covered by unit test rather than
by the live run above, since the scenario needs a turn whose newest user
message has no text block. The remaining intentional delta (empty
leading text block) was likewise not exercised against real traffic; it
cannot trigger while the first text block has content. Streaming
responses were not exercised (the stub replies non-streaming), and the
OpenAI/Gemini/batch call sites of `extract_user_query` were not run —
this change adds a keyword-only parameter with a default that preserves
their behaviour, but they were verified by test suite only.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A — proxy-internal change, no user-facing surface.
## Additional Notes
- Independent of #2706 despite touching the same feature area — verified
`conflicts=0` via `git merge-tree`; either can merge first.
- The follow-up this unblocks: `extract_user_query` does not strip
injected `<system-reminder>` boilerplate, so under harnesses that inject
context the resolved "query" is the boilerplate rather than the user's
question. That defect is shared by every consumer of the helper
(SmartCrusher relevance scoring on the Anthropic, OpenAI, Gemini and
batch paths), which is exactly why it wants a dedicated PR with a
before/after compression-quality measurement rather than riding along
here.
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -2628,19 +2628,11 @@ class AnthropicHandlerMixin:
|
||||
and self.config.ccr_proactive_expansion
|
||||
and ccr_workspace_key
|
||||
):
|
||||
# Extract user query from messages
|
||||
user_query = ""
|
||||
for msg in reversed(messages):
|
||||
if msg.get("role") == "user":
|
||||
content = msg.get("content", "")
|
||||
if isinstance(content, str):
|
||||
user_query = content
|
||||
elif isinstance(content, list):
|
||||
for block in content:
|
||||
if isinstance(block, dict) and block.get("type") == "text":
|
||||
user_query = block.get("text", "")
|
||||
break
|
||||
break
|
||||
# Relevance query for this turn. Same helper the compression
|
||||
# pipeline scores with, so both read the query the same way;
|
||||
# `latest_user_turn_only` keeps the previous behaviour of not
|
||||
# reaching back past the newest user turn.
|
||||
user_query = extract_user_query(messages, latest_user_turn_only=True)
|
||||
|
||||
if user_query:
|
||||
recommendations = self.ccr_context_tracker.analyze_query(
|
||||
|
||||
+14
-1
@@ -49,12 +49,23 @@ def fast_hash(data: str | bytes, length: int = 16) -> str:
|
||||
return hashlib.md5(data).hexdigest()[:length] # nosec B324
|
||||
|
||||
|
||||
def extract_user_query(messages: list[dict[str, Any]]) -> str:
|
||||
def extract_user_query(
|
||||
messages: list[dict[str, Any]], *, latest_user_turn_only: bool = False
|
||||
) -> str:
|
||||
"""Extract the most recent user question from messages.
|
||||
|
||||
Used to pass context through the compression pipeline so transforms like
|
||||
SmartCrusher can score items by relevance to the user's actual question,
|
||||
not just by statistical properties (position, anomaly, boundary).
|
||||
|
||||
Args:
|
||||
messages: Conversation messages, oldest first.
|
||||
latest_user_turn_only: Stop at the newest user message instead of
|
||||
walking back through earlier ones when it yields no text. Callers
|
||||
that treat the query as "what this turn is about" want this: on a
|
||||
tool_result continuation turn the newest user message carries no
|
||||
text, and resurrecting an older turn's question would make them
|
||||
act on a stale intent.
|
||||
"""
|
||||
for msg in reversed(messages):
|
||||
if msg.get("role") == "user":
|
||||
@@ -67,6 +78,8 @@ def extract_user_query(messages: list[dict[str, Any]]) -> str:
|
||||
text = str(block.get("text", "")).strip()
|
||||
if text:
|
||||
return text
|
||||
if latest_user_turn_only:
|
||||
return ""
|
||||
return ""
|
||||
|
||||
|
||||
|
||||
@@ -46,6 +46,33 @@ def test_extract_user_query_and_message_hashes() -> None:
|
||||
assert utils.extract_user_query(messages) == "latest question"
|
||||
assert utils.extract_user_query([{"role": "assistant", "content": "skip"}]) == ""
|
||||
|
||||
|
||||
def test_extract_user_query_latest_user_turn_only() -> None:
|
||||
"""The flag stops at the newest user turn instead of resurrecting an older one."""
|
||||
# A tool_result continuation: the newest user turn carries no text at all.
|
||||
messages = [
|
||||
{"role": "user", "content": "the original question"},
|
||||
{"role": "assistant", "content": "calling a tool"},
|
||||
{"role": "user", "content": [{"type": "tool_result", "content": "output"}]},
|
||||
]
|
||||
|
||||
# Default walks back and finds the older turn's question.
|
||||
assert utils.extract_user_query(messages) == "the original question"
|
||||
# Scoped to this turn, there is no question to score against.
|
||||
assert utils.extract_user_query(messages, latest_user_turn_only=True) == ""
|
||||
|
||||
|
||||
def test_extract_user_query_latest_turn_only_agrees_on_normal_turns() -> None:
|
||||
"""With text present on the newest user turn, the flag changes nothing."""
|
||||
messages = [
|
||||
{"role": "user", "content": "older question"},
|
||||
{"role": "assistant", "content": "answer"},
|
||||
{"role": "user", "content": [{"type": "text", "text": " newest question "}]},
|
||||
]
|
||||
|
||||
assert utils.extract_user_query(messages) == "newest question"
|
||||
assert utils.extract_user_query(messages, latest_user_turn_only=True) == "newest question"
|
||||
|
||||
hash_one = utils.compute_messages_hash(messages)
|
||||
hash_two = utils.compute_messages_hash(list(messages))
|
||||
assert hash_one == hash_two
|
||||
|
||||
Reference in New Issue
Block a user