mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:06:14 +08:00
feat/https-compose-caddy
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
16eaec1e95 |
fix(cache): keep prefix lineage across a replaced system tail (#3934)
## Description Claude Code can end a `/v1/messages` request with a temporary `role:"system"` reminder and replace it on the next turn. `SessionTrackerStore.resolve_tracker` records the full history, so the stored chain ends in the old reminder and is never a prefix of the next request: `_classify_history_canonical` returns `diverged` and the request gets a fresh, empty tracker. The confirmed frozen prefix and the previously forwarded messages are gone, the #3380 replay has nothing to replay, and any recompressed history re-writes the provider prompt cache. The reminder moves every turn, so this repeats for the whole session. This change lets lineage resolution set aside that one replaced trailing system message, only when nothing matched as recorded. It changes tracker selection only: forwarded bytes, the replay body, and breakpoint placement are untouched, and the existing replay already stops at the first changed message, so the new reminder goes out as the client sent it. Closes #3933 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cache/prefix_tracker.py`: new `_extends_past_system_tail(current, previous)`. It is true when `previous` ends in a `role:"system"` message, everything before that message includes user/assistant history and is unchanged in `current`, and `current` is longer than `previous`. Because `current` must be longer, siblings that only swap their final instruction stay separate. - `SessionTrackerStore.resolve_tracker`: after the existing exact/append, block-append and block-rewrite passes find nothing (and there are no rewrite candidates, so an ambiguous rewrite tie is not overridden), Anthropic requests consider chains with the same cache affinity that pass that check. Only a unique longest candidate is taken; two equal-length candidates start a fresh lineage, as before. The full current snapshot is stored as usual. - `extract_cache_stable_delta` (cache mode's prefix/delta split): a `diverged` request that replaced the previous trailing system reminder is split too. Every recorded message before the reminder, plus the model reply the handlers record after it, must match the current request position for position (canonical compare, same rule via `_extends_past_system_tail`). The previously forwarded bytes are replayed without the old reminder, and everything after it is the delta to compress. Before this, cache mode tagged these turns `cache_mode_prefix_mismatch` and forwarded the new messages raw. - `HEADROOM_TRANSIENT_SYSTEM_LINEAGE=0` (also `false`/`no`/`off`) turns off both the lineage fallback and the delta split. It uses the same parsing as `HEADROOM_STABLE_BOUNDARY_BREAKPOINT`. - `tests/test_cache/test_prefix_tracker.py`: - Tracker retention: the tracker and its confirmed frozen count are kept, and the existing overlay restores the unchanged prefix. - A reminder that moves and changes over several turns: full snapshots are stored. - Isolation: different cache affinity, OpenAI provider, changed earlier history, changed leading system message, changed historical system message, same-length siblings `[user: task, system: A]` vs `[user: task, system: B]`, ambiguous equal-length siblings, exact-match precedence, and the kill switch. - The delta split: with and without the recorded reply, a system message followed by a client turn is not treated as a reminder, a mismatched reply is rejected, and the kill switch disables it. - `tests/test_cache_mode_cold_start.py`: drives `handle_anthropic_messages` in `mode="cache"` with the real `SessionTrackerStore` over three turns. The first turn compresses a tool result and later turns replace the reminder. The test asserts that the third request replays the second request's compressed prefix byte-for-byte (ignoring `cache_control`), compresses the new tool result, and ends in the current reminder. ## Testing - [x] Unit tests pass (`pytest`) - targeted files below, not the whole suite - [x] Linting passes (`ruff check .`) - on the changed files, plus `ruff format --check` - [x] Type checking passes (`mypy headroom`) - on `headroom/cache/prefix_tracker.py` - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output New tests with the source at `main` ( |
||
|
|
6e712672cb |
fix(anthropic): stop the spurious compaction warning and drop dead handler code (#3678)
## Description
Three small cleanups in `headroom/proxy/handlers/anthropic.py`, found
while reading the handler end to end. One is a real bug, two are dead
code.
**Spurious "system prompt compaction FAILED" on every request.** With
`HEADROOM_SYSTEM_COMPACT` enabled and no `ContentRouter` in the
pipeline, the handler read `_sys_modified`, a local that is only
assigned on the router branch. The resulting `UnboundLocalError` was
caught by the surrounding `except Exception` and logged as a compaction
failure on every request, even though nothing had failed. The flag is
now only read on the branch that sets it.
**No-op OpenAI fallback stub.** `ProxyConfig.fallback_enabled` /
`fallback_provider` gated a block in the Anthropic error path whose body
was two comments and a log line saying "Attempting fallback to OpenAI".
No conversion or retry ever existed, and nothing else in `headroom/`,
`tests/` or `docs/` references either field. Both fields and the block
are gone.
**Redundant pre-upstream semaphore releases.**
`handle_anthropic_messages` wraps its body in a `try/finally` that calls
the idempotent `_finalize_pre_upstream()` on every exit. Six explicit
calls placed directly before early `return`s (413, invalid JSON,
message-count 400, cache hit, security 403, Bedrock error) therefore did
nothing. They are removed. The calls that deliberately release *before*
a long await (streaming and backend handoffs) are kept, since those
change timing rather than correctness.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: move the `if _sys_modified:`
bookkeeping inside the router branch that assigns it, so the compaction
success/skip path no longer raises `UnboundLocalError` into the failure
log.
- `headroom/proxy/handlers/anthropic.py`: delete the
`fallback_enabled`/`fallback_provider` block in the upstream-error path;
it contained no executable fallback.
- `headroom/proxy/models.py`: delete the two unused `ProxyConfig`
fields. No loader, CLI flag, env var or doc maps to them.
- `headroom/proxy/handlers/anthropic.py`: drop six `await
_finalize_pre_upstream()` calls that sit immediately before an early
`return` already covered by the handler's `finally`.
- `tests/test_system_compaction_handler.py`: new
`test_system_compaction_without_content_router_is_not_a_failure`, which
fails on `main` with `cannot access local variable '_sys_modified'` and
passes here.
- `tests/test_system_compaction_handler.py`: also
`test_system_compaction_with_content_router_records_the_transform`,
which stubs the router lookup and `compact_system_prompt` so the success
branch (transform tag + debug log) is exercised; closes the codecov
partial on the moved `if _sys_modified:` line.
- `tests/test_anthropic_pre_upstream_backpressure.py`: extend the
early-exit semaphore test to the three validation paths it did not cover
(413, invalid JSON, message-count 400), so removing the explicit
releases is proven safe rather than assumed.
- `tests/test_anthropic_stage_timings.py`,
`tests/test_cache_mode_cold_start.py`,
`tests/test_cold_start_fast_pass.py`: stop setting the removed config
fields.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest -q tests/test_anthropic_pre_upstream_backpressure.py tests/test_system_compaction_handler.py \
tests/test_anthropic_stage_timings.py tests/test_cache_mode_cold_start.py tests/test_cold_start_fast_pass.py
41 passed, 2 warnings in 7.6s
# same files with the new test against upstream/main's handler:
FAILED tests/test_system_compaction_handler.py::test_system_compaction_without_content_router_is_not_a_failure
E 'system prompt compaction FAILED: cannot access local variable '_sys_modified' where it is not associated with a value'
$ ruff check headroom/proxy/handlers/anthropic.py headroom/proxy/models.py tests/test_anthropic_pre_upstream_backpressure.py tests/test_system_compaction_handler.py
All checks passed!
$ mypy headroom/proxy/handlers/anthropic.py headroom/proxy/models.py
Success: no issues found in 2 source files
$ grep -rn 'fallback_enabled\|fallback_provider' headroom/ tests/ docs/
(no output)
```
## Real Behavior Proof
- Environment: macOS 25.6, Python 3.12 venv with the repo's compiled
`headroom._core`, fresh worktree of `upstream/main` @ `67eb910e`, the
three commits cherry-picked on top.
- Exact command / steps: read `handle_anthropic_messages` from the
semaphore acquire through the outer `finally` and confirmed
`_finalize_pre_upstream()` is idempotent and runs in `finally` on every
exit; grepped every reference to `fallback_enabled` /
`fallback_provider`; ran the five touched test files on the branch and
again with the new compaction test against `main`'s handler; ran `ruff
check` and `mypy` on the touched source files.
- Observed result: 40 tests pass on the branch. The new compaction test
reproduces the spurious failure log on `main` and passes here. No
remaining references to the deleted config fields. Lint and type check
clean on the touched files.
- Not tested: no live proxy traffic. The compaction fix only changes
which branch reads a local, the fallback removal deletes code with no
behavior, and the semaphore change is covered by the extended early-exit
test rather than by observing a real backlog. `ruff format --check`
flags `anthropic.py` on `main` already; this PR does not reformat
unrelated lines.
## Runtime Rollout Safety
- Rollout-managed feature(s): `HEADROOM_SYSTEM_COMPACT` (env-gated, off
by default). No new gate is introduced.
- Minimum rollout channel: n/a. The fix only affects the log line and
`transforms_applied` bookkeeping when compaction is enabled and no
router is present; upstream requests were already unaffected.
- Stable/default behavior changed: no. With the env var unset, none of
the changed lines execute. `ProxyConfig(fallback_enabled=...)` would now
raise `TypeError`, but no code path, doc or config loader ever passed
it.
- Kill switch / disable path: unset `HEADROOM_SYSTEM_COMPACT` restores
the pre-existing behavior for the compaction path; the other two changes
remove dead code and have no runtime switch.
- Unsafe override required: no.
- Qualification impact: none. No compression, cache or provider behavior
changes.
- Rollback path: `git revert` of the three commits; they touch seven
files and no persisted state or schema.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
A fourth commit that unified the budget and rate-limit preflight across
all provider handlers was deliberately left out. Open PR #3579 already
enforces the spend budget on the OpenAI and Gemini routes with two
approvals, and #3375 targets the same issue #3374. The remaining
rate-limit-key unification will be proposed separately once #3579 lands.
Documentation checkbox is unticked because `fallback_enabled` /
`fallback_provider` appear in no documentation and no user-facing
behavior changed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
|
aaeba0a319 |
fix(proxy): compress cache-mode cold starts and tag prefix-mismatch passthrough (#2365)
## Description
Since v0.31.0 shipped cache mode as the default (
|