mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 18:41:07 +08:00
feat/https-compose-caddy
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
0f3b0d8a21 |
fix(memory): resolve pi cwd before injecting project tools (#4618)
## Description Closes #4442. pi's system `<cwd>` block now resolves the memory project. If project scope is still unresolved with the `empty` fallback, the Anthropic handler no longer advertises unusable memory tools, including tools recorded earlier in the session. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Parse pi cwd only in system text. Keep main's line parser, header/CLI precedence and limited user-message fallback. - Reuse `is_project_unresolved` for context and sticky-tool replay. Client tools and global/user storage behavior stay intact. - Add initialized-handler scope regressions, replace source-inspection/forwarding-only checks, and document project scope in the published memory page. ## Testing - [x] Unit tests pass (`pytest`) — related suite and the normal critical gate, not the entire Python suite locally. - [x] Linting passes (`ruff check .`) — hosted lint passed on 5ace63e2; local filesystem limitation below. - [x] Type checking passes (`mypy headroom`) — normal full hook with its existing `--ignore-missing-imports` setting. - [x] New tests added for the scope regression - [x] Manual testing performed ### Test Output ```text # 16 related test files 329 passed, 12 skipped, 1 warning in 43.07s # ordinary pre-push: make ci-precheck 178 passed, 5 skipped, 1 warning in 10.87s ✅ ci-precheck PASSED — safe to push. uv run ruff format --check . 1860 files already formatted ``` Normal commit hooks (Ruff, formatter, full mypy, commitlint) and the complete pre-push path passed: Rust fmt/clippy/workspace tests, native build/import verification, critical Python tests and commitlint. The existing Starlette test-client warning was not filtered. Local whole-tree Ruff reports EXE002 on five **unchanged** OAuth plugin files: my `fuseblk` checkout exposes them as executable despite Git modes `100644`; `chmod` returns zero without changing the mode. The uncached check agrees. I did not change those files, add shebangs or ignore the rule. Hosted Ruff, formatting and full mypy passed on 5ace63e2; the local filesystem limitation remains. The first full CI caught a missing scope predicate on the existing lifecycle test double. Fixed in [ |
||
|
|
f25fcc58cd |
feat(proxy): demote cache-mode frozen-prefix restore log from WARNING to INFO (#4635)
## Description
In `--mode cache`, restoring the frozen prefix to preserve prompt cache
stability is the expected routine behavior across turns. However,
`headroom/proxy/handlers/openai.py` logged this occurrence at `WARNING`
level:
```python
logger.warning(
f"[{request_id}] Restored {restored_count} frozen prefix message(s) "
"to preserve cache stability (openai)"
)
```
On production gateways, this routine log line accounted for thousands of
warnings daily (approx. 65% of total container WARNING volume),
generating significant log noise and obscuring actual anomalies during
incident triage (as reported in #4505).
This PR demotes the log line from `WARNING` to `INFO`, matching the
convention of routine cache-mode stability logging in sibling handlers
and preserving all details (`request_id`, `restored_count`).
Closes #4505
## Type of Change
- [x] Performance improvement / log signal-to-noise improvement
- [x] Code quality / maintenance
## Changes Made
- `headroom/proxy/handlers/openai.py`:
- Changed `logger.warning` to `logger.info` when `restored_count > 0` in
`is_cache_mode` restore branch.
- `tests/test_proxy_system_prompt_immutable.py`:
- Added `test_cache_mode_frozen_prefix_restore_logs_at_info_level` to
verify that prefix restoration logs at `INFO` and produces zero
`WARNING` logs.
## Testing
- [x] Unit tests pass:
- `tests/test_proxy_system_prompt_immutable.py`: 9 passed
- `tests/test_proxy_handler_helpers.py`: 87 passed
- [x] Linting passes: `ruff check` on modified files
- [x] Formatting passes: `ruff format --check` on modified files
- [x] New regression test added covering log level verification
### Test Output
```text
tests/test_proxy_system_prompt_immutable.py ......... [100%]
======================= 9 passed, 3 warnings in 8.26s =======================
```
## Real Behavior Proof
- Environment: Windows, Python 3.12.16, offline environment.
- Exact command / steps:
1. Ran `TestClient` post request against `/v1/chat/completions` with
proxy configured in `mode="cache"`, `optimize=True`, `frozen_count=1`.
2. Captured logging output via pytest `caplog` fixture set to
`logging.INFO` on `headroom.proxy.handlers.openai.logger`.
3. Evaluated emitted log records when pipeline mutation triggers
`_restore_frozen_prefix`.
- Observed result:
- Exactly 1 record emitted with `levelno == logging.INFO` matching
`Restored 1 frozen prefix message(s) to preserve cache stability
(openai)`.
- Zero log records emitted at `WARNING` or higher for prefix
restoration.
- Not tested: Live production gateway log ingestion pipelines.
## Runtime Rollout Safety
- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: yes; routine log level demoted from
WARNING to INFO.
- Kill switch / disable path: N/A (revert commit)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I did not edit `CHANGELOG.md`
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4635"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
### Maintainer post-merge runtime evidence
Reviewed combined head `8f82977fd23f6d254ea4ebe5dce60f44d94dc485`,
including main `365f6fe52`. All 11 combined immutable-prompt/cache-mode
tests pass. The logging regression independently fails on current main
because it emits the previous WARNING record.
Runtime setup: Windows, Python 3.12.13, patched worktree source and the
existing local native extension. A real Uvicorn proxy and a separate
local HTTP provider ran on ephemeral loopback ports. Production HTTP
forwarding was retained; `_retry_request` was not patched. A
deterministic instrumented pipeline deliberately changed the frozen
system text, and a one-message frozen-prefix tracker exercised the real
restoration path.
Exact command from the isolated worktree, with
`PYTHONPATH=G:/git/headroom-review4635` and `HEADROOM_OFFLINE=1` set
through PowerShell environment assignments:
`G:/git/headroom/.venv/Scripts/python.exe
G:/git/headroom/.codex/review4635-loopback.py`.
Observed output: `{"status":200,"forwarded_system":"frozen
system","restore_records":1,"restore_level":"INFO"}`. The real upstream
received the restored original system text; exactly one restoration log
was emitted at INFO, with no restoration WARNING. The proxy and local
provider were shut down after the probe.
Not tested: production log aggregation, an external provider, production
compression/model performance, or a fresh local native compilation. The
injected deterministic pipeline is a trigger for restoration, not a
claim about a production compressor. Updated-head hosted qualification
is handled by maintainers; no contributor CI action is needed. Nothing
merged.
### Maintainer current-main requalification
Exact head `c32d16e0a762748c3d21c113d42a65f9d8ef9dc3`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.
Clean mechanical update from current upstream main. The production delta
remains exactly the frozen-prefix restoration log level, WARNING to
INFO; the message, request ID, restored count, original prefix
restoration and unrelated warnings are preserved. Thirteen cache-prompt
tests pass, and the logging regression independently fails with the main
WARNING implementation before restoring this PR's source. Ten shared
Gemini request-fixture cases pass after the test-only repair for
independently verified main failures. Changed-file Ruff/format and
whitespace checks pass.
The real TCP Uvicorn-to-loopback-provider probe on this combined tree
returns HTTP 200, forwards the restored original system text, and
records exactly one INFO restoration message. It uses a controlled
mutating pipeline/session tracker to exercise restoration; it does not
qualify model-based compression, live external providers or production
log ingestion. The existing native binary was supplied only for startup,
not as a fresh native-build claim. No new correctness finding and no
contributor production logic was rewritten.
Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.
### Maintainer current-main requalification
Exact head `5b3d67e97a7b6c7b7aabf75005784420187c51a7`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.
Clean main synchronization with upstream
|
||
|
|
36540379e0 |
fix(proxy): skip memory-context search on OpenAI paths in cache mode (#4634)
## Description Follow-up to #4606. In `--mode cache`, the proxy preserves prompt cache stability across turns by never mutating or appending to user turns. While #4606 addressed this for Anthropic's `/v1/messages` by skipping `search_and_format_context`, the three OpenAI-compatible paths (`/v1/chat/completions`, `/v1/responses`, and `/v1/responses` WebSocket) still executed the memory search and injected memory context into the user turn even in cache mode. This PR aligns the OpenAI handlers with Anthropic: when in cache mode with `inject_context` enabled, it logs `skipped_cache_mode`, stamps `memory_skip_reason="cache_mode"` in the request/frame tags, and skips `search_and_format_context`. Memory tool injection and user input bytes remain unchanged. Closes #4622 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement ## Changes Made - `headroom/proxy/handlers/openai.py`: - `handle_openai_chat`: skip `search_and_format_context` when `is_cache_mode` is active, stamp `tags["memory_skip_reason"] = "cache_mode"`, log `decision="skipped_cache_mode"`, and keep the latest user message unmutated. - `handle_openai_responses`: skip `search_and_format_context` when `is_cache_mode` is active, stamp `tags["memory_skip_reason"] = "cache_mode"`, and log `decision="skipped_cache_mode"`. - `handle_openai_responses_ws`: skip `search_and_format_context` when `is_cache_mode` is active, stamp `ws_tags["memory_skip_reason"] = "cache_mode"`, log `decision="skipped_cache_mode"`, and avoid input mutation. - `tests/test_proxy_system_prompt_immutable.py`: - Updated `test_cache_mode_skips_memory_context_search_openai_chat` to verify `memory_skip_reason="cache_mode"` on recorded outcomes and preservation of memory tools. - Updated `test_cache_mode_skips_memory_context_search_openai_responses` to verify `memory_skip_reason="cache_mode"` on recorded outcomes and preservation of memory tools. - `tests/test_codex_ws_per_frame_memory.py`: - Expanded `test_cache_mode_skips_memory_context_search_ws` with multiple response.create frames verifying input preservation, memory tool preservation, and `memory_skip_reason="cache_mode"` in turn outcome tags. ## Testing - [x] Unit tests pass: - `tests/test_proxy_system_prompt_immutable.py`: 10 passed - `tests/test_codex_ws_per_frame_memory.py`: 12 passed - [x] Linting passes: `ruff check` on modified files - [x] Formatting passes: `ruff format --check` on modified files - [x] New regression tests added for all 3 OpenAI paths in cache mode - [x] Manual testing performed ### Test Output ```text tests/test_proxy_system_prompt_immutable.py .......... [ 45%] tests/test_codex_ws_per_frame_memory.py ............ [100%] ======================= 22 passed, 4 warnings in 18.06s ======================= ``` ## Real Behavior Proof - Environment: Windows, Python 3.12.16, `HEADROOM_OFFLINE=1`. - Exact command / steps: 1. Ran `pytest tests/test_proxy_system_prompt_immutable.py` exercising `/v1/chat/completions` and `/v1/responses` in cache mode with mock memory handler and outcome recorder. 2. Ran `pytest tests/test_codex_ws_per_frame_memory.py -k test_cache_mode_skips_memory_context_search_ws` exercising two `response.create` frames in cache mode with mock WebSocket and outcome recorder. 3. Checked that `memory_handler.search_and_format_context` is never awaited on any path. 4. Checked that `memory_skip_reason == "cache_mode"` reaches `RequestOutcome.tags` across Chat, Responses, and WebSocket turns. 5. Checked that input bytes and client/memory tools (`client_tool`, `mem_search`) remain untouched in the forwarded payloads. - Observed result: - `memory_handler.search_and_format_context.assert_not_awaited()` passed on all routes. - `outcome.tags.get("memory_skip_reason") == "cache_mode"` passed on Chat, Responses, and WS frames. - Forwarded inputs matched raw client inputs without injected memory context. - Forwarded tools retained `client_tool` and memory tools (`mem_search`). - 22 tests passed across both test files. - Not tested: Live cloud provider endpoints. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: yes; in cache mode, memory context is no longer injected into OpenAI requests, ensuring next-turn prefix bytes remain byte-identical, and request/frame tags now record `memory_skip_reason="cache_mode"`. - Kill switch / disable path: N/A (revert commit) - Unsafe override required: no - Qualification impact: none - Rollback path: revert this PR ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I did not edit `CHANGELOG.md` --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
365f6fe523 |
fix(proxy): skip memory-context search in cache mode on OpenAI routes (#4632)
## Description Follow-up to #4600 / #4606 as requested in #4622. PR #4606 implemented this optimization for Anthropic `/v1/messages`. This PR brings the three OpenAI routes (`handle_openai_chat`, `handle_openai_responses`, and `handle_openai_responses_ws`) into alignment. In `--mode cache`, memory context injection is disabled so prompt prefix bytes remain byte-stable across turns. Previously, all three OpenAI paths still executed the memory-context search (`search_and_format_context`: embedding query, vector index lookups, graph searches) and appended the result to the latest user turn in cache mode, introducing latency (~1–37ms+ per turn) and altering the forwarded user turn. With this PR: 1. `handle_openai_chat`: checks `is_cache_mode(self.config.mode)` before executing `search_and_format_context`, emits `decision="skipped_cache_mode"`, and keeps the user turn intact. 2. `handle_openai_responses`: checks `is_cache_mode(self.config.mode)` before searching, emits `decision="skipped_cache_mode"`, and keeps `body["input"]` intact. 3. `handle_openai_responses_ws`: checks `is_cache_mode(self.config.mode)` before measuring `memory_context`, emits `decision="skipped_cache_mode"`, and leaves `ws_response_body["input"]` untouched. 4. Memory tool injection remains completely unchanged across all routes. Closes #4622 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement ## Changes Made - `headroom/proxy/handlers/openai.py`: Check `is_cache_mode(self.config.mode)` when `inject_context` is enabled across `handle_openai_chat`, `handle_openai_responses`, and `handle_openai_responses_ws`. Emit `decision="skipped_cache_mode"` with `bytes_injected=0` and bypass `search_and_format_context`. - `tests/test_proxy_system_prompt_immutable.py`: Added `test_cache_mode_skips_memory_context_search_openai_chat` and `test_cache_mode_skips_memory_context_search_openai_responses` to verify that `search_and_format_context` is never awaited in cache mode and user message / input fields remain byte-equal to client values. ## Testing - [x] Unit tests pass (`pytest tests/test_proxy_system_prompt_immutable.py`) - [x] Linting passes (`ruff check headroom/proxy/handlers/openai.py tests/test_proxy_system_prompt_immutable.py`) - [x] Formatting passes (`ruff format --check headroom/proxy/handlers/openai.py tests/test_proxy_system_prompt_immutable.py`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_proxy_system_prompt_immutable.py -q 10 passed, 1 warning in 31.92s $ ruff check headroom/proxy/handlers/openai.py tests/test_proxy_system_prompt_immutable.py All checks passed! $ ruff format --check headroom/proxy/handlers/openai.py tests/test_proxy_system_prompt_immutable.py 2 files already formatted ``` ## Real Behavior Proof - Environment: Linux / WSL2 Ubuntu, Python 3.13, local proxy test harness (`create_app` + `TestClient`). - Exact command / steps: Executed requests against `/v1/chat/completions` and `/v1/responses` with proxy configured in `--mode cache` and memory context enabled. Verified upstream payload captures and mock call history. - Observed result: In cache mode, the memory search is bypassed on all OpenAI requests; search_and_format_context was never awaited (0 calls), user message content and input fields remain byte-equal without injected memory context, and proxy emitted decision=skipped_cache_mode with bytes_injected=0. - Not tested: Live WebSocket connection to OpenAI upstream with streaming audio/realtime endpoints. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: no. Cache mode requires byte stability across turns; memory search is skipped in cache mode. Token mode is untouched. - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: git revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation: n/a - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - Aligns OpenAI Chat, Responses, and WebSocket routes with Anthropic behavior established in #4606. - Preserves memory tool injection without modification. Co-authored-by: shaiknaziya20 <naziyashaik5512@gmail.com> ### Maintainer combined-head validation Reviewed head `66a09054548423e33af47121ac7e87134d023fe4` includes a clean main sync through `12dca26aa` plus test-only maintainer follow-ups: the existing WebSocket dummy config now explicitly uses token mode, and a two-frame cache-mode regression covers skipped searches, unchanged input, retained memory tools, passthrough cancellation frames, and skip telemetry. No contributor runtime logic was changed. Windows / Python 3.12.13, with the existing local Rust extension: `.venv/Scripts/python.exe -m pytest tests/test_proxy_system_prompt_immutable.py tests/test_codex_ws_per_frame_memory.py tests/test_openai_codex_ws_lifecycle.py -q --tb=short --disable-warnings` => **89 passed**. The new WebSocket regression fails on current main because both frames still execute memory searches. Ruff check/format and main-relative whitespace checks pass. Additional runtime proof used a real Uvicorn proxy and a separate local HTTP upstream, both bound to ephemeral loopback ports, with production HTTP forwarding intact (no `_retry_request` patch). A deterministic recording memory backend returned `LOOPBACK_MEMORY` when searched, so the probe can distinguish an actual skip from an unconfigured/disabled memory path. The probe sent the same authenticated Chat and Responses requests in token mode, then cache mode, using user text `latest user prompt` and fixed Responses instructions. Exact invocation from the isolated worktree: `PYTHONPATH=G:/git/headroom-review4632 HEADROOM_OFFLINE=1 G:/git/headroom/.venv/Scripts/python.exe G:/git/headroom/.codex/review4632-loopback.py` (environment variables set with PowerShell `$env:` assignments). Results: **four HTTP 200 responses**; token mode performed one memory search per route and forwarded the returned memory text; cache mode performed zero searches per route, forwarded the original user text unchanged, and recorded `memory_skip_reason=cache_mode`. The proxy and upstream were shut down after the run. Limits: the loopback provider and recording memory backend are deterministic local fixtures; no external provider, embedding model, production memory service, load test, or real upstream WebSocket was exercised. WebSocket multi-frame/input/tool/telemetry behavior is covered by the focused regression above. Updated-head hosted qualification is inspected by maintainers; no contributor CI action is needed. Nothing merged. --------- Co-authored-by: shaiknaziya20 <naziyashaik5512@gmail.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
053925b03c |
fix(proxy): skip the memory-context search in cache mode (#4606)
## Description
In `--mode cache`, the Anthropic `/v1/messages` handler ran the full
memory-context search (`search_and_format_context`: embed the query,
query the vector index and graph) on every request, and only then
checked `is_cache_mode()` and discarded the result with
`decision=skipped_cache_mode`. Cache mode never injects memory context,
so the search was pure latency before the request goes upstream.
This PR checks cache mode first and skips the search entirely. The
`skipped_cache_mode` decision is still logged, and memory tools are
still injected.
Closes #4600
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: when `inject_context` is on
and the proxy is in cache mode, log `skipped_cache_mode` and skip
`search_and_format_context`. The old post-search cache-mode branch is
removed.
- Corrected the gate comment: `decide_memory_injection` only skips for
`HEADROOM_MEMORY_INJECTION_MODE=disabled`, not `tool` (as noted in the
issue).
- `tests/test_proxy_system_prompt_immutable.py`: added
`test_cache_mode_skips_memory_context_search`, which sends a cache-mode
request with memory on and asserts the search is never awaited and the
latest user turn is forwarded unchanged.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`): see Additional Notes
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
New test, before the fix:
```text
E AssertionError: Expected mock to not have been awaited. Awaited 1 times.
FAILED tests/test_proxy_system_prompt_immutable.py::test_cache_mode_skips_memory_context_search
```
After the fix:
```text
$ pytest tests/test_proxy_system_prompt_immutable.py -q
8 passed
$ pytest tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_memory_integration.py \
tests/test_memory_tool_mode.py tests/test_memory_decision.py \
tests/test_anthropic_pre_upstream_backpressure.py tests/test_proxy_pipeline_lifecycle.py \
tests/test_proxy_byte_faithful_forwarding.py -q
170 passed, 12 skipped
$ ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_system_prompt_immutable.py
All checks passed!
```
## Real Behavior Proof
- Environment: macOS, Python 3.12, local memory backend
(`all-MiniLM-L6-v2` ONNX embedder, hnswlib vector index),
`memory_storage_mode=global` with 50 seeded memories.
- Exact command / steps: an in-process benchmark that starts the real
proxy app (`create_app` + `TestClient`) with `--mode cache` and memory
enabled, stubs only the upstream call, and sends 100 `/v1/messages`
requests. It counts calls to `search_and_format_context` and times each
request end to end. Two alternating rounds of `main` (
|
||
|
|
9553d4292f |
fix(cache): retain prefix trackers for the provider cache lifetime (#4023)
## Description Session cleanup currently drops prefix trackers after the default 600-second session lifetime even when Anthropic's request selects a one-hour cache TTL. Losing the tracker also loses the frozen forwarded prefix and causes the cache-preserving overlay to replay different bytes. Closes #4017. ## Type of Change - [x] Bug fix - [x] Documentation update ## Changes Made - Resolve the request's existing Anthropic cache TTL before selecting the prefix lineage and pass that TTL through tracker selection/creation. - Retain each lineage for the greater of the configured session floor and its observed/current provider TTL plus the existing 60-second safety margin. - Keep the lifetime per lineage rather than changing global configuration: a five-minute session does not inherit another session's one-hour lifetime. - Preserve normal expiry beyond the provider lifetime and margin. - Honor configured cache TTL before a confirmed cache write; discard expired state before accepting a later request's longer TTL. - Re-register the live lineage family after lookup/cleanup so recreated sessions cannot share a tracker's frozen state across sibling conversations or tool profiles. - Update existing fake callbacks to the changed tracker contract, preserve all consumer byte-stability assertions, and remove one handler-to-store wiring-only test rather than re-pinning its internal identity assertions. - Document local tracker retention separately from provider cache availability. ## Testing - [x] Unit tests pass (`pytest`), including frozen-prefix byte stability and independent TTL expiry. - [x] Changed-file lint, configured pre-commit mypy and manual production timestamp/overlay checks pass. ### Test Output ```text Review correction: 273 passed, 2 dependency warnings in 83.79s {"expires_beyond_provider_ttl_and_margin": true, "five_minute_session_evicted": true, "forwarded_prefix_byte_exact": true, "initial_frozen": 1, "one_hour_session_retained": true, "resolved_ttl_seconds": 3600, "resumed_frozen": 1, "same_tracker_after_660s": true} ``` The mandatory pre-push `make ci-precheck` passed: workspace Rust fmt/clippy/tests, native extension build/import verification, the complete configured Python gate, and Commitlint. The native gate explicitly used the installed CPU `libonnxruntime.so.1.26.0` via `ORT_DYLIB_PATH`, without skipping cached-model assertions. ```text [build_rust_extension] headroom._core build + install + verify: OK ================== 177 passed, 4 skipped, 1 warning in 12.87s ================== ✅ ci-precheck PASSED — safe to push. ``` ```sh python -m pytest -q tests/test_cache/test_prefix_tracker.py python -m pytest -q tests/test_proxy_anthropic_cache_stability.py tests/test_cache_prefix_overlay.py tests/test_cache_ttl_preserved.py tests/test_cold_prefix.py ``` **74 passed** in tracker tests; **54 passed** in request/cache tests (one existing Starlette TestClient deprecation warning). Two new regressions fail on baseline because the request TTL is not accepted; a separate production runtime comparison demonstrates the actual pre-fix loss, not just the missing parameter. After rebasing onto current main `1ad92f0cbc3dea27676e253588100f64a2af6994`, ran those suites together with `tests/test_prefix_tracker_client_rewrote_reply.py`, `tests/test_cache_mode_delta_marker.py`, and `tests/test_content_router_mcp_small_result.py`: **150 passed**. The actual production timestamp/byte-retention smoke was repeated successfully on the rebased code, covering compatibility with main's newly merged client-reply normalization. Review regression and affected HTTP-consumer suite: **273 passed**, including tracker expiry, Anthropic/OpenAI cache stability, byte-faithful forwarding, header isolation, buffered timeouts, CCR injection, and immutable system prompts. All affected two-argument test callbacks now accept the TTL contract. The obsolete walker-call-order test was removed; wire-byte assertions remain. The new configured-TTL/expiry cases fail against the published baseline; the post-cleanup regression also verifies that an unrelated conversation gets a different tracker after recreation. Changed-file Ruff, formatting, installed pre-commit `mypy headroom`, and Commitlint hooks passed. Full-tree Ruff is locally blocked by executable filesystem permissions on five unchanged OAuth2 plugin files (tracked 100644). A separate full dev-environment mypy run found an unchanged optional LangChain `truthy-function` error; the configured isolated pre-commit hook passed. No unrelated source/permissions changed. ## Real Behavior Proof - Environment: Linux x86_64, CPython 3.13.14; unchanged `dea9092e19385f286b7f67a64f4d85de2708d756` and this branch; real `SessionTrackerStore`, `PrefixCacheTracker`, request TTL parser, token estimator and `overlay_cached_prefix`, no mocked store or mocked clock. - Exact command / steps: parse `cache_control: {type: ephemeral, ttl: 1h}` for `claude-sonnet-4-5`; establish one real cache-write-confirmed frozen message with different original/forwarded prefix bytes; age `_last_activity` to `time.time()-660` and make periodic cleanup due; resume the same lineage and compare the tracker, frozen count, and forwarded-prefix bytes. Also age beyond 3901 seconds and compare independent 300/3600-second sessions. - Before: resolved TTL 3600; after 660 seconds, tracker identity changes, frozen count drops **1→0**, and forwarded-prefix bytes differ. - Observed result: after the fix, after 660 seconds the same tracker remains, frozen count stays **1**, and forwarded-prefix bytes are identical. The one-hour session survives independently, the five-minute session expires, and the one-hour lineage still expires beyond TTL plus margin. - Review runtime proof against published head `e1a107df`: actual stores and prefix overlays confirm configured TTL retention before the first cache write, no resurrection of expired state by a later one-hour request, eventual expiry, and separate trackers after the maintainer's 3901-second cleanup/recreation sequence. On that exact baseline the unrelated lookup reuses the wrong tracker, although the overlay guard preserves the particular unrelated input bytes; after the fix both identity isolation and byte preservation hold. Identical messages under different tool profiles also receive a cold, separate tracker after the expired bare-key transition. - Not tested: Anthropic cloud cache availability, billed savings, or an actual one-hour wait. The local production state and byte invariants were exercised by aging actual timestamps, not by a paid upstream request. Preserving a tracker cannot guarantee that the provider retains its cache. ## Runtime Rollout Safety - Rollout-managed feature(s): none; repair of existing TTL-aware cache-prefix behavior. - Minimum rollout channel: stable; no rollout-managed feature is enabled. - Stable/default behavior changed: eligible local prefix trackers survive for their provider cache lifetime. Short-TTL/default sessions remain bounded by their own session floor. - Kill switch / disable path: the existing prefix-freeze disable path remains available; no new flag. - Unsafe override required: no. - Qualification impact: local Anthropic prefix lineage selection/retention, not cache pricing or subscription quota policy. - Rollback path: revert the commit to restore previous retention. No persistent schema change. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] Existing cache TTL parser and safety margin reused; cache docs updated. - [x] Regressions cover frozen-byte retention, expiry, mixed lifetimes, and existing request byte-stability behavior. - [x] `CHANGELOG.md` not edited; Conventional Commit title supplied. Remote CI results are not inferred from local verification. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
f0dcc02775 |
fix: A3 — byte-faithful Python forwarders; serialize canonical only when mutated
Eliminates P0-2 universally. Every Python forwarder (server.py
`_retry_request`, handlers/streaming.py `_stream_response`,
handlers/openai.py `_ws_http_fallback`, handlers/batch.py `_batch_passthrough`
+ batch-create + Google batch passthrough, handlers/anthropic.py CCR
continuation + batch endpoint) now switches from `httpx ... json=body` to
`httpx ... content=raw_bytes`. The default httpx JSON encoder was
re-serializing every request with `, `/`: ` separators and `\\uXXXX` ASCII
escapes — collapsing Anthropic prompt-cache hit-rate.
Forwarder strategy:
- unmutated body → forward `await request.body()` verbatim;
- mutated body → re-serialize once via the new
`serialize_body_canonical(body) -> bytes` helper (compact separators,
`ensure_ascii=False`, dict insertion order preserved).
`HEADROOM_PROXY_PYTHON_FORWARDER_MODE` env var configures the mode:
- `byte_faithful` (default) — the new behavior;
- `legacy_json_kwarg` — explicit operator opt-in for emergency rollback.
Documented in `docs/content/docs/configuration.mdx`. NOT a fallback —
unknown values raise loudly per build constraint #4.
`BodyMutationTracker` accompanies each request through the handler so
transform sites mark the tracker (`memory_injection`,
`image_compression`, `compression_*`, `batch_compression`,
`ccr_continuation`, etc.). At forwarder dispatch we additionally compare
the final body dict against the parsed original bytes as a structural
safety net — any silent mutation we missed still triggers canonical
re-serialization.
A2 follow-up: `handlers/openai.py:534-540` (Chat Completions memory
injection) was prepending a system message; replaced with
`append_text_to_latest_user_chat_message`, the OpenAI Chat Completions
analog of `_append_context_to_latest_non_frozen_user_turn`. The cache
hot zone (system messages) is now sacrosanct on /v1/chat/completions
too. Honors `HEADROOM_MEMORY_INJECTION_MODE=disabled`.
Structured logging: every forwarder emits an `event=outbound_request`
log line with `forwarder`, `path`, `body_bytes`, `body_mutated`,
`mutation_reasons`, `source` (passthrough|canonical|legacy),
`request_id`. Never logs Authorization or full body.
`_read_request_json` factored to share `_read_request_body_bytes` with
new `read_request_json_with_bytes` so the anthropic handler can capture
both the parsed dict and the original (decompressed) bytes.
Tests:
- `tests/test_proxy_byte_faithful_forwarding.py` (28 tests):
SHA-256 byte-equality on /v1/messages and streaming, unicode
preservation, numeric precision, mutation-tracker invariants,
canonical-serializer properties, legacy-mode rollback, OpenAI
Chat memory routing.
- Existing test mocks updated to accept the new `**kwargs` on
`_retry_request` (no behavior change).
- `tests/test_proxy_handlers_batch.py` updated to read the captured
`content=` bytes (formerly `json=`).
- One A2 test corrected (`test_anthropic_tool_sort_and_context_append_helpers`)
to match the live-zone-tail semantics introduced by A2.
Constraints satisfied: configurable env var; no new regex / hardcodes;
no silent fallback (`legacy_json_kwarg` is operator opt-in);
performant (`prepare_outbound_body_bytes` is O(1) for passthrough);
elegant single-responsibility helpers; structured tracing logs.
|
||
|
|
704fb2f19d |
fix: A2 — system prompt immutable; memory routes to live-zone tail; cache_aligner detector-only
P0-1: Delete `_inject_system_context` from `proxy/server.py`. Memory context now routes exclusively to the first text block of the latest non-frozen user message via `_append_context_to_latest_non_frozen_user_turn` (promoted to the canonical default in handlers/anthropic.py). Mirror applied to OpenAI Responses API at handlers/openai.py: `body["instructions"]` is no longer mutated; memory context appends to the latest user item in `body["input"]`. P2-23: Replace `headroom/transforms/cache_aligner.py` with a detector-only implementation. The legacy rewrite path (~400 LOC) is removed. The volatile- content detector uses no regex — UUIDs via `uuid.UUID`, ISO 8601 via `datetime.fromisoformat`, JWT shape via base64url segment-count check, hex hashes via length + `int(token, 16)` validation. Volatile findings surface through `cache_metrics`/`warnings`/`logger.warning`; the prompt is never mutated. Configurability: new env var `HEADROOM_MEMORY_INJECTION_MODE` with values `live_zone_tail` (default) and `disabled`. No `system_prompt` value — that path is permanently retired. Structured logs: every memory injection emits `event=memory_injection` with `decision`, `bytes_injected`, `query_hash` (BLAKE2b, never raw query), `session_id`, `request_id`. Auth is never logged. Tests: - Add `tests/test_proxy_system_prompt_immutable.py` (7 tests). - Add `tests/test_cache_aligner_detector_only.py` (20 tests). - Replace `tests/test_transforms/test_cache_aligner.py` (rewrite-path tests, 58 cases) with detector-only behavior. - Update `tests/test_acceptance.py::TestDateTrap` to pin the new detector-only contract. Acceptance: - `git grep -n "_inject_system_context\|_inject_to_system_or_instructions" headroom/` returns nothing. - `git grep -n "import re\|from re import" headroom/transforms/cache_aligner.py` returns nothing. - Targeted suite (`test_proxy_system_prompt_immutable.py`, `test_cache_aligner_detector_only.py`, `test_proxy_anthropic_cache_stability.py`, `test_acceptance.py::TestDateTrap`, `test_memory*.py`, `test_cli/`) green. |