mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
feat/https-compose-caddy
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
8c07cf4a0f |
fix(memory): bound retrieval queries before embedding (#4244)
## Description Closes #4223. The proxy now embeds a bounded, user-first retrieval query instead of concatenating entire tool outputs into a deadline-limited lookup. The raw `MemoryQuery` and its default SDK rendering still retain the full sources. Memory failures use the exception representation, so an empty `TimeoutError` is no longer a blank warning. The existing Anthropic deadline is unchanged and documented; this does not promise that a cold model or slow embedding server will finish within it. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Documentation update ## Changes Made - Bound proxy embedding input to 4096 characters, including delimiters; spend the budget on the latest user question and recent context tails. - Reserve question and context space inside the active local tokenizer's window, rather than relying on a character cap to prevent truncation. - Keep the full-fidelity SDK behavior when no budget is requested. - Make generic memory errors identifiable in the shared handler and provider handlers. - Cover Unicode, newest-context precedence, zero character budget and actual tokenizer-window retention; keep the full-fidelity SDK tests. ## Testing - [ ] Unit tests pass (pytest) — all 67 affected query, memory-handler and tokenizer-window regressions passed locally, not the entire Python suite. - [ ] Linting passes (ruff check .) — normal changed-file Ruff hooks passed. - [ ] Type checking passes (mypy headroom) — normal mypy hook passed. - [x] Manual testing performed ### Test Output ```text Affected query/native-memory/tool/auto-tail/tokenizer suites: 67 passed, 1 warning Ordinary make ci-precheck: PASS (181 critical Python tests, 6 warnings) ``` The focused-suite warning concerns assertion rewriting of the already-imported `asyncio` name; it was not suppressed. All five installed pytest plugins were loaded explicitly. Normal Ruff formatting/checking, mypy and commitlint hooks passed on `8eb10bfd`. The ordinary full pre-push gate passed on that same commit, including Rust workspace checks/tests, the native build and critical Python suites; the normal fast-forward fork push completed without a bypass. ## Real Behavior Proof - Environment: Linux x86-64, Python 3.13.14, the actual local ONNX model/session, SQLite and vector retrieval. The optional sentence-transformers CPU backend was also exercised in a separate complete CPU-wheel environment; no project dependency changes. - Exact command / steps: save a SQLite-corruption memory and an unrelated cooking memory through the real `LocalBackend`. Call `MemoryHandler.search_and_format_context` under a two-second `asyncio.wait_for` with the same long repeated question and SQLite tool error before and after the change. Record the real ONNX session's input IDs/attention mask without replacing inference; decode those IDs to check the final question and tool error. Verify the original `MemoryQuery` fields stay intact. - Observed result: both fragments now reach the model and the relevant memory is retrieved. Same synthetic input and default cache behavior; no fake embedding or search result: | Measurement | Before | After | | --- | ---: | ---: | | Submitted characters | 2594 | 1146 | | Tokens before model truncation | 560 | 256 | | Question tail and tool error reach model | No | Yes | | Relevant memory retrieved | No | Yes | | Warm lookup seconds | 0.076 | 0.241 | A separately configured 96-token ONNX model also retained both fragments. The real sentence-transformers CPU model and a previously uninitialized local ONNX backend passed the same retention check and generated real embeddings. The legacy messages-only path still submitted 4096 characters; the empty-timeout diagnostic still logged `TimeoutError()` and failed open. - Not tested: the reporter's Granite embedding server, cold-start deadlines or network latency, arbitrary remote tokenizer windows, live provider requests, Windows/macOS or GPU/MPS. Only the diagnostic failure was injected; successful persistence, embeddings and retrieval used real backends. ## Runtime Rollout Safety - Rollout-managed feature(s): existing automatic memory lookup; no new feature gate. - Minimum rollout channel: unchanged. - Stable/default behavior changed: proxy lookups use the bounded representation; the SDK's default full rendering is unchanged. - Kill switch / disable path: existing `--no-memory-context` or memory configuration. - Unsafe override required: no. - Qualification impact: actual retrieval, raw-input preservation and diagnostics checked locally. - Rollback path: revert the fix; no schema migration. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have made corresponding changes to the documentation - [x] I did not edit CHANGELOG.md — release-please owns it ## Additional Notes AI-assisted; no subagents. |
||
|
|
4e5a67a342 |
fix(memory): skip <system-reminder> blocks when building the retrieval query (#2195) (#2541)
## Description Addresses #2195 Finding 1. `extract_memory_query_sources` (the memory retrieval query builder) was extended to harvest text blocks from Anthropic list-shaped user turns — the standard Claude Code shape — but it joins **every** text block in the turn. Claude Code appends `<system-reminder>` harness blocks to essentially every user turn, so those get concatenated into the embedding input alongside the real question. Per the reporter's measurements (`all-MiniLM-L6-v2`): the clean question scored top cosine **0.748** against a stored memory; the same question wrapped in harness boilerplate scored **0.232**. The default `memory_min_similarity` floor is **0.3**, so the diluted query falls under the floor and **nothing is retrieved** — memory silently no-ops for Claude Code clients. The reporter explicitly warned that a naive "concatenate all text blocks" harvest would still retrieve nothing, which is exactly the current behavior. ## Fix Filter out text blocks whose text starts with `<system-reminder` when building `user_text`, so the retrieval query keys on the substantive question and the embedding isn't diluted by harness boilerplate. A turn that is only a system-reminder yields no `user_text` (as before). All other harvesting (tool_result blocks, OpenAI string content, assistant/tool context) is unchanged. Note: the reporter also asked to expose `memory_min_similarity` as an env var / CLI flag (it lives on `ProxyConfig` with no surface today). That is a sensible companion but is a separate config-plumbing change; I kept this PR focused on the retrieval-query bug so it stays easy to review, and I'm happy to follow up with the env/CLI surface. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/memory_query_policy.py`: in `extract_memory_query_sources`, skip `<system-reminder>` text blocks when assembling the user query from a list-shaped Anthropic user turn. - `tests/test_memory_query_policy.py`: regressions that a system-reminder block is excluded (real question kept) and that a reminder-only turn yields no user text. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_memory_query_policy.py -q 7 passed # with the fix reverted, the two new tests fail: the system-reminder text is # concatenated into user_text (the diluted-query behavior) $ uvx ruff@0.15.17 check headroom/proxy/memory_query_policy.py tests/test_memory_query_policy.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/memory_query_policy.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: called `extract_memory_query_sources` with a Claude Code-shaped user turn (real question text block + an appended `<system-reminder>` text block), and with a reminder-only turn; then reverted the source and re-ran. - Observed result: with the fix `user_text` is exactly `"how do I add caching to the auth handler?"` (no `system-reminder` substring), and a reminder-only turn yields `""`; with the fix reverted `user_text` includes the full `<system-reminder>...</system-reminder>` text (the diluted embedding input). Ran against the actual module. - Not tested: an end-to-end embedding + backend retrieval against a live memory DB measuring the cosine recovery (the dilution figures are the reporter's; this change removes the boilerplate from the query text that produces them). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
|
f542b70413 |
fix(proxy/memory): capture user text blocks for the retrieval query (#2064)
## Description
`extract_memory_query_sources` (`headroom/proxy/memory_query_policy.py`)
builds the text used to
retrieve relevant memories. It captures `latest_user` **only** when a
user message's `content` is
a plain `str`:
```python
if role == "user":
if isinstance(content, list):
_append_anthropic_tool_results(content, tool_outputs=..., lookback_tools=...)
elif isinstance(content, str) and not latest_user:
latest_user = content
```
But the standard Anthropic `/v1/messages` shape (used by Claude Code)
sends the user turn as a
**list of content blocks** — `content=[{"type":"text","text":"help me
refactor auth"}]`. That
routes into `_append_anthropic_tool_results`, which extracts only
`type=="tool_result"` blocks
and **never reads the `type=="text"` blocks** — so the actual user
prompt is discarded.
Downstream (`handlers/anthropic.py` → `MemoryQuery.from_messages` →
`to_embedding_input`):
- On a **first turn** (no prior assistant/tool context) the embedding
input is `""`, and the
memory handler then returns `None` — **memory injection is silently
skipped entirely**.
- With history present, the query is assembled from stale assistant/tool
context **minus the
current question**, so retrieval targets the wrong text.
Closes: no issue filed — found while auditing the memory retrieval query
policy.
## Fix
In the list-content user branch, also collect the `text` blocks into
`latest_user` (guarded by
`if not latest_user` so the latest turn wins), alongside the existing
tool-result extraction.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/memory_query_policy.py`: capture Anthropic user `text`
blocks into `latest_user`.
- `tests/test_memory_query_policy.py`: add
`test_extract_sources_captures_anthropic_user_text_blocks` and
`test_extract_sources_captures_user_text_alongside_tool_result`.
## Testing
- [x] New regression tests added (`tests/test_memory_query_policy.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/proxy/memory_query_policy.py tests/test_memory_query_policy.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the extraction logic
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: ran a text-block user turn (and a mixed
text+tool_result turn, a plain-string turn, and multiple user turns)
through the old and new logic.
- Observed result: the old logic drops the user text (empty query →
injection skipped); the new logic captures it, still gathers tool
output, and keeps the plain-string / latest-turn behavior:
```text
text-block user: OLD user_text='' NEW user_text='help me refactor auth'
MEMORY QUERY TEXT-BLOCK FIX VERIFIED (old drops user text; new captures it)
```
- Not tested: a full memory retrieval round-trip through the embedder
(needs the heavy stack). The fix is confined to
`extract_memory_query_sources` and the new tests drive it directly. Full
local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Small, contained fix in the query-source extractor; no new
dependencies. The existing
`test_extract_sources_handles_anthropic_tool_result_without_user_text`
still passes (its list turn has no text block).
- @JerrettDavis tagging you — this silently disables memory injection
for the standard Claude Code request shape on a first turn, so it seemed
worth surfacing. Thanks!
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
|
235c986c9c |
refactor(memory): isolate query construction policy (#1950)
## Description Extracts memory retrieval query construction policy into a pure helper module while preserving `MemoryQuery` as the public frozen value type. The dataclass now delegates source extraction and embedding-input rendering to policy helpers, keeping query construction separate from the value wrapper. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Added `headroom.proxy.memory_query_policy` for pure retrieval query source extraction and rendering. - Updated `MemoryQuery.to_embedding_input` and `MemoryQuery.from_messages` to delegate to the extracted policy. - Added direct tests for the policy boundary. - Kept the LiteLLM callback compatibility shim required for repo-wide type checking on fresh branches. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m pytest tests/test_memory_query_policy.py tests/test_memory_query.py tests/test_memory_invariants.py tests/test_litellm_callback.py tests/test_compress_api.py::TestLiteLLMCallback -q 30 passed in 6.69s python -m ruff check . All checks passed! python -m ruff format --check . 1095 files already formatted python -m mypy headroom --ignore-missing-imports Success: no issues found in 409 source files gitleaks protect --staged --no-banner --redact no leaks found ``` ## Real Behavior Proof - Environment: Windows, Python 3.13.13, local worktree based on `headroomlabs/main`. - Exact command / steps: ran focused memory query tests, memory invariant tests, LiteLLM callback tests, Ruff lint/format checks, mypy over `headroom`, and staged gitleaks scan. - Observed result: all local checks passed; staged secret scan found no leaks. - Not tested: full CI matrix and deployment flows; those are covered by GitHub Actions. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes Documentation and changelog updates are not applicable for this internal refactor. GitHub reported existing Dependabot alerts on the default branch during push; this PR does not change dependencies, and the staged secret scan is clean. |