mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
feat/https-compose-caddy
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
01d9194854 |
fix(ccr): serve headroom_retrieve when the model calls it alongside client tools (#4034)
## Description When the model calls `headroom_retrieve` in parallel with client tools, `CCRResponseHandler.handle_response` hands the whole turn back for the client to resolve (#839). The client has no `headroom_retrieve`, so it can never resolve that call: Claude Code answers `Error: No such tool available: headroom_retrieve`, and the model never gets the content it asked for. Parallel calls are the normal shape for agentic clients. A Claude Code subagent that greps and reads several files in one turn hits this regularly, and users report the error as "happens sometimes, usually with subagents". Measured on one machine's Claude Code transcripts (`~/.claude/projects`): 501 leaked `headroom_retrieve` calls since 2026-09-21. 484 of them were mixed turns (retrieve plus at least one client tool in the same assistant message), and 435 of those were in subagents. The other 17 were lone retrieves on the #3071 streaming path, which this PR does not touch. The client tools in a mixed turn have not run yet. This PR drops them from the turn, serves the retrieval in the continuation exactly as for a lone call, and lets the model decide again there, with the retrieved content in hand. The continuation is what the client receives, whatever it does: re-issue the client calls, change them, or withdraw them because the retrieved content says not to. The client never sees `headroom_retrieve` and never runs a call the model withdrew. Only when the continuation request fails does the client get the model's own mixed turn back (the #839 behaviour), since the model made no newer decision; a continuation that succeeded is kept even if a later retrieval round fails. A `headroom_retrieve` without a valid hash is never treated as a client call: such a turn is handed back unchanged. This applies to Anthropic and OpenAI chat only. In the Responses API a `function_call` can be the item a `reasoning` item requires next, and a Gemini `functionCall` part can carry the turn's thought signature, so dropping a sibling there could make the continuation invalid. Those providers keep the #839 pass-through unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `ccr/tool_calls.py`: new `drop_tool_calls(response, provider, calls)`. It returns a shallow copy of the response without the given tool-call objects, matched by identity. It handles Anthropic and OpenAI chat and returns the response unchanged for any other provider. - `ccr/response_handler.py`: on a mixed turn, `handle_response` trims the client calls and continues the normal retrieval loop. A successful continuation is returned as is. If the continuation request for a mixed round fails, it returns the original mixed turn instead of the trimmed one. When the provider is not trimmable, or one of the extra calls is a `headroom_retrieve` without a valid hash, it keeps the old skip-and-return. The model's own response object is not mutated. - Tests: the #839 handler test now asserts the new behaviour for Anthropic and OpenAI chat: one continuation carrying only the retrieve pair, and the re-issuing continuation returned. New handler tests return the continuation when it withdraws the client call in text, changes its arguments, or re-issues it unchanged; return the original turn when the continuation request fails; and keep a successful continuation when a later retrieval round fails. A malformed `headroom_retrieve` keeps the turn unchanged with no continuation. The Responses pass-through is pinned as unchanged. On the streaming SSE path, a re-issued client tool streams without `headroom_retrieve`, a continuation that withdraws the client tool streams the continuation, and a failed continuation streams both original blocks as a 200. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/ -q -k "ccr or CCR or retrieve or mixed" 1168 passed, 24 skipped, 15064 deselected, 3 warnings in 178.17s $ uv run --frozen --extra dev pytest <every test file naming headroom_retrieve, CCR_TOOL_NAME or ccr_response_handler> # at |
||
|
|
9f75f91dc5 |
fix(ccr): log the exception type when a CCR continuation call fails (#3137)
## Description
`response_handler.py` logs a failed CCR continuation call with only
`str(e)`:
```python
logger.error(f"CCR: Continuation API call failed: {e}")
```
For exception types whose `str()` is empty —
`httpx.TimeoutException('')`, a bare `Exception()`,
`httpx.ConnectError('')` — this produces a log line with nothing after
the colon:
```
CCR: Continuation API call failed:
```
The cause (type, message) is lost entirely. This was hit in production:
a continuation call failed and the only trace was this truncated line,
with no way to tell whether it was a timeout, a transport error, or
something else.
The fix logs `type(e).__name__` plus `repr(e)`, both of which are
non-empty for every exception type (a bare `Exception()` still reprs as
`Exception()`), so the cause always survives.
Closes #3129.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/ccr/response_handler.py`: the continuation-failure log now
uses `logger.error("CCR: Continuation API call failed: %s: %r",
type(e).__name__, e)`.
- `tests/test_ccr_response_handler.py`: added
`test_continuation_failure_logs_cause_for_empty_str_exception`, which
drives the continuation path with an `api_call_fn` that raises
`Exception("")` (empty `str`, non-empty `repr`) and asserts the logged
line names the type and is never a bare `...failed:`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
tests/test_ccr_response_handler.py -> 36 passed
uvx ruff@0.16.2 check headroom/ccr/response_handler.py tests/test_ccr_response_handler.py -> All checks passed!
uvx mypy@1.20.2 headroom/ccr/response_handler.py -> Success: no issues found
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.16.2 and mypy 1.20.2 via uvx.
- Exact command / steps: reverted the log line to `f"...: {e}"` and ran
the new test (`python -m pytest tests/test_ccr_response_handler.py -k
continuation_failure`) -> failed with the captured log `ERROR ... CCR:
Continuation API call failed:` (empty after the colon) and `assert
"Exception" in ...` failing; restored the fix and re-ran -> 36 passed.
- Observed result: before the fix an empty-`str` exception logged `CCR:
Continuation API call failed:` with no cause; after the fix it logs
`CCR: Continuation API call failed: Exception: Exception('')`.
- Not tested: a live upstream timeout in a running proxy; the
continuation loop is exercised directly with a raising `api_call_fn`,
which is the exact except branch that logs.
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is a diagnostic log line, not a
rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: only the text of one error log line.
Control flow is unchanged (the handler still degrades to the current
response).
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: operators can now see the cause of a failed CCR
continuation instead of a blank line.
- Rollback path: revert this PR; the log returns to interpolating
`str(e)`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal log line)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
The issue also raised, as a secondary "worth considering" point,
re-raising the exception so the caller can react rather than only seeing
a degraded response. I left the control flow as-is (the `break`/degrade
path is a deliberate "degradation, not outage" design) and scoped this
PR to the logging defect, which is the concrete data-loss the issue
reports. Happy to follow up on the re-raise separately if maintainers
want it.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
|
89319fbcad |
fix(ccr): guard empty/malformed OpenAI choices in _extract_assistant_message (#2389)
## Description
`CCRResponseHandler._extract_assistant_message` extracts the assistant
message from an upstream response while building the CCR
retrieval-continuation history. The OpenAI branch is not defensive about
an empty or malformed `choices` array:
```python
elif provider == "openai":
message = response.get("choices", [{}])[0].get("message", {})
```
`response.get("choices", [{}])` only falls back to `[{}]` when the key
is **absent**. When `choices` is present but empty (`[]`) or carries a
null first element (`[null]`), this raises on the success path:
- `choices: []` → `[][0]` → `IndexError`
- `choices: [null]` → `None.get(...)` → `AttributeError`
OpenAI-compatible gateways can return those shapes on content-filtered
or usage-only responses. The sibling **Google** branch a few lines below
already guards this (`candidates = response.get("candidates", []); if
candidates: ... else: parts = []`), and so does `ccr/tool_calls.py` (it
checks `isinstance(choices, list)`, non-empty, and
`isinstance(first_choice, dict)`). Only this OpenAI branch was missed.
## Fix
Guard the list and the first element the same way the siblings do:
```python
elif provider == "openai":
choices = response.get("choices")
first = choices[0] if isinstance(choices, list) and choices else {}
message = first.get("message", {}) if isinstance(first, dict) else {}
return {
"role": "assistant",
"content": message.get("content"),
"tool_calls": message.get("tool_calls"),
}
```
A well-formed response is unaffected; an empty/null/absent `choices` now
yields `{"role": "assistant", "content": None, "tool_calls": None}`
instead of raising.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/ccr/response_handler.py`: guard empty/non-list `choices` and
a non-dict first element in the OpenAI branch of
`_extract_assistant_message`.
- `tests/test_ccr_response_handler.py`: add
`TestExtractAssistantMessageEdgeCases` (empty `choices`, `[null]`,
absent, and the normal case).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/ccr/response_handler.py tests/test_ccr_response_handler.py
All checks passed!
$ uvx ruff@0.15.17 format --check headroom/ccr/response_handler.py tests/test_ccr_response_handler.py
2 files already formatted
# Verified against the REAL imported module (headroom.ccr.response_handler is
# light — no ML imports), so this ran locally in the project venv:
$ python -c "from headroom.ccr.response_handler import CCRResponseHandler as H; h=H(); \
assert h._extract_assistant_message({'choices': []}, 'openai') == {'role':'assistant','content':None,'tool_calls':None}"
# (no IndexError; normal case still extracts content/tool_calls)
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17`.
- Exact command / steps: imported the real `CCRResponseHandler` and
called `_extract_assistant_message` with `{"choices": []}`, `{"choices":
[null]}`, `{}` (absent), and a normal `{"choices": [{"message":
{...}}]}`.
- Observed result: the OLD code raised `IndexError` on `[]` and
`AttributeError` on `[null]`; the NEW code returns `{"role":
"assistant", "content": None, "tool_calls": None}` for all three
malformed shapes and still extracts `content`/`tool_calls` from a
well-formed response. Because `response_handler` has no ML imports, this
ran against the actual module, not a replica.
- Not tested: a live CCR retrieval round trip through a gateway that
emits empty choices; the added unit tests drive
`_extract_assistant_message` directly.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
`headroom/ccr/response_handler.py` is a light module (no ML imports), so
unlike most of my recent PRs I verified the fix by importing the real
class in the project venv (output above), in addition to the added unit
tests. This aligns the OpenAI branch with the already-defensive Google
branch and `ccr/tool_calls.py`.
|
||
|
|
c2fc4d3753 |
fix(ccr): make headroom_retrieve a hash-only full-content lookup (#1532)
The optional `query` parameter on headroom_retrieve routed retrieval through CompressionStore.search(), which BM25-scored the items inside a single cached blob and dropped everything below a 0.3 relevance floor. On small per-blob corpora with conversational queries this returned an empty result the large majority of the time, so the LLM saw "nothing found" for content that was actually present — pushing users to turn compression off entirely. Retrieval is fundamentally a hash lookup (this already matches the Rust proxy's CCR store, which is put/get only — "no BM25 search"). Remove the query/search path end to end and always return the full original content: Core (Python proxy): - tool schemas (anthropic/openai/google) drop the `query` property - parse_tool_call returns the hash (str | None) instead of (hash, query) - response handler, proxy POST/GET/tool-call handlers, the MCP retrieve tool, and the streaming feedback recorders retrieve by hash only - proactive context-tracker expansion always restores full content - delete CompressionStore.search() and its BM25 machinery (the bm25 module stays — it is still used by relevance/) - CCRToolCall.query, CCRToolResult.was_search, and ExpansionRecommendation.expand_full/search_query are removed Plugins (advertised a now-defunct query param to the LLM): - hermes (Python), openclaw + opencode (TypeScript) retrieve tools drop `query` from their schemas, signatures, request URLs, and tests Benchmarks/docs: - ccr_regression + adversarial benchmarks switch from store.search() to full hash retrieval (search input-injection tests repurposed to the hash, the only remaining input surface) - wiki/ARCHITECTURE.md, wiki/ccr.md, docs/content/docs/ccr.mdx, config.py and store docstrings updated to describe hash-only retrieval Tests updated to assert full-content retrieval and guard the removed surface; the full CCR/proxy/store/TOIN suite passes. ruff + mypy clean. ## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|
|
30078f8465 |
fix(ccr): skip CCR when model calls headroom_retrieve alongside user tools (#839)
## Summary - When the LLM calls `headroom_retrieve` **and** a non-CCR tool (e.g. `read_file`) in the same turn, the previous code attempted a continuation with only the CCR result - Anthropic requires every `tool_use` block to have a matching `tool_result` — the continuation was rejected with 400, a round-trip was wasted, and the original response (with unresolved `headroom_retrieve`) was returned anyway - Fix: if `other_calls` is non-empty alongside `ccr_calls`, log a warning and return the original response immediately — no continuation attempted ## Root cause `_parse_ccr_tool_calls` correctly separates CCR and non-CCR calls, but `handle_response` never checked `other_calls` before building the continuation. `_create_tool_result_message` only adds results for CCR calls, leaving the non-CCR `tool_use` blocks without matching `tool_result` entries. ## Files changed - `headroom/ccr/response_handler.py` — guard at top of `while` loop in `handle_response` - `tests/test_ccr_response_handler.py` — regression test: asserts `api_call_count == 0` and original response returned unchanged when model uses mixed tools ## Test plan - [x] `pytest tests/test_ccr_response_handler.py::TestCCRResponseHandling::test_handle_response_mixed_tools_skips_ccr` — passes - [x] `pytest tests/test_ccr_response_handler.py tests/test_ccr_response_handler_extra.py tests/test_ccr_tool_injection.py tests/test_ccr_tool_always_on.py` — 85 passed - [x] Pre-commit hooks (ruff, mypy) — clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
8f0754a622 |
Fix security vulnerabilities in memory and CCR systems
- Fix race condition in BatchContextStore.stats() by acquiring lock - Add atomic dict snapshot in get_memory_stats() to prevent RuntimeError - Add metadata key validation to prevent JSON path injection in SQLite - Parameterize LIMIT/OFFSET in SQLite queries to prevent SQL injection - Strengthen CCR hash validation to require exactly 24 hex characters - Add comprehensive security validation tests |
||
|
|
d724f14022 |
v0.2.2: Add CCR Response Handler, Context Tracker, and restructure docs
Features: - CCR Response Handler: Automatically intercepts and handles headroom_retrieve tool calls - CCR Context Tracker: Multi-turn awareness with proactive expansion of relevant compressed content - New CCR demo script showing before/after flow Documentation: - Restructured README from 885 lines to 190 lines for better DevEx - Split detailed docs into focused guides: ccr.md, sdk.md, configuration.md, text-compression.md, llmlingua.md, metrics.md, errors.md - Updated docs/README.md index with all new documentation Tests: - Added comprehensive tests for Response Handler (32 tests) - Added comprehensive tests for Context Tracker (32 tests) - All 977 tests passing |