mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat/https-compose-caddy
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
dcd03b9b4c |
fix(proxy): don't take Claude Code's "Tool loaded." note as the user query (#4714)
## Description Claude Code answers a ToolSearch call with `tool_reference` blocks plus a "Tool loaded." text block. `extract_user_query` takes the newest user text, so after an MCP tool is loaded the relevance query is "Tool loaded." and the question never reaches SmartCrusher. It's the same kind of bug as #4628 (`<system-reminder>`). Observed on Claude Code 2.1.319. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - In a turn whose tool_result carries `tool_reference` blocks, drop the exact "Tool loaded." text. Other text in that turn still counts, and "Tool loaded." outside such a turn is unchanged. - The `tool_reference` check is the stable signal; the exact string follows Claude Code's current wording. Happy to drop all text in such a turn instead if you prefer. ## Testing - [x] New tests: ToolSearch turn, ToolSearch turn with real user text, plain "Tool loaded." text. The first two fail without the fix. - [x] `ruff check`, `ruff format --check`; tests for `extract_user_query` callers pass. Fork CI: green. ### Test Output ``` $ pytest -q tests/test_utils.py 38 passed in 0.33s $ pytest -q tests/test_utils.py -k "tool_search or tool_loaded" 3 passed, 35 deselected in 0.18s $ ruff check headroom/utils.py tests/test_utils.py && ruff format --check headroom/utils.py tests/test_utils.py All checks passed! 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS arm64, Claude Code (Sonnet) through `headroom proxy` (PyPI 0.43.1 vs source), an MCP tool returning 100 of this repo's issues. - Exact command / steps: 12 questions "Who opened issue #N, and is it open or closed?"; scripts and answers: https://github.com/cherylaurelia/headroom-demo/tree/main/bench - Observed result: the logged query was "Tool loaded." on 0.43.1, and the question with this fix. Accuracy 2/12 → 12/12 together with the SmartCrusher fix in `fix/keep-query-matches` (2/12 with that fix alone). - Not tested: agents other than Claude Code. ## Runtime Rollout Safety - Rollout-managed feature(s): n/a - Minimum rollout channel: n/a - Stable/default behavior changed: only for ToolSearch result turns - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] I did **not** edit `CHANGELOG.md` Written with Claude Code, which also ran the tests and measurements above. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4714" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
12dca26aa1 |
fix: use all user text blocks for relevance scoring (#4628)
## Description
`extract_user_query` stops at the first text block in a user message.
Relevance scoring can therefore miss the rest of the request. A leading
Claude Code `<system-reminder>` can also be mistaken for the request.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- Join the user text blocks in order and remove complete reminder spans
without discarding text around them.
- Keep incomplete reminder tags and retain the existing latest-turn
fallback behavior.
- Cover multi-block prompts, reminders within both string and block
content, and a relevance-split example.
## Testing
- [x] Focused unit and integration tests pass
- [x] Linting passes (`ruff check .` and `ruff format --check .`)
- [x] New tests added for the bug
### Test Output
```text
Windows / Python 3.12: 108 passed, 2 warnings
Ubuntu / Python 3.12: 108 passed, 2 warnings
Ubuntu: ruff check ., ruff format --check ., mypy headroom --ignore-missing-imports passed
```
The added tests fail on the preceding revision. The Ubuntu run built the
Rust extension from source and tested the final PR code. [Run
details](https://github.com/bingdongni/headroom/actions/runs/41793014635).
## Real Behavior Proof
- Environment: Ubuntu, Python 3.12, `HEADROOM_OFFLINE=1`; Rust extension
built from source with `maturin build --profile ci`.
- Exact command / steps: Run the [deterministic input
script](https://github.com/bingdongni/headroom/blob/5750c34/scripts/contributor-proof.py)
against the base and patched source trees, with the same compiled
extension. The [verification
run](https://github.com/bingdongni/headroom/actions/runs/41793014635)
records these commands and output:
```sh
git worktree add --detach /tmp/headroom-base
|
||
|
|
ebc59356ec |
refactor(utils): deprecate the marker, JSON and hash helpers nothing calls (#4107)
## Description
This PR deprecates six helpers in `headroom/utils.py`. Nothing in the
product, the docs or the org's consumer repos calls them.
`headroom.utils` is imported by the org's consumer repos, so its public
functions count as public API, and these helpers are deprecated rather
than deleted.
Each helper now emits a `DeprecationWarning` with `stacklevel=2`, so the
warning points at the caller. Each docstring gets a `.. deprecated::`
note. Behaviour is otherwise unchanged.
| Deprecated | Use instead |
|---|---|
| `fast_hash` | `hashlib.md5(data).hexdigest()[:length]`, with `str`
input encoded as UTF-8 first. For `str` input at the default length,
`headroom.parser.compute_hash` gives the same value. |
| `create_dropped_context_marker` | `create_marker("dropped_context",
...)` |
| `create_truncated_marker` | `create_marker("truncated", ...)` |
| `extract_markers` | no replacement |
| `safe_json_loads` | `json.loads`, catching `ValueError` |
| `safe_json_dumps` | `json.dumps(obj, ensure_ascii=False,
separators=(",", ":"))` |
Evidence: a word-boundary search for each name across `headroom`,
`crates`, `plugins`, `sdk`, `examples`, `benchmarks`, `scripts`, `e2e`,
`docs`, `wiki`, `.github`, `pyproject.toml` and the consumer repos finds
only the definitions and `tests/test_utils.py`. Rust comments that
describe the old Python `safe_json_dumps` output format are the one
exception. So no product path triggers the new warnings.
Refs #4136
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
- `headroom/utils.py`: add a `warnings.warn(..., DeprecationWarning,
stacklevel=2)` and a `.. deprecated::` docstring note to each of the six
helpers.
- `tests/test_utils.py`: replace the asserts on these helpers with one
parametrized test, `test_deprecated_helpers_warn_and_still_work`.
- For each case it checks two things: the return value is unchanged, and
the `DeprecationWarning` names the helper.
- All the old cases are kept: valid and invalid JSON, several markers in
one string, and a dropped-context marker with and without a count.
- A new case covers `bytes` input to `fast_hash`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
# each new deprecation message escalated to an error, so any product call path would fail:
$ pytest -q -p no:cacheprovider -W "error:fast_hash is deprecated:DeprecationWarning" -W "error:safe_json_loads is deprecated:DeprecationWarning" -W "error:safe_json_dumps is deprecated:DeprecationWarning" -W "error:create_dropped_context_marker is deprecated:DeprecationWarning" -W "error:create_truncated_marker is deprecated:DeprecationWarning" -W "error:extract_markers is deprecated:DeprecationWarning" tests/test_utils.py tests/test_smart_crusher.py tests/test_storage tests/test_reporting.py tests/test_cache_aligner_prefix_stability.py tests/test_integration_metric_timestamps_utc.py tests/test_transforms
648 passed in 46.58s
$ python -W always::DeprecationWarning -c "from headroom import utils; utils.safe_json_dumps({'a': 1})"
<string>:1: DeprecationWarning: safe_json_dumps is deprecated and will be removed in a future release; call json.dumps(obj, ensure_ascii=False, separators=(',', ':')) instead.
$ uvx ruff@0.16.8 check headroom/utils.py tests/test_utils.py
All checks passed!
$ python -m mypy headroom/utils.py --ignore-missing-imports
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS, Python 3.12.13, this branch checked out in a
worktree, repo .venv; temp HOME and HEADROOM_WORKSPACE_DIR, offline.
- Exact command / steps: (1) The shared smoke harness `smoke.sh
<worktree>`: it starts a real proxy from this checkout against a local
fake upstream and sends one Anthropic and one OpenAI request with a
compressible tool result. (2) A direct command on this PR's own path:
`HOME=$TMP/home HEADROOM_WORKSPACE_DIR=$TMP/ws python -W
always::DeprecationWarning -c "from headroom import utils;
print(utils.fast_hash(b'hello', 8)); print(utils.safe_json_dumps({'a':
1}))"`.
- Observed result: Both requests return 200 and `/stats` shows savings,
with only the expected offline-mode warnings in proxy.log. The direct
command's output is below.
- Not tested: Outside callers of these helpers; that is the point of
deprecating first.
```text
$ bash smoke.sh <worktree>
branch: cleanup/core-misc-utils-deprecate @
|
||
|
|
dcec845851 |
refactor(ccr): read the expansion query through extract_user_query (#2707)
## Description
The CCR proactive-expansion path hand-rolled its own "most recent user
question" scan instead of calling `headroom.utils.extract_user_query`,
which every other consumer on this handler already uses. Two copies of
the same read meant any fix to how the query is identified had to be
made twice — and only one copy would get it.
No behaviour change intended for the normal case. This is groundwork:
there is a real defect in how that query is identified (the helper does
not strip injected `<system-reminder>` boilerplate, so under harnesses
that inject context the "query" is the boilerplate rather than the
user's question). Fixing that belongs in its own PR because it moves
SmartCrusher's keep/drop scoring on every provider path and needs a
before/after quality measurement. Collapsing to one call site first
means that fix lands in one place.
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: replace the inline 12-line
reversed-scan in the proactive-expansion block with
`extract_user_query(messages, latest_user_turn_only=True)`.
- `headroom/utils.py`: add `latest_user_turn_only` to
`extract_user_query`.
The flag exists to preserve behaviour exactly. `extract_user_query`
walks back to *earlier* user turns when the newest one yields no text;
the expansion path must not. On a `tool_result` continuation turn the
newest user turn carries no text at all, and scoring relevance against
an older turn's question would expand context for a stale intent — i.e.
without the flag this refactor would make expansion fire on turns where
it previously stayed silent.
One deliberate difference remains: within the chosen turn the helper
skips empty text blocks rather than committing to the first block it
sees, so an empty leading text block no longer suppresses expansion
outright. That is a no-op whenever the first text block has content,
which is the normal case.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_utils.py tests/test_proxy_ccr.py -q
======================== 25 passed, 3 warnings in 5.08s ========================
$ pytest tests/ -q -k "ccr or proactive or expansion or utils or memory_query or extract"
=== 2 failed, 782 passed, 22 skipped, 9409 deselected, 43 warnings in 43.49s ===
$ ruff check headroom/utils.py headroom/proxy/handlers/anthropic.py tests/test_utils.py
All checks passed!
$ ruff format --check headroom/utils.py headroom/proxy/handlers/anthropic.py tests/test_utils.py
3 files already formatted
$ mypy headroom/utils.py --ignore-missing-imports
Success: no issues found in 1 source file
```
The 2 failures are
`tests/test_cli_proxy_env.py::TestCLICompressionOnlyFlags::{test_ccr_defaults_on,test_no_ccr_flag}`
and are **pre-existing and unrelated** — they fail identically with this
change stashed, and pass under a clean `HOME`. The real
`~/.headroom/settings.json` leaks into those config tests.
Two new tests pin the flag: that it returns `""` on a `tool_result`
continuation turn where the default walks back to `"the original
question"`, and that it agrees with the default whenever the newest user
turn does carry text.
## Real Behavior Proof
The contract for a refactor like this is "the proxy makes the same
decisions". So it was verified by running the real proxy on both `main`
and this branch and diffing every observable.
- Environment: macOS 15 (arm64), Python 3.13.14. `headroom proxy --mode
token --backend anthropic --anthropic-api-url http://127.0.0.1:<stub>`,
isolated `HOME` per run so neither could touch a developer's live
logs/store, `HF_HOME` pointed at cached kompress weights so the
lossy+CCR-marker path is exercised and expansion can actually arm.
- Exact command / steps: three requests over one conversation through
the real HTTP path (`x-headroom-cwd: /tmp/proof`, `user-agent:
claude-code/1.4.2`): a user turn carrying the real
`~/.claude/rules/*.md` text (~8.2k tokens); then `assistant` + a short
user turn so that block becomes compressible and gets tracked as a CCR
entry; then a follow-up whose leading text block shares vocabulary with
it so `analyze_query` scores above threshold. A stub upstream recorded
every forwarded body. Identical request sequence against both builds.
- Observed result: identical on every axis — this branch resolves the
same query and therefore reaches the same expansion decision.
| observable | `main` | this branch |
|---|---|---|
| expanded hash | `f0cf4efb42373ec225f57725` |
`f0cf4efb42373ec225f57725` |
| items expanded | 1417 | 1417 |
| expansion firings in run | 2 | 2 |
| `tok_before` → `tok_after` per turn | 8170→9553, 8184→9567, 8245→11880
| 8170→9553, 8184→9567, 8245→11880 |
| forwarded body bytes upstream | 43,507 / 43,637 / 54,130 | 43,507 /
43,637 / 54,130 |
(The growth visible in those numbers is the pre-existing expansion
behaviour on `main`, unchanged here by design — it is what #2706
addresses. This PR only proves it does not *move*.)
- Not tested: the `tool_result` continuation turn — the case
`latest_user_turn_only` exists for — is covered by unit test rather than
by the live run above, since the scenario needs a turn whose newest user
message has no text block. The remaining intentional delta (empty
leading text block) was likewise not exercised against real traffic; it
cannot trigger while the first text block has content. Streaming
responses were not exercised (the stub replies non-streaming), and the
OpenAI/Gemini/batch call sites of `extract_user_query` were not run —
this change adds a keyword-only parameter with a default that preserves
their behaviour, but they were verified by test suite only.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A — proxy-internal change, no user-facing surface.
## Additional Notes
- Independent of #2706 despite touching the same feature area — verified
`conflicts=0` via `git merge-tree`; either can merge first.
- The follow-up this unblocks: `extract_user_query` does not strip
injected `<system-reminder>` boilerplate, so under harnesses that inject
context the resolved "query" is the boilerplate rather than the user's
question. That defect is shared by every consumer of the helper
(SmartCrusher relevance scoring on the Anthropic, OpenAI, Gemini and
batch paths), which is exactly why it wants a dedicated PR with a
before/after compression-quality measurement rather than riding along
here.
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
|
b625d188c5 |
fix(utils): normalize timezone-aware datetimes in format_timestamp (#3116)
## Description `format_timestamp` appended a bare `"Z"` to `dt.isoformat()`: ```python return dt.isoformat() + "Z" ``` That is correct only for a naive (assumed-UTC) datetime. For a timezone-aware datetime it produces an invalid ISO 8601 string, and for a non-UTC offset it mislabels the instant: ``` format_timestamp(datetime(2024,1,1,12, tzinfo=utc)) -> '2024-01-01T12:00:00+00:00Z' # invalid; fromisoformat rejects it format_timestamp(datetime(2024,1,1,12, tzinfo=utc+5h)) -> '2024-01-01T12:00:00+05:00Z' # labels 07:00Z as "Z" ``` This is reachable: the shipped integrations construct `RequestMetrics(timestamp=datetime.now(timezone.utc))` (aware) - e.g. `headroom/integrations/agno/model.py`, `.../agno/hooks.py`, `.../autogen`, `.../crewai`, `.../langchain` - and the storage layer persists `format_timestamp(metrics.timestamp)` to JSONL/SQLite. So those rows carried malformed timestamps that a strict parser (including Python's own `datetime.fromisoformat`) rejects, and any non-UTC offset silently recorded the wrong instant. The fix converts an aware datetime to UTC before formatting; a naive datetime is still assumed to be UTC. Output stays `...Z` and now round-trips through a strict parser. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/utils.py` (`format_timestamp`): convert timezone-aware datetimes to UTC (`astimezone(timezone.utc).replace(tzinfo=None)`) before appending `"Z"`; naive datetimes are unchanged. The `None` default now uses aware `datetime.now(timezone.utc)` and flows through the same normalization, producing the identical string as before. - `tests/test_utils.py`: added `test_format_timestamp_normalizes_aware_datetimes` (aware UTC round-trips through `fromisoformat`; a `+05:00` offset is converted to the correct UTC time). The existing naive-datetime assertions are retained. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text tests/test_utils.py -> 5 passed uvx ruff@0.16.2 check headroom/utils.py tests/test_utils.py -> All checks passed! uvx mypy@1.20.2 headroom/utils.py -> Success: no issues found ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1, ruff 0.16.2 and mypy 1.20.2 via uvx. - Exact command / steps: reproduced the defect directly (`format_timestamp(datetime(2024,1,1,12,0,0, tzinfo=timezone.utc))` -> `'2024-01-01T12:00:00+00:00Z'`, and `datetime.fromisoformat` on that string raised `Invalid isoformat string`); reverted the fix and ran `python -m pytest tests/test_utils.py::test_format_timestamp_normalizes_aware_datetimes` -> failed (`'2026-04-23T06:00:00+00:00Z' == '2026-04-23T06:00:00Z'`); restored the fix and ran the file -> 5 passed. - Observed result: after the fix, `format_timestamp(aware_utc)` returns `'2026-04-23T06:00:00Z'` and a `+05:00` datetime returns `'2026-04-23T01:00:00Z'`; naive input is unchanged. - Not tested: an end-to-end write/read against a live SQLite/JSONL metrics store; the format helper is exercised directly, and the storage layer calls it verbatim. ## Runtime Rollout Safety - Rollout-managed feature(s): none. This is a pure formatting helper, not a rollout-channel-gated runtime feature. - Minimum rollout channel: N/A. - Stable/default behavior changed: only for aware datetimes, which previously produced an invalid string. Naive datetimes and the `None` default produce the identical output as before. - Kill switch / disable path: N/A. - Unsafe override required: no. - Qualification impact: metrics timestamps written from integration paths are now valid ISO 8601 UTC and round-trip through strict parsers. - Rollback path: revert this PR; `format_timestamp` returns to appending `"Z"` unconditionally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A: internal helper, no user-facing docs) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes `parse_timestamp` already tolerates both forms (it strips a trailing `Z` before `fromisoformat`), so in-process round-trips kept working and hid the defect; the corruption is visible to external consumers of the persisted files and to any non-UTC offset. This PR fixes the writer, which is where the invalid/mislabeled value originates. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
0d458e5d45 | test: add missing type hints to FakeProvider in test_utils (#631) | ||
|
|
efd2ac1ca4 |
chore: renormalize line endings to LF
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but 74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings, violating that contract. Every macOS/Linux clone reports these files as "modified" on fresh checkout because git's diff engine sees the stored bytes don't match the attribute contract, even though the working tree and index match byte-for-byte. Running `git add --renormalize .` rewrites each affected blob so the stored form matches the attribute declaration. No semantic changes — every affected file's diff is "N insertions, N deletions" with inserts and deletes being the same lines modulo line endings. Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub blame skip this mechanical commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
38bf3e639c |
test: expand coverage across helper slices
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |