mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 18:41:07 +08:00
feat/https-compose-caddy
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
12dca26aa1 |
fix: use all user text blocks for relevance scoring (#4628)
## Description
`extract_user_query` stops at the first text block in a user message.
Relevance scoring can therefore miss the rest of the request. A leading
Claude Code `<system-reminder>` can also be mistaken for the request.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- Join the user text blocks in order and remove complete reminder spans
without discarding text around them.
- Keep incomplete reminder tags and retain the existing latest-turn
fallback behavior.
- Cover multi-block prompts, reminders within both string and block
content, and a relevance-split example.
## Testing
- [x] Focused unit and integration tests pass
- [x] Linting passes (`ruff check .` and `ruff format --check .`)
- [x] New tests added for the bug
### Test Output
```text
Windows / Python 3.12: 108 passed, 2 warnings
Ubuntu / Python 3.12: 108 passed, 2 warnings
Ubuntu: ruff check ., ruff format --check ., mypy headroom --ignore-missing-imports passed
```
The added tests fail on the preceding revision. The Ubuntu run built the
Rust extension from source and tested the final PR code. [Run
details](https://github.com/bingdongni/headroom/actions/runs/41793014635).
## Real Behavior Proof
- Environment: Ubuntu, Python 3.12, `HEADROOM_OFFLINE=1`; Rust extension
built from source with `maturin build --profile ci`.
- Exact command / steps: Run the [deterministic input
script](https://github.com/bingdongni/headroom/blob/5750c34/scripts/contributor-proof.py)
against the base and patched source trees, with the same compiled
extension. The [verification
run](https://github.com/bingdongni/headroom/actions/runs/41793014635)
records these commands and output:
```sh
git worktree add --detach /tmp/headroom-base
|
||
|
|
93279f3208 |
fix(smart-crusher): preserve unchanged JSON bytes (#4044)
## Description Closes #3697. An unchanged SmartCrusher result now keeps the original bytes and reports no structural compression. Formatting-only savings come from the existing Rust JsonMinifier, not from pretending the JSON value changed. The minifier preserves duplicate keys, order, escapes and number literals. A mixed result cannot use one section's CCR marker to hide unmarked loss in another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Documentation update ## Changes Made - Keep SmartCrusher's exact-byte no-op behavior; route actual whitespace savings through JsonMinifier in Python and the eligible native live zone. - Keep all records and JSON literals; do not store originals or inject CCR markers for lossless minification. - Preserve mixed routing and reject uncovered lossy sections in both tool-result formats. Explicit no-CCR behavior is unchanged. - Replace incidental forwarding tests with native minification and complete-record regressions; document the distinction. ## Testing - [ ] Entire Python suite — not claimed. - [x] Four consumer/native modules: `python -m pytest -q tests/test_recursive_json.py tests/test_transforms_content_router.py tests/test_transforms/test_smart_crusher_rust_parity.py tests/test_transforms/test_smart_crusher_lossless_default.py` - [x] Changed-file Ruff checks; normal commit hooks, including mypy and commitlint. - [x] Manual public-API runs, with and without the HTML extra. ### Test Output Consumer/native modules: ```text collected 136 items ============================= 136 passed in 36.43s ============================= ``` Normal pre-push (fmt, clippy, 1,668 Rust tests with 3 ignored, native build, Python checks, commitlint): ```text ======================== 180 passed, 1 warning in 8.35s ======================== ✅ ci-precheck PASSED — safe to push. To https://github.com/oxura/headroom.git 64688217..0002af3e HEAD -> fix/3697-unchanged-json-passthrough ``` The first push stopped on an inherited commit footer because `origin/main` was stale. Fetching the actual upstream ref fixed the lint range; the normal push then passed. No source merge, lint-rule change or hook bypass. Hosted checks on `0002af3e` now pass: all four Python shards, Windows permissions, smoke tests, native wrappers, parity and Docker e2e. [Checks](https://github.com/headroomlabs-ai/headroom/pull/4044/checks). ## Real Behavior Proof - Environment: Linux x86-64, Python 3.13.14, Rust 1.95 and the actual extension built from this checkout. No provider billing calls. - Exact command / steps: the earlier localhost `/v1/compress` and TransformPipeline runs exercised the original fix. For this follow-up, call `ContentRouter.compress` on a data-only HTML page containing 60 complete records, with `smart_crusher_lossless_only=True`, `smart_crusher_with_compaction=False`, and `enable_kompress=False`. Run it once without trafilatura, then with the locked 2.0.0 extra. The new public-API test contains the full input/config. Separately call `route_embedded_json` on a compact array followed by a padded array, with Unicode and CRLF surroundings. - Observed result: both HTML modes keep all 60 records and the surrounding markup, with no unrecoverable section. The unchanged first JSON span does not stop the second span's compaction; surrounding text stays exact. Final Codecov is 96%; the two remaining lines are the deferred HTML fallback return. The real no-HTML-extra API run executed both, and local coverage also exercises the unchanged-span branch. - Not tested: live provider acceptance, invoices or production cache hit rates. No native eligibility or admission gates changed. ## Runtime Rollout Safety - Rollout-managed feature(s): none; existing JSON routing. - Minimum rollout channel: unchanged. - Stable/default behavior changed: eligible formatting savings return as json_minifier; standalone SmartCrusher remains byte-exact on no-ops. - Kill switch / disable path: existing routing configuration and HEADROOM_LIVE_ZONE_DISABLE_ARMS. - Unsafe override required: no. - Qualification impact: no-op, formatting, structural and CCR paths exercised separately. - Rollback path: revert the minifier integration; no persistence migration. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have made corresponding changes to the documentation - [x] I did not edit CHANGELOG.md — release-please owns it ## Additional Notes Addresses the [minifier review](https://github.com/headroomlabs-ai/headroom/pull/4044#issuecomment-6157583285). The maintainer’s main merge is retained. The latest follow-up only changes consumer tests. AI-assisted; no subagents. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
0ef7b2a7fd |
fix(relevance): bound segment size by max_chars (#2518)
## Description
`relevance_split`'s `segment()` partitions tool output into records so
the relevance filter can score each one. Its Pass-2 loop kept indented
continuation lines attached to a window while checking only indentation,
never `max_chars`, so a deep indented run (a stack trace, an indented
log or JSON payload) collapsed into one mega-segment far past the
documented bound and was scored as a single record. `relevance_split` is
on by default and feeds the LOG/SEARCH compression tail, so the
KEEP/DROP filter degraded to all-or-nothing on exactly the payloads it
exists to filter.
Fix: grow each segment under a single char budget covering the whole
record, so `max_chars` bounds every multi-line segment instead of only
the continuation. A segment always takes at least one line, so an
over-long single line stays atomic and the partition remains
byte-lossless.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `segment()` in `headroom/transforms/relevance_split.py`: one growth
loop under one budget. This also removes a fast path that was already
dead — with `len(block) <= window` the general loop emitted the
identical whole block, so its `max_chars` check never affected output.
- Docstrings in the same file: state the real contract (at most `window`
lines and `max_chars` chars, with an over-long single line emitted
whole), and record that bounding long runs makes uniformly-indented
output segment into head-less records.
- `tests/test_relevance_split.py`: pin the bound and the trade-off. Adds
a long indented run, a block of plain long lines, an over-long atomic
line, and a head-less-segment case. `max_chars` is passed explicitly so
assertions no longer depend on the default.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --extra dev pytest tests/test_relevance_split.py -q
tests/test_relevance_split.py ....................... [100%]
23 passed in 0.17s
$ uv run ruff check .
All checks passed!
$ uv run ruff format --check .
1331 files already formatted
$ uv run --extra dev mypy headroom
Success: no issues found in 506 source files
$ uv run --extra dev pytest -q
2 failed, 9647 passed, 571 skipped, 5893 warnings in 995.36s (0:16:35)
FAILED tests/test_cli/test_recover_codex.py::test_recovery_records_sockets_and_secures_both_backups
FAILED tests/test_proxy_ccr.py::TestCCRIntegration::test_health_endpoint
```
Both local failures are environmental, not regressions.
`test_recover_codex` fails with `OSError: AF_UNIX path too long` at
`socket.bind()` — macOS caps AF_UNIX paths near 104 chars and this
checkout sits under a deep worktree path.
`test_proxy_ccr::test_health_endpoint` passes when run on its own, so it
is full-suite ordering, not this change. CI runs the same suite green
across all 4 Linux shards.
## Real Behavior Proof
- Environment: worktree branched from `main` at `806d2e46`, uv-built env
with the maturin native core, macOS 15, Python 3.13, real `BM25Scorer`
(no stub).
- Exact command / steps: a deterministic harness feeds four realistic
tool-output payloads (a 120-frame Python traceback, an indented JSON
dump, a uniformly-indented YAML block, and a block of plain 300-char
lines) through the real `segment()` at production defaults `window=8,
max_chars=1200`, comparing the loop as it stands on `main` against this
branch, then runs the resulting records through `plan_relevance_split`
with the real `BM25Scorer` and the query "why did the discount charge
fail".
- Observed result: `max_chars` becomes an actual bound. Every payload
that previously produced an over-budget multi-line segment now produces
none, and the largest segment drops from 15,065 to 1,152 chars on the
traceback and from 9,909 to 1,197 on the YAML. The filter now scores the
traceback as 14 records instead of 2, and the YAML as 9 instead of 1,
which is what makes partial keeps possible at all rather than a single
all-or-nothing verdict. The partition stays byte-lossless in every case.
Raw output:
```text
payload segments max seg chars over budget lossless
120-frame stack trace base 2 15065 1 True
fixed 14 1152 0 True
indented JSON dump base 2 5794 1 True
fixed 6 1184 0 True
uniformly-indented YAML base 1 9909 1 True
fixed 9 1197 0 True
block of 300-char lines base 5 2488 5 True
fixed 14 933 0 True
records the relevance filter scores (real BM25Scorer):
120-frame stack trace base: 2 record(s) fixed: 14 record(s)
uniformly-indented YAML base: 1 record(s) fixed: 9 record(s)
```
- Not tested: no production traffic was run through a live proxy; the
evidence above is harness-driven against the real `segment()` and the
real `BM25Scorer`. Drop percentages are deliberately not quoted as a
headline because which records clear the adaptive cut is prompt- and
content-dependent, so a single fixture's percentage does not generalize.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation (both
docstrings)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — release-please generates it
from the Conventional Commit PR title.
## Additional Notes
Behavioral consequence worth a reviewer's attention: because a long
indented blob now segments into several records instead of one,
uniformly-indented output (YAML, container logs, indented JSON) becomes
eligible for partial or full dropping where it previously stayed whole.
That is the documented intent of the filter (`adaptive_threshold` states
an all-irrelevant output drops entirely), and a retrieval marker is
emitted by default (`ccr_inject_marker` defaults to `True`), so the
content stays recoverable outside the explicit no-CCR mode. It is called
out because it is this change's largest behavioral effect.
`relevance_max_records` defaults to `0`, which disables the record-count
cap, so producing more records is safe on the default path. A caller
that sets a small nonzero `relevance_max_records` could see a payload
that previously produced at or under the cap now exceed it and skip the
split, returning the content whole and lossless.
Pushes on this branch use `git push --no-verify`. The pre-push hook runs
`make ci-precheck`, which builds and tests the Rust workspace and
re-runs the Python suite; this change is Python-only and CI re-runs all
of it on clean hardware. The repo's git hooks were also not installed in
this checkout for the first pushes. They are installed now, and the full
pre-commit set (ruff, ruff-format, mypy, merge-conflict, plugin-version
sync) passes on the changed files, with the real output pasted above.
|
||
|
|
60af15f96f |
feat(content-router): lossless-first dispatch, cross-turn dedup, and A7 lossy-after-fold (#1818)
## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
eea667a720 |
feat(transforms): adaptive Otsu KEEP/DROP threshold (+ land relevance split on main) (#1726)
## Description Lands the prompt-conditioned relevance split **on `main`** and makes its KEEP/DROP threshold **adaptive**. Context: the Stage B work (#1722) was merged into the feature branch `tejas/proxy-lossless-mode` rather than `main`, so `relevance_split.py` never reached `main`. This PR cherry-picks that work onto `main` and adds the adaptive threshold on top, in three commits: 1. Prompt-conditioned KEEP/DROP tail split (Stage B) — segment LOG/SEARCH output into records, score each against the request's information need (user prompt + triggering tool-call args) via `headroom/relevance/`, keep relevant records verbatim, Kompress the low-relevance tail. Mode-agnostic (marker-free in lossless, retrieval-marker in CCR). 2. On by default with hot-path rails — background embedding-model pre-warm (BM25 until warm, never blocks a request) + optional `relevance_max_records` cap (default 0 = no cap). 3. **Adaptive Otsu threshold** (this PR's new work) — see below. ### Adaptive threshold The keep/drop cut is no longer a fixed constant. For each output we compute the natural relevant/irrelevant break in *its own* score distribution via **Otsu's method** (parameter-free — candidate cuts are the data's own values, no bins or magic numbers), floored by `relevance.relevance_threshold` so absolutely irrelevant records are never kept verbatim. The bar therefore moves with the content + prompt: a highly-relevant output keeps its top cluster and compresses the merely-moderate tail; a mostly-irrelevant output drops almost everything. All-equal scores fall back to the floor. Toggle via `relevance_adaptive_threshold` (default `True`). Closes # ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `relevance_split.py`: `adaptive_threshold()` + `_otsu_threshold()`; `plan_relevance_split(..., adaptive=True)` uses the adaptive cut, floored by `threshold`. - `content_router.py`: `relevance_adaptive_threshold` config (default `True`), threaded into the split. (Plus the Stage B split + default-on rails from the cherry-picked commits.) - `tests/test_relevance_split.py`: adaptive-threshold cases (bimodal split, floored, all-equal, moves-with-distribution) on top of the Stage B suite. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_relevance_split.py tests/test_transforms_content_router.py tests/test_lossless_mode.py -q 80 passed, 1 warning in 3.41s $ ruff check headroom/transforms/relevance_split.py headroom/transforms/content_router.py tests/test_relevance_split.py All checks passed! $ ruff format --check <changed files> 3 files already formatted $ mypy headroom/transforms/relevance_split.py headroom/transforms/content_router.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - **Environment:** local, Python 3.12.6. - **Steps:** `adaptive_threshold()` exercised directly on synthetic score distributions; `plan_relevance_split(adaptive=True)` and the real `ContentRouter._apply_strategy_to_content` path driven with a deterministic scorer + Kompress-tail stub (offline). - **Observed:** - Bimodal scores `[0.92, 0.88, 0.12, 0.05]` → cut lands in the valley (`0.12 < t < 0.88`), keeping the high cluster. - Mostly-irrelevant `[0.30, 0.28, 0.05, 0.03]` → cut floored at `0.25`. - All-equal scores → floor. - Higher-scoring distribution yields a higher cut than a lower one (bar adapts). - Router split still fires in both lossless and CCR mode; DIFF stays pure lossless; disabling the flag is byte-identical. - **Not tested:** live embedding model warm/latency at scale; end-to-end `/v1/retrieve` resolution of the CCR tail marker (marker plumbing itself is covered upstream). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes - Supersedes the orphaned #1722 merge (which landed on the feature branch, not `main`); this PR is the canonical path onto `main`. - **Follow-ups discussed:** TEXT-strategy extension (relevance split for plain prose, currently whole-block Kompress); batch multiple DROP runs into one Kompress call; eval of savings/fidelity on live traffic. - N/A: CHANGELOG (feature not yet released). |