mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 16:45:34 +08:00
feat/https-compose-caddy
1
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7c9fbed61a |
fix(router): preserve unmarked HTML tool output (#3778)
## Description `HTMLExtractor` can reduce a tool result to article prose without a CCR retrieval marker. The existing tool ground-truth guard omitted `CompressionStrategy.HTML`, so the deleted script or structured data could not be recovered. This is independent of #3627's HTML-vs-MIXED routing change and #3774's test isolation change. Closes #3775. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `HTML` to the lossy/unmarked strategy sets used by `ContentRouter.apply` and the provider-neutral shell-output unit path. Unmarked HTML extraction now leaves tool output unchanged; a future extraction with a real CCR marker can still be accepted. - Add three real-`trafilatura` regression cases for Anthropic `tool_result`, OpenAI `role=tool`, and a structured `local_shell_call_output`. The tests failed on the original code and passed with the two set entries. - Install the existing `html` extra in CI's `test-extras` job and run the HTML extractor and new reversibility tests there. The main pytest shards currently install `[dev]` without `html`; the `[sandbox]` smoke job installs it but does not run pytest. ## Testing - [ ] Unit tests pass (`pytest`) — the full suite was not run; the focused run below passed. - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) — local Windows run has the unrelated errors listed below. - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text Before the source change: python -m pytest tests/test_html_tool_reversibility.py -q --tb=short 3 failed (unmarked HTML replaced both message shapes and the provider unit). After the source change, Windows Python 3.12.13 with trafilatura 2.0.0: python -m pytest tests/test_transforms/test_html_extractor.py tests/test_html_tool_reversibility.py tests/test_compression_units.py tests/test_compression_batches.py tests/test_content_router_tool_role_reversibility.py tests/test_transforms_content_router.py tests/test_no_ccr_lossy.py -q --tb=short 137 passed in 9.55s ruff check . All checks passed! ruff format --check . 1642 files already formatted git diff --check (no output) python -m pytest tests/test_transforms/test_dense_line_elider.py -q --tb=short 9 passed on current head after #3774's test isolation change merged into main. mypy headroom 2 errors in unmodified files on Windows: headroom/fileperms.py:71 (os.fchmod), headroom/integrations/langchain/chat_model.py:394 (truthy-function). Hosted Linux CI runs a separately pinned mypy version. ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.13, source checkout at `df8ebdf97` plus this patch, `uv sync --frozen --extra dev --extra html` using that interpreter, `trafilatura==2.0.0`. `ContentRouterConfig(enable_kompress=False, min_section_tokens=10)`, default CCR marker setting, local `gpt-4o` token counter. Windows selected the pure-Python content detector; no live provider request was made. - Exact command / steps: run this from the repository root after installing those extras: ```powershell @' from headroom.providers import OpenAIProvider from headroom.tokenizer import Tokenizer from headroom.transforms.content_router import ContentRouter, ContentRouterConfig js = "var a=1;function f(x){return x*2};" * 300 html = "<!doctype html><html><head><script>" + js + "</script></head><body><p>the answer is 42</p></body></html>" messages = [ {"role": "user", "content": "check the site"}, {"role": "assistant", "content": [{"type": "tool_use", "id": "t1", "name": "Bash", "input": {"command": "curl -s x"}}]}, {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "t1", "content": html}]}, {"role": "assistant", "content": "ok"}, {"role": "user", "content": "and now?"}, ] tokenizer = Tokenizer(OpenAIProvider().get_token_counter("gpt-4o"), "gpt-4o") router = ContentRouter(ContentRouterConfig(enable_kompress=False, min_section_tokens=10)) direct = router.compress(html, context="tool_result") block = router.apply(messages, tokenizer).messages[2]["content"][0]["content"] out = block if isinstance(block, str) else block[0]["text"] print("direct:", direct.strategy_used.value, len(html), "->", len(direct.compressed), repr(direct.compressed)) print("tool_result:", len(html), "->", len(out), "same:", out == html, "marker:", "Retrieve original: hash=" in out) '@ | .\.venv\Scripts\python.exe - ``` - Observed result: the unfixed router replaced 10,294 HTML characters with 16 unmarked characters; after this patch the tool result retained all 10,294 original characters. ```text Before: direct: html 10294 -> 16 'the answer is 42' tool_result: 10294 -> 16, same: False, marker: False After: direct: html 10294 -> 16 'the answer is 42' tool_result: 10294 -> 10294, same: True, marker: False ``` A second unmocked probe with the same fixture yielded `role=tool: 10294 -> 10294, same=True` and `local_shell_call_output: html, modified=False, same=True, reason=lossy_unrecoverable_tool_output`. - Not tested: the reporter's actual page, a live LLM/provider request, Linux/native detection, or a full local pytest run with `html` installed. ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: N/A. - Stable/default behavior changed: yes; tool outputs whose HTML extraction is lossy and unmarked are retained verbatim, so those outputs use more tokens. Direct `ContentRouter.compress()` extraction is unchanged. - Kill switch / disable path: `enable_html_extractor=False` prevents HTML extraction. `ccr_inject_marker=False` remains the existing intentional opt-out for the `ContentRouter.apply` guard. - Unsafe override required: none. - Qualification impact: CI's existing `test-extras` job now installs `[html]` and runs real HTML tests; main shards are untouched. The #3774 test isolation change is in current main, outside this PR diff. - Rollback path: revert this commit; no state migration or new dependency is introduced. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Current head `d361388e` merges latest `main`, including [#3790](https://github.com/headroomlabs-ai/headroom/pull/3790)'s fix for the unrelated SQLite test that failed in the previous CI run. On this head, that test, the 137 focused HTML/router/unit tests, and all 9 dense-line-elider tests passed locally; Ruff and `git diff --check` passed. Exact-head hosted checks are pending. ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A: no new public API or setting) - [ ] My changes generate no new warnings (full suite not verified locally) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (focused tests pass; full local suite was not run) - [x] I did **not** edit `CHANGELOG.md`, it is generated by release-please from my Conventional Commit PR title ## Additional Notes This keeps the existing reversible-tool-output contract: an unmarked lossy transform cannot replace tool ground truth. It deliberately does not add a retrieval store or change HTML routing. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |