Commit Graph
4 Commits
Author SHA1 Message Date
a8df6f919f fix(transforms): recognise code fences whose info string is not a bare word (#4291)
## Description

`split_into_sections()` in `headroom/transforms/mixed_content.py`
recognises an opening code fence with `^```(\w*)\s*$`. Common fence tags
with non-word characters don't match: ```` ```c++ ````, ```` ```c# ````,
```` ```objective-c ````, ```` ```js {1,3} ````, ```` ```python
title="app.py" ````. When the opener is missed:

1. the fence line and code are kept as plain text;
2. the block's **closing** ```` ``` ```` matches the pattern (empty
tag), so it is taken as an **opening** fence;
3. everything after it (prose, JSON, until the next ```` ``` ```` or end
of input) becomes one `SOURCE_CODE` section with language `unknown`,
goes to the code compressor, and is re-emitted wrapped in ````
```unknown ```` … ```` ``` ````.

So the code block loses its closing fence, and the prose and data that
follow it come back inside a fake code block. Through
`headroom.compress()` this turns a tool result's JSON into a
JSON-escaped string (`\n` sequences) inside that stray fence (see the
proof below). It only takes one C++ or C# snippet in a mixed tool result
or fetched page.

Closes: N/A (no matching issue found)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/transforms/mixed_content.py`:
- `_CODE_FENCE_PATTERN` is now `^```([^\n`]*)$`. Group 1 is the whole
info string, any text without a backtick, so ```` ```c++ ````, ````
```js {1,3} ```` and ```` ``` c++ ```` (space before the tag, valid
Markdown) all open a fence. A closing ```` ``` ```` and a plain ````
```python ```` match as before. A line with a later backtick (````
```foo``` ````) still doesn't.
- New `_fence_language()` takes the info string's first word as the
language, and returns `"unknown"` for a bare fence, as before.
- It's a single quantifier, so a 200,000-character fence line takes
about 2 ms. While working on this I measured an overlapping variant
(`([^\s`]*)[^\n`]*`) at 3.3 s on a 20,000-character line, and avoided
it.
- `tests/test_mixed_content_sections.py`:
- `test_split_accepts_fence_info_strings_beyond_word_characters`,
parametrized over `c++`, `c#`, `objective-c`, `js {1,3}`, `python
title="app.py"`, ` c++` and `python\r`, checks the sections come out
PLAIN_TEXT / SOURCE_CODE(lang) / PLAIN_TEXT / JSON_ARRAY with the code
body intact.
- `test_fence_pattern_stays_linear_on_a_long_unclosed_fence_line` keeps
a 50,000-character fence line under 0.5 s. That's far above the linear
cost (under 1 ms), so a loaded runner won't trip it, but a quadratic
pattern (seconds) would. It passes on `main` too; it's there so any
future widening stays linear, as in
`test_multiline_regex_blank_runs.py`.

## Testing

- [x] Unit tests pass (`pytest`): `tests/test_mixed_content*.py`,
`tests/test_content_router*.py`,
`tests/test_transforms_content_router.py`,
`tests/test_code_compressor*.py`,
`tests/test_multiline_regex_blank_runs.py`, `tests/test_transforms/`,
not the full suite
- [x] Linting passes (`ruff check .`): on the changed files
- [ ] Type checking passes (`mypy headroom`): not run
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

New tests on `main` (fix reverted, tests kept):

```text
$ python -m pytest tests/test_mixed_content_sections.py -q
FAILED tests/test_mixed_content_sections.py::test_split_accepts_fence_info_strings_beyond_word_characters[c++-c++]
FAILED tests/test_mixed_content_sections.py::test_split_accepts_fence_info_strings_beyond_word_characters[c#-c#]
FAILED tests/test_mixed_content_sections.py::test_split_accepts_fence_info_strings_beyond_word_characters[objective-c-objective-c]
FAILED tests/test_mixed_content_sections.py::test_split_accepts_fence_info_strings_beyond_word_characters[js {1,3}-js]
FAILED tests/test_mixed_content_sections.py::test_split_accepts_fence_info_strings_beyond_word_characters[python title="app.py"-python]
FAILED tests/test_mixed_content_sections.py::test_split_accepts_fence_info_strings_beyond_word_characters[ c++-c++]
========================= 6 failed, 6 passed in 1.53s ==========================
```

With the fix:

```text
$ python -m pytest tests/test_mixed_content*.py tests/test_content_router*.py tests/test_transforms_content_router.py \
    tests/test_code_compressor*.py tests/test_multiline_regex_blank_runs.py tests/test_transforms -q
====================== 1044 passed, 88 skipped in 32.30s =======================

$ ruff check headroom/transforms/mixed_content.py tests/test_mixed_content_sections.py
All checks passed!
$ ruff format --check headroom/transforms/mixed_content.py tests/test_mixed_content_sections.py
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS 27.0.1 (arm64), Python 3.12.11, source tree on this
branch (native `headroom._core` from the 0.41.0 wheel),
`HEADROOM_OFFLINE=1`, public `headroom.compress()` API
- Exact command / steps: called `headroom.compress(messages,
model="gpt-4o", model_limit=128000, protect_recent=0)` on an
OpenAI-format conversation whose tool result is: one intro line, a ````
```c++ ```` block of two declarations, 40 prose lines, then a 400-row
JSON array (13.5K tokens). I printed the `transforms_applied`, every
fence line in the compressed tool result and its tail, on `main` and on
this branch. (Through the proxy, the same content only reaches the
mixed-content path once it is old enough to be compressed, so I used the
library entry point to keep the repro to one call.)
- Observed result: both runs take the mixed path (`router:mixed`). On
`main` the output has three fence lines (`c++`, a stray `unknown`, and a
closing one), the C++ block lost its closing fence, and the JSON tail
comes back as a JSON-escaped string inside the stray fence. On this
branch the C++ block keeps both fences, the prose stays prose, and the
JSON is compacted to CSV rows as the structured path does elsewhere.

  ```text
  == main
  tokens 13474 -> 4289 | transforms: [..., 'router:mixed:0.35']
  fence lines: ['```c++', '```unknown', '```']
    | Here is the Header file that we wrote for the project today.
    | ```c++
    | int add(int a, int b);
    | int sub(int a, int b);
    |
    | ```unknown
| Paragraph 0 explains the Design choice that we made for this module
today.
tail:
'...397,item397,team-a,ok\\n398,item398,team-a,ok\\n399,item399,team-a,ok\\n"\n```'
  == this branch
  tokens 13474 -> 4285 | transforms: [..., 'router:mixed:0.29']
  fence lines: ['```c++', '```']
    | ```c++
    | int add(int a, int b);
    | int sub(int a, int b);
    | ```
    |
| Paragraph 0 explains the Design choice that we made for this module
today.
tail:
'...397,item397,team-a,ok\n398,item398,team-a,ok\n399,item399,team-a,ok\n'
  ```
- Not tested: the same content through the live proxy's mixed-content
path (in a short session the turn is protected or below the size gates,
so nothing is compressed on either branch); Rust — there is no Rust
mirror of `split_into_sections`.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: fences tagged with non-word
characters (or a space before the tag) are now parsed as code blocks.
Content with only `\w` tags (```` ```python ````, ```` ```json ````,
bare ```` ``` ````) splits exactly as before; the existing 565
router/transform tests pass unchanged.
- Kill switch / disable path: N/A
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- An **unclosed** fence still runs to the end of the input, as an
unclosed ```` ```python ```` already does on `main`. This PR only
changes which openers are recognised. Making unclosed fences fall back
to plain text would change behaviour for every tag, so it seems better
as its own PR.
- The router re-wraps a fenced section with ```` ```{section.language}
````, so trailing info-string metadata (`{1,3}`, `title=...`) is not
re-emitted. That was already true for every fence; this PR only fixes
the opener being missed.
- `_extract_fenced_command()` in `content_router.py` has its own
`[\w.-]+` fence class. It only extracts a shell command for
read-detection, so I left it alone to keep this PR to one change.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-04 22:44:06 -06:00
HetCreepandClaude Opus 5.5 abbade9b4c fix(mixed-content): do not carve ISO-timestamped log rows as search results (#4463)
## Description

The mixed-content splitter's `_is_search_line` ORs the legacy
`^\S+:\d+:` pattern with the detector's predicate. That pattern matches
ISO-timestamped log rows (`2026-09-23T10:05:07Z ...`), which the
detector has rejected since #3736, so the splitter carved them into a
SEARCH_RESULTS section and `_compress_mixed` sent them to the lossy
SearchCompressor. `_is_search_line` now returns False for rows matched
by `_TIMESTAMP_ROW_RE` (the regex the detector already uses) before the
OR; real `path:NN:content` grep lines still carve.

Fixes #4462

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/transforms/mixed_content.py`: import `_TIMESTAMP_ROW_RE`
from `lossless_compaction` (the detector already imports it from there)
and return False for such rows in `_is_search_line`; docstring note.
- `tests/test_mixed_content_sections.py`: new test
`test_split_does_not_carve_timestamped_log_rows_as_search` (timestamped
rows produce no SEARCH_RESULTS section; a `src/app.py:10:` grep line
after the same prose and fence still does).

## Testing

Rebased onto upstream/main `3fef15d3` (2026-10-24): the new tests pass
(5 passed); with only the source change reverted, 1 fails. The
real-behavior run below was done on `e995eb9f`.

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

(`ruff check` and `ruff format --check` were run on the two changed
files only. `mypy headroom/transforms/mixed_content.py` reports one
error in `headroom/providers/model_metadata.py:79`, a file this PR does
not touch.)

### Test Output

```text
# new test with only the mixed_content.py change reverted
FAILED tests/test_mixed_content_sections.py::test_split_does_not_carve_timestamped_log_rows_as_search
AssertionError: assert <ContentType.SEARCH_RESULTS: 'search'> not in [<ContentType.PLAIN_TEXT: 'text'>, <ContentType.SOURCE_CODE: 'source_code'>, <ContentType.SEARCH_RESULTS: 'search'>]
1 failed, 4 passed in 1.12s

# with the fix: the new file, test_mixed_content_scan_cache.py and the content-detector tests
285 passed in 3.31s

ruff check: All checks passed!
ruff format --check: 2 files already formatted
```

## Real Behavior Proof

- Environment: Ubuntu on WSL2, Python 3.12.15, upstream/main e995eb9f
plus this commit, Rust core `headroom/_core.abi3.so` copied (untracked)
from the official headroom-ai 0.41.0 wheel. The real proxy `python -m
headroom.proxy.server` on 127.0.0.1:18991 with
`ANTHROPIC_TARGET_API_URL` pointing at a local stub on 127.0.0.1:18990
that records the tool_result text it receives.
- Exact command / steps: one `POST /v1/messages` whose last user message
holds a `tool_result` of 3,970 chars (one prose line, a python fence, 41
timestamped INFO log rows) plus a text block. Run once with the
`mixed_content.py` change reverted (`git apply -R`) and once with it
applied. Also the in-process router run (`ContentRouter().compress`) on
the same text. The stub log shows each request arriving, so the proxy
was pointed at the stub in both runs. Scripts: `router_proof.py`,
`stub.py`, `client.py`, `run.sh`.
- Observed result: before, the router routes the 41 rows as
SEARCH_RESULTS (974 -> 138 tokens); the stub receives 5 rows with the
date rewritten (`2026:9:23T10:05:00Z`) and `[41 matches compressed to 5.
Retrieve more: hash=...]`; proxy PERF `tok_before=1397 tok_after=252`.
After, the rows are a PLAIN_TEXT section; the stub receives all 41 rows
byte-for-byte (41 lines starting `2026-09-23`), PERF `tok_saved=0`
(`route_counts` shows `ratio_too_high`: the text compressor did not
shrink them, so nothing was changed). Excerpts below.
- Not tested: a real Anthropic or OpenAI API (stub upstream only);
streaming responses; OpenAI-shaped requests; Windows or macOS; a Rust
core built from this checkout (the 0.41.0 wheel's was used); other
timestamp shapes beyond the four `_TIMESTAMP_ROW_RE` already covers; the
effect on token savings across real workloads (this payload gets no
savings after the fix, by design).

Router, before (`before_router.out`):

```text
section PLAIN_TEXT 0 0
section SOURCE_CODE 1 3
section SEARCH_RESULTS 4 44
route PLAIN_TEXT TEXT 15 -> 15
route SOURCE_CODE CODE_AWARE 3 -> 3
route SEARCH_RESULTS SEARCH 974 -> 138
orig chars 3974 compressed chars 630
log rows kept verbatim: 0 of 41
```

Router, after (`after_router.out`):

```text
section PLAIN_TEXT 0 0
section SOURCE_CODE 1 3
section PLAIN_TEXT 4 44
route PLAIN_TEXT TEXT 15 -> 15
route SOURCE_CODE CODE_AWARE 3 -> 3
route PLAIN_TEXT TEXT 974 -> 974
orig chars 3974 compressed chars 3976
log rows kept verbatim: 41 of 41
```

Proxy, before (stub upstream received):

```text
client sent body_bytes=4415; tool_result chars=3970; got HTTP 200
2026:9:23T10:05:00Z worker-0 INFO request 300 handled status=200 path=/api/items latency_ms=35
2026:9:23T10:09:07Z worker-1 INFO upstream shard 3 drained
2026:9:23T10:05:06Z worker-2 INFO request 306 handled status=200 path=/api/items latency_ms=41
2026:9:23T10:05:12Z worker-0 INFO request 312 handled status=200 path=/api/items latency_ms=47
2026:9:23T10:05:18Z worker-2 INFO request 318 handled status=200 path=/api/items latency_ms=53
[... and 36 more matches in 2026]
[41 matches compressed to 5. Retrieve more: hash=06f542f16b8e09223e33e3e7]
proxy: route_counts={'content_blocks': 1, 'cache_miss': 1} compressed=1 frozen=2 msgs=3 ... tok_before=1397 tok_after=252
```

Proxy, after:

```text
client sent body_bytes=4415; tool_result chars=3970; got HTTP 200
2026-09-23T10:05:00Z worker-0 INFO request 300 handled status=200 path=/api/items latency_ms=35
2026-09-23T10:05:06Z worker-2 INFO request 306 handled status=200 path=/api/items latency_ms=41
... (41 rows, all unchanged)
2026-09-23T10:09:07Z worker-1 INFO upstream shard 3 drained
proxy: route_counts={'ratio_too_high': 1, 'content_blocks': 1, 'cache_miss': 1} compressed=0 frozen=2 msgs=3 ... tok_before=1397 tok_after=1397
```

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: yes, narrowly: mixed payloads
containing ISO/syslog/slash-date/bare-clock timestamped rows no longer
send those rows to the SearchCompressor, so they are no longer reduced
to a few rows with a rewritten date
- Kill switch / disable path: none; no config involved
- Unsafe override required: no
- Qualification impact: none expected
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Design choice to review: the guard sits in the splitter's
`_is_search_line` and reuses the lossless fold's `_TIMESTAMP_ROW_RE`, so
the splitter and detector cannot drift apart again. The alternative is
dropping the legacy `^\S+:\d+:` pattern entirely; I kept it because the
comment at `mixed_content.py:40-46` says it is there so previously
carved lines still carve. Documentation: no doc describes this routing.

Produced with AI assistance (Claude Code) operated by me (HetCreep); the
reproduction was run on my machine on a clone of the repository.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4463"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 23:32:52 -05:00
lorenzozane f085c47402 fix(transforms): classify grep -A/-B/-C context lines as search results (#3599)
## Description

`grep -A`/`-B`/`-C` context lines (both the real GNU `path-NN-content`
shape and the reported `path:NN-content` shape) were classified as
`PLAIN_TEXT`, so code inside them was routed to the word-dropping
Kompress prose compressor and silently corrupted (`=` dropped from
assignments, whole lines removed) behind a plausible `[N words
compressed to M]` marker.

`content_detector._is_search_result_line` only accepted the
`path:NN:content` match shape, and the mixed-content splitter had the
same gap. This change recognizes both context-line shapes as
`SEARCH_RESULTS` with path-shape guards (prefix must contain `/` or `.`,
same `</>=` exclusions as the colon branch) so dates and dashed prose
stay out.

Closes #3580

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `_is_grep_context_line` predicate plus a
`_GREP_COLON_DASH_PATTERN` branch in
`headroom/transforms/content_detector.py`, sharing a
`_prefix_looks_like_path` guard so the three grep-line shapes cannot
drift.
- Taught the mixed-content splitter
(`headroom/transforms/mixed_content.py`) to carve the same shapes into
`SEARCH_RESULTS` sections via the shared predicate.
- Added one detection-signal table row in
`docs/content/docs/text-and-logs.mdx`.
- Added 6 regression tests (both separator shapes, mixed match/context
blocks with `--` separators, date/dashed-prose/markup guard cases).
- Thresholds, confidences, and the Rust production chain are untouched;
the SearchCompressor already parses `-` context lines and keeps them
verbatim.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
python -m pytest tests/test_transforms_content_detection.py tests/test_mixed_content_sections.py \
  tests/test_mixed_content_scan_cache.py tests/test_transforms_content_router.py \
  tests/test_transforms/test_content_router.py tests/test_content_router_detection_dedup.py \
  tests/test_transforms/test_detect_fallback_1123.py tests/test_search_compressor.py \
  tests/test_transforms_search_compressor.py tests/test_search_compressor_cjk.py \
  tests/test_bash_search_lossless_fold.py -q
527 passed in ~19s
ruff check: clean. ruff format --check: clean. git diff --check: clean.
```

`mypy` reports no errors in the two changed source files (10
pre-existing errors elsewhere, e.g. proxy/server.py), so the type-check
box above stays unchecked.

## Real Behavior Proof

- Environment: Linux, submitter worktree `.venv` (native
`headroom._core` built via maturin).
- Exact command / steps: fed real `grep -A` output through the content
router and the SearchCompressor; the exact `if manifest.scope != ...`
line from the issue was checked verbatim after compression.
- Observed result: match lines and context lines both route to `search`;
the SearchCompressor keeps the issue's line byte-identical (before the
fix, context lines routed to Kompress and were corrupted).
- Not tested: full `pytest tests/` suite, `cargo test --workspace`,
parity suite, and the live native (Magika/ONNX) classification path —
upstream CI still required.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A (not rollout-gated).
- Stable/default behavior changed: Yes — grep `-A`/`-B`/`-C` context
lines now classify as search results instead of prose (the intended fix
for #3580); all other inputs unchanged.
- Kill switch / disable path: Not applicable; no flag, no new
dependency, no I/O.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A.

## Additional Notes

- Prior attempt #3582 (closed unmerged) diagnosed the same area; this
fix additionally covers the reported `path:NN-content` shape, which a
dash-dash-only predicate would miss.
- Follow-up (not this PR): port the predicate to the Rust oracle
`crates/headroom-core/src/transforms/content_detector.rs`; the locked
production chain in `detection.rs` is intentionally untouched. No new
parity fixture was added for the same reason.
- Out of scope, unchanged pre-existing behavior: pathless single-file
`grep` lines (`30:def...`) and drive-letter paths (`C:\...`) still
classify as before.
2026-09-17 15:42:38 -07:00
JD Davis 9bacf4810f refactor(transforms): isolate mixed content parsing (#1939)
## Description

Extracts mixed-content parsing out of the large `ContentRouter` module
into a pure transform-domain module. The router still exports the
existing compatibility names, but section typing, mixed-content
indicators, section splitting, and JSON block extraction now live in a
focused domain object/function layer.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- Added `headroom.transforms.mixed_content` with `ContentSection`,
`mixed_content_indicators`, `is_mixed_content`, `split_into_sections`,
and JSON block extraction.
- Updated `ContentRouter` to delegate mixed-content debug indicators and
parsing to the new module while preserving legacy imports from
`content_router.py`.
- Added direct unit coverage for mixed-content detection, section
boundaries, and JSON delimiters inside string literals.
- Included the LiteLLM callback signature compatibility shim needed for
repo-wide mypy while the earlier architecture PRs are still open.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
python -m pytest tests/test_mixed_content_sections.py tests/test_transforms_content_router.py tests/test_litellm_callback.py tests/test_compress_api.py::TestLiteLLMCallback -q
50 passed in 6.82s

python -m ruff check .
All checks passed!

python -m ruff format --check .
1095 files already formatted

python -m mypy headroom --ignore-missing-imports
headroom\proxy\server.py:1457: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
headroom\proxy\server.py:1468: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
Success: no issues found in 409 source files
```

## Real Behavior Proof

- Environment: Windows, Python 3.13.13, clean worktree
`C:\git\headroom-pr-slice6`
- Exact command / steps: ran the pytest, Ruff, format, and mypy commands
listed above.
- Observed result: mixed-content parsing behavior remains covered
through existing router tests and new direct tests; repo-wide lint/type
checks pass.
- Not tested: full pytest suite and Docker/native CI jobs are left to
GitHub Actions.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A.

## Additional Notes

- Documentation, changelog, and screenshots are N/A for this internal
refactor.
- Manual UI testing is N/A; this is pure transform parsing logic.
- Comment checklist is unchecked because the extracted functions are
small and covered by direct tests.
2026-07-10 19:28:21 -05:00