mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description
The shared production `count_content_blocks` walker already counts
nested Anthropic tool-result/tool-use blocks on current main. This PR
adds direct OpenAI-provider regression coverage for that existing fix:
payload-sensitive counts, bounded nested media estimates, and the two
sample mapping orders. It makes no production counter change. Related
historical reports: #987, #828 and #813.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Add direct OpenAI regressions for tool-result/tool-use payload counts
and a lower count for a shorter tool-result fixture.
- Verify a nested 1 MB base64 image receives a bounded estimate rather
than text-tokenizing its bytes.
- Retain the sample mapping-order assertion with its scope clarified:
two sample maps have equal counts; this does not promise general
canonical key-order invariance. Main serializes the supplied mapping as
JSON.
- The production walker and both provider counters are unchanged by this
PR.
## Testing
- [x] Focused provider/shared-walker tests: 76 passed, one deprecation
warning.
- [x] Ruff lint and formatting pass on the changed test file.
- [x] Main-relative whitespace checks pass.
- [x] Independent fixture-count probe: 2809 tokens for the large
fixture, 26 for the short fixture.
- [x] Full hosted qualification passed on exact refreshed
head94c44fbe28065c7f31ef97ac2dcd51352d691dac (CI42232683655 and terminal
hosted rollup, no failures).
No old-main versus new-production-fix comparison is claimed; the
production fix is already on main.
## Real Behavior Proof
- Fresh maintainer environment: Windows, Python 3.12, tiktoken counter
for gpt-4o; current-main native DLL used for package startup, not
rebuilt here.
- Exact counter reproduction (run in the configured project
environment):
```python
from headroom.providers.openai import OpenAIProvider
c = OpenAIProvider().get_token_counter("gpt-4o")
big = "row data: " + ", ".join(f"item_{i}=value_{i}" for i in range(400))
small = "row data: item_0=value_0 [398 items compressed. hash=abc123]"
def message(text):
return [{"role": "user", "content": [{"type": "tool_result", "content": text}]}]
print(c.count_messages(message(big)), c.count_messages(message(small)))
```
- Observed: `2809 26`. These are counts for two supplied fixtures, not a
new compression run or an old-main/new-PR comparison; current main
already counts the blocks correctly.
- Focused verification: `pytest -q tests/test_providers/test_openai.py
tests/test_provider_counter_content_blocks.py` — 76 passed, one
deprecation warning.
- Not tested: full live `/v1/compress` HTTP flow, external provider
inference or a fresh branch-specific native build.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A
## Additional Notes
- The production fix for structured nested content landed on `main` via
the shared `count_content_blocks` walker; this branch adds the direct
OpenAI regressions on top of it.
- CHANGELOG is handled by Release Please for this repo, so that
checklist item is N/A.
````
````
Two things before you paste. First, check the boxes I left unchecked — I
marked mypy and "existing unit tests pass locally" as unchecked because
your own notes say you skipped the full suite, and I'd rather you not
assert something you didn't run. Tick them if you have since. Second, if
the template file in your checkout has drifted since #1839, diff against
`.github/PULL_REQUEST_TEMPLATE.md` before saving; the bot matches
headings literally.
Paste into the PR body, hit save, and the governance workflow should
re-run and swap the label.
If you'd rather I did the clicking, install the Claude for Chrome
extension and reconnect — then I can open the PR, fill the body, and
show you the diff before saving anything.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2681"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
### Maintainer current-main requalification
Exact head `94c44fbe28065c7f31ef97ac2dcd51352d691dac`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.
Clean current-main integration with upstream
c01cf07d94. Production shared
content-block walker already exists on main; remaining branch scope is
direct OpenAI regression tests only. Re-read current discussion and all3
Devin notes.76 provider/shared-walker tests pass,1warning; changed-test
Ruff lint/format and whitespace pass. Maintainer wording-only test
rename/docstring limits mapping-order assertion to its actual two
fixtures; assertions and counter logic unchanged. Actual counter probe
reports2809 large-fixture tokens versus26 short-fixture tokens using
gpt-4o; this is fixture counting, not an HTTP compression or old/new
production comparison. PR body is corrected to test-only scope and
provides exact reproducible code; no general canonical key-order
guarantee. Existing current-main DLL supports startup only, no new
native build/provider call. Formal approval awaits exact-head hosted
qualification; maintainers handle CI. Nothing merged.
Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.
---------
Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Co-authored-by: JD Davis <mxjerrett@gmail.com>