Commit Graph
12 Commits
Author SHA1 Message Date
Tejas ChopraandClaude Opus 5.5 4bf4ae870d refactor(parser): deprecate find_tool_units and get_message_content_text (#4108)
## Description

This PR deprecates two functions in `headroom/parser.py`,
`find_tool_units()` and `get_message_content_text()`. Nothing in the
product calls either one; their only callers are tests in
`tests/test_parser.py`. `find_tool_units` looks like a leftover of the
rolling-window stage, which has been retired.

`headroom.parser` is imported by the org's consumer repos, so these
functions are treated as public API and are deprecated rather than
deleted. Neither has a replacement.

Each function now emits a `DeprecationWarning` with `stacklevel=2`, and
its docstring gets a `.. deprecated::` note. Behaviour is otherwise
unchanged.

Evidence: a word-boundary search for both names across `headroom`,
`crates`, `plugins`, `sdk`, `examples`, `benchmarks`, `scripts`, `e2e`,
`docs`, `wiki`, `.github`, `pyproject.toml` and the consumer repos finds
only the definitions and `tests/test_parser.py`.

Refs #4136

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `headroom/parser.py`: add `warnings.warn(..., DeprecationWarning,
stacklevel=2)` and a `.. deprecated::` note to `find_tool_units` and
`get_message_content_text`.
- `tests/test_parser.py`: the existing tests for both functions stay.
They cover the behaviour during the deprecation period.
  - They get a `filterwarnings` mark, so they do not flood the output.
- Two small new tests assert that each warning fires and that the result
is unchanged.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
# both deprecation messages escalated to errors, so any product call path would fail:
$ pytest -q -p no:cacheprovider -W "error:find_tool_units is deprecated:DeprecationWarning" -W "error:get_message_content_text is deprecated:DeprecationWarning" tests/test_parser.py tests/test_transforms tests/test_compress_api.py
684 passed in 22.17s
$ uvx ruff@0.16.8 check headroom/parser.py tests/test_parser.py
All checks passed!
$ python -m mypy headroom/parser.py --ignore-missing-imports
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS, Python 3.12.13, this branch checked out in a
worktree, repo .venv; temp HOME and HEADROOM_WORKSPACE_DIR, offline.
- Exact command / steps: (1) The shared smoke harness `smoke.sh
<worktree>`: it starts a real proxy from this checkout against a local
fake upstream and sends one Anthropic and one OpenAI request with a
compressible tool result. (2) A direct command on this PR's own path:
`HOME=$TMP/home HEADROOM_WORKSPACE_DIR=$TMP/ws python -W
always::DeprecationWarning -c "from headroom import parser;
print(parser.find_tool_units([{'role': 'assistant', 'tool_calls':
[{'id': 'c1'}]}, {'role': 'tool', 'tool_call_id': 'c1'}]));
print(parser.get_message_content_text({'content': 'hi'}))"`.
- Observed result: Both requests return 200 and `/stats` shows savings,
with only the expected offline-mode warnings in proxy.log. The direct
command's output is below.
- Not tested: Outside callers of these functions; that is the point of
deprecating first.

```text
$ bash smoke.sh <worktree>
branch: cleanup/core-misc-parser-deprecate @ d31b7f7b
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines ($SMOKE_TMP/ws/logs/proxy-65488.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: $SMOKE_TMP)

$ HOME=$TMP/home HEADROOM_WORKSPACE_DIR=$TMP/ws python -W always::DeprecationWarning -c "from headroom import parser; print(parser.find_tool_units([{'role': 'assistant', 'tool_calls': [{'id': 'c1'}]}, {'role': 'tool', 'tool_call_id': 'c1'}])); print(parser.get_message_content_text({'content': 'hi'}))"
<string>:1: DeprecationWarning: find_tool_units is deprecated and will be removed in a future release; it has no replacement.
<string>:1: DeprecationWarning: get_message_content_text is deprecated and will be removed in a future release; it has no replacement.
[(0, [1])]
hi
```

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A; deprecation only.
- Stable/default behavior changed: No. Return values are unchanged, and
only direct callers see a warning.
- Kill switch / disable path: Standard Python warning filters.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.

When the functions are removed in a later release, `TestFindToolUnits`,
`TestGetMessageContentText` and
`TestStreamingToolCallObjects.test_find_tool_units_with_sdk_tool_call`
go with them. `_coerce_tool_call_to_dict` stays, because
`parse_message_to_blocks` uses it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:03:48 -07:00
Zhibo Zhang (Amur)andJD Davis d0e9c4ca48 fix(parser): stop counting HTML comments twice in waste signals (#3942)
## Root cause

In `detect_waste_signals`, the HTML noise signal joins matches from two
patterns:

```python
html_matches = HTML_TAG_PATTERN.findall(text) + HTML_COMMENT_PATTERN.findall(text)
```

`HTML_TAG_PATTERN` is `<[^>]+>`, which matches a full `<!-- ... -->`
comment (the span contains no bare `>`), and `HTML_COMMENT_PATTERN`
matches the same span again. Every comment is therefore counted twice,
and tags inside a comment are counted a third time as separate noise.

Measured on a text containing a single 608-character comment:
`html_noise_tokens` reports 304 instead of 152. The inflated signal
feeds `WasteSignals.total()`, which skews compression thresholds and
overreports noise in telemetry.

## Fix

Strip comments from the text before tag matching:

```python
comment_matches = HTML_COMMENT_PATTERN.findall(text)
html_matches = HTML_TAG_PATTERN.findall(HTML_COMMENT_PATTERN.sub("", text)) + comment_matches
```

Each comment is now counted exactly once (via the comment pattern), and
tags inside comments are no longer counted as separate noise. Plain tag
counting is unchanged: tags outside comments still match the same way on
the comment-free remainder.

## Test

Added four regression tests to `TestDetectWasteSignals`:

- a 608-character comment reports exactly its own token count (failed
before: reported double)
- tags inside a comment are not counted separately (failed before)
- mixed tags and comments each contribute exactly once (failed before)
- plain tags keep their existing count (guard, passes before and after)

Full `tests/test_parser.py` suite: 98 passed. `ruff check` and `ruff
format --check` clean on both touched files.

Same waste-signal accounting accuracy area as #1102 (whitespace signal
always zero), which this follows.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3942"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-04 13:55:28 -07:00
f19bc9a9c3 fix(parser): whitespace waste signal always reported zero (#1102)
## Description

`detect_waste_signals` in `headroom/parser.py` computed the
`whitespace_tokens` waste signal as `0` for every input, so collapsible
whitespace never contributed to `WasteSignals.total()`.

The root cause was this normalization step:

```python
normalized_text = " ".join(ws_matches)
```

That keeps each matched whitespace run verbatim and only inserts one
extra space between runs. The normalized text is therefore never shorter
than the original matched whitespace text. This PR collapses each
matched run to one space instead:

```python
normalized_text = " "  len(ws_matches)
```

## Type of Change

- [x] Bug fix
- [x] Test coverage
- [x] Changelog entry

## Changes Made

- Fixed the whitespace waste calculation in `headroom/parser.py`.
- Tightened the existing excessive-whitespace test so it asserts real
savings instead of allowing `0`.
- Added a regression test proving the signal scales with long whitespace
runs.

## Testing

### Test Output

```text
88 parser tests passed
ruff check: passed
ruff format --check: passed
```

- [x] `uv run pytest tests/test_parser.py` -> `88 passed`
- [x] `ruff check`
- [x] `ruff format --check`

## Real Behavior Proof

- Environment: current PR head, pure-Python parser path.
- Exact command / steps: ran `uv run pytest tests/test_parser.py`, Ruff,
and formatting checks with long whitespace-run coverage.
- Observed result: 88 parser tests pass; fixture waste is 2 tokens and a
realistic padded log reports 65 rather than zero.
- Not tested: compiled Rust, hnswlib, torch, or live proxy paths because
the changed path is pure Python.

Before this fix, the existing whitespace fixture and a padded-log case
both returned `signals.whitespace_tokens == 0`, even though the matched
whitespace was collapsible.

After this fix:

```text
fixture whitespace_tokens: 2
realistic padded log whitespace_tokens: 65
```

The new tests fail against `main` and pass with this branch.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Scope Notes

I did not retest compiled Rust, `hnswlib`, torch, or live proxy paths on
this Windows machine. The changed path is pure Python and covered by
`tests/test_parser.py`.

Co-authored-by: nyxst4ck <289980115+nyxst4ck@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-01 22:20:44 -05:00
Tejas ChopraandClaude Opus 5 96dfc7005f chore: finish the org and docs-domain migration (#3389)
Two stale identities left over from moves that were only half applied.

## GitHub org

`install/state.py` already migrated Docker images to
`ghcr.io/headroomlabs-ai/headroom` and carries a
`_DEPRECATED_IMAGE_REPO` constant that rewrites the old path on load —
so this move was decided long ago. Everything else still said
`chopratejas/headroom`.

Rewritten across packaging (`pyproject.toml`, `Cargo.toml`), both plugin
marketplaces, the agent-hooks manifests, install scripts, CI templates,
crate docs and tests.

**Two deliberate exclusions:**
- **`ghcr.io/chopratejas/headroom` is untouched everywhere.** It is the
*input* to `_migrate_deprecated_image()` and to the tests covering it.
Rewriting it would silently disable the migration.
- **`CHANGELOG.md` keeps its 238 historical references.** GitHub's
rename redirect resolves them, and rewriting would bury this diff in
noise.

**HuggingFace model ids are NOT affected and were never in scope.**
`chopratejas/kompress-base`, `kompress-v2-base`, `kompress-finance`,
`technique-router`, `technique-router-onnx` and
`siglip-image-encoder-onnx` are account-scoped identifiers on a
different service. All six were validated against the HF API before this
change:

| Model | HTTP |
|---|---|
| `kompress-base` | 200 |
| `kompress-v2-base` | 200 |
| `technique-router` | 200 |
| `technique-router-onnx` | 200 |
| `siglip-image-encoder-onnx` | 200 |
| `kompress-finance` | **401 (private)** |

The 401 is unrelated to this migration, but worth knowing:
`kompress_compressor.py:1255` references it, so that path is not
reachable by an OSS user without access.

**One functional fix, not cosmetic:** `_marketplace_source()`
(`headroom/cli/init.py`) returned the old org as its fallback, so plugin
installs were pointed at the retired repo.

## Docs domain

The site moved to `docs.headroomlabs.ai`, but eleven references still
named the vercel.app host — including **`llms.txt` (27 links)**, which
is served at `docs.headroomlabs.ai/llms.txt` and read by LLM crawlers,
the **dashboard's own Documentation link**, and a `headroom wrap` help
message. The docs app's canonical-URL fallbacks were fixed separately in
#3388.

## Also folded in

`RUST_DEV.md` corrections found while verifying the above:
- `make build-wheel` points at `Cargo.toml`, not `pyproject.toml`
- the Rust toolchain is pinned, not tracking `stable`
- `headroom-simulators` was missing from the workspace listing
- the Python `CompressionStore` defaults to **SQLite**, not in-memory,
when `HEADROOM_CCR_BACKEND` is unset (verified by execution) — this one
matters for multi-worker reasoning

And the install scripts pointed at `docs/docker-install.md`, a path that
no longer exists; they now point at the live docs page.

## Verification

297 tests pass, including the plugin-manifest, install-state and
release-workflow suites that assert on these exact strings. `ruff check`
clean. Unrelated lockfile churn (`uv.lock`, both `package-lock.json`) is
deliberately excluded.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-02 15:53:35 +05:30
Lakshya Sharma 5986c2260f fix(agno): tolerate streaming tool-call SDK objects in parser (#1312) (#1336)
## Description

`HeadroomAgnoModel` blows up as soon as you stream a response that
includes a tool call:

```
ERROR    Error in Agent run: 'ChoiceDeltaToolCall' object has no attribute 'get'
```

When streaming, Agno hands us `Message.tool_calls` as the raw OpenAI SDK
objects (`ChoiceDeltaToolCall`), not the OpenAI-style dicts we get on
the non-streaming path. Those objects are pydantic models — attribute
access only, no `.get()`. Our shared parser in `headroom/parser.py`
walks `tool_calls` and calls `tc.get("function", {})` / `tc.get("id")`,
so it throws `AttributeError`, and the Agno wrapper surfaces that as a
`RunErrorEvent` that kills the run.

I reproduced the exact error against `parse_message_to_blocks` with a
stand-in object before writing the fix.

Closes #1312

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `parser.py`: added a small `_coerce_tool_call_to_dict()` helper that
takes a tool_call which might be a dict or a provider SDK object and
returns the canonical OpenAI dict (reading `.function.name` /
`.function.arguments` via `getattr`). Wired it into both `.get()` sites,
`parse_message_to_blocks` and `find_tool_units`. Dicts pass straight
through (same object, no copy); `None` or anything unexpected degrades
to `{}` instead of raising. The proxy, langchain, and strands
integrations go through this same parser, so they get the same
hardening.
- `integrations/agno/model.py`: normalize `tool_calls` to dicts in
`_convert_messages_to_openai`, so the Agno `Message` objects we rebuild
and hand back also carry clean dicts and Agno's own re-serialization
can't trip over the same thing.

## Testing

- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_parser.py -q
93 passed
# 87 existing + 6 new regression tests in TestStreamingToolCallObjects.

$ python -m pytest tests/test_integrations/agno/test_model.py -q
59 skipped
# These skip locally because agno isn't installed here
# (pytestmark = skipif(not AGNO_AVAILABLE)); they run in CI. The new
# test_convert_messages_normalizes_streaming_tool_call_objects is in this file.
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.10, local clone. agno and the Rust
`headroom._core` extension aren't installed/built in this checkout.
- Exact command / steps: built a stand-in `ChoiceDeltaToolCall`
(attribute access, no `.get()`, nested `.function.name`/`.arguments`)
matching the OpenAI SDK streaming type, ran it through
`parse_message_to_blocks` and `find_tool_units` before and after the
change, then ran the parser suite.
- Observed result: before the fix I got `AttributeError:
'ChoiceDeltaToolCall' object has no attribute 'get'` — the exact error
from the issue. After the fix the same input produces a proper
`tool_call` block (correct `tool_call_id` / `function_name`) and
`find_tool_units` pairs the assistant call with its tool response.
Parser suite is green at 93 passed.
- Not tested: a full live `agent.run(stream=True)` against a real
OpenAI-compatible backend, since agno isn't installed here. That path is
covered by the Agno test in CI. I reproduced the failure at the parser
boundary instead, which is where the actual crash happens.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- No docs change — this is an internal robustness fix at the parsing
boundary, no user-facing API.
- CHANGELOG.md is generated from the Conventional Commit subject via
release-please, so the `fix(agno):` commit gets picked up on its own.
- I went with two layers (parser + the Agno boundary) on purpose so
neither our pipeline nor Agno's re-serialization can hit it. Since the
parser helper is shared, the proxy/langchain/strands paths are covered
too.
2026-06-24 09:52:15 -05:00
Focused Instabilityandintegration-check 7d4ae86ec0 feat(parser): detect re-issued identical tool calls as reread waste (#909)
Fixes #908

## Problem

Reread waste detection matches `tool_result` blocks by exact
`content_hash` only. Two gaps hide a common waste pattern — the agent
re-issuing the *same tool call* and paying full price for a
near-identical result:

1. **Byte-different results escape matching.** Same tool, same
arguments, but the second result differs trivially (embedded mtimes,
timestamps, ordering) → different hash, zero reread counted.
2. **Anthropic `tool_use` parts were dropped entirely** in
`parse_message_to_blocks` — only OpenAI-style `message.tool_calls`
produced `tool_call` blocks, so Anthropic/Strands traffic had no
call-side record at all.

## Fix

- Parse Anthropic `tool_use` / Strands `toolUse` content parts into
`tool_call` blocks (same shape as the OpenAI path: `function_name`,
`tool_call_id` flags).
- Tag every `tool_call` block with a canonical `call_key` = hash(name +
arguments re-serialized with sorted keys), so `'{"path": "a.py",
"lines": 100}'` (OpenAI JSON string) and `{"lines": 100, "path":
"a.py"}` (Anthropic dict) hash equal — covered by a cross-format parity
test.
- Second reread pass in `parse_messages` groups calls by `call_key`:
repeat invocations past the existing `REREAD_ADJACENT_GAP` polling guard
count their **result** tokens into `reread_tokens`, subject to the
existing `REREAD_MIN_TOKENS` floor. Results already counted by the
content-hash pass are skipped, so byte-identical repeats are never
double-counted.

No new `WasteSignals` field — a byte-different re-fetch of an identical
call is reread waste by the existing definition. Detection is
Python-only (`parser.py`); no Rust parity surface.

## Proof

Re-reading the same file twice, 7 messages apart, second serve differing
only by an mtime line:

```
main:        tool_call blocks: 2, reread_tokens: 0
this branch: tool_call blocks: 2, reread_tokens: 381
```

## Testing

- 11 new tests (`TestCallArgMatchReread`): changed-result repeat counted
(OpenAI + Anthropic + Strands formats), byte-identical repeat counted
exactly once, polling gap skipped, different args not matched, sub-floor
results skipped, repeat without result ignored, canonical-key
normalization, cross-format call_key parity.
- Full `tests/test_parser.py` suite: 87 passed. Consumer regression
sweep (reporting, config, request outcome, read lifecycle,
observability, storage): 188 passed.
- `ruff check` + `ruff format --check` + `mypy headroom/parser.py`
clean.

Co-authored-by: integration-check <integration@local>
2026-06-12 17:16:56 -05:00
Focused InstabilityandAsh Rhodes 5f1d88ad27 feat: detect re-served tool results as over-compression waste signal (#854)
Closes #853

## What

Adds a `reread` waste signal: identical `tool_result` content appearing
at more than one message position means the agent re-fetched something
already in context — the dominant failure signature of over-compression
(Manus context-engineering; JetBrains "Complexity Trap",
arXiv:2508.21433). Per-request savings can't see this cost; this signal
makes it visible.

- `WasteSignals.reread_tokens` — new field, in `total()`, exported as
`"reread"` in `to_dict()`.
- `parse_messages()` groups `tool_result` blocks by their **existing**
`content_hash` and counts every repeat beyond the first serve. No new
hashing or tokenization; one O(blocks) dict pass.
- `REREAD_MIN_TOKENS = 50` guard: short outputs ("ok", empty diffs)
legitimately repeat and are skipped. Duplicates within a single message
(same `source_index`) are not counted.
- Works across all formats the parser already normalizes to
`tool_result` blocks: OpenAI `role=tool`, Anthropic `tool_result`,
Strands/Bedrock `toolResult` (#813/#815).
- Flows through existing generic plumbing with zero handler changes:
pipeline → `RequestOutcome.waste_signals` → Prometheus
`headroom_waste_signal_tokens_total{signal="reread"}` → dashboard "Waste
Detected" panel. Dashboard gains label/color entries for the new key.

## Tests

7 new tests in `tests/test_parser.py::TestRereadDetection` (red before,
green after): OpenAI + Anthropic format detection, repeat-counting
semantics (first serve free), single-occurrence, short-duplicate guard,
same-message guard, `total()`/`to_dict()` participation. Updated 2
exact-shape assertions in `tests/test_config.py`.

Local runs: `tests/test_parser.py` (72 passed), `tests/test_config.py` +
outcome/reporting/observability/storage/proxy-hooks suites (190 passed),
`tests/test_canonical_pipeline.py` +
`tests/test_proxy_pipeline_lifecycle.py` (11 passed). `ruff check` +
`ruff format --check` clean.

## Real behavior proof

**Setup:** macOS (Darwin 25.5), Python 3.11.9, this branch, real proxy
server (`python -m headroom.proxy.server --port 18970
--anthropic-api-url http://127.0.0.1:18971`) with a local mock Anthropic
upstream returning a canned `/v1/messages` response (no real key
needed).

**Steps:** POSTed an Anthropic-format conversation to the live proxy:
agent fetches a 14 KB JSON log array via `get_logs` tool, then fetches
the identical content again under a different `tool_use_id` (the
re-read).

**Observed result** — `curl http://127.0.0.1:18970/metrics` after the
request:

```
# HELP headroom_waste_signal_tokens_total Tokens attributed to detected waste signals
# TYPE headroom_waste_signal_tokens_total counter
headroom_waste_signal_tokens_total{signal="json_bloat"} 9858
headroom_waste_signal_tokens_total{signal="reread"} 4935
```

`reread` = 4935 tokens, exactly the second serve of the ~4.9k-token tool
result (json_bloat counts both occurrences ≈ 2×). `/stats` shows the
same: `"waste_signals": {"json_bloat": 9858, "reread": 4935, ...}` —
which is what the dashboard panel renders.

Also verified the negative path live: a conversation whose tool results
contain non-compressible plain code text produced no waste-signal
entries (the pipeline only attributes waste when compression actually
engaged, unchanged behavior).

**Not tested:** Gemini `functionResponse` path (parser doesn't produce
`tool_result` blocks for it — pre-existing gap tracked in #819);
dashboard rendering only verified via the `/stats` payload the panel
binds to, not a browser screenshot.

## Out of scope (per #853)

Tool-call argument matching, compression-marker attribution,
tokens-per-task metric, cache hit-rate panel.

---------

Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-11 13:07:04 -05:00
Focused InstabilityandAsh Rhodes 929698af10 fix(parser): detect waste signals in Anthropic tool_result content blocks (#815)
## Description

The dashboard's "What Headroom Removed" panel (waste signals) stays
permanently empty for Anthropic-format traffic.
`parse_message_to_blocks()` only extracted text from content parts with
`type == "text"`, so the `tool_result` blocks that carry the bulk of
agentic conversations (Claude Code, and aider/cursor/copilot in
anthropic mode) were invisible to `detect_waste_signals()`. The pipeline
then reported `waste_signals=None` and `/stats` returned
`"waste_signals": {}` forever.

This PR emits a dedicated `tool_result` Block per Anthropic
`tool_result` content part — handling both string-form content and the
nested text-block-list form — with waste detection and a `tool_call_id`
pairing flag. OpenAI chat-completions behavior is unchanged (parity test
included).

Fixes #813

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/parser.py`: new `_extract_tool_result_text()` helper;
`parse_message_to_blocks()` collects `tool_result` content parts and
emits a `Block(kind="tool_result")` per part with waste signals and
`tool_call_id` flags
- `tests/test_parser.py`: 7 new tests — nested text-list form, string
form, mixed text+tool_result, empty content, non-text inner blocks,
`parse_messages` aggregation, and waste parity with the OpenAI `role:
"tool"` format

## Testing

- [x] Unit tests pass (`pytest tests/test_parser.py` — 60 passed)
- [x] Linting passes (`ruff check`, `ruff format --check`)
- [x] New tests added for new functionality
- [x] Manual testing performed (real pipeline run below)

Also ran `tests/test_pipeline.py`, `tests/test_canonical_pipeline.py`,
`tests/test_proxy_pipeline_lifecycle.py`: 3 failures there are
pre-existing on a clean `upstream/main` checkout (verified via `git
stash`) and unrelated to this change.

## Test Output

```
$ pytest tests/test_parser.py -q
60 passed in 0.14s

$ ruff check headroom/parser.py tests/test_parser.py
All checks passed!
```

## Real behavior proof

Real `TransformPipeline` (CacheAligner + ContentRouter, same
construction as the proxy server) over an Anthropic-format conversation
with four large JSON `tool_result` blocks:

```
# before this fix
before=37265 after=16013 saved=21252
transforms: ['router:protected:user_message', 'router:tool_result:smart_crusher']
waste_signals: None          <- SmartCrusher removed 21k tokens, dashboard shows nothing

# after this fix (identical input)
before=37265 after=16013 saved=21252
transforms: ['router:protected:user_message', 'router:tool_result:smart_crusher']
waste_signals: {'json_bloat': 37140, 'html_noise': 0, 'base64': 0, 'whitespace': 0, 'dynamic_date': 0, 'repetition': 0}
```

Parser-level parity (same JSON payload, both wire formats):

```
anthropic tool_result waste total: 0      -> 1745 after fix
openai role:"tool" waste total:    1745   (unchanged)
```

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] CHANGELOG.md — not edited manually; release-please generates
entries from the conventional `fix:` commit

## Additional Notes

Scoped to the Anthropic `tool_result` parsing bug per issue #813. Two
related-but-separate gaps noted there: `handle_openai_responses` (codex)
never computes waste signals at all, and Gemini `functionResponse` parts
are preserved verbatim — both deserve their own issues/PRs.

---------

Co-authored-by: Ash Rhodes <ashley.rhodes@king.com>
2026-06-10 21:06:28 -05:00
chopratejas 5e2186c42a Add multi-provider memory system with auto-detection
- Add MemoryToolAdapter for unified memory across providers
- Anthropic: Uses native memory tool (memory_20250818) for subscription safety
- OpenAI/Gemini/Others: Uses function calling format
- All providers share the same semantic vector store backend
- Simplify CLI to single --memory flag with auto-detection
- Add proper resource cleanup (close methods) to fix test isolation
- Update README with memory documentation
2026-02-01 14:42:50 -08:00
Prakersh MaheshwariandClaude Opus 4.5 f83f92a51b fix: Handle Anthropic format tool_use/tool_result as atomic units
Root Cause:
The `find_tool_units()` function in `parser.py` only detected OpenAI
format tool calls (assistant.tool_calls + role="tool" messages), not
Anthropic format (assistant.content[type=tool_use] + user.content[type=tool_result]).

This caused RollingWindow and IntelligentContext transforms to treat
Anthropic tool_use and tool_result as separate, independently droppable
messages. When context needed to be trimmed, the assistant message with
tool_use could be dropped while keeping the user message with tool_result,
creating orphaned tool_result blocks.

When sent to the Anthropic API, this produces the error:
"unexpected tool_use_id found in tool_result blocks"

Changes:
1. parser.py: Extended `find_tool_units()` to detect Anthropic format:
   - Scan user messages for content blocks with type="tool_result"
   - Scan assistant messages for content blocks with type="tool_use"
   - Map tool_use_id to corresponding response message indices

2. rolling_window.py: Extended `_get_protected_indices()` to protect
   Anthropic format tool pairs:
   - Detect tool_use blocks in assistant.content
   - Find and protect matching user messages with tool_result blocks

3. tests/test_parser.py: Added 4 new tests for Anthropic format:
   - test_anthropic_format_tool_use_and_result
   - test_anthropic_format_multiple_tool_uses
   - test_anthropic_format_orphaned_tool_result
   - test_mixed_openai_and_anthropic_formats

Test Results: 82 passed (including 4 new Anthropic format tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 00:26:25 +05:30
chopratejas e4a41faa33 Fix all ruff lint and format errors for CI
- Fix E402: Move module-level imports to top of file
- Fix F401: Add noqa for availability check imports
- Fix F402: Rename loop variables shadowing imports
- Fix E722: Replace bare except with except Exception
- Fix B904: Add exception chaining (from e)
- Fix F811: Remove duplicate imports
- Fix B027: Add noqa for empty close() method
- Fix E741: Rename ambiguous variable l -> label
- Fix I001: Import sorting issues
- Apply ruff format to all 106 files

All 902 tests pass.
2026-01-10 15:33:44 -08:00
chopratejas 9c7d4512d6 Initial commit: Headroom SDK - LLM context optimization toolkit
A comprehensive SDK for optimizing LLM context windows, reducing token
usage while preserving critical information for AI agents.

Core Features:
- SmartCrusher: Statistical compression of tool outputs (70-85% reduction)
- CacheAligner: Prefix optimization for prompt cache hits
- RollingWindow: Intelligent context window management
- BM25/Hybrid relevance scoring for smart item selection

Integrations:
- OpenAI and Anthropic provider support
- LangChain integration (ChatModel, Callbacks, Runnable)
- MCP (Model Context Protocol) integration for tool compression

Test Coverage:
- 372 tests passing across all modules
- 35 performance benchmarks
- Real-world agent evaluations with 88% token savings

Key Components:
- headroom/transforms/: Core compression transforms
- headroom/providers/: OpenAI and Anthropic support
- headroom/integrations/: LangChain and MCP integrations
- headroom/relevance/: BM25 and hybrid scoring
- headroom/pricing/: Model pricing registry
- benchmarks/: Performance benchmark suite
- examples/: Usage examples and demos
2026-01-06 23:16:58 -08:00