Commit Graph
9 Commits
Author SHA1 Message Date
703755e873 fix(pricing): add Claude Haiku 5.5 rates, 1M context and its 100K long-prompt card (#4037)
## Current maintainer qualification

Reviewed head 595803d786 after the
current-main sync. The PR-specific pricing implementation is unchanged
from the previous reviewed head.

Independent validation: 202 focused Python pricing/provider/savings
tests passed, four skipped. The Rust proxy binary built from this head.
Through the running Rust HTTP proxy and a separate loopback upstream,
requests carrying 100,000 and 100,001 input tokens produced the expected
native-ledger request costs of $0.0105 and $0.0525005 respectively,
including 1,000 output tokens. The default stable rollout channel was
used, with no unsafe feature override.

The HTTP replay uses controlled upstream usage; it does not claim live
Anthropic inference or independently verified provider billing. The
previously recorded missing native HTTP boundary check is now covered.

Current main is an ancestor. Hosted checks on this head are terminal: 49
successful, 2 skipped, with no failures. The prior shared Rust advisory
hold is cleared. Final merge approval remains human-only; nothing has
been merged by this review.
## Description

Claude Haiku 5.5 (`claude-haiku-5-5`) is released and had no rows in
Headroom's built-in Anthropic tables or the Rust proxy's vendored
LiteLLM snapshot. This PR adds it, including its second rate card for
prompts over 100K tokens, and makes every cost path read a model's
long-context threshold from its own catalog row instead of assuming
200K.

**Why the missing rows mispriced instead of failing:** without LiteLLM
data for the model, `AnthropicProvider` falls through to the `haiku`
pattern default ($0.80 / $4, 200K), which is 8x the real input rate and
a fifth of the real window. The Rust `context_window_for` is
exact-match, so the id fell through to the 128K default.

Haiku 5.5's rate card ([Anthropic
pricing](https://platform.claude.com/docs/en/about-claude/pricing),
matching upstream LiteLLM's `claude-haiku-5-5` rows): $0.10 / $0.50
input / output and $0.01 cache read for prompts up to 100K tokens; every
rate is 5x ($0.50 / $2.50 / $0.05) above 100K. 1M context, 128K max
output.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

All values in $ / MTok (input / output / cache read).

| Model | Before (fallback) | After | Context |
|---|---|---|---|
| `claude-haiku-5-5`, prompt <= 100K | *(no row: `haiku` tier default)*
$0.80 / $4 / $0.08 | **$0.10 / $0.50 / $0.01** | 200K -> **1M** |
| `claude-haiku-5-5`, prompt > 100K | $0.80 / $4 / $0.08 | **$0.50 /
$2.50 / $0.05** | |

- `headroom/providers/anthropic.py`:
- Adds Haiku 5.5 to `ANTHROPIC_PRICING` and `ANTHROPIC_CONTEXT_LIMITS`.
- The long-context premium becomes a small per-model table,
`_LONG_CONTEXT_TIERS` (prefixes, threshold, multipliers), so Haiku 5.5's
100K card sits next to the existing Sonnet 4 / 4.5 200K tier. The Sonnet
tier's threshold and multipliers are unchanged.
  ```diff
   _apply_long_context_premium(model, pricing, input_tokens)
- if input_tokens <= 200K or model not in Sonnet 4 / 4.5: return pricing
  -  return pricing * Sonnet premium
  +  for prefixes, threshold, premium in _LONG_CONTEXT_TIERS:
  +    if model starts with prefixes:
+ return pricing if input_tokens <= threshold else pricing * premium
  +  return pricing
  ```
- `crates/headroom-proxy/data/model_prices_and_context_window.json`:
appends the 18 upstream LiteLLM `claude-haiku-5-5` keys verbatim
(first-party, Bedrock regional/global/Mantle/GovCloud, Vertex, Azure,
OpenRouter, Perplexity). No existing entry changes or is removed;
extended rather than re-pulled for the reasons in #3841.
- **Per-model long-context tiers on the stats paths** (Devin finding 1).
Each catalog row publishes at most one `*_above_<N>k_tokens` tier (100K,
128K, 200K, 256K or 272K); the threshold now comes from the row, not a
shared 200K constant.
- `crates/headroom-proxy/src/observability/pricing.rs`: the price book
stores a `PriceCard` (base card + optional long-context card parsed from
the row's tier fields; a missing tier field falls back to that rate's
base price). `lookup` still returns the base card; `lookup_card` returns
both, and `PriceCard::for_prompt` picks one.
- `crates/headroom-proxy/src/observability/ledger.rs`: `record` picks
the card from the billed prompt (uncached + cache read + cache write)
before pricing spend and savings.
- `headroom/pricing/counterfactual.py`: `long_context_threshold(model)`
/ `long_context_tier(info)`; `resolve_rates` reads the row's own tier,
including its combined 1h + long-tier write rate where published.
`CacheMix.is_long_context` takes the threshold.
- `headroom/proxy/cost.py`, `headroom/proxy/savings_tracker.py`: bucket
and price at the model's own threshold.
- Side effect on other tiered rows, both intended: the Rust stats now
bill Sonnet 4 / 4.5 above 200K at its long card (the Python cost card
already did), and both paths switch GPT-5.x at 272K and the 128K-tier
rows at 128K with their own rates. Before, those rows crossed a 200K
line, found no `*_above_200k_tokens` fields and stayed on the base card.
- **Compression that crosses a tier credits the whole avoided premium**
(Devin re-review). When compression keeps a prompt under the threshold
its uncompressed form would have passed, the saving is the uncompressed
bill minus the actual bill, not the removed tokens at the cheap rate.
Rust: `PriceCard::compression_savings_usd` (input and output); Python:
`long_context_premium_avoided_usd`, used by
`estimate_request_savings_usd` and by `CostTracker` (which also adds the
output premium). The list companion still prices only the removed
tokens, so budget enforcement stays monotonic. The warm-prefix test in
`tests/test_cost_card_attribution.py` sent 200K with 10K saved, which
crosses Sonnet 4.5's tier; its counts are halved (same mix) so it stays
inside one tier.
- **Short ids no longer inherit a newer release** (Devin finding 2).
`get_context_limit` and `_get_pricing` matched substrings in both
directions, so `claude-haiku-5` resolved to Haiku 5.5 and
`claude-sonnet-4` to Sonnet 4.6's 1M window. `_is_release_of` accepts a
longer id containing the known one (dated or provider-wrapped) or an
undated / `-latest` alias of a dated key, and rejects ids one version
segment apart in either direction.
- `crates/headroom-proxy/src/compression/model_limits.rs`: the 1M-window
test now covers `claude-haiku-5-5`, `us.anthropic.claude-haiku-5-5` and
`vertex_ai/claude-haiku-5-5`.
- Rust tests: Haiku 5.5 base and >100K cards, per-model threshold
(Sonnet 4.5 at 200K), flat-rated model, missing tier field; ledger bills
150K in / 5K out at the long card, and cached tokens count toward the
threshold.
- `tests/test_providers/test_anthropic.py`: Haiku 5.5 price, dated-id
and 1M-context cases, cost cases at and above the 100K threshold, the
`claude-haiku-5` regression and an `_is_release_of` table.
- `tests/test_cost_card_attribution.py`,
`tests/test_savings_provider_matrix.py`: Haiku 5.5 cost card and savings
at 90K vs 150K.
- `tests/test_counterfactual_pricing.py`: the long-tier 1h test now
reads the published combined rate, and a synthetic row covers the
derived case.
- `tests/test_provider_model_fallback.py`: the "future haiku" probe
moves from `claude-haiku-5-20260101` to `claude-haiku-6-20270101`,
mirroring the opus probe change in #3841.
- `tests/test_thinking_compactor_version_gate.py`: `claude-haiku-5-5` is
treated as a Claude 5 model.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`) (ruff 0.16.8, on the changed
files)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ HOME=$(mktemp -d) pytest -q <119 pricing / cost / savings / stats / provider / learn modules>
2097 passed, 18 skipped, 5 warnings

$ cargo test -p headroom-proxy
577 passed, 0 failed (41 suites)
$ cargo clippy -p headroom-proxy --all-targets -- -D warnings
(clean)
$ cargo fmt -p headroom-proxy --check
(clean)

$ ruff check <changed .py files>          # ruff 0.16.8
All checks passed!
$ ruff format --check <changed .py files>
(clean)
$ mypy headroom --ignore-missing-imports  # mypy 1.20.2
Success: no issues found in 566 source files
```

`HOME` is isolated because a developer `~/.headroom/models.json` leaks
into `TestAnthropicConfigLoading`.

## Real Behavior Proof

- Environment: macOS, the repo `.venv`. `origin/main` and this PR head
were each checked out as a git worktree and run with the same
interpreter and an isolated `HOME`.
- Exact command / steps: a script built `AnthropicProvider()`, stubbed
`estimate_cost_from_tokens` and `_get_litellm_clients` to force the
built-in fallback (the LiteLLM-unavailable or offline path), then priced
two requests and read the context limit. For the Rust side it applied
`context_window_for` semantics (exact key, `mode == "chat"`,
`max_input_tokens`, 128K default) to each worktree's vendored JSON. For
the review fixes, a second script ran against the first PR commit
(`b0503796`) and this head: the fallback lookups for short ids, a
`CostTracker` 150K-in / 5K-out Haiku 5.5 turn with 10K saved, and
`resolve_rates(..., long_context=True)`; the Rust ledger case is the
`long_prompt_bills_at_the_models_long_context_card` test.
- Observed result: Haiku 5.5 now prices and sizes correctly on every
path, and short ids keep their tier default (tables below).
  ```text
origin/main this PR
context limit (Python fallback) 200,000 (inferred 'haiku' tier)
1,000,000
50K in / 5K out $0.060000 $0.007500
150K in (120K cached) / 5K out $0.053600 $0.033500 (100K+ card)
Rust context_window_for 128,000 1,000,000
  ```
Hand check of the last row: 30K × $0.50 + 120K × $0.05 + 5K × $2.50 per
MTok = $0.0335.

  Review fixes, first PR commit vs this head:
  ```text
b0503796 this head
get_context_limit("claude-haiku-5") 1,000,000 200,000 ('haiku' tier)
_get_pricing("claude-haiku-5") $0.10 / $0.50 / $0.01 $0.80 / $4 / $0.08
get_context_limit("claude-sonnet-4") 1,000,000 (Sonnet 4.6) 200,000
(Sonnet 4)
CostTracker 150K in / 5K out: output cost $0.002500 $0.012500
CostTracker 10K saved: cache-aware savings $0.001000 $0.005000
resolve_rates(haiku-5-5, long).uncached $0.10 / MTok $0.50 / MTok
Rust ledger 150K in / 5K out $0.0175 (base card, computed) $0.0875
(ledger unit test)
  ```

Compression crossing the tier, Haiku 5.5, 110K uncached compressed to
90K (unit tests on this head; the "before" column is the old rule's
arithmetic):
  ```text
removed tokens at base this head
Rust ledger compression savings $0.002 $0.046
estimate_request_savings_usd["compression"] $0.002 $0.046
CostTracker cache-aware savings (1K output) $0.002 $0.048 (incl. $0.002
output premium)
  ```
- Not tested: live provider API billing. The controlled native HTTP
replay now covers the 100K boundary, as documented above.
- The Rust proxy end to end with a live >100K Haiku 5.5 request; the
ledger is covered by unit tests only.
- Bedrock-style ids (`us.anthropic.claude-haiku-5-5`) get the base card
without the 100K premium on the Python fallback path, the same
limitation the Sonnet 4 / 4.5 tier already has.
  - No live API call was made.

## Maintainer follow-up qualification

Latest qualification head: `f885cc4c8afa574d867820583886687b0bae12a9`.
The pricing code and the 413-Python/73-Rust/TCP-replay validation below
are unchanged from `a45ae8c3e`. The first hosted shard-3 run exposed
five existing main integration failures:
`test_proxy_dependency_gate_matches_onnx_platform_markers` still patched
a removed eager-import hook. The follow-up aligns those existing tests
with lazy `find_spec` discovery and its missing-spec return value. The
five failures reproduce locally before; all eight dependency-contract
cases and 91 CLI/stateless/logging tests pass after, with
Ruff/format/whitespace passing. The current-head CI must finish before
approval.

Published maintainer follow-up at `a45ae8c3e` (normal push from the
unchanged `c326e2046` head): forward generated `output_tokens` into the
shared compression estimator through Prometheus and the durable lifetime
recorder, and add the long-card minus base-card output premium only when
compression crosses the model's threshold. `output_tokens_saved` remains
a separate output-shaping layer. The documented unpriced direct tracker
fallback is preserved.

Integrated current main `463c1a43e30fc33cb256a3f455daf28043e53b05` to
resolve the pricing conflict. The resolution retains canonical
model-specific cache billing, independent cache-rate provenance and
explicit-zero rates, while probing missing billed cache rates at the
model's own long-context threshold. New regressions cover both contracts
together.

Verification after the conflict resolution:

- 413 Python tests passed; one known Windows directory-fsync test
deselected after it failed identically on the unchanged PR head; three
existing warnings. This includes 20 new
output-premium/boundary/metadata/merged-cache-billing cases, pricing,
provider fallback, persistent metrics, cache TTL, tool-schema and output
savings suites.
- 73 Rust observability tests passed: `cargo test -p headroom-proxy
--lib observability --locked`.
- Ruff lint/format and whitespace checks pass. Focused Linux-platform
mypy passes for the four pricing/metrics modules. Full local mypy in
this extras-rich environment reports 19 errors in
`relevance/embedding.py`, `memory/backends/direct_mem0.py`,
`integrations/langchain/chat_model.py` and
`integrations/crewai/agents.py`; it is not claimed green.
- Before the fix, both public priced recorders report $0.046 for
110K→90K input with 1K generated output; after, both report $0.048.
Noncrossing and no-output controls pass, and separately saved output is
not conflated with the avoided premium.

Actual Windows/Python 3.12 TCP replay: real uvicorn proxy, separate
loopback HTTP upstream, and the real content-router/SmartCrusher
pipeline. A 304,903-character JSON tool result becomes 139,003
characters, saving 55,974 locally counted tokens. The controlled
upstream reports 90,000 input and 1,000 output tokens. On the unchanged
PR head the lifetime estimator reports $0.063987 and fails; with the fix
and after the main integration it reports the expected $0.065987. The
persistent Prometheus compression metric matches, and output-shaping
savings remain zero. This exercises real compression and handlers with
controlled usage/rates; no live provider billing or Rust HTTP runtime is
claimed.

Current-head GitHub CI and edited review bodies still require
qualification before approval.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: cost estimates and the context window
reported for `claude-haiku-5-5` and the 18 added LiteLLM keys;
long-prompt spend and savings on tiered models now use each model's own
threshold; short ids such as `claude-haiku-5` fall back to their tier
default instead of a newer release
- Kill switch / disable path: n/a (data tables)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (n/a: no
user-facing docs list per-model prices)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

n/a

## Additional Notes

The 100K card applies on every path: the Python fallback
(`_LONG_CONTEXT_TIERS`), LiteLLM's own estimate, the cost card and
savings counterfactual, and the Rust stats ledger.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4037"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <tejas@headroomlabs.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 17:54:57 -05:00
Kenneth WongandJerrettDavis 0d99c56dc5 fix(pricing): add Claude Sonnet 5.5 / Opus 5.5 / Fable 5.1, correct Sonnet 5 rates (#3841)
## Description

Claude Sonnet 5.5 and Opus 5.5 are released, and Anthropic made Claude
Sonnet 5's launch price its standard rate. This PR updates Headroom's
built-in Anthropic fallback price and context tables and the Rust
proxy's vendored LiteLLM snapshot to match current [Anthropic
pricing](https://platform.claude.com/docs/en/about-claude/pricing)
(verified 2026-09-29).

From the pricing page, footnote 3:

> The $2/$10 per million input/output token pricing for Claude Sonnet 5,
announced at launch as introductory pricing through August 31, 2026, is
now the standard price. The previously scheduled increase to $3/$15 per
million input/output tokens on September 1, 2026 will not occur.

This reverses the deliberate $3/$15 pin from #1767, which assumed that
increase would happen.

**Why the missing rows mispriced instead of failing:**
`AnthropicProvider._get_pricing` falls back to substring matching in
dict insertion order, so `claude-sonnet-5-5` silently inherited the
`claude-sonnet-5` row and `claude-fable-5-1` inherited Fable 5's $1
cache read. The Rust `context_window_for` is exact-match, so every
Claude 5-generation id fell through to the 128K default.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

All values in $ / MTok (input / output / cache read).

| Model | Before (fallback) | After | Context |
|---|---|---|---|
| `claude-sonnet-5-5` | *(no row: matched `claude-sonnet-5`)* $3 / $15 /
$0.30 | **$2 / $10 / $0.20** | 1M |
| `claude-sonnet-5` | $3 / $15 / $0.30 | **$2 / $10 / $0.20** | 1M |
| `claude-opus-5-5` | *(no row: opus tier default)* $5 / $25 / $0.50 |
**$4 / $20 / $0.20** | 1M |
| `claude-opus-5` | *(no row: opus tier default)* | $5 / $25 / $0.50 |
1M |
| `claude-fable-5-1` | *(no row: matched `claude-fable-5`)* $10 / $50 /
$1.00 | **$10 / $50 / $0.25** | 1M |

- `headroom/providers/anthropic.py`:
- Adds the four new models to `ANTHROPIC_PRICING` and
`ANTHROPIC_CONTEXT_LIMITS` and corrects `claude-sonnet-5`.
- Newer ids are listed before their prefixes so dated or suffixed ids
(`…-5-5-20261001`, `…-5-5[1m]`) resolve to their own row.
- The `_PATTERN_DEFAULTS` tier rates are unchanged: they also cover
older unlisted models such as Sonnet 3.7.
- `crates/headroom-proxy/data/model_prices_and_context_window.json`:
appends the 113 missing upstream Claude 5-generation keys (first-party,
Bedrock regional/global/Mantle, Vertex, Azure, Databricks, OpenRouter
and similar). No existing entry changes or is removed. The snapshot is
extended rather than re-pulled with `scripts/refresh_model_limits.sh`:
- Upstream has since pruned 258 retired ids the proxy still serves, e.g.
`claude-3-7-sonnet-20250219` and `claude-opus-4-1`. Those would regress
from 200K to the 128K default.
- A full re-pull overwrites the hand-edited DeepSeek rate card from
#3575.
- `crates/headroom-proxy/src/compression/model_limits.rs`: new
`claude_5_generation_models_present` test.
- `tests/test_providers/test_anthropic.py`:
  - The Sonnet 5 expectation moves to $2/$10/$0.20.
- New parametrized pricing and 1M-context tests for the new models,
including the prefix-collision cases.
- `tests/test_provider_model_fallback.py`: the "future opus" probe moves
from `claude-opus-5-20260101` to `claude-opus-6-20270101`. Opus 5 is now
a real 1M entry, so the old id no longer exercised the pattern fallback.
This mirrors the existing `claude-sonnet-6` probe.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`) (run on the changed module,
`headroom/providers/anthropic.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ cargo test -p headroom-proxy --lib compression::model_limits
test compression::model_limits::tests::claude_5_generation_models_present ... ok
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 271 filtered out

$ HOME=$(mktemp -d) pytest -q tests/test_providers/test_anthropic.py tests/test_provider_model_fallback.py \
    tests/test_cost_tracker_totals.py tests/test_pricing_from_litellm.py tests/test_pricing.py \
    tests/test_openai_pricing_resolution.py
138 passed, 2 warnings

$ ruff check headroom/providers/anthropic.py tests/test_providers/test_anthropic.py tests/test_provider_model_fallback.py
All checks passed!
$ ruff format --check (same files)
3 files already formatted
$ cargo fmt -p headroom-proxy -- --check
(clean)
$ mypy headroom/providers/anthropic.py
Success: no issues found in 1 source file
```

On the broader pricing, cost and savings subset, 773 pass. The same 12
failures and 3 collection errors occur on unmodified `main` in my
environment, all `ModuleNotFoundError` from optional dev deps (`respx`
and others) that aren't installed locally. `HOME` is isolated because a
developer `~/.headroom/models.json` leaks into
`TestAnthropicConfigLoading`.

## Real Behavior Proof

- Environment: macOS, the repo `.venv` (Python 3.10). `origin/main` and
this PR head were each checked out as a git worktree and run with the
same interpreter and an isolated `HOME`.
- Exact command / steps: priced the same request through the built-in
fallback on `origin/main` and on this PR head (steps below).
- A script built `AnthropicProvider()`, stubbed
`estimate_cost_from_tokens` to return `None` to force the built-in
fallback (the LiteLLM-unavailable or offline path), and priced 1M input
tokens (800K cached) plus 100K output tokens for each model.
- For the Rust side, `context_window_for` semantics (exact key, `mode ==
"chat"`, `max_input_tokens`, 128K default) were applied to each
worktree's vendored JSON.
- Observed result: `origin/main` bills Sonnet 5.5 at $3/$15 and gives
every Claude 5-generation id a 128K Rust window; this PR bills $2/$10
and gives them 1M.
  ```text
  origin/main
claude-sonnet-5-5 rates={'input': 3.0, 'output': 15.0, 'cached_input':
0.3} cost=$2.3400
claude-sonnet-5 rates={'input': 3.0, 'output': 15.0, 'cached_input':
0.3} cost=$2.3400
claude-opus-5-5 rates={'input': 5.0, 'output': 25.0, 'cached_input':
0.5} cost=$3.9000
claude-fable-5-1 rates={'input': 10.0, 'output': 50.0, 'cached_input':
1.0} cost=$7.8000
rust context_window_for: sonnet-5-5 / global.anthropic.claude-sonnet-5-5
/ opus-5-5 / fable-5-1 = 128000

  this PR
claude-sonnet-5-5 rates={'input': 2.0, 'output': 10.0, 'cached_input':
0.2} cost=$1.5600
claude-sonnet-5 rates={'input': 2.0, 'output': 10.0, 'cached_input':
0.2} cost=$1.5600
claude-opus-5-5 rates={'input': 4.0, 'output': 20.0, 'cached_input':
0.2} cost=$2.9600
claude-fable-5-1 rates={'input': 10.0, 'output': 50.0, 'cached_input':
0.25} cost=$7.2000
    rust context_window_for: all four = 1000000
  ```
The live LiteLLM `model_cost` already reports the "after" rates for all
of these, so the primary pricing path is unchanged. This fixes the
fallback path and the Rust proxy.
- Not tested: no live upstream Sonnet 5.5 request, no dashboard browser
check (details below).
  - No live upstream requests were sent against Claude Sonnet 5.5.
- The dashboard savings tile was not exercised in a browser. It consumes
the same `estimate_cost` path shown above.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This is static pricing and context
metadata.
- Minimum rollout channel: n/a
- Stable/default behavior changed: yes, fallback cost estimates and the
Rust compression budget for Claude 5-generation ids only (details
below).
- Fallback cost estimates for the listed models drop to Anthropic's
published rates.
- The Rust proxy's compression budget for Claude 5-generation ids rises
from 128K to their real 1M window. The proxy now compresses those
requests later instead of over-trimming them.
- Kill switch / disable path: `HEADROOM_MODEL_LIMITS` or
`~/.headroom/models.json` can override any model's context limit or
pricing.
- Unsafe override required: no
- Qualification impact: none expected. Existing models' rows and
vendored entries are byte-identical.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (n/a: no
user-facing docs list model prices)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this) (my original commit did; @JerrettDavis removed it in 2cfd4f64,
thanks)

## Screenshots (if applicable)

n/a

## Additional Notes

Not in scope: the static `_build_bedrock_fallback_map` and
`_VERTEX_MODEL_MAP` in `headroom/backends/litellm.py` stop at Claude
4.6. They are routing maps, not pricing, and are only used when
inference-profile discovery fails. They need verified per-model Bedrock
and Vertex ids, so they should be a separate PR.

https://claude.ai/code/session_01E5Hryq8EfuB4LAnu81TtJz

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-29 11:15:38 -05:00
Morteza RastgooandClaude Fable 5.1 73a6edbe83 fix: never grep-fold timestamped logs, size-weight savings, warn on no-op model limits (#3419)
Three independent fixes from evaluating Headroom in front of a self-hosted vLLM gateway, plus review follow-ups.

- compaction: `_GREP_ROW_RE` matched timestamped log lines (`2026-09-02 14:30:00 [FATAL] ...`, syslog `Aug 16 11:03:22 ...`) as `path:line:content` rows, so search_heading hoisted the date+hour into a heading and the model saw `30:00 [FATAL] ...`. Byte-reversible, so the inverse check could not catch it; guard at the row matcher. Zero false positives on 5,921 real grep rows. Adds a `HEADROOM_LOSSLESS_COMPACTION=0` kill-switch, read per call so the proxy's runtime-env hot-sync applies.
- proxy/cost: `avg_compression_pct` is now weighted by original tokens instead of a mean of per-request ratios, so one tiny highly-compressible request no longer dominates the headline.
- providers/anthropic: warn when `HEADROOM_MODEL_LIMITS` parses but carries neither `context_limits` nor `pricing`, naming the expected shape. Stays quiet when another provider's namespaced section (e.g. `{"openai": {...}}`) carries the keys.
- docs: document `HEADROOM_LOSSLESS_COMPACTION` in the env table.

Co-authored-by: Morteza Rastgoo <5219339+Morteza-Rastgoo@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbB9CAngCNrB3uXNqgHGZe
2026-09-04 10:30:12 +05:30
Abhay Singh 3ed8f76019 fix(providers): don't crash on a non-object HEADROOM_MODEL_LIMITS / models.json (#3089)
## Description

`_load_custom_model_config` in both `headroom/providers/anthropic.py`
and `headroom/providers/openai.py` loads the operator's custom model
configuration from `HEADROOM_MODEL_LIMITS` (a JSON string or a file
path) and `~/.headroom/models.json`, then reads it with
`loaded.get(...)`:

```python
loaded = json.loads(env_config)          # or json.load(f)
anthropic_config = loaded.get("anthropic", loaded)
```

The `try` guards only `except (json.JSONDecodeError, OSError)`. When the
value is **valid JSON but not an object** (a JSON array, number, string,
bool, or `null`), `json.loads` succeeds and returns a non-dict, so
`loaded.get(...)` raises `AttributeError` — which is *not* one of the
caught types. Instead of the intended warn-and-fall-back-to-defaults, a
misconfigured `HEADROOM_MODEL_LIMITS` (e.g.
`HEADROOM_MODEL_LIMITS='[1,2,3]'` or `'"gpt-4"'`) crashes provider
initialization. The same gap exists in the `models.json` branch of both
providers.

## Fix

After each load, validate `isinstance(loaded, dict)` and raise
`ValueError` with a clear message, and broaden the handler from `except
(json.JSONDecodeError, OSError)` to `except (ValueError, OSError)`.
`json.JSONDecodeError` is a subclass of `ValueError`, so this strictly
supersets the previous handling: every previously-caught malformed value
still warns and falls back, and a valid-JSON-but-non-object value now
does too, instead of crashing.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/providers/anthropic.py` and `headroom/providers/openai.py`
(`_load_custom_model_config`): add an `isinstance(loaded, dict)` guard
(raising `ValueError`) after the env-var load and after the
`models.json` load, and change both `except` clauses to `(ValueError,
OSError)`.
- `tests/test_provider_model_fallback.py`: added parametrized
`test_non_object_env_var_falls_back_to_defaults` (array / string /
number / bool / null) for both providers, and
`test_non_object_config_file_falls_back_to_defaults` for a non-object
`models.json`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added

### Test Output

```text
tests/test_provider_model_fallback.py  44 passed
# uvx ruff@0.15.22 check  -> All checks passed!
# uvx mypy@1.20.2 headroom/providers/anthropic.py headroom/providers/openai.py -> Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.22 and mypy 1.20.2 via uvx.
- Exact command / steps: reverted both providers and ran the new
regressions to capture the bug (`python -m pytest
tests/test_provider_model_fallback.py::TestAnthropicConfigLoading::test_non_object_env_var_falls_back_to_defaults
tests/test_provider_model_fallback.py::TestOpenAIConfigLoading::test_non_object_env_var_falls_back_to_defaults
tests/test_provider_model_fallback.py::TestAnthropicConfigLoading::test_non_object_config_file_falls_back_to_defaults`
-> 11 failed with `AttributeError` on `loaded.get` across the
array/string/number/bool/null shapes); restored the fix; re-ran the full
file (`python -m pytest tests/test_provider_model_fallback.py` -> 44
passed); then `uvx ruff@0.15.22 format`, `uvx ruff@0.15.22 check`, and
`uvx mypy@1.20.2` on both providers.
- Observed result: before the fix, `HEADROOM_MODEL_LIMITS='[1,2,3]'` (or
`'"gpt-4"'`, `'42'`, `'true'`, `'null'`) raised `AttributeError` out of
`_load_custom_model_config`; after the fix the same values log a warning
and the loader returns the default `{"context_limits": {}, "pricing":
{}[, "encodings": {}]}`, and a well-formed object config is unchanged.
- Not tested: a live proxy boot with a corrupt `HEADROOM_MODEL_LIMITS`
(the loader is exercised directly, which is the exact function provider
init calls).

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This is defensive parsing in the
provider model-config loader, not a rollout-channel-gated runtime
feature.
- Minimum rollout channel: N/A (no rollout-managed behavior).
- Stable/default behavior changed: only for a previously-crashing input.
A non-object `HEADROOM_MODEL_LIMITS` / `models.json` now warns and uses
built-in defaults instead of raising. Well-formed object configs are
parsed exactly as before.
- Kill switch / disable path: N/A — remove or correct the malformed
config value to load custom limits.
- Unsafe override required: no.
- Qualification impact: a corrupt or mistyped model-limits value
degrades to built-in defaults with a warning rather than failing
provider init.
- Rollback path: revert this PR; the loader returns to catching only
`json.JSONDecodeError`/`OSError`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title

## Additional Notes

Both providers carry the same loader shape, so the guard and the widened
`except` are applied identically to keep them in sync. The message names
the offending source (`HEADROOM_MODEL_LIMITS` vs the resolved
config-file path) so the warning is actionable.
2026-08-17 20:18:58 -07:00
Tejas Chopra 0cb72f45b2 fix(providers): stop a shorter model family shadowing a longer one (#2762)
## Description

> **Stacked on #2761** — that PR splits `_lookup_encoding_name` out of
`_get_encoding_name_for_model`, which this one builds on. Please merge
#2761 first; the diff here will shrink to just this commit afterwards.

`_MODEL_ENCODINGS` and `_CONTEXT_LIMITS` are matched by prefix,
iterating in **plain dict order** — so the first *inserted* prefix wins
rather than the most specific one. `gpt-4.1` matched the `gpt-4` entry:

| model | resolved | actual | |
|---|---|---|---|
| `gpt-4.1` | 8192 | 1,047,576 | **128× under** |
| `gpt-4.1-mini` | 8192 | 1,047,576 | **128× under** |
| `gpt-4.1-nano` | 8192 | 1,047,576 | **128× under** |
| `gpt-4-32k-0613` | 8192 | 32,768 | 4× under |
| `gpt-5` / `-mini` / `-nano` | 128,000 | 400,000 | fell to
unknown-model default |
| `o4-mini` | 128,000 | 200,000 | fell to unknown-model default |

A 128× under-estimate matters because the context limit is what tells
the proxy how much headroom is left: it treats a 1M-context model as
nearly full and compresses accordingly.

The same shadowing picked the **wrong encoding** — `gpt-4.1` got
`cl100k_base` instead of `o200k_base`. Measured cost of that:

```text
                cl100k    o200k    error
python code        420      420    +0.0%
json blob          555      555    +0.0%
logs               580      580    +0.0%
english            201      201    +0.0%
CJK                600      450   +33.3%
```

So the encoding half is narrow but real — it only bites CJK content,
which the repo already treats as a case worth testing
(`tests/test_evals_cjk_tokenization.py`).

**Scope honestly:** `get_context_limit` consults LiteLLM *before* this
table, so the limit half only surfaces where LiteLLM is absent or does
not know the model. That is not hypothetical — the `litellm` dependency
carries a `python_version < '3.14'` marker (`pyproject.toml:56`), so
**any install on Python 3.14+ has no LiteLLM** and this table is
load-bearing. The encoding half never had a LiteLLM fallback and was
always wrong.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Both prefix loops now iterate `sorted(..., key=len, reverse=True)` —
longest prefix wins. This is the root-cause fix: it also protects the
*next* model added to these tables.
- Added the missing families: `gpt-4.1` (+`-mini`/`-nano`), `gpt-5`
(+`-mini`/`-nano`), `o4-mini` to both tables.
- Left `supports_model`'s prefix loop alone — it only returns a bool, so
order cannot change its answer.

Not touched: `_PRICING`. `gpt-4.1`/`gpt-5` also fall through to the
GPT-4o pricing tier, which skews cost reporting, but that is a separate
concern with its own verification burden (published rates, staleness
window) and does not belong in a tokenizer-correctness fix.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` + `ruff format`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality

### Test Output

12 of 24 new cases fail without the fix; the 12 that pass are the "must
not regress" rows (`gpt-4`, `gpt-4-turbo`, `gpt-4o`, `o3`,
`gpt-3.5-turbo`) — included precisely so the longest-prefix change can't
quietly move them:

```text
$ git stash push headroom/providers/openai.py && pytest tests/test_openai_model_table_resolution.py -q
FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4.1-mini-1047576]
FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4.1-nano-1047576]
FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4.1-2025-04-14-1047576]
FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4-32k-0613-32768]
FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-5-400000]
FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-5-mini-400000]
FAILED ...::test_context_limit_prefers_the_most_specific_prefix[o4-mini-200000]
FAILED ...::test_encoding_prefers_the_most_specific_prefix[gpt-4.1-o200k_base]
FAILED ...::test_encoding_prefers_the_most_specific_prefix[gpt-4.1-mini-o200k_base]
FAILED ...::test_encoding_prefers_the_most_specific_prefix[gpt-4.1-2025-04-14-o200k_base]
FAILED ...::test_cjk_is_not_over_counted_for_gpt_41
12 failed, 12 passed in 0.70s

$ git stash pop && pytest tests/test_openai_model_table_resolution.py -q
24 passed in 0.43s
```

Regression check — 119 suites touching openai / cost / savings / token /
compress / outcome / budget, this branch vs clean `main` in the same
environment, comparing failure *sets*:

```text
branch : 5 failed, 1486 passed, 86 skipped in 111.01s
main   : 5 failed, 1453 passed, 86 skipped in 132.07s

NEW failures introduced: (none)

pre-existing on both:
  test_bundled_tools_savings.py::test_compressed_payload_preserves_answer_anthropic
  test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4]   (needs `transformers`)
  test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4]   (needs `transformers`)
  test_image_compressor_singleton_reuse.py::test_onnx_router_is_built_once_and_cached
  test_openai_streaming_backend.py::...test_litellm_vertex_streaming_preserves_max_tokens_and_vendor_fields
```

```text
$ ruff check headroom/providers/openai.py tests/test_openai_model_table_resolution.py
All checks passed!
$ mypy headroom/providers/openai.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- **Environment:** macOS, Python 3.13.7, isolated worktree at
`upstream/main` (`ad56dd38`), no `litellm` installed (matching a Python
3.14+ install, where the dep marker excludes it).
- **Exact command / steps:** resolve context limit + encoding for each
model against published OpenAI values, before and after.
- **Observed result:**

```text
before                              after
model               limit  enc      model               limit      enc
gpt-4.1              8192  cl100k   gpt-4.1           1047576  o200k_base
gpt-4.1-mini         8192  cl100k   gpt-4.1-mini      1047576  o200k_base
gpt-4.1-nano         8192  cl100k   gpt-4.1-nano      1047576  o200k_base
gpt-4.1-2025-04-14   8192  cl100k   gpt-4.1-2025-04-14 1047576 o200k_base
gpt-4-32k-0613       8192  cl100k   gpt-4-32k-0613      32768  cl100k_base
gpt-5              128000  o200k    gpt-5              400000  o200k_base
o4-mini            128000  o200k    o4-mini            200000  o200k_base

unchanged: gpt-4=8192/cl100k, gpt-4-turbo=128000/cl100k,
           gpt-4o=128000/o200k, o3=200000, gpt-3.5-turbo=16385/cl100k
```
2026-08-04 00:34:38 -07:00
Kenneth Wong e84ca980cf feat(anthropic): add Claude 5 family pricing & align current rates (#1767)
## Summary

Adds Claude 5 generation metadata and aligns the Anthropic fallback
pricing / context-limit tables in `headroom/providers/anthropic.py` with
current [Anthropic
pricing](https://platform.claude.com/docs/en/about-claude/pricing)
(verified 2026-07-04). Several Claude 4.x entries carried stale rates,
the new Claude 5 models (Fable 5, Opus 4.8, Sonnet 5) had no metadata,
and the generic fallback tests disagreed with the provider tests.

Supersedes #1485 (rebased onto latest `main`, squashed to one commit,
extended with Sonnet 5 / Fable 5 and the requested fallback-test fixes).

## Changes

All values `$ / MTok`; `cached_input` = prompt-cache read = 0.1× input.

| Tier | Model | Before | After | Context |
|---|---|---|---|---|
| Fable | `claude-fable-5` | — *(new)* | $10 / $50 / $1.00 | **1M** |
| Opus | `claude-opus-4-8` | — *(new)* | $5 / $25 / $0.50 | **1M** |
| Opus | `claude-opus-4-7` | $15 / $75 / $1.50 | $5 / $25 / $0.50 | 1M |
| Opus | `claude-opus-4-6` | $15 / $75 / $1.50 | $5 / $25 / $0.50 | 1M |
| Opus | `claude-opus-4-5-20251101` | $15 / $75 / $1.50 | $5 / $25 /
$0.50 | 200K |
| Sonnet | `claude-sonnet-5` | — *(new)* | $3 / $15 / $0.30 | **1M** |
| Sonnet | `claude-sonnet-4-6` | — *(new)* | $3 / $15 / $0.30 | **1M** |
| Sonnet | `claude-sonnet-4-5` | — *(new)* | $3 / $15 / $0.30 | 200K |
| Haiku | `claude-haiku-4-5-20251001` | $0.80 / $4 / $0.08 *(3.5 rates)*
| $1 / $5 / $0.10 | 200K |

`claude-sonnet-4-20250514` and all Claude 3.x / 3.5.x entries were
already correct — left unchanged.

The **1M-context** entries (Fable 5, Opus 4.8, Sonnet 5, Sonnet 4.6) are
functional, not cosmetic: they ship in the long-context tier, and
without explicit entries the `sonnet` / `opus` pattern defaults would
report 200K.

Sonnet 5 is pinned to the **standard** Sonnet tier ($3 / $15 / $0.30);
Anthropic's introductory rate ($2 / $10 through Aug 31 2026) is
intentionally not encoded to avoid a time-dependent fixture.

## Fallback-model tests (addresses review on #1485)

`_PATTERN_DEFAULTS["opus"]` is aligned to the current Opus tier ($5 /
$25 / $0.50) so the generic fallback suite and the provider-specific
suite agree:

- `test_pricing_for_known_models` — Opus 4.5 pins $5 / $25 / $0.50
- `test_pattern_based_inference_opus` — unknown-opus fallback now $5 /
$25
- `test_cost_estimation_for_new_models` — fixture estimate corrected
$22.5 → $7.5
- `test_pattern_based_inference_sonnet` — retargeted to
`claude-sonnet-6-*` (the old `claude-sonnet-5-*` probe now
prefix-matches the real `claude-sonnet-5` key)

New provider coverage:

- `test_get_context_limit_claude_5_family` — Fable 5 / Opus 4.8 / Sonnet
5 all 1M
- `test_pricing_claude_5_family` — exact rate table for the 3 new models

Full suite: **48 passed**.

## Source

https://platform.claude.com/docs/en/about-claude/pricing
2026-07-03 12:09:35 -07:00
julienguarinoandJulien Guarino 17ecad9d89 fix(gemini): resolve Google model capabilities through ModelRegistry (#1276)
## Description

Google model capability lookup was still tied to static provider tables
for support checks and context limits. That made plausible future Gemini
model ids fail token counting or context lookup even when they clearly
belonged to the Google provider family.

This change adds a tolerant `ModelRegistry.resolve()` runtime lookup
path and routes the Google provider through it. Exact built-in registry
matches still win first, LiteLLM pricing metadata can supply live limits
when available, and provider-scoped family fallbacks cover future Gemini
ids without letting Google claim unrelated models.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `ModelRegistry.resolve()` as a tolerant runtime capability
resolver.
- Added provider-scoped Google/Gemini family fallbacks for plausible
future model ids.
- Added support for LiteLLM-style `gemini/gemini-...` model ids in
provider inference and family fallback matching.
- Updated `GoogleProvider.supports_model()` and
`GoogleProvider.get_context_limit()` to use the shared model registry
path.
- Added regression tests for future Gemini ids, legacy Gemini context
limits, and unrelated model rejection.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run --no-project --with pytest --with opentelemetry-api --with pydantic --with tiktoken --with litellm --with click --with rich python -B -m pytest tests/test_provider_model_fallback.py tests/test_models.py

65 passed

uv run --no-project --with ruff ruff check headroom/models/registry.py headroom/providers/google.py tests/test_provider_model_fallback.py tests/test_models.py

All checks passed!

uv run --no-project --with ruff ruff format --check headroom/models/registry.py headroom/providers/google.py tests/test_provider_model_fallback.py tests/test_models.py

4 files already formatted
```

## Real Behavior Proof

- Environment: macOS arm64 local checkout, Python 3.13 virtualenv for
editable install; deployed smoke test in a Cloud Run staging service
using an earlier commit from this fork branch before the review
follow-up.
- Exact command / steps: installed `headroom-ai[langchain]` from the
fork branch in the staging service, triggered long-context requests that
activate Headroom's LangChain compression path, then checked Cloud Run
logs after 2026-06-22 12:20 Europe/Paris.
- Observed result: Headroom initialized successfully, compressed
conversation memory (`23255 -> 5618 chars`), and no logs matched the
previous model-resolution failure signatures (`not recognized as a
Google model`, `Unknown context limit`).
- Not tested: staging was not rerun after the `gemini/gemini-...` review
follow-up; that prefix path is covered by local regression tests. Full
repository `uv run pytest` on local macOS is currently blocked by a
native `maturin`/`esaxx-rs` compile failure (`fatal error: 'cstdint'
file not found`). Type checking was not run.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- Documentation changes are not included because this is a runtime
compatibility fix with no public API or user-facing configuration
change.
- Full local test execution should be retried in CI or a Linux
environment where the native Rust extension build is healthy.

Co-authored-by: Julien Guarino <julien.guarino@fashiondata.io>
2026-06-26 23:31:56 -05:00
JerrettDavis d00c6739e1 Fix CI regressions for cache benchmark work 2026-04-04 22:33:44 -05:00
chopratejas 905c229251 Add AST-based code compression and custom model configuration
CodeAwareCompressor:
- Tree-sitter based AST parsing for Python, JS, TS, Go, Rust, Java, C, C++
- Preserves imports, signatures, type annotations, error handlers
- Guarantees syntactically valid output
- Uses tree-sitter-language-pack for broad language support

ContentRouter:
- Intelligent compression orchestrator
- Auto-routes content to optimal compressor based on type detection
- Source hint support for high-confidence routing

Custom Model Configuration:
- HEADROOM_MODEL_LIMITS env var and ~/.headroom/models.json support
- Pattern-based inference for unknown models (opus/sonnet/haiku tiers)
- Support for Claude 4.5, Claude 4, o3, o3-mini
- Graceful fallback - never crashes on unknown models
2026-01-14 13:46:55 -08:00