fix(providers): isolate resolved Anthropic context limits (#4593)

## Description

Addresses the order-dependent Anthropic context-limit resolution
reported in #4390. The proxy reuses its provider instance: resolving
`gpt-4` first currently makes a later `gpt-4-turbo` lookup inherit 8,192
tokens through alias matching, although LiteLLM reports 128,000.

Keep dynamically resolved limits in an exact-model cache so each model
reaches its own metadata lookup. The broader client-supplied
context-limit/header proposal in #4390 remains a separate feature.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Separate resolved limits from the built-in/configured entries used for
release and alias matching.
- Preserve exact-result caching, configured aliases, `[1m]` handling,
and recognition of previously resolved models.
- Cover both lookup orders, both LiteLLM limit fields, unknown-model and
Claude fallbacks, date aliases, repeated lookups, and explicit
overrides.

## Testing

- [x] Relevant unit tests pass
- [x] Linting passes (`python -m ruff check .`)
- [x] Type checking passes for the Linux target (`python -m mypy
headroom --ignore-missing-imports --platform linux`)
- [x] New regression tests added
- [x] Real provider-runtime replay performed

### Test Output

```text
New regressions against fc1ba891: 14 failed, 3 passed
python -m pytest tests/test_providers tests/test_provider_model_fallback.py tests/test_provider_model_metadata.py tests/test_provider_registry.py tests/test_provider_registry_extended.py tests/test_proxy_anthropic_model_sanitization.py
304 passed

Makefile ci-precheck-python test selection, using the locally built Rust extension:
178 passed, 5 skipped

python -m ruff check .
All checks passed!
python -m ruff format . --check
1860 files already formatted
python -m mypy headroom --ignore-missing-imports --platform linux
Success: no issues found in 579 source files

cargo fmt --all -- --check: passed
cargo clippy --workspace -- -D warnings: passed
commitlint --from origin/main --to HEAD --config .commitlintrc.json: passed
```

Windows validation limits: native Windows mypy reports seven existing
POSIX-attribute errors (`os.fchmod`, `O_DIRECTORY`, `O_NOFOLLOW`,
`O_NONBLOCK`), with the same errors reproduced using the original
provider source. The local mypy hook was skipped after the equivalent
Linux-target check above passed. Other applicable pre-commit and
commit-message hooks passed.

`cargo test --workspace` compiled successfully with Rust 1.95.0,
`x86_64-pc-windows-gnu`, then the unchanged `headroom-core` test
executable exited with `0xc0000005` before running tests; invoking it
with `--list` produced the same failure. Rust tests therefore have no
local passing result. The Windows Make wrapper also fails to interpret
the shell recipe (`command was unexpected at this time`). Push uses the
hook's documented `--no-verify` option after the explicit checks above.
The Rust extension was built from this checkout, installed editable, and
import-verified with `DiffCompressor` and `SmartCrusher`.

## Real Behavior Proof

- Environment: Windows x64, Python 3.14.5, LiteLLM 1.101.3, default
`ProxyConfig`, locally built editable Headroom. Each sequence uses the
actual shared Anthropic provider from `build_proxy_provider_runtime` and
LiteLLM's installed model catalog.
- Exact steps, run with the checkout's Python:

```python
from headroom.providers.registry import build_proxy_provider_runtime
from headroom.proxy.models import ProxyConfig
import litellm

for order in [("gpt-4", "gpt-4-turbo"), ("gpt-4-turbo", "gpt-4")]:
    runtime = build_proxy_provider_runtime(ProxyConfig())
    provider = runtime.pipeline_provider("anthropic")
    assert runtime.pipeline_provider("anthropic") is provider
    actual = {model: provider.get_context_limit(model) for model in order}
    expected = {model: litellm.get_model_info(model)["max_input_tokens"] for model in order}
    print(order, actual, expected)
    assert actual == expected
```

- Before: short-ID-first returns `gpt-4=8192, gpt-4-turbo=8192`; reverse
order returns the correct values.
- After: both orders return `gpt-4=8192, gpt-4-turbo=128000`, matching
the real catalog.
- Not tested: live model-provider requests, OpenCode traffic, or
provider billing. The reproduction exercises the production
runtime/provider objects without mocks or online inference calls. The
complete Python suite was not run.

## Runtime Rollout Safety

- Rollout-managed features/channel: existing default provider
resolution; no new rollout flag.
- Stable/default behavior changed: resolved model IDs retain their own
context limits across successive requests.
- Disable/rollback: configure an exact context limit through the
existing configuration, or revert this commit.
- Unsafe override required: none.
- Qualification impact: provider-resolution regressions and runtime
replay above.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] Code follows the project's style guidelines
- [x] Added regressions that fail before the fix
- [x] Relevant new and existing Python tests pass locally
- [x] `CHANGELOG.md` is unchanged
- Documentation: existing configuration and public API remain
applicable.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4593"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
emme1t
2026-11-03 23:44:06 -06:00
committed by GitHub
co-authored by JD Davis
parent 66e1467682
commit 51258ef14a
2 changed files with 79 additions and 6 deletions
+12 -6
View File
@@ -811,6 +811,10 @@ class AnthropicProvider(Provider):
if context_limits:
self._context_limits.update(context_limits)
# Dynamically resolved limits belong to an exact model id. Keep them
# separate from the configured entries used for release/alias matching.
self._context_limit_cache: dict[str, int] = {}
@property
def name(self) -> str:
return "anthropic"
@@ -856,6 +860,8 @@ class AnthropicProvider(Provider):
# Check explicit and loaded limits
if model in self._context_limits:
return self._context_limits[model]
if model in self._context_limit_cache:
return self._context_limit_cache[model]
# Releases and aliases of a known id (e.g. "claude-3-5-sonnet" matches
# "claude-3-5-sonnet-20241022"); see `_is_release_of`.
@@ -871,11 +877,11 @@ class AnthropicProvider(Provider):
if info:
if "max_input_tokens" in info and info["max_input_tokens"] is not None:
limit = int(info["max_input_tokens"])
self._context_limits[model] = limit
self._context_limit_cache[model] = limit
return limit
if "max_tokens" in info and info["max_tokens"] is not None:
limit = int(info["max_tokens"])
self._context_limits[model] = limit
self._context_limit_cache[model] = limit
return limit
except Exception as e:
logger.debug(f"LiteLLM get_model_info failed for {model}: {e}")
@@ -886,20 +892,20 @@ class AnthropicProvider(Provider):
limit = cast(int, _PATTERN_DEFAULTS[tier]["context"])
self._warn_unknown_model(model, limit, f"inferred from '{tier}' tier")
# Cache for future calls
self._context_limits[model] = limit
self._context_limit_cache[model] = limit
return limit
# Fallback for unknown Claude models
if model.startswith("claude"):
limit = cast(int, _UNKNOWN_CLAUDE_DEFAULT["context"])
self._warn_unknown_model(model, limit, "using default Claude limit")
self._context_limits[model] = limit
self._context_limit_cache[model] = limit
return limit
# Non-Claude model - use conservative default
limit = 128000
self._warn_unknown_model(model, limit, "unknown provider, using conservative default")
self._context_limits[model] = limit
self._context_limit_cache[model] = limit
return limit
def _warn_unknown_model(self, model: str, limit: int, reason: str) -> None:
@@ -916,7 +922,7 @@ class AnthropicProvider(Provider):
def supports_model(self, model: str) -> bool:
"""Check if this provider supports the given model."""
model = sanitize_anthropic_model_id(model)
if model in self._context_limits:
if model in self._context_limits or model in self._context_limit_cache:
return True
# Check prefix matches - support all Claude models
return model.startswith("claude")
+67
View File
@@ -3,6 +3,73 @@
import pytest
class TestContextLimitCache:
@pytest.mark.parametrize("metadata_field", ["max_input_tokens", "max_tokens"])
@pytest.mark.parametrize(
("first", "second", "metadata", "expected"),
[
("gpt-4", "gpt-4-turbo", {"gpt-4": 8192, "gpt-4-turbo": 128000}, (8192, 128000)),
("gpt-4-turbo", "gpt-4", {"gpt-4": 8192, "gpt-4-turbo": 128000}, (128000, 8192)),
("gateway", "gateway-large", {"gateway-large": 1000000}, (128000, 1000000)),
("gateway", "gateway-unknown", {"gateway": 64000}, (64000, 128000)),
(
"claude-future-sonnet",
"claude-future-sonnet-extended",
{"claude-future-sonnet-extended": 1000000},
(200000, 1000000),
),
("gateway", "gateway-sonnet", {}, (128000, 200000)),
(
"claude-future",
"claude-future-long",
{"claude-future-long": 640000},
(200000, 640000),
),
(
"gateway-20260101",
"gateway",
{"gateway-20260101": 64000, "gateway": 128000},
(64000, 128000),
),
],
)
def test_each_model_keeps_its_own_resolved_limit(
self, monkeypatch, first, second, metadata, expected, metadata_field
):
import headroom.providers.anthropic as anthropic_module
lookups = []
def get_model_info(model):
lookups.append(model)
return {metadata_field: metadata[model]} if model in metadata else None
monkeypatch.setattr(
anthropic_module, "_get_litellm_clients", lambda: (None, get_model_info)
)
provider = anthropic_module.AnthropicProvider(warn=False)
assert (provider.get_context_limit(first), provider.get_context_limit(second)) == expected
assert (provider.get_context_limit(first), provider.get_context_limit(second)) == expected
assert lookups == [first, second]
assert provider.supports_model(first)
assert provider.supports_model(second)
def test_configured_aliases_and_1m_suffix_keep_their_limits(self, monkeypatch):
import headroom.providers.anthropic as anthropic_module
monkeypatch.setattr(anthropic_module, "_get_litellm_clients", lambda: (None, None))
provider = anthropic_module.AnthropicProvider(
context_limits={"gateway": 64000, "gateway-exact": 96000}, warn=False
)
assert provider.get_context_limit("gateway-20260101") == 64000
assert provider.get_context_limit("gateway-exact") == 96000
assert provider.get_context_limit("gateway[1m]") == 1000000
assert provider.get_context_limit("gateway") == 64000
assert provider.get_context_limit("claude-sonnet-4-5-20250929") == 200000
class TestAnthropicModelSanitization:
def test_sanitize_model_id_removes_ansi_escape_sequences(self):
from headroom.providers.anthropic import sanitize_anthropic_model_id