Files
headroom/tests/test_provider_registry.py
Luis Araya 931e5b9666 fix(anthropic): route /api/hello connectivity probes to Anthropic (#3336) (#4059)
## Description

Claude Code (and other Anthropic SDK clients) uses `HEAD /api/hello` or
`GET /api/hello` as an unauthenticated connectivity probe to verify
upstream reachability. Because this probe carries no Anthropic
authentication headers, the unrouted passthrough in
`select_passthrough_base_url` fell through to the default OpenAI target,
opening an unexpected outbound connection to `api.openai.com` (which
returns 404) on an Anthropic-only setup.

This mirrors the Copilot completion routing fix (#3076): `/api/hello` is
an Anthropic-specific endpoint and is now explicitly routed to the
Anthropic target even when no authentication headers are present.

Closes #3336

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/providers/route_specs.py`: add `GET` and `HEAD` `/api/hello`
to `ANTHROPIC_PASSTHROUGH_ROUTES`.
- `headroom/providers/proxy_targets.py`: add `is_anthropic_hello_path()`
and route `/api/hello` to `api_target(proxy, "anthropic")` in
`select_passthrough_base_url()`.
- `headroom/providers/registry.py`: accept optional `path` in
`select_passthrough_base_url()` to route `/api/hello` to Anthropic
target.
- `tests/test_provider_route_specs.py`,
`tests/test_provider_proxy_targets.py`,
`tests/test_provider_proxy_routes.py`,
`tests/test_provider_registry.py`: update existing route and target
assertions.
- `tests/test_anthropic_hello_probe_routing.py`: new targeted regression
tests for path normalization, base URL resolution, and unauthenticated
end-to-end `HEAD` and `GET` routing.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_provider_route_specs.py tests/test_provider_proxy_targets.py tests/test_provider_proxy_routes.py tests/test_provider_registry.py tests/test_anthropic_hello_probe_routing.py -q
..................................................................       [100%]
66 passed, 1 warning in 56.04s

$ ruff check headroom/providers/route_specs.py headroom/providers/proxy_targets.py headroom/providers/registry.py tests/test_anthropic_hello_probe_routing.py
All checks passed!

$ ruff format --check headroom/providers/route_specs.py headroom/providers/proxy_targets.py headroom/providers/registry.py tests/test_anthropic_hello_probe_routing.py
4 files already formatted

$ mypy headroom/providers/route_specs.py headroom/providers/proxy_targets.py headroom/providers/registry.py tests/test_anthropic_hello_probe_routing.py --follow-imports=silent --ignore-missing-imports
Success: no issues found in 4 source files
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.14.6, local test server with
Anthropic and OpenAI target stubs.
- Exact command / steps: Sent unauthenticated `HEAD /api/hello`
(carrying `User-Agent: Bun/1.1.20`) and `GET /api/hello` requests to the
proxy using `fastapi.testclient.TestClient`.
- Observed result: Both requests routed directly to
`https://api.anthropic.test` with HTTP 200; `fake_passthrough` recorded
`base_url == "https://api.anthropic.test"` and `provider ==
"anthropic"`. Other unauthenticated endpoints continue falling through
to `https://api.openai.test`.
- Not tested: Live Anthropic egress against production Claude API.

### Maintainer live qualification (2026-10-08)

- Environment: actual Windows 11, Python 3.12.13, uvicorn proxy over
loopback TCP. Provider API-key environment variables were empty; the
client sent no authentication headers.
- Exact steps: start the unchanged proxy from this head with
`anthropic_api_url=https://api.anthropic.com` and the OpenAI target
deliberately set to unreachable `http://127.0.0.1:1`; send keyless `HEAD
/api/hello` and `GET /api/hello` through the proxy with httpx.
- Observed: public Anthropic returned **HEAD 200 with zero body bytes**
and **GET 200 with `{"message": "hello"}`**. No handler or provider call
was mocked. A second real TCP probe against a loopback Anthropic test
server captured exactly HEAD and GET `/api/hello`, both 200, while the
OpenAI target remained unreachable. **66 focused provider-routing tests
passed**.
- Scope: connectivity canaries only; no inference request or
credentialed API action was made. Hosted CI was held by GitHub's fork
workflow-approval gate, which the maintainer approved after reviewing
this exact head; CI is now running and PR approval remains pending until
it qualifies.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: `/api/hello` probes now forward to
Anthropic rather than OpenAI. Other unauthenticated routes remain
unchanged.
- Kill switch / disable path: N/A.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4059"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
2026-10-08 22:30:20 -05:00

507 lines
17 KiB
Python

from __future__ import annotations
import logging
import pytest
from headroom.providers.registry import (
ProviderApiOverrides,
build_proxy_provider_runtime,
create_proxy_backend,
format_backend_status,
resolve_api_overrides,
resolve_api_targets,
resolve_extra_headers,
)
from headroom.proxy.models import ProxyConfig
def test_resolve_api_overrides_prefers_explicit_values_over_environment(monkeypatch) -> None:
monkeypatch.setenv("ANTHROPIC_TARGET_API_URL", "https://env.anthropic.example/v1")
monkeypatch.setenv("OPENAI_TARGET_API_URL", "https://env.openai.example/v1")
monkeypatch.setenv("VERTEX_TARGET_API_URL", "https://env-vertex-aiplatform.example/v1")
overrides = resolve_api_overrides(
anthropic_api_url="https://cli.anthropic.example/v1",
openai_api_url=None,
gemini_api_url=None,
cloudcode_api_url=None,
vertex_api_url="https://cli-vertex-aiplatform.example/v1",
)
assert overrides == ProviderApiOverrides(
anthropic="https://cli.anthropic.example/v1",
openai="https://env.openai.example/v1",
gemini=None,
cloudcode=None,
vertex="https://cli-vertex-aiplatform.example/v1",
)
def test_resolve_api_targets_normalizes_trailing_v1() -> None:
targets = resolve_api_targets(
ProviderApiOverrides(
anthropic="https://anthropic.example/v1/",
openai="https://openai.example/v1",
gemini="https://gemini.example/v1",
cloudcode="https://cloudcode.example/v1/",
vertex="https://vertex.example/v1/",
)
)
assert targets.anthropic == "https://anthropic.example"
assert targets.openai == "https://openai.example"
assert targets.gemini == "https://gemini.example"
assert targets.cloudcode == "https://cloudcode.example"
assert targets.vertex == "https://vertex.example"
def test_copilot_openai_target_routes_anthropic_to_copilot() -> None:
"""When the OpenAI target is a Copilot host and no Anthropic override is set,
the Anthropic target must default to the same Copilot host.
Copilot serves Claude models via its Anthropic surface (``/v1/messages``) on
the same host. Without this, Claude requests fell back to api.anthropic.com
and 401'd with the Copilot bearer ("Invalid bearer token", #3247).
"""
targets = resolve_api_targets(
ProviderApiOverrides(
anthropic=None,
openai="https://api.githubcopilot.com",
gemini=None,
cloudcode=None,
vertex=None,
)
)
assert targets.openai == "https://api.githubcopilot.com"
assert targets.anthropic == "https://api.githubcopilot.com"
def test_explicit_anthropic_override_wins_over_copilot_default() -> None:
"""An explicit Anthropic target is never overridden by the Copilot default."""
targets = resolve_api_targets(
ProviderApiOverrides(
anthropic="https://api.anthropic.com",
openai="https://api.githubcopilot.com",
gemini=None,
cloudcode=None,
vertex=None,
)
)
assert targets.anthropic == "https://api.anthropic.com"
def test_non_copilot_openai_target_leaves_anthropic_default() -> None:
"""A non-Copilot OpenAI target must not touch the Anthropic default."""
targets = resolve_api_targets(
ProviderApiOverrides(
anthropic=None,
openai="https://api.openai.com",
gemini=None,
cloudcode=None,
vertex=None,
)
)
assert targets.anthropic == "https://api.anthropic.com"
def test_proxy_config_exposes_provider_api_overrides() -> None:
config = ProxyConfig(
anthropic_api_url="https://anthropic.example",
openai_api_url="https://openai.example",
gemini_api_url=None,
cloudcode_api_url="https://cloudcode.example",
vertex_api_url="https://vertex.example",
)
assert config.provider_api_overrides == ProviderApiOverrides(
anthropic="https://anthropic.example",
openai="https://openai.example",
gemini=None,
cloudcode="https://cloudcode.example",
vertex="https://vertex.example",
)
def test_format_backend_status_for_anyllm() -> None:
assert (
format_backend_status(
backend="anyllm",
anyllm_provider="groq",
bedrock_region="us-central1",
)
== "Groq via any-llm"
)
def test_format_backend_status_for_anthropic_direct() -> None:
assert (
format_backend_status(
backend="anthropic",
anyllm_provider="ignored",
bedrock_region=None,
)
== "ANTHROPIC (direct API)"
)
def test_proxy_provider_runtime_routes_model_metadata_and_passthrough() -> None:
runtime = build_proxy_provider_runtime(ProxyConfig())
assert runtime.model_metadata_provider({"x-api-key": "test"}) == "anthropic"
assert runtime.model_metadata_provider({}) == "openai"
assert (
runtime.select_passthrough_base_url({"x-api-key": "test"}) == runtime.api_targets.anthropic
)
assert (
runtime.select_passthrough_base_url({"x-goog-api-key": "test"})
== runtime.api_targets.gemini
)
assert runtime.select_passthrough_base_url({"api-key": "azure", "x-headroom-base-url": ""}) == (
runtime.api_targets.openai
)
assert runtime.select_passthrough_base_url({}, "/api/hello") == runtime.api_targets.anthropic
assert runtime.select_passthrough_base_url({}, "/other/path") == runtime.api_targets.openai
def test_create_proxy_backend_handles_missing_litellm_backend(caplog) -> None:
logger = logging.getLogger("test")
with caplog.at_level(logging.WARNING):
missing = create_proxy_backend(
backend="bedrock",
anyllm_provider="ignored",
bedrock_region="us-east-1",
logger=logger,
litellm_backend_cls=lambda provider, region, profile_name=None: (_ for _ in ()).throw(
ImportError("missing")
),
)
assert missing is None
assert "LiteLLM backend not available" in caplog.text
def test_create_proxy_backend_logs_structured_failure_details(caplog) -> None:
logger = logging.getLogger("test")
with caplog.at_level(logging.ERROR):
missing = create_proxy_backend(
backend="bedrock",
anyllm_provider="ignored",
bedrock_region="us-east-1",
logger=logger,
litellm_backend_cls=lambda provider, region, profile_name=None: (_ for _ in ()).throw(
RuntimeError("boom")
),
)
assert missing is None
assert "backend initialization failed: backend=litellm-bedrock provider=bedrock error=boom" in (
caplog.text
)
def test_proxy_provider_runtime_loaders_cache_backend_types(monkeypatch) -> None:
import headroom.providers.registry as registry
anyllm_loads = 0
litellm_loads = 0
class FakeAnyLLMBackend:
pass
class FakeLiteLLMBackend:
pass
def fake_import(name, globals=None, locals=None, fromlist=(), level=0):
nonlocal anyllm_loads, litellm_loads
if name == "headroom.backends.anyllm":
anyllm_loads += 1
return type("Module", (), {"AnyLLMBackend": FakeAnyLLMBackend})()
if name == "headroom.backends.litellm":
litellm_loads += 1
return type("Module", (), {"LiteLLMBackend": FakeLiteLLMBackend})()
raise AssertionError(name)
monkeypatch.setattr(registry, "AnyLLMBackendType", None)
monkeypatch.setattr(registry, "LiteLLMBackendType", None)
monkeypatch.setattr("builtins.__import__", fake_import)
assert registry._load_anyllm_backend() is FakeAnyLLMBackend
assert registry._load_anyllm_backend() is FakeAnyLLMBackend
assert registry._load_litellm_backend() is FakeLiteLLMBackend
assert registry._load_litellm_backend() is FakeLiteLLMBackend
assert anyllm_loads == 1
assert litellm_loads == 1
def test_proxy_provider_runtime_transport_helpers_handle_missing_usage() -> None:
import headroom.providers.registry as registry
class Storage:
def __init__(self) -> None:
self.saved = []
def save(self, metrics) -> None:
self.saved.append(metrics)
client = type(
"Client",
(),
{
"_storage": Storage(),
"_original": type(
"Original",
(),
{
"chat": type(
"Chat",
(),
{
"completions": type(
"Completions",
(),
{
"create": staticmethod(
lambda **kwargs: type("Resp", (), {"usage": None})()
)
},
)()
},
)(),
"messages": type(
"Messages",
(),
{
"create": staticmethod(
lambda **kwargs: type("Resp", (), {"usage": None})()
)
},
)(),
},
)(),
},
)()
openai_metrics = type("Metrics", (), {"tokens_output": 0, "cached_tokens": 0})()
anthropic_metrics = type("Metrics", (), {"tokens_output": 0, "cached_tokens": 0})()
registry._call_openai_transport(
client,
model="gpt-4o",
messages=[],
stream=False,
metrics=openai_metrics,
)
registry._call_anthropic_transport(
client,
model="claude",
messages=[],
stream=False,
metrics=anthropic_metrics,
)
assert openai_metrics.tokens_output == 0
assert openai_metrics.cached_tokens == 0
assert anthropic_metrics.tokens_output == 0
assert anthropic_metrics.cached_tokens == 0
assert len(client._storage.saved) == 2
def test_proxy_provider_runtime_transport_helpers_handle_usage_without_optional_cache_fields() -> (
None
):
import headroom.providers.registry as registry
class Storage:
def __init__(self) -> None:
self.saved = []
def save(self, metrics) -> None:
self.saved.append(metrics)
client = type(
"Client",
(),
{
"_storage": Storage(),
"_original": type(
"Original",
(),
{
"chat": type(
"Chat",
(),
{
"completions": type(
"Completions",
(),
{
"create": staticmethod(
lambda **kwargs: type(
"Resp",
(),
{
"usage": type(
"Usage",
(),
{"completion_tokens": 7},
)()
},
)()
)
},
)()
},
)(),
"messages": type(
"Messages",
(),
{
"create": staticmethod(
lambda **kwargs: type(
"Resp",
(),
{
"usage": type(
"Usage",
(),
{"output_tokens": 5},
)()
},
)()
)
},
)(),
},
)(),
},
)()
openai_metrics = type("Metrics", (), {"tokens_output": 0, "cached_tokens": 0})()
anthropic_metrics = type("Metrics", (), {"tokens_output": 0, "cached_tokens": 0})()
registry._call_openai_transport(
client,
model="gpt-4o",
messages=[],
stream=False,
metrics=openai_metrics,
)
registry._call_anthropic_transport(
client,
model="claude",
messages=[],
stream=False,
metrics=anthropic_metrics,
)
assert openai_metrics.tokens_output == 7
assert openai_metrics.cached_tokens == 0
assert anthropic_metrics.tokens_output == 5
assert anthropic_metrics.cached_tokens == 0
assert len(client._storage.saved) == 2
def test_proxy_provider_runtime_openai_transport_handles_prompt_details_without_cached_tokens() -> (
None
):
import headroom.providers.registry as registry
class Storage:
def __init__(self) -> None:
self.saved = []
def save(self, metrics) -> None:
self.saved.append(metrics)
client = type(
"Client",
(),
{
"_storage": Storage(),
"_original": type(
"Original",
(),
{
"chat": type(
"Chat",
(),
{
"completions": type(
"Completions",
(),
{
"create": staticmethod(
lambda **kwargs: type(
"Resp",
(),
{
"usage": type(
"Usage",
(),
{
"completion_tokens": 9,
"prompt_tokens_details": type(
"Details",
(),
{},
)(),
},
)()
},
)()
)
},
)()
},
)()
},
)(),
},
)()
metrics = type("Metrics", (), {"tokens_output": 0, "cached_tokens": 0})()
registry._call_openai_transport(
client,
model="gpt-4o",
messages=[],
stream=False,
metrics=metrics,
)
assert metrics.tokens_output == 9
assert metrics.cached_tokens == 0
assert len(client._storage.saved) == 1
def test_resolve_extra_headers_cli_wins_over_env(monkeypatch) -> None:
monkeypatch.setenv("ANTHROPIC_TARGET_API_HEADERS", '{"Env-Header": "env-value"}')
result = resolve_extra_headers('{"Cli-Header": "cli-value"}', "ANTHROPIC_TARGET_API_HEADERS")
assert result == {"Cli-Header": "cli-value"}
def test_resolve_extra_headers_falls_back_to_env(monkeypatch) -> None:
monkeypatch.setenv("OPENAI_TARGET_API_HEADERS", '{"Env-Header": "env-value"}')
result = resolve_extra_headers(None, "OPENAI_TARGET_API_HEADERS")
assert result == {"Env-Header": "env-value"}
def test_resolve_extra_headers_unset_returns_none(monkeypatch) -> None:
monkeypatch.delenv("ANTHROPIC_TARGET_API_HEADERS", raising=False)
assert resolve_extra_headers(None, "ANTHROPIC_TARGET_API_HEADERS") is None
def test_resolve_extra_headers_invalid_json_raises(monkeypatch) -> None:
with pytest.raises(ValueError):
resolve_extra_headers("not json", "ANTHROPIC_TARGET_API_HEADERS")
def test_resolve_extra_headers_non_object_raises(monkeypatch) -> None:
with pytest.raises(ValueError):
resolve_extra_headers('["a", "b"]', "ANTHROPIC_TARGET_API_HEADERS")
def test_resolve_extra_headers_non_string_value_raises(monkeypatch) -> None:
with pytest.raises(ValueError):
resolve_extra_headers('{"Key": 123}', "ANTHROPIC_TARGET_API_HEADERS")