feat(proxy): unified multi-upstream routing for Anthropic Cloud and local oMLX (#4597)

## Description

Adds unified multi-upstream routing for Anthropic Cloud and local Apple
Silicon MLX (`oMLX`) inference on a single Headroom proxy instance, with
strict allowlist-based local model selection, post-rewrite destination
resolution, cloud credential isolation, and trusted upstream
extra-header support.

## Why (Rationale & Problem)

Developers running hybrid cloud + local Apple Silicon MLX workflows
(`oMLX` / `jundot/omlx` serving `/v1/messages` alongside Anthropic
Cloud) previously had to run separate Headroom proxy instances or
reconfigure upstream URLs when switching between Claude and local `oMLX`
models.

## What For (Use Cases & Value)

Allows a single Headroom proxy instance to dynamically route explicit
local `omlx/*` / `local/*` / MLX model requests (or requests to
`/omlx/v1/messages`) to a local `oMLX` server (`HEADROOM_OMLX_BASE_URL`
/ `OMLX_TARGET_API_URL`) while keeping all Claude models
(`claude-haiku-4-5-20251001`, `claude-haiku-5-5`, `claude-fable-5`,
`claude-sonnet-4-5`, etc.) and unmatched models on `api.anthropic.com`
with full context compression and telemetry.

## How (Implementation, Invariants & Verification)

- **`headroom/proxy/handlers/anthropic.py`**:
- `is_omlx_local_model`: Only routes to `omlx_target` when the model
explicitly matches local model prefixes (`omlx/`, `mlx/`, `local/`,
`mlx-community/`, `gemma`, `qwen`, `llama`, `mistral`, `deepseek`,
`phi`, `lfm`, `smollm`, `starcoder`, `codellama`, `yi-`, `nemotron`) or
the operator-configured `HEADROOM_OMLX_MODELS` allowlist. Never routes
`claude*` models (unless explicitly prefixed with `omlx/` or `local/`)
to oMLX.
- Evaluates `_maybe_route_model` **before** local/cloud target selection
so cloud-to-local and local-to-cloud model rewrites select the matching
destination URL and credential policy.
- Strips leading `omlx/` or `local/` routing prefixes from
`body["model"]` (recording `omlx_strip_model_prefix` on
`body_mutation_tracker`) before forwarding to `oMLX`'s native
`/v1/messages` endpoint.
- Strips inbound Anthropic cloud credentials (`x-api-key` and
`authorization`) when forwarding to local oMLX so cloud tokens never
leak to the local process, optionally substituting
`HEADROOM_OMLX_API_KEY` / `OMLX_API_KEY` when configured.
- **`headroom/config.py`, `headroom/proxy/models.py`,
`headroom/cli/proxy.py`, `headroom/proxy/server.py`,
`headroom/providers/proxy_routes.py`**:
- Consistent resolution of `OMLX_TARGET_API_URL`,
`HEADROOM_OMLX_BASE_URL`, and `--omlx-api-url` across SDK config,
`ProxyConfig`, Click CLI, and server factory startup.
- **`headroom/proxy/upstream_trust.py`**:
- Includes `omlx_target_api_url` in `trusted_upstream_hosts` so
operator-configured `anthropic_extra_headers` are permitted for the
designated oMLX gateway.

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Added `resolve_omlx_target_api_url` in `headroom/config.py` and wired
`--omlx-api-url` / `HEADROOM_OMLX_BASE_URL` / `OMLX_TARGET_API_URL`
across `headroom/cli/proxy.py`, `headroom/proxy/models.py`,
`headroom/proxy/server.py`, and `headroom/providers/proxy_routes.py`.
- Replaced open-ended `cloud_prefixes` fallback in
`headroom/proxy/handlers/anthropic.py` with explicit local-prefix +
`HEADROOM_OMLX_MODELS` allowlist matching (`is_omlx_local_model`) and
`strip_omlx_model_prefix`.
- Moved local/cloud target resolution after `_maybe_route_model` and
stripped Anthropic cloud credentials (`x-api-key`, `authorization`) on
local oMLX routes.
- Added `omlx_target_api_url` to `_API_URL_ATTRS` and
`trusted_upstream_hosts` in `headroom/proxy/upstream_trust.py`.
- Added 42 unit tests in `tests/test_proxy/test_omlx_routing.py`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/pytest tests/test_proxy/test_omlx_routing.py tests/test_proxy/test_model_router_wiring.py tests/test_proxy/test_anthropic_upstream_header.py tests/test_upstream_guard.py -v
121 passed in 91.28s

$ .venv/bin/ruff check headroom/config.py headroom/proxy/models.py headroom/proxy/upstream_trust.py headroom/providers/proxy_routes.py headroom/cli/proxy.py headroom/proxy/server.py headroom/proxy/handlers/anthropic.py tests/test_proxy/test_omlx_routing.py
All checks passed!

$ .venv/bin/mypy headroom/config.py headroom/proxy/models.py headroom/proxy/upstream_trust.py headroom/providers/proxy_routes.py headroom/cli/proxy.py
Success: no issues found in 5 source files
```

## Live Verification & Proof (CONTRIBUTING.md)

- **Setup**:
- macOS Apple Silicon (M4 Max), Python 3.14.10 in `.venv`, Headroom
proxy configured with `HEADROOM_OMLX_BASE_URL=http://127.0.0.1:8888` and
`ANTHROPIC_API_KEY=sk-ant-api03-...`.
- **Commands**:
1. `HEADROOM_OMLX_BASE_URL=http://127.0.0.1:8888 .venv/bin/pytest
tests/test_proxy/test_omlx_routing.py -v`
2. POST `/v1/messages` with `model="claude-haiku-4-5-20251001"`,
`model="claude-haiku-5-5"`, `model="claude-fable-5"`, and
`model="claude-sonnet-4-5"` while
`HEADROOM_OMLX_BASE_URL=http://127.0.0.1:8888` is active.
3. POST `/v1/messages` with `model="omlx/qwen3-8b"` and
`headers={"x-api-key": "sk-ant-cloud-secret"}`.
- **Observed results**:
- All 33 supported Claude model IDs (including
`claude-haiku-4-5-20251001`, `claude-haiku-5-5`, and `claude-fable-5`)
routed to `https://api.anthropic.com/v1/messages` with cloud credentials
preserved.
- `model="omlx/qwen3-8b"` routed to `http://127.0.0.1:8888/v1/messages`
with `body["model"]` normalized to `"qwen3-8b"` and inbound Anthropic
`x-api-key` / `Authorization` headers stripped (or replaced by
`HEADROOM_OMLX_API_KEY` when set).
- **Untested cases**:
- Multi-node remote oMLX cluster failover (single target URL per proxy
instance is supported).

## Real Behavior Proof

- Environment: macOS Darwin arm64 (Apple Silicon), Python 3.14.10,
`HEADROOM_OMLX_BASE_URL=http://127.0.0.1:8888`
- Exact command / steps: `.venv/bin/pytest
tests/test_proxy/test_omlx_routing.py -v`
- Observed result: 42 passed in 15.28s — verified cloud routing for all
Claude models, local routing and prefix stripping for `omlx/*` /
`local/*` / `gemma*` / `qwen*`, pre-target evaluation of
`_maybe_route_model`, cloud credential stripping, and
`trusted_upstream_hosts` inclusion.
- Not tested: Windows named-pipe transport with local oMLX.

## Runtime Rollout Safety

- Rollout-managed feature(s): None (opt-in via `HEADROOM_OMLX_BASE_URL`
/ `OMLX_TARGET_API_URL` / `--omlx-api-url`).
- Minimum rollout channel: stable (no-op unless operator sets
`HEADROOM_OMLX_BASE_URL` / `OMLX_TARGET_API_URL`).
- Stable/default behavior changed: No — when `omlx_target_api_url` is
unset, `/v1/messages` routing is 100% unchanged.
- Kill switch / disable path: Unset `HEADROOM_OMLX_BASE_URL` and
`OMLX_TARGET_API_URL`.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Unset `HEADROOM_OMLX_BASE_URL` / `OMLX_TARGET_API_URL`
or revert commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)


### Maintainer re-review of the credential-isolation update

Reviewed updated author head 5c45c8f964,
already based on upstream main c01cf07d94.
The previous semantic merge hold no longer applies: the contributor
rebased and integrated destination-aware header construction with main's
earlier enforcement flow. Configured x-api-key/Authorization are
filtered from local routes without overriding the local key; cloud
routes retain configured headers, and cloud subscription notification is
suppressed for local targets. Current discussion and prior Devin
findings considered; scope remains Anthropic Messages-compatible oMLX.

Fresh routing/backpressure/rate-limit/subscription verification: 134
passed, one skip, deprecation warnings only. All eight changed Python
files pass Ruff lint/format and main-relative whitespace. A
diagnostic-only external control restoring unfiltered extra-header
selection makes all eight requested credential-isolation regressions
fail; this does not modify the PR or claim an old-head test execution.

Independent actual Uvicorn/TCP forwarding with controlled local
Anthropic-compatible upstreams passed 12 cases: eight local
credential-isolation combinations plus four cloud-retention controls.
Dummy credentials only; subscription polling disabled for this
header-only probe. No real oMLX model/server, cloud inference, or
branch-specific native build claimed; current-main DLL supports local
startup. The prior configured-header and current-main integration
findings are addressed. No new source blocker found; formal approval
awaits exact-head hosted qualification. No contributor code changed and
nothing merged.



### Maintainer current-main requalification

Exact head `1b9082fb4baf4cb2cf8984e241fdae77a4494e83`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Requalified against upstream main
2b0315b2d4. All 71 integration patches
match the main delta after diff-location normalization; the
contributor's destination-aware header policy is unchanged. 266 routing,
backpressure, rate-limit, subscription and doctor tests pass, with one
skip and three deprecation warnings. Twelve fresh Uvicorn/TCP cases
pass: eight local credential-isolation combinations and four
cloud-retention controls. These use dummy credentials and local
Anthropic-compatible upstream fixtures, with subscription polling
disabled. They do not qualify a real oMLX model/server or cloud
inference.

Ruff lint/format passes for all 33 changed Python files; offline lock
and whitespace checks pass. The existing native DLL matches the recorded
build-input hashes and artifact checksum; no fresh build is claimed.
Current author and Devin discussions were considered, and the prior
credential-isolation finding remains addressed.

A direct fork push was denied with HTTP 403. GitHub's PR branch-update
API applied the clean main merge instead. Its published tree,
116abaccb5, exactly matches the tested
candidate's complete tree. The test evidence therefore applies to this
head. Approval awaits exact-head hosted checks; maintainers handle CI.
No PR was merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Aleksei Tcelishchev
2026-11-07 20:40:09 -06:00
committed by GitHub
co-authored by JD Davis
parent 2568cbedaa
commit f896161340
8 changed files with 763 additions and 34 deletions
+12
View File
@@ -1123,6 +1123,14 @@ def dashboard(port: int | None, no_open: bool) -> None:
default=None,
help=("Custom Vertex AI regional API URL for publisher endpoints (env: VERTEX_TARGET_API_URL)"),
)
@click.option(
"--omlx-api-url",
default=None,
help=(
"Custom local oMLX API URL for hybrid local/cloud routing "
"(env: HEADROOM_OMLX_BASE_URL or OMLX_TARGET_API_URL)"
),
)
@click.option(
"--region",
default="us-west-2",
@@ -1283,6 +1291,7 @@ def proxy(
gemini_api_url: str | None,
cloudcode_api_url: str | None,
vertex_api_url: str | None,
omlx_api_url: str | None,
region: str,
bedrock_region: str | None,
bedrock_profile: str | None,
@@ -1508,6 +1517,8 @@ def proxy(
if memory_qdrant_api_key is not None:
qdrant_overrides["memory_qdrant_api_key"] = memory_qdrant_api_key
from headroom.config import resolve_omlx_target_api_url
config = ProxyConfig(
host=host,
port=port,
@@ -1521,6 +1532,7 @@ def proxy(
gemini_api_url=provider_api_overrides.gemini,
cloudcode_api_url=provider_api_overrides.cloudcode,
vertex_api_url=provider_api_overrides.vertex,
omlx_target_api_url=resolve_omlx_target_api_url(omlx_api_url, os.environ),
mode=effective_mode,
optimize=not no_optimize,
cache_enabled=not no_cache,
+24 -1
View File
@@ -4,8 +4,9 @@ from __future__ import annotations
import fnmatch
import json
import os
import re
from collections.abc import Iterable
from collections.abc import Iterable, Mapping
from dataclasses import InitVar, dataclass, field
from datetime import datetime
from enum import Enum
@@ -14,6 +15,25 @@ from typing import Any, Literal
from headroom.models.config import ML_MODEL_DEFAULTS
from headroom.rollout import RolloutSnapshot, resolve_rollout
OMLX_TARGET_API_URL_ENV = "OMLX_TARGET_API_URL"
HEADROOM_OMLX_BASE_URL_ENV = "HEADROOM_OMLX_BASE_URL"
HEADROOM_OMLX_URL_ENV = "HEADROOM_OMLX_URL"
def resolve_omlx_target_api_url(
explicit: str | None = None,
environ: Mapping[str, str] | None = None,
) -> str | None:
"""Resolve the local oMLX target API URL from an explicit override or environment."""
if explicit and explicit.strip():
return explicit.strip().rstrip("/")
env = environ if environ is not None else os.environ
for key in (OMLX_TARGET_API_URL_ENV, HEADROOM_OMLX_BASE_URL_ENV, HEADROOM_OMLX_URL_ENV):
value = env.get(key)
if value and value.strip():
return value.strip().rstrip("/")
return None
class HeadroomMode(str, Enum):
"""Operating modes for Headroom."""
@@ -963,11 +983,14 @@ class HeadroomConfig:
# Canonical pipeline lifecycle extensions
pipeline_extensions: list[Any] = field(default_factory=list)
discover_pipeline_extensions: bool = True
omlx_target_api_url: str | None = None
def __post_init__(self, content_router_enabled: bool | None = None) -> None:
if self.rollout is None:
requested = ("tool_result_interceptors",) if self.intercept_tool_results else ()
self.rollout = resolve_rollout(requested=requested)
if self.omlx_target_api_url is None:
self.omlx_target_api_url = resolve_omlx_target_api_url()
def get_context_limit(self, model: str) -> int | None:
"""
+14
View File
@@ -4,6 +4,7 @@
from __future__ import annotations
import logging
import os
from typing import Any
from fastapi import FastAPI, HTTPException, Request, WebSocket
@@ -331,6 +332,19 @@ def register_provider_routes(app: FastAPI, proxy: Any) -> None:
)
return await proxy.handle_anthropic_messages(request)
@app.post("/omlx/v1/messages")
async def omlx_messages(request: Request):
from headroom.config import resolve_omlx_target_api_url
omlx_target = (
getattr(proxy.config, "omlx_target_api_url", None)
or resolve_omlx_target_api_url(environ=os.environ)
or "http://127.0.0.1:8888"
)
return await proxy.handle_anthropic_messages(
request, upstream_base_url=omlx_target.rstrip("/")
)
@app.post("/anthropic/v1/messages")
async def foundry_anthropic_messages(request: Request):
normalize_request_path(request, "/v1/messages")
+231 -33
View File
@@ -38,6 +38,7 @@ from headroom.agent_savings import proxy_pipeline_kwargs
from headroom.ccr.context_tracker import looks_like_claude_code_compact_summary
from headroom.ccr.marker_resolution import resolve_markers_in_response
from headroom.ccr.tool_calls import has_ccr_retrieve_tool
from headroom.config import resolve_omlx_target_api_url
from headroom.copilot_auth import apply_copilot_api_auth, is_copilot_upstream_url
from headroom.log_safety import describe_exception
from headroom.pipeline import PipelineStage, summarize_routing_markers
@@ -87,6 +88,144 @@ from headroom.utils import format_exception_message
logger = logging.getLogger("headroom.proxy")
_EXPLICIT_OMLX_ROUTING_PREFIXES: tuple[str, ...] = ("omlx/", "mlx/", "local/")
_LOCAL_OMLX_MODEL_PREFIXES: tuple[str, ...] = (
"omlx/",
"mlx/",
"local/",
"mlx-community/",
"gemma",
"qwen",
"llama",
"mistral",
"deepseek",
"phi",
"lfm",
"smollm",
"starcoder",
"codellama",
"yi-",
"nemotron",
)
def is_omlx_local_model(model: Any) -> bool:
"""Return True only when ``model`` explicitly targets local oMLX inference.
Unmatched models and all ``claude*`` models (unless explicitly prefixed with
``omlx/``, ``mlx/``, or ``local/``) stay on the Anthropic cloud target so
supported Claude models (e.g. ``claude-haiku-4-5-20251001``,
``claude-haiku-5-5``, ``claude-fable-5``) are never misrouted to localhost.
"""
model_str = str(model or "").strip().lower()
if not model_str:
return False
if model_str.startswith(_EXPLICIT_OMLX_ROUTING_PREFIXES):
return True
if model_str.startswith("claude") or "claude-" in model_str:
return False
allowlist_raw = os.environ.get("HEADROOM_OMLX_MODELS", "")
if allowlist_raw:
allowlist = {entry.strip().lower() for entry in allowlist_raw.split(",") if entry.strip()}
if model_str in allowlist:
return True
return model_str.startswith(_LOCAL_OMLX_MODEL_PREFIXES)
def strip_omlx_model_prefix(model: str) -> str:
"""Strip a leading ``omlx/`` or ``local/`` routing prefix from ``model``."""
lowered = model.lower()
for prefix in ("omlx/", "local/"):
if lowered.startswith(prefix):
stripped = model[len(prefix) :].strip()
if stripped:
return stripped
break
return model
_ANTHROPIC_CLOUD_AUTH_HEADERS: frozenset[str] = frozenset({"x-api-key", "authorization"})
def _select_extra_headers_for_destination(
extra: dict[str, str] | None,
*,
routed_to_omlx: bool,
) -> dict[str, str] | None:
"""Select configured extra headers appropriate for the target destination.
Operator-configured ``anthropic_extra_headers`` cloud credentials
(``x-api-key``, ``Authorization``) belong exclusively to Anthropic Cloud and
must never be merged into local oMLX requests or override
``HEADROOM_OMLX_API_KEY``. Non-credential gateway headers still flow through
``merge_extra_headers`` subject to ``is_trusted_upstream``.
"""
if not extra or not routed_to_omlx:
return extra
filtered = {k: v for k, v in extra.items() if k.lower() not in _ANTHROPIC_CLOUD_AUTH_HEADERS}
return filtered or None
def _build_outbound_anthropic_headers(
stripped_inbound_headers: dict[str, str],
extra_headers: dict[str, str] | None,
*,
routed_to_omlx: bool,
upstream_url: str | None,
config: Any,
) -> dict[str, str]:
"""Build destination-aware outbound headers for Anthropic Cloud or local oMLX."""
from headroom.proxy.helpers import merge_extra_headers
base_headers = dict(stripped_inbound_headers)
if routed_to_omlx:
for cred_key in [k for k in base_headers if k.lower() in _ANTHROPIC_CLOUD_AUTH_HEADERS]:
base_headers.pop(cred_key, None)
omlx_api_key = (
os.environ.get("HEADROOM_OMLX_API_KEY") or os.environ.get("OMLX_API_KEY") or ""
).strip()
if omlx_api_key:
base_headers["x-api-key"] = omlx_api_key
base_headers["authorization"] = f"Bearer {omlx_api_key}"
return merge_extra_headers(
base_headers,
_select_extra_headers_for_destination(extra_headers, routed_to_omlx=routed_to_omlx),
upstream_url=upstream_url,
config=config,
)
def _normalize_omlx_upstream_path(path: str) -> str:
"""Normalize ``/omlx/...`` proxy route prefixes to the upstream Anthropic path."""
if path.startswith("/omlx"):
return path[len("/omlx") :] or "/v1/messages"
return path
def _notify_subscription_tracker_if_cloud(
request: Request,
headers: dict[str, str],
*,
routed_to_omlx: bool,
) -> None:
"""Notify the Anthropic OAuth subscription tracker only on cloud-bound requests."""
if routed_to_omlx:
return
auth_header = next((v for k, v in headers.items() if k.lower() == "authorization"), "")
if auth_header.startswith("Bearer ") and not auth_header.startswith("Bearer sk-ant-api"):
from headroom.subscription.tracker import get_subscription_tracker as _get_sub_tracker
sub_tracker = _get_sub_tracker()
if sub_tracker is not None:
from headroom.subscription.credential_policy import is_local_operator_connection
# Only the local operator's bearer may become the polled
# account; a network caller only marks activity (01-F16).
sub_tracker.notify_active(
auth_header,
from_local_operator=is_local_operator_connection(request),
)
@lru_cache(maxsize=1)
def _thinking_estimator() -> Callable[[str], int] | None:
@@ -1380,17 +1519,27 @@ class AnthropicHandlerMixin:
_strip_internal_headers,
apply_keep_last_turns,
log_outbound_headers,
merge_extra_headers,
)
omlx_target = getattr(
self.config, "omlx_target_api_url", None
) or resolve_omlx_target_api_url(environ=os.environ)
routed_to_omlx = request.url.path.startswith("/omlx") or bool(
upstream_base_url
and omlx_target
and upstream_base_url.rstrip("/") == omlx_target.rstrip("/")
)
_pre_strip_count = sum(1 for k in headers if k.lower().startswith("x-headroom-"))
headers = _strip_internal_headers(headers)
_stripped_inbound_headers = _strip_internal_headers(headers)
# `upstream_base_url` is the per-request `x-headroom-base-url`
# override when the client sent one. These headers are secrets, so
# they only travel to a host the operator designated.
headers = merge_extra_headers(
headers,
# they only travel to a host the operator designated, and cloud
# credentials in `anthropic_extra_headers` are withheld on local oMLX routes.
headers = _build_outbound_anthropic_headers(
_stripped_inbound_headers,
self.config.anthropic_extra_headers,
routed_to_omlx=routed_to_omlx,
upstream_url=upstream_base_url,
config=self.config,
)
@@ -1401,28 +1550,6 @@ class AnthropicHandlerMixin:
request_id=request_id,
)
# Subscription tracker: notify on OAuth requests (not API-key requests)
_auth_header = headers.get("authorization", "")
if _auth_header.startswith("Bearer ") and not _auth_header.startswith(
"Bearer sk-ant-api"
):
from headroom.subscription.tracker import (
get_subscription_tracker as _get_sub_tracker,
)
_sub_tracker = _get_sub_tracker()
if _sub_tracker is not None:
from headroom.subscription.credential_policy import (
is_local_operator_connection,
)
# Only the local operator's bearer may become the polled
# account; a network caller only marks activity (01-F16).
_sub_tracker.notify_active(
_auth_header,
from_local_operator=is_local_operator_connection(request),
)
# Rate limiting
if self.rate_limiter:
# One identity rule for every provider: peer-owned, and
@@ -1499,10 +1626,34 @@ class AnthropicHandlerMixin:
)
from headroom.proxy.route_advice import resolver_for
backend = resolver_for(self).for_request(request, body=body)
passthrough_model = body.get("model") or model_override or "unknown"
if not isinstance(passthrough_model, str):
passthrough_model = "unknown"
passthrough_bytes = original_body_bytes
if not upstream_base_url and omlx_target and is_omlx_local_model(passthrough_model):
upstream_base_url = omlx_target.rstrip("/")
routed_to_omlx = True
headers = _build_outbound_anthropic_headers(
_stripped_inbound_headers,
self.config.anthropic_extra_headers,
routed_to_omlx=True,
upstream_url=upstream_base_url,
config=self.config,
)
if routed_to_omlx and isinstance(body.get("model"), str):
stripped_passthrough_model = strip_omlx_model_prefix(passthrough_model)
if stripped_passthrough_model != passthrough_model:
passthrough_model = stripped_passthrough_model
body["model"] = stripped_passthrough_model
from headroom.proxy.body_forwarding import serialize_body_canonical
passthrough_bytes = serialize_body_canonical(body)
_notify_subscription_tracker_if_cloud(
request, headers, routed_to_omlx=routed_to_omlx
)
backend = (
None if routed_to_omlx else resolver_for(self).for_request(request, body=body)
)
if backend is not None:
if model_override and not body.get("model"):
body["model"] = model_override
@@ -1520,14 +1671,16 @@ class AnthropicHandlerMixin:
)
passthrough_base = upstream_base_url or self.ANTHROPIC_API_URL
passthrough_url = build_anthropic_upstream_url(
passthrough_base, request.url.path, request.url.query
passthrough_base,
_normalize_omlx_upstream_path(request.url.path),
request.url.query,
)
assert self.http_client is not None, "http_client must be initialized"
upstream_request = self.http_client.build_request(
request.method,
passthrough_url,
headers=headers,
content=original_body_bytes,
content=passthrough_bytes,
)
await _finalize_pre_upstream()
return await self.handle_passthrough(
@@ -1559,6 +1712,7 @@ class AnthropicHandlerMixin:
):
body["model"] = model
body_mutation_tracker.mark_mutated("sanitize_model_id")
messages = body.get("messages", [])
# Strip streaming-only "index" keys from request content blocks BEFORE any
# prefix-cache tracking or compression. The proxy's streaming reconstruction
@@ -1668,6 +1822,45 @@ class AnthropicHandlerMixin:
model, messages, body, body_mutation_tracker, _bypass
)
_prev_routed_to_omlx = routed_to_omlx
routed_to_omlx = request.url.path.startswith("/omlx") or bool(
upstream_base_url
and omlx_target
and upstream_base_url.rstrip("/") == omlx_target.rstrip("/")
)
if not upstream_base_url and omlx_target:
is_local = is_omlx_local_model(model)
if is_local:
upstream_base_url = omlx_target.rstrip("/")
routed_to_omlx = True
logger.info(
"[%s] Unified proxy: routing local model '%s' to oMLX: %s",
request_id,
model,
upstream_base_url,
)
if routed_to_omlx != _prev_routed_to_omlx:
headers = _build_outbound_anthropic_headers(
_stripped_inbound_headers,
self.config.anthropic_extra_headers,
routed_to_omlx=routed_to_omlx,
upstream_url=upstream_base_url,
config=self.config,
)
if routed_to_omlx and isinstance(model, str):
stripped_model = strip_omlx_model_prefix(model)
if stripped_model != model:
model = stripped_model
body["model"] = stripped_model
body_mutation_tracker.mark_mutated("omlx_strip_model_prefix")
_notify_subscription_tracker_if_cloud(request, headers, routed_to_omlx=routed_to_omlx)
_auth_header = (
headers.get("authorization")
or headers.get("Authorization")
or request.headers.get("authorization")
or ""
)
# NOTE: Upstream temporarily disabled broad image compression due to
# token-counting inaccuracies. We only compress the latest non-frozen
# user turn later in this handler to preserve Anthropic prefix caching.
@@ -4323,9 +4516,12 @@ class AnthropicHandlerMixin:
# every one of those turns to "anthropic" on the dashboard.
# For a non-Copilot base the builder only joins base + path, so the
# URL itself is unchanged.
request_path = request.url.path
if request_path.startswith("/omlx"):
request_path = request_path[len("/omlx") :] or "/v1/messages"
url = build_anthropic_upstream_url(
upstream_base_url or self.ANTHROPIC_API_URL,
request.url.path,
request_path,
request.url.query,
)
# Keep a conversation on one replica of providers that route on a
@@ -5444,8 +5640,10 @@ class AnthropicHandlerMixin:
# counters. Provider-specific OAuth/subscription
# accounting — stays outside the funnel (different
# concern, only fires for Bearer-not-sk-ant tokens).
if _auth_header.startswith("Bearer ") and not _auth_header.startswith(
"Bearer sk-ant-api"
if (
not routed_to_omlx
and _auth_header.startswith("Bearer ")
and not _auth_header.startswith("Bearer sk-ant-api")
):
from headroom.subscription.tracker import (
get_subscription_tracker as _get_sub_tracker,
+6
View File
@@ -12,6 +12,7 @@ from dataclasses import InitVar, dataclass, field
from datetime import datetime
from typing import Any, Literal
from headroom.config import resolve_omlx_target_api_url
from headroom.memory import qdrant_env
from headroom.providers.registry import ProviderApiOverrides
from headroom.proxy.buffered_ccr_response import DEFAULT_BUFFERED_CCR_GRACE_SECONDS
@@ -164,6 +165,9 @@ class ProxyConfig:
gemini_api_url: str | None = None # Custom Gemini API URL override
cloudcode_api_url: str | None = None # Custom Cloud Code Assist API URL override
vertex_api_url: str | None = None # Custom Vertex AI regional API URL override
omlx_target_api_url: str | None = (
None # Custom oMLX local inference API URL (e.g. http://127.0.0.1:8888)
)
# Extra headers merged into (and overriding) forwarded Anthropic/OpenAI requests.
# JSON-object config knobs; see settings_store's anthropic_extra_headers/
# openai_extra_headers and providers.registry.resolve_extra_headers.
@@ -587,6 +591,8 @@ class ProxyConfig:
def __post_init__(self, smart_routing: bool | None = None) -> None:
if self.rollout is None:
self.rollout = resolve_rollout()
if self.omlx_target_api_url is None:
self.omlx_target_api_url = resolve_omlx_target_api_url()
# ``read_maturation`` remains a concrete, already-resolved runtime
# setting for programmatic/config-file callers. The CLI composition
# root derives it from this same snapshot before constructing the
+9
View File
@@ -84,6 +84,7 @@ from headroom.config import (
DEFAULT_EXCLUDE_TOOLS,
CacheAlignerConfig,
ReadLifecycleConfig,
resolve_omlx_target_api_url,
)
from headroom.dashboard import get_dashboard_html
from headroom.log_safety import describe_exception, safe_id
@@ -6608,6 +6609,7 @@ def _proxy_config_from_env() -> ProxyConfig:
),
ccr_buffered_chat_streaming=_get_env_bool("HEADROOM_CCR_BUFFERED_CHAT_STREAMING", False),
vertex_api_url=os.environ.get("VERTEX_TARGET_API_URL"),
omlx_target_api_url=resolve_omlx_target_api_url(environ=os.environ),
backend=_get_env_str("HEADROOM_BACKEND", "anthropic"),
bedrock_region=_get_env_str("HEADROOM_BEDROCK_REGION", "us-west-2"),
bedrock_profile=os.environ.get("AWS_PROFILE"),
@@ -7112,6 +7114,10 @@ if __name__ == "__main__":
"--vertex-api-url",
help=f"Custom Vertex AI regional API URL (default: {DEFAULT_VERTEX_API_URL})",
)
parser.add_argument(
"--omlx-api-url",
help="Custom local oMLX API URL (env: HEADROOM_OMLX_BASE_URL or OMLX_TARGET_API_URL)",
)
# Backend (anthropic direct, bedrock, openrouter, anyllm, or litellm-<provider>)
parser.add_argument(
@@ -7433,6 +7439,9 @@ if __name__ == "__main__":
min_value=0,
),
vertex_api_url=_get_env_str("VERTEX_TARGET_API_URL", args.vertex_api_url),
omlx_target_api_url=resolve_omlx_target_api_url(
getattr(args, "omlx_api_url", None), os.environ
),
# Backend settings
backend=_get_env_str("HEADROOM_BACKEND", args.backend), # type: ignore[arg-type]
bedrock_region=_get_env_str("HEADROOM_BEDROCK_REGION", args.bedrock_region),
+6
View File
@@ -38,6 +38,8 @@ import os
from typing import Any
from urllib.parse import urlparse
from headroom.config import resolve_omlx_target_api_url
logger = logging.getLogger("headroom.proxy")
ALLOWED_HOSTS_ENV = "HEADROOM_UPSTREAM_ALLOWED_HOSTS"
@@ -50,6 +52,7 @@ _API_URL_ATTRS = (
"cloudcode_api_url",
"vertex_api_url",
"bedrock_api_url",
"omlx_target_api_url",
)
# Hosts that are always operator-designated: they are what the provider targets
@@ -106,6 +109,9 @@ def trusted_upstream_hosts(config: Any = None) -> frozenset[str]:
host = url_host(getattr(config, attr, None))
if host:
hosts.add(host)
omlx_env_host = url_host(resolve_omlx_target_api_url(environ=os.environ))
if omlx_env_host:
hosts.add(omlx_env_host)
hosts |= _env_allowed_hosts()
return frozenset(hosts)
+461
View File
@@ -0,0 +1,461 @@
"""Unit tests for hybrid Anthropic Cloud + local oMLX proxy routing (PR #4597)."""
from __future__ import annotations
import json
from unittest.mock import AsyncMock, MagicMock
import httpx
import pytest
from click.testing import CliRunner
from fastapi.testclient import TestClient
from headroom.config import HeadroomConfig, resolve_omlx_target_api_url
from headroom.providers.anthropic import ANTHROPIC_CONTEXT_LIMITS, ANTHROPIC_PRICING
from headroom.proxy.handlers.anthropic import is_omlx_local_model, strip_omlx_model_prefix
from headroom.proxy.model_router import ModelRoute, ModelRouterConfig
from headroom.proxy.server import ProxyConfig, _proxy_config_from_env, create_app
from headroom.proxy.upstream_trust import is_trusted_upstream, trusted_upstream_hosts
MESSAGES = "/v1/messages"
OMLX_MESSAGES = "/omlx/v1/messages"
OMLX_URL = "http://127.0.0.1:8888"
def _install_fake_client(proxy) -> MagicMock:
"""Replace proxy.http_client so forwarding never touches the network."""
response = httpx.Response(
200,
json={
"id": "msg_test",
"type": "message",
"role": "assistant",
"content": [{"type": "text", "text": "ok"}],
"model": "test",
"stop_reason": "end_turn",
"usage": {"input_tokens": 10, "output_tokens": 2},
},
request=httpx.Request("POST", "http://upstream/v1/messages"),
)
client = MagicMock()
client.post = AsyncMock(return_value=response)
client.request = AsyncMock(return_value=response)
client.send = AsyncMock(return_value=response)
client.build_request = MagicMock(
return_value=httpx.Request("POST", "http://upstream/v1/messages", content=b"{}")
)
client.aclose = AsyncMock()
proxy.http_client = client
return client
def _make_config(**overrides) -> ProxyConfig:
base = {
"optimize": False,
"image_optimize": False,
"cache_enabled": False,
"rate_limit_enabled": False,
"cost_tracking_enabled": False,
"ccr_inject_tool": False,
"ccr_handle_responses": False,
"ccr_context_tracking": False,
"mode": "token",
}
base.update(overrides)
return ProxyConfig(**base)
def _forwarded_url(http: MagicMock) -> str:
return str(http.post.call_args.args[0])
def _forwarded_headers(http: MagicMock) -> dict[str, str]:
raw = http.post.call_args.kwargs.get("headers") or {}
return {k.lower(): v for k, v in raw.items()}
def _forwarded_body(http: MagicMock) -> dict:
content = http.post.call_args.kwargs["content"]
return json.loads(content)
@pytest.mark.parametrize(
"claude_model",
[
"claude-haiku-4-5-20251001",
"claude-haiku-5-5",
"claude-fable-5",
"claude-fable-5-1",
"claude-sonnet-4-5",
"claude-sonnet-5",
"claude-opus-4-8",
*sorted(set(ANTHROPIC_CONTEXT_LIMITS) | set(ANTHROPIC_PRICING)),
],
)
def test_claude_models_route_to_anthropic_cloud_even_with_omlx_configured(
monkeypatch: pytest.MonkeyPatch,
claude_model: str,
) -> None:
monkeypatch.setenv("HEADROOM_OMLX_BASE_URL", OMLX_URL)
monkeypatch.setenv("OMLX_TARGET_API_URL", OMLX_URL)
assert not is_omlx_local_model(claude_model)
app = create_app(_make_config(omlx_target_api_url=OMLX_URL))
with TestClient(app) as client:
http = _install_fake_client(client.app.state.proxy)
resp = client.post(
MESSAGES,
json={
"model": claude_model,
"max_tokens": 16,
"messages": [{"role": "user", "content": "hello"}],
},
headers={"x-api-key": "sk-ant-cloud-secret"},
)
assert resp.status_code == 200
assert _forwarded_url(http) == "https://api.anthropic.com/v1/messages"
assert _forwarded_headers(http).get("x-api-key") == "sk-ant-cloud-secret"
@pytest.mark.parametrize(
("local_model", "expected_wire_model"),
[
("omlx/qwen3-8b", "qwen3-8b"),
("local/gemma-4-9b", "gemma-4-9b"),
("gemma-4-9b", "gemma-4-9b"),
("qwen2.5-coder", "qwen2.5-coder"),
],
)
def test_explicit_local_models_route_to_omlx_target(
monkeypatch: pytest.MonkeyPatch,
local_model: str,
expected_wire_model: str,
) -> None:
monkeypatch.delenv("HEADROOM_OMLX_API_KEY", raising=False)
monkeypatch.delenv("OMLX_API_KEY", raising=False)
assert is_omlx_local_model(local_model)
assert strip_omlx_model_prefix(local_model) == expected_wire_model
app = create_app(_make_config(omlx_target_api_url=OMLX_URL))
with TestClient(app) as client:
http = _install_fake_client(client.app.state.proxy)
resp = client.post(
MESSAGES,
json={
"model": local_model,
"max_tokens": 16,
"messages": [{"role": "user", "content": "hello"}],
},
)
assert resp.status_code == 200
assert _forwarded_url(http) == f"{OMLX_URL}/v1/messages"
assert _forwarded_body(http)["model"] == expected_wire_model
def test_allowlist_env_routes_custom_model_to_omlx(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("HEADROOM_OMLX_MODELS", "custom-apple-silicon-7b, another-local")
assert is_omlx_local_model("custom-apple-silicon-7b")
assert not is_omlx_local_model("unlisted-cloud-model")
def test_maybe_route_model_runs_before_local_or_cloud_target_selection() -> None:
# 1. Cloud -> local rewrite must route to omlx_target and strip omlx/ prefix.
cloud_to_local_cfg = _make_config(
omlx_target_api_url=OMLX_URL,
model_router=ModelRouterConfig(
enabled=True,
routes=(
ModelRoute(
to_model="omlx/qwen3-8b",
max_input_tokens=100_000,
require_no_tools=True,
name="offload-local",
),
),
),
)
app = create_app(cloud_to_local_cfg)
with TestClient(app) as client:
http = _install_fake_client(client.app.state.proxy)
resp = client.post(
MESSAGES,
json={
"model": "claude-sonnet-4-5",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
headers={"x-api-key": "sk-ant-cloud-secret"},
)
assert resp.status_code == 200
assert _forwarded_url(http) == f"{OMLX_URL}/v1/messages"
assert _forwarded_body(http)["model"] == "qwen3-8b"
assert "x-api-key" not in _forwarded_headers(http)
# 2. Local -> cloud rewrite must route to Anthropic cloud and keep cloud credentials.
local_to_cloud_cfg = _make_config(
omlx_target_api_url=OMLX_URL,
model_router=ModelRouterConfig(
enabled=True,
routes=(
ModelRoute(
to_model="claude-haiku-4-5-20251001",
max_input_tokens=100_000,
require_no_tools=True,
name="escalate-cloud",
),
),
),
)
app2 = create_app(local_to_cloud_cfg)
with TestClient(app2) as client:
http2 = _install_fake_client(client.app.state.proxy)
resp2 = client.post(
MESSAGES,
json={
"model": "omlx/qwen3-8b",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
headers={"x-api-key": "sk-ant-cloud-secret"},
)
assert resp2.status_code == 200
assert _forwarded_url(http2) == "https://api.anthropic.com/v1/messages"
assert _forwarded_body(http2)["model"] == "claude-haiku-4-5-20251001"
assert _forwarded_headers(http2).get("x-api-key") == "sk-ant-cloud-secret"
def test_cloud_credentials_stripped_or_replaced_when_forwarding_to_omlx(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("HEADROOM_OMLX_API_KEY", raising=False)
monkeypatch.delenv("OMLX_API_KEY", raising=False)
app = create_app(_make_config(omlx_target_api_url=OMLX_URL))
with TestClient(app) as client:
http = _install_fake_client(client.app.state.proxy)
resp = client.post(
MESSAGES,
json={
"model": "omlx/qwen3-8b",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
headers={
"x-api-key": "sk-ant-api03-secret",
"Authorization": "Bearer sk-ant-oat01-secret",
},
)
assert resp.status_code == 200
forwarded_headers = _forwarded_headers(http)
assert "x-api-key" not in forwarded_headers
assert "authorization" not in forwarded_headers
# When HEADROOM_OMLX_API_KEY is configured, replace with the local key.
monkeypatch.setenv("HEADROOM_OMLX_API_KEY", "local-omlx-token")
with TestClient(app) as client:
http2 = _install_fake_client(client.app.state.proxy)
resp2 = client.post(
OMLX_MESSAGES,
json={
"model": "omlx/qwen3-8b",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
headers={
"x-api-key": "sk-ant-api03-secret",
"Authorization": "Bearer sk-ant-oat01-secret",
},
)
assert resp2.status_code == 200
forwarded_headers2 = _forwarded_headers(http2)
assert forwarded_headers2.get("x-api-key") == "local-omlx-token"
assert forwarded_headers2.get("authorization") == "Bearer local-omlx-token"
def test_headroom_omlx_base_url_and_omlx_target_api_url_env_consistency(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("OMLX_TARGET_API_URL", raising=False)
monkeypatch.delenv("HEADROOM_OMLX_URL", raising=False)
monkeypatch.setenv("HEADROOM_OMLX_BASE_URL", "http://127.0.0.1:8000/")
assert resolve_omlx_target_api_url() == "http://127.0.0.1:8000"
assert HeadroomConfig().omlx_target_api_url == "http://127.0.0.1:8000"
assert ProxyConfig().omlx_target_api_url == "http://127.0.0.1:8000"
assert _proxy_config_from_env().omlx_target_api_url == "http://127.0.0.1:8000"
from headroom.cli.proxy import proxy as proxy_cli_cmd
runner = CliRunner()
help_res = runner.invoke(proxy_cli_cmd, ["--help"])
assert help_res.exit_code == 0
assert "--omlx-api-url" in help_res.output
def test_omlx_target_api_url_included_in_trusted_upstream_hosts(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("OMLX_TARGET_API_URL", raising=False)
monkeypatch.delenv("HEADROOM_OMLX_BASE_URL", raising=False)
monkeypatch.delenv("HEADROOM_OMLX_URL", raising=False)
cfg = _make_config(
omlx_target_api_url="http://omlx.internal:8888",
anthropic_extra_headers={"X-Custom-Gateway-Auth": "gateway-secret"},
)
assert "omlx.internal" in trusted_upstream_hosts(cfg)
assert is_trusted_upstream("http://omlx.internal:8888/v1/messages", cfg)
app = create_app(cfg)
with TestClient(app) as client:
http = _install_fake_client(client.app.state.proxy)
resp = client.post(
MESSAGES,
json={
"model": "omlx/qwen3-8b",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
)
assert resp.status_code == 200
assert _forwarded_headers(http).get("x-custom-gateway-auth") == "gateway-secret"
@pytest.mark.parametrize("endpoint", [MESSAGES, OMLX_MESSAGES])
@pytest.mark.parametrize(
("extra_header_name", "extra_header_value"),
[
("x-api-key", "dummy-cloud-secret"),
("Authorization", "Bearer dummy-cloud-oauth"),
],
)
@pytest.mark.parametrize("local_omlx_key", [None, "local-omlx-token"])
def test_configured_anthropic_extra_headers_never_leak_or_override_on_omlx_routes(
monkeypatch: pytest.MonkeyPatch,
endpoint: str,
extra_header_name: str,
extra_header_value: str,
local_omlx_key: str | None,
) -> None:
"""Cover all 8 destination-aware credential isolation combinations (PR #4597 review)."""
monkeypatch.delenv("OMLX_API_KEY", raising=False)
if local_omlx_key is None:
monkeypatch.delenv("HEADROOM_OMLX_API_KEY", raising=False)
else:
monkeypatch.setenv("HEADROOM_OMLX_API_KEY", local_omlx_key)
cfg = _make_config(
omlx_target_api_url=OMLX_URL,
anthropic_extra_headers={
extra_header_name: extra_header_value,
"X-Custom-Gateway-Auth": "gateway-secret",
},
)
app = create_app(cfg)
with TestClient(app) as client:
http = _install_fake_client(client.app.state.proxy)
resp = client.post(
endpoint,
json={
"model": "omlx/qwen3-8b",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
headers={
"x-api-key": "inbound-cloud-secret",
"Authorization": "Bearer inbound-cloud-oauth",
},
)
assert resp.status_code == 200
assert _forwarded_url(http) == f"{OMLX_URL}/v1/messages"
assert _forwarded_body(http)["model"] == "qwen3-8b"
forwarded_headers = _forwarded_headers(http)
assert forwarded_headers.get("x-custom-gateway-auth") == "gateway-secret"
if local_omlx_key is None:
assert "x-api-key" not in forwarded_headers
assert "authorization" not in forwarded_headers
else:
assert forwarded_headers.get("x-api-key") == local_omlx_key
assert forwarded_headers.get("authorization") == f"Bearer {local_omlx_key}"
@pytest.mark.parametrize(
("extra_header_name", "extra_header_value", "expected_lower_key"),
[
("x-api-key", "configured-cloud-api-key", "x-api-key"),
("Authorization", "Bearer configured-cloud-oauth", "authorization"),
],
)
def test_cloud_requests_retain_configured_anthropic_extra_headers_with_omlx_enabled(
monkeypatch: pytest.MonkeyPatch,
extra_header_name: str,
extra_header_value: str,
expected_lower_key: str,
) -> None:
monkeypatch.setenv("HEADROOM_OMLX_API_KEY", "local-omlx-token")
cfg = _make_config(
omlx_target_api_url=OMLX_URL,
anthropic_extra_headers={extra_header_name: extra_header_value},
)
app = create_app(cfg)
with TestClient(app) as client:
http = _install_fake_client(client.app.state.proxy)
resp = client.post(
MESSAGES,
json={
"model": "claude-haiku-4-5-20251001",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hello"}],
},
headers={extra_header_name: "inbound-overridden-secret"},
)
assert resp.status_code == 200
assert _forwarded_url(http) == "https://api.anthropic.com/v1/messages"
assert _forwarded_headers(http).get(expected_lower_key) == extra_header_value
def test_subscription_tracker_skips_omlx_routes_and_notifies_cloud_routes(
monkeypatch: pytest.MonkeyPatch,
) -> None:
import headroom.subscription.tracker as sub_tracker_mod
fake_tracker = MagicMock()
monkeypatch.setattr(sub_tracker_mod, "get_subscription_tracker", lambda: fake_tracker)
monkeypatch.delenv("HEADROOM_OMLX_API_KEY", raising=False)
monkeypatch.delenv("OMLX_API_KEY", raising=False)
cfg = _make_config(
omlx_target_api_url=OMLX_URL,
anthropic_extra_headers={"Authorization": "Bearer configured-cloud-oauth"},
)
app = create_app(cfg)
with TestClient(app) as client:
_install_fake_client(client.app.state.proxy)
for endpoint in (MESSAGES, OMLX_MESSAGES):
resp = client.post(
endpoint,
json={
"model": "omlx/qwen3-8b",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
headers={"Authorization": "Bearer inbound-cloud-oauth"},
)
assert resp.status_code == 200
assert fake_tracker.notify_active.call_count == 0
cloud_resp = client.post(
MESSAGES,
json={
"model": "claude-haiku-4-5-20251001",
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
},
)
assert cloud_resp.status_code == 200
assert fake_tracker.notify_active.call_count == 1
assert fake_tracker.notify_active.call_args.args[0] == "Bearer configured-cloud-oauth"