fix(grok): route CLI traffic to api.x.ai on shared proxies (#2693)

## Description

`headroom wrap grok` points the Grok CLI at the local proxy via
`GROK_MODELS_BASE_URL` and, when it starts the proxy, sets
`openai_api_url=https://api.x.ai`. When Claude/Codex already own port
8787, wrap reuses that process without restart (other wrappers
attached), so process-wide `OPENAI_API_URL` stays
`https://api.openai.com`.

Grok CLI cannot stamp `x-headroom-base-url` (no custom attribution
headers). Inference then goes to OpenAI with xAI session tokens and
returns 401. Observed on the wire:

- `x-xai-token-auth: xai-grok-cli`
- `user-agent: grok-pager/... grok-shell/...`
- outbound `POST https://api.openai.com/v1/chat/completions` status 401

This PR adds per-request Grok CLI detection and routes chat / models /
responses to `https://api.x.ai` when those signals are present and the
OpenAI target is still its default, without restarting the shared proxy
or changing Claude/Codex paths. An operator-configured gateway is never
bypassed.

Not covered by #2626 (attribution / UA map only when custom base is
already `api.x.ai`).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `is_grok_cli_request()` in `providers/grok/runtime.py` using
narrow wire signals: exact `x-xai-token-auth: xai-grok-cli` and the
`grok-pager/` / `grok-shell/` UA product tokens, compared per
whitespace-split token so wrappers like `litellm-grok/1.0` cannot match
- Resolve OpenAI chat and responses final upstream as:
`x-headroom-base-url` then Grok CLI then process `OPENAI_API_URL`
- Route `GET /v1/models` and OpenAI-classified passthrough through the
same Grok to xAI base
- Add unit tests for detector, resolver priority, and passthrough/models
base selection
- Add `tests/test_grok_shared_proxy_call_sites.py`, which drives
`handle_openai_chat` and `handle_openai_responses` against a stubbed
transport and asserts the outbound URL — reverting either call site to
the old `custom_only or OPENAI_API_URL` formula turns these red
- Gate Grok to xAI routing on the OpenAI target still being its default,
via one shared `route_grok_to_xai()` predicate used by both the
chat/responses resolver and the models/passthrough path, so an
operator-configured gateway is never bypassed

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` / `ruff format`)
- [x] New tests added for new functionality when applicable
- [x] Manual testing performed

### Test Output

```text
# Grok routing: call sites, detector, resolver priority, passthrough/models
uv run pytest tests/test_grok_shared_proxy_call_sites.py \
  tests/test_provider_grok.py tests/test_provider_proxy_targets.py \
  tests/test_proxy/test_openai_upstream_header.py \
  tests/test_openai_codex_routing.py -q
============================== 48 passed in 0.46s ==============================

uv run ruff check .
All checks passed!

uv run ruff format --check .
1331 files already formatted

uv run mypy headroom --ignore-missing-imports
Success: no issues found in 506 source files

# Full suite (uv, Python 3.14, tests + scripts/tests) — honesty note
==== 15 failed, 9480 passed, 592 skipped in 553.64s ====
# Replaying those 15 as a set gives an identical 11 failed / 4 passed at the
# merge base, at HEAD~1 and at HEAD, so this branch introduces no failure.
# Causes are local-env: missing cargo, missing litellm extra, Windows-only
# asyncio path, AF_UNIX path length. The 4 test_proxy_health.py kompress tests
# pass in isolation at every ref and fail only under the full suite
# (pre-existing ordering pollution; the new test file is not the cause --
# it runs clean immediately before test_proxy_health.py).
# tests/test_memory_eval.py excluded: it imports litellm at collection time.
# Fork CI pytest not yet run (action_required until maintainer approves).
```

## Real Behavior Proof

- Environment: macOS arm64; Grok CLI 0.2.117; PR branch
`fix/grok-shared-proxy-xai-routing` (0.33.1-dev) running as the only
proxy on 8787 with `openai_api_url=null` and `--backend anthropic` (same
shared-proxy shape as Claude/Codex-started 8787). Stock Headroom 0.33.0
was first used to capture the before failure.
- Exact command / steps: 1) Before: with stock 0.33.0 on 8787, run
`headroom wrap grok` (or attach via `GROK_MODELS_BASE_URL`) and inspect
`~/.headroom/logs/proxy.log`. 2) After: replace the process with this
branch (`uv run python -m headroom.cli proxy --host 127.0.0.1 --port
8787 --backend anthropic`, no OpenAI target override). 3)
`GROK_MODELS_BASE_URL=http://127.0.0.1:8787/v1 grok -p "Reply with
exactly one word: pong" --always-approve`. 4) Confirm health still shows
`openai_api_url: null` and inspect proxy log outbound lines.
- Observed result: Before (stock): Grok chat with `x-xai-token-auth:
xai-grok-cli` forwarded to `https://api.openai.com/v1/chat/completions`
→ 401. After (this branch): same Grok CLI on the same port; `/v1/models`
for `user-agent: grok-shell/0.2.117` returned 200; chat outbound was
`POST path=https://api.x.ai/v1/chat/completions` with status 200; CLI
printed `pong` and exited 0. Codex models list against the same proxy
still returned 200.
- Not tested: multi-day soak with Claude + Codex + Grok all attached for
hours; production PyPI release build; full-suite green under fork CI
(local full run had 7 env-only failures unrelated to this change);
savings attribution still labels provider as openai without custom base
(orthogonal, #2626).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title

## Screenshots (if applicable)

N/A — evidence is CLI output and proxy logs (redacted auth).

## Additional Notes

- Explicitly out of scope: savings attribution as `provider=xai` without
custom base (see #2626), multi-provider matrix for Kimi/Mistral,
hard-coded per-agent ports.
- ChatGPT Codex auth still wins on `/v1/responses` before this chain
runs.
- Priority: `x-headroom-base-url` > Grok CLI signals (only while
`OPENAI_API_URL` is still the default) > process OpenAI default. The
gate matters because an operator who points the proxy at a gateway
(LiteLLM, Azure, self-hosted vLLM) chose it for every OpenAI-compatible
client: a client User-Agent must not silently bypass it, nor carry that
gateway's `OPENAI_TARGET_API_HEADERS` to a different vendor. The
shared-proxy case this PR fixes runs with `openai_api_url: null`, so it
is unaffected.
- Fork PR workflows may show `action_required` until a maintainer
approves first-time external CI runs.
## Runtime Rollout Safety

- Rollout-managed feature(s): none; routing is selected per request from
narrow Grok CLI wire signals.
- Minimum rollout channel: normal stable release after focused routing
and credential-isolation coverage passes.
- Stable/default behavior changed: only Grok-fingerprinted requests on a
proxy whose OpenAI target remains the default route to api.x.ai;
explicit client base URLs and operator-configured gateways retain
priority.
- Kill switch / disable path: use an explicit non-default OpenAI
gateway/target, or human-revert this routing change.
- Unsafe override required: none.
- Qualification impact: Grok chat, Responses, models, passthrough, and
cross-vendor header-isolation regressions must remain green.
- Rollback path: human-revert the PR; no persisted state or migration is
involved.

---------

Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
This commit is contained in:
Peter
2026-10-01 00:11:46 -05:00
committed by GitHub
co-authored by Jerrett Davis
parent 81a8a28da3
commit ef1c528e92
10 changed files with 652 additions and 32 deletions
+14 -2
View File
@@ -1,5 +1,17 @@
"""Grok CLI provider helpers."""
from .runtime import DEFAULT_API_URL, PROXY_ENV_KEY, build_launch_env, proxy_base_url
from .runtime import (
DEFAULT_API_URL,
PROXY_ENV_KEY,
build_launch_env,
is_grok_cli_request,
proxy_base_url,
)
__all__ = ["DEFAULT_API_URL", "PROXY_ENV_KEY", "build_launch_env", "proxy_base_url"]
__all__ = [
"DEFAULT_API_URL",
"PROXY_ENV_KEY",
"build_launch_env",
"is_grok_cli_request",
"proxy_base_url",
]
+37
View File
@@ -10,6 +10,43 @@ from headroom.proxy.project_context import with_project_prefix
DEFAULT_API_URL = "https://api.x.ai"
PROXY_ENV_KEY = "GROK_MODELS_BASE_URL"
# Official Grok CLI / Grok Build stamps this on inference requests (observed on
# grok-shell 0.2.x). Used for per-request xAI routing when the shared proxy's
# process-wide OPENAI target is still api.openai.com (Claude/Codex-started).
_XAI_TOKEN_AUTH_HEADER = "x-xai-token-auth"
_XAI_TOKEN_AUTH_VALUE = "xai-grok-cli"
# UA product tokens emitted by Grok CLI 0.2.x. Matched against whitespace-split
# tokens so unrelated clients ("litellm-grok/1.0") cannot collide.
_GROK_UA_PREFIXES = ("grok-pager/", "grok-shell/")
def _header_value(headers: Mapping[str, str], name: str) -> str | None:
"""Case-insensitive header lookup for plain mappings and Starlette Headers."""
lowered = name.lower()
for key, value in headers.items():
if key.lower() == lowered:
return value
return None
def is_grok_cli_request(headers: Mapping[str, str]) -> bool:
"""Return True when inbound headers identify the official Grok CLI.
Grok cannot stamp ``x-headroom-base-url`` (no custom attribution headers),
so shared-proxy routing must recognize the CLI from wire signals instead.
Detection is intentionally narrow: only the official token-auth marker and
known Grok UA product tokens (prefix match on whitespace-split tokens) —
never model-id heuristics.
"""
token_auth = _header_value(headers, _XAI_TOKEN_AUTH_HEADER)
if token_auth is not None and token_auth.strip().lower() == _XAI_TOKEN_AUTH_VALUE:
return True
user_agent = _header_value(headers, "user-agent")
if not user_agent:
return False
return any(token.startswith(_GROK_UA_PREFIXES) for token in user_agent.lower().split())
def proxy_base_url(port: int) -> str:
"""Return the local proxy base URL used by Grok CLI integrations."""
+19 -4
View File
@@ -39,6 +39,9 @@ from headroom.providers.openai_responses import (
from headroom.providers.proxy_targets import (
api_target as _api_target,
)
from headroom.providers.proxy_targets import (
openai_compatible_base_url as _openai_compatible_base_url,
)
from headroom.providers.proxy_targets import (
select_passthrough_base_url as _select_passthrough_base_url,
)
@@ -490,23 +493,35 @@ def register_provider_routes(app: FastAPI, proxy: Any) -> None:
@app.get("/v1/models")
async def list_models(request: Request):
provider_name = proxy.provider_runtime.model_metadata_provider(dict(request.headers))
headers = dict(request.headers)
provider_name = proxy.provider_runtime.model_metadata_provider(headers)
base_url = (
_openai_compatible_base_url(proxy, headers)
if provider_name == "openai"
else _api_target(proxy, provider_name)
)
return await handle_model_metadata_endpoint(
proxy,
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url=_api_target(proxy, provider_name),
provider_api_base_url=base_url,
provider_name=provider_name,
)
@app.get("/v1/models/{model_id}")
async def get_model(request: Request, model_id: str):
provider_name = proxy.provider_runtime.model_metadata_provider(dict(request.headers))
headers = dict(request.headers)
provider_name = proxy.provider_runtime.model_metadata_provider(headers)
base_url = (
_openai_compatible_base_url(proxy, headers)
if provider_name == "openai"
else _api_target(proxy, provider_name)
)
return await handle_model_metadata_endpoint(
proxy,
request,
endpoint=model_metadata_get_endpoint(model_id),
provider_api_base_url=_api_target(proxy, provider_name),
provider_api_base_url=base_url,
provider_name=provider_name,
)
+46
View File
@@ -13,6 +13,9 @@ from headroom.copilot_auth import (
)
from headroom.providers.codex import resolve_codex_routing
from headroom.providers.codex.endpoints import CHATGPT_BACKEND_API_URL
from headroom.providers.codex.runtime import DEFAULT_API_URL as DEFAULT_OPENAI_API_URL
from headroom.providers.grok.runtime import DEFAULT_API_URL as XAI_API_URL
from headroom.providers.grok.runtime import is_grok_cli_request
from headroom.providers.vertex import vertex_target_for_location as _vertex_target_for_location
from headroom.proxy.upstream_guard import is_safe_upstream_url
@@ -39,6 +42,43 @@ def vertex_target_for_location(proxy: Any, location: str) -> str:
logger = logging.getLogger("headroom.proxy")
def route_grok_to_xai(headers: Mapping[str, str], openai_target: str) -> bool:
"""Return True when Grok CLI traffic should be redirected to ``api.x.ai``.
Grok CLI cannot set ``x-headroom-base-url``, so a shared proxy started for
Claude/Codex has to recognize it from wire signals or it forwards xAI
session tokens to ``api.openai.com``.
Only applies while the OpenAI target is still the default. An operator who
pointed the proxy at a gateway (LiteLLM, Azure, self-hosted vLLM) chose it
for every OpenAI-compatible client; a client User-Agent must not silently
bypass that.
This gate is URL policy only. It does not keep operator-configured
``OPENAI_TARGET_API_HEADERS`` away from xAI — those extras are configured
independently of the target URL, so a default-URL proxy can still redirect
here. The direct OpenAI HTTP handlers enforce credential isolation
separately by suppressing configured extras when their OpenAI-compatible
upstream candidate is the xAI host. Configured backend transports retain
their existing header policy.
"""
if not is_grok_cli_request(headers):
return False
return openai_target.rstrip("/") == DEFAULT_OPENAI_API_URL
def openai_compatible_base_url(proxy: Any, headers: Mapping[str, str]) -> str:
"""Resolve upstream for OpenAI-compatible metadata/passthrough traffic.
Routes official Grok CLI to ``api.x.ai`` so ``GET /v1/models`` and catch-all
passthrough succeed on a shared proxy whose OpenAI target is the default.
"""
target = api_target(proxy, "openai")
if route_grok_to_xai(headers, target):
return XAI_API_URL
return target
def select_passthrough_base_url(
proxy: Any, headers: Mapping[str, str], path: str | None = None
) -> str:
@@ -95,4 +135,10 @@ def select_passthrough_base_url(
# extension sends none of them — so a request that took one of those
# branches is not Copilot's and keeps the upstream it asked for.
return copilot_completions_base_url()
if provider_name == "openai":
# Grok CLI reaches passthrough the same way, recognized by wire signals
# rather than path, and only while the OpenAI target is still the
# default. Checked after Copilot because that branch keys on a path only
# Copilot emits, so it is the narrower claim on this fall-through.
return openai_compatible_base_url(proxy, headers)
return target
+87 -22
View File
@@ -75,6 +75,8 @@ from headroom.providers.codex.runtime import (
resolve_codex_routing_headers as _resolve_codex_routing_headers,
)
from headroom.providers.copilot import model_prefers_responses_api
from headroom.providers.grok.runtime import DEFAULT_API_URL as XAI_API_URL
from headroom.providers.proxy_targets import route_grok_to_xai
from headroom.proxy.auth_mode import (
classify_auth_mode,
classify_client,
@@ -489,6 +491,26 @@ def _append_request_query(url: str, query: str) -> str:
return f"{url}{separator}{query}"
def _xai_hostname(url: str) -> str | None:
"""Return ``url``'s hostname with any fully-qualified trailing dot removed.
``https://api.x.ai.`` resolves to the same host as ``https://api.x.ai`` but
``urlparse`` reports a distinct hostname, so a client-supplied
``x-headroom-base-url`` could otherwise slip past the comparison below.
"""
hostname = urlparse(url).hostname
return hostname.rstrip(".") if hostname else hostname
def _is_xai_upstream(upstream_base_url: str) -> bool:
"""Return whether the selected upstream is the official xAI API host.
Compares the parsed hostname only: a path, scheme or port variation of
``api.x.ai`` is still xAI and must not receive OpenAI-side credentials.
"""
return _xai_hostname(upstream_base_url) == _xai_hostname(XAI_API_URL)
def _normalize_origin(origin: str) -> str | None:
parsed = urlparse(origin.strip())
if not parsed.scheme or not parsed.hostname:
@@ -1996,10 +2018,36 @@ class OpenAIHandlerMixin:
Honors the ``x-headroom-base-url`` request header so OpenAI-compatible
gateways (LiteLLM, CPA, self-hosted vLLM, Azure OpenAI) route through
the dedicated ``/v1/chat/completions`` and ``/v1/responses`` handlers,
not just the generic passthrough route that already honors it. Falls
back to the configured ``OPENAI_API_URL`` (``OPENAI_TARGET_API_URL``).
not just the generic passthrough route that already honors it.
When the header is absent, official Grok CLI requests (identified by
``x-xai-token-auth`` / Grok UA tokens) route to ``api.x.ai`` so a
shared proxy started for Claude/Codex does not forward Grok session
tokens to ``api.openai.com`` — but only while ``OPENAI_API_URL`` is
still the default, so a configured gateway is never bypassed.
Otherwise falls back to the configured ``OPENAI_API_URL``
(``OPENAI_TARGET_API_URL``).
"""
return _resolve_openai_upstream_base(request.headers) or self.OPENAI_API_URL
custom = _resolve_openai_upstream_base(request.headers)
if custom is not None:
return custom
if route_grok_to_xai(request.headers, self.OPENAI_API_URL):
return XAI_API_URL
return self.OPENAI_API_URL
def _openai_extra_headers_for_upstream(self, upstream_base_url: str) -> dict[str, str] | None:
"""Return configured OpenAI extras for a direct non-xAI upstream.
``openai_extra_headers`` is operator-owned and scoped to the OpenAI
target (an API key for a gateway, a tenant header, ...). ``api.x.ai`` is
reached with the *client's* own xAI credential, so those extras must
never travel there — ``merge_extra_headers`` overrides same-named keys,
so a configured ``Authorization`` would both leak the operator's OpenAI
credential and clobber the client's ``Bearer xai-...``.
"""
if _is_xai_upstream(upstream_base_url):
return None
return self.config.openai_extra_headers
@staticmethod
def _strict_previous_turn_frozen_count(
@@ -3562,12 +3610,17 @@ class OpenAIHandlerMixin:
_pre_strip_count_chat = sum(1 for k in headers if k.lower().startswith("x-headroom-"))
headers = _strip_internal_headers(headers)
# `custom_upstream_base_url` is the per-request `x-headroom-base-url`
# override resolved above. Secrets only go to designated hosts.
# Configured backends own their destination and authentication, so they
# retain the existing extra-header policy. The direct path selects
# extras from the resolved OpenAI-compatible upstream. Direct custom
# upstreams must also pass the operator-designated-host check before
# receiving those extras.
headers = merge_extra_headers(
headers,
self.config.openai_extra_headers,
upstream_url=custom_upstream_base_url,
self.config.openai_extra_headers
if self.anthropic_backend is not None
else self._openai_extra_headers_for_upstream(upstream_base_url),
upstream_url=(None if self.anthropic_backend is not None else custom_upstream_base_url),
config=self.config,
)
log_outbound_headers(
@@ -3575,19 +3628,19 @@ class OpenAIHandlerMixin:
stripped_count=_pre_strip_count_chat,
request_id=request_id,
)
upstream_base_url = _resolve_openai_upstream_base(request.headers)
custom_upstream_base_url = _resolve_openai_upstream_base(request.headers)
handler_path = (
_resolve_openai_handler_path(
request.headers,
handler_path=_OPENAI_CHAT_COMPLETIONS_PATH,
)
if upstream_base_url is not None
if custom_upstream_base_url is not None
else "/v1/chat/completions"
)
_, custom_chat_provider = _custom_base_passthrough_telemetry(
request.method,
handler_path,
upstream_base_url or "",
custom_upstream_base_url or "",
)
openai_chat_outcome_provider = custom_chat_provider or "openai"
@@ -5069,9 +5122,13 @@ class OpenAIHandlerMixin:
},
)
# Direct OpenAI API (no backend configured)
# Direct OpenAI API (no backend configured). Reuse the upstream resolved
# once at request entry (custom base → Grok CLI → process default): the
# local ``custom_upstream_base_url`` above is custom-header only, and the
# same value already decided which extra headers were merged, so routing
# and header policy cannot drift apart.
url = build_copilot_upstream_url(
upstream_base_url or self.OPENAI_API_URL,
upstream_base_url,
handler_path,
)
url = _append_request_query(url, request.url.query)
@@ -5846,16 +5903,21 @@ class OpenAIHandlerMixin:
_pre_strip_count_resp = sum(1 for k in headers if k.lower().startswith("x-headroom-"))
headers = _strip_internal_headers(headers)
# This handler also honors `x-headroom-base-url` (resolved further
# below); resolve it here too so the secret headers are gated on the
# real destination rather than merged before it is known.
# Resolve the client-supplied upstream once: the secret-header gate,
# the routing decision, and the CCR streaming decision all need it.
upstream_base_url = _resolve_openai_upstream_base(request.headers)
# Client header and resolved candidate are different values. CCR and
# the secret-header gate see only x-headroom-base-url (None when the
# client did not set one). Routing uses the resolved candidate:
# that header, then a Grok CLI fingerprint, then the process default.
# Extras are chosen from the resolved host so OpenAI-target headers
# cannot ride a Grok request to api.x.ai. Mixed ChatGPT-auth plus
# Grok signals still withhold extras, even though the ChatGPT branch
# below sends the request to chatgpt.com.
custom_upstream_base_url = _resolve_openai_upstream_base(request.headers)
upstream_base_url = custom_upstream_base_url
openai_upstream_base_url = self._resolve_openai_upstream(request)
headers = merge_extra_headers(
headers,
self.config.openai_extra_headers,
upstream_url=upstream_base_url,
self._openai_extra_headers_for_upstream(openai_upstream_base_url),
upstream_url=custom_upstream_base_url,
config=self.config,
)
# Mirror the WS handler: never forward Codex's client-only lite header
@@ -6189,11 +6251,14 @@ class OpenAIHandlerMixin:
else:
handler_path = (
_resolve_openai_handler_path(request.headers, handler_path=_OPENAI_RESPONSES_PATH)
if upstream_base_url is not None
if custom_upstream_base_url is not None
else "/v1/responses"
)
# Reuse the OpenAI-compatible candidate resolved at request entry.
# In this non-ChatGPT branch it is also the actual upstream, keeping
# direct routing and header selection aligned.
url = build_copilot_upstream_url(
upstream_base_url or self.OPENAI_API_URL,
openai_upstream_base_url,
handler_path,
)
url = _append_request_query(url, request.url.query)
+305
View File
@@ -0,0 +1,305 @@
"""Call-site regressions for Grok routing and OpenAI-header isolation.
The tests assert the outbound URL and headers together. A routing-only unit test
would miss operator credentials leaking after a Grok request selects xAI, while a
header-only test could pass if a handler simply stopped merging extras everywhere.
"""
from __future__ import annotations
from collections.abc import Callable
from types import SimpleNamespace
from typing import Any, NamedTuple, cast
import anyio
import pytest
from headroom.cache.prefix_tracker import SessionTrackerStore
from tests.test_openai_codex_routing import (
_build_request,
_DummyOpenAIHandler,
_DummyTokenizer,
_jwt,
)
@pytest.fixture(autouse=True)
def _allow_client_chosen_test_upstreams(monkeypatch: pytest.MonkeyPatch) -> None:
"""Allowlist the client-chosen upstreams these tests name.
``x-headroom-base-url`` is validated by the SSRF guard before it is honored,
and in allowlist mode only these hosts pass. Naming them keeps the header
policy under test here rather than DNS: ``litellm.internal`` is deliberately
unresolvable, and the ``api.x.ai`` cases would otherwise depend on a live
lookup. The trailing-dot form is a distinct host to the guard, which matches
on the parsed hostname without normalizing it.
This is the SSRF allowlist, not the designated-upstream list that governs
which hosts may receive operator secrets, so the header assertions below
still exercise the real policy.
"""
monkeypatch.setenv("HEADROOM_ALLOWED_BASE_URLS", "litellm.internal,api.x.ai,api.x.ai.")
class _MinimalConfig(SimpleNamespace):
"""Treat unset optional chat features as disabled."""
def __getattr__(self, name: str) -> None:
return None
class _ChatHandler(_DummyOpenAIHandler):
"""Add the state read by the full chat-completions path."""
anthropic_backend: Any
ccr_response_handler: Any
def __init__(self) -> None:
super().__init__()
self.config = _MinimalConfig(**vars(self.config))
self.cache = None
self.session_tracker_store = cast(SimpleNamespace, SessionTrackerStore())
_GROK_HEADERS = {
"Authorization": "Bearer xai-redacted",
"x-xai-token-auth": "xai-grok-cli",
"User-Agent": "grok-pager/0.2.117 grok-shell/0.2.117 (macos; aarch64)",
}
_OPENAI_CLIENT_HEADERS = {
"Authorization": "Bearer sk-test",
"User-Agent": "codex-tui/0.146.0",
}
_OPERATOR_EXTRAS = {
"X-Operator-Sentinel": "must-not-leak",
"Authorization": "Bearer operator-openai-key",
}
_SENTINEL_ONLY_EXTRAS = {"X-Operator-Sentinel": "must-not-leak"}
_SENTINEL_VALUE = "must-not-leak"
_OPERATOR_KEY = "Bearer operator-openai-key"
_CLIENT_XAI_KEY = "Bearer xai-redacted"
@pytest.fixture(autouse=True)
def _stub_tokenizer(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr("headroom.tokenizers.get_tokenizer", lambda model: _DummyTokenizer())
class _Outbound(NamedTuple):
url: str
headers: dict[str, str]
def _lower_unique(headers: dict[str, str]) -> dict[str, str]:
"""Normalize names and fail if duplicate case variants would go on wire."""
lowered = {key.lower(): value for key, value in headers.items()}
assert len(lowered) == len(headers), f"duplicate header names on wire: {sorted(headers)}"
return lowered
def _outbound(captured: tuple[str, str, dict, dict] | None) -> _Outbound:
assert captured is not None
_method, url, headers, _body = captured
return _Outbound(url=url, headers=_lower_unique(headers))
def _run_chat(
request_headers: dict[str, str],
*,
model: str = "grok-4",
extras: dict[str, str] = _OPERATOR_EXTRAS,
openai_api_url: str | None = None,
) -> _Outbound:
handler = _ChatHandler()
handler.config.openai_extra_headers = dict(extras)
if openai_api_url is not None:
handler.OPENAI_API_URL = openai_api_url
request = _build_request(
{"model": model, "messages": [{"role": "user", "content": "hello"}]},
request_headers,
path="/v1/chat/completions",
)
anyio.run(handler.handle_openai_chat, request)
return _outbound(handler.captured_request)
def _run_responses(
request_headers: dict[str, str],
*,
model: str = "grok-4",
extras: dict[str, str] = _OPERATOR_EXTRAS,
openai_api_url: str | None = None,
) -> _Outbound:
handler = _DummyOpenAIHandler()
handler.config.openai_extra_headers = dict(extras)
if openai_api_url is not None:
handler.OPENAI_API_URL = openai_api_url
request = _build_request({"model": model, "input": "hello"}, request_headers)
anyio.run(handler.handle_openai_responses, request)
return _outbound(handler.captured_request)
_Runner = Callable[..., _Outbound]
@pytest.mark.parametrize(
("run", "expected_url"),
[
(_run_chat, "https://api.x.ai/v1/chat/completions"),
(_run_responses, "https://api.x.ai/v1/responses"),
],
ids=("chat", "responses"),
)
def test_grok_call_sites_withhold_operator_extras_from_xai(run: _Runner, expected_url: str) -> None:
outbound = run(_GROK_HEADERS)
assert outbound.url == expected_url
assert "x-operator-sentinel" not in outbound.headers
assert outbound.headers["authorization"] == _CLIENT_XAI_KEY
@pytest.mark.parametrize(
("run", "expected_url"),
[
(_run_chat, "https://api.openai.com/v1/chat/completions"),
(_run_responses, "https://api.openai.com/v1/responses"),
],
ids=("chat", "responses"),
)
def test_ordinary_openai_call_sites_still_merge_operator_extras(
run: _Runner, expected_url: str
) -> None:
outbound = run(_OPENAI_CLIENT_HEADERS, model="gpt-4o-mini")
assert outbound.url == expected_url
assert outbound.headers["x-operator-sentinel"] == _SENTINEL_VALUE
assert outbound.headers["authorization"] == _OPERATOR_KEY
@pytest.mark.parametrize(
("run", "expected_url"),
[
(_run_chat, "https://litellm.internal/v1/chat/completions"),
(_run_responses, "https://litellm.internal/v1/responses"),
],
ids=("chat", "responses"),
)
def test_configured_gateway_outranks_grok_and_keeps_operator_extras(
run: _Runner, expected_url: str
) -> None:
outbound = run(_GROK_HEADERS, openai_api_url="https://litellm.internal")
assert outbound.url == expected_url
assert outbound.headers["x-operator-sentinel"] == _SENTINEL_VALUE
assert outbound.headers["authorization"] == _OPERATOR_KEY
@pytest.mark.parametrize(
("run", "base_url", "expected_url", "withhold"),
[
(
_run_chat,
"https://litellm.internal",
"https://litellm.internal/v1/chat/completions",
True,
),
(
_run_chat,
"https://api.x.ai/v1",
"https://api.x.ai/v1/v1/chat/completions",
True,
),
(
_run_responses,
"https://api.x.ai:8443",
"https://api.x.ai:8443/v1/responses",
True,
),
(
_run_responses,
"https://api.x.ai.",
"https://api.x.ai./v1/responses",
True,
),
],
ids=("untrusted-non-xai", "xai-subpath", "xai-port", "xai-trailing-dot"),
)
def test_custom_base_url_applies_hostname_scoped_header_policy(
run: _Runner,
base_url: str,
expected_url: str,
withhold: bool,
) -> None:
outbound = run(
{"Authorization": _CLIENT_XAI_KEY, "x-headroom-base-url": base_url},
)
assert outbound.url == expected_url
if withhold:
assert "x-operator-sentinel" not in outbound.headers
assert outbound.headers["authorization"] == _CLIENT_XAI_KEY
else:
assert outbound.headers["x-operator-sentinel"] == _SENTINEL_VALUE
assert outbound.headers["authorization"] == _OPERATOR_KEY
def test_configured_xai_target_withholds_ambient_openai_extras() -> None:
outbound = _run_responses(_GROK_HEADERS, openai_api_url="https://api.x.ai")
assert outbound.url == "https://api.x.ai/v1/responses"
assert "x-operator-sentinel" not in outbound.headers
assert outbound.headers["authorization"] == _CLIENT_XAI_KEY
def test_chatgpt_auth_wins_routing_while_xai_candidate_withholds_extras() -> None:
token = _jwt({"https://api.openai.com/auth": {"chatgpt_account_id": "acct-from-jwt"}})
outbound = _run_responses(
{**_GROK_HEADERS, "Authorization": f"Bearer {token}"},
model="gpt-5.4",
extras=_SENTINEL_ONLY_EXTRAS,
)
assert outbound.url == "https://chatgpt.com/backend-api/codex/responses"
assert outbound.headers["chatgpt-account-id"] == "acct-from-jwt"
assert "x-operator-sentinel" not in outbound.headers
class _RecordingBackend:
name = "litellm"
def __init__(self) -> None:
self.captured_headers: dict[str, str] | None = None
async def send_openai_message(self, body: dict, headers: dict) -> SimpleNamespace:
self.captured_headers = dict(headers)
return SimpleNamespace(
status_code=200,
error=None,
body={
"id": "chatcmpl-backend-1",
"choices": [{"index": 0, "message": {"role": "assistant", "content": "ok"}}],
"usage": {"prompt_tokens": 2, "completion_tokens": 1},
},
)
def test_configured_chat_backend_retains_its_operator_extras() -> None:
handler = _ChatHandler()
handler.config.openai_extra_headers = dict(_OPERATOR_EXTRAS)
backend = _RecordingBackend()
handler.anthropic_backend = backend
handler.ccr_response_handler = None
request = _build_request(
{"model": "grok-4", "messages": [{"role": "user", "content": "hello"}]},
_GROK_HEADERS,
path="/v1/chat/completions",
)
response = anyio.run(handler.handle_openai_chat, request)
assert response.status_code == 200
assert backend.captured_headers is not None
headers = _lower_unique(backend.captured_headers)
assert headers["x-operator-sentinel"] == _SENTINEL_VALUE
assert headers["authorization"] == _OPERATOR_KEY
assert handler.captured_request is None
+3 -3
View File
@@ -401,7 +401,7 @@ class _ZdrResponsesHandler(_DummyOpenAIHandler):
)
def _build_request(body: dict, headers: dict[str, str]) -> Request:
def _build_request(body: dict, headers: dict[str, str], path: str = "/v1/responses") -> Request:
payload = json.dumps(body).encode("utf-8")
async def receive():
@@ -413,8 +413,8 @@ def _build_request(body: dict, headers: dict[str, str]) -> Request:
"http_version": "1.1",
"method": "POST",
"scheme": "https",
"path": "/v1/responses",
"raw_path": b"/v1/responses",
"path": path,
"raw_path": path.encode("utf-8"),
"query_string": b"",
"headers": [
(key.lower().encode("utf-8"), value.encode("utf-8")) for key, value in headers.items()
+30 -1
View File
@@ -1,6 +1,11 @@
from __future__ import annotations
from headroom.providers.grok import PROXY_ENV_KEY, build_launch_env, proxy_base_url
from headroom.providers.grok import (
PROXY_ENV_KEY,
build_launch_env,
is_grok_cli_request,
proxy_base_url,
)
from headroom.providers.grok.install import build_install_env
@@ -8,6 +13,30 @@ def test_grok_proxy_base_url_uses_local_headroom_proxy() -> None:
assert proxy_base_url(8787) == "http://127.0.0.1:8787/v1"
def test_is_grok_cli_request_matches_xai_token_auth() -> None:
# Official Grok CLI stamps x-xai-token-auth: xai-grok-cli on inference.
assert is_grok_cli_request({"x-xai-token-auth": "xai-grok-cli"}) is True
assert is_grok_cli_request({"X-Xai-Token-Auth": "xai-grok-cli"}) is True
assert is_grok_cli_request({"x-xai-token-auth": "other"}) is False
def test_is_grok_cli_request_matches_known_user_agents() -> None:
assert (
is_grok_cli_request(
{"user-agent": "grok-pager/0.2.117 grok-shell/0.2.117 (macos; aarch64)"}
)
is True
)
assert is_grok_cli_request({"user-agent": "grok-shell/0.2.112"}) is True
# Non-Grok OpenAI-compatible clients must not match, including wrappers
# whose name merely contains "grok" — they carry OpenAI credentials.
assert is_grok_cli_request({"user-agent": "codex-tui/0.146.0"}) is False
assert is_grok_cli_request({"user-agent": "litellm-grok/1.0"}) is False
assert is_grok_cli_request({"user-agent": "my-grok-shell/1.0"}) is False
assert is_grok_cli_request({"user-agent": "grok/0.1.0"}) is False
assert is_grok_cli_request({}) is False
def test_grok_build_launch_env_sets_models_base_url() -> None:
env, display = build_launch_env(9999, environ={})
+42
View File
@@ -4,6 +4,7 @@ from unittest.mock import patch
from headroom.providers.proxy_targets import (
api_target,
openai_compatible_base_url,
select_passthrough_base_url,
vertex_target_for_location,
)
@@ -78,3 +79,44 @@ def test_select_passthrough_base_url_handles_special_auth_modes() -> None:
"https://legacy.anthropic.test"
)
assert select_passthrough_base_url(proxy, {}) == "https://legacy.openai.test"
# A configured OpenAI target outranks Grok wire signals (see
# ``test_openai_compatible_base_url_respects_configured_openai_target``).
assert (
select_passthrough_base_url(
proxy,
{
"x-xai-token-auth": "xai-grok-cli",
"user-agent": "grok-shell/0.2.117",
},
)
== "https://legacy.openai.test"
)
def test_openai_compatible_base_url_routes_grok_to_xai_on_default_target() -> None:
"""The shared-proxy case this routing exists for: OpenAI target untouched."""
proxy = _proxy(OPENAI_API_URL="https://api.openai.com")
grok_headers = {"user-agent": "grok-pager/0.2.117 grok-shell/0.2.117"}
assert openai_compatible_base_url(proxy, grok_headers) == "https://api.x.ai"
assert openai_compatible_base_url(proxy, {}) == "https://api.openai.com"
# Also via passthrough, which is the catch-all entry point Grok CLI hits.
assert select_passthrough_base_url(proxy, grok_headers) == "https://api.x.ai"
def test_openai_compatible_base_url_respects_configured_openai_target() -> None:
"""An operator gateway is chosen for every OpenAI-compatible client.
A client User-Agent must not silently bypass it, nor carry the gateway's
``OPENAI_TARGET_API_HEADERS`` to a different vendor.
"""
proxy = _proxy(OPENAI_API_URL="https://legacy.openai.test")
assert (
openai_compatible_base_url(
proxy,
{"user-agent": "grok-pager/0.2.117 grok-shell/0.2.117"},
)
== "https://legacy.openai.test"
)
assert openai_compatible_base_url(proxy, {}) == "https://legacy.openai.test"
@@ -10,6 +10,11 @@ These tests pin the resolution contract:
- header present → its value wins
- header absent → configured ``OPENAI_API_URL`` fallback
- header empty or whitespace-only → fallback (no blanking)
- Grok CLI wire signals → ``api.x.ai``, but only while the OpenAI target is
still the default (a configured gateway is never bypassed)
The chat/responses call sites are covered end to end in
``tests/test_grok_shared_proxy_call_sites.py``.
"""
from __future__ import annotations
@@ -97,3 +102,67 @@ def test_header_with_subpath_preserves_path() -> None:
# Trailing slash is normalized away, not doubled.
trailing = _FakeRequest({"x-headroom-base-url": "https://gateway.example/api/v1/"})
assert proxy._resolve_openai_upstream(trailing) == "https://gateway.example/api/v1"
def test_grok_cli_routes_to_xai_when_process_default_is_openai() -> None:
"""Shared proxy started by Claude/Codex keeps OPENAI_API_URL=api.openai.com.
Grok CLI cannot stamp x-headroom-base-url; wire signals must still route
chat completions to api.x.ai so xAI session tokens are not sent to OpenAI.
"""
proxy = _stub_proxy("https://api.openai.com")
request = _FakeRequest(
{
"authorization": "Bearer redacted",
"x-xai-token-auth": "xai-grok-cli",
"user-agent": "grok-pager/0.2.117 grok-shell/0.2.117 (macos; aarch64)",
}
)
assert proxy._resolve_openai_upstream(request) == "https://api.x.ai"
def test_grok_cli_ua_alone_routes_to_xai() -> None:
proxy = _stub_proxy("https://api.openai.com")
request = _FakeRequest({"user-agent": "grok-shell/0.2.112 (macos; aarch64)"})
assert proxy._resolve_openai_upstream(request) == "https://api.x.ai"
def test_custom_base_url_still_wins_over_grok_signals() -> None:
"""Explicit x-headroom-base-url remains highest priority (gateway override)."""
proxy = _stub_proxy("https://api.openai.com")
request = _FakeRequest(
{
"x-headroom-base-url": "https://gateway.example",
"x-xai-token-auth": "xai-grok-cli",
"user-agent": "grok-shell/0.2.112",
}
)
assert proxy._resolve_openai_upstream(request) == "https://gateway.example"
def test_non_grok_clients_keep_process_openai_default() -> None:
proxy = _stub_proxy("https://api.openai.com")
request = _FakeRequest(
{
"authorization": "Bearer sk-test",
"user-agent": "codex-tui/0.146.0",
}
)
assert proxy._resolve_openai_upstream(request) == "https://api.openai.com"
def test_grok_signals_do_not_override_a_configured_gateway() -> None:
"""Grok routing is a fallback for the default target, not an override."""
proxy = _stub_proxy("https://gateway.internal")
request = _FakeRequest(
{
"x-xai-token-auth": "xai-grok-cli",
"user-agent": "grok-shell/0.2.117",
}
)
assert proxy._resolve_openai_upstream(request) == "https://gateway.internal"