fix(grok): preserve xAI model context metadata (#3312) (#3315)

## Description

Wrapped Grok Shell receives xAI model metadata with `context_length`,
but Grok's custom-endpoint parser reads `context_window` and falls back
to 256K when that alias is absent. Normalize the successful xAI
model-list response at Headroom's provider metadata boundary, preserving
the upstream value and the `grok-4.6` model id. Closes #3312.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Translate a positive xAI `context_length` value to Grok's missing
`context_window` field for the resolved `api.x.ai` model list.
- Preserve model ids, original xAI fields, explicit context aliases,
response status, and non-xAI model metadata.
- Leave backend selection unchanged, so this fixes the 256K context
fallback without rerouting inference from ChatCompletions to Responses.
- Remove entity validators when rewriting model-list bodies, preventing
cached metadata from being validated against the unmodified upstream
body.

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_providers/test_grok_model_metadata.py
tests/test_provider_model_metadata.py
tests/test_provider_proxy_routes.py tests/test_provider_grok.py
tests/test_cli/test_wrap_grok.py -q`)
- [x] Linting passes (`uv run ruff check
headroom/providers/grok/model_metadata.py
headroom/providers/model_metadata.py headroom/proxy/handlers/openai.py
tests/test_providers/test_grok_model_metadata.py
tests/test_provider_model_metadata.py
tests/test_provider_proxy_routes.py`)
- [x] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality when applicable
- [ ] Manual testing performed

### Test Output

```text
uv run pytest tests/test_providers/test_grok_model_metadata.py tests/test_provider_model_metadata.py tests/test_provider_proxy_routes.py tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py -q
46 passed, 1 warning in 8.41s

uv run ruff check headroom/providers/grok/model_metadata.py headroom/providers/model_metadata.py headroom/proxy/handlers/openai.py tests/test_providers/test_grok_model_metadata.py tests/test_provider_model_metadata.py tests/test_provider_proxy_routes.py
All checks passed!

uv run ruff format headroom/providers/grok/model_metadata.py headroom/providers/model_metadata.py headroom/proxy/handlers/openai.py tests/test_providers/test_grok_model_metadata.py tests/test_provider_model_metadata.py tests/test_provider_proxy_routes.py --check
6 files already formatted

uv run mypy headroom
Success: no issues found in 529 source files
```

## Real Behavior Proof

- Environment: Windows, Headroom development build, focused route tests
with a local OpenAI-compatible response fixture.
- Exact command / steps: Run
`tests/test_provider_proxy_routes.py::test_issue_3312_grok_model_metadata`;
the fixture sends an xAI-shaped `/v1/models` entry containing `id:
grok-4.6` and `context_length: 500000` through the real provider route.
- Observed result: The patched route returns `id: grok-4.6`, preserves
`context_length: 500000`, and adds `context_window: 500000`.
- Not tested: live Grok Shell 1.0.5 session against `api.x.ai`

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable/default.
- Stable/default behavior changed: Yes, wrapped xAI model discovery now
exposes the upstream context window to Grok.
- Kill switch / disable path: Not applicable, revert the change to
disable it.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert the change; model-list responses return to the
previous metadata shape.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`, it is generated by
release-please from the Conventional Commit PR title.

## Screenshots (if applicable)

Not applicable for this provider metadata change.

## Additional Notes

The change is scoped to the xAI model-list schema boundary. There is no
runtime rollout switch; the default behavior changes only for wrapped
xAI model discovery, and reverting the change restores the previous
metadata response. Live Grok Shell 1.0.5 verification against `api.x.ai`
was not run; the available local binary is Grok 1.0.0.
This commit is contained in:
Rod Boev
2026-09-25 00:00:59 -05:00
committed by GitHub
parent b0c19a25f0
commit c81378c878
6 changed files with 480 additions and 2 deletions
+55
View File
@@ -0,0 +1,55 @@
"""xAI model metadata normalization for Grok's custom endpoint."""
from __future__ import annotations
from collections.abc import Mapping
from typing import Any
from urllib.parse import urlsplit
from headroom.providers.grok.runtime import DEFAULT_API_URL
_XAI_API_HOST = urlsplit(DEFAULT_API_URL).hostname
def is_xai_model_list_target(base_url: str) -> bool:
"""Return whether ``base_url`` is the exact xAI model-list target."""
parsed = urlsplit(base_url)
return (
parsed.hostname is not None
and _XAI_API_HOST is not None
and parsed.hostname.lower() == _XAI_API_HOST.lower()
and parsed.path.rstrip("/") in {"", "/v1"}
)
def normalize_xai_model_metadata(payload: Any) -> Any:
"""Add Grok's context alias to eligible xAI model-list entries."""
if not isinstance(payload, dict) or not isinstance(payload.get("data"), list):
return None
normalized_entries: list[Any] = []
changed = False
for entry in payload["data"]:
if not isinstance(entry, Mapping):
normalized_entries.append(entry)
continue
context_length = entry.get("context_length")
if (
isinstance(context_length, int)
and not isinstance(context_length, bool)
and context_length > 0
and "context_window" not in entry
and "contextWindow" not in entry
):
normalized_entry = dict(entry)
normalized_entry["context_window"] = context_length
normalized_entries.append(normalized_entry)
changed = True
else:
normalized_entries.append(entry)
if not changed:
return None
normalized_payload = dict(payload)
normalized_payload["data"] = normalized_entries
return normalized_payload
+46 -2
View File
@@ -2,13 +2,19 @@
from __future__ import annotations
import json
from dataclasses import dataclass
from typing import Any, cast
from typing import Any, NoReturn, cast
from fastapi import Request
from fastapi.responses import Response
from headroom.providers.codex.model_metadata import handle_chatgpt_model_metadata
from headroom.providers.grok.model_metadata import (
is_xai_model_list_target,
normalize_xai_model_metadata,
)
from headroom.proxy.helpers import sanitize_forwarded_response_headers
@dataclass(frozen=True, slots=True)
@@ -23,6 +29,10 @@ class ModelMetadataEndpoint:
MODEL_METADATA_LIST_ENDPOINT = ModelMetadataEndpoint("/v1/models", "/backend-api/models")
def _reject_non_standard_json_constant(value: str) -> NoReturn:
raise ValueError(f"non-standard JSON constant: {value}")
def model_metadata_get_endpoint(model_id: str) -> ModelMetadataEndpoint:
"""Return the single-model metadata endpoint for ``model_id``."""
return ModelMetadataEndpoint(
@@ -49,7 +59,7 @@ async def handle_model_metadata_endpoint(
if chatgpt_response is not None:
return chatgpt_response
return cast(
response = cast(
Response,
await proxy.handle_passthrough(
request,
@@ -58,3 +68,37 @@ async def handle_model_metadata_endpoint(
provider_name,
),
)
if (
endpoint == MODEL_METADATA_LIST_ENDPOINT
and 200 <= response.status_code < 300
and is_xai_model_list_target(provider_api_base_url)
):
normalized_content: bytes | None = None
try:
payload = json.loads(
response.body,
parse_constant=_reject_non_standard_json_constant,
)
normalized_payload = normalize_xai_model_metadata(payload)
if normalized_payload is not None:
normalized_content = json.dumps(
normalized_payload,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
except (TypeError, UnicodeError, ValueError):
normalized_payload = None
if normalized_payload is not None and normalized_content is not None:
headers = sanitize_forwarded_response_headers(
response.headers,
"etag",
"last-modified",
"cache-control",
)
headers["content-type"] = "application/json"
return Response(
content=normalized_content,
status_code=response.status_code,
headers=headers,
)
return response
+6
View File
@@ -10989,6 +10989,12 @@ class OpenAIHandlerMixin:
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
response_headers = _sanitize_forwarded_response_headers(
response.headers,
"etag",
"last-modified",
"cache-control",
)
response_headers["content-type"] = "application/json"
# Passthrough request: forwarded upstream with no transforms.
+250
View File
@@ -94,3 +94,253 @@ def test_handle_model_metadata_endpoint_falls_back_to_selected_provider(monkeypa
assert response.json() == {"provider": "anthropic", "sub_path": "models"}
assert calls == [("https://api.anthropic.test", "models", "anthropic")]
def test_grok_dispatch_adapts_xai_model_list(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return Response(
content=b'{"object":"list","data":[{"id":"grok-4.6","context_length":500000}]}',
status_code=200,
headers={"content-type": "application/json"},
)
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai/v1",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.json()["data"] == [
{"id": "grok-4.6", "context_length": 500000, "context_window": 500000}
]
def test_grok_response_preserves_status_and_safe_headers_without_stale_framing(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return Response(
content=b'{"data":[{"id":"grok","context_length":1}]}',
status_code=206,
headers={
"content-type": "application/json",
"x-upstream": "kept",
"etag": '"stale"',
"last-modified": "Thu, 01 Jan 1970 00:00:00 GMT",
"cache-control": "max-age=60",
"content-length": "44",
"content-encoding": "gzip",
"transfer-encoding": "chunked",
},
)
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.status_code == 206
assert response.headers["x-upstream"] == "kept"
assert response.headers.get("etag") is None
assert response.headers.get("last-modified") is None
assert response.headers.get("cache-control") is None
assert response.headers.get("content-encoding") is None
assert response.headers.get("transfer-encoding") is None
assert response.json()["data"][0]["context_window"] == 1
def test_grok_non_success_and_non_json_responses_are_unchanged(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
responses = {
"/error": Response(
content=b'{"data":[{"context_length":500000}]}',
status_code=500,
headers={"x-upstream": "error"},
),
"/non-json": Response(
content=b"upstream text",
status_code=200,
headers={"x-upstream": "text"},
),
"/surrogate": Response(
content=b'{"data":[{"id":"grok","context_length":1,"label":"\\ud800"}]}',
status_code=200,
headers={"x-upstream": "surrogate"},
),
"/non-finite": Response(
content=b'{"data":[{"id":"grok","context_length":1}],"extra":NaN}',
status_code=200,
headers={"x-upstream": "non-finite"},
),
}
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return responses[request.url.path]
app = FastAPI()
@app.get("/{kind}")
async def probe(request: Request, kind: str):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai",
provider_name="openai",
)
with TestClient(app) as client:
error = client.get("/error")
non_json = client.get("/non-json")
surrogate = client.get("/surrogate")
non_finite = client.get("/non-finite")
assert error.status_code == 500
assert error.content == b'{"data":[{"context_length":500000}]}'
assert error.headers["x-upstream"] == "error"
assert non_json.status_code == 200
assert non_json.content == b"upstream text"
assert non_json.headers["x-upstream"] == "text"
assert surrogate.content == b'{"data":[{"id":"grok","context_length":1,"label":"\\ud800"}]}'
assert surrogate.headers["x-upstream"] == "surrogate"
assert non_finite.content == b'{"data":[{"id":"grok","context_length":1}],"extra":NaN}'
assert non_finite.headers["x-upstream"] == "non-finite"
def test_grok_response_unchanged_preserves_entity_validators(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
return Response(
content=b'{"data":[{"id":"grok","context_length":1,"context_window":1}]}',
status_code=200,
headers={
"content-type": "application/json",
"etag": '"current"',
"last-modified": "Thu, 01 Jan 1970 00:00:00 GMT",
"cache-control": "max-age=60",
},
)
app = FastAPI()
@app.get("/probe")
async def probe(request: Request):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT,
provider_api_base_url="https://api.x.ai",
provider_name="openai",
)
with TestClient(app) as client:
response = client.get("/probe")
assert response.headers["etag"] == '"current"'
assert response.headers["last-modified"] == "Thu, 01 Jan 1970 00:00:00 GMT"
assert response.headers["cache-control"] == "max-age=60"
def test_grok_negative_space_bypasses_non_xai_detail_and_aliases(monkeypatch) -> None:
async def no_chatgpt_metadata(http_client, request: Request, upstream_path: str) -> None:
return None
monkeypatch.setattr(
"headroom.providers.model_metadata.handle_chatgpt_model_metadata",
no_chatgpt_metadata,
)
class Proxy:
http_client = "h2"
async def handle_passthrough(self, request, base_url, sub_path="", provider_name=""):
if request.url.path == "/non-xai":
content = b'{"data":[{"context_length":500000}]}'
elif request.url.path == "/detail":
content = b'{"id":"grok-4.6","context_length":500000}'
else:
content = b'{"data":[{"context_length":500000,"contextWindow":123}]}'
return Response(content=content, status_code=200, media_type="application/json")
app = FastAPI()
@app.get("/{kind}")
async def probe(request: Request, kind: str):
return await handle_model_metadata_endpoint(
Proxy(),
request,
endpoint=MODEL_METADATA_LIST_ENDPOINT
if kind == "list"
else model_metadata_get_endpoint("grok"),
provider_api_base_url="https://api.x.ai"
if kind != "non-xai"
else "https://api.openai.com",
provider_name="openai",
)
with TestClient(app) as client:
non_xai = client.get("/non-xai")
detail = client.get("/detail")
alias = client.get("/list")
assert non_xai.content == b'{"data":[{"context_length":500000}]}'
assert detail.content == b'{"id":"grok-4.6","context_length":500000}'
assert alias.json()["data"][0]["contextWindow"] == 123
+59
View File
@@ -1261,6 +1261,11 @@ def test_anthropic_model_metadata_strips_ansi_model_ids() -> None:
{"id": "claude-sonnet-4-5[1m]", "object": "model"},
],
},
headers={
"etag": '"stale"',
"last-modified": "Thu, 01 Jan 1970 00:00:00 GMT",
"cache-control": "max-age=60",
},
)
async def aclose(self) -> None:
@@ -1272,6 +1277,9 @@ def test_anthropic_model_metadata_strips_ansi_model_ids() -> None:
response = client.get("/v1/models", headers={"x-api-key": "sk-ant-test"})
assert response.status_code == 200
assert response.headers.get("etag") is None
assert response.headers.get("last-modified") is None
assert response.headers.get("cache-control") is None
assert response.json()["data"] == [
{"id": "claude-opus-4-8", "object": "model"},
{"id": "claude-sonnet-4-5", "object": "model"},
@@ -1309,6 +1317,57 @@ def test_anthropic_model_detail_path_strips_ansi_model_id() -> None:
]
def test_issue_3312_grok_model_metadata() -> None:
fixture = {
"model": "grok-4.6",
"context_length": 500000,
}
class FakeAsyncClient:
async def request(self, method, url, **kwargs): # type: ignore[no-untyped-def]
return httpx.Response(
200,
json={
"object": "list",
"data": [
{
"id": fixture["model"],
"object": "model",
"context_length": fixture["context_length"],
}
],
},
)
async def aclose(self) -> None:
return None
app = create_app(
ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
anthropic_api_url="https://api.anthropic.test",
openai_api_url="https://api.x.ai",
gemini_api_url="https://api.gemini.test",
cloudcode_api_url="https://cloudcode.test",
vertex_api_url="https://vertex.test",
)
)
with TestClient(app) as client:
client.app.state.proxy.http_client = FakeAsyncClient()
response = client.get("/v1/models", headers={"authorization": "Bearer sk-xai-test"})
entry = response.json()["data"][0]
assert response.status_code == 200
assert entry == {
"id": "grok-4.6",
"object": "model",
"context_length": 500000,
"context_window": 500000,
}
def test_anthropic_messages_strips_ansi_model_id_before_upstream() -> None:
class FakeAsyncClient:
def __init__(self) -> None:
@@ -0,0 +1,64 @@
from __future__ import annotations
import pytest
from headroom.providers.grok.model_metadata import (
is_xai_model_list_target,
normalize_xai_model_metadata,
)
from headroom.providers.grok.runtime import DEFAULT_API_URL
@pytest.mark.parametrize(
("url", "expected"),
[
(DEFAULT_API_URL, True),
(f"{DEFAULT_API_URL}/v1/", True),
("https://api.x.ai.evil.test", False),
("https://api.x.ai/v2", False),
("https://api.openai.com", False),
],
)
def test_xai_model_list_target_uses_exact_host_and_optional_v1_path(
url: str, expected: bool
) -> None:
assert is_xai_model_list_target(url) is expected
def test_normalize_xai_model_metadata_copies_only_positive_integer_context_length() -> None:
payload = {
"object": "list",
"data": [
{"id": "grok-4.6", "context_length": 500000, "nested": {"keep": True}},
{"id": "zero", "context_length": 0},
{"id": "negative", "context_length": -1},
{"id": "bool", "context_length": True},
{"id": "string", "context_length": "500000"},
{"id": "snake", "context_length": 10, "context_window": 20},
{"id": "camel", "context_length": 10, "contextWindow": 30},
"not-an-entry",
],
}
normalized = normalize_xai_model_metadata(payload)
assert normalized["object"] == "list"
assert normalized["data"][0] == {
"id": "grok-4.6",
"context_length": 500000,
"context_window": 500000,
"nested": {"keep": True},
}
assert normalized["data"][1:] == payload["data"][1:]
assert payload["data"][0] == {
"id": "grok-4.6",
"context_length": 500000,
"nested": {"keep": True},
}
def test_normalize_xai_model_metadata_returns_none_when_no_entry_changes() -> None:
assert normalize_xai_model_metadata({"data": [{"id": "grok", "context_length": 0}]}) is None
assert normalize_xai_model_metadata({"data": [{"id": "grok", "contextWindow": 10}]}) is None
assert normalize_xai_model_metadata({"data": []}) is None
assert normalize_xai_model_metadata({"data": {}}) is None