mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): honor x-headroom-base-url on provider passthrough routes (#4596)
## Description Provider passthrough routes (`/v1/embeddings`, `/v1/moderations`, `/v1/audio/*`, `/v1/messages/count_tokens`, `/v1beta/models`, `/v1beta/cachedContents`, `/api/hello`) ignored the per-request `x-headroom-base-url` override: their wrapper passed the fixed configured target to `handle_passthrough()`, which never reads the header, so callers that multiplex several custom OpenAI-compatible / Anthropic / Gemini providers through one proxy got these requests delivered to the default upstream. The `/v1/messages`, `/v1/alpha/search` and catch-all routes already validate and use the header; this family was the outlier. No upstream issue: this is the route-family variant of #4302; #4308/#4330 do not touch these handlers. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made <details> <summary>details</summary> - `headroom/providers/proxy_routes.py`: honor `x-headroom-base-url` like the catch-all path — SSRF guard (unsafe -> 400), trimmed custom base, bounded telemetry via `_custom_base_passthrough_telemetry`, then `handle_passthrough`. No header -> previous behavior. - `tests/test_provider_proxy_routes.py`: custom-base assertions for embeddings, count_tokens and Gemini models, plus an unsafe-URL 400 case. ## Testing <details> <summary>details</summary> - [x] Unit tests pass (`pytest tests/test_provider_proxy_routes.py`: 25 passed) - [x] Linting passes (`ruff check` on both files) - [ ] Type checking passes (Python repo; no mypy gate for these modules) - [x] New tests added - [x] Manual testing performed — see Real Behavior Proof ### Test Output <!-- Paste relevant command output or artifact links here --> <details><summary>commands and output</summary> ```text pytest tests/test_provider_proxy_routes.py -q -> 25 passed pytest <that file> tests/test_custom_base_passthrough_telemetry.py \ tests/test_proxy/test_anthropic_upstream_header.py tests/test_proxy/test_openai_upstream_header.py -q -> 43 passed all *passthrough* test files (14) -> 167 passed, 19 skipped ruff check <two files> -> All checks passed ``` Before the source change the new assertions fail: `assert 'https://api.openai.test' == 'https://custom.example/base'`. </details> </details> ## Real Behavior Proof <details> <summary>details</summary> - Environment: macOS, Python 3.13, `create_app(ProxyConfig(...))` with `HEADROOM_ALLOWED_BASE_URLS=127.0.0.1`, two real local HTTP upstreams (default A, custom B). - Exact command / steps: requests through the app TestClient, i.e. the real `handle_passthrough` path, not a mocked handler. - Observed result: embeddings/count_tokens/Gemini requests with the header reached the custom upstream and returned 200 (`/custom/v1/embeddings`, `/anthropic/v1/messages/count_tokens`, `/gemini/v1beta/models`); the same embeddings call without the header reached the default upstream; an unsafe URL got 400 "Rejected unsafe upstream base URL"; downstream logs showed the default upstream receiving only `POST /v1/embeddings`. - Not tested: the full pytest suite and CI; Windows; a live external provider (local upstreams stand in). Telemetry label is empty for a generic custom host, same as the catch-all path. </details> </details> ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: N/A (no rollout-managed feature). - Stable/default behavior changed: no - only requests that send the x-headroom-base-url header take the new branch. - Kill switch / disable path: remove the header (or point the provider base URL back); the previous path is the default. - Unsafe override required: no - unsafe URLs are rejected with 400 as on the other routes. - Qualification impact: none expected. - Rollback path: revert the commit; no state changes. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (none needed: matches the documented custom-base contract) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Screenshots (if applicable) N/A — no UI change. ## Additional Notes Reuses the existing helpers instead of adding a second policy, so telemetry stays bounded (request host is not a stats label, per #3759). The local `_core` debug wheel needed to start the proxy lifespan is build output and not in the diff. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4596" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JD Davis
parent
51258ef14a
commit
5c8772b721
@@ -108,6 +108,25 @@ def _register_provider_passthrough_route(
|
||||
spec: ProviderPassthroughRoute,
|
||||
) -> None:
|
||||
async def provider_passthrough(request: Request):
|
||||
# Honor the per-request upstream override like /v1/messages and the
|
||||
# catch-all do. Clients that speak one wire format but authenticate
|
||||
# against a custom OpenAI-compatible gateway must reach that gateway
|
||||
# for embeddings/audio/metadata too, not the default provider target
|
||||
# (the same gap as the model-metadata routes, #4302). Telemetry keeps
|
||||
# the route spec labels: they are bounded by this module, so the
|
||||
# request-controlled host never becomes a stats label (#3759), while
|
||||
# `handle_passthrough` still records the outcome for these known routes.
|
||||
custom_base = request.headers.get("x-headroom-base-url", "").strip()
|
||||
if custom_base:
|
||||
if not await is_safe_upstream_url_async(custom_base):
|
||||
logger.warning("rejecting unsafe x-headroom-base-url: %r", custom_base)
|
||||
raise HTTPException(status_code=400, detail="Rejected unsafe upstream base URL")
|
||||
return await proxy.handle_passthrough(
|
||||
request,
|
||||
custom_base.rstrip("/"),
|
||||
spec.sub_path,
|
||||
spec.provider_name,
|
||||
)
|
||||
return await proxy.handle_passthrough(
|
||||
request,
|
||||
_api_target(proxy, spec.provider_name),
|
||||
|
||||
@@ -144,6 +144,41 @@ def test_provider_passthrough_routes_forward_expected_targets(monkeypatch) -> No
|
||||
assert client.get("/v1beta/models").json()["provider"] == "gemini"
|
||||
assert client.get("/v1beta/models/demo").json()["sub_path"] == "models"
|
||||
assert client.post("/v1beta/models/demo:embedContent").json()["sub_path"] == "embedContent"
|
||||
# Provider passthrough routes must honor the per-request upstream like
|
||||
# /v1/messages and the catch-all do. Without this, a custom
|
||||
# OpenAI-compatible provider's embeddings/audio/metadata requests are
|
||||
# forwarded to the default provider target and fail with 401.
|
||||
custom_embeddings = client.post(
|
||||
"/v1/embeddings",
|
||||
headers={"x-headroom-base-url": "https://custom.example/base/"},
|
||||
).json()
|
||||
assert custom_embeddings["base_url"] == "https://custom.example/base"
|
||||
assert custom_embeddings["path"] == "/v1/embeddings"
|
||||
# Known provider routes keep their bounded labels so handle_passthrough
|
||||
# records the outcome for the overridden upstream; the custom host is
|
||||
# never turned into a label.
|
||||
assert custom_embeddings["sub_path"] == "embeddings"
|
||||
assert custom_embeddings["provider"] == "openai"
|
||||
custom_count_tokens = client.post(
|
||||
"/v1/messages/count_tokens",
|
||||
headers={"x-headroom-base-url": "https://custom.example/anthropic/"},
|
||||
).json()
|
||||
assert custom_count_tokens["base_url"] == "https://custom.example/anthropic"
|
||||
assert custom_count_tokens["sub_path"] == "count_tokens"
|
||||
assert custom_count_tokens["provider"] == "anthropic"
|
||||
custom_gemini = client.get(
|
||||
"/v1beta/models",
|
||||
headers={"x-headroom-base-url": "https://custom.example/gemini/"},
|
||||
).json()
|
||||
assert custom_gemini["base_url"] == "https://custom.example/gemini"
|
||||
assert custom_gemini["sub_path"] == "models"
|
||||
assert custom_gemini["provider"] == "gemini"
|
||||
# An unsafe override is rejected instead of being relayed.
|
||||
unsafe_embeddings = client.post(
|
||||
"/v1/embeddings",
|
||||
headers={"x-headroom-base-url": "http://127.0.0.1:9"},
|
||||
)
|
||||
assert unsafe_embeddings.status_code == 400
|
||||
assert client.post(
|
||||
"/v1/projects/p/locations/us-central1/publishers/google/models/gemini-2.0-flash:generateContent"
|
||||
).json() == {
|
||||
|
||||
Reference in New Issue
Block a user