Files
headroom/tests/test_provider_vertex_runtime.py
a9c1ac5368 fix(providers/vertex): Vertex route multi-region locations (#3802)
## Description

Vertex AI's `us` and `eu` **multi-region** locations use a third
hostname shape
(`aiplatform.<location>.rep.googleapis.com`) that's distinct from both
the `global` endpoint and single-region endpoints
(`<location>-aiplatform.googleapis.com`). Headroom's
`vertex_target_for_location` only knew about the latter two shapes, so
`/locations/us/...` requests were forwarded to
`https://us-aiplatform.googleapis.com`, which 404s.

Closes #3798

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/providers/vertex/runtime.py`: `vertex_target_for_location`
now routes bare-word locations other than `global` (currently `us`,
`eu`) to`https://aiplatform.<location>.rep.googleapis.com`. This is a
structural rule (no hyphen ⇒ multi-region), not a `{"us", "eu"}`
allowlist — GCP single-region ids are always hyphenated (`us-central1`,
`europe-west4`, ...), so any multi-region Google adds later is covered
without a follow-up patch. The existing `global`/empty check is
unchanged and still runs first, since `global` is also a bare word and
needs its own (hostless) branch.
- `tests/test_provider_vertex_runtime.py`: added
`test_vertex_target_for_location_uses_rep_hostname_for_multi_region`
covering `us`, `eu`, and a hypothetical future bare-word multi-region.

## Testing

- [x] Unit tests pass (`pytest`)
- [x]  Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ uv run ruff check headroom/providers/vertex/runtime.py tests/test_provider_vertex_runtime.py
All checks passed!

$ uv run mypy headroom/providers/vertex/runtime.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: local dev sandbox; no GCP ADC / Vertex project
configured, so the exact repro in #3798 (`headroom wrap claude` against
a real `us` multi-region Vertex endpoint) could not be run end-to-end
here.
- Exact command / steps: ran the new unit test asserting
`vertex_target_for_location(DEFAULT_VERTEX_API_URL, "us") ==
"https://aiplatform.us.rep.googleapis.com"` (and `"eu"`), matching the
hostname format documented in Google's own multi-region announcement
([cloud.google.com
blog](https://cloud.google.com/blog/products/ai-machine-learning/multi-region-endpoints-for-claude-available-on-vertex-ai)).
- Observed result: assertion passes; existing
single-region/global/explicit-gateway tests are unaffected.
- Not tested: a live request against a real Vertex `us`/`eu`
multi-region project through `headroom wrap claude`.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Requests whose Vertex `location` is a
bare word other than `global` (currently only `us`/`eu`) now resolve to
`aiplatform.<location>.rep.googleapis.com` instead of the
previously-wrong `<location>-aiplatform.googleapis.com`, which always
404'd. Hyphenated single-region locations and `global` are unchanged.
- Kill switch / disable path: The existing `ANTHROPIC_VERTEX_BASE_URL` /
`--vertex-api-url` override already bypasses this location-derivation
logic entirely if a hardcoded upstream host is needed.
- Unsafe override required: No
- Qualification impact: None known
- Rollback path: Revert commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x]I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3802"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-05 08:33:07 -05:00

147 lines
5.4 KiB
Python

from __future__ import annotations
from urllib.parse import urlsplit
import pytest
from headroom.providers.registry import DEFAULT_VERTEX_API_URL
from headroom.providers.vertex import (
VERTEX_ANTHROPIC_PROVIDER_NAME,
VERTEX_COUNT_TOKENS,
VERTEX_GENERATE_CONTENT,
VERTEX_GOOGLE_PROVIDER_NAME,
VERTEX_RAW_PREDICT,
VERTEX_STREAM_GENERATE_CONTENT,
VERTEX_STREAM_RAW_PREDICT,
VertexPublisherAction,
is_vertex_anthropic_publisher,
is_vertex_google_publisher,
vertex_anthropic_target,
vertex_publisher_provider_name,
vertex_target_for_location,
)
def test_vertex_publisher_classification_is_explicit() -> None:
assert is_vertex_google_publisher("google") is True
assert is_vertex_google_publisher("anthropic") is False
assert is_vertex_anthropic_publisher("anthropic") is True
assert is_vertex_anthropic_publisher("google") is False
def test_vertex_provider_names_are_provider_owned() -> None:
assert VERTEX_GOOGLE_PROVIDER_NAME == "vertex:google"
assert VERTEX_ANTHROPIC_PROVIDER_NAME == "vertex:anthropic"
assert vertex_publisher_provider_name("mistral") == "vertex:mistral"
def test_vertex_publisher_actions_are_named_values() -> None:
assert VERTEX_GENERATE_CONTENT == VertexPublisherAction("generateContent")
assert VERTEX_STREAM_GENERATE_CONTENT == VertexPublisherAction("streamGenerateContent")
assert VERTEX_COUNT_TOKENS == VertexPublisherAction("countTokens")
assert VERTEX_RAW_PREDICT == VertexPublisherAction("rawPredict")
assert VERTEX_STREAM_RAW_PREDICT == VertexPublisherAction(
"streamRawPredict",
force_stream=True,
)
def test_vertex_anthropic_target_adds_v1_only_for_versionless_routes() -> None:
assert vertex_anthropic_target("https://europe-west1-aiplatform.googleapis.com") == (
"https://europe-west1-aiplatform.googleapis.com"
)
assert (
vertex_anthropic_target(
"https://europe-west1-aiplatform.googleapis.com/",
versionless_route=True,
)
== "https://europe-west1-aiplatform.googleapis.com/v1"
)
def test_vertex_target_for_location_derives_regional_hosts_from_default_target() -> None:
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, "europe-west1") == (
"https://europe-west1-aiplatform.googleapis.com"
)
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, "global") == (
"https://aiplatform.googleapis.com"
)
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, "") == (
"https://aiplatform.googleapis.com"
)
def test_vertex_target_for_location_uses_rep_hostname_for_multi_region() -> None:
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, "us") == (
"https://aiplatform.us.rep.googleapis.com"
)
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, "eu") == (
"https://aiplatform.eu.rep.googleapis.com"
)
def test_vertex_target_for_location_falls_back_for_unsupported_bare_word_locations() -> None:
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, "asia") == (
"https://asia-aiplatform.googleapis.com"
)
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, "foo") == (
"https://foo-aiplatform.googleapis.com"
)
def test_vertex_target_for_location_honors_explicit_gateway() -> None:
assert vertex_target_for_location("https://vertex-gateway.internal", "europe-west1") == (
"https://vertex-gateway.internal"
)
_VERTEX_PUBLIC_ENDPOINT = "https://aiplatform.googleapis.com"
@pytest.mark.parametrize(
"region",
["us-central1", "europe-west4", "asia-northeast1", "me-central1", "us-east5"],
)
def test_vertex_target_for_location_accepts_real_regions(region: str) -> None:
assert vertex_target_for_location(DEFAULT_VERTEX_API_URL, region) == (
f"https://{region}-aiplatform.googleapis.com"
)
@pytest.mark.parametrize(
"malicious",
[
"169.254.169.254#", # fragment delimiter -> cloud metadata IP (the reported PoC)
"127.0.0.1:44919#", # host:port + fragment
"169.254.169.254/latest/meta-data/iam#", # path injection
"169.254.169.254:80", # port injection
"evil.example", # dotted host
"foo@evil.example", # userinfo delimiter
"us_central1", # underscore (not a region)
"US-CENTRAL1", # uppercase
"-leading", # leading hyphen
"trailing-", # trailing hyphen
"a--b", # empty hyphen group
],
)
def test_vertex_target_for_location_rejects_ssrf_payloads(malicious: str) -> None:
"""A non-region ``location`` must never carry an attacker-chosen host into
the interpolated Vertex hostname (CWE-918). It falls back to the public
endpoint, and the parsed host is always the legitimate Vertex host — never
a metadata IP, loopback, or injected authority.
"""
target = vertex_target_for_location(DEFAULT_VERTEX_API_URL, malicious)
assert target == _VERTEX_PUBLIC_ENDPOINT
parsed = urlsplit(target)
assert parsed.hostname == "aiplatform.googleapis.com"
assert parsed.port is None
def test_vertex_target_for_location_ssrf_fallback_only_on_default_target() -> None:
"""A validated non-region value still cannot override an explicitly
configured gateway (that path returns the operator's target verbatim and
is not user-derived)."""
assert vertex_target_for_location("https://gw.internal", "169.254.169.254#") == (
"https://gw.internal"
)