mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description
Two routes in the Rust front (`crates/headroom-proxy`) attach the host's
own cloud credentials to whatever request reaches them, without checking
the caller:
- **Bedrock native** (`/model/{id}/invoke`, `/converse`,
`/invoke-with-response-stream`, `/converse-stream`) is on by default and
signs every request with SigV4 using the AWS credentials found on the
host.
- **Vertex** (`…/publishers/anthropic/models/{m}:rawPredict` /
`:streamRawPredict`) is mounted unconditionally, replaces the caller's
`Authorization` with a bearer from the host's Google ADC, and forwards
to the generic `--upstream`, whatever host that is. So with a non-Google
upstream (the Python proxy, another provider) the host's Google token is
sent there. `--vertex-adc-scope` was parsed but never used.
This PR makes both routes explicit opt-ins, only sends the ADC bearer to
a Google API host, and uses the configured scope.
**Dependency:** these routes still do not authenticate callers
themselves. Callers are expected to be authenticated by the inbound
`HEADROOM_PROXY_TOKEN` gate added in #4840 (which also moves the default
listen address to loopback). This PR is based on `main` and does not
require #4840 to merge first, but operators who enable either route on a
non-loopback address need #4840 to restrict who can use them.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `config.rs`: `--enable-bedrock-native` /
`HEADROOM_PROXY_ENABLE_BEDROCK_NATIVE` now defaults to `false`. New
`--enable-vertex-native` / `HEADROOM_PROXY_ENABLE_VERTEX_NATIVE`
(default `false`) and `--vertex-allow-non-google-upstream` /
`HEADROOM_PROXY_VERTEX_ALLOW_NON_GOOGLE_UPSTREAM` (default `false`).
`Config::for_test` keeps both routes on so the existing route tests run
unchanged.
- `proxy.rs`: the Vertex route is mounted only when enabled; with it
off, Vertex paths go through the catch-all with the caller's own headers
(as the Python proxy's Vertex passthrough expects). The ADC token source
is built with `--vertex-adc-scope`. The "Bedrock route not mounted"
startup line is now `info`, since off is the default.
- `vertex/raw_predict.rs`: before fetching the ADC bearer, the handler
refuses (502 `vertex upstream is not a Google API host`,
`vertex_upstream_not_google` error log) unless `--upstream` is
`https://` on a `*.googleapis.com` host (regional, global and
`*.p.googleapis.com` endpoints), or the operator set
`--vertex-allow-non-google-upstream=true` (local emulators, gateways).
No token is fetched on that path.
- `RUST_DEV.md`: the three flags in the configuration table, with what
enabling them means.
**Behaviour change for operators:** deployments that used the Rust
front's native Bedrock or Vertex signing must now pass
`--enable-bedrock-native true` / `--enable-vertex-native true` (or the
env vars). Vertex with a non-Google `--upstream` additionally needs
`--vertex-allow-non-google-upstream true`. Without the flags, those
paths are forwarded to `--upstream` unsigned with the client's own
credentials (Claude Code in Vertex mode keeps sending its own ADC token,
which is what `docs/content/docs/claude-code-vertex.mdx` describes).
`scripts/qualify_cargo_dependencies.py` already passes
`--enable-bedrock-native true`.
## Testing
- [x] `cargo test -p headroom-proxy` (45 test binaries, 596 tests pass)
- [x] `cargo clippy -p headroom-proxy --all-targets -- -D warnings` and
`cargo fmt --all -- --check` clean
- [x] New tests added for new functionality
- [x] Manual testing performed
- [ ] `pytest` / `ruff` / `mypy`: N/A, no Python changes
New tests in `tests/integration_cloud_routes_opt_in.rs` build the config
from the real CLI defaults (`CliArgs` → `Config::from_cli`), give the
app host AWS credentials and a host ADC bearer, and record what a
`wiremock` upstream receives:
- `default_config_does_not_sign_bedrock_requests`: no `AWS4-HMAC-SHA256`
Authorization on `invoke`, `invoke-with-response-stream`, `converse`.
- `default_config_never_attaches_host_adc_bearer`: `rawPredict` and
`streamRawPredict` reach the upstream without the host bearer.
- `vertex_bearer_is_not_sent_to_a_non_google_upstream`: with the route
enabled and an `http://127.0.0.1` upstream, 502 and the upstream sees
nothing.
- `vertex_opt_in_still_signs_when_non_google_upstream_is_allowed`: the
explicit escape hatch still works.
- `configured_adc_scope_reaches_token_source`.
- Unit test `vertex::raw_predict::tests::google_api_host_check`
(`http://`, `googleapis.com.example.net`, `example-googleapis.com`,
loopback are rejected).
### Test Output
Before (the three tests that compile against `main`, run with this PR's
`src/` changes stashed):
```text
test configured_adc_scope_reaches_token_source ... FAILED
test default_config_never_attaches_host_adc_bearer ... FAILED
test default_config_does_not_sign_bedrock_requests ... FAILED
test result: FAILED. 0 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.77s
```
After:
```text
$ cargo test -p headroom-proxy --test integration_cloud_routes_opt_in
test configured_adc_scope_reaches_token_source ... ok
test vertex_opt_in_still_signs_when_non_google_upstream_is_allowed ... ok
test vertex_bearer_is_not_sent_to_a_non_google_upstream ... ok
test default_config_never_attaches_host_adc_bearer ... ok
test default_config_does_not_sign_bedrock_requests ... ok
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.69s
```
## Real Behavior Proof
- Environment: macOS (Darwin 27.2), debug builds of `headroom-proxy`
from `origin/main` (c21b0813) and this branch. AWS credentials supplied
to the proxy through `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` (fake
values), IMDS disabled. Upstream and Bedrock endpoint are a small Python
recorder on `127.0.0.1:18788` that prints the `Authorization` it
receives.
- Exact command / steps: `headroom-proxy --listen 127.0.0.1:18787
--upstream http://127.0.0.1:18788 --bedrock-endpoint
http://127.0.0.1:18788` (no Bedrock/Vertex flags), then `curl -d
'<anthropic body>'
http://127.0.0.1:18787/model/anthropic.claude-3-haiku-20240307-v1:0/invoke`
with no client credentials. Repeated on this branch with
`--enable-bedrock-native true`.
- Observed result (key id and signature redacted):
```text
== origin/main (no flags)
POST /model/anthropic.claude-3-haiku-20240307-v1:0/invoke, no client credentials -> 200
UPSTREAM SAW {"path": "/model/...:0/invoke", "authorization": "AWS4-HMAC-SHA256 Credential=<redacted>/20261113/us-east-1/bedrock/aws4_request, SignedHeaders=..., Signature=<sig>"}
== this branch (no flags)
POST /model/anthropic.claude-3-haiku-20240307-v1:0/invoke, no client credentials -> 200
UPSTREAM SAW {"path": "/model/...:0/invoke", "authorization": null}
== this branch, --enable-bedrock-native true
UPSTREAM SAW {"path": "/model/...:0/invoke", "authorization": "AWS4-HMAC-SHA256 Credential=<redacted>/20261113/us-east-1/bedrock/aws4_request, SignedHeaders=..., Signature=<sig>"}
```
- Not tested: Vertex against real Google ADC and a real
`*.googleapis.com` upstream (covered by the integration tests with a
static token source; no GCP credentials on the test machine).
## Runtime Rollout Safety
- Rollout-managed feature(s): `native_bedrock` (existing). The CLI flag
already maps `false` to a rollout disable, so with the new default the
feature reports disabled unless `--enable-bedrock-native true` is
passed. Vertex is not rollout-managed.
- Minimum rollout channel: unchanged
- Stable/default behavior changed: yes, both native routes are off by
default
- Kill switch / disable path: the routes are now off unless enabled
- Unsafe override required: no
- Qualification impact: none, the qualification script already passes
`--enable-bedrock-native true`
- Rollback path: pass `--enable-bedrock-native true` /
`--enable-vertex-native true` (plus `--vertex-allow-non-google-upstream
true` if the upstream is not a Google host), or revert this PR
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
Fix adapted from a proposal by HetCreep.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4851"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Reported by @JoaoMarcos44. Thank you for your contribution! 🩵
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>