mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): judge NAT64 upstream addresses by their embedded IPv4 (#4179)
## Description On a DNS64 network the SSRF guard rejects every client-supplied upstream whose host has only an A record, so `x-headroom-base-url` is silently ignored. In the OpenAI handler, the request then goes to the configured upstream (`api.openai.com` by default) with the custom gateway's API key. The user gets OpenAI's 401 for a key that was never meant for OpenAI. I hit this while verifying #4164 on Windows with OpenCode 2.0.26. That issue is already fixed by #3999 and isn't touched here. A custom `@ai-sdk/openai-compatible` provider pointed at `https://api.deepseek.com/v1` was routed correctly by the plugin, but the proxy logged `ignoring unsafe x-headroom-base-url override: 'https://api.deepseek.com'`. On this network the resolver answers `api.deepseek.com` with `3.173.21.63` plus the synthesized `64:ff9b::3ad:153f`. Root cause: #3195 added NAT64 handling so the embedded IPv4 is "extracted and judged on its own", but it runs after the category checks. `ipaddress` reports every address in `64:ff9b::/96` as `is_reserved`, because the prefix sits in `::/8`. So `_is_internal_address` returned before the NAT64 branch could run, and the branch was unreachable. The existing `NAT64 embedding loopback` case passed only because the whole prefix was rejected. On `main`: ```text 64:ff9b::808:808 reserved True private False global True literal 64:ff9b::808:808 (8.8.8.8 via NAT64) -> False literal 8.8.8.8 -> True ``` ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/upstream_guard.py`: unwrap the RFC 6052 well-known prefix (`64:ff9b::/96`) at the top of `_is_internal_address`, and judge the embedded IPv4 by the same rules. `64:ff9b::7f00:1`, `64:ff9b::a00:1` and `64:ff9b::a9fe:a9fe` stay blocked; `64:ff9b::808:808` is allowed. The unreachable NAT64 branch after the `is_global` test is removed. - The RFC 8215 local-use prefix (`64:ff9b:1::/48`) is deliberately not unwrapped. Where its IPv4 bits sit depends on the operator's prefix length, so the low 32 bits are not reliably the destination. It stays rejected as non-global, which is what already happened in practice. - `tests/test_upstream_guard.py`: - a NAT64-embedded public IPv4 is added to `test_allows_public`; - NAT64-embedded RFC1918 and metadata addresses, and a local-use address embedding a public IPv4, are added to the blocked cases; - a new test resolves a hostname to a DNS64 answer (A plus synthesized AAAA) and checks that it is accepted with both addresses pinned. - No docs change: `security-model.mdx` already says "NAT64-embedded IPv4 is unwrapped before the check". This makes that true. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - run on the changed files - [x] Type checking passes (`mypy headroom`) - run on the changed module - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # new and changed cases (-k "allows_public or NAT64 or dns64"), fix not applied FAILED tests/test_upstream_guard.py::test_allows_public[https://[64:ff9b::808:808]/v1] FAILED tests/test_upstream_guard.py::test_a_dns64_answer_for_a_public_host_is_accepted_and_pinned 2 failed, 8 passed, 51 deselected # same selection, with the fix 10 passed, 51 deselected # mutations of the fix: each must fail at least one test M1 drop the unwrap KILLED (2 tests) M2 unwrap but always allow KILLED (3 tests) M3 also unwrap the local-use /48 KILLED (1 test) M4 judge the IPv6 form, not the embedded KILLED (5 tests) # test_upstream_guard, test_provider_proxy_routes, test_provider_proxy_targets, # test_provider_registry_extended, test_proxy_openai_cache_stability, # test_proxy_semantic_cache_key_integration main: 4 failed, 116 passed this PR: 4 failed, 121 passed ruff 0.16.8 check / format --check (changed files): All checks passed! / 2 files already formatted mypy 1.19.1 headroom/proxy/upstream_guard.py: no errors in the changed module ``` The 4 failures are the same tests on `main` and on this branch. They are specific to my machine: `socksio` is not installed, the global starlette `TestClient` rejects `client=`, and Python is 3.11.9. The 7 mypy errors are the Windows-only `os.fchmod` / `os.O_*` ones in `fileperms.py` and `astgrep.py`, identical on `main`. ## Real Behavior Proof - Environment: Windows 11, Python 3.11.9, on a network whose resolver does DNS64 (`api.deepseek.com` -> `3.173.21.63` plus the synthesized `64:ff9b::3ad:153f`). OpenCode `@opencode/cli` 2.0.26 with an isolated HOME and a custom `@ai-sdk/openai-compatible` provider `mygw` (`baseURL: https://api.deepseek.com/v1`, fake key). Headroom from `main` (976aa714; `upstream_guard.py` is identical at0126fba6) and from this branch (7decb067). - Exact command / steps: `HEADROOM_OFFLINE=1 headroom wrap opencode --port 8799 --no-serena --no-mcp -v -- run -m mygw/deepseek-chat "reply with the single word ok"`, then read `~/.headroom/logs/proxy-8799.log`. `HEADROOM_OFFLINE` only stops model downloads; the guard's verdict for `https://api.deepseek.com` is `False` on `main` with or without it. - Observed result: on `main` the proxy logs `ignoring unsafe x-headroom-base-url override: 'https://api.deepseek.com'`, then `outbound_request forwarder=streaming method=POST path=https://api.openai.com/v1/chat/completions`, and OpenCode prints OpenAI's `Incorrect API key provided: sk-fake-***********************heck. You can find your API key at https://platform.openai.com/account/api-keys.` With this branch there is no warning, the proxy logs `outbound_request forwarder=streaming method=POST path=https://api.deepseek.com/v1/chat/completions`, and OpenCode prints DeepSeek's own `Authentication Fails, Your api key: ****heck is invalid`, the expected 401 for a fake key. - Not tested: Linux or macOS on a live DNS64 network; an IPv6-only network where the synthesized address is the only route (this network is dual-stack, so the A record was also pinned); deployments that use the RFC 8215 local-use prefix or an operator-specific NAT64 prefix. ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: N/A - Stable/default behavior changed: Narrowly. A client-supplied upstream that resolves into `64:ff9b::/96` with a public embedded IPv4 is now accepted instead of rejected. Every other verdict is unchanged. - Kill switch / disable path: None added. `HEADROOM_ALLOWED_BASE_URLS` still restricts client-supplied upstreams to listed hosts. - Unsafe override required: No - Qualification impact: None - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A: the security-model page already describes this behavior) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - Why the order matters: `is_reserved` has to stay for the rest of `::/8`, so the well-known prefix is unwrapped before the category checks instead of exempting it from them. - Out of scope: when an override is rejected, the request still goes to the configured upstream with the caller's key. That is the documented "ignored rather than fatal" behavior from #3195, and this PR doesn't change it. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4179" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JD Davis
parent
9e5dac7c74
commit
486afaac74
@@ -256,19 +256,21 @@ def destination_listed(
|
||||
return (scheme, host.lower(), port) in origins
|
||||
|
||||
|
||||
# RFC 6052 / RFC 8215: these IPv6 prefixes embed an IPv4 address in their low
|
||||
# 32 bits, and `ipaddress` reports the well-known one as globally routable. On a
|
||||
# NAT64 network `64:ff9b::7f00:1` reaches 127.0.0.1, so the embedded address is
|
||||
# what has to be judged. 6to4, Teredo and IPv4-mapped forms are already caught
|
||||
# by the `is_global` test below.
|
||||
_NAT64_PREFIXES = (
|
||||
ipaddress.IPv6Network("64:ff9b::/96"),
|
||||
ipaddress.IPv6Network("64:ff9b:1::/48"),
|
||||
)
|
||||
# RFC 6052 well-known prefix. A DNS64 resolver answers a host that has only an
|
||||
# A record with `64:ff9b::<its IPv4>`, and the NAT64 translator delivers that to
|
||||
# the IPv4 address in the low 32 bits, so the embedded address is what has to
|
||||
# be judged: `64:ff9b::7f00:1` reaches 127.0.0.1. It has to be judged *before*
|
||||
# the category checks, because `ipaddress` calls the whole prefix reserved (it
|
||||
# sits in ::/8) -- judging the IPv6 form rejected every IPv4-only upstream on a
|
||||
# DNS64 network. The local-use prefix (RFC 8215, 64:ff9b:1::/48) is not
|
||||
# unwrapped: where its IPv4 bits sit depends on the operator's prefix length,
|
||||
# so it stays rejected as non-global. 6to4, Teredo and IPv4-mapped forms are
|
||||
# already caught by the `is_global` test below.
|
||||
_NAT64_WELL_KNOWN_PREFIX = ipaddress.IPv6Network("64:ff9b::/96")
|
||||
|
||||
|
||||
def _nat64_embedded_ipv4(addr: ipaddress.IPv6Address) -> ipaddress.IPv4Address | None:
|
||||
if not any(addr in prefix for prefix in _NAT64_PREFIXES):
|
||||
if addr not in _NAT64_WELL_KNOWN_PREFIX:
|
||||
return None
|
||||
try:
|
||||
return ipaddress.IPv4Address(int(addr) & 0xFFFFFFFF)
|
||||
@@ -281,6 +283,10 @@ def _is_internal_address(ip: str) -> bool:
|
||||
addr = ipaddress.ip_address(ip)
|
||||
except ValueError:
|
||||
return True # unparseable (e.g. scoped link-local) -> treat as unsafe
|
||||
if isinstance(addr, ipaddress.IPv6Address):
|
||||
embedded = _nat64_embedded_ipv4(addr)
|
||||
if embedded is not None:
|
||||
return _is_internal_address(str(embedded))
|
||||
if (
|
||||
addr.is_private
|
||||
or addr.is_loopback
|
||||
@@ -298,10 +304,6 @@ def _is_internal_address(ip: str) -> bool:
|
||||
# learns about.
|
||||
if not addr.is_global:
|
||||
return True
|
||||
if isinstance(addr, ipaddress.IPv6Address):
|
||||
embedded = _nat64_embedded_ipv4(addr)
|
||||
if embedded is not None and _is_internal_address(str(embedded)):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
|
||||
@@ -42,7 +42,16 @@ def test_blocks_internal_and_invalid(url: str) -> None:
|
||||
assert is_safe_upstream_url(url) is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", ["https://8.8.8.8/v1", "https://1.1.1.1/", "wss://9.9.9.9/rt"])
|
||||
@pytest.mark.parametrize(
|
||||
"url",
|
||||
[
|
||||
"https://8.8.8.8/v1",
|
||||
"https://1.1.1.1/",
|
||||
"wss://9.9.9.9/rt",
|
||||
# NAT64 well-known prefix embedding 8.8.8.8: the translator delivers it there.
|
||||
"https://[64:ff9b::808:808]/v1",
|
||||
],
|
||||
)
|
||||
def test_allows_public(url: str) -> None:
|
||||
assert is_safe_upstream_url(url) is True
|
||||
|
||||
@@ -63,7 +72,12 @@ def test_allows_public(url: str) -> None:
|
||||
("6to4 embedding loopback", "http://[2002:7f00:1::]/"),
|
||||
("6to4 embedding RFC1918", "http://[2002:a00:1::]/"),
|
||||
("NAT64 embedding loopback", "http://[64:ff9b::7f00:1]/"),
|
||||
("NAT64 embedding RFC1918", "http://[64:ff9b::a00:1]/"),
|
||||
("NAT64 embedding metadata", "http://[64:ff9b::a9fe:a9fe]/"),
|
||||
("NAT64 local-use prefix", "http://[64:ff9b:1::7f00:1]/"),
|
||||
# Local-use embedding depends on the operator's prefix length, so the
|
||||
# low 32 bits are not reliably the destination: never unwrapped.
|
||||
("NAT64 local-use prefix embedding a public IPv4", "http://[64:ff9b:1::808:808]/"),
|
||||
("teredo", "http://[2001:0::7f00:1]/"),
|
||||
("IPv4-mapped metadata", "http://[::ffff:169.254.169.254]/"),
|
||||
("IPv4-mapped loopback", "http://[::ffff:127.0.0.1]/"),
|
||||
@@ -582,6 +596,33 @@ def test_a_check_pins_every_address_it_accepted(monkeypatch: pytest.MonkeyPatch)
|
||||
assert upstream_guard.validated_addresses("multi.homed.example") == ("8.8.8.8", "1.1.1.1")
|
||||
|
||||
|
||||
def test_a_dns64_answer_for_a_public_host_is_accepted_and_pinned(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""On a DNS64 network an IPv4-only host also gets a synthesized `64:ff9b::` AAAA.
|
||||
|
||||
Judging that AAAA as reserved rejected every IPv4-only upstream there, and
|
||||
the OpenAI handler then sent the request -- key included -- to the
|
||||
configured upstream instead.
|
||||
"""
|
||||
from headroom.proxy import upstream_guard
|
||||
|
||||
def dns64_answer(*args: object, **kwargs: object) -> list[object]:
|
||||
return [
|
||||
(None, None, None, None, ("8.8.8.8", 443)),
|
||||
(None, None, None, None, ("64:ff9b::808:808", 443, 0, 0)),
|
||||
]
|
||||
|
||||
monkeypatch.setattr(socket, "getaddrinfo", dns64_answer)
|
||||
upstream_guard.clear_validated_addresses()
|
||||
|
||||
assert is_safe_upstream_url("https://ipv4-only.example/v1") is True
|
||||
assert upstream_guard.validated_addresses("ipv4-only.example") == (
|
||||
"8.8.8.8",
|
||||
"64:ff9b::808:808",
|
||||
)
|
||||
|
||||
|
||||
def test_a_rejected_or_allowlisted_destination_pins_nothing(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
|
||||
Reference in New Issue
Block a user