mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): key rate limits by peer unless the caller authenticated (#3860)
## Why The proxy's request and token rate limits could be walked around by any caller the proxy cannot authenticate (finding 01-F3). Each provider handler keyed its buckets on a client-supplied header: - **OpenAI:** an HMAC of the full `Authorization` / `api-key` value, else one shared `"default"` bucket. Rotating the header gave a fresh bucket per request, and all callers without a credential shared one bucket. - **Anthropic:** the first 16 characters of the key plus the peer. That is mostly the common `sk-ant-api03-` prefix. - **Gemini:** the first 20 characters of `x-goog-api-key`, with no peer at all. ## What changes - New `headroom/proxy/rate_limit_identity.py` with one rule for the Anthropic, OpenAI and Gemini handlers. `rate_limit_identity()` returns a plain string key: - Every key starts with the peer from `resolve_client_ip`, which honours `X-Forwarded-For` only from `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` peers. IPv6 peers are grouped by `/64`; an IPv4-mapped IPv6 address counts as the IPv4 peer. - **Trusted** requests get one bucket per provider credential: `peer:<addr>|cred:<hmac>`, an HMAC of the whole credential with a process-local key, so bucket names hold no credential material. A request is trusted if it presented a valid `HEADROOM_PROXY_TOKEN` or came directly from loopback. A trusted-gateway peer is trusted to supply the client address, not to vouch that the caller authenticated, so a forwarded request without the token is untrusted and keyed by the forwarded client address. A request from a gateway address that carries no `X-Forwarded-For` is judged as a direct request, so a loopback gateway CIDR does not change how direct localhost callers are keyed. Distinct principals behind one address keep separate limits, as #3364 intended. - **Untrusted** requests (no token, remote caller) are charged per peer (`peer:<addr>`). The credential header is ignored. - The security gate sets `request.state.proxy_authenticated = True` when a caller presents a valid proxy token. - The per-handler key code is removed (`_openai_rate_limit_key` and the Anthropic and Gemini inline keys). `rate_limiter.py` is unchanged, and so are token-limit semantics. ## Behaviour change Requests the proxy cannot authenticate (no `HEADROOM_PROXY_TOKEN`, remote caller) are now limited per client address. Changing the API-key header no longer gives a new allowance. Authenticated and direct-loopback callers keep one allowance per provider credential. Behind a gateway, set `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` so each forwarded client address gets its own limit; to keep one allowance per credential there, callers must also send the proxy token. The limiter is quota smoothing and abuse control for unauthenticated callers. It is not a security boundary for authenticated callers, and this PR does not make it one. ## Tests - `tests/test_proxy/test_rate_limit_identity.py`: untrusted keys ignore the credential; trusted keys are per credential and include the peer; the whole credential is used, not a prefix; bucket names carry no credential material; loopback is trusted; a request relayed by a trusted-gateway peer (including one on loopback) is not trusted without the proxy token; with `127.0.0.0/8` configured as a gateway, direct loopback callers stay per credential, and a relayed request with an unusable `X-Forwarded-For` still counts as relayed; IPv6 `/64` grouping; IPv4-mapped addresses. - `tests/test_proxy_openai_rate_limit_key.py`, end to end through `create_app` with `--rpm 1`: authenticated and loopback callers with distinct keys do not share a bucket; an unauthenticated remote caller who rotates `api-key` or `Authorization`, or drops the credential, gets 429 on the second request; two different unauthenticated peers do not share a bucket; through a trusted gateway (`10.0.0.2`, fixed `X-Forwarded-For: 203.0.113.9`) a caller rotating `api-key` gets 429, distinct forwarded clients do not share, and token-authenticated callers stay per credential; with `127.0.0.0/8` as the gateway CIDR, two direct loopback callers with different keys both pass while a caller relayed through it with a fixed forwarded address gets 429 on the second key. The unauthenticated cases fail on `main`. - Existing rate-limiter and TPM suites pass unchanged. Manual run: `headroom proxy --host 0.0.0.0 --rpm 2` with six requests from a LAN address, each with a different `Authorization` key. On `main` all six reach the upstream. On this branch without a proxy token, the third and later get 429. With the proxy token sent, all six pass, one bucket per credential. Not tested: the Rust `headroom-proxy` binary, which has its own limiter. ## Merge order / conflicts Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 → #3863. #3852 makes the untrusted case rare by refusing token-less non-loopback binds; this PR makes it safe when an operator opts out of that. Textual conflict: #3862 adds an import next to the one this PR adds in `handlers/anthropic.py` and `handlers/openai.py`. Whichever merges second keeps both imports. Revised 2026-10-01 after self-review: `rate_limiter.py` is back to `main`; the separate trusted/untrusted pools, per-owner cap and overflow bucket were removed. Revised 2026-10-02 after review: gateway membership only drops the credential from the key when the gateway actually forwarded a client address, so direct localhost callers keep per-credential buckets under a loopback gateway CIDR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3860" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JD Davis
parent
3fdf18eded
commit
6fb7cadfe4
@@ -51,7 +51,6 @@ from headroom.proxy.buffered_ccr_response import (
|
||||
buffered_ccr_asgi_call,
|
||||
)
|
||||
from headroom.proxy.compression_decision import CompressionDecision
|
||||
from headroom.proxy.forwarded_headers import resolve_client_ip
|
||||
from headroom.proxy.handlers._debug_dump import _debug_dump_mode, _redact_debug_value
|
||||
from headroom.proxy.helpers import (
|
||||
extract_tags,
|
||||
@@ -66,6 +65,7 @@ from headroom.proxy.model_router import estimate_input_tokens
|
||||
from headroom.proxy.nonstream_sse_policy import should_recover_sse_reply
|
||||
from headroom.proxy.outcome import RequestOutcome
|
||||
from headroom.proxy.output_shaper import shaper_enabled_for, steering_allowed_for
|
||||
from headroom.proxy.rate_limit_identity import rate_limit_identity
|
||||
from headroom.proxy.tenant_key import resolve_tenant_key, set_request_tenant_key
|
||||
from headroom.proxy.thinking_tokens import ThinkingTokens, extract_thinking_tokens
|
||||
from headroom.utils import format_exception_message
|
||||
@@ -1318,18 +1318,10 @@ class AnthropicHandlerMixin:
|
||||
|
||||
# Rate limiting
|
||||
if self.rate_limiter:
|
||||
api_key = headers.get("x-api-key", "")
|
||||
if not api_key:
|
||||
auth = headers.get("authorization", "")
|
||||
if auth.startswith("Bearer "):
|
||||
api_key = auth[7:]
|
||||
# Phase F PR-F4: trust ``X-Forwarded-For`` for the rate-limit
|
||||
# key only when the connecting peer is in
|
||||
# ``HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS``; otherwise we use
|
||||
# the direct peer IP and a malicious client cannot rotate
|
||||
# rate-limit buckets by forging headers.
|
||||
client_ip = resolve_client_ip(request) or "unknown"
|
||||
rate_key = f"{api_key[:16]}:{client_ip}" if api_key else client_ip
|
||||
# One identity rule for every provider: peer-owned, and
|
||||
# credential-scoped only for proxy-token / direct loopback
|
||||
# callers (headroom/proxy/rate_limit_identity.py).
|
||||
rate_key = rate_limit_identity(request, headers)
|
||||
allowed, wait_seconds = await self.rate_limiter.check_request(rate_key)
|
||||
if not allowed:
|
||||
await self.metrics.record_rate_limited(
|
||||
|
||||
@@ -24,6 +24,7 @@ from headroom.proxy.compression_decision import CompressionDecision
|
||||
from headroom.proxy.helpers import COMPRESSION_TIMEOUT_SECONDS, extract_tags
|
||||
from headroom.proxy.identity import resolve_memory_identity
|
||||
from headroom.proxy.outcome import RequestOutcome
|
||||
from headroom.proxy.rate_limit_identity import rate_limit_identity
|
||||
from headroom.proxy.token_counting import gemini_output_tokens
|
||||
|
||||
logger = logging.getLogger("headroom.proxy")
|
||||
@@ -441,9 +442,10 @@ class GeminiHandlerMixin:
|
||||
)
|
||||
memory_decision.apply_to_tags(tags)
|
||||
|
||||
# Rate limiting (use Gemini API key)
|
||||
# Rate limiting: one identity rule for every provider
|
||||
# (headroom/proxy/rate_limit_identity.py).
|
||||
if self.rate_limiter:
|
||||
rate_key = headers.get("x-goog-api-key", "default")[:20]
|
||||
rate_key = rate_limit_identity(request, headers)
|
||||
allowed, wait_seconds = await self.rate_limiter.check_request(rate_key)
|
||||
if not allowed:
|
||||
await self.metrics.record_rate_limited(provider=provider_name, source="headroom")
|
||||
|
||||
@@ -10,12 +10,10 @@ import contextlib
|
||||
import contextvars
|
||||
import copy
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import ssl
|
||||
import threading
|
||||
import time
|
||||
@@ -38,6 +36,7 @@ from headroom.proxy.helpers import (
|
||||
from headroom.proxy.identity import resolve_memory_identity
|
||||
from headroom.proxy.loopback_guard import is_loopback_host
|
||||
from headroom.proxy.modes import is_cache_mode
|
||||
from headroom.proxy.rate_limit_identity import rate_limit_identity
|
||||
from headroom.proxy.stage_timer import StageTimer, emit_stage_timings_log
|
||||
from headroom.proxy.upstream_guard import is_safe_upstream_url
|
||||
from headroom.proxy.ws_headers import WS_HOP_BY_HOP_HEADERS
|
||||
@@ -129,30 +128,6 @@ _CCR_HASH_RE = re.compile(
|
||||
r"(?:Retrieve (?:more|original): hash=|<<ccr:)([a-fA-F0-9]{12,24})(?=[^a-fA-F0-9]|$)"
|
||||
)
|
||||
_BARE_CCR_HASH_RE = re.compile(r"[a-fA-F0-9]{12,24}")
|
||||
_OPENAI_RATE_KEY_SECRET = secrets.token_bytes(32)
|
||||
|
||||
|
||||
def _openai_rate_limit_key(headers: dict[str, str]) -> str:
|
||||
"""Return the credential identity used by the OpenAI rate limiter.
|
||||
|
||||
OpenAI-compatible gateways may authenticate with either a bearer token or
|
||||
an ``api-key`` header. Deriving an identity from the complete value keeps common
|
||||
prefixes distinct without retaining recoverable credential material in
|
||||
bucket keys. The secret is process-local, like the limiter state.
|
||||
Requests without either retain the existing shared fallback bucket.
|
||||
"""
|
||||
authorization = headers.get("authorization")
|
||||
api_key = headers.get("api-key")
|
||||
if authorization:
|
||||
kind, credential = "authorization", authorization
|
||||
elif api_key:
|
||||
kind, credential = "api-key", api_key
|
||||
else:
|
||||
return "default"
|
||||
# API credentials are identifiers, not passwords to verify. A process-keyed
|
||||
# HMAC keeps bucket keys opaque without a password KDF on the request path.
|
||||
digest = hmac.digest(_OPENAI_RATE_KEY_SECRET, f"{kind}:{credential}".encode(), "sha256").hex()
|
||||
return f"{kind}:{digest}"
|
||||
|
||||
|
||||
def _response_ccr_hashes(messages: list[dict[str, Any]], markers: list[str]) -> list[str]:
|
||||
@@ -3874,7 +3849,7 @@ class OpenAIHandlerMixin:
|
||||
|
||||
# Rate limiting
|
||||
if self.rate_limiter:
|
||||
rate_key = _openai_rate_limit_key(headers)
|
||||
rate_key = rate_limit_identity(request, headers)
|
||||
allowed, wait_seconds = await self.rate_limiter.check_request(rate_key)
|
||||
if not allowed:
|
||||
await self.metrics.record_rate_limited(
|
||||
@@ -6219,7 +6194,7 @@ class OpenAIHandlerMixin:
|
||||
|
||||
# Rate limiting
|
||||
if self.rate_limiter:
|
||||
rate_key = _openai_rate_limit_key(headers)
|
||||
rate_key = rate_limit_identity(request, headers)
|
||||
allowed, wait_seconds = await self.rate_limiter.check_request(rate_key)
|
||||
if not allowed:
|
||||
await self.metrics.record_rate_limited(provider="openai", source="headroom")
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Who a request is charged to, for the proxy's request/token rate limits.
|
||||
|
||||
One rule for every provider handler (Anthropic, OpenAI, Gemini). Before this
|
||||
module each handler keyed its buckets differently, and every rule could be
|
||||
walked around (VAPT finding 01-F3):
|
||||
|
||||
* OpenAI keyed on an HMAC of the full ``Authorization`` / ``api-key`` value, so
|
||||
a caller rotating that header got a fresh bucket per request; callers with no
|
||||
credential all shared one ``"default"`` bucket.
|
||||
* Anthropic keyed on the first 16 characters of the key plus the peer address —
|
||||
mostly the common ``sk-ant-api03-`` prefix.
|
||||
* Gemini keyed on the first 20 characters of ``x-goog-api-key`` with no peer.
|
||||
|
||||
The rule now:
|
||||
|
||||
1. Every identity is **owned by the peer** — ``resolve_client_ip``, which honours
|
||||
``X-Forwarded-For`` only from ``HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS`` peers.
|
||||
IPv6 peers are grouped by ``/64`` so rotating addresses inside one
|
||||
allocation does not rotate buckets.
|
||||
2. For a **trusted** request the provider credential is part of the bucket, so
|
||||
distinct principals behind one address (a team gateway, a NAT) do not share a
|
||||
limit — decision D2, one proxy is shared by several principals. A request is
|
||||
trusted when it presented the proxy token (the security gate records that on
|
||||
``request.state``) or came directly from loopback. A trusted-gateway peer is
|
||||
trusted to report the caller's address, not to vouch that the caller
|
||||
authenticated, so forwarded requests without the token are untrusted. A
|
||||
direct request from a gateway address that forwards no client address is
|
||||
judged like any other direct request.
|
||||
3. For an **untrusted** request (no token configured and a remote caller) the
|
||||
credential is ignored: the bucket is the peer. Rotating a header cannot mint
|
||||
a new bucket for someone the proxy cannot authenticate.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
import ipaddress
|
||||
import secrets
|
||||
from typing import Any
|
||||
|
||||
from headroom.proxy.forwarded_headers import load_trusted_gateway_cidrs, resolve_client_ip
|
||||
from headroom.proxy.forwarded_policy import peer_is_trusted_gateway
|
||||
from headroom.proxy.loopback_guard import is_loopback_host
|
||||
|
||||
# Set by the security gate when a caller presented a valid HEADROOM_PROXY_TOKEN.
|
||||
PROXY_AUTHENTICATED_STATE_ATTR = "proxy_authenticated"
|
||||
|
||||
# Checked in order; the first present header is the caller's provider credential.
|
||||
_CREDENTIAL_HEADERS = ("authorization", "x-api-key", "api-key", "x-goog-api-key")
|
||||
|
||||
# Process-local key: bucket names never contain recoverable credential material.
|
||||
_IDENTITY_SECRET = secrets.token_bytes(32)
|
||||
|
||||
|
||||
def _peer_group(ip: str) -> str:
|
||||
"""Normalise a peer address; IPv6 is grouped by its /64."""
|
||||
if not ip:
|
||||
return "peer:unknown"
|
||||
try:
|
||||
addr = ipaddress.ip_address(ip)
|
||||
except ValueError:
|
||||
return f"peer:{ip[:64]}"
|
||||
if isinstance(addr, ipaddress.IPv6Address):
|
||||
if addr.ipv4_mapped is not None:
|
||||
return f"peer:{addr.ipv4_mapped}"
|
||||
network = ipaddress.IPv6Network((int(addr) >> 64 << 64, 64))
|
||||
return f"peer:{network}"
|
||||
return f"peer:{addr}"
|
||||
|
||||
|
||||
def _credential(headers: Any) -> str | None:
|
||||
for name in _CREDENTIAL_HEADERS:
|
||||
try:
|
||||
value = headers.get(name)
|
||||
except AttributeError:
|
||||
return None
|
||||
if value:
|
||||
return f"{name}:{value}"
|
||||
return None
|
||||
|
||||
|
||||
def _relayed_by_trusted_gateway(request: Any) -> bool:
|
||||
"""True when a trusted-gateway peer forwarded a client address for this request.
|
||||
|
||||
Any ``X-Forwarded-For`` value counts, even one the forwarded-header policy
|
||||
cannot use, so a relayed request is never mistaken for a direct one.
|
||||
"""
|
||||
client = getattr(request, "client", None)
|
||||
host = getattr(client, "host", None) if client is not None else None
|
||||
if not host or not peer_is_trusted_gateway(host, load_trusted_gateway_cidrs()):
|
||||
return False
|
||||
headers = getattr(request, "headers", None)
|
||||
return bool(headers is not None and headers.get("x-forwarded-for"))
|
||||
|
||||
|
||||
def is_trusted_request(request: Any) -> bool:
|
||||
"""True when the caller presented the proxy token, or connected directly from loopback.
|
||||
|
||||
A request relayed by a trusted gateway (even one on loopback) is keyed by
|
||||
the forwarded client address unless it presented the proxy token: the
|
||||
gateway vouches for that address, not for the caller's credential. A direct
|
||||
request from a loopback gateway address, with no forwarded client address,
|
||||
is still a direct loopback caller.
|
||||
"""
|
||||
state = getattr(request, "state", None)
|
||||
if state is not None and getattr(state, PROXY_AUTHENTICATED_STATE_ATTR, False):
|
||||
return True
|
||||
if _relayed_by_trusted_gateway(request):
|
||||
return False
|
||||
client = getattr(request, "client", None)
|
||||
host = getattr(client, "host", None) if client is not None else None
|
||||
return bool(host and is_loopback_host(host))
|
||||
|
||||
|
||||
def rate_limit_identity(request: Any, headers: Any = None) -> str:
|
||||
"""Return the bucket this request is charged to. See the module docstring."""
|
||||
headers = headers if headers is not None else getattr(request, "headers", {})
|
||||
peer = _peer_group(resolve_client_ip(request) or "")
|
||||
credential = _credential(headers) if is_trusted_request(request) else None
|
||||
if credential is None:
|
||||
return peer
|
||||
digest = hmac.digest(_IDENTITY_SECRET, credential.encode("utf-8", "replace"), "sha256")
|
||||
return f"{peer}|cred:{digest.hex()[:32]}"
|
||||
@@ -4058,6 +4058,10 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
|
||||
rejection = JSONResponse(status_code=401, content={"error": "unauthorized"})
|
||||
_apply_security_headers(rejection)
|
||||
return rejection
|
||||
# The caller proved it holds the proxy token. The rate limiter
|
||||
# keys authenticated callers per credential and everyone else
|
||||
# per peer (headroom/proxy/rate_limit_identity.py).
|
||||
request.state.proxy_authenticated = True
|
||||
|
||||
# The credential has done its job; take it off the request so no handler
|
||||
# can forward it upstream. Runs on the exempt paths too, since a loopback
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
"""Rate-limit identity (VAPT 01-F3)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.proxy.rate_limit_identity import rate_limit_identity
|
||||
|
||||
|
||||
def _request(peer: str, *, authenticated: bool = False, headers=None):
|
||||
return SimpleNamespace(
|
||||
client=SimpleNamespace(host=peer),
|
||||
headers=headers or {},
|
||||
state=SimpleNamespace(proxy_authenticated=authenticated),
|
||||
)
|
||||
|
||||
|
||||
# ── identity ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_untrusted_identity_ignores_the_credential() -> None:
|
||||
a = rate_limit_identity(_request("203.0.113.9", headers={"authorization": "Bearer k1"}))
|
||||
b = rate_limit_identity(_request("203.0.113.9", headers={"x-api-key": "k2"}))
|
||||
c = rate_limit_identity(_request("203.0.113.9"))
|
||||
assert a == b == c == "peer:203.0.113.9"
|
||||
|
||||
|
||||
def test_trusted_identity_is_per_credential_and_owned_by_the_peer() -> None:
|
||||
a = rate_limit_identity(
|
||||
_request("203.0.113.9", authenticated=True, headers={"authorization": "Bearer k1"})
|
||||
)
|
||||
b = rate_limit_identity(
|
||||
_request("203.0.113.9", authenticated=True, headers={"authorization": "Bearer k2"})
|
||||
)
|
||||
assert a != b
|
||||
assert a.startswith("peer:203.0.113.9|cred:") and b.startswith("peer:203.0.113.9|cred:")
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"headers",
|
||||
[
|
||||
{"authorization": "Bearer sk-ant-api03-SAME-PREFIX-A"},
|
||||
{"x-api-key": "sk-ant-api03-SAME-PREFIX-A"},
|
||||
{"x-goog-api-key": "AIzaSyD-SAME-PREFIX-A"},
|
||||
],
|
||||
)
|
||||
def test_trusted_identity_uses_the_whole_credential_not_a_prefix(headers) -> None:
|
||||
other = {k: v[:-1] + "B" for k, v in headers.items()}
|
||||
a = rate_limit_identity(_request("127.0.0.1", headers=headers))
|
||||
b = rate_limit_identity(_request("127.0.0.1", headers=other))
|
||||
assert a != b
|
||||
|
||||
|
||||
def test_bucket_names_carry_no_credential_material() -> None:
|
||||
ident = rate_limit_identity(
|
||||
_request("127.0.0.1", headers={"authorization": "Bearer sk-live-secret-value"})
|
||||
)
|
||||
assert "sk-live" not in ident
|
||||
assert "secret" not in ident
|
||||
|
||||
|
||||
def test_loopback_is_trusted_without_a_token() -> None:
|
||||
keyed = {"authorization": "Bearer k1"}
|
||||
assert rate_limit_identity(_request("127.0.0.1", headers=keyed)) != rate_limit_identity(
|
||||
_request("127.0.0.1")
|
||||
)
|
||||
|
||||
|
||||
def test_trusted_gateway_peer_alone_does_not_authenticate_the_caller(monkeypatch) -> None:
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "10.20.0.0/16")
|
||||
keyed = {"authorization": "Bearer k1"}
|
||||
assert rate_limit_identity(_request("10.20.3.4", headers=keyed)) == "peer:10.20.3.4"
|
||||
assert "|cred:" in rate_limit_identity(_request("10.20.3.4", authenticated=True, headers=keyed))
|
||||
|
||||
|
||||
def test_loopback_gateway_does_not_authenticate_forwarded_callers(monkeypatch) -> None:
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "127.0.0.0/8")
|
||||
keyed = {"authorization": "Bearer k1", "x-forwarded-for": "203.0.113.9"}
|
||||
assert rate_limit_identity(_request("127.0.0.1", headers=keyed)) == "peer:203.0.113.9"
|
||||
|
||||
|
||||
def test_loopback_gateway_config_keeps_direct_loopback_callers_per_credential(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
"""A loopback gateway CIDR only matters for requests that forward a client address."""
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "127.0.0.0/8")
|
||||
a = rate_limit_identity(_request("127.0.0.1", headers={"authorization": "Bearer k1"}))
|
||||
b = rate_limit_identity(_request("127.0.0.1", headers={"authorization": "Bearer k2"}))
|
||||
assert a != b
|
||||
assert a.startswith("peer:127.0.0.1|cred:") and b.startswith("peer:127.0.0.1|cred:")
|
||||
|
||||
|
||||
def test_unusable_forwarded_address_still_counts_as_relayed(monkeypatch) -> None:
|
||||
"""A gateway-relayed request with an empty leftmost hop is not a direct caller."""
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "127.0.0.0/8")
|
||||
keyed = {"authorization": "Bearer k1", "x-forwarded-for": ", 203.0.113.9"}
|
||||
assert "|cred:" not in rate_limit_identity(_request("127.0.0.1", headers=keyed))
|
||||
|
||||
|
||||
def test_ipv6_peers_are_grouped_by_slash_64() -> None:
|
||||
a = rate_limit_identity(_request("2001:db8:1:2::1"))
|
||||
b = rate_limit_identity(_request("2001:db8:1:2:ffff:ffff:ffff:ffff"))
|
||||
c = rate_limit_identity(_request("2001:db8:1:3::1"))
|
||||
assert a == b
|
||||
assert a != c
|
||||
|
||||
|
||||
def test_ipv4_mapped_ipv6_is_the_ipv4_peer() -> None:
|
||||
assert rate_limit_identity(_request("::ffff:203.0.113.9")) == rate_limit_identity(
|
||||
_request("203.0.113.9")
|
||||
)
|
||||
@@ -20,7 +20,6 @@ from opentelemetry.sdk.metrics.export import InMemoryMetricReader
|
||||
|
||||
from headroom.backends.base import BackendResponse
|
||||
from headroom.observability import HeadroomOtelMetrics, reset_otel_metrics, set_otel_metrics
|
||||
from headroom.proxy.handlers.openai import _openai_rate_limit_key
|
||||
from headroom.proxy.server import ProxyConfig, create_app
|
||||
from tests.test_observability_metrics import _collect_metrics
|
||||
|
||||
@@ -1287,6 +1286,12 @@ def test_openai_chat_custom_base_flood_cannot_grow_the_provider_set(
|
||||
# The flood hosts are unresolvable on purpose; without this stub the SSRF
|
||||
# guard would drop the override and relabel the flood as "openai".
|
||||
monkeypatch.setattr("headroom.proxy.handlers.openai.is_safe_upstream_url", lambda url: True)
|
||||
# Bucket by the raw credential so the stub limiter below can refuse just
|
||||
# ``limited_auth``; the identity rule itself is covered by its own tests.
|
||||
monkeypatch.setattr(
|
||||
"headroom.proxy.handlers.openai.rate_limit_identity",
|
||||
lambda request, headers: headers.get("authorization", ""),
|
||||
)
|
||||
# Isolated sinks: a fresh lifetime store and workspace, so the exact
|
||||
# counts below are this test's traffic and nothing reaches ~/.headroom.
|
||||
monkeypatch.setenv("HEADROOM_SAVINGS_PATH", str(tmp_path / "savings.json"))
|
||||
@@ -1298,7 +1303,7 @@ def test_openai_chat_custom_base_flood_cannot_grow_the_provider_set(
|
||||
"""Headroom's own limiter, refusing only ``limited_auth``."""
|
||||
|
||||
async def check_request(self, key: str = "default") -> tuple[bool, float]:
|
||||
return key != _openai_rate_limit_key({"authorization": limited_auth}), 1.0
|
||||
return key != limited_auth, 1.0
|
||||
|
||||
async def check_tokens(self, key: str, tokens: int) -> tuple[bool, float]:
|
||||
return True, 0.0
|
||||
|
||||
@@ -1,4 +1,15 @@
|
||||
"""Regression coverage for OpenAI credential-scoped rate limiting (#3364)."""
|
||||
"""Credential-scoped rate limiting (#3364) under the shared identity rule (01-F3).
|
||||
|
||||
#3364 made distinct OpenAI credentials use distinct buckets. VAPT finding 01-F3
|
||||
showed the other side of that: a caller the proxy cannot authenticate could
|
||||
rotate the credential header to mint a fresh bucket per request. Both hold now:
|
||||
|
||||
* an **authenticated** caller (proxy token) or a direct loopback caller gets
|
||||
one bucket per provider credential — #3364's behaviour;
|
||||
* an **unauthenticated remote** caller is charged per peer, whatever credential
|
||||
header it sends — including one relayed by a trusted gateway, which is
|
||||
charged per forwarded client address.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -11,34 +22,37 @@ pytest.importorskip("fastapi")
|
||||
|
||||
from fastapi.testclient import TestClient # noqa: E402
|
||||
|
||||
from headroom.proxy.handlers.openai import _openai_rate_limit_key # noqa: E402
|
||||
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
|
||||
|
||||
TOKEN = "hrpt_rate_limit_test_token_0123456789"
|
||||
|
||||
def test_openai_rate_limit_key_accepts_api_key_header() -> None:
|
||||
assert _openai_rate_limit_key({"api-key": "gateway-key-A"}) != _openai_rate_limit_key(
|
||||
{"api-key": "gateway-key-B"}
|
||||
)
|
||||
CHAT = (
|
||||
"/v1/chat/completions",
|
||||
{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "hello"}], "stream": False},
|
||||
{
|
||||
"id": "chatcmpl-test",
|
||||
"object": "chat.completion",
|
||||
"choices": [
|
||||
{"index": 0, "message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}
|
||||
],
|
||||
"usage": {"prompt_tokens": 3, "completion_tokens": 1, "total_tokens": 4},
|
||||
},
|
||||
)
|
||||
RESPONSES = (
|
||||
"/v1/responses",
|
||||
{"model": "gpt-4o-mini", "input": "hello", "stream": False},
|
||||
{
|
||||
"id": "resp-test",
|
||||
"object": "response",
|
||||
"status": "completed",
|
||||
"output": [],
|
||||
"usage": {"input_tokens": 3, "output_tokens": 1, "total_tokens": 4},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_openai_rate_limit_key_uses_complete_credential() -> None:
|
||||
shared_prefix = "a" * 20
|
||||
assert _openai_rate_limit_key({"api-key": f"{shared_prefix}-A"}) != _openai_rate_limit_key(
|
||||
{"api-key": f"{shared_prefix}-B"}
|
||||
)
|
||||
|
||||
|
||||
def test_openai_rate_limit_key_prefers_authorization() -> None:
|
||||
headers = {"authorization": "Bearer primary", "api-key": "secondary"}
|
||||
assert _openai_rate_limit_key(headers) == _openai_rate_limit_key(
|
||||
{"authorization": "Bearer primary"}
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", ["chat", "responses"])
|
||||
def test_distinct_api_keys_do_not_share_openai_rate_limit_bucket(monkeypatch, endpoint) -> None:
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
config = ProxyConfig(
|
||||
def _config(**kwargs) -> ProxyConfig:
|
||||
return ProxyConfig(
|
||||
optimize=False,
|
||||
cache_enabled=False,
|
||||
rate_limit_enabled=True,
|
||||
@@ -51,47 +65,177 @@ def test_distinct_api_keys_do_not_share_openai_rate_limit_bucket(monkeypatch, en
|
||||
ccr_context_tracking=False,
|
||||
image_optimize=False,
|
||||
retry_enabled=False,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
if endpoint == "chat":
|
||||
path = "/v1/chat/completions"
|
||||
body = {
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [{"role": "user", "content": "hello"}],
|
||||
"stream": False,
|
||||
}
|
||||
upstream_body = {
|
||||
"id": "chatcmpl-test",
|
||||
"object": "chat.completion",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {"role": "assistant", "content": "ok"},
|
||||
"finish_reason": "stop",
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 3, "completion_tokens": 1, "total_tokens": 4},
|
||||
}
|
||||
else:
|
||||
path = "/v1/responses"
|
||||
body = {"model": "gpt-4o-mini", "input": "hello", "stream": False}
|
||||
upstream_body = {
|
||||
"id": "resp-test",
|
||||
"object": "response",
|
||||
"status": "completed",
|
||||
"output": [],
|
||||
"usage": {"input_tokens": 3, "output_tokens": 1, "total_tokens": 4},
|
||||
}
|
||||
|
||||
with TestClient(create_app(config)) as client:
|
||||
proxy = client.app.state.proxy
|
||||
proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *args, **kwargs: httpx.Response(200, json=upstream_body)
|
||||
def _two_requests(client, path, body, first_headers, second_headers):
|
||||
proxy = client.app.state.proxy
|
||||
return (
|
||||
proxy,
|
||||
client.post(path, headers=first_headers, json=body),
|
||||
client.post(path, headers=second_headers, json=body),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", [CHAT, RESPONSES], ids=["chat", "responses"])
|
||||
def test_authenticated_callers_get_one_bucket_per_credential(monkeypatch, endpoint) -> None:
|
||||
"""#3364: distinct provider credentials from an authenticated caller do not share."""
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
path, body, upstream_body = endpoint
|
||||
with TestClient(create_app(_config(proxy_token=TOKEN)), client=("203.0.113.9", 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
proxy, first, second = _two_requests(
|
||||
client,
|
||||
path,
|
||||
body,
|
||||
{"x-headroom-proxy-token": TOKEN, "api-key": "gateway-key-A"},
|
||||
{"x-headroom-proxy-token": TOKEN, "api-key": "gateway-key-B"},
|
||||
)
|
||||
assert first.status_code == 200
|
||||
assert second.status_code == 200
|
||||
assert proxy._retry_request.await_count == 2
|
||||
|
||||
first = client.post(path, headers={"api-key": "gateway-key-A"}, json=body)
|
||||
second = client.post(path, headers={"api-key": "gateway-key-B"}, json=body)
|
||||
|
||||
assert first.status_code == 200
|
||||
assert second.status_code == 200
|
||||
assert proxy._retry_request.await_count == 2
|
||||
@pytest.mark.parametrize("endpoint", [CHAT, RESPONSES], ids=["chat", "responses"])
|
||||
def test_loopback_callers_get_one_bucket_per_credential(monkeypatch, endpoint) -> None:
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
path, body, upstream_body = endpoint
|
||||
with TestClient(create_app(_config()), client=("127.0.0.1", 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
proxy, first, second = _two_requests(
|
||||
client, path, body, {"api-key": "key-A"}, {"api-key": "key-B"}
|
||||
)
|
||||
assert (first.status_code, second.status_code) == (200, 200)
|
||||
assert proxy._retry_request.await_count == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", [CHAT, RESPONSES], ids=["chat", "responses"])
|
||||
@pytest.mark.parametrize(
|
||||
"rotating",
|
||||
[
|
||||
({"api-key": "rotated-1"}, {"api-key": "rotated-2"}),
|
||||
({"authorization": "Bearer rotated-1"}, {"authorization": "Bearer rotated-2"}),
|
||||
({"api-key": "rotated-1"}, {}),
|
||||
],
|
||||
ids=["api-key", "authorization", "credential-then-none"],
|
||||
)
|
||||
def test_unauthenticated_remote_caller_cannot_rotate_credentials_into_new_buckets(
|
||||
monkeypatch, endpoint, rotating
|
||||
) -> None:
|
||||
"""01-F3: with no proxy token, a remote caller is charged per peer."""
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
path, body, upstream_body = endpoint
|
||||
with TestClient(create_app(_config()), client=("203.0.113.9", 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
proxy, first, second = _two_requests(client, path, body, *rotating)
|
||||
assert first.status_code == 200
|
||||
assert second.status_code == 429
|
||||
assert proxy._retry_request.await_count == 1
|
||||
|
||||
|
||||
def test_unauthenticated_remote_callers_on_different_peers_do_not_share(monkeypatch) -> None:
|
||||
"""The fix is per peer, not one global bucket for everyone unauthenticated."""
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
path, body, upstream_body = CHAT
|
||||
app = create_app(_config())
|
||||
statuses = []
|
||||
for peer in ("203.0.113.9", "198.51.100.4"):
|
||||
with TestClient(app, client=(peer, 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
statuses.append(client.post(path, json=body).status_code)
|
||||
assert statuses == [200, 200]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", [CHAT, RESPONSES], ids=["chat", "responses"])
|
||||
def test_unauthenticated_caller_behind_trusted_gateway_cannot_rotate_credentials(
|
||||
monkeypatch, endpoint
|
||||
) -> None:
|
||||
"""A trusted gateway vouches for the client address, not for the caller."""
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "10.0.0.0/8")
|
||||
path, body, upstream_body = endpoint
|
||||
forwarded = {"x-forwarded-for": "203.0.113.9"}
|
||||
with TestClient(create_app(_config()), client=("10.0.0.2", 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
proxy, first, second = _two_requests(
|
||||
client,
|
||||
path,
|
||||
body,
|
||||
{**forwarded, "api-key": "rotated-A"},
|
||||
{**forwarded, "api-key": "rotated-B"},
|
||||
)
|
||||
assert first.status_code == 200
|
||||
assert second.status_code == 429
|
||||
assert proxy._retry_request.await_count == 1
|
||||
|
||||
|
||||
def test_trusted_gateway_keys_unauthenticated_callers_by_forwarded_address(monkeypatch) -> None:
|
||||
"""Distinct forwarded clients behind one gateway keep distinct buckets."""
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "10.0.0.0/8")
|
||||
path, body, upstream_body = CHAT
|
||||
with TestClient(create_app(_config()), client=("10.0.0.2", 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
statuses = [
|
||||
client.post(path, headers={"x-forwarded-for": ip}, json=body).status_code
|
||||
for ip in ("203.0.113.9", "198.51.100.4")
|
||||
]
|
||||
assert statuses == [200, 200]
|
||||
|
||||
|
||||
def test_authenticated_caller_behind_trusted_gateway_gets_one_bucket_per_credential(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "10.0.0.0/8")
|
||||
path, body, upstream_body = CHAT
|
||||
headers = {"x-forwarded-for": "203.0.113.9", "x-headroom-proxy-token": TOKEN}
|
||||
with TestClient(create_app(_config(proxy_token=TOKEN)), client=("10.0.0.2", 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
_, first, second = _two_requests(
|
||||
client, path, body, {**headers, "api-key": "key-A"}, {**headers, "api-key": "key-B"}
|
||||
)
|
||||
assert (first.status_code, second.status_code) == (200, 200)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", [CHAT, RESPONSES], ids=["chat", "responses"])
|
||||
def test_loopback_gateway_config_direct_and_forwarded_callers(monkeypatch, endpoint) -> None:
|
||||
"""One loopback gateway CIDR: direct callers stay per credential, relayed ones per client.
|
||||
|
||||
Direct loopback callers (no ``X-Forwarded-For``) with distinct keys keep
|
||||
distinct buckets. A caller relayed through the same loopback gateway with a
|
||||
fixed forwarded address cannot rotate keys into a fresh bucket.
|
||||
"""
|
||||
monkeypatch.setenv("HEADROOM_SKIP_UPSTREAM_CHECK", "1")
|
||||
monkeypatch.setenv("HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS", "127.0.0.0/8")
|
||||
path, body, upstream_body = endpoint
|
||||
forwarded = {"x-forwarded-for": "203.0.113.9"}
|
||||
with TestClient(create_app(_config()), client=("127.0.0.1", 1)) as client:
|
||||
client.app.state.proxy._retry_request = AsyncMock(
|
||||
side_effect=lambda *a, **k: httpx.Response(200, json=upstream_body)
|
||||
)
|
||||
statuses = [
|
||||
client.post(path, headers=headers, json=body).status_code
|
||||
for headers in (
|
||||
{"api-key": "direct-A"},
|
||||
{"api-key": "direct-B"},
|
||||
{**forwarded, "api-key": "rotated-A"},
|
||||
{**forwarded, "api-key": "rotated-B"},
|
||||
)
|
||||
]
|
||||
assert statuses == [200, 200, 200, 429]
|
||||
|
||||
Reference in New Issue
Block a user