mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(security): poll usage only with the operator's own credential (#3863)
## Why Two background usage pollers spent callers' credentials (finding 01-F16): - The Claude subscription tracker adopted any caller's OAuth `Authorization: Bearer` as the account to poll (`notify_active` stored it and `_maybe_poll` used it). - The Codex `/wham/usage` refresher polled with any caller's bearer and `ChatGPT-Account-Id`. On a proxy shared by several callers, a network caller's credential was therefore sent on a request they never made, and their subscription usage appeared on the operator's dashboard. ## What changes - New `headroom/subscription/credential_policy.py` with `is_local_operator_connection()`. It is true only for a direct loopback peer with no `Forwarded`, `X-Real-IP` or `X-Forwarded-*` header (any of the family, including `-Proto` and `-Host` alone). It fails closed on a missing peer. Trusted-gateway CIDRs do not count, because a gateway fronts other principals. - `SubscriptionTracker.notify_active(token, *, from_local_operator=False)` always marks the tracker active, but stores the bearer for polling only when `from_local_operator` is true. - `maybe_schedule_usage_poll(headers, *, from_local_operator=False)` returns without polling unless `from_local_operator` is true. - Both flags default to the safe answer, so a new call site cannot adopt a credential by accident. The Anthropic handler and the Codex WebSocket relay pass `is_local_operator_connection(...)`. Unchanged: `_maybe_poll` ordering, and the fallback to the operator's own credential (`CLAUDE_CODE_OAUTH_TOKEN` or the Claude Code credentials file) when no bearer has been learned. The Copilot quota poller already uses only the host's own GitHub token. ## Behaviour change - Usage pollers never poll with a network caller's credential. A bearer learned from traffic comes only from a direct local caller; otherwise the pollers use the operator-configured credential as before. - A proxy fronted by a gateway on the same host no longer learns a token from gateway traffic. Set `CLAUDE_CODE_OAUTH_TOKEN` there if you want the subscription window. - A single developer running Claude Code on the same machine is unaffected, including when the OAuth token lives in the macOS Keychain rather than a file. ## Tests `tests/test_subscription_poll_credential_policy.py`: - The policy: IPv4 and IPv6 loopback callers are the local operator; LAN peers, a missing peer and each forwarding header are not, including `X-Forwarded-Proto` alone and `X-Forwarded-Host` alone. - The tracker: a network caller marks activity but is never adopted; a local operator's bearer is adopted; a foreign bearer never reaches a usage poll. - The Codex trigger is not driven by a network caller. - End to end through the real `/v1/messages` handler: a LAN caller's bearer is never adopted. The tracker, foreign-bearer, Codex and handler tests fail on `main`. Existing tracker and Codex tests that assert adoption now pass `from_local_operator=True`, so they still test what they meant to. Manual run: `headroom proxy --host 0.0.0.0` with no Claude credentials on the host and the usage URL pointed at a local capture server. On `main`, a LAN caller's bearer was used for polling. On this branch only the loopback operator's bearer was used. Not tested: a live Codex ChatGPT session end to end (the trigger is covered by unit tests). ## Merge order Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 → #3863. This PR goes last. #3891 stops the proxy token being forwarded upstream; this PR stops the pollers adopting any foreign bearer. They touch different files. Revised 2026-10-01 after self-review: removed the `HEADROOM_SUBSCRIPTION_TRAFFIC_TOKEN` setting and reverted the poll re-ordering, so the change is limited to which callers' bearers are stored. Revised 2026-10-01 after review: the local-operator check now rejects every `X-Forwarded-*` header, not just `X-Forwarded-For`, so a gateway sending only `-Proto` or `-Host` fails closed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3863" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
90a68e0291
commit
65e63c0883
@@ -1264,7 +1264,16 @@ class AnthropicHandlerMixin:
|
||||
|
||||
_sub_tracker = _get_sub_tracker()
|
||||
if _sub_tracker is not None:
|
||||
_sub_tracker.notify_active(_auth_header)
|
||||
from headroom.subscription.credential_policy import (
|
||||
is_local_operator_connection,
|
||||
)
|
||||
|
||||
# Only the local operator's bearer may become the polled
|
||||
# account; a network caller only marks activity (01-F16).
|
||||
_sub_tracker.notify_active(
|
||||
_auth_header,
|
||||
from_local_operator=is_local_operator_connection(request),
|
||||
)
|
||||
|
||||
# Rate limiting
|
||||
if self.rate_limiter:
|
||||
|
||||
@@ -7598,8 +7598,16 @@ class OpenAIHandlerMixin:
|
||||
from headroom.subscription.codex_rate_limits import (
|
||||
maybe_schedule_usage_poll,
|
||||
)
|
||||
from headroom.subscription.credential_policy import (
|
||||
is_local_operator_connection,
|
||||
)
|
||||
|
||||
maybe_schedule_usage_poll(ws_headers)
|
||||
# The poll spends the caller's own bearer: local operator only
|
||||
# (01-F16).
|
||||
maybe_schedule_usage_poll(
|
||||
ws_headers,
|
||||
from_local_operator=is_local_operator_connection(websocket),
|
||||
)
|
||||
|
||||
try:
|
||||
# ChatGPT-auth sessions no longer need upstream x-codex-* headers on
|
||||
|
||||
@@ -463,6 +463,7 @@ async def _fetch_and_store_usage(url: str, headers: dict[str, str]) -> None:
|
||||
def maybe_schedule_usage_poll(
|
||||
request_headers: dict[str, str],
|
||||
*,
|
||||
from_local_operator: bool = False,
|
||||
url: str = CODEX_USAGE_URL,
|
||||
min_interval_s: float = USAGE_POLL_MIN_INTERVAL_S,
|
||||
) -> bool:
|
||||
@@ -471,7 +472,14 @@ def maybe_schedule_usage_poll(
|
||||
Safe to call on every Codex request: scoped to ChatGPT-session traffic via
|
||||
:func:`_build_usage_headers` and internally throttled to at most one live
|
||||
poll per ``min_interval_s``. Returns ``True`` when a poll was scheduled.
|
||||
|
||||
The poll spends the *caller's* bearer, so it runs only when
|
||||
``from_local_operator`` is true (the handler passes
|
||||
:func:`headroom.subscription.credential_policy.is_local_operator_connection`).
|
||||
A network caller on a shared proxy never drives it (VAPT 01-F16).
|
||||
"""
|
||||
if not from_local_operator:
|
||||
return False
|
||||
headers = _build_usage_headers(request_headers)
|
||||
if headers is None:
|
||||
return False
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Which credential a background usage poller may use (VAPT 01-F16).
|
||||
|
||||
Usage pollers (the Claude subscription tracker, the Codex ``/wham/usage``
|
||||
refresher) call a provider on the proxy's behalf and show the result on the
|
||||
operator's dashboard. The account they poll must be the **operator's**, so a
|
||||
credential *learned from traffic* is adopted only from the **local operator**: a
|
||||
loopback peer whose request was not forwarded by another hop. On a shared proxy (several principals behind one Headroom), a network caller's
|
||||
``Authorization`` bearer must never become the polled account — that would spend
|
||||
the caller's credential on a request they never made and publish their usage on
|
||||
someone else's dashboard. Without a learned token the pollers fall back to the
|
||||
operator-configured credential (``CLAUDE_CODE_OAUTH_TOKEN`` or the proxy user's
|
||||
own Claude Code credentials file), as before.
|
||||
|
||||
This keeps the single-developer experience: Claude Code on the same machine,
|
||||
where the OAuth token often lives in the OS keychain rather than a file, still
|
||||
gets its subscription window without extra setup.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from headroom.proxy.loopback_guard import is_loopback_host
|
||||
|
||||
# Any of these means the request crossed another hop (a gateway or reverse
|
||||
# proxy on the same host), so the loopback peer is not the end caller. The
|
||||
# whole ``X-Forwarded-*`` family counts (``-For``, ``-Proto``, ``-Host``, and
|
||||
# any added later), so a gateway that sends only one of them still fails closed.
|
||||
_FORWARDING_HEADERS = frozenset({"forwarded", "x-real-ip"})
|
||||
_FORWARDING_HEADER_PREFIX = "x-forwarded-"
|
||||
|
||||
|
||||
def is_local_operator_connection(conn: Any) -> bool:
|
||||
"""True only for a direct loopback caller (HTTP request or WebSocket).
|
||||
|
||||
Fails closed: a missing peer address, a non-loopback peer, or any
|
||||
forwarding header returns ``False``. Trusted-gateway CIDRs deliberately do
|
||||
not count — a gateway fronts other principals by definition.
|
||||
"""
|
||||
client = getattr(conn, "client", None)
|
||||
host = getattr(client, "host", None) if client is not None else None
|
||||
if not isinstance(host, str) or not is_loopback_host(host):
|
||||
return False
|
||||
headers = getattr(conn, "headers", None)
|
||||
if headers is not None:
|
||||
try:
|
||||
for name in headers.keys():
|
||||
name = name.lower()
|
||||
if name in _FORWARDING_HEADERS or name.startswith(_FORWARDING_HEADER_PREFIX):
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
return True
|
||||
@@ -179,11 +179,15 @@ class SubscriptionTracker(QuotaTracker):
|
||||
# Proxy integration hooks
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def notify_active(self, token: str) -> None:
|
||||
def notify_active(self, token: str, *, from_local_operator: bool = False) -> None:
|
||||
"""Called by the proxy handler when an OAuth request comes through.
|
||||
|
||||
Stores the token for polling and marks the tracker as recently active.
|
||||
Only processes Bearer tokens that look like OAuth (not API keys).
|
||||
Marks the tracker as recently active. The caller's bearer is remembered
|
||||
for polling **only** when ``from_local_operator`` is true — the handler
|
||||
passes :func:`headroom.subscription.credential_policy.is_local_operator_connection`.
|
||||
A network caller on a shared proxy never becomes the polled account
|
||||
(VAPT 01-F16). Only Bearer tokens that look like OAuth (not API keys)
|
||||
are considered.
|
||||
"""
|
||||
if not token or not token.startswith("Bearer "):
|
||||
return
|
||||
@@ -192,8 +196,10 @@ class SubscriptionTracker(QuotaTracker):
|
||||
if raw.startswith("sk-ant-api"):
|
||||
return
|
||||
with self._lock:
|
||||
self._current_token = raw
|
||||
self._state.last_active_at = _utc_now()
|
||||
if not from_local_operator:
|
||||
return
|
||||
self._current_token = raw
|
||||
prefix = raw[:8]
|
||||
self._full_tokens[prefix] = self._full_tokens.get(prefix, 0) + 1
|
||||
|
||||
|
||||
@@ -383,14 +383,17 @@ class TestUsagePollGating:
|
||||
# No running event loop -> cannot schedule.
|
||||
assert (
|
||||
maybe_schedule_usage_poll(
|
||||
{"authorization": "Bearer a.b.c", "chatgpt-account-id": "acct"}
|
||||
{"authorization": "Bearer a.b.c", "chatgpt-account-id": "acct"},
|
||||
from_local_operator=True,
|
||||
)
|
||||
is False
|
||||
)
|
||||
|
||||
def test_maybe_schedule_skips_non_codex(self):
|
||||
async def run():
|
||||
return maybe_schedule_usage_poll({"authorization": "Bearer a.b.c"})
|
||||
return maybe_schedule_usage_poll(
|
||||
{"authorization": "Bearer a.b.c"}, from_local_operator=True
|
||||
)
|
||||
|
||||
assert asyncio.run(run()) is False
|
||||
|
||||
@@ -409,8 +412,8 @@ class TestUsagePollGating:
|
||||
|
||||
async def run():
|
||||
req = {"authorization": "Bearer a.b.c", "chatgpt-account-id": "acct"}
|
||||
first = maybe_schedule_usage_poll(req, min_interval_s=60.0)
|
||||
second = maybe_schedule_usage_poll(req, min_interval_s=60.0)
|
||||
first = maybe_schedule_usage_poll(req, from_local_operator=True, min_interval_s=60.0)
|
||||
second = maybe_schedule_usage_poll(req, from_local_operator=True, min_interval_s=60.0)
|
||||
# Let the scheduled task run.
|
||||
await asyncio.sleep(0)
|
||||
return first, second
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
"""01-F16: a caller's bearer must never become the account a usage poller polls.
|
||||
|
||||
On a shared proxy, the subscription tracker used to adopt *any* caller's OAuth
|
||||
bearer as the account to poll, and the Codex ``/wham/usage`` refresher polled
|
||||
with any caller's bearer + ChatGPT account id. The result was published on the
|
||||
operator's dashboard and the caller's credential was spent on a request they
|
||||
never made. Now only a bearer from the local operator (direct loopback, not
|
||||
forwarded) may be adopted; otherwise the operator-configured credential is used.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from starlette.datastructures import Headers
|
||||
|
||||
from headroom.subscription import codex_rate_limits
|
||||
from headroom.subscription.credential_policy import is_local_operator_connection
|
||||
from headroom.subscription.tracker import SubscriptionTracker
|
||||
|
||||
|
||||
def _conn(host: str | None, headers: dict[str, str] | Headers | None = None) -> SimpleNamespace:
|
||||
client = SimpleNamespace(host=host) if host is not None else None
|
||||
return SimpleNamespace(client=client, headers=headers or {})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Policy
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize("host", ["127.0.0.1", "::1"])
|
||||
def test_direct_loopback_caller_is_the_local_operator(host: str) -> None:
|
||||
assert is_local_operator_connection(_conn(host)) is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"conn",
|
||||
[
|
||||
_conn("10.0.0.7"),
|
||||
_conn("192.168.1.20"),
|
||||
_conn(None),
|
||||
_conn("127.0.0.1", {"x-forwarded-for": "203.0.113.9"}),
|
||||
_conn("127.0.0.1", {"forwarded": "for=203.0.113.9"}),
|
||||
_conn("127.0.0.1", {"x-real-ip": "203.0.113.9"}),
|
||||
_conn("127.0.0.1", {"x-forwarded-proto": "https"}),
|
||||
_conn("127.0.0.1", {"x-forwarded-host": "public.example"}),
|
||||
_conn("127.0.0.1", {"X-Forwarded-Port": "443"}),
|
||||
_conn("127.0.0.1", Headers({"X-Forwarded-Proto": "https"})),
|
||||
],
|
||||
ids=[
|
||||
"lan",
|
||||
"lan2",
|
||||
"no-peer",
|
||||
"xff",
|
||||
"forwarded",
|
||||
"x-real-ip",
|
||||
"xfp-only",
|
||||
"xfh-only",
|
||||
"xf-port",
|
||||
"starlette-xfp",
|
||||
],
|
||||
)
|
||||
def test_network_forwarded_or_unknown_callers_are_not(conn) -> None: # noqa: ANN001
|
||||
assert is_local_operator_connection(conn) is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Subscription tracker
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _tracker(tmp_path) -> SubscriptionTracker: # noqa: ANN001
|
||||
return SubscriptionTracker(persist_path=tmp_path / "sub.json")
|
||||
|
||||
|
||||
def test_network_caller_marks_activity_but_is_never_adopted(tmp_path) -> None: # noqa: ANN001
|
||||
tracker = _tracker(tmp_path)
|
||||
tracker.notify_active("Bearer tenant-b-oauth-token")
|
||||
assert tracker._current_token is None
|
||||
assert tracker.is_active() is True
|
||||
|
||||
|
||||
def test_local_operator_bearer_is_adopted(tmp_path) -> None: # noqa: ANN001
|
||||
tracker = _tracker(tmp_path)
|
||||
tracker.notify_active("Bearer operator-oauth-token", from_local_operator=True)
|
||||
assert tracker._current_token == "operator-oauth-token"
|
||||
|
||||
|
||||
def test_foreign_bearer_never_reaches_the_usage_poll(tmp_path, monkeypatch) -> None: # noqa: ANN001
|
||||
monkeypatch.delenv("CLAUDE_CODE_OAUTH_TOKEN", raising=False)
|
||||
monkeypatch.setenv("CLAUDE_CONFIG_DIR", str(tmp_path / "no-claude"))
|
||||
tracker = _tracker(tmp_path)
|
||||
tracker.notify_active("Bearer tenant-b-oauth-token") # network caller
|
||||
polled: list[str | None] = []
|
||||
|
||||
async def fetch(token: str | None): # noqa: ANN202
|
||||
polled.append(token)
|
||||
return None
|
||||
|
||||
tracker._client.fetch = fetch # type: ignore[method-assign]
|
||||
asyncio.run(tracker._maybe_poll())
|
||||
# The poll ran (the caller marked activity) but with no adopted token, so
|
||||
# the client falls back to the operator's own credential.
|
||||
assert polled == [None]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Codex /wham/usage refresher
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
CODEX_HEADERS = {"authorization": "Bearer tenant-b.jwt", "chatgpt-account-id": "acct-b"}
|
||||
|
||||
|
||||
def test_codex_poll_is_not_driven_by_a_network_caller(monkeypatch) -> None: # noqa: ANN001
|
||||
fetched: list[dict] = []
|
||||
|
||||
async def _fake_fetch(url, headers): # noqa: ANN001
|
||||
fetched.append(headers)
|
||||
|
||||
monkeypatch.setattr(codex_rate_limits, "_fetch_and_store_usage", _fake_fetch)
|
||||
codex_rate_limits.get_codex_rate_limit_state()._last_poll_monotonic = 0.0
|
||||
|
||||
async def run() -> bool:
|
||||
scheduled = codex_rate_limits.maybe_schedule_usage_poll(CODEX_HEADERS, min_interval_s=0.0)
|
||||
await asyncio.sleep(0)
|
||||
return scheduled
|
||||
|
||||
assert asyncio.run(run()) is False
|
||||
assert fetched == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# End to end through the real Anthropic handler
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _drive_handler(client_host: str, monkeypatch, tmp_path) -> SubscriptionTracker: # noqa: ANN001
|
||||
pytest.importorskip("fastapi")
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from headroom.proxy.server import ProxyConfig, create_app
|
||||
from headroom.subscription import tracker as tracker_mod
|
||||
|
||||
tracker = _tracker(tmp_path)
|
||||
monkeypatch.setattr(tracker_mod, "get_subscription_tracker", lambda: tracker)
|
||||
app = create_app(
|
||||
ProxyConfig(
|
||||
optimize=False,
|
||||
cache_enabled=False,
|
||||
rate_limit_enabled=False,
|
||||
cost_tracking_enabled=False,
|
||||
log_requests=False,
|
||||
ccr_inject_tool=False,
|
||||
ccr_handle_responses=False,
|
||||
ccr_context_tracking=False,
|
||||
image_optimize=False,
|
||||
)
|
||||
)
|
||||
with TestClient(app, client=(client_host, 50000)) as client:
|
||||
|
||||
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "msg_1",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"content": [{"type": "text", "text": "ok"}],
|
||||
"usage": {"input_tokens": 1, "output_tokens": 1},
|
||||
},
|
||||
)
|
||||
|
||||
client.app.state.proxy._retry_request = _fake_retry
|
||||
resp = client.post(
|
||||
"/v1/messages",
|
||||
headers={
|
||||
"authorization": "Bearer tenant-oauth-token",
|
||||
"anthropic-version": "2023-06-01",
|
||||
},
|
||||
json={
|
||||
"model": "claude-haiku-4-5",
|
||||
"max_tokens": 8,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
return tracker
|
||||
|
||||
|
||||
def test_handler_never_adopts_a_network_callers_bearer(monkeypatch, tmp_path) -> None: # noqa: ANN001
|
||||
tracker = _drive_handler("127.0.0.1", monkeypatch, tmp_path)
|
||||
# Sanity: a direct loopback caller is still adopted (single-developer case).
|
||||
assert tracker._current_token == "tenant-oauth-token"
|
||||
|
||||
monkeypatch.setenv("HEADROOM_ALLOW_UNAUTHENTICATED_BIND", "1")
|
||||
tracker = _drive_handler("10.1.2.3", monkeypatch, tmp_path / "net")
|
||||
assert tracker._current_token is None
|
||||
@@ -57,7 +57,7 @@ def test_tracker_notify_active_update_and_basic_state(monkeypatch: pytest.Monkey
|
||||
tracker.notify_active("Bearer sk-ant-api-key")
|
||||
assert tracker._current_token is None
|
||||
|
||||
tracker.notify_active("Bearer oauth-token-123")
|
||||
tracker.notify_active("Bearer oauth-token-123", from_local_operator=True)
|
||||
assert tracker._current_token == "oauth-token-123"
|
||||
assert tracker._full_tokens["oauth-to"] == 1
|
||||
assert tracker.is_active() is True
|
||||
@@ -180,7 +180,7 @@ async def test_maybe_poll_success_updates_state_and_metrics(
|
||||
) -> None:
|
||||
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
|
||||
tracker = SubscriptionTracker()
|
||||
tracker.notify_active("Bearer live-oauth-token")
|
||||
tracker.notify_active("Bearer live-oauth-token", from_local_operator=True)
|
||||
|
||||
snapshot = _make_snapshot()
|
||||
discrepancies = [WindowDiscrepancy(kind="cache_miss", description="miss", severity="warning")]
|
||||
@@ -226,7 +226,7 @@ async def test_maybe_poll_runs_transcript_scan_off_event_loop(
|
||||
multi-second ~/.claude/projects scan wedges the proxy every poll interval."""
|
||||
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
|
||||
tracker = SubscriptionTracker()
|
||||
tracker.notify_active("Bearer live-oauth-token")
|
||||
tracker.notify_active("Bearer live-oauth-token", from_local_operator=True)
|
||||
|
||||
snapshot = _make_snapshot()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user