fix(subscription): poll with the refreshed credentials-file OAuth token (#3916)

## Description

Fixes #3913.

`_maybe_poll` resolved the token to poll with as `token or bg_token`,
which
prefers `self._current_token` over `read_cached_oauth_token()`:

```python
if not is_active:
    bg_token = read_cached_oauth_token()
    if not bg_token:
        return
    token = token or bg_token
```

`self._current_token` is a snapshot of the last proxied `Authorization`
header,
set by `notify_active` when a request passes through. Claude Code
refreshes its
OAuth token by rewriting `~/.claude/.credentials.json`, which does
**not** touch
that in-memory value — and an idle client sends no proxied request, so
`_current_token` is never replaced. Every poll therefore went out with
the
pre-rotation token, Anthropic answered 401, `SubscriptionClient.fetch`
mapped
that to `None`, and the tracker recorded `fetch returned None` on each
interval
while `polled_at` stopped advancing.

The bug is that the *fresh* credential source was only consulted as a
fallback
when the remembered one was empty — the one case where it does not help.

## Changes Made

- Consult `read_cached_oauth_token()` on **every** poll, not only when
inactive.
- Poll with the credentials-file token first — it is the source that
refreshes.
- Fall back to the remembered header token only when it is a *different*
token
that could still be live (no credentials file, polling off a proxied
header),
  so a normal poll still costs exactly one request.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Testing

- [x] Unit tests pass
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ HEADROOM_REQUIRE_RUST_CORE=false python -m pytest \
    tests/test_subscription_base.py tests/test_subscription_client.py \
    tests/test_subscription_contribution.py tests/test_subscription_session_tracking.py \
    tests/test_subscription_tracker.py tests/test_subscription_window_render.py \
    tests/test_copilot_subscription_smoke.py -q
51 passed
```

Three new regression tests in `tests/test_subscription_tracker.py`.
Against the
unpatched `tracker.py`, two of them fail and pass only with this change:

```text
FAILED test_maybe_poll_prefers_refreshed_credentials_token_over_stale_header
  - assert ['stale-header-token'] == ['refreshed-file-token']
FAILED test_maybe_poll_falls_back_to_header_token_when_file_token_rejected
  - assert ['header-only-token'] == ['revoked-file-token', 'header-only-token']
```

`test_maybe_poll_single_request_when_file_token_matches_header` pins the
one-request-per-poll guarantee.

`python -m ruff check` and `python -m ruff format --check` clean on both
files.

### Pre-existing failures, unrelated

`tests/test_oauth_bearer_routing.py` and
`tests/test_license_env_and_usage_reporting.py`
have 10 failures in this environment. They fail identically on unpatched
`main`
(verified by stashing this change), so they are not caused by this PR.

## Real Behavior Proof

- Environment: Python 3.13.5, Linux 5.x, `headroom` checked out at
`main` (`c46e74d`) with this one-file change applied. No live Anthropic
traffic — `SubscriptionClient` is stubbed to reproduce the 401, and
`read_cached_oauth_token` is pointed at a real credentials file written
to a temp directory, so the token-precedence logic runs for real rather
than being mocked at the call site.
- Exact command / steps: run a script that (1) writes a credentials file
holding `claudeAiOauth.accessToken = "tok-POST-REFRESH"` with a future
`expiresAt`; (2) constructs `SubscriptionTracker` and calls
`notify_active("Bearer tok-PRE-ROTATION")` to capture a proxied header
token from *before* the rotation — the state an idle Claude Code session
leaves behind, since no further request flows through the proxy; (3)
stubs `_client.fetch` to return `None` for `tok-PRE-ROTATION` (what the
usage API's 401 becomes) and a real `SubscriptionSnapshot` otherwise;
(4) awaits `tracker._maybe_poll()` once and prints the tokens the stub
saw plus the tracker state.
- Observed result: on unpatched `main` the poll sent
`['tok-PRE-ROTATION']`, ended with `poll_errors: 1`, `last_error: "fetch
returned None"` and no snapshot. With this change the same script sent
`['tok-POST-REFRESH']`, ended with `poll_errors: 0`, `last_error: None`,
and stored the snapshot. Same input and same stub, so the difference is
the token-precedence change alone. `mypy
headroom/subscription/tracker.py --ignore-missing-imports` → `Success:
no issues found in 1 source file`; `ruff check` and `ruff format
--check` clean on both touched files.
- Not tested: live traffic against
`https://api.anthropic.com/api/oauth/usage`, a real Claude Code OAuth
rotation, the proxy-mode `/stats` and dashboard end-to-end output,
`maybe_poll_on_demand`, Windows or macOS, and the poll loop's
long-running cadence over hours. The `read_cached_oauth_token` path
returns `None` on a malformed or expired credentials file, in which case
behaviour is unchanged from before (the header token is used) — covered
by the fallback test, not by a live run.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This corrects which existing token
the already-registered `SubscriptionTracker` polls with; it does not
add, gate, or change any `headroom.rollout` feature.
- Minimum rollout channel: all channels. `subscription` /
`subscription_window` reporting is available in every channel today, so
the fix applies everywhere rather than being canary-gated.
- Stable/default behavior changed: yes, deliberately — a poll that
previously sent a stale token and recorded `fetch returned None` now
sends the credentials-file token and can succeed. That is the bug being
fixed, and it is what #3913 reports as `/stats` showing growing
`poll_errors` and a `polled_at` frozen for hours.
- Kill switch / disable path: none needed. Setting `enabled=False` on
the subscription tracker (the existing config surface, exercised by
`tests/test_subscription_tracker.py::test_tracker_notify_active_update_and_basic_state`)
already stops all polling. Disabling the subscription tracker entirely
restores pre-existing behaviour, which is the documented workaround in
#3913 and should not be needed after this change.
- Unsafe override required: no.
- Qualification impact: subscription polling is not part of feature
qualification; no rollout snapshot digest or enabled-feature set
changes, so `RolloutSnapshot` is byte-identical before and after.
- Rollback path: revert this single commit. There is no migration, no
persisted-format change and no state to unwind — `_maybe_poll` holds no
new state between calls, and the only persistent writes are the existing
`_persist_state()` snapshot writes.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Additional Notes

- The `tests/test_oauth_bearer_routing.py` and
`tests/test_license_env_and_usage_reporting.py` failures reported above
are pre-existing in this sandbox and reproduce identically on unpatched
`main` (verified by stashing this change and re-running), so they are
not caused by this PR.
- N/A checklist items: no documentation change is needed — this is a bug
fix with no user-facing configuration or CLI surface, and no
`CHANGELOG.md` was touched (release-please generates it from the commit
title).
- Scope note: this deliberately does *not* clear `_current_token` on a
401, which #3913 lists as optional. Preferring the refreshable source
removes the staleness rather than papering over it, and touching the
remembered token would interact with `notify_active`'s token-prefix
accounting (`self._full_tokens`) that the contribution stats depend on.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3916"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Yi-111-a <241260369@qq.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
yi111
2026-10-05 08:30:26 -05:00
committed by GitHub
co-authored by Yi-111-a JD Davis
parent 2de582875b
commit 2dc9fc22f0
3 changed files with 132 additions and 11 deletions
+23 -8
View File
@@ -399,16 +399,31 @@ class SubscriptionTracker(QuotaTracker):
is_active = self._state.is_active(active_window_s=self._active_window_s)
token = self._current_token
if not is_active:
# Try background poll using credentials file token
from headroom.subscription.client import read_cached_oauth_token
# Always consult the credentials file: it is the source Claude Code
# refreshes in place, so it holds the *current* token. `self._current_token`
# is a snapshot of the last proxied Authorization header and goes stale
# as soon as Claude Code rotates its OAuth token — it is only replaced
# when a request flows through the proxy, which an idle client never
# does. Preferring it (`token or bg_token`) therefore pinned polling to
# an expired token and surfaced as permanent `poll_errors` /
# "fetch returned None" with a stale `polled_at` (issue #3913).
from headroom.subscription.client import read_cached_oauth_token
bg_token = read_cached_oauth_token()
if not bg_token:
return
token = token or bg_token
bg_token = read_cached_oauth_token()
if not is_active and not bg_token:
return
snapshot = None
# Fresh credentials-file token first, so a normal poll costs one request.
if bg_token:
snapshot = await self._client.fetch(bg_token)
# Only spend a second request on the remembered header token when it is
# a *different* token that could still be the live one (no credentials
# file, e.g. CLAUDE_CODE_OAUTH_TOKEN absent and polling off a proxied
# header).
if snapshot is None and token and token != bg_token:
snapshot = await self._client.fetch(token)
snapshot = await self._client.fetch(token)
if snapshot is None:
with self._lock:
self._state.mark_error("fetch returned None")
@@ -103,9 +103,11 @@ def test_foreign_bearer_never_reaches_the_usage_poll(tmp_path, monkeypatch) -> N
tracker._client.fetch = fetch # type: ignore[method-assign]
asyncio.run(tracker._maybe_poll())
# The poll ran (the caller marked activity) but with no adopted token, so
# the client falls back to the operator's own credential.
assert polled == [None]
# Activity was marked, but with no adopted local-operator token and no
# operator-configured credential there is nothing to poll — and specifically
# the foreign bearer must never be spent. Skip the unnecessary fetch(None).
assert polled == []
assert "tenant-b-oauth-token" not in {t for t in polled if t}
# ---------------------------------------------------------------------------
+104
View File
@@ -174,6 +174,106 @@ async def test_maybe_poll_handles_inactive_and_none_snapshot(
assert tracker._state.poll_errors == 1
@pytest.mark.asyncio
async def test_maybe_poll_prefers_refreshed_credentials_token_over_stale_header(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Regression (#3913): the credentials file is where Claude Code writes the
*refreshed* OAuth token. Polling the remembered Authorization-header token
first pinned every poll to an expired credential, so `/stats` reported
permanent `poll_errors` and a `polled_at` that never advanced."""
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
tracker = SubscriptionTracker()
# Token captured from a proxied local-operator request, before Claude Code rotated it.
tracker.notify_active("Bearer stale-header-token", from_local_operator=True)
monkeypatch.setattr(
"headroom.subscription.client.read_cached_oauth_token", lambda: "refreshed-file-token"
)
snapshot = _make_snapshot()
attempted: list[str | None] = []
async def fetch_snapshot(token: str | None):
attempted.append(token)
# The stale header token is what Anthropic rejects with 401 -> None.
if token == "stale-header-token":
return None
return snapshot
tracker._client = SimpleNamespace(fetch=fetch_snapshot)
monkeypatch.setattr(tracker, "_persist_state", lambda: None)
await tracker._maybe_poll()
assert attempted == ["refreshed-file-token"]
assert tracker.latest_snapshot is snapshot
assert tracker._state.poll_errors == 0
assert tracker._state.last_error is None
@pytest.mark.asyncio
async def test_maybe_poll_falls_back_to_header_token_when_file_token_rejected(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The remembered header token is still the only credential when no
credentials file exists, so a rejection of the file token must fall back to
it rather than mark the tracker errored."""
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
tracker = SubscriptionTracker()
tracker.notify_active("Bearer header-only-token", from_local_operator=True)
monkeypatch.setattr(
"headroom.subscription.client.read_cached_oauth_token", lambda: "revoked-file-token"
)
snapshot = _make_snapshot()
attempted: list[str | None] = []
async def fetch_snapshot(token: str | None):
attempted.append(token)
if token == "revoked-file-token":
return None
return snapshot
tracker._client = SimpleNamespace(fetch=fetch_snapshot)
monkeypatch.setattr(tracker, "_persist_state", lambda: None)
await tracker._maybe_poll()
assert attempted == ["revoked-file-token", "header-only-token"]
assert tracker.latest_snapshot is snapshot
assert tracker._state.poll_errors == 0
@pytest.mark.asyncio
async def test_maybe_poll_single_request_when_file_token_matches_header(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Identical tokens must not be spent twice — the common case for a proxy
session whose credentials file holds the same token as the header."""
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
tracker = SubscriptionTracker()
tracker.notify_active("Bearer shared-token", from_local_operator=True)
monkeypatch.setattr(
"headroom.subscription.client.read_cached_oauth_token", lambda: "shared-token"
)
attempted: list[str | None] = []
async def fetch_snapshot(token: str | None):
attempted.append(token)
return _make_snapshot()
tracker._client = SimpleNamespace(fetch=fetch_snapshot)
monkeypatch.setattr(tracker, "_persist_state", lambda: None)
await tracker._maybe_poll()
assert attempted == ["shared-token"]
@pytest.mark.asyncio
async def test_maybe_poll_success_updates_state_and_metrics(
monkeypatch: pytest.MonkeyPatch,
@@ -181,6 +281,9 @@ async def test_maybe_poll_success_updates_state_and_metrics(
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
tracker = SubscriptionTracker()
tracker.notify_active("Bearer live-oauth-token", from_local_operator=True)
# Pin the credentials-file/env source so this exercises the adopted
# local-operator header path, not whatever token the host happens to hold.
monkeypatch.setattr("headroom.subscription.client.read_cached_oauth_token", lambda: None)
snapshot = _make_snapshot()
discrepancies = [WindowDiscrepancy(kind="cache_miss", description="miss", severity="warning")]
@@ -227,6 +330,7 @@ async def test_maybe_poll_runs_transcript_scan_off_event_loop(
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
tracker = SubscriptionTracker()
tracker.notify_active("Bearer live-oauth-token", from_local_operator=True)
monkeypatch.setattr("headroom.subscription.client.read_cached_oauth_token", lambda: None)
snapshot = _make_snapshot()