fix(subscription): log a failed OTEL window update instead of passing silently (#4131)

## Description

`SubscriptionTracker._maybe_poll` exports the subscription window to
OTEL after each poll. That export ended in `except Exception: pass`, so
a broken or misconfigured exporter left no trace at all. It now logs
`Subscription OTEL metrics update failed: <error>` at DEBUG. Polling is
unaffected.

This was split out of the mcp-registry-atomic-writes branch, where it
was off theme.

Refs #4136
## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/subscription/tracker.py`: `except Exception: pass` becomes
`except Exception as exc: logger.debug(...)`. The catch stays broad on
purpose: OTEL is optional observability, and no exporter failure may
break subscription polling. It is now logged, not swallowed.
- Tests: new `test_maybe_poll_logs_a_failed_otel_update` in
`tests/test_subscription_tracker.py`. Its exporter raises; the test
asserts the snapshot is kept, `last_error` stays `None`, and the debug
record is written.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ pytest -q -p no:cacheprovider tests/test_subscription_tracker.py tests/test_subscription_contribution.py
16 passed
# New test against origin/main tracker.py:
1 failed (no log record)
$ uvx ruff@0.16.8 check headroom/subscription/tracker.py tests/test_subscription_tracker.py
All checks passed!
$ mypy headroom/subscription/tracker.py --ignore-missing-imports
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS, Python from the repo `.venv`, worktree off
origin/main.
- Exact command / steps: the new test drives `_maybe_poll` with a fake
client and an OTEL exporter that raises `RuntimeError("exporter down")`.
- Observed result: the poll completes, and `Subscription OTEL metrics
update failed: exporter down` is logged at DEBUG on
`headroom.subscription.tracker`. Before: nothing.
- Smoke harness (real proxy started from this checkout against a local
fake upstream; temp HOME and workspace, offline):

```text
branch: cleanup/cli-integrations-subscription-otel-log @ 08ba2d32
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines (<tmp>/ws/logs/proxy-63721.log):
   2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
   2 headroom.transforms.kompress_compressor - WARNING - Kompress: 'onnx/kompress-int8-wo.onnx' for chopratejas/kompress-v2-base is not cached and HEADROOM_OFFLINE forbids fetching it (HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.). Compression falls back to the non-ML path.
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
   4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
(smoke dir: <tmp>)
```
- Direct runtime command on this PR's own path (temp HOME,
HEADROOM_WORKSPACE_DIR and CODEX_HOME):

```text
$ python otel_run.py  # real SubscriptionTracker._maybe_poll, fake usage client, OTEL exporter that raises; DEBUG logging on
## otel
DEBUG headroom.subscription.tracker: Subscription OTEL metrics update failed: exporter down
poll_count: 1 last_error: None
## clean
poll_count: 1 last_error: None
```
- Not tested: a real OTEL exporter failure.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A, not rollout-managed.
- Stable/default behavior changed: No. One DEBUG log line on exporter
failure, once per poll.
- Kill switch / disable path: None needed.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.
- No docs impact.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4131"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Tejas Chopra
2026-10-10 08:55:24 -07:00
committed by GitHub
co-authored by Claude Opus 5.5
parent f2153fec2b
commit 7d9dd55e7c
2 changed files with 43 additions and 2 deletions
+2 -2
View File
@@ -457,8 +457,8 @@ class SubscriptionTracker(QuotaTracker):
from headroom.observability.metrics import get_otel_metrics
get_otel_metrics().record_subscription_window(self._state.to_dict())
except Exception:
pass
except Exception as exc:
logger.debug("Subscription OTEL metrics update failed: %s", exc)
def _maybe_reset_contribution(self, snapshot: SubscriptionSnapshot) -> None:
"""Reset contribution counters when the 5h window rolls over."""
+41
View File
@@ -363,6 +363,47 @@ async def test_maybe_poll_runs_transcript_scan_off_event_loop(
assert seen["thread_id"] != loop_thread_id
@pytest.mark.asyncio
async def test_maybe_poll_logs_a_failed_otel_update(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""A broken OTEL exporter must not break polling, but must leave a trace."""
monkeypatch.setattr(SubscriptionTracker, "_load_persisted_state", lambda self: None)
tracker = SubscriptionTracker()
tracker.notify_active("Bearer live-oauth-token", from_local_operator=True)
monkeypatch.setattr("headroom.subscription.client.read_cached_oauth_token", lambda: None)
snapshot = _make_snapshot()
async def fetch_snapshot(token: str | None) -> SubscriptionSnapshot:
return snapshot
def broken_exporter(state: dict) -> None:
raise RuntimeError("exporter down")
tracker._client = SimpleNamespace(fetch=fetch_snapshot)
monkeypatch.setattr(
tracker_module, "_compute_window_tokens_for_snapshot", lambda snap: WindowTokens(input=7)
)
monkeypatch.setattr(tracker_module, "_detect_discrepancies", lambda snap, tokens: [])
monkeypatch.setattr(tracker, "_persist_state", lambda: None)
monkeypatch.setitem(
sys.modules,
"headroom.observability.metrics",
SimpleNamespace(
get_otel_metrics=lambda: SimpleNamespace(record_subscription_window=broken_exporter)
),
)
with caplog.at_level("DEBUG", logger="headroom.subscription.tracker"):
await tracker._maybe_poll()
assert tracker.latest_snapshot is snapshot
assert tracker._state.last_error is None
assert "Subscription OTEL metrics update failed: exporter down" in [
r.getMessage() for r in caplog.records if r.name == "headroom.subscription.tracker"
]
def test_persist_and_load_state_round_trip(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
persist_path = tmp_path / "tracker-state.json"
tracker = SubscriptionTracker(persist_path=persist_path)