mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 16:45:34 +08:00
## Description
Make an explicit HEADROOM_TELEMETRY=off or --no-telemetry choice stop
the anonymous upload beacon as well as local telemetry, while preserving
the separately configured HEADROOM_BEACON=on override and existing
offline/DO_NOT_TRACK policy.
Existing persistent manifests already contain explicit off values
written by the old installer defaults. Those installs stay opted out
after upgrading unless deliberately reconfigured or reinstalled. The PR
does not guess which saved off values were intentional. New installs
omit an unspecified telemetry choice so they retain the existing default
beacon policy.
## Type of Change
- [x] Bug fix (telemetry opt-out behavior)
- [x] Documentation update
## Changes Made
- Apply an explicit local-telemetry opt-out to the beacon enable
decision and report the reason in telemetry status.
- Recheck enablement before sending buffered session telemetry.
- Preserve unspecified, enabled, and disabled telemetry choices when
generating installation manifests.
- Update operator documentation and regression coverage.
## Testing
- [x] Maintainer telemetry, beacon, and snapshot suites passed after the
current-main sync.
- [x] Whitespace check passed.
### Test Output
```text
python -m pytest tests/test_telemetry_warning.py tests/test_proxy_telemetry_env.py tests/test_beacon_schema_v2.py tests/test_beacon_gzip.py tests/test_original_snapshot_in_place_hook.py -q
122 passed, 1 warning in 21.19s
```
A previous broader installer run also reproduced the persistent-service
embedder assertion on current main independently; that baseline failure
was not attributed to this PR. The other 34 planner tests passed in that
qualification. Hosted CI is qualifying head
d2f8582f97.
## Real Behavior Proof
- Environment: isolated Windows/Python 3.12.13 maintainer worktree with
actual telemetry/beacon decision logic and proxy fixtures.
- Exact command / steps: run the five pytest suites shown above,
including explicit opt-out, override, and buffered-session regressions.
- Observed result: all 122 selected tests passed after incorporating
current main. Earlier installer-choice regression coverage and the
independent main comparison are recorded in the qualification evidence.
- Not tested: independent maintainer live upload traffic to the
production collector or migration of real persistent installs. The
author's live status/proxy before/after observations are preserved below
and remain attributed to the author.
## Runtime Rollout Safety
- Rollout-managed feature(s): existing telemetry and beacon controls; no
new rollout framework toggle.
- Minimum rollout channel: existing deployment policy applies.
- Stable/default behavior changed: an explicit telemetry opt-out also
disables beacon uploads unless a permissible explicit beacon-on override
is configured. Saved legacy off values remain off; new unspecified
install choices are no longer persisted as off.
- Kill switch / disable path: HEADROOM_BEACON=off, DO_NOT_TRACK=1, or
offline mode under the existing policy.
- Unsafe override required: no.
- Qualification impact: operator intent and existing manifest behavior
must be considered during upgrade; hosted CI and human review remain
required.
- Rollback path: retain the prior qualified release; configure
HEADROOM_BEACON explicitly where appropriate rather than rewriting saved
user intent.
## Review Readiness
- [ ] I have performed a self-review
- [ ] This PR is ready for human review
Maintainer verification is documented above; author readiness
attestations have not been filled on the author's behalf.
## Additional Notes
The original author warning, description, follow-up exclusions, and
supplied evidence are preserved below.
> **Behaviour change for existing installs.** Read this before merging.
Every manifest that `headroom install` saved before this change already
contains `HEADROOM_TELEMETRY=off` and `--no-telemetry`, because the
planner wrote them by default. Those manifests are replayed as-is, so
**existing persistent installs will report the beacon OFF after
upgrading** until they are reinstalled. A loader migration can't tell
those defaults apart from a real opt-out, and turning uploads back on
for people who opted out would be worse, so this PR doesn't migrate. New
installs keep the beacon on by default.
#### The test we ran
A Daytona sandbox running Headroom Enterprise 0.4.3 (core 0.39.1), with
Codex CLI 0.162.1 and Claude Code connected to real GitHub, Context7 and
Fetch MCP servers, working through the workshop tasks.
#### What we found
`headroom telemetry --show` and the README say telemetry can be turned
off, and users reasonably set **`HEADROOM_TELEMETRY=off`** (or pass
`--no-telemetry`). But the anonymous upload beacon was gated only on
`HEADROOM_BEACON` / `DO_NOT_TRACK` / offline mode. With
`HEADROOM_TELEMETRY=off` it kept uploading, and `headroom telemetry`
still printed `Upload beacon: ON (opt-out)`.
#### The fix
- New `beacon_off_reason()` in `headroom/telemetry/beacon.py` names the
setting that turns the beacon off. `is_beacon_enabled()` delegates to
it. The order is:
1. offline mode;
2. `DO_NOT_TRACK`;
3. `HEADROOM_BEACON`, whose explicit `on` still wins over local stats
being off;
4. an explicit **`HEADROOM_TELEMETRY` off-value**: off / false / 0 / no
/ disable / disabled, the same `_OFF_VALUES` the module already uses.
`HEADROOM_TELEMETRY=on` still means only "local stats". It never
authorises upload.
- A session buffered before an opt-out is not sent afterwards:
`post_session_event` (heartbeat, idle close) and the exit flush re-check
`is_beacon_enabled()` at send time.
- `headroom telemetry` prints `Upload beacon: OFF
(HEADROOM_TELEMETRY=off)` (or whichever switch is in effect), and the
proxy startup log does the same.
- `headroom install`: since `off` now stops the beacon, the planner
writes `HEADROOM_TELEMETRY` / `--telemetry` / `--no-telemetry` only for
an explicit choice (`telemetry_enabled` is now tri-state, `None` = no
choice). Local stats stay off without the env var because they are
opt-in. `scripts/install.sh` and `install.ps1` already passed
`--no-telemetry` only on an explicit flag.
- Docs: README, proxy, security-model and limitations pages, the
dashboard tooltip, and the `--no-telemetry` help text.
- `test_banner_beacon_default_on_is_surfaced` asserted the old policy
(`--no-telemetry` leaves the beacon on). It now checks the default case
instead.
#### Before / after
| env | `is_beacon_enabled()` on origin/main | this PR | `headroom
telemetry` (this PR) |
|---|---|---|---|
| none | True | True | `ON (opt-out)` |
| `HEADROOM_TELEMETRY=off` | **True** | **False** | `OFF
(HEADROOM_TELEMETRY=off)` |
| `HEADROOM_TELEMETRY=off` + `HEADROOM_BEACON=on` | True | True | `ON
(opt-out)` |
| `HEADROOM_BEACON=off` | False | False | `OFF (HEADROOM_BEACON=off)` |
| `headroom install` with no flag → manifest | `HEADROOM_TELEMETRY=off`
+ `--no-telemetry` | neither written | — |
#### Tests
-
`tests/test_telemetry_warning.py::test_telemetry_off_turns_the_beacon_off_and_says_so`,
parametrized over the 7 off spellings: fails on origin/main, passes
here.
-
`tests/test_telemetry_warning.py::test_beacon_on_overrides_telemetry_off`.
-
`tests/test_telemetry_warning.py::test_buffered_session_is_not_sent_after_telemetry_off`:
fails on the first commit of this PR, passes now.
-
`tests/test_install/test_planner.py::test_install_writes_telemetry_only_when_chosen`,
through the CLI's `_build_deployment_manifest`: the no-flag case fails
on origin/main.
- Related suites pass: install, telemetry warning, beacon schema v2,
install CLI and proxy health (435 passed).
- ruff check, ruff format --check and mypy pass.
#### Follow-ups (not in this PR)
- `headroom init` passes `telemetry_enabled=True`, so its manifests turn
local stats on (`cli/init.py:718`). This doesn't affect the beacon.
- The `--telemetry` help text still says "anonymous telemetry", but the
flag only turns on local stats.
#### Real behavior proof
- **Setup:** macOS 26.7, Python 3.12.13. origin/main and this branch
were run side by side with `HEADROOM_BEACON` and `DO_NOT_TRACK` unset.
- `HEADROOM_TELEMETRY=off headroom telemetry`:
- origin/main: `Upload beacon: ON (opt-out)`, `is_beacon_enabled():
True`
- this branch: `Upload beacon: OFF (HEADROOM_TELEMETRY=off)`,
`is_beacon_enabled(): False`
- `headroom proxy --no-telemetry --no-optimize`, startup log:
- origin/main: `Anonymous upload beacon: ENABLED (default — session
summaries are sent to Headroom Labs). Opt out: HEADROOM_BEACON=off or
DO_NOT_TRACK=1`
- this branch: `Anonymous upload beacon: DISABLED
(HEADROOM_TELEMETRY=off)`
- **Not tested:** a real `headroom install apply` of a persistent
service on this branch; the manifest change is covered through
`_build_deployment_manifest`. Upgrading an existing install was not
tested either; see the warning at the top.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
487 lines
20 KiB
Python
487 lines
20 KiB
Python
"""Tests for anonymous telemetry warning feature.
|
||
|
||
Covers:
|
||
- is_telemetry_warn_enabled() feature flag
|
||
- format_telemetry_notice() helper
|
||
- proxy CLI banner includes telemetry status
|
||
- wrap CLI prints telemetry notice
|
||
- /stats endpoint exposes anon_telemetry_shipping flag
|
||
"""
|
||
|
||
from types import SimpleNamespace
|
||
from unittest.mock import patch
|
||
|
||
import pytest
|
||
|
||
click = pytest.importorskip("click")
|
||
from click.testing import CliRunner # noqa: E402
|
||
|
||
from headroom.telemetry.beacon import ( # noqa: E402
|
||
format_telemetry_notice,
|
||
is_beacon_enabled,
|
||
is_telemetry_warn_enabled,
|
||
)
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# is_telemetry_warn_enabled
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
class TestIsTelemetryWarnEnabled:
|
||
"""Tests for the HEADROOM_TELEMETRY_WARN feature flag."""
|
||
|
||
def test_enabled_by_default(self, monkeypatch):
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
assert is_telemetry_warn_enabled() is True
|
||
|
||
@pytest.mark.parametrize("value", ["off", "OFF", "false", "0", "no", "disable", "disabled"])
|
||
def test_disabled_by_env_var(self, monkeypatch, value):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY_WARN", value)
|
||
assert is_telemetry_warn_enabled() is False
|
||
|
||
@pytest.mark.parametrize("value", ["on", "ON", "1", "yes", "true"])
|
||
def test_enabled_by_truthy_env_var(self, monkeypatch, value):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY_WARN", value)
|
||
assert is_telemetry_warn_enabled() is True
|
||
|
||
|
||
@pytest.mark.parametrize("value", ["off", "OFF", "false", "0", "no", "disable", "disabled"])
|
||
def test_telemetry_off_turns_the_beacon_off_and_says_so(monkeypatch, value):
|
||
"""``HEADROOM_TELEMETRY=off`` means "send nothing": it used to leave the
|
||
upload beacon on, and only ``HEADROOM_BEACON=off`` stopped it."""
|
||
from headroom.cli.main import main
|
||
|
||
for var in ("HEADROOM_BEACON", "DO_NOT_TRACK", "HEADROOM_OFFLINE"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", value)
|
||
|
||
assert is_beacon_enabled() is False
|
||
output = CliRunner().invoke(main, ["telemetry"]).output
|
||
assert f"Upload beacon: OFF (HEADROOM_TELEMETRY={value.lower()})" in output
|
||
|
||
|
||
def test_buffered_session_is_not_sent_after_telemetry_off(monkeypatch):
|
||
"""A session recorded while the beacon was on must not upload once the
|
||
operator opts out, at heartbeat/idle close or at exit."""
|
||
from headroom.telemetry import session as S
|
||
|
||
for var in ("HEADROOM_BEACON", "DO_NOT_TRACK", "HEADROOM_OFFLINE"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
sent: list[dict] = []
|
||
monkeypatch.setattr(S, "_post_blocking", lambda payload, **kwargs: sent.append(payload))
|
||
aggregator = S.SessionAggregator(S.post_session_event, idle_s=900.0, flush_s=100.0)
|
||
monkeypatch.setattr(S, "_aggregator", aggregator)
|
||
outcome = SimpleNamespace(provider="openai", model="gpt-5.4", tokens_saved=10, tags={})
|
||
aggregator.record(outcome, now=1_000.0)
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||
|
||
aggregator.record(outcome, now=1_150.0) # past flush_s: heartbeat
|
||
aggregator.record(outcome, now=3_000.0) # past idle_s: idle close
|
||
S._flush_at_exit()
|
||
|
||
assert sent == []
|
||
|
||
|
||
def test_beacon_on_overrides_telemetry_off(monkeypatch):
|
||
"""HEADROOM_BEACON is the beacon's own switch and still wins."""
|
||
for var in ("DO_NOT_TRACK", "HEADROOM_OFFLINE"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "on")
|
||
|
||
assert is_beacon_enabled() is True
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# format_telemetry_notice
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
class TestFormatTelemetryNotice:
|
||
"""Tests for format_telemetry_notice()."""
|
||
|
||
def test_returns_notice_when_telemetry_on(self, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
notice = format_telemetry_notice()
|
||
assert notice != ""
|
||
assert "ENABLED" in notice
|
||
assert "HEADROOM_TELEMETRY=off" in notice
|
||
assert "--no-telemetry" in notice
|
||
|
||
def test_empty_when_telemetry_off(self, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
assert format_telemetry_notice() == ""
|
||
|
||
def test_beacon_is_announced_by_default(self, monkeypatch):
|
||
"""The beacon is opt-out, so the notice is the only place a user finds
|
||
out it is running. Silence here is how anonymous telemetry becomes a
|
||
trust incident."""
|
||
for var in ("HEADROOM_TELEMETRY", "HEADROOM_BEACON", "DO_NOT_TRACK"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
notice = format_telemetry_notice()
|
||
# Substance, not the exact phrasing: what is sent (counters), what is
|
||
# not (prompts), and how to turn it off. The wording widened at schema
|
||
# v2 — "compression stats" stopped describing a payload that also
|
||
# carries cache behaviour, session shape and configuration.
|
||
assert "usage counters" in notice
|
||
assert "HEADROOM_BEACON=off" in notice
|
||
|
||
def test_beacon_notice_names_what_is_not_sent(self, monkeypatch):
|
||
"""Vague reassurance is worse than none. The notice has to name the
|
||
three things users actually worry about."""
|
||
for var in ("HEADROOM_TELEMETRY", "HEADROOM_BEACON", "DO_NOT_TRACK"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
notice = format_telemetry_notice()
|
||
assert "never prompts" in notice
|
||
assert "code" in notice
|
||
assert "file paths" in notice
|
||
|
||
def test_silent_when_beacon_disabled_and_no_local(self, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
for var in ("HEADROOM_TELEMETRY", "DO_NOT_TRACK"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
assert format_telemetry_notice() == ""
|
||
|
||
def test_do_not_track_silences_the_beacon_notice(self, monkeypatch):
|
||
monkeypatch.setenv("DO_NOT_TRACK", "1")
|
||
for var in ("HEADROOM_TELEMETRY", "HEADROOM_BEACON"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
assert format_telemetry_notice() == ""
|
||
|
||
def test_empty_when_warn_flag_off(self, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY_WARN", "off")
|
||
assert format_telemetry_notice() == ""
|
||
|
||
def test_prefix_is_applied(self, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
notice = format_telemetry_notice(prefix=" ")
|
||
assert notice.startswith(" ")
|
||
|
||
def test_no_prefix_by_default(self, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
notice = format_telemetry_notice()
|
||
# Default prefix is "" so the string should start with "Telemetry"
|
||
assert notice.startswith("Telemetry:")
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# proxy CLI banner
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
class TestProxyCLITelemetryBanner:
|
||
"""Proxy CLI startup banner must include telemetry status.
|
||
|
||
Every scenario below sets HEADROOM_BEACON=off explicitly, isolating the
|
||
local-telemetry-only wording (mirrors the convention already used by
|
||
TestFormatTelemetryNotice, e.g. test_returns_notice_when_telemetry_on).
|
||
Before the beacon-disclosure fix the banner never looked at the beacon
|
||
at all, so these tests would have passed unchanged whether the beacon
|
||
was on or off -- which is exactly the bug: an operator relying on this
|
||
banner had no way to tell the two apart. The two tests at the bottom of
|
||
this class (*_beacon_default_on_is_surfaced,
|
||
*_beacon_off_local_off_says_fully_off) cover the beacon-on-by-default
|
||
case the old banner never surfaced.
|
||
"""
|
||
|
||
@pytest.fixture
|
||
def runner(self):
|
||
return CliRunner()
|
||
|
||
def test_banner_shows_telemetry_enabled(self, runner, monkeypatch):
|
||
# Telemetry is opt-in: it only shows ENABLED once explicitly turned on.
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy"])
|
||
|
||
assert "Telemetry:" in result.output
|
||
assert "ENABLED" in result.output
|
||
|
||
def test_banner_disabled_by_default(self, runner, monkeypatch):
|
||
# The whole point of opt-in: unset env => local telemetry off, banner
|
||
# says so and surfaces how to opt in. (Beacon pinned off here so this
|
||
# isolates the local-only wording; see the beacon-specific tests
|
||
# below for the on-by-default beacon case.)
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY", raising=False)
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy"])
|
||
|
||
assert "Telemetry:" in result.output
|
||
assert "OFF" in result.output
|
||
assert "HEADROOM_TELEMETRY=on" in result.output
|
||
|
||
def test_telemetry_flag_opts_in(self, runner, monkeypatch):
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY", raising=False)
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy", "--telemetry"])
|
||
|
||
assert "Telemetry:" in result.output
|
||
assert "ENABLED" in result.output
|
||
|
||
def test_banner_shows_telemetry_disabled(self, runner, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy"])
|
||
|
||
assert "Telemetry:" in result.output
|
||
assert "OFF" in result.output
|
||
|
||
def test_no_telemetry_flag_disables(self, runner, monkeypatch):
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY", raising=False)
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy", "--no-telemetry"])
|
||
|
||
assert "Telemetry:" in result.output
|
||
assert "OFF" in result.output
|
||
|
||
def test_banner_shows_opt_out_instructions_when_enabled(self, runner, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy"])
|
||
|
||
assert "HEADROOM_TELEMETRY=off" in result.output or "--no-telemetry" in result.output
|
||
|
||
def test_banner_beacon_default_on_is_surfaced(self, runner, monkeypatch):
|
||
"""With no switch set the banner discloses the opt-out beacon rather than
|
||
printing a bare "Telemetry: DISABLED" for the local-stats switch alone."""
|
||
for var in ("HEADROOM_TELEMETRY", "HEADROOM_BEACON", "DO_NOT_TRACK"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy"])
|
||
|
||
assert "Telemetry:" in result.output
|
||
assert "usage counters" in result.output
|
||
assert "HEADROOM_BEACON=off" in result.output
|
||
telemetry_line = next(
|
||
line for line in result.output.splitlines() if line.strip().startswith("Telemetry:")
|
||
)
|
||
assert "DISABLED" not in telemetry_line
|
||
|
||
def test_banner_beacon_off_and_local_off_says_fully_off(self, runner, monkeypatch):
|
||
"""Only when BOTH switches are off does the banner claim nothing is
|
||
happening -- the one case where a bare OFF claim is actually true."""
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.main import main
|
||
|
||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||
result = runner.invoke(main, ["proxy"])
|
||
|
||
assert "Telemetry:" in result.output
|
||
assert "OFF" in result.output
|
||
assert "compression stats" not in result.output
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# wrap CLI telemetry notice
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
class TestWrapCLITelemetryNotice:
|
||
"""_print_telemetry_notice() is called from wrap commands."""
|
||
|
||
def test_print_notice_outputs_when_telemetry_on(self, monkeypatch, capsys):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
|
||
from headroom.cli.wrap import _print_telemetry_notice
|
||
|
||
_print_telemetry_notice()
|
||
captured = capsys.readouterr()
|
||
assert "Telemetry" in captured.out
|
||
assert "HEADROOM_TELEMETRY=off" in captured.out
|
||
|
||
def test_print_notice_announces_beacon_by_default(self, monkeypatch, capsys):
|
||
for var in ("HEADROOM_TELEMETRY", "HEADROOM_BEACON", "DO_NOT_TRACK"):
|
||
monkeypatch.delenv(var, raising=False)
|
||
monkeypatch.delenv("HEADROOM_TELEMETRY_WARN", raising=False)
|
||
|
||
from headroom.cli.wrap import _print_telemetry_notice
|
||
|
||
_print_telemetry_notice()
|
||
captured = capsys.readouterr()
|
||
assert "usage counters" in captured.out
|
||
assert "HEADROOM_BEACON=off" in captured.out
|
||
|
||
def test_print_notice_silent_when_telemetry_off(self, monkeypatch, capsys):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
|
||
from headroom.cli.wrap import _print_telemetry_notice
|
||
|
||
_print_telemetry_notice()
|
||
captured = capsys.readouterr()
|
||
assert captured.out == ""
|
||
|
||
def test_print_notice_silent_when_warn_flag_off(self, monkeypatch, capsys):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY_WARN", "off")
|
||
|
||
from headroom.cli.wrap import _print_telemetry_notice
|
||
|
||
_print_telemetry_notice()
|
||
captured = capsys.readouterr()
|
||
assert captured.out == ""
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# /stats endpoint – anon_telemetry_shipping flag
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
@pytest.mark.asyncio
|
||
class TestStatsEndpointTelemetryFlag:
|
||
"""The /stats endpoint's anon_telemetry_shipping field must reflect the
|
||
live HEADROOM_BEACON state, not a hardcoded constant.
|
||
|
||
Previously this field was hardcoded to False on the premise that "the
|
||
anonymous telemetry beacon was removed" -- it was not (telemetry/beacon.py
|
||
and telemetry/session.py fully implement and wire it via
|
||
record_outcome() -> SessionAggregator -> a POST to
|
||
headroom-beacon.headroom-beacon.workers.dev, gated on is_beacon_enabled(),
|
||
which is ON BY DEFAULT). An operator polling /stats to confirm nothing
|
||
ships externally got a false assurance regardless of their actual
|
||
HEADROOM_BEACON setting. These two tests previously asserted `False`
|
||
unconditionally, encoding the same incorrect premise as the field they
|
||
were testing.
|
||
"""
|
||
|
||
pytest.importorskip("fastapi")
|
||
|
||
async def test_stats_anon_telemetry_shipping_reflects_beacon_on(self, monkeypatch):
|
||
# HEADROOM_BEACON is on by default (BEACON_DEFAULT_ON = True) and
|
||
# HEADROOM_TELEMETRY does not touch it -- setting local telemetry on
|
||
# must not by itself make this field misreport False.
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "on")
|
||
monkeypatch.delenv("HEADROOM_BEACON", raising=False)
|
||
monkeypatch.delenv("DO_NOT_TRACK", raising=False)
|
||
from headroom.proxy.server import ProxyConfig, create_app
|
||
|
||
app = create_app(
|
||
ProxyConfig(
|
||
cache_enabled=False,
|
||
rate_limit_enabled=False,
|
||
cost_tracking_enabled=False,
|
||
)
|
||
)
|
||
|
||
from httpx import ASGITransport, AsyncClient
|
||
|
||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
|
||
resp = await client.get("/stats")
|
||
|
||
assert resp.status_code == 200
|
||
data = resp.json()
|
||
assert "anon_telemetry_shipping" in data
|
||
assert data["anon_telemetry_shipping"] is True
|
||
|
||
async def test_stats_anon_telemetry_shipping_false_when_beacon_off(self, monkeypatch):
|
||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||
monkeypatch.setenv("HEADROOM_BEACON", "off")
|
||
from headroom.proxy.server import ProxyConfig, create_app
|
||
|
||
app = create_app(
|
||
ProxyConfig(
|
||
cache_enabled=False,
|
||
rate_limit_enabled=False,
|
||
cost_tracking_enabled=False,
|
||
)
|
||
)
|
||
|
||
from httpx import ASGITransport, AsyncClient
|
||
|
||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
|
||
resp = await client.get("/stats")
|
||
|
||
assert resp.status_code == 200
|
||
data = resp.json()
|
||
assert "anon_telemetry_shipping" in data
|
||
assert data["anon_telemetry_shipping"] is False
|
||
|
||
async def test_stats_anon_telemetry_shipping_false_when_do_not_track(self, monkeypatch):
|
||
# DO_NOT_TRACK silences the beacon regardless of HEADROOM_BEACON.
|
||
monkeypatch.setenv("DO_NOT_TRACK", "1")
|
||
monkeypatch.delenv("HEADROOM_BEACON", raising=False)
|
||
from headroom.proxy.server import ProxyConfig, create_app
|
||
|
||
app = create_app(
|
||
ProxyConfig(
|
||
cache_enabled=False,
|
||
rate_limit_enabled=False,
|
||
cost_tracking_enabled=False,
|
||
)
|
||
)
|
||
|
||
from httpx import ASGITransport, AsyncClient
|
||
|
||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
|
||
resp = await client.get("/stats")
|
||
|
||
assert resp.status_code == 200
|
||
data = resp.json()
|
||
assert data["anon_telemetry_shipping"] is False
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# telemetry __init__ exports
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
class TestTelemetryModuleExports:
|
||
"""New helpers must be exported from headroom.telemetry."""
|
||
|
||
def test_is_telemetry_warn_enabled_exported(self):
|
||
from headroom.telemetry import is_telemetry_warn_enabled as fn
|
||
|
||
assert callable(fn)
|
||
|
||
def test_is_telemetry_enabled_exported(self):
|
||
from headroom.telemetry import is_telemetry_enabled as fn
|
||
|
||
assert callable(fn)
|
||
|
||
def test_format_telemetry_notice_exported(self):
|
||
from headroom.telemetry import format_telemetry_notice as fn
|
||
|
||
assert callable(fn)
|