mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(telemetry): let HEADROOM_TELEMETRY=off stop the anonymous beacon (#4235)
## Description
Make an explicit HEADROOM_TELEMETRY=off or --no-telemetry choice stop
the anonymous upload beacon as well as local telemetry, while preserving
the separately configured HEADROOM_BEACON=on override and existing
offline/DO_NOT_TRACK policy.
Existing persistent manifests already contain explicit off values
written by the old installer defaults. Those installs stay opted out
after upgrading unless deliberately reconfigured or reinstalled. The PR
does not guess which saved off values were intentional. New installs
omit an unspecified telemetry choice so they retain the existing default
beacon policy.
## Type of Change
- [x] Bug fix (telemetry opt-out behavior)
- [x] Documentation update
## Changes Made
- Apply an explicit local-telemetry opt-out to the beacon enable
decision and report the reason in telemetry status.
- Recheck enablement before sending buffered session telemetry.
- Preserve unspecified, enabled, and disabled telemetry choices when
generating installation manifests.
- Update operator documentation and regression coverage.
## Testing
- [x] Maintainer telemetry, beacon, and snapshot suites passed after the
current-main sync.
- [x] Whitespace check passed.
### Test Output
```text
python -m pytest tests/test_telemetry_warning.py tests/test_proxy_telemetry_env.py tests/test_beacon_schema_v2.py tests/test_beacon_gzip.py tests/test_original_snapshot_in_place_hook.py -q
122 passed, 1 warning in 21.19s
```
A previous broader installer run also reproduced the persistent-service
embedder assertion on current main independently; that baseline failure
was not attributed to this PR. The other 34 planner tests passed in that
qualification. Hosted CI is qualifying head
d2f8582f97.
## Real Behavior Proof
- Environment: isolated Windows/Python 3.12.13 maintainer worktree with
actual telemetry/beacon decision logic and proxy fixtures.
- Exact command / steps: run the five pytest suites shown above,
including explicit opt-out, override, and buffered-session regressions.
- Observed result: all 122 selected tests passed after incorporating
current main. Earlier installer-choice regression coverage and the
independent main comparison are recorded in the qualification evidence.
- Not tested: independent maintainer live upload traffic to the
production collector or migration of real persistent installs. The
author's live status/proxy before/after observations are preserved below
and remain attributed to the author.
## Runtime Rollout Safety
- Rollout-managed feature(s): existing telemetry and beacon controls; no
new rollout framework toggle.
- Minimum rollout channel: existing deployment policy applies.
- Stable/default behavior changed: an explicit telemetry opt-out also
disables beacon uploads unless a permissible explicit beacon-on override
is configured. Saved legacy off values remain off; new unspecified
install choices are no longer persisted as off.
- Kill switch / disable path: HEADROOM_BEACON=off, DO_NOT_TRACK=1, or
offline mode under the existing policy.
- Unsafe override required: no.
- Qualification impact: operator intent and existing manifest behavior
must be considered during upgrade; hosted CI and human review remain
required.
- Rollback path: retain the prior qualified release; configure
HEADROOM_BEACON explicitly where appropriate rather than rewriting saved
user intent.
## Review Readiness
- [ ] I have performed a self-review
- [ ] This PR is ready for human review
Maintainer verification is documented above; author readiness
attestations have not been filled on the author's behalf.
## Additional Notes
The original author warning, description, follow-up exclusions, and
supplied evidence are preserved below.
> **Behaviour change for existing installs.** Read this before merging.
Every manifest that `headroom install` saved before this change already
contains `HEADROOM_TELEMETRY=off` and `--no-telemetry`, because the
planner wrote them by default. Those manifests are replayed as-is, so
**existing persistent installs will report the beacon OFF after
upgrading** until they are reinstalled. A loader migration can't tell
those defaults apart from a real opt-out, and turning uploads back on
for people who opted out would be worse, so this PR doesn't migrate. New
installs keep the beacon on by default.
#### The test we ran
A Daytona sandbox running Headroom Enterprise 0.4.3 (core 0.39.1), with
Codex CLI 0.162.1 and Claude Code connected to real GitHub, Context7 and
Fetch MCP servers, working through the workshop tasks.
#### What we found
`headroom telemetry --show` and the README say telemetry can be turned
off, and users reasonably set **`HEADROOM_TELEMETRY=off`** (or pass
`--no-telemetry`). But the anonymous upload beacon was gated only on
`HEADROOM_BEACON` / `DO_NOT_TRACK` / offline mode. With
`HEADROOM_TELEMETRY=off` it kept uploading, and `headroom telemetry`
still printed `Upload beacon: ON (opt-out)`.
#### The fix
- New `beacon_off_reason()` in `headroom/telemetry/beacon.py` names the
setting that turns the beacon off. `is_beacon_enabled()` delegates to
it. The order is:
1. offline mode;
2. `DO_NOT_TRACK`;
3. `HEADROOM_BEACON`, whose explicit `on` still wins over local stats
being off;
4. an explicit **`HEADROOM_TELEMETRY` off-value**: off / false / 0 / no
/ disable / disabled, the same `_OFF_VALUES` the module already uses.
`HEADROOM_TELEMETRY=on` still means only "local stats". It never
authorises upload.
- A session buffered before an opt-out is not sent afterwards:
`post_session_event` (heartbeat, idle close) and the exit flush re-check
`is_beacon_enabled()` at send time.
- `headroom telemetry` prints `Upload beacon: OFF
(HEADROOM_TELEMETRY=off)` (or whichever switch is in effect), and the
proxy startup log does the same.
- `headroom install`: since `off` now stops the beacon, the planner
writes `HEADROOM_TELEMETRY` / `--telemetry` / `--no-telemetry` only for
an explicit choice (`telemetry_enabled` is now tri-state, `None` = no
choice). Local stats stay off without the env var because they are
opt-in. `scripts/install.sh` and `install.ps1` already passed
`--no-telemetry` only on an explicit flag.
- Docs: README, proxy, security-model and limitations pages, the
dashboard tooltip, and the `--no-telemetry` help text.
- `test_banner_beacon_default_on_is_surfaced` asserted the old policy
(`--no-telemetry` leaves the beacon on). It now checks the default case
instead.
#### Before / after
| env | `is_beacon_enabled()` on origin/main | this PR | `headroom
telemetry` (this PR) |
|---|---|---|---|
| none | True | True | `ON (opt-out)` |
| `HEADROOM_TELEMETRY=off` | **True** | **False** | `OFF
(HEADROOM_TELEMETRY=off)` |
| `HEADROOM_TELEMETRY=off` + `HEADROOM_BEACON=on` | True | True | `ON
(opt-out)` |
| `HEADROOM_BEACON=off` | False | False | `OFF (HEADROOM_BEACON=off)` |
| `headroom install` with no flag → manifest | `HEADROOM_TELEMETRY=off`
+ `--no-telemetry` | neither written | — |
#### Tests
-
`tests/test_telemetry_warning.py::test_telemetry_off_turns_the_beacon_off_and_says_so`,
parametrized over the 7 off spellings: fails on origin/main, passes
here.
-
`tests/test_telemetry_warning.py::test_beacon_on_overrides_telemetry_off`.
-
`tests/test_telemetry_warning.py::test_buffered_session_is_not_sent_after_telemetry_off`:
fails on the first commit of this PR, passes now.
-
`tests/test_install/test_planner.py::test_install_writes_telemetry_only_when_chosen`,
through the CLI's `_build_deployment_manifest`: the no-flag case fails
on origin/main.
- Related suites pass: install, telemetry warning, beacon schema v2,
install CLI and proxy health (435 passed).
- ruff check, ruff format --check and mypy pass.
#### Follow-ups (not in this PR)
- `headroom init` passes `telemetry_enabled=True`, so its manifests turn
local stats on (`cli/init.py:718`). This doesn't affect the beacon.
- The `--telemetry` help text still says "anonymous telemetry", but the
flag only turns on local stats.
#### Real behavior proof
- **Setup:** macOS 26.7, Python 3.12.13. origin/main and this branch
were run side by side with `HEADROOM_BEACON` and `DO_NOT_TRACK` unset.
- `HEADROOM_TELEMETRY=off headroom telemetry`:
- origin/main: `Upload beacon: ON (opt-out)`, `is_beacon_enabled():
True`
- this branch: `Upload beacon: OFF (HEADROOM_TELEMETRY=off)`,
`is_beacon_enabled(): False`
- `headroom proxy --no-telemetry --no-optimize`, startup log:
- origin/main: `Anonymous upload beacon: ENABLED (default — session
summaries are sent to Headroom Labs). Opt out: HEADROOM_BEACON=off or
DO_NOT_TRACK=1`
- this branch: `Anonymous upload beacon: DISABLED
(HEADROOM_TELEMETRY=off)`
- **Not tested:** a real `headroom install apply` of a persistent
service on this branch; the manifest change is covered through
`_build_deployment_manifest`. Upgrading an existing install was not
tested either; see the warning at the top.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
ff36931653
commit
1a40b3ce67
@@ -624,8 +624,8 @@ prompts, completions, code or file paths. It exists so we can see when a release
|
||||
regresses a compression ratio across real workloads rather than only our own test
|
||||
corpus.
|
||||
|
||||
Turn it off with `HEADROOM_BEACON=off`, the `DO_NOT_TRACK=1` convention, or
|
||||
`HEADROOM_OFFLINE=1`. The full field list is in
|
||||
Turn it off with `HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off`, the
|
||||
`DO_NOT_TRACK=1` convention, or `HEADROOM_OFFLINE=1`. The full field list is in
|
||||
[the proxy docs](https://docs.headroomlabs.ai/docs/proxy).
|
||||
|
||||
## Headroom for teams
|
||||
|
||||
@@ -174,10 +174,12 @@ this page's "nothing leaves the machine" framing as universal:
|
||||
compression ratios, skip reasons, provider and model ids, failure counts, plus
|
||||
OS and architecture. Never prompts, completions, code, or file paths -- the
|
||||
collector allowlists fields server-side and drops the rest. Disable it with
|
||||
`HEADROOM_BEACON=off`, the cross-tool `DO_NOT_TRACK=1` convention, or by
|
||||
`HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off`, the cross-tool `DO_NOT_TRACK=1` convention, or by
|
||||
running offline. `is_beacon_enabled()` is fail-open: an unrecognized value
|
||||
uploads rather than staying silent. See [Proxy](/docs/proxy) for the full
|
||||
field list.
|
||||
|
||||
These are deliberately separate so that enabling local `/stats` does not
|
||||
silently start an upload on the next release.
|
||||
silently start an upload on the next release. The one crossing only ever turns
|
||||
the upload off: an explicit `HEADROOM_TELEMETRY=off` also stops the beacon
|
||||
(unless `HEADROOM_BEACON=on`).
|
||||
|
||||
@@ -24,7 +24,7 @@ headroom proxy \
|
||||
--budget 100.0
|
||||
```
|
||||
|
||||
Two independent telemetry switches exist. `HEADROOM_TELEMETRY=on` (or `--telemetry`) turns on **local-only** in-process usage stats that power your own `/stats`, `/metrics`, and dashboard — off by default, and nothing under this switch leaves the machine.
|
||||
Two telemetry switches exist. `HEADROOM_TELEMETRY=on` (or `--telemetry`) turns on **local-only** in-process usage stats that power your own `/stats`, `/metrics`, and dashboard — off by default, and nothing under this switch leaves the machine.
|
||||
|
||||
Separately, an **anonymous usage beacon is on by default** (opt-out). Once a session goes idle it POSTs a small summary of how compression behaved to a Headroom Labs collector.
|
||||
|
||||
@@ -36,7 +36,7 @@ Separately, an **anonymous usage beacon is on by default** (opt-out). Once a ses
|
||||
|
||||
The install id is a random UUID generated on first run and stored in your config directory. It is deliberately not derived from hostname, MAC address, or any hardware property — delete the file and you get a new one.
|
||||
|
||||
Disable it with `HEADROOM_BEACON=off`, the `DO_NOT_TRACK=1` convention, or `HEADROOM_OFFLINE=1` (which also disables update checks and license reporting).
|
||||
Disable it with `HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off` (or `--no-telemetry`), the `DO_NOT_TRACK=1` convention, or `HEADROOM_OFFLINE=1` (which also disables update checks and license reporting). `HEADROOM_BEACON=on` keeps the beacon on even with local stats off. `headroom telemetry` names the setting that turned it off.
|
||||
|
||||
## CLI options
|
||||
|
||||
|
||||
@@ -343,7 +343,7 @@ Beyond your configured LLM providers, Headroom may contact:
|
||||
|
||||
| Destination | Purpose | Default | Disable / pre-provision |
|
||||
|---|---|---|---|
|
||||
| `headroom-beacon.headroom-beacon.workers.dev` | Anonymous session summaries to Headroom Labs | **On** — see [The beacon](#the-beacon-is-on-by-default) | `HEADROOM_BEACON=off`, `DO_NOT_TRACK=1`, or `HEADROOM_OFFLINE` |
|
||||
| `headroom-beacon.headroom-beacon.workers.dev` | Anonymous session summaries to Headroom Labs | **On** — see [The beacon](#the-beacon-is-on-by-default) | `HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off`, `DO_NOT_TRACK=1`, or `HEADROOM_OFFLINE` |
|
||||
| `pypi.org` | Daily version check for update notices | On | `HEADROOM_UPDATE_CHECK=off`; also skipped in `--stateless`, in CI, and from source checkouts |
|
||||
| `huggingface.co` | Downloads models for the optional ML compression paths | On first use of those features | Pre-provision the model cache, or leave the features disabled |
|
||||
| `app.headroomlabs.ai` | License validation and usage reporting | Only with a license key configured | Omit the license key |
|
||||
@@ -379,14 +379,17 @@ Prompts, completions, file contents, tool results, and file paths are **not** in
|
||||
payload. The install id is a random UUID rather than a machine fingerprint, and resets
|
||||
if you delete `<config dir>/install_id`.
|
||||
|
||||
Three controls switch it off, checked in this order:
|
||||
Four controls switch it off, checked in this order:
|
||||
|
||||
```bash
|
||||
HEADROOM_OFFLINE=1 # offline mode — disables all egress, not just the beacon
|
||||
DO_NOT_TRACK=1 # the cross-tool convention, honoured
|
||||
HEADROOM_BEACON=off # also false / 0 / no / disable / disabled
|
||||
HEADROOM_OFFLINE=1 # offline mode — disables all egress, not just the beacon
|
||||
DO_NOT_TRACK=1 # the cross-tool convention, honoured
|
||||
HEADROOM_BEACON=off # also false / 0 / no / disable / disabled; =on keeps it on
|
||||
HEADROOM_TELEMETRY=off # same off-values; also what --no-telemetry sets
|
||||
```
|
||||
|
||||
`headroom telemetry` names the setting that turned it off.
|
||||
|
||||
The destination can also be redirected with `HEADROOM_TELEMETRY_ENDPOINT`, which is
|
||||
useful if you would rather terminate the reports at a receiver you control than block
|
||||
them outright.
|
||||
@@ -401,8 +404,8 @@ assuming a typo fails safe.
|
||||
|
||||
`HEADROOM_TELEMETRY` is a **different switch**, off by default and opt-in. It aggregates
|
||||
stats locally for the in-process collector and the `/stats` and `/v1/telemetry`
|
||||
endpoints, and sends nothing off-host. Enabling it does not authorise beacon upload, and
|
||||
disabling it does not suppress the beacon — the two are deliberately independent.
|
||||
endpoints, and sends nothing off-host. Enabling it does not authorise beacon upload; an
|
||||
explicit `HEADROOM_TELEMETRY=off` does suppress the beacon, unless `HEADROOM_BEACON=on`.
|
||||
|
||||
Operational metrics are separate again: OpenTelemetry export
|
||||
(`HEADROOM_OTEL_METRICS_*`) is opt-in and targets a collector you specify.
|
||||
|
||||
@@ -604,7 +604,8 @@ def _build_deployment_manifest(
|
||||
region=region,
|
||||
proxy_mode=proxy_mode,
|
||||
memory_enabled=memory,
|
||||
telemetry_enabled=telemetry and not no_telemetry,
|
||||
# --no-telemetry wins when both are given; neither leaves it unset.
|
||||
telemetry_enabled=False if no_telemetry else (True if telemetry else None),
|
||||
no_rate_limit=no_rate_limit,
|
||||
image=image,
|
||||
no_http2=no_http2,
|
||||
@@ -875,7 +876,7 @@ def _echo_installed(manifest: DeploymentManifest, *, prefix: str = "Installed pe
|
||||
@click.option(
|
||||
"--no-telemetry",
|
||||
is_flag=True,
|
||||
help="Force anonymous telemetry off in the runtime (already the default).",
|
||||
help="Turn off local stats and the anonymous upload beacon in the runtime.",
|
||||
)
|
||||
@click.option(
|
||||
"--no-rate-limit",
|
||||
@@ -1092,7 +1093,7 @@ def install_apply(
|
||||
@click.option(
|
||||
"--no-telemetry",
|
||||
is_flag=True,
|
||||
help="Force anonymous telemetry off in the runtime (already the default).",
|
||||
help="Turn off local stats and the anonymous upload beacon in the runtime.",
|
||||
)
|
||||
@click.option(
|
||||
"--image",
|
||||
|
||||
@@ -1112,7 +1112,7 @@ def dashboard(port: int, no_open: bool) -> None:
|
||||
@click.option(
|
||||
"--no-telemetry",
|
||||
is_flag=True,
|
||||
help="Force anonymous usage telemetry off (already the default; env: HEADROOM_TELEMETRY=off)",
|
||||
help="Turn off local stats and the anonymous upload beacon (env: HEADROOM_TELEMETRY=off)",
|
||||
)
|
||||
@click.option(
|
||||
"--stateless",
|
||||
|
||||
@@ -85,7 +85,7 @@ def telemetry(show: bool, as_json: bool) -> None:
|
||||
headroom telemetry Current state and the two switches
|
||||
headroom telemetry --show The exact payload, field by field
|
||||
"""
|
||||
from headroom.telemetry.beacon import is_beacon_enabled, is_telemetry_enabled
|
||||
from headroom.telemetry.beacon import beacon_off_reason, is_telemetry_enabled
|
||||
from headroom.telemetry.session import (
|
||||
DEFAULT_ENDPOINT,
|
||||
SCHEMA_VERSION,
|
||||
@@ -94,7 +94,8 @@ def telemetry(show: bool, as_json: bool) -> None:
|
||||
resource_attributes,
|
||||
)
|
||||
|
||||
beacon_on = is_beacon_enabled()
|
||||
beacon_off_by = beacon_off_reason()
|
||||
beacon_on = beacon_off_by is None
|
||||
local_on = is_telemetry_enabled()
|
||||
known_id = read_install_id()
|
||||
# Reported because it changes what goes on the wire, and because while the
|
||||
@@ -134,7 +135,7 @@ def telemetry(show: bool, as_json: bool) -> None:
|
||||
)
|
||||
return
|
||||
|
||||
click.echo(f"Upload beacon: {'ON (opt-out)' if beacon_on else 'OFF'}")
|
||||
click.echo(f"Upload beacon: {'ON (opt-out)' if beacon_on else f'OFF ({beacon_off_by})'}")
|
||||
click.echo(f"Local stats: {'ON' if local_on else 'OFF (opt-in)'}")
|
||||
click.echo(f"Endpoint: {DEFAULT_ENDPOINT}")
|
||||
click.echo(
|
||||
@@ -143,5 +144,8 @@ def telemetry(show: bool, as_json: bool) -> None:
|
||||
)
|
||||
click.echo(f"Install ID: {known_id or '(none yet — created on first upload)'}")
|
||||
click.echo()
|
||||
click.echo("Turn the beacon off: HEADROOM_BEACON=off (or DO_NOT_TRACK=1)")
|
||||
if beacon_on:
|
||||
click.echo(
|
||||
"Turn the beacon off: HEADROOM_BEACON=off (or HEADROOM_TELEMETRY=off, DO_NOT_TRACK=1)"
|
||||
)
|
||||
click.echo("See the exact payload: headroom telemetry --show")
|
||||
|
||||
@@ -152,7 +152,7 @@
|
||||
</div>
|
||||
<template x-if="stats.anon_telemetry_shipping">
|
||||
<div class="inline-flex items-center gap-1.5 rounded-full border border-amber-500/40 bg-amber-500/10 px-2.5 py-1"
|
||||
title="The anonymous telemetry beacon is on: compression stats only, never prompts, code, or file paths. Turn it off with HEADROOM_BEACON=off or DO_NOT_TRACK=1.">
|
||||
title="The anonymous telemetry beacon is on: compression stats only, never prompts, code, or file paths. Turn it off with HEADROOM_BEACON=off, HEADROOM_TELEMETRY=off or DO_NOT_TRACK=1.">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"
|
||||
class="w-3 h-3 text-amber-400 shrink-0">
|
||||
<path fill-rule="evenodd"
|
||||
|
||||
@@ -106,7 +106,9 @@ class DeploymentManifest:
|
||||
proxy_mode: str = "cache"
|
||||
memory_enabled: bool = False
|
||||
memory_db_path: str = ""
|
||||
telemetry_enabled: bool = True
|
||||
# None: the operator chose neither --telemetry nor --no-telemetry. The
|
||||
# default stays True for manifests saved before the field existed.
|
||||
telemetry_enabled: bool | None = True
|
||||
image: str = "ghcr.io/headroomlabs-ai/headroom:latest"
|
||||
service_name: str = "headroom"
|
||||
container_name: str = "headroom-persistent"
|
||||
|
||||
@@ -150,7 +150,7 @@ def build_manifest(
|
||||
region: str | None,
|
||||
proxy_mode: str,
|
||||
memory_enabled: bool,
|
||||
telemetry_enabled: bool,
|
||||
telemetry_enabled: bool | None,
|
||||
image: str,
|
||||
no_rate_limit: bool = False,
|
||||
no_http2: bool = False,
|
||||
@@ -201,9 +201,12 @@ def build_manifest(
|
||||
base_env["HEADROOM_ANYLLM_PROVIDER"] = anyllm_provider
|
||||
if region:
|
||||
base_env["HEADROOM_REGION"] = region
|
||||
# Telemetry is opt-in (off by default). Write the value explicitly so the
|
||||
# generated manifest is unambiguous and doesn't depend on the runtime default.
|
||||
base_env["HEADROOM_TELEMETRY"] = "on" if telemetry_enabled else "off"
|
||||
# Write the operator's explicit choice only. ``off`` also stops the upload
|
||||
# beacon (telemetry/beacon.py), so writing it by default would turn the
|
||||
# beacon off for every install nobody asked to opt out. Local stats stay
|
||||
# off without it: they are opt-in.
|
||||
if telemetry_enabled is not None:
|
||||
base_env["HEADROOM_TELEMETRY"] = "on" if telemetry_enabled else "off"
|
||||
if memory_enabled:
|
||||
base_env["HEADROOM_MEMORY_ENABLED"] = "1"
|
||||
# Grok / Grok Build need proxy upstream = xAI. Only auto-set when no other
|
||||
@@ -262,7 +265,8 @@ def build_manifest(
|
||||
"--backend",
|
||||
backend,
|
||||
]
|
||||
proxy_args.append("--telemetry" if telemetry_enabled else "--no-telemetry")
|
||||
if telemetry_enabled is not None:
|
||||
proxy_args.append("--telemetry" if telemetry_enabled else "--no-telemetry")
|
||||
# Agentic CLI targets (Claude Code, Codex) burst well above 60 req/min.
|
||||
# Persist the flag so reinstalls don't silently reintroduce throttling.
|
||||
# (see: https://github.com/headroomlabs-ai/headroom/issues/1350)
|
||||
|
||||
@@ -202,7 +202,7 @@ from headroom.subscription.tracker import (
|
||||
get_subscription_tracker,
|
||||
)
|
||||
from headroom.telemetry import get_telemetry_collector
|
||||
from headroom.telemetry.beacon import is_beacon_enabled, is_telemetry_enabled
|
||||
from headroom.telemetry.beacon import beacon_off_reason, is_beacon_enabled, is_telemetry_enabled
|
||||
from headroom.telemetry.toin import get_toin
|
||||
from headroom.transforms import (
|
||||
CacheAligner,
|
||||
@@ -2376,10 +2376,11 @@ class HeadroomProxy(
|
||||
if is_beacon_enabled():
|
||||
logger.info(
|
||||
"Anonymous upload beacon: ENABLED (default — session summaries are sent to "
|
||||
"Headroom Labs). Opt out: HEADROOM_BEACON=off or DO_NOT_TRACK=1"
|
||||
"Headroom Labs). Opt out: HEADROOM_BEACON=off, HEADROOM_TELEMETRY=off or "
|
||||
"DO_NOT_TRACK=1"
|
||||
)
|
||||
else:
|
||||
logger.info("Anonymous upload beacon: DISABLED")
|
||||
logger.info("Anonymous upload beacon: DISABLED (%s)", beacon_off_reason())
|
||||
|
||||
self.pipeline_extensions.emit(
|
||||
PipelineStage.POST_START,
|
||||
|
||||
@@ -8,7 +8,8 @@ Never leaves the machine.
|
||||
|
||||
``HEADROOM_BEACON`` — **on by default, opt-out.** Uploads an anonymous,
|
||||
content-free session summary to Headroom Labs. Disable with
|
||||
``HEADROOM_BEACON=off``, ``DO_NOT_TRACK=1``, or offline mode. See
|
||||
``HEADROOM_BEACON=off``, ``HEADROOM_TELEMETRY=off``, ``DO_NOT_TRACK=1``, or
|
||||
offline mode. See
|
||||
:mod:`headroom.telemetry.session` for the exact payload and
|
||||
``deploy/beacon/worker.js`` for what the receiver does with it.
|
||||
|
||||
@@ -17,7 +18,9 @@ collector via ``HEADROOM_OTEL_METRICS_*`` — a destination you control, that
|
||||
Headroom Labs never sees.
|
||||
|
||||
Keeping these separate matters: an operator who turned on local stats has not
|
||||
thereby agreed to upload anything, and must not start doing so on upgrade.
|
||||
thereby agreed to upload anything, and must not start doing so on upgrade. The
|
||||
one crossing is deliberate and only ever turns the upload off: an explicit
|
||||
``HEADROOM_TELEMETRY=off`` reads as "send nothing", so the beacon honours it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -56,40 +59,53 @@ def is_telemetry_enabled() -> bool:
|
||||
BEACON_DEFAULT_ON = True
|
||||
|
||||
|
||||
def is_beacon_enabled() -> bool:
|
||||
"""Check if the anonymous upload beacon is enabled.
|
||||
def beacon_off_reason() -> str | None:
|
||||
"""Name the setting that turns the anonymous upload beacon off, or ``None`` when on.
|
||||
|
||||
Default is :data:`BEACON_DEFAULT_ON`. Three things switch it off regardless,
|
||||
in this order:
|
||||
Default is :data:`BEACON_DEFAULT_ON`. Checked in this order, first match wins:
|
||||
|
||||
* offline mode — no network is no network;
|
||||
* ``DO_NOT_TRACK`` — the cross-tool convention, honoured because a user who
|
||||
has already stated this preference should not have to state it again;
|
||||
* ``HEADROOM_BEACON=off`` (or false/0/no/disable/disabled).
|
||||
* ``HEADROOM_BEACON`` — the beacon's own switch; an explicit on-value keeps
|
||||
the beacon on even when local stats are off;
|
||||
* ``HEADROOM_TELEMETRY=off`` (or false/0/no/disable/disabled) — people set
|
||||
the telemetry switch to mean "send nothing", so an explicit off stops the
|
||||
upload too.
|
||||
|
||||
Deliberately a *separate* switch from ``HEADROOM_TELEMETRY``. That flag
|
||||
means "aggregate stats locally so ``/stats`` has something to show" — data
|
||||
that never leaves the machine. Overloading it to also authorise upload
|
||||
would mean every operator who had turned on local stats begins transmitting
|
||||
the moment they upgrade, having answered a different question.
|
||||
Only an explicit *off* on ``HEADROOM_TELEMETRY`` reaches the beacon. Its
|
||||
*on* still means only "aggregate stats locally": turning on local stats must
|
||||
never be read as agreeing to upload.
|
||||
|
||||
The result is what ``headroom telemetry`` prints, so it is the variable and
|
||||
value as the operator wrote them (lowercased), e.g. ``HEADROOM_TELEMETRY=off``.
|
||||
"""
|
||||
from headroom.offline import OFFLINE_ENV, is_offline
|
||||
|
||||
if is_offline():
|
||||
return f"{OFFLINE_ENV} (offline mode)"
|
||||
dnt = os.environ.get("DO_NOT_TRACK", "").lower().strip()
|
||||
if dnt and dnt not in _OFF_VALUES:
|
||||
return f"DO_NOT_TRACK={dnt}"
|
||||
raw = os.environ.get("HEADROOM_BEACON", "").lower().strip()
|
||||
if raw in _OFF_VALUES:
|
||||
return f"HEADROOM_BEACON={raw}"
|
||||
if raw in _ON_VALUES:
|
||||
return None
|
||||
local = os.environ.get("HEADROOM_TELEMETRY", "").lower().strip()
|
||||
if local in _OFF_VALUES:
|
||||
return f"HEADROOM_TELEMETRY={local}"
|
||||
return None if BEACON_DEFAULT_ON else "default (opt-in)"
|
||||
|
||||
|
||||
def is_beacon_enabled() -> bool:
|
||||
"""Check if the anonymous upload beacon is enabled; see :func:`beacon_off_reason`.
|
||||
|
||||
Note the asymmetry with :func:`is_telemetry_enabled`, which is fail-closed:
|
||||
this one is fail-open by construction while ``BEACON_DEFAULT_ON`` is True,
|
||||
so an unrecognised value uploads rather than staying silent.
|
||||
"""
|
||||
from headroom.offline import is_offline
|
||||
|
||||
if is_offline():
|
||||
return False
|
||||
dnt = os.environ.get("DO_NOT_TRACK", "").lower().strip()
|
||||
if dnt and dnt not in _OFF_VALUES:
|
||||
return False
|
||||
raw = os.environ.get("HEADROOM_BEACON", "").lower().strip()
|
||||
if raw in _OFF_VALUES:
|
||||
return False
|
||||
if raw in _ON_VALUES:
|
||||
return True
|
||||
return BEACON_DEFAULT_ON
|
||||
return beacon_off_reason() is None
|
||||
|
||||
|
||||
def is_telemetry_warn_enabled() -> bool:
|
||||
|
||||
@@ -1768,7 +1768,13 @@ def post_session_event(payload: dict[str, Any]) -> None:
|
||||
ponytail: unbounded thread spawn is safe only because the close rate is
|
||||
bounded by the idle timeout. Revisit if anything else starts emitting here.
|
||||
"""
|
||||
from headroom.telemetry.beacon import is_beacon_enabled
|
||||
|
||||
try:
|
||||
# Re-checked at send time: a session buffered before an opt-out must
|
||||
# not upload after it (heartbeats and idle closes send through here).
|
||||
if not is_beacon_enabled():
|
||||
return
|
||||
threading.Thread(
|
||||
target=_post_blocking,
|
||||
args=(payload,),
|
||||
@@ -1812,9 +1818,17 @@ def _flush_at_exit() -> None:
|
||||
Still does not cover SIGKILL or a closed laptop. Heartbeats bound the loss
|
||||
there to one flush interval.
|
||||
"""
|
||||
from headroom.telemetry.beacon import is_beacon_enabled
|
||||
|
||||
aggregator = _aggregator
|
||||
if aggregator is None:
|
||||
return
|
||||
try:
|
||||
if not is_beacon_enabled(): # opted out since the session was buffered
|
||||
return
|
||||
except Exception:
|
||||
logger.debug("telemetry: beacon gate failed at exit", exc_info=True)
|
||||
return
|
||||
aggregator.flush_all(emit=lambda payload: _post_blocking(payload, timeout=_EXIT_POST_TIMEOUT_S))
|
||||
|
||||
|
||||
|
||||
@@ -56,6 +56,44 @@ def test_build_manifest_for_persistent_docker_sets_expected_defaults() -> None:
|
||||
assert "HEADROOM_KOMPRESS_BACKEND" not in manifest.base_env
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("telemetry", "no_telemetry", "env", "flag"),
|
||||
[
|
||||
(False, False, None, None),
|
||||
(True, False, "on", "--telemetry"),
|
||||
(False, True, "off", "--no-telemetry"),
|
||||
(True, True, "off", "--no-telemetry"),
|
||||
],
|
||||
)
|
||||
def test_install_writes_telemetry_only_when_chosen(telemetry, no_telemetry, env, flag) -> None:
|
||||
"""``HEADROOM_TELEMETRY=off`` now stops the upload beacon, so an install with
|
||||
no telemetry flag must not write it (or ``--no-telemetry``) by default."""
|
||||
from headroom.cli.install import _build_deployment_manifest
|
||||
|
||||
manifest = _build_deployment_manifest(
|
||||
preset=InstallPreset.PERSISTENT_DOCKER.value,
|
||||
runtime="docker",
|
||||
scope="user",
|
||||
provider_mode="manual",
|
||||
targets=(),
|
||||
profile="default",
|
||||
port=8787,
|
||||
backend="anthropic",
|
||||
anyllm_provider=None,
|
||||
region=None,
|
||||
proxy_mode="token",
|
||||
memory=False,
|
||||
telemetry=telemetry,
|
||||
no_telemetry=no_telemetry,
|
||||
image="ghcr.io/headroomlabs-ai/headroom:latest",
|
||||
no_http2=False,
|
||||
)
|
||||
|
||||
assert manifest.base_env.get("HEADROOM_TELEMETRY") == env
|
||||
flags = {"--telemetry", "--no-telemetry"} & set(manifest.proxy_args)
|
||||
assert flags == ({flag} if flag else set())
|
||||
|
||||
|
||||
def test_build_manifest_docker_runtime_never_installs_native_supervisor() -> None:
|
||||
manifest = build_manifest(
|
||||
profile="default",
|
||||
|
||||
@@ -8,6 +8,7 @@ Covers:
|
||||
- /stats endpoint exposes anon_telemetry_shipping flag
|
||||
"""
|
||||
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
@@ -17,6 +18,7 @@ from click.testing import CliRunner # noqa: E402
|
||||
|
||||
from headroom.telemetry.beacon import ( # noqa: E402
|
||||
format_telemetry_notice,
|
||||
is_beacon_enabled,
|
||||
is_telemetry_warn_enabled,
|
||||
)
|
||||
|
||||
@@ -43,6 +45,53 @@ class TestIsTelemetryWarnEnabled:
|
||||
assert is_telemetry_warn_enabled() is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["off", "OFF", "false", "0", "no", "disable", "disabled"])
|
||||
def test_telemetry_off_turns_the_beacon_off_and_says_so(monkeypatch, value):
|
||||
"""``HEADROOM_TELEMETRY=off`` means "send nothing": it used to leave the
|
||||
upload beacon on, and only ``HEADROOM_BEACON=off`` stopped it."""
|
||||
from headroom.cli.main import main
|
||||
|
||||
for var in ("HEADROOM_BEACON", "DO_NOT_TRACK", "HEADROOM_OFFLINE"):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
monkeypatch.setenv("HEADROOM_TELEMETRY", value)
|
||||
|
||||
assert is_beacon_enabled() is False
|
||||
output = CliRunner().invoke(main, ["telemetry"]).output
|
||||
assert f"Upload beacon: OFF (HEADROOM_TELEMETRY={value.lower()})" in output
|
||||
|
||||
|
||||
def test_buffered_session_is_not_sent_after_telemetry_off(monkeypatch):
|
||||
"""A session recorded while the beacon was on must not upload once the
|
||||
operator opts out, at heartbeat/idle close or at exit."""
|
||||
from headroom.telemetry import session as S
|
||||
|
||||
for var in ("HEADROOM_BEACON", "DO_NOT_TRACK", "HEADROOM_OFFLINE"):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
sent: list[dict] = []
|
||||
monkeypatch.setattr(S, "_post_blocking", lambda payload, **kwargs: sent.append(payload))
|
||||
aggregator = S.SessionAggregator(S.post_session_event, idle_s=900.0, flush_s=100.0)
|
||||
monkeypatch.setattr(S, "_aggregator", aggregator)
|
||||
outcome = SimpleNamespace(provider="openai", model="gpt-5.4", tokens_saved=10, tags={})
|
||||
aggregator.record(outcome, now=1_000.0)
|
||||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||||
|
||||
aggregator.record(outcome, now=1_150.0) # past flush_s: heartbeat
|
||||
aggregator.record(outcome, now=3_000.0) # past idle_s: idle close
|
||||
S._flush_at_exit()
|
||||
|
||||
assert sent == []
|
||||
|
||||
|
||||
def test_beacon_on_overrides_telemetry_off(monkeypatch):
|
||||
"""HEADROOM_BEACON is the beacon's own switch and still wins."""
|
||||
for var in ("DO_NOT_TRACK", "HEADROOM_OFFLINE"):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||||
monkeypatch.setenv("HEADROOM_BEACON", "on")
|
||||
|
||||
assert is_beacon_enabled() is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# format_telemetry_notice
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -228,28 +277,19 @@ class TestProxyCLITelemetryBanner:
|
||||
assert "HEADROOM_TELEMETRY=off" in result.output or "--no-telemetry" in result.output
|
||||
|
||||
def test_banner_beacon_default_on_is_surfaced(self, runner, monkeypatch):
|
||||
"""The actual bug: HEADROOM_TELEMETRY=off / --no-telemetry alone used
|
||||
to make the banner print a bare "Telemetry: DISABLED" even though the
|
||||
anonymous upload beacon (a separate, on-by-default switch) was still
|
||||
active and never mentioned. After the fix, an operator who sets only
|
||||
--no-telemetry still sees the beacon disclosed."""
|
||||
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
|
||||
monkeypatch.delenv("HEADROOM_BEACON", raising=False)
|
||||
monkeypatch.delenv("DO_NOT_TRACK", raising=False)
|
||||
"""With no switch set the banner discloses the opt-out beacon rather than
|
||||
printing a bare "Telemetry: DISABLED" for the local-stats switch alone."""
|
||||
for var in ("HEADROOM_TELEMETRY", "HEADROOM_BEACON", "DO_NOT_TRACK"):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
|
||||
from headroom.cli.main import main
|
||||
|
||||
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
|
||||
result = runner.invoke(main, ["proxy", "--no-telemetry"])
|
||||
result = runner.invoke(main, ["proxy"])
|
||||
|
||||
assert "Telemetry:" in result.output
|
||||
assert "usage counters" in result.output
|
||||
assert "HEADROOM_BEACON=off" in result.output
|
||||
# The old banner's bare "DISABLED" claim must not appear on the
|
||||
# Telemetry line specifically (the banner has an unrelated
|
||||
# "Memory: DISABLED" line that must not make this assertion a
|
||||
# false pass) -- that combination is exactly the false assurance
|
||||
# this fix removes.
|
||||
telemetry_line = next(
|
||||
line for line in result.output.splitlines() if line.strip().startswith("Telemetry:")
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user