fix(telemetry): let HEADROOM_TELEMETRY=off stop the anonymous beacon (#4235)

## Description

Make an explicit HEADROOM_TELEMETRY=off or --no-telemetry choice stop
the anonymous upload beacon as well as local telemetry, while preserving
the separately configured HEADROOM_BEACON=on override and existing
offline/DO_NOT_TRACK policy.

Existing persistent manifests already contain explicit off values
written by the old installer defaults. Those installs stay opted out
after upgrading unless deliberately reconfigured or reinstalled. The PR
does not guess which saved off values were intentional. New installs
omit an unspecified telemetry choice so they retain the existing default
beacon policy.

## Type of Change
- [x] Bug fix (telemetry opt-out behavior)
- [x] Documentation update

## Changes Made
- Apply an explicit local-telemetry opt-out to the beacon enable
decision and report the reason in telemetry status.
- Recheck enablement before sending buffered session telemetry.
- Preserve unspecified, enabled, and disabled telemetry choices when
generating installation manifests.
- Update operator documentation and regression coverage.

## Testing
- [x] Maintainer telemetry, beacon, and snapshot suites passed after the
current-main sync.
- [x] Whitespace check passed.

### Test Output
```text
python -m pytest tests/test_telemetry_warning.py tests/test_proxy_telemetry_env.py tests/test_beacon_schema_v2.py tests/test_beacon_gzip.py tests/test_original_snapshot_in_place_hook.py -q
122 passed, 1 warning in 21.19s
```

A previous broader installer run also reproduced the persistent-service
embedder assertion on current main independently; that baseline failure
was not attributed to this PR. The other 34 planner tests passed in that
qualification. Hosted CI is qualifying head
d2f8582f97.

## Real Behavior Proof
- Environment: isolated Windows/Python 3.12.13 maintainer worktree with
actual telemetry/beacon decision logic and proxy fixtures.
- Exact command / steps: run the five pytest suites shown above,
including explicit opt-out, override, and buffered-session regressions.
- Observed result: all 122 selected tests passed after incorporating
current main. Earlier installer-choice regression coverage and the
independent main comparison are recorded in the qualification evidence.
- Not tested: independent maintainer live upload traffic to the
production collector or migration of real persistent installs. The
author's live status/proxy before/after observations are preserved below
and remain attributed to the author.

## Runtime Rollout Safety
- Rollout-managed feature(s): existing telemetry and beacon controls; no
new rollout framework toggle.
- Minimum rollout channel: existing deployment policy applies.
- Stable/default behavior changed: an explicit telemetry opt-out also
disables beacon uploads unless a permissible explicit beacon-on override
is configured. Saved legacy off values remain off; new unspecified
install choices are no longer persisted as off.
- Kill switch / disable path: HEADROOM_BEACON=off, DO_NOT_TRACK=1, or
offline mode under the existing policy.
- Unsafe override required: no.
- Qualification impact: operator intent and existing manifest behavior
must be considered during upgrade; hosted CI and human review remain
required.
- Rollback path: retain the prior qualified release; configure
HEADROOM_BEACON explicitly where appropriate rather than rewriting saved
user intent.

## Review Readiness
- [ ] I have performed a self-review
- [ ] This PR is ready for human review

Maintainer verification is documented above; author readiness
attestations have not been filled on the author's behalf.

## Additional Notes

The original author warning, description, follow-up exclusions, and
supplied evidence are preserved below.
> **Behaviour change for existing installs.** Read this before merging.
Every manifest that `headroom install` saved before this change already
contains `HEADROOM_TELEMETRY=off` and `--no-telemetry`, because the
planner wrote them by default. Those manifests are replayed as-is, so
**existing persistent installs will report the beacon OFF after
upgrading** until they are reinstalled. A loader migration can't tell
those defaults apart from a real opt-out, and turning uploads back on
for people who opted out would be worse, so this PR doesn't migrate. New
installs keep the beacon on by default.

#### The test we ran

A Daytona sandbox running Headroom Enterprise 0.4.3 (core 0.39.1), with
Codex CLI 0.162.1 and Claude Code connected to real GitHub, Context7 and
Fetch MCP servers, working through the workshop tasks.

#### What we found

`headroom telemetry --show` and the README say telemetry can be turned
off, and users reasonably set **`HEADROOM_TELEMETRY=off`** (or pass
`--no-telemetry`). But the anonymous upload beacon was gated only on
`HEADROOM_BEACON` / `DO_NOT_TRACK` / offline mode. With
`HEADROOM_TELEMETRY=off` it kept uploading, and `headroom telemetry`
still printed `Upload beacon: ON (opt-out)`.

#### The fix

- New `beacon_off_reason()` in `headroom/telemetry/beacon.py` names the
setting that turns the beacon off. `is_beacon_enabled()` delegates to
it. The order is:
  1. offline mode;
  2. `DO_NOT_TRACK`;
3. `HEADROOM_BEACON`, whose explicit `on` still wins over local stats
being off;
4. an explicit **`HEADROOM_TELEMETRY` off-value**: off / false / 0 / no
/ disable / disabled, the same `_OFF_VALUES` the module already uses.

`HEADROOM_TELEMETRY=on` still means only "local stats". It never
authorises upload.
- A session buffered before an opt-out is not sent afterwards:
`post_session_event` (heartbeat, idle close) and the exit flush re-check
`is_beacon_enabled()` at send time.
- `headroom telemetry` prints `Upload beacon: OFF
(HEADROOM_TELEMETRY=off)` (or whichever switch is in effect), and the
proxy startup log does the same.
- `headroom install`: since `off` now stops the beacon, the planner
writes `HEADROOM_TELEMETRY` / `--telemetry` / `--no-telemetry` only for
an explicit choice (`telemetry_enabled` is now tri-state, `None` = no
choice). Local stats stay off without the env var because they are
opt-in. `scripts/install.sh` and `install.ps1` already passed
`--no-telemetry` only on an explicit flag.
- Docs: README, proxy, security-model and limitations pages, the
dashboard tooltip, and the `--no-telemetry` help text.
- `test_banner_beacon_default_on_is_surfaced` asserted the old policy
(`--no-telemetry` leaves the beacon on). It now checks the default case
instead.

#### Before / after

| env | `is_beacon_enabled()` on origin/main | this PR | `headroom
telemetry` (this PR) |
|---|---|---|---|
| none | True | True | `ON (opt-out)` |
| `HEADROOM_TELEMETRY=off` | **True** | **False** | `OFF
(HEADROOM_TELEMETRY=off)` |
| `HEADROOM_TELEMETRY=off` + `HEADROOM_BEACON=on` | True | True | `ON
(opt-out)` |
| `HEADROOM_BEACON=off` | False | False | `OFF (HEADROOM_BEACON=off)` |
| `headroom install` with no flag → manifest | `HEADROOM_TELEMETRY=off`
+ `--no-telemetry` | neither written | — |

#### Tests

-
`tests/test_telemetry_warning.py::test_telemetry_off_turns_the_beacon_off_and_says_so`,
parametrized over the 7 off spellings: fails on origin/main, passes
here.
-
`tests/test_telemetry_warning.py::test_beacon_on_overrides_telemetry_off`.
-
`tests/test_telemetry_warning.py::test_buffered_session_is_not_sent_after_telemetry_off`:
fails on the first commit of this PR, passes now.
-
`tests/test_install/test_planner.py::test_install_writes_telemetry_only_when_chosen`,
through the CLI's `_build_deployment_manifest`: the no-flag case fails
on origin/main.
- Related suites pass: install, telemetry warning, beacon schema v2,
install CLI and proxy health (435 passed).
- ruff check, ruff format --check and mypy pass.

#### Follow-ups (not in this PR)

- `headroom init` passes `telemetry_enabled=True`, so its manifests turn
local stats on (`cli/init.py:718`). This doesn't affect the beacon.
- The `--telemetry` help text still says "anonymous telemetry", but the
flag only turns on local stats.

#### Real behavior proof

- **Setup:** macOS 26.7, Python 3.12.13. origin/main and this branch
were run side by side with `HEADROOM_BEACON` and `DO_NOT_TRACK` unset.
- `HEADROOM_TELEMETRY=off headroom telemetry`:
- origin/main: `Upload beacon: ON (opt-out)`, `is_beacon_enabled():
True`
- this branch: `Upload beacon: OFF (HEADROOM_TELEMETRY=off)`,
`is_beacon_enabled(): False`
- `headroom proxy --no-telemetry --no-optimize`, startup log:
- origin/main: `Anonymous upload beacon: ENABLED (default — session
summaries are sent to Headroom Labs). Opt out: HEADROOM_BEACON=off or
DO_NOT_TRACK=1`
- this branch: `Anonymous upload beacon: DISABLED
(HEADROOM_TELEMETRY=off)`
- **Not tested:** a real `headroom install apply` of a persistent
service on this branch; the manifest change is covered through
`_build_deployment_manifest`. Upgrading an existing install was not
tested either; see the warning at the top.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
Tejas Chopra
2026-10-16 14:32:06 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent ff36931653
commit 1a40b3ce67
15 changed files with 195 additions and 70 deletions
+2 -2
View File
@@ -624,8 +624,8 @@ prompts, completions, code or file paths. It exists so we can see when a release
regresses a compression ratio across real workloads rather than only our own test
corpus.
Turn it off with `HEADROOM_BEACON=off`, the `DO_NOT_TRACK=1` convention, or
`HEADROOM_OFFLINE=1`. The full field list is in
Turn it off with `HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off`, the
`DO_NOT_TRACK=1` convention, or `HEADROOM_OFFLINE=1`. The full field list is in
[the proxy docs](https://docs.headroomlabs.ai/docs/proxy).
## Headroom for teams
+4 -2
View File
@@ -174,10 +174,12 @@ this page's "nothing leaves the machine" framing as universal:
compression ratios, skip reasons, provider and model ids, failure counts, plus
OS and architecture. Never prompts, completions, code, or file paths -- the
collector allowlists fields server-side and drops the rest. Disable it with
`HEADROOM_BEACON=off`, the cross-tool `DO_NOT_TRACK=1` convention, or by
`HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off`, the cross-tool `DO_NOT_TRACK=1` convention, or by
running offline. `is_beacon_enabled()` is fail-open: an unrecognized value
uploads rather than staying silent. See [Proxy](/docs/proxy) for the full
field list.
These are deliberately separate so that enabling local `/stats` does not
silently start an upload on the next release.
silently start an upload on the next release. The one crossing only ever turns
the upload off: an explicit `HEADROOM_TELEMETRY=off` also stops the beacon
(unless `HEADROOM_BEACON=on`).
+2 -2
View File
@@ -24,7 +24,7 @@ headroom proxy \
--budget 100.0
```
Two independent telemetry switches exist. `HEADROOM_TELEMETRY=on` (or `--telemetry`) turns on **local-only** in-process usage stats that power your own `/stats`, `/metrics`, and dashboard — off by default, and nothing under this switch leaves the machine.
Two telemetry switches exist. `HEADROOM_TELEMETRY=on` (or `--telemetry`) turns on **local-only** in-process usage stats that power your own `/stats`, `/metrics`, and dashboard — off by default, and nothing under this switch leaves the machine.
Separately, an **anonymous usage beacon is on by default** (opt-out). Once a session goes idle it POSTs a small summary of how compression behaved to a Headroom Labs collector.
@@ -36,7 +36,7 @@ Separately, an **anonymous usage beacon is on by default** (opt-out). Once a ses
The install id is a random UUID generated on first run and stored in your config directory. It is deliberately not derived from hostname, MAC address, or any hardware property — delete the file and you get a new one.
Disable it with `HEADROOM_BEACON=off`, the `DO_NOT_TRACK=1` convention, or `HEADROOM_OFFLINE=1` (which also disables update checks and license reporting).
Disable it with `HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off` (or `--no-telemetry`), the `DO_NOT_TRACK=1` convention, or `HEADROOM_OFFLINE=1` (which also disables update checks and license reporting). `HEADROOM_BEACON=on` keeps the beacon on even with local stats off. `headroom telemetry` names the setting that turned it off.
## CLI options
+10 -7
View File
@@ -343,7 +343,7 @@ Beyond your configured LLM providers, Headroom may contact:
| Destination | Purpose | Default | Disable / pre-provision |
|---|---|---|---|
| `headroom-beacon.headroom-beacon.workers.dev` | Anonymous session summaries to Headroom Labs | **On** — see [The beacon](#the-beacon-is-on-by-default) | `HEADROOM_BEACON=off`, `DO_NOT_TRACK=1`, or `HEADROOM_OFFLINE` |
| `headroom-beacon.headroom-beacon.workers.dev` | Anonymous session summaries to Headroom Labs | **On** — see [The beacon](#the-beacon-is-on-by-default) | `HEADROOM_BEACON=off`, `HEADROOM_TELEMETRY=off`, `DO_NOT_TRACK=1`, or `HEADROOM_OFFLINE` |
| `pypi.org` | Daily version check for update notices | On | `HEADROOM_UPDATE_CHECK=off`; also skipped in `--stateless`, in CI, and from source checkouts |
| `huggingface.co` | Downloads models for the optional ML compression paths | On first use of those features | Pre-provision the model cache, or leave the features disabled |
| `app.headroomlabs.ai` | License validation and usage reporting | Only with a license key configured | Omit the license key |
@@ -379,14 +379,17 @@ Prompts, completions, file contents, tool results, and file paths are **not** in
payload. The install id is a random UUID rather than a machine fingerprint, and resets
if you delete `<config dir>/install_id`.
Three controls switch it off, checked in this order:
Four controls switch it off, checked in this order:
```bash
HEADROOM_OFFLINE=1 # offline mode — disables all egress, not just the beacon
DO_NOT_TRACK=1 # the cross-tool convention, honoured
HEADROOM_BEACON=off # also false / 0 / no / disable / disabled
HEADROOM_OFFLINE=1 # offline mode — disables all egress, not just the beacon
DO_NOT_TRACK=1 # the cross-tool convention, honoured
HEADROOM_BEACON=off # also false / 0 / no / disable / disabled; =on keeps it on
HEADROOM_TELEMETRY=off # same off-values; also what --no-telemetry sets
```
`headroom telemetry` names the setting that turned it off.
The destination can also be redirected with `HEADROOM_TELEMETRY_ENDPOINT`, which is
useful if you would rather terminate the reports at a receiver you control than block
them outright.
@@ -401,8 +404,8 @@ assuming a typo fails safe.
`HEADROOM_TELEMETRY` is a **different switch**, off by default and opt-in. It aggregates
stats locally for the in-process collector and the `/stats` and `/v1/telemetry`
endpoints, and sends nothing off-host. Enabling it does not authorise beacon upload, and
disabling it does not suppress the beacon — the two are deliberately independent.
endpoints, and sends nothing off-host. Enabling it does not authorise beacon upload; an
explicit `HEADROOM_TELEMETRY=off` does suppress the beacon, unless `HEADROOM_BEACON=on`.
Operational metrics are separate again: OpenTelemetry export
(`HEADROOM_OTEL_METRICS_*`) is opt-in and targets a collector you specify.
+4 -3
View File
@@ -604,7 +604,8 @@ def _build_deployment_manifest(
region=region,
proxy_mode=proxy_mode,
memory_enabled=memory,
telemetry_enabled=telemetry and not no_telemetry,
# --no-telemetry wins when both are given; neither leaves it unset.
telemetry_enabled=False if no_telemetry else (True if telemetry else None),
no_rate_limit=no_rate_limit,
image=image,
no_http2=no_http2,
@@ -875,7 +876,7 @@ def _echo_installed(manifest: DeploymentManifest, *, prefix: str = "Installed pe
@click.option(
"--no-telemetry",
is_flag=True,
help="Force anonymous telemetry off in the runtime (already the default).",
help="Turn off local stats and the anonymous upload beacon in the runtime.",
)
@click.option(
"--no-rate-limit",
@@ -1092,7 +1093,7 @@ def install_apply(
@click.option(
"--no-telemetry",
is_flag=True,
help="Force anonymous telemetry off in the runtime (already the default).",
help="Turn off local stats and the anonymous upload beacon in the runtime.",
)
@click.option(
"--image",
+1 -1
View File
@@ -1112,7 +1112,7 @@ def dashboard(port: int, no_open: bool) -> None:
@click.option(
"--no-telemetry",
is_flag=True,
help="Force anonymous usage telemetry off (already the default; env: HEADROOM_TELEMETRY=off)",
help="Turn off local stats and the anonymous upload beacon (env: HEADROOM_TELEMETRY=off)",
)
@click.option(
"--stateless",
+8 -4
View File
@@ -85,7 +85,7 @@ def telemetry(show: bool, as_json: bool) -> None:
headroom telemetry Current state and the two switches
headroom telemetry --show The exact payload, field by field
"""
from headroom.telemetry.beacon import is_beacon_enabled, is_telemetry_enabled
from headroom.telemetry.beacon import beacon_off_reason, is_telemetry_enabled
from headroom.telemetry.session import (
DEFAULT_ENDPOINT,
SCHEMA_VERSION,
@@ -94,7 +94,8 @@ def telemetry(show: bool, as_json: bool) -> None:
resource_attributes,
)
beacon_on = is_beacon_enabled()
beacon_off_by = beacon_off_reason()
beacon_on = beacon_off_by is None
local_on = is_telemetry_enabled()
known_id = read_install_id()
# Reported because it changes what goes on the wire, and because while the
@@ -134,7 +135,7 @@ def telemetry(show: bool, as_json: bool) -> None:
)
return
click.echo(f"Upload beacon: {'ON (opt-out)' if beacon_on else 'OFF'}")
click.echo(f"Upload beacon: {'ON (opt-out)' if beacon_on else f'OFF ({beacon_off_by})'}")
click.echo(f"Local stats: {'ON' if local_on else 'OFF (opt-in)'}")
click.echo(f"Endpoint: {DEFAULT_ENDPOINT}")
click.echo(
@@ -143,5 +144,8 @@ def telemetry(show: bool, as_json: bool) -> None:
)
click.echo(f"Install ID: {known_id or '(none yet — created on first upload)'}")
click.echo()
click.echo("Turn the beacon off: HEADROOM_BEACON=off (or DO_NOT_TRACK=1)")
if beacon_on:
click.echo(
"Turn the beacon off: HEADROOM_BEACON=off (or HEADROOM_TELEMETRY=off, DO_NOT_TRACK=1)"
)
click.echo("See the exact payload: headroom telemetry --show")
+1 -1
View File
@@ -152,7 +152,7 @@
</div>
<template x-if="stats.anon_telemetry_shipping">
<div class="inline-flex items-center gap-1.5 rounded-full border border-amber-500/40 bg-amber-500/10 px-2.5 py-1"
title="The anonymous telemetry beacon is on: compression stats only, never prompts, code, or file paths. Turn it off with HEADROOM_BEACON=off or DO_NOT_TRACK=1.">
title="The anonymous telemetry beacon is on: compression stats only, never prompts, code, or file paths. Turn it off with HEADROOM_BEACON=off, HEADROOM_TELEMETRY=off or DO_NOT_TRACK=1.">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"
class="w-3 h-3 text-amber-400 shrink-0">
<path fill-rule="evenodd"
+3 -1
View File
@@ -106,7 +106,9 @@ class DeploymentManifest:
proxy_mode: str = "cache"
memory_enabled: bool = False
memory_db_path: str = ""
telemetry_enabled: bool = True
# None: the operator chose neither --telemetry nor --no-telemetry. The
# default stays True for manifests saved before the field existed.
telemetry_enabled: bool | None = True
image: str = "ghcr.io/headroomlabs-ai/headroom:latest"
service_name: str = "headroom"
container_name: str = "headroom-persistent"
+9 -5
View File
@@ -150,7 +150,7 @@ def build_manifest(
region: str | None,
proxy_mode: str,
memory_enabled: bool,
telemetry_enabled: bool,
telemetry_enabled: bool | None,
image: str,
no_rate_limit: bool = False,
no_http2: bool = False,
@@ -201,9 +201,12 @@ def build_manifest(
base_env["HEADROOM_ANYLLM_PROVIDER"] = anyllm_provider
if region:
base_env["HEADROOM_REGION"] = region
# Telemetry is opt-in (off by default). Write the value explicitly so the
# generated manifest is unambiguous and doesn't depend on the runtime default.
base_env["HEADROOM_TELEMETRY"] = "on" if telemetry_enabled else "off"
# Write the operator's explicit choice only. ``off`` also stops the upload
# beacon (telemetry/beacon.py), so writing it by default would turn the
# beacon off for every install nobody asked to opt out. Local stats stay
# off without it: they are opt-in.
if telemetry_enabled is not None:
base_env["HEADROOM_TELEMETRY"] = "on" if telemetry_enabled else "off"
if memory_enabled:
base_env["HEADROOM_MEMORY_ENABLED"] = "1"
# Grok / Grok Build need proxy upstream = xAI. Only auto-set when no other
@@ -262,7 +265,8 @@ def build_manifest(
"--backend",
backend,
]
proxy_args.append("--telemetry" if telemetry_enabled else "--no-telemetry")
if telemetry_enabled is not None:
proxy_args.append("--telemetry" if telemetry_enabled else "--no-telemetry")
# Agentic CLI targets (Claude Code, Codex) burst well above 60 req/min.
# Persist the flag so reinstalls don't silently reintroduce throttling.
# (see: https://github.com/headroomlabs-ai/headroom/issues/1350)
+4 -3
View File
@@ -202,7 +202,7 @@ from headroom.subscription.tracker import (
get_subscription_tracker,
)
from headroom.telemetry import get_telemetry_collector
from headroom.telemetry.beacon import is_beacon_enabled, is_telemetry_enabled
from headroom.telemetry.beacon import beacon_off_reason, is_beacon_enabled, is_telemetry_enabled
from headroom.telemetry.toin import get_toin
from headroom.transforms import (
CacheAligner,
@@ -2376,10 +2376,11 @@ class HeadroomProxy(
if is_beacon_enabled():
logger.info(
"Anonymous upload beacon: ENABLED (default — session summaries are sent to "
"Headroom Labs). Opt out: HEADROOM_BEACON=off or DO_NOT_TRACK=1"
"Headroom Labs). Opt out: HEADROOM_BEACON=off, HEADROOM_TELEMETRY=off or "
"DO_NOT_TRACK=1"
)
else:
logger.info("Anonymous upload beacon: DISABLED")
logger.info("Anonymous upload beacon: DISABLED (%s)", beacon_off_reason())
self.pipeline_extensions.emit(
PipelineStage.POST_START,
+41 -25
View File
@@ -8,7 +8,8 @@ Never leaves the machine.
``HEADROOM_BEACON`` — **on by default, opt-out.** Uploads an anonymous,
content-free session summary to Headroom Labs. Disable with
``HEADROOM_BEACON=off``, ``DO_NOT_TRACK=1``, or offline mode. See
``HEADROOM_BEACON=off``, ``HEADROOM_TELEMETRY=off``, ``DO_NOT_TRACK=1``, or
offline mode. See
:mod:`headroom.telemetry.session` for the exact payload and
``deploy/beacon/worker.js`` for what the receiver does with it.
@@ -17,7 +18,9 @@ collector via ``HEADROOM_OTEL_METRICS_*`` — a destination you control, that
Headroom Labs never sees.
Keeping these separate matters: an operator who turned on local stats has not
thereby agreed to upload anything, and must not start doing so on upgrade.
thereby agreed to upload anything, and must not start doing so on upgrade. The
one crossing is deliberate and only ever turns the upload off: an explicit
``HEADROOM_TELEMETRY=off`` reads as "send nothing", so the beacon honours it.
"""
from __future__ import annotations
@@ -56,40 +59,53 @@ def is_telemetry_enabled() -> bool:
BEACON_DEFAULT_ON = True
def is_beacon_enabled() -> bool:
"""Check if the anonymous upload beacon is enabled.
def beacon_off_reason() -> str | None:
"""Name the setting that turns the anonymous upload beacon off, or ``None`` when on.
Default is :data:`BEACON_DEFAULT_ON`. Three things switch it off regardless,
in this order:
Default is :data:`BEACON_DEFAULT_ON`. Checked in this order, first match wins:
* offline mode — no network is no network;
* ``DO_NOT_TRACK`` — the cross-tool convention, honoured because a user who
has already stated this preference should not have to state it again;
* ``HEADROOM_BEACON=off`` (or false/0/no/disable/disabled).
* ``HEADROOM_BEACON`` — the beacon's own switch; an explicit on-value keeps
the beacon on even when local stats are off;
* ``HEADROOM_TELEMETRY=off`` (or false/0/no/disable/disabled) — people set
the telemetry switch to mean "send nothing", so an explicit off stops the
upload too.
Deliberately a *separate* switch from ``HEADROOM_TELEMETRY``. That flag
means "aggregate stats locally so ``/stats`` has something to show" — data
that never leaves the machine. Overloading it to also authorise upload
would mean every operator who had turned on local stats begins transmitting
the moment they upgrade, having answered a different question.
Only an explicit *off* on ``HEADROOM_TELEMETRY`` reaches the beacon. Its
*on* still means only "aggregate stats locally": turning on local stats must
never be read as agreeing to upload.
The result is what ``headroom telemetry`` prints, so it is the variable and
value as the operator wrote them (lowercased), e.g. ``HEADROOM_TELEMETRY=off``.
"""
from headroom.offline import OFFLINE_ENV, is_offline
if is_offline():
return f"{OFFLINE_ENV} (offline mode)"
dnt = os.environ.get("DO_NOT_TRACK", "").lower().strip()
if dnt and dnt not in _OFF_VALUES:
return f"DO_NOT_TRACK={dnt}"
raw = os.environ.get("HEADROOM_BEACON", "").lower().strip()
if raw in _OFF_VALUES:
return f"HEADROOM_BEACON={raw}"
if raw in _ON_VALUES:
return None
local = os.environ.get("HEADROOM_TELEMETRY", "").lower().strip()
if local in _OFF_VALUES:
return f"HEADROOM_TELEMETRY={local}"
return None if BEACON_DEFAULT_ON else "default (opt-in)"
def is_beacon_enabled() -> bool:
"""Check if the anonymous upload beacon is enabled; see :func:`beacon_off_reason`.
Note the asymmetry with :func:`is_telemetry_enabled`, which is fail-closed:
this one is fail-open by construction while ``BEACON_DEFAULT_ON`` is True,
so an unrecognised value uploads rather than staying silent.
"""
from headroom.offline import is_offline
if is_offline():
return False
dnt = os.environ.get("DO_NOT_TRACK", "").lower().strip()
if dnt and dnt not in _OFF_VALUES:
return False
raw = os.environ.get("HEADROOM_BEACON", "").lower().strip()
if raw in _OFF_VALUES:
return False
if raw in _ON_VALUES:
return True
return BEACON_DEFAULT_ON
return beacon_off_reason() is None
def is_telemetry_warn_enabled() -> bool:
+14
View File
@@ -1768,7 +1768,13 @@ def post_session_event(payload: dict[str, Any]) -> None:
ponytail: unbounded thread spawn is safe only because the close rate is
bounded by the idle timeout. Revisit if anything else starts emitting here.
"""
from headroom.telemetry.beacon import is_beacon_enabled
try:
# Re-checked at send time: a session buffered before an opt-out must
# not upload after it (heartbeats and idle closes send through here).
if not is_beacon_enabled():
return
threading.Thread(
target=_post_blocking,
args=(payload,),
@@ -1812,9 +1818,17 @@ def _flush_at_exit() -> None:
Still does not cover SIGKILL or a closed laptop. Heartbeats bound the loss
there to one flush interval.
"""
from headroom.telemetry.beacon import is_beacon_enabled
aggregator = _aggregator
if aggregator is None:
return
try:
if not is_beacon_enabled(): # opted out since the session was buffered
return
except Exception:
logger.debug("telemetry: beacon gate failed at exit", exc_info=True)
return
aggregator.flush_all(emit=lambda payload: _post_blocking(payload, timeout=_EXIT_POST_TIMEOUT_S))
+38
View File
@@ -56,6 +56,44 @@ def test_build_manifest_for_persistent_docker_sets_expected_defaults() -> None:
assert "HEADROOM_KOMPRESS_BACKEND" not in manifest.base_env
@pytest.mark.parametrize(
("telemetry", "no_telemetry", "env", "flag"),
[
(False, False, None, None),
(True, False, "on", "--telemetry"),
(False, True, "off", "--no-telemetry"),
(True, True, "off", "--no-telemetry"),
],
)
def test_install_writes_telemetry_only_when_chosen(telemetry, no_telemetry, env, flag) -> None:
"""``HEADROOM_TELEMETRY=off`` now stops the upload beacon, so an install with
no telemetry flag must not write it (or ``--no-telemetry``) by default."""
from headroom.cli.install import _build_deployment_manifest
manifest = _build_deployment_manifest(
preset=InstallPreset.PERSISTENT_DOCKER.value,
runtime="docker",
scope="user",
provider_mode="manual",
targets=(),
profile="default",
port=8787,
backend="anthropic",
anyllm_provider=None,
region=None,
proxy_mode="token",
memory=False,
telemetry=telemetry,
no_telemetry=no_telemetry,
image="ghcr.io/headroomlabs-ai/headroom:latest",
no_http2=False,
)
assert manifest.base_env.get("HEADROOM_TELEMETRY") == env
flags = {"--telemetry", "--no-telemetry"} & set(manifest.proxy_args)
assert flags == ({flag} if flag else set())
def test_build_manifest_docker_runtime_never_installs_native_supervisor() -> None:
manifest = build_manifest(
profile="default",
+54 -14
View File
@@ -8,6 +8,7 @@ Covers:
- /stats endpoint exposes anon_telemetry_shipping flag
"""
from types import SimpleNamespace
from unittest.mock import patch
import pytest
@@ -17,6 +18,7 @@ from click.testing import CliRunner # noqa: E402
from headroom.telemetry.beacon import ( # noqa: E402
format_telemetry_notice,
is_beacon_enabled,
is_telemetry_warn_enabled,
)
@@ -43,6 +45,53 @@ class TestIsTelemetryWarnEnabled:
assert is_telemetry_warn_enabled() is True
@pytest.mark.parametrize("value", ["off", "OFF", "false", "0", "no", "disable", "disabled"])
def test_telemetry_off_turns_the_beacon_off_and_says_so(monkeypatch, value):
"""``HEADROOM_TELEMETRY=off`` means "send nothing": it used to leave the
upload beacon on, and only ``HEADROOM_BEACON=off`` stopped it."""
from headroom.cli.main import main
for var in ("HEADROOM_BEACON", "DO_NOT_TRACK", "HEADROOM_OFFLINE"):
monkeypatch.delenv(var, raising=False)
monkeypatch.setenv("HEADROOM_TELEMETRY", value)
assert is_beacon_enabled() is False
output = CliRunner().invoke(main, ["telemetry"]).output
assert f"Upload beacon: OFF (HEADROOM_TELEMETRY={value.lower()})" in output
def test_buffered_session_is_not_sent_after_telemetry_off(monkeypatch):
"""A session recorded while the beacon was on must not upload once the
operator opts out, at heartbeat/idle close or at exit."""
from headroom.telemetry import session as S
for var in ("HEADROOM_BEACON", "DO_NOT_TRACK", "HEADROOM_OFFLINE"):
monkeypatch.delenv(var, raising=False)
sent: list[dict] = []
monkeypatch.setattr(S, "_post_blocking", lambda payload, **kwargs: sent.append(payload))
aggregator = S.SessionAggregator(S.post_session_event, idle_s=900.0, flush_s=100.0)
monkeypatch.setattr(S, "_aggregator", aggregator)
outcome = SimpleNamespace(provider="openai", model="gpt-5.4", tokens_saved=10, tags={})
aggregator.record(outcome, now=1_000.0)
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
aggregator.record(outcome, now=1_150.0) # past flush_s: heartbeat
aggregator.record(outcome, now=3_000.0) # past idle_s: idle close
S._flush_at_exit()
assert sent == []
def test_beacon_on_overrides_telemetry_off(monkeypatch):
"""HEADROOM_BEACON is the beacon's own switch and still wins."""
for var in ("DO_NOT_TRACK", "HEADROOM_OFFLINE"):
monkeypatch.delenv(var, raising=False)
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
monkeypatch.setenv("HEADROOM_BEACON", "on")
assert is_beacon_enabled() is True
# ---------------------------------------------------------------------------
# format_telemetry_notice
# ---------------------------------------------------------------------------
@@ -228,28 +277,19 @@ class TestProxyCLITelemetryBanner:
assert "HEADROOM_TELEMETRY=off" in result.output or "--no-telemetry" in result.output
def test_banner_beacon_default_on_is_surfaced(self, runner, monkeypatch):
"""The actual bug: HEADROOM_TELEMETRY=off / --no-telemetry alone used
to make the banner print a bare "Telemetry: DISABLED" even though the
anonymous upload beacon (a separate, on-by-default switch) was still
active and never mentioned. After the fix, an operator who sets only
--no-telemetry still sees the beacon disclosed."""
monkeypatch.setenv("HEADROOM_TELEMETRY", "off")
monkeypatch.delenv("HEADROOM_BEACON", raising=False)
monkeypatch.delenv("DO_NOT_TRACK", raising=False)
"""With no switch set the banner discloses the opt-out beacon rather than
printing a bare "Telemetry: DISABLED" for the local-stats switch alone."""
for var in ("HEADROOM_TELEMETRY", "HEADROOM_BEACON", "DO_NOT_TRACK"):
monkeypatch.delenv(var, raising=False)
from headroom.cli.main import main
with patch("headroom.proxy.server.run_server", side_effect=SystemExit(0)):
result = runner.invoke(main, ["proxy", "--no-telemetry"])
result = runner.invoke(main, ["proxy"])
assert "Telemetry:" in result.output
assert "usage counters" in result.output
assert "HEADROOM_BEACON=off" in result.output
# The old banner's bare "DISABLED" claim must not appear on the
# Telemetry line specifically (the banner has an unrelated
# "Memory: DISABLED" line that must not make this assertion a
# false pass) -- that combination is exactly the false assurance
# this fix removes.
telemetry_line = next(
line for line in result.output.splitlines() if line.strip().startswith("Telemetry:")
)