Commit Graph
9 Commits
Author SHA1 Message Date
7b6cb77905 fix(install): keep the HuggingFace cache on a writable mount in Docker presets (#4207) (#4209)
## Description

With the `persistent-docker` preset, Kompress never loads. The container
runs as the host user (`--user 501:20`) with `HOME=/tmp/headroom-home`,
but the image never creates that directory. Docker creates it as
`root:root 0755` to hold the bind mounts, so the proxy cannot create
`~/.cache` and the HuggingFace download of the Kompress model fails with
`Permission denied (os error 13)`. The log never shows that error: it
reports a missing dependency instead (`Kompress requires onnxruntime or
torch`), pointing the operator at reinstalling extras that are already
present.

Fix: point `HF_HOME` and `XDG_CACHE_HOME` at the bind-mounted,
host-owned `.headroom` workspace so the cache is writable regardless of
who Docker made the container's `$HOME`; and keep the underlying
`OSError` in both `kompress_compressor.py` log sites instead of masking
it.

Closes #4207

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/runtime.py`: `build_runtime_command()` now emits
`HF_HOME` and `XDG_CACHE_HOME` as real `--env` flags pointing under the
bind-mounted `.headroom` workspace (`<container_home>/.headroom/cache`),
and applies them to the persistent-container args path, not only the
dev/foreground path.
- `headroom/install/runtime.py`: added `_append_cache_env()` and wired
it through both command builders.
- `scripts/install.sh` and `docker/docker-compose.native.yml`: set
`HF_HOME`/`XDG_CACHE_HOME` under the writable `.headroom` mount for the
same preset.
- `headroom/transforms/kompress_compressor.py`: `_create_onnx_session()`
appends the underlying cause to its `No loadable ONNX artifact` message;
`_load_kompress()` re-raises the real ONNX failure (chained) instead of
reporting a missing dependency when onnxruntime is present but loading
failed.

## Testing

- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ .venv/bin/python -m pytest tests/test_install/ tests/test_kompress_failsafe.py -q
278 passed, 11 skipped, 1 failed

The single failure is tests/test_install/test_runtime.py::test_build_runtime_command_lets_the_host_dashboard_read_its_data
which fails with ImportError: FastAPI required in this environment. It fails identically on unmodified main here,
so it is a local environment gap, not this change.

New regression tests, all passing:
- test_build_runtime_command_keeps_hf_cache_on_the_writable_mount
- test_build_runtime_command_persistent_path_keeps_hf_cache_on_the_writable_mount
- test_no_loadable_artifact_message_carries_the_underlying_error
- test_onnx_failure_while_onnxruntime_available_reports_the_real_error
- test_missing_dependency_still_reports_install_hint

$ bash -n scripts/install.sh          # clean
$ python -c "import yaml,sys;yaml.safe_load(open('docker/docker-compose.native.yml'))"   # clean
```

## Real Behavior Proof

- Environment: Linux, Python venv at `.venv`, repo at this branch. No
Docker daemon available on this machine.
- Exact command / steps: `pytest tests/test_install/
tests/test_kompress_failsafe.py -q`; asserted the generated `docker run`
argv from both `build_runtime_command()` and
`build_persistent_container_args()` contains `--env HF_HOME` / `--env
XDG_CACHE_HOME` pointing under the `.headroom` mount.
- Observed result: all new tests pass; both argv builders now emit the
cache env pointing at the writable host-owned mount.
- Not tested: the end-to-end container run from the issue's reproduction
steps — no Docker daemon here. The fix is verified at the level of the
generated `docker run` argv and the unit tests above.

## Runtime Rollout Safety

- Rollout-managed feature(s): Kompress proxy model loading under the
`persistent-docker` preset.
- Minimum rollout channel: all channels that use the persistent-docker
preset; the change only affects env vars for cache directories.
- Stable/default behavior changed: no. Default (non-persistent) paths
are unchanged; persistent paths gain two `--env` flags pointing at an
existing writable mount.
- Kill switch / disable path: unset/override `HF_HOME`/`XDG_CACHE_HOME`,
or run without the persistent-docker preset.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this commit; the preset returns to the previous
(broken) cache behavior.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Docs untouched: no user-facing docs describe
`HF_HOME`/`XDG_CACHE_HOME` for this preset, so no documentation change
was needed (hence the unchecked box).
- No lint/type-check run: `ruff`/`mypy` are not installed in this
environment.
- No Docker daemon available, so the container-level e2e reproduction
was not executed — flagged under "Not tested" above.

---------

Co-authored-by: Shivam Maurya <shivam@programmingwithmaurya.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: JerrettDavis <jerrett@headroomlabs.ai>
2026-10-24 08:38:28 -07:00
ee731d7ea8 fix(proxy): refuse token-less non-loopback binds, gate operator routes (#3852)
## Why

Four related gaps in the proxy's network gate (findings 01-F1, 01-F2,
01-F5, 01-F9):

- **Open bind.** A non-loopback bind with no `HEADROOM_PROXY_TOKEN` only
logged a warning, then served the `/v1/*` relay, with the operator's
upstream credentials, to every peer that could reach the port.
- **Missing peer address.** `is_loopback_host(None)` returned `True`. A
Unix-domain socket, an ASGI adapter that leaves `scope["client"]` unset,
or a hand-built request therefore turned off the token gate, the
`/admin` and `/debug` guards and the WebSocket gate.
- **Operator data routes.** `/stats-history` (including CSV export),
`/quota`, `/subscription-window` and `/metrics` were served to any
network caller.
- **WebSocket loopback exemption** checked only the peer IP, not the
`Host` header, so it had no DNS-rebinding defence.

## What changes

- New `headroom/proxy/bind_policy.py`. A non-loopback bind (`0.0.0.0`,
`::`, a LAN address or a hostname) with no token is refused at three
places: `create_app` (embedders and the multi-worker factory),
`run_server` (one exit with code 2 before uvicorn forks) and the
`headroom proxy` CLI (a clear error before the banner, exit code 1). An
absent host means uvicorn's default `127.0.0.1`.
- `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` acknowledges an open bind when
the runtime already restricts reachability, such as a container
published on `127.0.0.1`. The `proxy_open_bind` warning is still logged
on that path, and the banner says the bind is open.
- `is_loopback_host(None)` now returns `False` (fail closed), matching
`identity.py`.
- `/stats-history`, `/quota` and `/subscription-window` answer only: a
non-loopback caller that passed the token gate, a loopback caller (peer
and `Host` header both loopback), a trusted dashboard client, or, under
an acknowledged token-less bind, the container's own default-gateway TCP
peer with a loopback `Host` (the host browser reaching a
`127.0.0.1`-published container). Everyone else gets 404, as with
`/admin`. Settings writes keep their stricter rule with no token
short-cut.
- `/metrics` answers a non-loopback caller that passed the token gate, a
loopback caller, a connecting peer in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` (whatever it forwards), or a
resolved client in the dashboard CIDRs. For CIDR peers a hostname `Host`
is fine, so Prometheus scrapers work; a cross-origin browser request is
still refused.
- The token short-cut applies only to non-loopback requests. The gate
exempts loopback peers without checking `Host`, so a loopback request
has not proven it holds the token and must pass the loopback `Host`
check. This blocks a DNS-rebound browser on the operator's machine.
- The `/dashboard` shell stays ungated. It is a static template, like
the already public `/dashboard/static` assets. Gating it would break the
dashboard for containers published on host loopback, where the peer is
the bridge gateway and `Host` is `localhost`. The data routes behind it
are gated.
- The WebSocket loopback exemption requires a loopback peer and a
loopback `Host` header. A loopback peer with a foreign `Host` can still
connect with the token.
- Launchers that always publish on `127.0.0.1` set the acknowledgement
themselves: `headroom install` (docker preset), `scripts/install.sh` and
`scripts/install.ps1` (through one helper that adds the acknowledgement
only together with the `127.0.0.1` publication), and
`docker/docker-compose.native.yml`.
- Docs: `proxy.mdx` documents the new variable and which routes the
token also gates; `docker-install` (docs and wiki) and the
`docker-compose.yml` comment show the acknowledgement in the
loopback-published example. `e2e/docker-bind-security.sh` sets it and
checks that the same launch without it refuses to start.

## Behaviour change (breaking)

- `headroom proxy --host 0.0.0.0`, or `HEADROOM_HOST` set to any
non-loopback address, without `HEADROOM_PROXY_TOKEN` now refuses to
start. The error names both remedies. Set a token, bind loopback, or set
`HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` when the port is already
confined. Anyone running the image by hand with `--host 0.0.0.0` and no
token must add one of these.
- Without a token, `/stats-history`, `/quota`, `/subscription-window`
and `/metrics` return 404 to non-loopback callers outside a trusted
CIDR. Prometheus scrapers on another host must be in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` or send the token.
- Code that relied on a request with no peer address being treated as
loopback now gets the non-loopback path.
- The default loopback run is unchanged.

## Tests

- `tests/test_proxy_bind_policy.py`: the refuse/allow matrix
(non-loopback, loopback, unset host, token, acknowledgement values), the
warning on the acknowledged path, `create_app`, the `run_server` exit
before uvicorn, and the CLI (refusal, `HEADROOM_HOST` from env,
acknowledged start, offline banner, start with a token).
- `tests/test_proxy_loopback_gating.py`: each data route against
non-loopback, loopback, trusted-CIDR and token callers; the dashboard
shell stays reachable; the host of an acknowledged, loopback-published
container reads the data routes while another bridge peer, a forwarded
gateway address and a foreign `Host` do not; CSV export is not served to
network callers; `/metrics` across loopback, both CIDR lists,
out-of-range peers, cross-origin browsers and token callers, including a
trusted gateway relaying a scraper outside its own range. A regression
test shows that a token on the proxy does not exempt a loopback peer
from the `Host` check; it fails without the fix.
- `tests/test_proxy_hardening.py`: the HTTP gate with no peer address,
WebSocket `Host` cases, `require_loopback` and `/debug` with no peer.
- Installer tests: `headroom install` adds the acknowledgement, and the
shell and PowerShell wrappers add it only together with the loopback
publication. Launcher-level tests start the proxy with the environment
and `--host` that `build_runtime_command()` and
`docker-compose.native.yml` produce and check that the host browser
(bridge-gateway peer) gets `/stats-history` and another container does
not.
- Tests that bound `0.0.0.0` for unrelated reasons now set a token or
the acknowledgement. Tests that read operator routes use an explicit
loopback peer.

On `main`, 14 of the gating and hardening tests fail and the bind-policy
module does not exist. Manual runs: token-less `--host 0.0.0.0` exits
with the refusal from both the CLI and `run_server`; with the
acknowledgement, the data routes are 404 to a LAN peer and to a loopback
peer with a foreign `Host`; with a token, a LAN peer gets 401 without it
and 200 with it; the default loopback bind starts as before.

Not tested: a real Unix-domain-socket deployment and a real Prometheus
scrape (both covered by tests only).

## Merge order

Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 →
#3863. This PR goes first: it is the gate the others assume, and it is
the release-note event.

## Not in this PR

- Enforcing the token for loopback callers when one is configured.

Revised 2026-10-02 after review: `/metrics` matches gateway CIDRs
against the TCP peer, and the operator data routes serve the host of a
loopback-published container (bridge-gateway peer under the launchers'
bind acknowledgement) so the default Docker dashboard works.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3852"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:27:35 -05:00
Rod BoevandJerrettDavis b0c19a25f0 fix(security): reject unauthenticated public proxy binds (#3316)
## Description

Headroom's published image and maintained Docker launchers could expose
the
proxy on every host interface without a proxy token. This change moves
the
security boundary to the deployment artifacts: published images default
to a
container-loopback listener, supported local launchers publish only on
host
loopback, and deliberate public deployments retain explicit token
protection.
The application factory remains transport-neutral.

Closes #3283

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Set both published image stages to a container-loopback default while
keeping explicit host overrides available.
- Changed native Compose and the maintained Bash and PowerShell
launchers to use an internal `0.0.0.0` listener with host-loopback
publication.
- Preserved the top-level Compose token requirement and the existing
HTTP and WebSocket credential behavior for deliberate public binds.
- Scoped differential-capture token injection to the internal
`headroom-proxy` destination and kept sensitive headers redacted in
captured records.
- Added focused launch-contract, native-core, capture, and Docker
publication checks, plus a base/head harness that tolerates a base
already containing the fix.
- Updated Docker, installer, capture, and mirrored documentation,
including an unauthenticated local loopback quickstart and an
authenticated deliberate-public example.

## Testing

- [x] Focused unit and integration tests pass
- [x] Linting and formatting checks pass
- [ ] Type checking passes
- [x] New tests added for the changed deployment and capture behavior
- [ ] Manual testing performed

### Test Output

```text
uv run python -m pytest tests/test_proxy_hardening.py tests/test_install/test_native_installers.py tests/test_docker_compose_persistence.py tests/test_network_diff_capture.py tests/test_rust_core_smoke.py -q
76 passed, 1 skipped, 1 warning in 23.45s

uv run ruff check tests/test_install/test_native_installers.py tests/test_install/test_runtime.py
All checks passed!

bash -n e2e/docker-bind-security.sh
passed

bash e2e/docker-bind-security.sh --base-image headroom-t3283-base --head-image headroom-t3283-head --fixture tests/fixtures/docker-bind-exposure.json --base-ref origin/main
base_ref=origin/main ... runtime_slim_config=loopback rust_core=loaded local_completion=200 attacker_loopback=000 ws_status=403
docker bind security: fixture forwarding, loopback isolation, built-image health, slim-image loopback config, and token controls passed
```

## Real Behavior Proof

- Environment: Windows, Docker Desktop 29.6.2, current `origin/main`
base, head image, and a real Docker bridge peer.
- Exact command / steps: run the focused security/runtime tests, Compose
configuration checks, then `bash e2e/docker-bind-security.sh` against
the current base and head image definitions.
- Observed result: `76 passed, 1 skipped`; the built-image harness
passed base/head health, local completion, non-loopback isolation,
authenticated public HTTP/WebSocket controls, loopback runtime-slim
configuration, and direct Rust-core loading.
- Not tested: runtime-slim proxy service health on this host, because
its unrelated ONNXRuntime import path exits with code 139; the full
hosted matrix and a future published GHCR image remain CI-owned.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Unsafe override required: no
- Minimum rollout channel: next release containing the Docker and
launcher changes
- Stable/default behavior changed: published images and maintained local
Docker launchers now use loopback-safe publication defaults
- Kill switch / disable path: revert the deployment-artifact changes
before release; deliberate public deployments remain explicitly
operator-controlled
- Qualification impact: Docker users relying on all-interface default
publication must opt into public publication and configure the existing
proxy token
- Deliberate public deployment: use an explicit public host publication,
set `HEADROOM_PROXY_TOKEN`, and send the matching bearer or
`X-Headroom-Proxy-Token` credential
- Local deployment: use the maintained loopback publication and no
mandatory token
- Rollback path: revert the deployment-artifact and documentation
changes together

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove the behavior is effective
- [x] Focused tests pass locally with these changes
- [x] I did **not** edit `CHANGELOG.md`; release-please generates
release notes from the conventional PR title

## Additional Notes

The issue fixture remains provenance for the unauthenticated
publication. Native
`headroom proxy` defaults and the application-level authentication
owners are
unchanged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-24 23:58:46 -05:00
Tejas Chopra e0ce4b1d48 fix: remove rtk and lean-ctx CLI context tools (#2677)
## Description

Removes both third-party CLI context tools — **rtk** and **lean-ctx** —
and with them the context-tool selector itself. Headroom no longer
downloads, installs or configures either one, and there is no
replacement.

The previous pass (#2344) gated only three entry points inside
`headroom/cli/wrap.py`. That left the feature reachable in practice:

| Gap | Effect |
|---|---|
| `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global
--auto-patch` from bash/PowerShell, **bypassing the Python gate
entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook
regardless of `HEADROOM_RTK` |
| `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was
broken by default**: `rtk_required=True` met a gate returning `None` →
`SystemExit(1)`. Invisible because all 8 openhands tests patched
`_ensure_rtk_binary` to a fake path |
| `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to
`rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker
polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) |
| No cleanup path | Nothing removed artifacts an earlier default had
installed, so a machine that once ran the old default kept rtk in the
loop forever (#1669, #1955) |

Also worth noting: the rtk binary download had **no SHA or signature
verification** — only `rtk --version` as a smoke test.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages,
`headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` /
`_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` /
`--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap
subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the
dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine
getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers,
`benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path
filters.

**Fails loudly, not silently** — `--context-tool` / `--no-context-tool`
/ `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in
shell profiles, aliases and CI jobs, and accepting them as a no-op would
read as Headroom having quietly stopped working. The installers reject
them too, which matters more than it looks: their arg parsers forward
the first unknown flag **and everything after it** to the wrapped tool,
so a leftover `--no-rtk` would have silently swallowed a following
`--port` and then been ignored downstream.

**New `headroom/context_tool_cleanup.py`** — deleting the code cannot
help a machine that already ran the old default, since the hooks,
binaries and injected guidance are durable on disk.
`purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and
removes the registered hook entries, the generated hook scripts, the
Headroom-managed `~/.local/bin` symlinks, the vendored
`~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server
entry and the marker-fenced instruction blocks. Deliberately
conservative: idempotent, **skips** a malformed config rather than
overwriting it, and only unlinks a symlink resolving inside Headroom's
own bin dir so a user's own build is untouched. It reports on
**stderr**, because `wrap/unwrap openclaw --prepare-only` emit
machine-readable JSON on stdout as their entire contract. Skipped for
`wrap selfheal` (runs from a SessionStart hook; must not race Claude
Code's writer for `~/.claude.json`) and for `--help`, which must stay
read-only.

**Client-config hardening** (discovered while investigating a "corrupted
Serena settings file" report) — `wrap.py` reset a settings file to `{}`
when an existing file would not parse, then wrote that back. One
hand-edited typo or a transient `EACCES`/`EINTR` on a valid file
destroyed the user's `permissions`, `env` and `hooks`, on **every
`headroom wrap claude`**. It now refuses to write. Separately,
`fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`),
fixing all 14 non-atomic client-config writes at once; it follows
symlinks rather than replacing them (dotfile managers) and preserves an
existing file's mode.

**Deliberately kept** — `rtk` stays in the wrapper-peel list in
`transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as
shell-command grammar, so `rtk cat f` is still classified as a file read
for anyone running their own rtk install, which the purge intentionally
leaves alone.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
All checks passed!

$ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
1255 files already formatted

$ mypy headroom/
Success: no issues found in 508 source files

$ pytest tests/test_context_tool_cleanup.py -q
11 passed

$ pytest tests/test_fsutil.py -q
12 passed

$ pytest tests/test_cli/test_wrap_codex.py -q            # 89 tests
89 passed in 431.68s
$ pytest tests/test_cli/test_wrap_opencode.py -q
39 passed in 257.46s
$ pytest tests/test_cli/test_wrap_helpers.py -q
45 passed
$ pytest tests/test_paths.py -q
75 passed
$ pytest tests/test_cli/test_unwrap_claude.py -q
14 passed
$ pytest tests/test_proxy_savings_history.py -q
39 passed
$ pytest tests/test_cli/test_wrap_copilot.py -q
27 passed
$ pytest tests/test_cli/test_wrap_zcode.py -q
20 passed
$ pytest tests/test_subscription_tracker.py -q
9 passed
$ pytest tests/test_proxy_dashboard_stats_cache.py -q
5 passed, 1 skipped
```

Repo-wide grep for 14 removed symbols (`headroom.rtk`,
`headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`,
`_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`,
`wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`,
`tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`,
`*.html`: **zero hits**.

Notable test changes: `test_wrap_openhands.py` no longer patches
`_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0
unpatched — the regression that was previously masked.
`test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed
(every test drove RTK instruction injection). A new
`test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed`
proves a pre-removal `subscription_state.json` still loads.

## Real Behavior Proof

- **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @
this branch, real `~/.headroom` and `~/.claude` on the dev machine.
- **Exact command / steps and observed result:**

```text
# 1. Retired flag fails loudly instead of silently no-op'ing
$ headroom wrap codex --prepare-only --context-tool rtk
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they
rewrote shell commands through a third-party binary Headroom no longer manages.
Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL;
`headroom wrap` uninstalls what they left behind on first run.

$ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ...

# 2. install.sh rejects the retired flags (extracted parse_wrap_args harness)
['--no-rtk', '--port', '9999']   rc=1  ERROR: CLI context tools ... Drop --no-rtk
['--context-tool=rtk']           rc=1  ERROR: CLI context tools ... Drop --context-tool
$ bash -n scripts/install.sh   # syntax OK

# 3. Purge ran against the real machine, which had all the orphaned artifacts
$ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..."
  removed ~/.headroom/bin/lean-ctx        (51 MB)
  removed ~/.headroom/bin/rtk             (7.7 MB)
  removed ~/.local/bin/rtk                (symlink into ~/.headroom/bin)
  removed ~/.claude/hooks/rtk-rewrite.sh
  removed 8 lean-ctx-* hook scripts
# ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged
# → ~59 MB reclaimed, no unrelated key touched

# 4. stdout stays machine-readable while the purge reports (planted a fake artifact)
$ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err
$ cat out
{"enabled":true,"config":{"proxyPort":8787,...}}     # parses as JSON
$ cat err
Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk

# 5. --help is inert (planted artifact survives), a real run purges
$ headroom wrap codex --help   → artifact survived: CORRECT
$ headroom wrap openclaw --prepare-only → purged: CORRECT

# 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json
top-level keys 90 -> 90;  projects 19 -> 19;  LOST keys: none
all content outside mcpServers byte-identical: True
```

Dashboard rendered via the Playwright test after the panel removal:
"Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`,
and "Token Usage" reads Before Compression → Proxy Removed → After
Compression with no "Filtered (this session)" row. Nothing below the
removed panel broke.

- **Not tested:** Windows and Linux (macOS only) — `install.ps1` is
verified by brace-balance and inspection, not executed, since no `pwsh`
is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated
but not run; it needs the Docker e2e image. `serena project index`
interaction is exercised in the stacked base PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

**Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge
that first; this PR's base should then be retargeted to `main`, or it
will read as containing that fix too.

**Breaking-change migration for users:**
- Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`,
`--context-tool`, `--no-context-tool` from any alias, script or CI job,
and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error
rather than being ignored, so the failure is immediate and
self-explaining.
- Previously-installed artifacts are purged automatically on the next
`wrap`/`unwrap`; no manual cleanup needed.
- `headroom perf --json` no longer carries a `cli_filtering` key, and
`/stats` no longer returns a `context_tool` section.

**Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from
`README.md`,
`docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`,
`docs/observability.md` and the matching `wiki/` pages.
`REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED
rather than deleted, to keep the planning record.

**Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as
dead code — its only caller was the `except KeyboardInterrupt` guarding
the binary download, so with no download there is nothing slow left to
interrupt.
2026-07-30 22:59:41 -07:00
c3db8e47f8 fix(install): default docker image to headroomlabs-ai GHCR registry (#1867) (#2039)
## Description

The GitHub repository was transferred from `chopratejas/headroom` to
`headroomlabs-ai/headroom`. GitHub 301-redirects transferred repos for
web and git operations, but **GitHub Container Registry (GHCR) does
not** — the old package `ghcr.io/chopratejas/headroom` is now orphaned
and frozen (its `latest` tag stopped advancing at `0.27.0`), while CI
publishes new images to `ghcr.io/headroomlabs-ai/headroom` (the workflow
derives the path from `${{ github.repository }}`).

Headroom's install tooling still defaulted to the dead path, so
`headroom install apply --preset persistent-docker`, `headroom init`,
and the standalone install scripts all pulled a stale `0.27.0` image
instead of the current release.

This changes every docker-image **default** to
`ghcr.io/headroomlabs-ai/headroom:latest`.

Closes #1867

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/models.py` — `InstallManifest.image` default.
- `headroom/cli/install.py` — `--image` Click option default.
- `headroom/cli/init.py` — two `InstallManifest(...)` image args.
- `scripts/install.sh` / `scripts/install.ps1` — `IMAGE_DEFAULT` /
`$ImageDefault` plus the `--image` help-text default.
- `docker/docker-compose.native.yml` — image default in both services.
- `tests/test_install/test_planner.py` (4) and
`tests/test_install/test_runtime.py` (5) — updated the assertions that
pinned the old image (including `assert "<image>" in command`), so they
now verify the corrected registry threads through the planner and docker
runtime command.

Scope note: `github.com/chopratejas/...` links and the plugin
marketplace slug are intentionally **not** changed — GitHub redirects
those, so they still work. Only the genuinely-dead GHCR image references
are touched. No new dependency, no new abstraction.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_install/ -q
99 passed, 1 skipped in 48.34s

$ uv run ruff check headroom/install/models.py headroom/cli/install.py headroom/cli/init.py
All checks passed!

$ grep -rn "ghcr.io/chopratejas/headroom" headroom/ scripts/ docker/ tests/
# (no source matches — every default now points at headroomlabs-ai)
```

The install-test assertions pinned the old image, so they fail against
the old defaults and pass after the fix — they are the regression guard.

## Real Behavior Proof

- **Environment:** Windows 11, Python 3.13.5, headroom installed from
this branch (editable, via `uv`).
- **Exact command / steps:** The dead-registry claim is verifiable at
the registry level, independent of a release:
  ```text
docker pull ghcr.io/chopratejas/headroom:latest # old default → 0.27.0
(frozen / orphaned)
docker pull ghcr.io/headroomlabs-ai/headroom:latest # new default →
current release
  ```
And the install pipeline now emits the correct image (covered by
`tests/test_install/test_runtime.py`, which asserts the resolved docker
command contains `ghcr.io/headroomlabs-ai/headroom:latest`).
- **Observed result:** `grep` confirms no `ghcr.io/chopratejas/headroom`
default remains in code, scripts, compose, or tests;
`tests/test_install/` is green (99 passed) with the corrected image
asserted end-to-end through planner → runtime command.
- **Not tested:** A live `docker pull` of both tags on this specific
machine (no local Docker daemon guaranteed) — the registry difference is
reproducible by anyone running the two `docker pull` commands above; and
an end-to-end `headroom install apply --preset persistent-docker`
against a real Docker host.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- Documentation checklist item is N/A — no user-facing docs surface
beyond the CHANGELOG entry.
- `mypy` left unchecked — not run as part of this verification; the
change is a string-default swap with no type-level surface.
- The `--image` help text and `docker-compose.native.yml` were included
so every user-facing default is consistent; only the dead GHCR image
string was changed.

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-13 14:01:28 -04:00
Rod Boev 361adcd1a0 fix(dashboard): distinguish unavailable RTK from zero stats in Docker (#1901)
## Description

Dockerized Headroom shows `0` for RTK/context-tool dashboard figures
whenever the `rtk` binary isn't reachable inside the proxy's runtime —
indistinguishable from "genuinely nothing saved yet." The backend
already computes this distinction (an `installed`/`available` flag on
the context-tool stats payload) but it never reaches two of the JSON
surfaces the dashboard reads from, and the dashboard template never
checks the one surface that already has it. This PR threads that
existing availability flag through to both surfaces and updates the
dashboard to show a distinct "not installed" message instead of a bare
`0`, plus a short Docker note so operators know `rtk` needs to be
installed inside the container for those figures to populate at all.

Closes #1831

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/server.py`: reuse the existing context-tool
`installed` flag as one `available` boolean, add it to
`savings.by_layer.cli_filtering` in `/stats`, and add it to the curated
`cli_filtering` block in `/stats-history`; corrected that endpoint's
stale docstring claim that `cli_filtering` is `None` whenever RTK is
absent.
- `headroom/dashboard/templates/dashboard.html`: added
`cliFilteringAvailable`/`historyCliFilteringAvailable` getters and used
them to show a "not installed" message instead of `0` in the session
view's Token Usage panel and Token Savings breakdown, and to keep the
Historical tab's lifetime card hidden (its existing behavior) instead of
showing a stale zero.
- `docker-compose.yml` and `docker/docker-compose.native.yml`: added a
one-line comment noting that `rtk` needs to be installed inside the
container for CLI-filtering dashboard figures to populate.
- `docs/content/docs/docker-install.mdx`: added a note to the existing
Notes section about the same requirement.
- Added focused pytest coverage for the new JSON field on both endpoints
(installed, not-installed, and hard-failure cases) and a new Playwright
spec covering the rendered not-installed / genuine-zero / Historical-tab
states.

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_proxy_dashboard_stats_cache.py
tests/test_proxy_savings_history.py -q`)
- [x] Linting passes (`uv run ruff check .`)
- [ ] Type checking passes (`uv run mypy headroom`) or explain N/A
truthfully
- [x] New tests added for new functionality when applicable
- [x] Manual testing performed

### Test Output

```text
uv run pytest tests/test_proxy_dashboard_stats_cache.py tests/test_proxy_savings_history.py -q
51 passed, 1 skipped, 1 failed

uv run ruff check headroom/proxy/server.py tests/test_proxy_dashboard_stats_cache.py tests/test_proxy_savings_history.py tests/test_dashboard_context_tool_availability_playwright.py
All checks passed!
```

The one failure (`test_savings_tracker_save_fsyncs_parent_directory`) is
pre-existing and unrelated to this change; it reproduces identically on
a clean `origin/main` checkout with this diff removed (Windows
filesystem fsync behavior).

## Real Behavior Proof

- Environment: Windows sandbox, Python (uv-managed), no live Docker
container
- Exact command / steps: `GET /stats` and `GET /stats-history` against a
`TestClient` app with the context-tool stats source monkeypatched to a
not-installed payload (mirrors the exact shape
`_context_tool_zero_payload` produces when `rtk` is absent), then the
same with an installed-but-zero payload
- Observed result: `savings.by_layer.cli_filtering.available` and
`/stats-history`'s `cli_filtering.available` are `False` for the
not-installed payload and `True` for the installed-but-zero payload,
matching the pre-existing `context_tool.available` field; the new
Playwright spec exercises the corresponding dashboard rendering states
and runs in CI's "Dashboard Playwright" check
- Not tested: real rendering in a live browser against a live Docker
container (this sandbox cannot run the CI-only Dashboard Playwright job
locally); the fix is proved locally at the JSON-contract level and the
rendering claim is proved by the contributed CI-executed Playwright spec

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

CHANGELOG.md was intentionally left unchanged — release automation
derives changelog entries from conventional commits per this repo's
convention, and this is a dashboard/docs clarity fix rather than a new
user-facing command or config option. Type checking was not re-run in
isolation for this change; it's covered by the repo's CI lint job.
2026-07-09 09:39:16 -04:00
JerrettDavisandClaude Opus 4.7 4a87753713 feat(docker): forward HEADROOM_WORKSPACE_DIR and HEADROOM_CONFIG_DIR into containers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 19:19:25 -05:00
JerrettDavisandCopilot 21896a095c feat: add persistent install lifecycle management
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 13:47:05 -05:00
JerrettDavisandCopilot 38b1483a76 feat(cli): add Docker-native install flow and parity docs
Add system-native install scripts and host wrappers for running Headroom from Docker while keeping wrapped tools on the host. Document the Docker-native path, add a complete CLI reference with help output and parity details, and add support for root help/version aliases and proxy env-based binding behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-10 23:27:24 -05:00