Commit Graph
6 Commits
Author SHA1 Message Date
natoriandJD Davis 1b7977cecd fix: honor configured port in container startup and healthcheck (#2436)
## Description

Deploying on a non-default port produced a working proxy that the
surrounding
tooling reported as broken. Tracing the probe turned up a second defect
upstream of it: the container could not change its port via
`HEADROOM_PORT`
at all.

`headroom proxy --port` is a click option with `envvar="HEADROOM_PORT"`,
and an
explicit CLI argument beats the envvar. The Dockerfile baked `--port
8787` into
CMD, so `HEADROOM_PORT` was inert for a plain `docker run` — while
`docker-compose.yml` already passes `command: ["--host", "0.0.0.0"]`
with no
port, which is the intended shape. The Dockerfile was the outlier.

The two halves have to move together. An earlier revision of this change
fixed
only the HEALTHCHECK; that made the probe follow `HEADROOM_PORT` while
the
listener stayed pinned to 8787, and the container went from misleadingly
healthy to genuinely unhealthy. Evidence in Real Behavior Proof below.

Closes #2432

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `Dockerfile`, both runtime stages: dropped the baked `--port` from CMD
so
`HEADROOM_PORT` stays authoritative. `docker run` with no env still
binds
  8787 (click's default), so the published default is unchanged.
- `Dockerfile` (`runtime-slim-base`): shell-form HEALTHCHECK so
  `${HEADROOM_PORT:-8787}` expands at probe time.
- `Dockerfile` (`runtime-slim`): distroless, no shell — the port is
resolved
  inside Python instead, keeping exec form.
- `headroom/cli/doctor.py`: added `resolve_probe_port()`. `--port` /
`HEADROOM_PORT` wins, then an unambiguous deployment manifest (`default`
profile, or a lone named profile), then 8787. `headroom deploy --port N`
writes N into the deployment's environment, not into the shell `doctor`
runs
in, which is why the manifest tier is needed; with several named
profiles and
  no explicit request there is no non-arbitrary pick, so it stays on the
  default and the existing per-deployment section reports each one.
  `list_manifests()` is now read once and shared with that check.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom --ignore-missing-imports`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

Full suite, run inside this repo's own builder image so the
maturin-compiled
Rust core is the one under test:

```text
9 failed, 9248 passed, 853 skipped, 2230 warnings in 384.56s (0:06:24)
```

All 9 failures reproduce unchanged on `origin/main` (c400f908) in the
same
container. Restricted to just those files, base and branch are
identical:

```text
origin/main c400f908: 13 failed, 119 passed, 15 skipped, 66 warnings in 14.57s
this branch:          13 failed, 119 passed, 15 skipped, 66 warnings in 14.58s
```

They are `test_cli/test_wrap_helpers.py`, `test_image_compression.py`,
`test_memory_bridge.py`, `test_proxy/test_cc_switch_reconciler.py` and
`test_startup_log_noise.py`. Counts differ between the full and
file-scoped
runs because `test_memory_bridge` and `test_startup_log_noise` are
order-dependent — also true on the base.

```text
$ mypy headroom --ignore-missing-imports
pyproject.toml: note: unused section(s): module = ['mlx.*']
Success: no issues found in 505 source files
```

New and changed tests, green on this branch:

```text
tests/test_docker_healthcheck_port.py ......                   [100%]
6 passed in 0.15s

tests/test_cli_doctor.py .............................         [100%]
67 passed in 0.68s
```

Both go red with only the tests applied on top of `origin/main`:

```text
FAILED tests/test_docker_healthcheck_port.py::test_no_healthcheck_hardcodes_the_default_port
FAILED tests/test_docker_healthcheck_port.py::test_every_healthcheck_resolves_the_port_from_the_environment
FAILED tests/test_docker_healthcheck_port.py::test_distroless_stage_healthcheck_stays_shell_free
FAILED tests/test_docker_healthcheck_port.py::test_debian_stage_healthcheck_expands_the_port_variable
FAILED tests/test_docker_healthcheck_port.py::test_no_stage_bakes_a_port_into_cmd
5 failed, 1 passed in 0.23s

tests/test_cli_doctor.py:
E   ImportError: cannot import name 'DEFAULT_PROXY_PORT' from 'headroom.cli.doctor'
```

## Real Behavior Proof

- Environment: macOS 15 (Darwin 25.3.0) arm64, Docker 29.1.3, image
built from this branch with `docker build -t headroom:pr2432 .`; control
is the published `ghcr.io/headroomlabs-ai/headroom:latest`.
- Exact command / steps: started three containers — the patched image
with `HEADROOM_PORT=8789`, the patched image with no env at all, and the
published image with `HEADROOM_PORT=8790` as a control — then read
`docker ps` health and probed `/readyz` from inside each. Full commands
below.
- Observed result: the patched image binds 8789 and its probe follows,
reporting healthy; with no env it still binds 8787, so the published
default is unchanged; the published image ignores `HEADROOM_PORT`
entirely and stays on 8787. Output below.
- Not tested: `headroom deploy` was not driven end to end — the manifest
was written directly rather than produced by the deploy command, so the
manifest *contents* are assumed to match what deploy records (`port` is
the field `headroom install status` already reads). Non-Docker
supervisors were not exercised.

### Containers

```bash
docker run -d --name hr-fixed   -e HEADROOM_PORT=8789 -p 8789:8789 headroom:pr2432
docker run -d --name hr-default                       -p 8791:8787 headroom:pr2432
docker run -d --name hr-control -e HEADROOM_PORT=8790 -p 8790:8790 \
    ghcr.io/headroomlabs-ai/headroom:latest
```

```text
NAMES        STATUS
hr-fixed     Up (healthy)   HEADROOM_PORT=8789 -> listener on 8789
hr-default   Up (healthy)   no env             -> listener on 8787
hr-control   Up (healthy)   HEADROOM_PORT=8790 -> listener on 8787
```

Probing from inside each container confirms which port is actually
serving:

```text
hr-fixed:   8789=OK
hr-default: 8787=OK
hr-control: 8787=OK
```

`hr-control` is the published image ignoring `HEADROOM_PORT` entirely —
the CMD
half of the bug. Baked image config, before and after:

```text
published: ["CMD","curl","--fail","--silent","http://127.0.0.1:8787/readyz"]
this PR:   ["CMD-SHELL","curl --fail --silent \"http://127.0.0.1:${HEADROOM_PORT:-8787}/readyz\""]
```

The distroless stage was verified the same way — `docker build --target
runtime-slim` with `-e HEADROOM_PORT=8795` reports healthy and answers
`GET /readyz` with 200 from the host.

### doctor

Running the real CLI against a seeded `default` deployment manifest
recording
port 8789, with `HEADROOM_PORT` unset in the shell — the reported
situation,
where the port lives in the deployment rather than in the shell:

```text
origin/main c400f908: doctor probed port: 8787
this branch:          doctor probed port: 8789
```

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from
      my Conventional Commit PR title (a CI guard enforces this)

## Screenshots (if applicable)

N/A — no UI surface.

## Additional Notes

"I have made corresponding changes to the documentation" is checked in
the sense
that the `--port` help text now describes the new resolution order; no
prose
docs referenced the baked CMD port. `README.md` and the wiki reference
`headroom proxy --port 8787` as a host command, which is unaffected.

`docker/differential-network-capture/docker-compose.yml` passes `--port
8787`
explicitly in its own `command:`, which still overrides everything and
keeps
working.

Adjacent, deliberately not in this PR: `docker-compose.yml`'s
healthcheck is
hardcoded to 8787 the same way. It is inert today because that file
never sets
`HEADROOM_PORT`, and making it configurable also means parameterising
the
published `ports:` mapping — a separate change. Happy to open a
follow-up if
you want it.

## Runtime Rollout Safety

- Rollout-managed feature(s): Docker runtime port selection and
`headroom doctor` probe-port resolution.
- Minimum rollout channel: Standard Docker image release and persistent
Docker deployments.
- Stable/default behavior changed: No. Unset `HEADROOM_PORT` continues
to use port 8787.
- Kill switch / disable path: No new feature flag; revert the image or
this PR.
- Unsafe override required: No.
- Qualification impact: None. No data or schema migration; existing
port-8787 deployments remain compatible.
- Rollback path: Redeploy the previous image or revert this PR; no
migration cleanup is required.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2436"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:36:54 -05:00
ee731d7ea8 fix(proxy): refuse token-less non-loopback binds, gate operator routes (#3852)
## Why

Four related gaps in the proxy's network gate (findings 01-F1, 01-F2,
01-F5, 01-F9):

- **Open bind.** A non-loopback bind with no `HEADROOM_PROXY_TOKEN` only
logged a warning, then served the `/v1/*` relay, with the operator's
upstream credentials, to every peer that could reach the port.
- **Missing peer address.** `is_loopback_host(None)` returned `True`. A
Unix-domain socket, an ASGI adapter that leaves `scope["client"]` unset,
or a hand-built request therefore turned off the token gate, the
`/admin` and `/debug` guards and the WebSocket gate.
- **Operator data routes.** `/stats-history` (including CSV export),
`/quota`, `/subscription-window` and `/metrics` were served to any
network caller.
- **WebSocket loopback exemption** checked only the peer IP, not the
`Host` header, so it had no DNS-rebinding defence.

## What changes

- New `headroom/proxy/bind_policy.py`. A non-loopback bind (`0.0.0.0`,
`::`, a LAN address or a hostname) with no token is refused at three
places: `create_app` (embedders and the multi-worker factory),
`run_server` (one exit with code 2 before uvicorn forks) and the
`headroom proxy` CLI (a clear error before the banner, exit code 1). An
absent host means uvicorn's default `127.0.0.1`.
- `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` acknowledges an open bind when
the runtime already restricts reachability, such as a container
published on `127.0.0.1`. The `proxy_open_bind` warning is still logged
on that path, and the banner says the bind is open.
- `is_loopback_host(None)` now returns `False` (fail closed), matching
`identity.py`.
- `/stats-history`, `/quota` and `/subscription-window` answer only: a
non-loopback caller that passed the token gate, a loopback caller (peer
and `Host` header both loopback), a trusted dashboard client, or, under
an acknowledged token-less bind, the container's own default-gateway TCP
peer with a loopback `Host` (the host browser reaching a
`127.0.0.1`-published container). Everyone else gets 404, as with
`/admin`. Settings writes keep their stricter rule with no token
short-cut.
- `/metrics` answers a non-loopback caller that passed the token gate, a
loopback caller, a connecting peer in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` (whatever it forwards), or a
resolved client in the dashboard CIDRs. For CIDR peers a hostname `Host`
is fine, so Prometheus scrapers work; a cross-origin browser request is
still refused.
- The token short-cut applies only to non-loopback requests. The gate
exempts loopback peers without checking `Host`, so a loopback request
has not proven it holds the token and must pass the loopback `Host`
check. This blocks a DNS-rebound browser on the operator's machine.
- The `/dashboard` shell stays ungated. It is a static template, like
the already public `/dashboard/static` assets. Gating it would break the
dashboard for containers published on host loopback, where the peer is
the bridge gateway and `Host` is `localhost`. The data routes behind it
are gated.
- The WebSocket loopback exemption requires a loopback peer and a
loopback `Host` header. A loopback peer with a foreign `Host` can still
connect with the token.
- Launchers that always publish on `127.0.0.1` set the acknowledgement
themselves: `headroom install` (docker preset), `scripts/install.sh` and
`scripts/install.ps1` (through one helper that adds the acknowledgement
only together with the `127.0.0.1` publication), and
`docker/docker-compose.native.yml`.
- Docs: `proxy.mdx` documents the new variable and which routes the
token also gates; `docker-install` (docs and wiki) and the
`docker-compose.yml` comment show the acknowledgement in the
loopback-published example. `e2e/docker-bind-security.sh` sets it and
checks that the same launch without it refuses to start.

## Behaviour change (breaking)

- `headroom proxy --host 0.0.0.0`, or `HEADROOM_HOST` set to any
non-loopback address, without `HEADROOM_PROXY_TOKEN` now refuses to
start. The error names both remedies. Set a token, bind loopback, or set
`HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` when the port is already
confined. Anyone running the image by hand with `--host 0.0.0.0` and no
token must add one of these.
- Without a token, `/stats-history`, `/quota`, `/subscription-window`
and `/metrics` return 404 to non-loopback callers outside a trusted
CIDR. Prometheus scrapers on another host must be in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` or send the token.
- Code that relied on a request with no peer address being treated as
loopback now gets the non-loopback path.
- The default loopback run is unchanged.

## Tests

- `tests/test_proxy_bind_policy.py`: the refuse/allow matrix
(non-loopback, loopback, unset host, token, acknowledgement values), the
warning on the acknowledged path, `create_app`, the `run_server` exit
before uvicorn, and the CLI (refusal, `HEADROOM_HOST` from env,
acknowledged start, offline banner, start with a token).
- `tests/test_proxy_loopback_gating.py`: each data route against
non-loopback, loopback, trusted-CIDR and token callers; the dashboard
shell stays reachable; the host of an acknowledged, loopback-published
container reads the data routes while another bridge peer, a forwarded
gateway address and a foreign `Host` do not; CSV export is not served to
network callers; `/metrics` across loopback, both CIDR lists,
out-of-range peers, cross-origin browsers and token callers, including a
trusted gateway relaying a scraper outside its own range. A regression
test shows that a token on the proxy does not exempt a loopback peer
from the `Host` check; it fails without the fix.
- `tests/test_proxy_hardening.py`: the HTTP gate with no peer address,
WebSocket `Host` cases, `require_loopback` and `/debug` with no peer.
- Installer tests: `headroom install` adds the acknowledgement, and the
shell and PowerShell wrappers add it only together with the loopback
publication. Launcher-level tests start the proxy with the environment
and `--host` that `build_runtime_command()` and
`docker-compose.native.yml` produce and check that the host browser
(bridge-gateway peer) gets `/stats-history` and another container does
not.
- Tests that bound `0.0.0.0` for unrelated reasons now set a token or
the acknowledgement. Tests that read operator routes use an explicit
loopback peer.

On `main`, 14 of the gating and hardening tests fail and the bind-policy
module does not exist. Manual runs: token-less `--host 0.0.0.0` exits
with the refusal from both the CLI and `run_server`; with the
acknowledgement, the data routes are 404 to a LAN peer and to a loopback
peer with a foreign `Host`; with a token, a LAN peer gets 401 without it
and 200 with it; the default loopback bind starts as before.

Not tested: a real Unix-domain-socket deployment and a real Prometheus
scrape (both covered by tests only).

## Merge order

Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 →
#3863. This PR goes first: it is the gate the others assume, and it is
the release-note event.

## Not in this PR

- Enforcing the token for loopback callers when one is configured.

Revised 2026-10-02 after review: `/metrics` matches gateway CIDRs
against the TCP peer, and the operator data routes serve the host of a
loopback-published container (bridge-gateway peer under the launchers'
bind acknowledgement) so the default Docker dashboard works.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3852"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:27:35 -05:00
Rod BoevandJerrettDavis b0c19a25f0 fix(security): reject unauthenticated public proxy binds (#3316)
## Description

Headroom's published image and maintained Docker launchers could expose
the
proxy on every host interface without a proxy token. This change moves
the
security boundary to the deployment artifacts: published images default
to a
container-loopback listener, supported local launchers publish only on
host
loopback, and deliberate public deployments retain explicit token
protection.
The application factory remains transport-neutral.

Closes #3283

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Set both published image stages to a container-loopback default while
keeping explicit host overrides available.
- Changed native Compose and the maintained Bash and PowerShell
launchers to use an internal `0.0.0.0` listener with host-loopback
publication.
- Preserved the top-level Compose token requirement and the existing
HTTP and WebSocket credential behavior for deliberate public binds.
- Scoped differential-capture token injection to the internal
`headroom-proxy` destination and kept sensitive headers redacted in
captured records.
- Added focused launch-contract, native-core, capture, and Docker
publication checks, plus a base/head harness that tolerates a base
already containing the fix.
- Updated Docker, installer, capture, and mirrored documentation,
including an unauthenticated local loopback quickstart and an
authenticated deliberate-public example.

## Testing

- [x] Focused unit and integration tests pass
- [x] Linting and formatting checks pass
- [ ] Type checking passes
- [x] New tests added for the changed deployment and capture behavior
- [ ] Manual testing performed

### Test Output

```text
uv run python -m pytest tests/test_proxy_hardening.py tests/test_install/test_native_installers.py tests/test_docker_compose_persistence.py tests/test_network_diff_capture.py tests/test_rust_core_smoke.py -q
76 passed, 1 skipped, 1 warning in 23.45s

uv run ruff check tests/test_install/test_native_installers.py tests/test_install/test_runtime.py
All checks passed!

bash -n e2e/docker-bind-security.sh
passed

bash e2e/docker-bind-security.sh --base-image headroom-t3283-base --head-image headroom-t3283-head --fixture tests/fixtures/docker-bind-exposure.json --base-ref origin/main
base_ref=origin/main ... runtime_slim_config=loopback rust_core=loaded local_completion=200 attacker_loopback=000 ws_status=403
docker bind security: fixture forwarding, loopback isolation, built-image health, slim-image loopback config, and token controls passed
```

## Real Behavior Proof

- Environment: Windows, Docker Desktop 29.6.2, current `origin/main`
base, head image, and a real Docker bridge peer.
- Exact command / steps: run the focused security/runtime tests, Compose
configuration checks, then `bash e2e/docker-bind-security.sh` against
the current base and head image definitions.
- Observed result: `76 passed, 1 skipped`; the built-image harness
passed base/head health, local completion, non-loopback isolation,
authenticated public HTTP/WebSocket controls, loopback runtime-slim
configuration, and direct Rust-core loading.
- Not tested: runtime-slim proxy service health on this host, because
its unrelated ONNXRuntime import path exits with code 139; the full
hosted matrix and a future published GHCR image remain CI-owned.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Unsafe override required: no
- Minimum rollout channel: next release containing the Docker and
launcher changes
- Stable/default behavior changed: published images and maintained local
Docker launchers now use loopback-safe publication defaults
- Kill switch / disable path: revert the deployment-artifact changes
before release; deliberate public deployments remain explicitly
operator-controlled
- Qualification impact: Docker users relying on all-interface default
publication must opt into public publication and configure the existing
proxy token
- Deliberate public deployment: use an explicit public host publication,
set `HEADROOM_PROXY_TOKEN`, and send the matching bearer or
`X-Headroom-Proxy-Token` credential
- Local deployment: use the maintained loopback publication and no
mandatory token
- Rollback path: revert the deployment-artifact and documentation
changes together

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove the behavior is effective
- [x] Focused tests pass locally with these changes
- [x] I did **not** edit `CHANGELOG.md`; release-please generates
release notes from the conventional PR title

## Additional Notes

The issue fixture remains provenance for the unauthenticated
publication. Native
`headroom proxy` defaults and the application-level authentication
owners are
unchanged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-24 23:58:46 -05:00
Tejas ChopraandTejas Chopra 481e0b83d5 fix(docker): publish compose ports on loopback only (#3061)
## Description

`docker compose up -d` published every service on `0.0.0.0`, and none of
the three authenticates an inbound caller by default:

| port | service | default auth |
|---|---|---|
| 8787 | proxy | `/v1/*` data plane open unless `HEADROOM_PROXY_TOKEN`
is set |
| 6333/6334 | Qdrant | **none at all** — holds embeddings derived from
prompts |
| 7474/7687 | Neo4j | `NEO4J_AUTH` falls back to `neo4j/devpassword`,
published in this file |

So the shipped default handed any peer on the surrounding network a
relay through the proxy plus direct read/write on the vector and graph
stores built from the operator's own prompt content. The proxy already
warns about exactly this shape at `headroom/proxy/server.py:3289` — the
compose file just never took its own advice.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Changes Made

- Pinned all five published ports to `127.0.0.1`.
- Documented in the file header how to expose the proxy deliberately,
pairing the port override with `HEADROOM_PROXY_TOKEN` rather than
leaving that implicit.
- Added a commented `HEADROOM_PROXY_TOKEN` entry to the proxy service
environment.
- Added a regression test asserting every published port names a
loopback host IP.

## Testing

- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes — N/A (YAML + test only)
- [x] New tests added for new functionality

### Test Output

```text
$ .venv/bin/python -m pytest tests/test_docker_compose_persistence.py -q
3 passed in 0.14s

$ docker compose -f docker-compose.yml config    # validates
headroom-proxy   host_ip=127.0.0.1  published=8787 -> 8787
neo4j            host_ip=127.0.0.1  published=7474 -> 7474
neo4j            host_ip=127.0.0.1  published=7687 -> 7687
qdrant           host_ip=127.0.0.1  published=6333 -> 6333
qdrant           host_ip=127.0.0.1  published=6334 -> 6334
```

Against the parent commit:

```text
FAILED test_top_level_compose_publishes_only_to_loopback
E   AssertionError: headroom-proxy: port '8787:8787' publishes on all interfaces
```

## Real Behavior Proof

- Environment: macOS 15 (darwin 25.4.0), Docker Compose v2 available
locally.
- Exact command / steps: `docker compose -f docker-compose.yml config
--format json` before and after, comparing the resolved `host_ip` on
every published port.
- Observed result: before, no port carried a `host_ip` (Docker binds
`0.0.0.0`); after, all five resolve to `host_ip=127.0.0.1`. The compose
file still validates.
- Not tested: bringing the stack up and probing the ports from a second
machine on the LAN — the assertion is made against Docker's own resolved
configuration rather than a live two-host network.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: yes — the compose stack is no longer
reachable from other machines by default.
- Kill switch / disable path: override `ports:` in a
`docker-compose.override.yml`; the header documents this and pairs it
with `HEADROOM_PROXY_TOKEN`.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation (the
compose header)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

**This is a deliberate breaking change for one workflow**: anyone
reaching the compose proxy from another machine will need to override
`ports:`. That is exactly the configuration that was unsafe, so it
should break loudly rather than silently. `http://localhost:8787` from
the host is unchanged, the container still listens on `0.0.0.0`
internally, and service-to-service traffic on the compose network is
unaffected.

Scope note: I fixed all three services rather than only the proxy.
Closing 8787 while leaving an unauthenticated Qdrant and a
default-password Neo4j published on `0.0.0.0` would not have improved
the security posture.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
2026-08-16 19:05:29 -07:00
Vinay Gupta 38074888ac fix(docker): report source build version (#1862)
## Description

Closes #1858

Docker/Compose source builds could report stale or misleading version
information: the dashboard initially rendered a hardcoded `v0.3.0`, then
`/health` replaced it with installed package metadata, which can be
stale when building locally from `main` without release metadata in the
image.

This change makes source Docker Compose builds report an explicit
source-build identity, removes the stale dashboard fallback, and keeps
CLI/doctor version checks from treating source-build labels as
release-version drift.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `HEADROOM_VERSION` / `HEADROOM_BUILD_VERSION` runtime version
overrides and optional packaged `_build_info.py` metadata.
- Teach Docker Compose source builds to pass a `source-build` sentinel
that the Dockerfile expands to `source-build+g<sha>` when git metadata
is available, or `source-build+sha256.<digest>` otherwise.
- Keep release/published image builds on normal package metadata when
`HEADROOM_BUILD_VERSION` is unset.
- Include only minimal `.git` metadata in the Docker build context so
the source-build label can identify the checkout without copying git
objects.
- Treat source-build labels and raw hashes as non-release labels in
`wrap` and `doctor`, avoiding false stale-proxy restarts and drift
warnings.
- Replace the dashboard hardcoded `0.3.0` fallback with `loading` /
`unknown` and format non-release build labels without a `v` prefix.
- Include the runtime version in proxy startup logs, `/health`,
`/livez`, and OTEL service version reporting.

## Testing

- [x] Unit tests pass (`pytest` in GitHub CI)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
GitHub CI: all checks passing
- CI: build, build-wheel, lint, test shards, test-extras, test-agno, test-dashboard-ui
- Docker: docker-native-e2e, docker-wrap-e2e, docker-init-e2e
- Native wrappers: macOS, Windows, Ubuntu
- Security: CodeQL, gitleaks, pip-audit
- Governance: template, label, merge-conflicts, commitlint

$ HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=/Users/vinaygupta/Desktop/git/headroom-fix-1858-version-mismatch pytest tests/test_package_init_lazy.py::test_version_prefers_explicit_build_env tests/test_package_init_lazy.py::test_version_label_helpers_only_prefix_release_versions tests/test_package_init_lazy.py::test_version_uses_packaged_build_metadata tests/test_package_init_lazy.py::test_observability_version_uses_runtime_version tests/test_docker_compose_persistence.py tests/test_cli_doctor.py::TestProxyLiveness::test_up_leaves_source_label_unprefixed tests/test_cli_doctor.py::TestVersionDrift::test_non_release_version_labels_skip_drift_comparison tests/test_cli/test_wrap_persistent.py::test_proxy_version_restart_ignores_non_release_source_labels tests/test_proxy_dashboard_stats_cache.py::test_dashboard_uses_cached_stats_and_lazy_history_feed_polling -q
13 passed, 1 warning

$ uvx ruff==0.15.17 check .
All checks passed!

$ uvx ruff==0.15.17 format --check .
1058 files already formatted

$ uvx mypy==1.20.2 headroom --ignore-missing-imports
Success: no issues found in 407 source files

$ git diff --check
# no output

$ docker compose config
# resolved headroom-proxy build args include HEADROOM_BUILD_VERSION: source-build

$ HEADROOM_BUILD_VERSION=6266a1d docker compose config
# explicit override is preserved as HEADROOM_BUILD_VERSION: 6266a1d

$ docker build --check --build-arg HEADROOM_BUILD_VERSION=source-build .
Check complete, no warnings found.
```

## Real Behavior Proof

- Environment: macOS local checkout, Python 3.13.5, Docker Desktop
builder `desktop-linux`, plus GitHub Actions CI.
- Exact command / steps: `docker compose config`,
`HEADROOM_BUILD_VERSION=6266a1d docker compose config`, and `docker
build --check --build-arg HEADROOM_BUILD_VERSION=source-build .`.
- Observed result: Compose defaults the top-level `headroom-proxy` build
arg to the `source-build` sentinel, preserves explicit overrides, and
Dockerfile syntax/check validation passes for the source-build path.
- Not tested: Full end-to-end release publishing flow; this PR only
changes local/source-build reporting.
- CI proof: GitHub Actions completed successfully across Docker E2E, CI
test shards, lint/type checks, native wrapper checks, security checks,
and PR governance.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally/CI with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

Docs and changelog are N/A for this runtime-reporting bug fix. The PR is
open and ready for review with all GitHub checks passing.
2026-07-08 13:32:04 -05:00
Vinay Gupta 5e29c06aaf fix(docker): persist headroom workspace in compose (#1839)
## Description

Pin the top-level Docker Compose proxy service to Headroom's canonical
writable workspace under the existing `headroom_workspace` named volume.

Closes #1835

The dashboard's durable savings/history data is loaded from
`proxy_savings.json` via `HEADROOM_WORKSPACE_DIR`; logs, session stats,
TOIN, config, and default workspace state are also derived from that
root. The top-level compose file already mounted
`/home/nonroot/.headroom`, but it relied on image/user home resolution
instead of exporting the canonical workspace env. This makes the
official compose contract explicit and matches the Docker-native
compose/runtime path behavior.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Set `HOME=/home/nonroot` for the top-level compose proxy service.
- Set `HEADROOM_WORKSPACE_DIR=/home/nonroot/.headroom` and
`HEADROOM_CONFIG_DIR=/home/nonroot/.headroom/config` so dashboard
savings/history, logs, config, memory state, session stats, and TOIN
resolve into the persisted named volume.
- Added a regression test that locks the top-level compose persistence
wiring.

## Testing

- [x] Unit tests pass (`pytest`) — focused local tests and full CI test
matrix passed
- [x] Linting passes (`ruff check .`) — local Ruff and CI lint passed
- [x] Type checking passes (`mypy headroom`) — local mypy and CI lint
passed
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ rtk pytest tests/test_docker_compose_persistence.py
Pytest: 1 passed

$ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py
Pytest: 76 passed

$ rtk uvx ruff check tests/test_docker_compose_persistence.py
All checks passed!

$ rtk docker compose config
services:
  headroom-proxy:
    environment:
      HEADROOM_CONFIG_DIR: /home/nonroot/.headroom/config
      HEADROOM_HOST: 0.0.0.0
      HEADROOM_WORKSPACE_DIR: /home/nonroot/.headroom
      HOME: /home/nonroot
    volumes:
      - type: volume
        source: headroom_workspace
        target: /home/nonroot/.headroom
```

Attempted broader proxy stats-history coverage, but this local checkout
does not have the native extension built:

```text
$ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py tests/test_proxy_savings_history.py::test_stats_history_persists_across_restarts_and_stats_stays_compatible
ModuleNotFoundError: No module named 'headroom._core'
```

Attempted project-managed Ruff, but `uv run` tried to build the editable
package first and hit the known local native build issue before Ruff
could execute:

```text
$ rtk uv run ruff check tests/test_docker_compose_persistence.py
error: failed to run custom build command for `esaxx-rs v0.1.10`
fatal error: 'cstdint' file not found
```

## Real Behavior Proof

- Environment: local clean clone at current upstream `main`, branch
`fix/1835-docker-compose-persistence`.
- Exact command / steps: `rtk docker compose config` from the repo root.
- Observed result: Compose renders `HOME`, `HEADROOM_WORKSPACE_DIR`, and
`HEADROOM_CONFIG_DIR` under `/home/nonroot/.headroom`, and the
`headroom_workspace` named volume targets that same path.
- Not tested: full Docker image build or live `docker compose up`
restart cycle; full pytest/mypy not run locally because this checkout
lacks the built `headroom._core` extension.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- All non-skipped GitHub Actions checks are green after the rebase onto
`main`; skipped jobs are path-gated.
- The dashboard's recent request table is still an in-memory tail and is
expected to be empty after a proxy restart. This PR targets durable
dashboard savings/history and other workspace-backed files.
- `HEADROOM_LOG_FILE=/home/nonroot/.headroom/requests.jsonl` remains an
optional operator setting; persisted request JSONL is not replayed into
the dashboard after restart.
- The docs/CHANGELOG checklist items are N/A for this narrow compose
configuration fix.
2026-07-06 08:35:06 -07:00