mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Why Four related gaps in the proxy's network gate (findings 01-F1, 01-F2, 01-F5, 01-F9): - **Open bind.** A non-loopback bind with no `HEADROOM_PROXY_TOKEN` only logged a warning, then served the `/v1/*` relay, with the operator's upstream credentials, to every peer that could reach the port. - **Missing peer address.** `is_loopback_host(None)` returned `True`. A Unix-domain socket, an ASGI adapter that leaves `scope["client"]` unset, or a hand-built request therefore turned off the token gate, the `/admin` and `/debug` guards and the WebSocket gate. - **Operator data routes.** `/stats-history` (including CSV export), `/quota`, `/subscription-window` and `/metrics` were served to any network caller. - **WebSocket loopback exemption** checked only the peer IP, not the `Host` header, so it had no DNS-rebinding defence. ## What changes - New `headroom/proxy/bind_policy.py`. A non-loopback bind (`0.0.0.0`, `::`, a LAN address or a hostname) with no token is refused at three places: `create_app` (embedders and the multi-worker factory), `run_server` (one exit with code 2 before uvicorn forks) and the `headroom proxy` CLI (a clear error before the banner, exit code 1). An absent host means uvicorn's default `127.0.0.1`. - `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` acknowledges an open bind when the runtime already restricts reachability, such as a container published on `127.0.0.1`. The `proxy_open_bind` warning is still logged on that path, and the banner says the bind is open. - `is_loopback_host(None)` now returns `False` (fail closed), matching `identity.py`. - `/stats-history`, `/quota` and `/subscription-window` answer only: a non-loopback caller that passed the token gate, a loopback caller (peer and `Host` header both loopback), a trusted dashboard client, or, under an acknowledged token-less bind, the container's own default-gateway TCP peer with a loopback `Host` (the host browser reaching a `127.0.0.1`-published container). Everyone else gets 404, as with `/admin`. Settings writes keep their stricter rule with no token short-cut. - `/metrics` answers a non-loopback caller that passed the token gate, a loopback caller, a connecting peer in `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` (whatever it forwards), or a resolved client in the dashboard CIDRs. For CIDR peers a hostname `Host` is fine, so Prometheus scrapers work; a cross-origin browser request is still refused. - The token short-cut applies only to non-loopback requests. The gate exempts loopback peers without checking `Host`, so a loopback request has not proven it holds the token and must pass the loopback `Host` check. This blocks a DNS-rebound browser on the operator's machine. - The `/dashboard` shell stays ungated. It is a static template, like the already public `/dashboard/static` assets. Gating it would break the dashboard for containers published on host loopback, where the peer is the bridge gateway and `Host` is `localhost`. The data routes behind it are gated. - The WebSocket loopback exemption requires a loopback peer and a loopback `Host` header. A loopback peer with a foreign `Host` can still connect with the token. - Launchers that always publish on `127.0.0.1` set the acknowledgement themselves: `headroom install` (docker preset), `scripts/install.sh` and `scripts/install.ps1` (through one helper that adds the acknowledgement only together with the `127.0.0.1` publication), and `docker/docker-compose.native.yml`. - Docs: `proxy.mdx` documents the new variable and which routes the token also gates; `docker-install` (docs and wiki) and the `docker-compose.yml` comment show the acknowledgement in the loopback-published example. `e2e/docker-bind-security.sh` sets it and checks that the same launch without it refuses to start. ## Behaviour change (breaking) - `headroom proxy --host 0.0.0.0`, or `HEADROOM_HOST` set to any non-loopback address, without `HEADROOM_PROXY_TOKEN` now refuses to start. The error names both remedies. Set a token, bind loopback, or set `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` when the port is already confined. Anyone running the image by hand with `--host 0.0.0.0` and no token must add one of these. - Without a token, `/stats-history`, `/quota`, `/subscription-window` and `/metrics` return 404 to non-loopback callers outside a trusted CIDR. Prometheus scrapers on another host must be in `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` or send the token. - Code that relied on a request with no peer address being treated as loopback now gets the non-loopback path. - The default loopback run is unchanged. ## Tests - `tests/test_proxy_bind_policy.py`: the refuse/allow matrix (non-loopback, loopback, unset host, token, acknowledgement values), the warning on the acknowledged path, `create_app`, the `run_server` exit before uvicorn, and the CLI (refusal, `HEADROOM_HOST` from env, acknowledged start, offline banner, start with a token). - `tests/test_proxy_loopback_gating.py`: each data route against non-loopback, loopback, trusted-CIDR and token callers; the dashboard shell stays reachable; the host of an acknowledged, loopback-published container reads the data routes while another bridge peer, a forwarded gateway address and a foreign `Host` do not; CSV export is not served to network callers; `/metrics` across loopback, both CIDR lists, out-of-range peers, cross-origin browsers and token callers, including a trusted gateway relaying a scraper outside its own range. A regression test shows that a token on the proxy does not exempt a loopback peer from the `Host` check; it fails without the fix. - `tests/test_proxy_hardening.py`: the HTTP gate with no peer address, WebSocket `Host` cases, `require_loopback` and `/debug` with no peer. - Installer tests: `headroom install` adds the acknowledgement, and the shell and PowerShell wrappers add it only together with the loopback publication. Launcher-level tests start the proxy with the environment and `--host` that `build_runtime_command()` and `docker-compose.native.yml` produce and check that the host browser (bridge-gateway peer) gets `/stats-history` and another container does not. - Tests that bound `0.0.0.0` for unrelated reasons now set a token or the acknowledgement. Tests that read operator routes use an explicit loopback peer. On `main`, 14 of the gating and hardening tests fail and the bind-policy module does not exist. Manual runs: token-less `--host 0.0.0.0` exits with the refusal from both the CLI and `run_server`; with the acknowledgement, the data routes are 404 to a LAN peer and to a loopback peer with a foreign `Host`; with a token, a LAN peer gets 401 without it and 200 with it; the default loopback bind starts as before. Not tested: a real Unix-domain-socket deployment and a real Prometheus scrape (both covered by tests only). ## Merge order Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 → #3863. This PR goes first: it is the gate the others assume, and it is the release-note event. ## Not in this PR - Enforcing the token for loopback callers when one is configured. Revised 2026-10-02 after review: `/metrics` matches gateway CIDRs against the TCP peer, and the operator data routes serve the host of a loopback-published container (bridge-gateway peer under the launchers' bind acknowledgement) so the default Docker dashboard works. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3852" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
258 lines
10 KiB
Python
258 lines
10 KiB
Python
"""Bind policy: a non-loopback bind with no inbound token is refused, not warned about.
|
|
|
|
Before this change the proxy logged ``proxy_open_bind`` and started anyway,
|
|
so any launcher that set ``--host 0.0.0.0`` / ``HEADROOM_HOST=0.0.0.0`` without
|
|
``HEADROOM_PROXY_TOKEN`` served the ``/v1/*`` relay to every peer on the
|
|
network. The policy is evaluated in :mod:`headroom.proxy.bind_policy` and
|
|
enforced at three layers -- ``create_app``, ``run_server`` and the CLI -- each
|
|
of which is covered here, plus the explicit operator acknowledgement
|
|
(``HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1``) that keeps the documented
|
|
"container binds 0.0.0.0, runtime publishes on 127.0.0.1" shape working.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from unittest.mock import patch
|
|
|
|
import pytest
|
|
|
|
pytest.importorskip("fastapi")
|
|
|
|
from click.testing import CliRunner # noqa: E402
|
|
|
|
from headroom.cli.main import main # noqa: E402
|
|
from headroom.exceptions import ConfigurationError # noqa: E402
|
|
from headroom.proxy.bind_policy import ( # noqa: E402
|
|
OPEN_BIND_ACK_ENV,
|
|
PROXY_TOKEN_ENV,
|
|
OpenBindRefused,
|
|
enforce_bind_policy,
|
|
evaluate_bind_policy,
|
|
)
|
|
from headroom.proxy.models import ProxyConfig # noqa: E402
|
|
from headroom.proxy.server import create_app, run_server # noqa: E402
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _clean_bind_env(monkeypatch: pytest.MonkeyPatch):
|
|
"""The policy reads the process env; never let the developer's shell leak in."""
|
|
monkeypatch.delenv(PROXY_TOKEN_ENV, raising=False)
|
|
monkeypatch.delenv(OPEN_BIND_ACK_ENV, raising=False)
|
|
|
|
|
|
# ───────────────────────────── evaluate ────────────────────────────────────
|
|
|
|
|
|
@pytest.mark.parametrize("host", ["0.0.0.0", "::", "192.168.1.20", "myhost.internal", "[::]"])
|
|
def test_non_loopback_bind_without_token_is_refused(host: str) -> None:
|
|
decision = evaluate_bind_policy(host, None, environ={})
|
|
assert decision.open_bind is True
|
|
assert decision.refused is True
|
|
# The message must name both remedies so the operator can act on it.
|
|
assert PROXY_TOKEN_ENV in decision.message()
|
|
assert OPEN_BIND_ACK_ENV in decision.message()
|
|
|
|
|
|
@pytest.mark.parametrize("host", ["127.0.0.1", "localhost", "::1", "127.5.5.5", "::ffff:127.0.0.1"])
|
|
def test_loopback_bind_without_token_is_fine(host: str) -> None:
|
|
decision = evaluate_bind_policy(host, None, environ={})
|
|
assert decision.loopback is True
|
|
assert decision.open_bind is False
|
|
assert decision.refused is False
|
|
|
|
|
|
@pytest.mark.parametrize("host", [None, "", " "])
|
|
def test_unset_host_means_uvicorn_default_loopback(host: str | None) -> None:
|
|
"""An absent host is uvicorn's 127.0.0.1, never 'unknown therefore public'."""
|
|
decision = evaluate_bind_policy(host, None, environ={})
|
|
assert decision.host == "127.0.0.1"
|
|
assert decision.refused is False
|
|
|
|
|
|
def test_token_makes_any_bind_acceptable() -> None:
|
|
assert evaluate_bind_policy("0.0.0.0", "s3cr3t", environ={}).refused is False
|
|
# Same rule the security gate uses: the env var is the fallback source.
|
|
assert (
|
|
evaluate_bind_policy("0.0.0.0", None, environ={PROXY_TOKEN_ENV: "s3cr3t"}).refused is False
|
|
)
|
|
# Whitespace-only is not a token.
|
|
assert evaluate_bind_policy("0.0.0.0", " ", environ={}).refused is True
|
|
|
|
|
|
@pytest.mark.parametrize("value", ["1", "true", "TRUE", "yes", "on", " On "])
|
|
def test_acknowledgement_env_allows_the_open_bind(value: str) -> None:
|
|
decision = evaluate_bind_policy("0.0.0.0", None, environ={OPEN_BIND_ACK_ENV: value})
|
|
assert decision.open_bind is True
|
|
assert decision.acknowledged is True
|
|
assert decision.refused is False
|
|
|
|
|
|
@pytest.mark.parametrize("value", ["0", "false", "no", "", "maybe"])
|
|
def test_non_truthy_acknowledgement_still_refuses(value: str) -> None:
|
|
assert evaluate_bind_policy("0.0.0.0", None, environ={OPEN_BIND_ACK_ENV: value}).refused
|
|
|
|
|
|
# ───────────────────────────── enforce ─────────────────────────────────────
|
|
|
|
|
|
def test_enforce_raises_a_configuration_error_subclass() -> None:
|
|
with pytest.raises(OpenBindRefused) as exc_info:
|
|
enforce_bind_policy("0.0.0.0", None, environ={})
|
|
assert isinstance(exc_info.value, ConfigurationError)
|
|
assert exc_info.value.decision.host == "0.0.0.0"
|
|
|
|
|
|
def test_enforce_keeps_the_open_bind_warning_when_acknowledged() -> None:
|
|
"""Log-based monitoring keyed on ``proxy_open_bind`` must keep working."""
|
|
records: list[logging.LogRecord] = []
|
|
handler = logging.Handler()
|
|
handler.emit = records.append # type: ignore[method-assign]
|
|
log = logging.getLogger("headroom.proxy.bind_policy")
|
|
log.addHandler(handler)
|
|
previous = log.level
|
|
log.setLevel(logging.DEBUG)
|
|
try:
|
|
decision = enforce_bind_policy("0.0.0.0", None, environ={OPEN_BIND_ACK_ENV: "1"})
|
|
finally:
|
|
log.removeHandler(handler)
|
|
log.setLevel(previous)
|
|
assert decision.acknowledged is True
|
|
messages = [r.getMessage() for r in records]
|
|
assert any("event=proxy_open_bind " in m for m in messages), messages
|
|
assert not any("proxy_open_bind_refused" in m for m in messages)
|
|
|
|
|
|
# ───────────────────────────── create_app ──────────────────────────────────
|
|
|
|
|
|
def _config(**overrides) -> ProxyConfig:
|
|
return ProxyConfig(
|
|
optimize=False,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
**overrides,
|
|
)
|
|
|
|
|
|
def test_create_app_refuses_open_bind() -> None:
|
|
with pytest.raises(OpenBindRefused):
|
|
create_app(_config(host="0.0.0.0", proxy_token=None))
|
|
|
|
|
|
def test_create_app_accepts_open_bind_with_token() -> None:
|
|
assert create_app(_config(host="0.0.0.0", proxy_token="s3cr3t")) is not None
|
|
|
|
|
|
def test_create_app_accepts_acknowledged_open_bind(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
monkeypatch.setenv(OPEN_BIND_ACK_ENV, "1")
|
|
assert create_app(_config(host="0.0.0.0", proxy_token=None)) is not None
|
|
|
|
|
|
def test_create_app_default_loopback_bind_needs_nothing() -> None:
|
|
assert create_app(_config()) is not None
|
|
|
|
|
|
# ───────────────────────────── run_server ──────────────────────────────────
|
|
|
|
|
|
def test_run_server_exits_before_uvicorn_on_open_bind(capsys: pytest.CaptureFixture[str]) -> None:
|
|
"""One clean exit code, not N workers dying in create_app."""
|
|
called = []
|
|
with (
|
|
patch("headroom.proxy.server.uvicorn.run", lambda *a, **k: called.append(1)),
|
|
pytest.raises(SystemExit) as exc_info,
|
|
):
|
|
run_server(_config(host="0.0.0.0", proxy_token=None), print_banner=False)
|
|
assert exc_info.value.code == 2
|
|
assert called == []
|
|
err = capsys.readouterr().err
|
|
assert "Refusing to bind '0.0.0.0'" in err
|
|
assert PROXY_TOKEN_ENV in err and OPEN_BIND_ACK_ENV in err
|
|
|
|
|
|
def test_run_server_proceeds_with_token() -> None:
|
|
called = []
|
|
with patch("headroom.proxy.server.uvicorn.run", lambda *a, **k: called.append(k)):
|
|
run_server(_config(host="0.0.0.0", proxy_token="s3cr3t"), print_banner=False)
|
|
assert len(called) == 1
|
|
assert called[0]["host"] == "0.0.0.0"
|
|
|
|
|
|
# ───────────────────────────── CLI ─────────────────────────────────────────
|
|
|
|
|
|
def test_cli_refuses_open_bind_with_actionable_error() -> None:
|
|
runner = CliRunner()
|
|
called = []
|
|
with patch("headroom.proxy.server.run_server", lambda config, **kw: called.append(config)):
|
|
result = runner.invoke(main, ["proxy", "--host", "0.0.0.0"])
|
|
assert result.exit_code != 0, result.output
|
|
assert called == []
|
|
assert "Refusing to bind '0.0.0.0'" in result.output
|
|
assert PROXY_TOKEN_ENV in result.output
|
|
assert OPEN_BIND_ACK_ENV in result.output
|
|
# A refusal is an error message, never a traceback.
|
|
assert "Traceback" not in result.output
|
|
|
|
|
|
def test_cli_env_host_is_refused_too() -> None:
|
|
"""``HEADROOM_HOST=0.0.0.0`` in a systemd unit is the common real-world shape."""
|
|
runner = CliRunner()
|
|
with patch("headroom.proxy.server.run_server", lambda config, **kw: None):
|
|
result = runner.invoke(main, ["proxy"], env={"HEADROOM_HOST": "0.0.0.0"})
|
|
assert result.exit_code != 0, result.output
|
|
assert "Refusing to bind '0.0.0.0'" in result.output
|
|
|
|
|
|
def test_cli_acknowledged_open_bind_starts_and_says_so() -> None:
|
|
runner = CliRunner()
|
|
captured = {}
|
|
with patch(
|
|
"headroom.proxy.server.run_server",
|
|
lambda config, **kw: captured.setdefault("config", config),
|
|
):
|
|
result = runner.invoke(
|
|
main,
|
|
["proxy", "--host", "0.0.0.0"],
|
|
env={OPEN_BIND_ACK_ENV: "1"},
|
|
catch_exceptions=False,
|
|
)
|
|
assert result.exit_code == 0, result.output
|
|
assert captured["config"].host == "0.0.0.0"
|
|
assert "UNAUTHENTICATED" in result.output
|
|
assert OPEN_BIND_ACK_ENV in result.output
|
|
|
|
|
|
def test_cli_offline_banner_still_flags_acknowledged_open_bind() -> None:
|
|
"""Offline mode must not hide that /v1/* is unauthenticated."""
|
|
runner = CliRunner()
|
|
with patch("headroom.proxy.server.run_server", lambda config, **kw: None):
|
|
result = runner.invoke(
|
|
main,
|
|
["proxy", "--host", "0.0.0.0"],
|
|
env={OPEN_BIND_ACK_ENV: "1", "HEADROOM_OFFLINE": "1"},
|
|
catch_exceptions=False,
|
|
)
|
|
assert result.exit_code == 0, result.output
|
|
assert "OFFLINE" in result.output
|
|
assert "UNAUTHENTICATED" in result.output
|
|
|
|
|
|
def test_cli_open_bind_with_token_starts() -> None:
|
|
runner = CliRunner()
|
|
captured = {}
|
|
with patch(
|
|
"headroom.proxy.server.run_server",
|
|
lambda config, **kw: captured.setdefault("config", config),
|
|
):
|
|
result = runner.invoke(
|
|
main,
|
|
["proxy", "--host", "0.0.0.0"],
|
|
env={PROXY_TOKEN_ENV: "s3cr3t"},
|
|
catch_exceptions=False,
|
|
)
|
|
assert result.exit_code == 0, result.output
|
|
assert "inbound token REQUIRED" in result.output
|