mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(install): distinguish protocol targets in status (#4040)
## Description Closes #2615. `headroom install status` labelled the default backend as the generic `Backend`, even when an OpenAI target was configured independently. This made a healthy OpenAI-only deployment look like an Anthropic-only deployment. ## Type of Change - [x] Bug fix - [x] Documentation update ## Changes Made - Label the fallback/backend choice as `Default backend`. - Show `Anthropic target` and `OpenAI target` separately as `configured` or `default`, based on the existing live loopback health configuration. - Do not print upstream target URLs or their embedded credentials. - Preserve the existing non-dict health-config guard and all lifecycle/routing behavior. - Document the distinction in the existing persistent-installs page. - Cover configured/default/unknown targets, missing/non-dict/partial/malformed health configuration, manifest fallback, credential withholding, and actual HTTP status probes. ## Testing - Complete install/CLI owner suites: **296 passed, 5 skipped**, one existing Starlette/httpx deprecation warning. - Ruff lint and format checks for both changed Python files: passed. - `mypy headroom/cli/install.py`: passed, one source file checked. - Installed commit hooks: passed (including whole-project mirrored mypy and commit lint). - Required `make ci-precheck`, run by the actual pre-push hook: passed — Rust fmt, strict Clippy, full workspace tests, native extension build/import, **177 Python passed / 4 skipped** (one existing provider-counting warning), and commitlint. - First pre-push attempt hit the unchanged native `first_dispatch_never_blocks_on_model_init` 500 ms bound at 592.8 ms. The isolated test and then the complete unchanged gate passed. No threshold, test, native source, or gate was modified or skipped. - Independent installed-LangChain whole-project typing failure at `chat_model.py:394` is tracked in #4039; this changed CLI file's scoped check passes. No claim that the CLI change fixes that separate error. - Hosted Codecov's automated advisory reports 0% patch coverage for the three display lines. The real CLI before/after runs above exercise them; no wording-only mock test was added to manufacture coverage. At the observed SHA, there is no failing Codecov check; remaining hosted jobs were still in progress. ## Real Behavior Proof Linux x86_64, Python 3.13.14; base `3c418f6f0b190ad1060413c0eadd77f20ab3cf46`. Created a real manifest with the production planner and atomic persistence API, then ran the actual `headroom install agent run` and `headroom install status` CLI. HOME, canonical workspace/config roots, profiles, logs, and ports were isolated under the project verification directory. No system supervisor or provider configuration was installed or modified. Compression/caching were disabled through supported raw proxy flags because this is status-display proof, not compression proof; the anonymous beacon was disabled. Before, live `/health` reports `backend=anthropic`, a configured OpenAI URL, and no configured Anthropic URL; status is running/healthy but ends with `Backend: anthropic` only. After, the same configuration reports: ```text Default backend: anthropic Anthropic target: default OpenAI target: configured ``` A second actual run with both protocol URLs configured reports both targets as configured. The intentionally unreachable Anthropic test endpoint reports `Healthy: no` there; configured is not misrepresented as reachable/healthy. A fake secret embedded in each verification URL does not appear in status output. Agents were explicitly terminated after each observation; no graceful-exit claim. Not tested: paid upstream requests, a running compression workload, macOS, Windows, or a real platform supervisor installation. ## Runtime Rollout Safety No channel, feature flag, routing, auth, or persisted-data format changes. Only operator display changes. Revert this commit to restore the old display; deployment profiles remain valid. ## Review Readiness - [x] Changed consumers and existing health-config contract reviewed - [x] Actual before/after CLI surface verified ## Checklist - [x] Existing documentation updated - [x] No warnings introduced in the changed Python files - [x] No CHANGELOG edit (release-please owns it) AI-assisted: omp / GPT-6.1 Sol, without subagents. No hosted-CI claim. ## Follow-up qualification at2969351e3dWhen the health response withholds configuration or omits a target field, status now reports `unknown` instead of asserting the default. Malformed target values also remain unknown. Explicit null/empty URL fields report default; configured string URLs report configured without exposing their contents. Reproduce the committed HTTP regressions from a configured repository Python environment: ```sh python -m pytest tests/test_cli/test_install_cli.py -k status -q ``` For the actual proxy/CLI replay, save the complete script in [the maintainer evidence comment](https://github.com/headroomlabs-ai/headroom/pull/4040#issuecomment-6062359335) as `verify_status.py` in the repository root, then run `python verify_status.py`. It creates temporary production manifests and an actual uvicorn proxy, verifies direct health configuration and configuration withheld by the real health guard through a relay, and checks that status never prints URLs/credentials. The same script fails on the original39e7d46fahead and passes on this head. This is not a real Docker networking or OS-supervisor installation claim. Independent Windows/Python 3.12 verification: 319 install/CLI tests pass, 2 platform skips, one existing warning; Ruff lint/format and Linux-platform mypy over 566 files pass. The complete suite also reproduced one unrelated Windows planner test mismatch: it expects persistent-service defaults although unchanged Windows planning intentionally converts that preset to persistent-task (#2552). That test was deselected in the follow-up suite; no planner source/test changed. Exact setup, limitations, before/after output assertions, and script are in the linked evidence.
This commit is contained in:
@@ -90,6 +90,8 @@ headroom install remove
|
||||
|
||||
`apply` creates or updates a named deployment profile, stores its manifest under `~/.headroom/deploy/<profile>/manifest.json`, applies reversible configuration changes, and starts the selected runtime.
|
||||
|
||||
`status` distinguishes the default backend from the Anthropic and OpenAI protocol targets. When the health response includes a target's configuration, that target is shown as `configured` or `default`. If configuration is unavailable or omits a target, that target is shown as `unknown`; this does not mean it is unset. For example, a Docker bridge peer may not pass the loopback health-config gate. Target URLs and credentials are not printed. An OpenAI-compatible deployment can therefore have `Default backend: anthropic` and `OpenAI target: configured` at the same time.
|
||||
|
||||
## Presets and runtime kinds
|
||||
|
||||
### Presets
|
||||
|
||||
+16
-2
@@ -1178,9 +1178,23 @@ def install_status(profile: str) -> None:
|
||||
# reach `.get('backend', ...)` and crash with AttributeError. Guard on
|
||||
# isinstance, mirroring wrap.py's _proxy_health_config.
|
||||
config = payload.get("config")
|
||||
if not isinstance(config, dict):
|
||||
anthropic_target = openai_target = "unknown"
|
||||
if isinstance(config, dict):
|
||||
anthropic_url = config.get("anthropic_api_url")
|
||||
if "anthropic_api_url" in config and (
|
||||
anthropic_url is None or isinstance(anthropic_url, str)
|
||||
):
|
||||
anthropic_target = "configured" if anthropic_url else "default"
|
||||
openai_url = config.get("openai_api_url")
|
||||
if "openai_api_url" in config and (openai_url is None or isinstance(openai_url, str)):
|
||||
openai_target = "configured" if openai_url else "default"
|
||||
else:
|
||||
# Network health probes may intentionally omit credential-bearing
|
||||
# config. Missing information does not mean a target is unset.
|
||||
config = {}
|
||||
click.echo(f"Backend: {config.get('backend', manifest.backend)}")
|
||||
click.echo(f"Default backend: {config.get('backend', manifest.backend)}")
|
||||
click.echo(f"Anthropic target: {anthropic_target}")
|
||||
click.echo(f"OpenAI target: {openai_target}")
|
||||
|
||||
|
||||
@install.command("start")
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from threading import Thread
|
||||
from types import SimpleNamespace
|
||||
|
||||
import click
|
||||
@@ -13,6 +15,7 @@ from headroom.cli.main import main
|
||||
from headroom.install.models import DeploymentManifest, ManagedMutation
|
||||
from headroom.install.planner import build_tool_envs
|
||||
from headroom.install.state import load_manifest as load_state_manifest
|
||||
from headroom.install.state import save_manifest_strict
|
||||
|
||||
|
||||
def test_require_manifest_resolves_single_profile_when_default_missing(monkeypatch):
|
||||
@@ -449,67 +452,165 @@ def test_install_apply_explicit_env_overrides_captured(monkeypatch) -> None:
|
||||
assert captured["extra_env"]["ANTHROPIC_TARGET_API_URL"] == "https://explicit.internal/v1"
|
||||
|
||||
|
||||
def test_install_status_includes_backend_from_health_probe(monkeypatch) -> None:
|
||||
runner = CliRunner()
|
||||
|
||||
class Manifest:
|
||||
targets = ["claude"]
|
||||
tool_envs = {}
|
||||
profile = "default"
|
||||
preset = "persistent-service"
|
||||
runtime_kind = "python"
|
||||
supervisor_kind = "service"
|
||||
scope = "user"
|
||||
port = 8787
|
||||
backend = "anthropic"
|
||||
health_url = "http://127.0.0.1:8787/readyz"
|
||||
|
||||
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
|
||||
@pytest.mark.parametrize(
|
||||
("anthropic_url", "openai_url", "anthropic_target", "openai_target"),
|
||||
[
|
||||
(None, None, "default", "default"),
|
||||
("", "", "default", "default"),
|
||||
("https://anthropic-user:fake-secret@anthropic.invalid/v1", None, "configured", "default"),
|
||||
(None, "https://openai-user:fake-secret@openai.invalid/v1", "default", "configured"),
|
||||
(
|
||||
"https://anthropic-user:fake-secret@anthropic.invalid/v1",
|
||||
"https://openai-user:fake-secret@openai.invalid/v1",
|
||||
"configured",
|
||||
"configured",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_install_status_includes_protocol_targets_from_health_probe(
|
||||
monkeypatch, anthropic_url, openai_url, anthropic_target, openai_target
|
||||
) -> None:
|
||||
monkeypatch.setattr(inst, "load_manifest", lambda profile: _status_manifest(profile))
|
||||
monkeypatch.setattr(inst, "runtime_status", lambda manifest: "running")
|
||||
monkeypatch.setattr(inst, "probe_ready", lambda url: True)
|
||||
monkeypatch.setattr(
|
||||
"headroom.cli.install.runtime_status",
|
||||
lambda manifest: "running",
|
||||
)
|
||||
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: True)
|
||||
monkeypatch.setattr(
|
||||
"headroom.cli.install.probe_json",
|
||||
lambda url: {"config": {"backend": "anthropic"}},
|
||||
inst,
|
||||
"probe_json",
|
||||
lambda url: {
|
||||
"config": {
|
||||
"backend": "openai",
|
||||
"anthropic_api_url": anthropic_url,
|
||||
"openai_api_url": openai_url,
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
result = runner.invoke(main, ["install", "status"])
|
||||
result = CliRunner().invoke(main, ["install", "status"])
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "Status: running" in result.output
|
||||
assert "Healthy: yes" in result.output
|
||||
assert "Backend: anthropic" in result.output
|
||||
assert "Default backend: openai" in result.output
|
||||
assert f"Anthropic target: {anthropic_target}" in result.output
|
||||
assert f"OpenAI target: {openai_target}" in result.output
|
||||
for secret in ("fake-secret", "anthropic-user", "openai-user", ".invalid"):
|
||||
assert secret not in result.output
|
||||
|
||||
|
||||
def test_install_status_survives_non_dict_config(monkeypatch) -> None:
|
||||
"""A health payload whose `config` is a non-dict (e.g. a different service
|
||||
answering on the port returns config: null) must not crash the command."""
|
||||
runner = CliRunner()
|
||||
@pytest.mark.parametrize(
|
||||
"payload",
|
||||
[
|
||||
{"status": "healthy"},
|
||||
{"config": {}},
|
||||
{"config": {"backend": "anthropic"}},
|
||||
{"config": None},
|
||||
{"config": "not-a-config"},
|
||||
{"config": []},
|
||||
{"config": 42},
|
||||
{"config": {"anthropic_api_url": False, "openai_api_url": 0}},
|
||||
{"config": {"anthropic_api_url": [], "openai_api_url": {}}},
|
||||
{"config": {"anthropic_api_url": ["malformed"], "openai_api_url": {"url": "bad"}}},
|
||||
],
|
||||
)
|
||||
def test_install_status_survives_unavailable_config(monkeypatch, payload) -> None:
|
||||
"""Redacted or malformed config is unknown, not evidence of unset URLs."""
|
||||
monkeypatch.setattr(inst, "load_manifest", lambda profile: _status_manifest(profile))
|
||||
monkeypatch.setattr(inst, "runtime_status", lambda manifest: "running")
|
||||
monkeypatch.setattr(inst, "probe_ready", lambda url: True)
|
||||
monkeypatch.setattr(inst, "probe_json", lambda url: payload)
|
||||
|
||||
class Manifest:
|
||||
targets = ["claude"]
|
||||
tool_envs = {}
|
||||
profile = "default"
|
||||
preset = "persistent-service"
|
||||
runtime_kind = "python"
|
||||
supervisor_kind = "service"
|
||||
scope = "user"
|
||||
port = 8787
|
||||
backend = "anthropic"
|
||||
health_url = "http://127.0.0.1:8787/readyz"
|
||||
result = CliRunner().invoke(main, ["install", "status"])
|
||||
|
||||
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
|
||||
monkeypatch.setattr("headroom.cli.install.runtime_status", lambda manifest: "running")
|
||||
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: True)
|
||||
monkeypatch.setattr("headroom.cli.install.probe_json", lambda url: {"config": None})
|
||||
|
||||
result = runner.invoke(main, ["install", "status"])
|
||||
|
||||
# No AttributeError; Backend falls back to the manifest value.
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "Backend: anthropic" in result.output
|
||||
assert "Default backend: anthropic" in result.output
|
||||
assert "Anthropic target: unknown" in result.output
|
||||
assert "OpenAI target: unknown" in result.output
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("payload", "anthropic_target", "openai_target"),
|
||||
[
|
||||
({"status": "healthy", "ready": True}, "unknown", "unknown"),
|
||||
({"config": {}}, "unknown", "unknown"),
|
||||
(
|
||||
{"config": {"anthropic_api_url": None, "openai_api_url": None}},
|
||||
"default",
|
||||
"default",
|
||||
),
|
||||
(
|
||||
{"config": {"openai_api_url": "https://user:fake-secret@openai.invalid/v1"}},
|
||||
"unknown",
|
||||
"configured",
|
||||
),
|
||||
({"config": {"anthropic_api_url": ""}}, "default", "unknown"),
|
||||
({"config": {"anthropic_api_url": False, "openai_api_url": None}}, "unknown", "default"),
|
||||
],
|
||||
)
|
||||
def test_install_status_probes_live_health_response(
|
||||
tmp_path, monkeypatch, payload, anthropic_target, openai_target
|
||||
) -> None:
|
||||
"""Exercise manifest persistence, HTTP probes and CLI rendering together.
|
||||
|
||||
The redacted response is the shape returned to an untrusted Docker bridge
|
||||
peer. This fixture is an HTTP server, not a Docker networking test.
|
||||
"""
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path / "workspace"))
|
||||
monkeypatch.setenv("HEADROOM_CONFIG_DIR", str(tmp_path / "config"))
|
||||
monkeypatch.delenv("HEADROOM_DEPLOYMENT_PROFILE", raising=False)
|
||||
requests = []
|
||||
|
||||
class HealthHandler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
requests.append(self.path)
|
||||
body = json.dumps({"ready": True} if self.path == "/readyz" else payload).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, format, *args):
|
||||
pass
|
||||
|
||||
with ThreadingHTTPServer(("127.0.0.1", 0), HealthHandler) as server:
|
||||
thread = Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
try:
|
||||
port = server.server_address[1]
|
||||
save_manifest_strict(
|
||||
DeploymentManifest(
|
||||
profile="status-proof",
|
||||
preset="persistent-task",
|
||||
runtime_kind="python",
|
||||
supervisor_kind="none",
|
||||
scope="user",
|
||||
provider_mode="manual",
|
||||
targets=[],
|
||||
port=port,
|
||||
host="127.0.0.1",
|
||||
backend="anthropic",
|
||||
health_url=f"http://127.0.0.1:{port}/readyz",
|
||||
base_env={
|
||||
"OPENAI_TARGET_API_URL": "https://stale:fake-secret@stale.invalid/v1"
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
result = CliRunner().invoke(main, ["install", "status", "--profile", "status-proof"])
|
||||
finally:
|
||||
server.shutdown()
|
||||
thread.join(timeout=5)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert requests == ["/health", "/readyz"]
|
||||
assert "Status: stopped" in result.output
|
||||
assert "Healthy: yes" in result.output
|
||||
assert "Default backend: anthropic" in result.output
|
||||
assert f"Anthropic target: {anthropic_target}" in result.output
|
||||
assert f"OpenAI target: {openai_target}" in result.output
|
||||
assert "fake-secret" not in result.output
|
||||
assert ".invalid" not in result.output
|
||||
|
||||
|
||||
def test_install_restart_uses_internal_helpers(monkeypatch) -> None:
|
||||
|
||||
Reference in New Issue
Block a user