mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description
`headroom doctor`'s routing check matches the configured
`ANTHROPIC_BASE_URL` against:
```python
_LOOPBACK_URL_RE = re.compile(r"https?://(?:127\.0\.0\.1|localhost):(\d+)")
```
This misses two loopback URL forms that route correctly through the
proxy, so `check_claude_routing` (and `check_codex_routing`, which uses
the parallel `_CODEX_BASE_URL_RE`) reports a **correctly-routed**
session as "not routed" and sends the user off to debug a working setup:
1. **IPv6 loopback literal** — `http://[::1]:8787` never matches, so an
IPv6-configured route reads as unrouted.
2. **Case** — the regex is case-sensitive, but URL schemes and hostnames
are not. `HTTP://Localhost:8787` or `http://LOCALHOST:8787` (valid) both
fail to match.
```text
http://127.0.0.1:8787 -> MATCH
http://[::1]:8787 -> NO MATCH -> doctor: "not routed" (false negative)
HTTP://Localhost:8787 -> NO MATCH -> doctor: "not routed" (false negative)
```
This is the same false-negative class the routing check was already
corrected for twice (#3205, #3213): a diagnostic that tells a routed
user they are not routed is actively misleading.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Replace loopback URL prefix matching with a shared
urllib.parse.urlsplit classifier. Require an exact loopback host,
HTTP(S), an explicit valid port, and no userinfo; support IPv6 loopback
and case-insensitive schemes/hosts.
- Retain active Codex provider selection and classify its extracted
base_url through the same parser. Remote URLs, hostname-suffix
lookalikes and malformed ports remain warnings.
- Add routing/classifier regressions for accepted URL variants and
rejected lookalikes. Health probes retain current main behavior; this
change does not add IPv6-only health probing.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
tests/test_cli_doctor.py -> 95 passed in 1.19s
(4 of the 5 new variants FAIL on pre-fix code — [::1] and the upper-cased URLs — verified via git stash)
uvx ruff@0.16.2 check headroom/cli/doctor.py tests/test_cli_doctor.py -> All checks passed!
uvx mypy@1.20.2 headroom/cli/doctor.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.16.2 and mypy 1.20.2 via uvx.
- Exact command / steps: with `ANTHROPIC_BASE_URL=http://[::1]:8787`
(and the upper-cased variants) in a settings file,
`check_claude_routing(path, 8787)` returned WARN "not routed" before the
fix and PASS "routed" after; `http://127.0.0.1:8788` still WARNs on the
port mismatch and `https://gateway.corp.example/v1` still WARNs as
non-proxy.
- Observed result: IPv6 and case-variant loopback URLs now classify as
routed; port-mismatch and non-loopback WARNs are unchanged.
- Not tested: no live `headroom doctor` invocation against a running
proxy; the pure classifier `check_claude_routing` is exercised directly
(it is what the command calls).
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is the `headroom doctor`
diagnostic classifier, not a rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: only the false-negative is corrected
— additional valid loopback forms now classify as routed. Port-mismatch,
Ollama-collision, and non-loopback WARNs are unchanged, and no URL that
was previously WARNed as non-proxy now passes except genuine loopback
forms.
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this PR.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal behavior)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3255"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
### Maintainer current-main requalification
Exact head `922badceae5536e91e63f299685d77ff3bc3ce29`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.
Clean mechanical sync with upstream main
c01cf07d94. Re-read current discussion and
all three Devin notes. Exact URL parsing preserves active-provider
selection, IPv6/mixed-case support and rejection of userinfo, suffix
lookalikes, invalid ports and remote hosts. Current-main port-discovery
behavior is retained. All131 doctor tests pass;2-file Ruff lint/format
and main-relative whitespace pass. Devin IPv6-only health-probe
observation concerns the unchanged IPv4-only doctor probe implementation
already on main; this PR changes routing classification, not
health-probe binding, so the inherited limitation is not used to
withhold source approval. No fresh live IPv6-only doctor/proxy or
provider call is claimed; current verification is classifier/CLI tests,
not broader runtime proof. The description Changes Made section is being
corrected to describe urlsplit rather than the superseded regex
implementation. No contributor routing logic rewritten. Formal approval
awaits exact-head hosted qualification; maintainers handle CI. Nothing
merged.
Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
1190 lines
48 KiB
Python
1190 lines
48 KiB
Python
"""Tests for `headroom doctor`."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from dataclasses import dataclass
|
|
|
|
import pytest
|
|
from click.testing import CliRunner
|
|
|
|
import headroom.cli.doctor as doctor_mod
|
|
from headroom.cli.doctor import (
|
|
DEFAULT_PROXY_PORT,
|
|
FAIL,
|
|
PASS,
|
|
SKIP,
|
|
WARN,
|
|
check_budget,
|
|
check_claude_desktop,
|
|
check_claude_remote_control_gate,
|
|
check_claude_routing,
|
|
check_codex_routing,
|
|
check_deployments,
|
|
check_kompress_health,
|
|
check_proxy_liveness,
|
|
check_savings,
|
|
check_shell_env,
|
|
check_version_drift,
|
|
resolve_probe_port,
|
|
)
|
|
from headroom.cli.main import main
|
|
from headroom.providers.claude.runtime import remote_control_gate_message
|
|
|
|
LIVEZ_OK = {
|
|
"service": "headroom-proxy",
|
|
"status": "healthy",
|
|
"alive": True,
|
|
"version": "0.26.0",
|
|
"uptime_seconds": 260135.0,
|
|
}
|
|
|
|
STATS_OK = {
|
|
"persistent_savings": {
|
|
"lifetime": {"tokens_saved": 17_583_102, "compression_savings_usd": 7.81701},
|
|
"display_session": {"last_activity_at": "2026-06-12T12:00:00Z"},
|
|
},
|
|
"cost": {"budget_limit_usd": 10.0, "budget_period": "daily"},
|
|
}
|
|
|
|
|
|
class TestKompressHealth:
|
|
def test_missing_health_endpoint_skips(self):
|
|
result = check_kompress_health(None)
|
|
assert result.status == SKIP
|
|
assert "health endpoint" in result.summary
|
|
|
|
def test_older_proxy_without_component_warns(self):
|
|
result = check_kompress_health({"checks": {}})
|
|
assert result.status == WARN
|
|
assert "readiness" in result.summary
|
|
|
|
def test_disabled_passes(self):
|
|
result = check_kompress_health({"checks": {"kompress": {"enabled": False}}})
|
|
assert result.status == PASS
|
|
assert result.summary == "disabled"
|
|
|
|
def test_ready_reports_backend(self):
|
|
result = check_kompress_health(
|
|
{"checks": {"kompress": {"enabled": True, "ready": True, "backend": "onnx"}}}
|
|
)
|
|
assert result.status == PASS
|
|
assert result.summary == "ready (onnx)"
|
|
|
|
def test_cold_model_warns_with_action(self):
|
|
result = check_kompress_health(
|
|
{"checks": {"kompress": {"enabled": True, "ready": False, "status": "degraded"}}}
|
|
)
|
|
assert result.status == WARN
|
|
assert "passing through" in result.summary
|
|
assert "/debug/warmup" in (result.hint or "")
|
|
|
|
|
|
class TestProxyLiveness:
|
|
def test_down_is_fail_with_hint(self):
|
|
result = check_proxy_liveness(None, "http://127.0.0.1:8787")
|
|
assert result.status == FAIL
|
|
assert "headroom proxy" in (result.hint or "")
|
|
|
|
def test_up_mentions_version_and_uptime(self):
|
|
result = check_proxy_liveness(LIVEZ_OK, "http://127.0.0.1:8787")
|
|
assert result.status == PASS
|
|
assert "v0.26.0" in result.summary
|
|
assert "3d" in result.summary
|
|
|
|
def test_up_leaves_source_label_unprefixed(self):
|
|
livez = {**LIVEZ_OK, "version": "source-build+sha.abcdef123456"}
|
|
result = check_proxy_liveness(livez, "http://127.0.0.1:8787")
|
|
assert result.status == PASS
|
|
assert "source-build+sha.abcdef123456" in result.summary
|
|
assert "vsource-build" not in result.summary
|
|
|
|
|
|
class TestVersionDrift:
|
|
def test_match_passes(self):
|
|
assert check_version_drift(LIVEZ_OK, "0.26.0").status == PASS
|
|
|
|
def test_mismatch_warns_with_restart_hint(self):
|
|
result = check_version_drift(LIVEZ_OK, "0.27.0")
|
|
assert result.status == WARN
|
|
assert "drift" in result.summary
|
|
assert "restart" in (result.hint or "")
|
|
|
|
def test_proxy_down_skips(self):
|
|
assert check_version_drift(None, "0.26.0").status == SKIP
|
|
|
|
def test_unknown_version_warns(self):
|
|
assert check_version_drift({"version": "unknown"}, "0.26.0").status == WARN
|
|
assert check_version_drift(LIVEZ_OK, "unknown").status == WARN
|
|
|
|
@pytest.mark.parametrize(
|
|
("running", "installed"),
|
|
[
|
|
("source-build+g6266a1d774b5", "0.26.0"),
|
|
("source-build+sha.abcdef123456", "0.26.0"),
|
|
("6266a1d", "0.26.0"),
|
|
("0.26.0+gabcdef0", "0.26.0"),
|
|
("0.26.0", "source-build+sha.abcdef123456"),
|
|
],
|
|
)
|
|
def test_non_release_version_labels_skip_drift_comparison(self, running, installed):
|
|
result = check_version_drift({"version": running}, installed)
|
|
assert result.status == SKIP
|
|
assert "drift" not in result.summary
|
|
|
|
|
|
class TestClaudeRouting:
|
|
def test_missing_file_warns(self, tmp_path):
|
|
result = check_claude_routing(tmp_path / "settings.json", 8787)
|
|
assert result.status == WARN
|
|
assert "wrap claude" in (result.hint or "")
|
|
|
|
def test_malformed_json_warns(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text("{not json", encoding="utf-8")
|
|
assert check_claude_routing(path, 8787).status == WARN
|
|
|
|
@pytest.mark.parametrize("body", ["[]", "null", "42", '"a string"'])
|
|
def test_non_object_json_warns(self, tmp_path, body):
|
|
# Valid JSON that isn't an object parses cleanly, so it slips past the
|
|
# JSONDecodeError guard; the later payload.get("env") must not crash the
|
|
# diagnostic command that is being run precisely because the config is
|
|
# suspect.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(body, encoding="utf-8")
|
|
assert check_claude_routing(path, 8787).status == WARN
|
|
|
|
def test_no_env_key_warns(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(json.dumps({"env": {}}), encoding="utf-8")
|
|
assert check_claude_routing(path, 8787).status == WARN
|
|
|
|
def test_correct_url_passes(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_routing(path, 8787).status == PASS
|
|
|
|
@pytest.mark.parametrize(
|
|
"base_url",
|
|
[
|
|
"http://localhost:8787",
|
|
"http://[::1]:8787", # IPv6 loopback literal
|
|
"http://[::1]:8787/v1",
|
|
"HTTP://Localhost:8787", # scheme/host are case-insensitive
|
|
"http://LOCALHOST:8787",
|
|
],
|
|
)
|
|
def test_loopback_url_variants_pass(self, tmp_path, base_url):
|
|
"""A correctly-routed loopback URL must not read as 'not routed'.
|
|
|
|
The routing regex only matched lowercase ``127.0.0.1``/``localhost`` with
|
|
an explicit port, so an IPv6 (``[::1]``) or upper-cased loopback URL — a
|
|
valid route — produced a misleading WARN (the false-negative class of
|
|
#3205 / #3213).
|
|
"""
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(json.dumps({"env": {"ANTHROPIC_BASE_URL": base_url}}), encoding="utf-8")
|
|
assert check_claude_routing(path, 8787).status == PASS
|
|
|
|
def test_port_mismatch_warns(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8788"}}),
|
|
encoding="utf-8",
|
|
)
|
|
result = check_claude_routing(path, 8787)
|
|
assert result.status == WARN
|
|
assert "8788" in result.summary
|
|
|
|
def test_non_headroom_url_warns(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "https://gateway.corp.example/v1"}}),
|
|
encoding="utf-8",
|
|
)
|
|
result = check_claude_routing(path, 8787)
|
|
assert result.status == WARN
|
|
assert "gateway.corp.example" in result.summary
|
|
|
|
@pytest.mark.parametrize(
|
|
"base_url",
|
|
[
|
|
# Userinfo lookalike: a prefix match sees "localhost:8787" but the
|
|
# URL's real host is evil.example (the loopback text is credentials).
|
|
"http://localhost:8787@evil.example/v1",
|
|
"http://127.0.0.1:8787@evil.example/v1",
|
|
# Userinfo present at all — a real Headroom route never carries it.
|
|
"http://evil.example@127.0.0.1:8787",
|
|
# Hostname-suffix lookalikes: not an exact loopback host.
|
|
"http://localhost.evil.example:8787",
|
|
"http://127.0.0.1.evil.example:8787",
|
|
],
|
|
)
|
|
def test_loopback_lookalikes_warn(self, tmp_path, base_url):
|
|
"""URL classification (not prefix matching) must reject userinfo and
|
|
hostname-suffix lookalikes that resolve to a non-loopback host, so
|
|
``doctor`` never reports an attacker-controlled destination as routed."""
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(json.dumps({"env": {"ANTHROPIC_BASE_URL": base_url}}), encoding="utf-8")
|
|
assert check_claude_routing(path, 8787).status == WARN
|
|
|
|
def test_foundry_url_passes(self, tmp_path):
|
|
# In Foundry mode ANTHROPIC_BASE_URL is absent by design; routing lives
|
|
# in ANTHROPIC_FOUNDRY_BASE_URL. doctor must recognize it as routed.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps(
|
|
{
|
|
"env": {
|
|
"CLAUDE_CODE_USE_FOUNDRY": "1",
|
|
"ANTHROPIC_FOUNDRY_BASE_URL": "http://127.0.0.1:8787",
|
|
}
|
|
}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_routing(path, 8787).status == PASS
|
|
|
|
def test_foundry_without_base_url_warns(self, tmp_path):
|
|
# Foundry mode on but no upstream URL configured: still unrouted.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"CLAUDE_CODE_USE_FOUNDRY": "1"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_routing(path, 8787).status == WARN
|
|
|
|
def test_foundry_url_ignored_without_flag(self, tmp_path):
|
|
# Without CLAUDE_CODE_USE_FOUNDRY the foundry URL is not consulted, so a
|
|
# settings file carrying only the foundry key reads as unrouted.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_FOUNDRY_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_routing(path, 8787).status == WARN
|
|
|
|
|
|
class TestClaudeDesktop:
|
|
def test_no_desktop_dir_produces_no_row(self, tmp_path):
|
|
# #2925: absent Desktop -> no row, so it never contradicts a routed CLI.
|
|
assert check_claude_desktop(tmp_path / "Claude") is None
|
|
|
|
def test_desktop_present_warns_about_bypass(self, tmp_path):
|
|
desktop = tmp_path / "Claude"
|
|
desktop.mkdir()
|
|
result = check_claude_desktop(desktop)
|
|
assert result is not None
|
|
assert result.name == "claude desktop"
|
|
assert result.status == WARN
|
|
assert "bypass" in result.summary
|
|
assert "#869" in (result.hint or "")
|
|
|
|
def test_doctor_appends_desktop_row_when_present(self, tmp_path, monkeypatch):
|
|
# Integration: the entrypoint surfaces the Desktop row when detected.
|
|
desktop = tmp_path / "Claude"
|
|
desktop.mkdir()
|
|
monkeypatch.setattr(doctor_mod, "claude_desktop_config_dir", lambda: desktop)
|
|
monkeypatch.setattr(doctor_mod, "probe_json", lambda *a, **k: None)
|
|
monkeypatch.setattr(doctor_mod, "list_manifests", lambda: [])
|
|
result = CliRunner().invoke(main, ["doctor", "--json"])
|
|
payload = json.loads(result.output)
|
|
rows = {c["name"]: c for c in payload["checks"]}
|
|
assert "claude desktop" in rows
|
|
assert rows["claude desktop"]["status"] == WARN
|
|
|
|
def test_doctor_omits_desktop_row_when_absent(self, tmp_path, monkeypatch):
|
|
monkeypatch.setattr(doctor_mod, "claude_desktop_config_dir", lambda: tmp_path / "Claude")
|
|
monkeypatch.setattr(doctor_mod, "probe_json", lambda *a, **k: None)
|
|
monkeypatch.setattr(doctor_mod, "list_manifests", lambda: [])
|
|
result = CliRunner().invoke(main, ["doctor", "--json"])
|
|
payload = json.loads(result.output)
|
|
assert "claude desktop" not in {c["name"] for c in payload["checks"]}
|
|
|
|
|
|
class TestClaudeRemoteControlGate:
|
|
def test_settings_custom_base_warns(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
result = check_claude_remote_control_gate(path, {})
|
|
assert result is not None
|
|
assert result.status == WARN
|
|
assert remote_control_gate_message("ANTHROPIC_BASE_URL from settings") in result.summary
|
|
|
|
def test_shell_env_custom_base_warns(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text("{}", encoding="utf-8")
|
|
result = check_claude_remote_control_gate(
|
|
path, {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}
|
|
)
|
|
assert result is not None
|
|
assert result.status == WARN
|
|
assert remote_control_gate_message("ANTHROPIC_BASE_URL in shell") in result.summary
|
|
|
|
def test_no_custom_base_no_warning(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "https://api.anthropic.com"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_remote_control_gate(path, {}) is None
|
|
|
|
@pytest.mark.parametrize("body", ["[]", "null", "42"])
|
|
def test_non_object_settings_does_not_crash(self, tmp_path, body):
|
|
# A valid-but-non-object settings file parses past the JSONDecodeError
|
|
# guard; payload.get("env") must not raise AttributeError. The shell env
|
|
# still drives the gate, so a custom base there still warns.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(body, encoding="utf-8")
|
|
result = check_claude_remote_control_gate(
|
|
path, {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}
|
|
)
|
|
assert result is not None
|
|
assert result.status == WARN
|
|
|
|
def test_api_key_auth_suppresses_warning(self, tmp_path):
|
|
# Issue #1779: a PAYG / API-key session never had Remote Control, so the
|
|
# gate warning must not fire even behind a custom base URL.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_remote_control_gate(path, {"ANTHROPIC_API_KEY": "sk-ant-api-x"}) is None
|
|
|
|
def test_settings_api_key_suppresses_warning(self, tmp_path):
|
|
# An API key configured in settings.json (not just the shell) also means
|
|
# a non-subscription session — stay silent.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps(
|
|
{
|
|
"env": {
|
|
"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787",
|
|
"ANTHROPIC_API_KEY": "sk-ant-api-x",
|
|
}
|
|
}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_remote_control_gate(path, {}) is None
|
|
|
|
def test_version_resolver_not_called_without_custom_base(self, tmp_path):
|
|
# The `claude --version` subprocess is expensive (Node CLI cold start);
|
|
# the check must not invoke the resolver when no custom base URL exists.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "https://api.anthropic.com"}}),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
def boom() -> tuple[int, int, int]:
|
|
raise AssertionError("resolver must not run when no custom base URL")
|
|
|
|
assert check_claude_remote_control_gate(path, {}, version_resolver=boom) is None
|
|
|
|
def test_version_resolver_not_called_for_api_key_auth(self, tmp_path):
|
|
# PAYG sessions are suppressed before version matters — no subprocess.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
def boom() -> tuple[int, int, int]:
|
|
raise AssertionError("resolver must not run for API-key auth")
|
|
|
|
assert (
|
|
check_claude_remote_control_gate(
|
|
path, {"ANTHROPIC_API_KEY": "sk-ant-api-x"}, version_resolver=boom
|
|
)
|
|
is None
|
|
)
|
|
|
|
def test_version_resolver_called_once_and_honored(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
calls: list[int] = []
|
|
|
|
def resolver() -> tuple[int, int, int]:
|
|
calls.append(1)
|
|
return (2, 1, 196)
|
|
|
|
# Shell env ALSO custom so both loop sources are live — still one call.
|
|
result = check_claude_remote_control_gate(
|
|
path,
|
|
{"ANTHROPIC_BASE_URL": "http://127.0.0.1:9999"},
|
|
version_resolver=resolver,
|
|
)
|
|
assert result is not None
|
|
assert "2.1.196" in result.summary
|
|
assert calls == [1]
|
|
|
|
def test_version_resolver_pre_gate_version_suppresses(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert (
|
|
check_claude_remote_control_gate(path, {}, version_resolver=lambda: (2, 1, 195)) is None
|
|
)
|
|
|
|
def test_malformed_settings_base_url_does_not_crash(self, tmp_path):
|
|
# Issue #1779: settings.json is user-edited; a typo'd IPv6 literal made
|
|
# urlparse raise ValueError("Invalid IPv6 URL") and crashed doctor.
|
|
# Malformed values degrade to "no host" and the check stays silent —
|
|
# check_claude_routing separately flags unusable URLs.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://[::1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_remote_control_gate(path, {}) is None
|
|
|
|
def test_malformed_shell_base_url_does_not_crash(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text("{}", encoding="utf-8")
|
|
assert check_claude_remote_control_gate(path, {"ANTHROPIC_BASE_URL": "http://["}) is None
|
|
|
|
def test_pre_gate_version_suppresses_warning(self, tmp_path):
|
|
# Older Claude Code does not gate RC on the base URL — no false alarm.
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
assert check_claude_remote_control_gate(path, {}, version=(2, 1, 195)) is None
|
|
|
|
def test_gated_version_warns_with_exact_version(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
result = check_claude_remote_control_gate(path, {}, version=(2, 1, 196))
|
|
assert result is not None
|
|
assert result.status == WARN
|
|
assert "2.1.196" in result.summary
|
|
assert "disables" in result.summary
|
|
# Sibling gates are co-reported in the hint (#746 / #1158).
|
|
assert "#746" in (result.hint or "")
|
|
assert "#1158" in (result.hint or "")
|
|
|
|
def test_settings_check_still_routes(self, tmp_path):
|
|
path = tmp_path / "settings.json"
|
|
path.write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
result = check_claude_routing(path, 8787)
|
|
assert result.status == PASS
|
|
|
|
|
|
class TestClaudeRoutingScope:
|
|
"""Project-scoped routing must not read as "not routed" (#3205).
|
|
|
|
`headroom init claude` without --global writes
|
|
`.claude/settings.local.json`. Reading only `~/.claude/settings.json`
|
|
reported not-routed for sessions that were genuinely routed and actively
|
|
compressing, which sent one team hand-checking `ps eww` on every session.
|
|
"""
|
|
|
|
@staticmethod
|
|
def _settings(path, base_url): # noqa: ANN001, ANN205
|
|
path.parent.mkdir(parents=True, exist_ok=True)
|
|
body = {"env": {"ANTHROPIC_BASE_URL": base_url}} if base_url else {"env": {}}
|
|
path.write_text(json.dumps(body), encoding="utf-8")
|
|
return path
|
|
|
|
def test_project_local_settings_count_as_routed(self, tmp_path):
|
|
user = tmp_path / "user" / "settings.json"
|
|
project = self._settings(
|
|
tmp_path / "proj" / ".claude" / "settings.local.json", "http://127.0.0.1:8787"
|
|
)
|
|
|
|
result = check_claude_routing(user, 8787, [project])
|
|
|
|
assert result.status == PASS
|
|
assert "settings.local.json" in result.summary or "settings.local.json" in str(result)
|
|
|
|
def test_project_settings_json_counts_as_routed(self, tmp_path):
|
|
user = tmp_path / "user" / "settings.json"
|
|
project = self._settings(
|
|
tmp_path / "proj" / ".claude" / "settings.json", "http://127.0.0.1:8787"
|
|
)
|
|
|
|
assert check_claude_routing(user, 8787, [project]).status == PASS
|
|
|
|
def test_project_scope_takes_precedence_over_user_scope(self, tmp_path):
|
|
"""Claude layers project over user, so the reported port follows suit."""
|
|
user = self._settings(tmp_path / "user" / "settings.json", "http://127.0.0.1:9999")
|
|
project = self._settings(
|
|
tmp_path / "proj" / ".claude" / "settings.local.json", "http://127.0.0.1:8787"
|
|
)
|
|
|
|
assert check_claude_routing(user, 8787, [project]).status == PASS
|
|
|
|
def test_falls_back_to_user_scope_when_project_has_no_base_url(self, tmp_path):
|
|
user = self._settings(tmp_path / "user" / "settings.json", "http://127.0.0.1:8787")
|
|
project = self._settings(tmp_path / "proj" / ".claude" / "settings.local.json", "")
|
|
|
|
assert check_claude_routing(user, 8787, [project]).status == PASS
|
|
|
|
def test_still_warns_when_nothing_routes(self, tmp_path):
|
|
user = self._settings(tmp_path / "user" / "settings.json", "")
|
|
project = self._settings(tmp_path / "proj" / ".claude" / "settings.local.json", "")
|
|
|
|
assert check_claude_routing(user, 8787, [project]).status == WARN
|
|
|
|
def test_missing_project_file_is_skipped_not_fatal(self, tmp_path):
|
|
user = self._settings(tmp_path / "user" / "settings.json", "http://127.0.0.1:8787")
|
|
absent = tmp_path / "proj" / ".claude" / "settings.local.json"
|
|
|
|
assert check_claude_routing(user, 8787, [absent]).status == PASS
|
|
|
|
def test_unparseable_project_file_surfaces_rather_than_reporting_not_routed(self, tmp_path):
|
|
project = tmp_path / "proj" / ".claude" / "settings.local.json"
|
|
project.parent.mkdir(parents=True, exist_ok=True)
|
|
project.write_text("{not json", encoding="utf-8")
|
|
user = tmp_path / "user" / "settings.json"
|
|
|
|
result = check_claude_routing(user, 8787, [project])
|
|
|
|
assert result.status == WARN
|
|
assert "could not parse" in result.summary
|
|
|
|
def test_no_project_paths_preserves_original_behaviour(self, tmp_path):
|
|
user = self._settings(tmp_path / "user" / "settings.json", "http://127.0.0.1:8787")
|
|
|
|
assert check_claude_routing(user, 8787).status == PASS
|
|
|
|
|
|
class TestCodexRouting:
|
|
def test_missing_file_warns(self, tmp_path):
|
|
assert check_codex_routing(tmp_path / "config.toml", 8787).status == WARN
|
|
|
|
def test_marker_block_right_port_passes(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "headroom"\n'
|
|
"[model_providers.headroom]\n"
|
|
'base_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
assert check_codex_routing(path, 8787).status == PASS
|
|
|
|
def test_preserved_provider_id_right_port_passes(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "codex-lb"\n'
|
|
"[model_providers.codex-lb]\n"
|
|
'base_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
result = check_codex_routing(path, 8787)
|
|
assert result.status == PASS
|
|
assert result.hint is None
|
|
|
|
def test_preserved_provider_id_port_mismatch_warns(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "codex-lb"\n'
|
|
"[model_providers.codex-lb]\n"
|
|
'base_url = "http://localhost:9999/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
result = check_codex_routing(path, 8787)
|
|
assert result.status == WARN
|
|
assert "9999" in result.summary
|
|
|
|
def test_active_provider_takes_precedence_over_headroom_block(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "corp"\n'
|
|
"[model_providers.corp]\n"
|
|
'base_url = "https://gateway.corp.example/v1"\n'
|
|
"[model_providers.headroom]\n"
|
|
'base_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
result = check_codex_routing(path, 8787)
|
|
assert result.status == WARN
|
|
assert "gateway.corp.example" in result.summary
|
|
|
|
def test_port_mismatch_warns(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'[model_providers.headroom]\nbase_url = "http://127.0.0.1:9999/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
result = check_codex_routing(path, 8787)
|
|
assert result.status == WARN
|
|
assert "9999" in result.summary
|
|
|
|
def test_root_openai_base_url_on_builtin_provider_passes(self, tmp_path):
|
|
"""Headroom Desktop's layout: built-in `openai` provider, root base URL."""
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "openai"\n'
|
|
'openai_base_url = "http://127.0.0.1:6767/v1"\n'
|
|
'[projects."/work/app"]\n'
|
|
'trust_level = "trusted"\n',
|
|
encoding="utf-8",
|
|
)
|
|
result = check_codex_routing(path, 6767)
|
|
assert result.status == PASS, result.summary
|
|
assert check_codex_routing(path, 8787).status == WARN
|
|
|
|
def test_root_openai_base_url_without_model_provider_passes(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text('openai_base_url = "http://127.0.0.1:8787/v1"\n', encoding="utf-8")
|
|
assert check_codex_routing(path, 8787).status == PASS
|
|
|
|
def test_openai_base_url_inside_a_table_is_not_root(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "openai"\n'
|
|
"[profiles.fast]\n"
|
|
'openai_base_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
assert check_codex_routing(path, 8787).status == WARN
|
|
|
|
def test_root_openai_base_url_beats_an_inactive_headroom_table(self, tmp_path):
|
|
"""No root `model_provider`: Codex uses its built-in `openai`, not the table."""
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'openai_base_url = "http://127.0.0.1:6767/v1"\n'
|
|
"[model_providers.headroom]\n"
|
|
'base_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
assert check_codex_routing(path, 6767).status == PASS
|
|
assert check_codex_routing(path, 8787).status == WARN
|
|
|
|
def test_profile_model_provider_does_not_hide_the_default_route(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'openai_base_url = "http://127.0.0.1:6767/v1"\n'
|
|
"[profiles.work]\n"
|
|
'model_provider = "azure"\n',
|
|
encoding="utf-8",
|
|
)
|
|
assert check_codex_routing(path, 6767).status == PASS
|
|
|
|
def test_no_marker_warns(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text('model = "gpt-5"\n', encoding="utf-8")
|
|
assert check_codex_routing(path, 8787).status == WARN
|
|
|
|
@pytest.mark.parametrize(
|
|
"base_url",
|
|
[
|
|
"http://localhost:8787@evil.example/v1", # userinfo lookalike
|
|
"http://localhost.evil.example:8787/v1", # hostname suffix lookalike
|
|
"https://gateway.corp.example/v1", # plainly remote
|
|
],
|
|
)
|
|
def test_non_loopback_base_url_warns(self, tmp_path, base_url):
|
|
"""The Headroom block's base_url is classified through the same URL
|
|
parser, so a non-loopback (or lookalike) base_url reads as not routed
|
|
rather than passing on a prefix match."""
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
f'[model_providers.headroom]\nbase_url = "{base_url}"\n',
|
|
encoding="utf-8",
|
|
)
|
|
result = check_codex_routing(path, 8787)
|
|
assert result.status == WARN
|
|
|
|
def test_base_url_read_from_headroom_block_not_earlier_provider(self, tmp_path):
|
|
"""A base_url in an earlier provider block must not be mistaken for the
|
|
Headroom route; only the Headroom block's own base_url counts."""
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
"[model_providers.openai]\n"
|
|
'base_url = "https://api.openai.com/v1"\n'
|
|
"\n"
|
|
"[model_providers.headroom]\n"
|
|
'base_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
assert check_codex_routing(path, 8787).status == PASS
|
|
|
|
def test_marker_block_without_base_url_warns(self, tmp_path):
|
|
"""A provider block that exists but has no base_url must not read as
|
|
routed: with no base_url there is nothing pointing Codex at the proxy,
|
|
so it WARNs rather than falling through to PASS."""
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "headroom"\n'
|
|
"[model_providers.headroom]\n"
|
|
"requires_openai_auth = true\n",
|
|
encoding="utf-8",
|
|
)
|
|
result = check_codex_routing(path, 8787)
|
|
assert result.status == WARN
|
|
assert "base_url" in result.summary
|
|
assert "wrap" in (result.hint or "")
|
|
|
|
def test_garbage_bytes_warn_not_crash(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_bytes(b"\xff\xfe garbage \x00")
|
|
assert check_codex_routing(path, 8787).status == WARN
|
|
|
|
# -- requires_openai_auth (#3206) ------------------------------------
|
|
# Codex attaches no Authorization header to a custom provider unless the
|
|
# block carries requires_openai_auth. A ChatGPT-OAuth user then 401s on
|
|
# every request with "Missing bearer" while doctor reported green -- the
|
|
# reason one report went 15h before anyone could see the cause.
|
|
|
|
@staticmethod
|
|
def _routed(tmp_path, *, requires_auth: bool):
|
|
path = tmp_path / "config.toml"
|
|
block = (
|
|
"[model_providers.headroom]\n"
|
|
'base_url = "http://127.0.0.1:8787/v1"\n'
|
|
"supports_websockets = true\n"
|
|
)
|
|
if requires_auth:
|
|
block += "requires_openai_auth = true\n"
|
|
path.write_text(block, encoding="utf-8")
|
|
return path
|
|
|
|
@staticmethod
|
|
def _chatgpt_auth(tmp_path):
|
|
(tmp_path / "auth.json").write_text('{"auth_mode": "chatgpt"}', encoding="utf-8")
|
|
|
|
def test_chatgpt_auth_without_requires_openai_auth_warns(self, tmp_path):
|
|
path = self._routed(tmp_path, requires_auth=False)
|
|
self._chatgpt_auth(tmp_path)
|
|
|
|
result = check_codex_routing(path, 8787)
|
|
|
|
assert result.status == WARN
|
|
assert "Authorization" in result.summary
|
|
|
|
def test_chatgpt_auth_with_requires_openai_auth_passes(self, tmp_path):
|
|
path = self._routed(tmp_path, requires_auth=True)
|
|
self._chatgpt_auth(tmp_path)
|
|
|
|
assert check_codex_routing(path, 8787).status == PASS
|
|
|
|
def test_preserved_provider_id_without_requires_openai_auth_warns(self, tmp_path):
|
|
path = tmp_path / "config.toml"
|
|
path.write_text(
|
|
'model_provider = "codex-lb"\n'
|
|
"[model_providers.codex-lb]\n"
|
|
'base_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
self._chatgpt_auth(tmp_path)
|
|
|
|
result = check_codex_routing(path, 8787)
|
|
assert result.status == WARN
|
|
assert "Authorization" in result.summary
|
|
|
|
def test_api_key_user_without_requires_openai_auth_still_passes(self, tmp_path):
|
|
"""API-key users must not be nagged -- the flag would break them (#406)."""
|
|
path = self._routed(tmp_path, requires_auth=False)
|
|
(tmp_path / "auth.json").write_text('{"OPENAI_API_KEY": "sk-test"}', encoding="utf-8")
|
|
|
|
assert check_codex_routing(path, 8787).status == PASS
|
|
|
|
def test_no_auth_json_does_not_warn(self, tmp_path):
|
|
path = self._routed(tmp_path, requires_auth=False)
|
|
|
|
assert check_codex_routing(path, 8787).status == PASS
|
|
|
|
|
|
class TestShellEnv:
|
|
def test_unset_warns(self):
|
|
result = check_shell_env({}, 8787)
|
|
assert result.status == WARN
|
|
assert "bypasses" in result.summary
|
|
|
|
def test_matching_anthropic_url_passes(self):
|
|
env = {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}
|
|
assert check_shell_env(env, 8787).status == PASS
|
|
|
|
def test_localhost_also_passes(self):
|
|
env = {"OPENAI_BASE_URL": "http://localhost:8787/v1"}
|
|
assert check_shell_env(env, 8787).status == PASS
|
|
|
|
def test_other_url_warns(self):
|
|
env = {"ANTHROPIC_BASE_URL": "https://api.anthropic.com"}
|
|
assert check_shell_env(env, 8787).status == WARN
|
|
|
|
def test_ollama_launch_url_names_the_collision(self):
|
|
# `ollama launch claude` points Claude Code at Ollama's :11434, which
|
|
# outranks the persistent Headroom route (issue #2199). The diagnostic
|
|
# must name Ollama, not tell the user to re-probe port 11434.
|
|
env = {"ANTHROPIC_BASE_URL": "http://127.0.0.1:11434"}
|
|
result = check_shell_env(env, 8787)
|
|
assert result.status == WARN
|
|
assert "Ollama" in result.summary
|
|
assert "#2199" in (result.hint or "")
|
|
assert "--port 11434" not in (result.hint or "")
|
|
|
|
|
|
class TestSavings:
|
|
def test_from_stats_passes_with_totals(self, tmp_path):
|
|
result = check_savings(STATS_OK, tmp_path / "missing.json")
|
|
assert result.status == PASS
|
|
assert "17,583,102" in result.summary
|
|
assert "$7.82" in result.summary
|
|
|
|
def test_falls_back_to_file_when_proxy_down(self, tmp_path):
|
|
savings_file = tmp_path / "proxy_savings.json"
|
|
savings_file.write_text(
|
|
json.dumps(
|
|
{
|
|
"lifetime": {"tokens_saved": 500, "compression_savings_usd": 0.01},
|
|
"display_session": {"last_activity_at": "2026-06-12T11:00:00Z"},
|
|
}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
result = check_savings(None, savings_file)
|
|
assert result.status == PASS
|
|
assert "500" in result.summary
|
|
assert str(savings_file) in result.summary
|
|
|
|
def test_no_data_warns(self, tmp_path):
|
|
assert check_savings(None, tmp_path / "missing.json").status == WARN
|
|
|
|
def test_zero_tokens_warns(self, tmp_path):
|
|
stats = {"persistent_savings": {"lifetime": {"tokens_saved": 0}}}
|
|
assert check_savings(stats, tmp_path / "missing.json").status == WARN
|
|
|
|
|
|
class TestBudget:
|
|
def test_proxy_down_skips(self):
|
|
assert check_budget(None).status == SKIP
|
|
|
|
def test_cost_tracking_disabled_warns(self):
|
|
assert check_budget({"cost": None}).status == WARN
|
|
|
|
def test_old_proxy_without_keys_warns(self):
|
|
result = check_budget({"cost": {"savings_usd": 1.0}})
|
|
assert result.status == WARN
|
|
assert "older version" in result.summary
|
|
|
|
def test_unset_budget_warns_with_hint(self):
|
|
result = check_budget({"cost": {"budget_limit_usd": None}})
|
|
assert result.status == WARN
|
|
assert "--budget" in (result.hint or "")
|
|
|
|
def test_configured_budget_passes(self):
|
|
result = check_budget(STATS_OK)
|
|
assert result.status == PASS
|
|
assert "$10.0/daily" in result.summary
|
|
|
|
def test_estimated_basis_share_is_reported_without_warning(self):
|
|
"""#2713: spend booked from a token estimate is surfaced, not warned on.
|
|
|
|
A provider that never reports a usage breakdown would otherwise sit at a
|
|
permanent WARN, so this stays informational.
|
|
"""
|
|
result = check_budget(
|
|
{
|
|
"cost": {
|
|
"budget_limit_usd": 10.0,
|
|
"budget_period": "daily",
|
|
"budget_estimated_basis": "count",
|
|
"budget_basis": {"estimated_usd": 1.24, "estimated_pct": 62.3},
|
|
}
|
|
}
|
|
)
|
|
assert result.status == PASS
|
|
assert "62% of period spend ($1.2400)" in result.summary
|
|
assert "Headroom token estimates" in result.summary
|
|
|
|
def test_all_measured_spend_adds_no_note(self):
|
|
result = check_budget(
|
|
{
|
|
"cost": {
|
|
"budget_limit_usd": 10.0,
|
|
"budget_period": "daily",
|
|
"budget_estimated_basis": "count",
|
|
"budget_basis": {"estimated_usd": 0.0, "estimated_pct": 0.0},
|
|
}
|
|
}
|
|
)
|
|
assert result.summary == "$10.0/daily budget enforced"
|
|
|
|
def test_non_default_basis_policy_is_named(self):
|
|
result = check_budget(
|
|
{
|
|
"cost": {
|
|
"budget_limit_usd": 10.0,
|
|
"budget_period": "daily",
|
|
"budget_estimated_basis": "block",
|
|
}
|
|
}
|
|
)
|
|
assert "estimated-basis policy: block" in result.summary
|
|
|
|
def test_missing_basis_fields_degrade_quietly(self):
|
|
"""`doctor` must still work against a proxy predating these fields."""
|
|
result = check_budget({"cost": {"budget_limit_usd": 10.0, "budget_period": "daily"}})
|
|
assert result.status == PASS
|
|
assert result.summary == "$10.0/daily budget enforced"
|
|
|
|
malformed = check_budget(
|
|
{"cost": {"budget_limit_usd": 10.0, "budget_period": "daily", "budget_basis": "nope"}}
|
|
)
|
|
assert malformed.status == PASS
|
|
|
|
|
|
@dataclass
|
|
class _FakeManifest:
|
|
profile: str
|
|
health_url: str
|
|
port: int = 8787
|
|
|
|
|
|
class TestDeployments:
|
|
def test_no_manifests_omits_section(self):
|
|
assert check_deployments([]) is None
|
|
|
|
def test_all_healthy_passes(self):
|
|
manifests = [_FakeManifest("default", "http://127.0.0.1:8787/readyz")]
|
|
result = check_deployments(manifests, probe=lambda url: {"ready": True})
|
|
assert result is not None and result.status == PASS
|
|
|
|
def test_unhealthy_fails_naming_profile(self):
|
|
manifests = [_FakeManifest("prod", "http://127.0.0.1:9999/readyz")]
|
|
result = check_deployments(manifests, probe=lambda url: None)
|
|
assert result is not None and result.status == FAIL
|
|
assert "prod" in result.summary
|
|
|
|
|
|
class TestResolveProbePort:
|
|
def test_explicit_port_wins_over_deployment(self):
|
|
manifests = [_FakeManifest("default", "", port=8789)]
|
|
assert resolve_probe_port(9000, manifests) == 9000
|
|
|
|
def test_default_profile_deployment_port_is_preferred(self):
|
|
manifests = [
|
|
_FakeManifest("prod", "", port=9999),
|
|
_FakeManifest("default", "", port=8789),
|
|
]
|
|
assert resolve_probe_port(None, manifests) == 8789
|
|
|
|
def test_single_non_default_deployment_is_used(self):
|
|
assert resolve_probe_port(None, [_FakeManifest("prod", "", port=9999)]) == 9999
|
|
|
|
def test_builtin_default_without_deployments(self):
|
|
assert resolve_probe_port(None, []) == DEFAULT_PROXY_PORT
|
|
|
|
def test_ambiguous_named_deployments_fall_back_to_the_default(self):
|
|
"""Several named profiles and no request: any pick would be arbitrary."""
|
|
manifests = [
|
|
_FakeManifest("prod", "", port=9998),
|
|
_FakeManifest("staging", "", port=9999),
|
|
]
|
|
assert resolve_probe_port(None, manifests) == DEFAULT_PROXY_PORT
|
|
|
|
|
|
class TestDoctorCommand:
|
|
@pytest.fixture
|
|
def runner(self):
|
|
return CliRunner()
|
|
|
|
@pytest.fixture
|
|
def isolated(self, tmp_path, monkeypatch):
|
|
"""Point all filesystem/network surfaces at controlled fakes."""
|
|
monkeypatch.setattr(doctor_mod, "claude_settings_path", lambda: tmp_path / "settings.json")
|
|
monkeypatch.setattr(doctor_mod, "codex_config_path", lambda: tmp_path / "config.toml")
|
|
monkeypatch.setattr(
|
|
doctor_mod, "codex_project_config_path", lambda: tmp_path / "project-codex.toml"
|
|
)
|
|
monkeypatch.setattr(doctor_mod, "savings_path", lambda: tmp_path / "savings.json")
|
|
monkeypatch.setattr(doctor_mod, "list_manifests", lambda: [])
|
|
for var in (
|
|
"ANTHROPIC_API_KEY",
|
|
"ANTHROPIC_AUTH_TOKEN",
|
|
"ANTHROPIC_BASE_URL",
|
|
"CLAUDE_CODE_USE_BEDROCK",
|
|
"CLAUDE_CODE_USE_FOUNDRY",
|
|
"CLAUDE_CODE_USE_VERTEX",
|
|
"OPENAI_BASE_URL",
|
|
"HEADROOM_PORT",
|
|
):
|
|
monkeypatch.delenv(var, raising=False)
|
|
return tmp_path
|
|
|
|
def _probe(self, livez, stats):
|
|
def fake_probe(url, timeout=2.0):
|
|
if url.endswith("/livez"):
|
|
return livez
|
|
if url.endswith("/stats"):
|
|
return stats
|
|
return None
|
|
|
|
return fake_probe
|
|
|
|
def test_proxy_down_exits_2(self, runner, isolated, monkeypatch):
|
|
monkeypatch.setattr(doctor_mod, "probe_json", self._probe(None, None))
|
|
result = runner.invoke(main, ["doctor"])
|
|
assert result.exit_code == 2
|
|
assert "not reachable" in result.output
|
|
|
|
def test_conflicting_claude_auth_is_a_redacted_failure(self, runner, isolated, monkeypatch):
|
|
settings = isolated / "settings.json"
|
|
settings.write_text('{"env":{"ANTHROPIC_AUTH_TOKEN":"token-value"}}', encoding="utf-8")
|
|
monkeypatch.setenv("ANTHROPIC_API_KEY", "api-value")
|
|
monkeypatch.setattr(doctor_mod, "probe_json", self._probe(None, None))
|
|
|
|
result = runner.invoke(main, ["doctor", "--json"])
|
|
|
|
assert result.exit_code == 2
|
|
payload = json.loads(result.output)
|
|
auth = next(check for check in payload["checks"] if check["name"] == "claude auth")
|
|
assert auth["status"] == "fail"
|
|
assert "shell environment" in auth["summary"]
|
|
assert str(settings) in auth["summary"]
|
|
assert "api-value" not in result.output
|
|
assert "token-value" not in result.output
|
|
|
|
def test_warnings_only_exits_1(self, runner, isolated, monkeypatch):
|
|
monkeypatch.setattr(doctor_mod, "probe_json", self._probe(LIVEZ_OK, STATS_OK))
|
|
monkeypatch.setattr(doctor_mod, "get_version", lambda: "0.26.0")
|
|
# proxy healthy, but clients unwrapped + shell env unset -> warns
|
|
result = runner.invoke(main, ["doctor"])
|
|
assert result.exit_code == 1
|
|
|
|
def test_remote_control_warning_exits_1(self, runner, isolated, monkeypatch):
|
|
monkeypatch.setattr(doctor_mod, "probe_json", self._probe(LIVEZ_OK, STATS_OK))
|
|
monkeypatch.setattr(doctor_mod, "get_version", lambda: "0.26.0")
|
|
monkeypatch.setattr(doctor_mod, "detect_claude_code_version", lambda: None)
|
|
(isolated / "settings.json").write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}}),
|
|
encoding="utf-8",
|
|
)
|
|
(isolated / "config.toml").write_text(
|
|
'[model_providers.headroom]\nbase_url = "http://127.0.0.1:8787/v1"\n',
|
|
encoding="utf-8",
|
|
)
|
|
result = runner.invoke(
|
|
main, ["doctor"], env={"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787"}
|
|
)
|
|
assert result.exit_code == 1, result.output
|
|
assert "Remote Control" in result.output
|
|
|
|
def test_json_output_parses(self, runner, isolated, monkeypatch):
|
|
monkeypatch.setattr(doctor_mod, "probe_json", self._probe(LIVEZ_OK, STATS_OK))
|
|
result = runner.invoke(main, ["doctor", "--json", "--port", "8787"])
|
|
payload = json.loads(result.output)
|
|
assert payload["port"] == 8787
|
|
assert {c["name"] for c in payload["checks"]} >= {"proxy", "version", "budget"}
|
|
assert all(c["status"] in ("pass", "warn", "fail", "skip") for c in payload["checks"])
|
|
|
|
def test_port_option_changes_probe_url(self, runner, isolated, monkeypatch):
|
|
seen: list[str] = []
|
|
|
|
def recording_probe(url, timeout=2.0):
|
|
seen.append(url)
|
|
return None
|
|
|
|
monkeypatch.setattr(doctor_mod, "probe_json", recording_probe)
|
|
runner.invoke(main, ["doctor", "--port", "9999"])
|
|
assert "http://127.0.0.1:9999/livez" in seen
|
|
|
|
def test_port_env_var_respected(self, runner, isolated, monkeypatch):
|
|
seen: list[str] = []
|
|
|
|
def recording_probe(url, timeout=2.0):
|
|
seen.append(url)
|
|
return None
|
|
|
|
monkeypatch.setattr(doctor_mod, "probe_json", recording_probe)
|
|
runner.invoke(main, ["doctor"], env={"HEADROOM_PORT": "9999"})
|
|
assert "http://127.0.0.1:9999/livez" in seen
|
|
|
|
def test_deployment_port_probed_when_shell_has_no_override(self, runner, isolated, monkeypatch):
|
|
"""`headroom deploy --port N` only sets HEADROOM_PORT inside the deployment."""
|
|
seen: list[str] = []
|
|
|
|
def recording_probe(url, timeout=2.0):
|
|
seen.append(url)
|
|
return None
|
|
|
|
monkeypatch.setattr(
|
|
doctor_mod,
|
|
"list_manifests",
|
|
lambda: [_FakeManifest("default", "http://127.0.0.1:8789/readyz", port=8789)],
|
|
)
|
|
monkeypatch.setattr(doctor_mod, "probe_json", recording_probe)
|
|
runner.invoke(main, ["doctor"])
|
|
assert "http://127.0.0.1:8789/livez" in seen
|
|
assert not any("127.0.0.1:8787" in url for url in seen)
|
|
|
|
def test_names_the_live_port_the_agents_are_configured_for(self, runner, isolated, monkeypatch):
|
|
"""Headroom Desktop runs on 6767 and records no deployment manifest."""
|
|
monkeypatch.setenv("HEADROOM_PORT_DISCOVERY", "1")
|
|
(isolated / "settings.json").write_text(
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:6767"}}),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
def desktop_probe(url, timeout=2.0):
|
|
return LIVEZ_OK if url == "http://127.0.0.1:6767/livez" else None
|
|
|
|
monkeypatch.setattr(doctor_mod, "probe_json", desktop_probe)
|
|
result = runner.invoke(main, ["doctor", "--json"])
|
|
proxy = next(c for c in json.loads(result.output)["checks"] if c["name"] == "proxy")
|
|
assert "a Headroom proxy IS running on port 6767" in proxy["summary"]
|
|
|
|
@pytest.mark.parametrize(
|
|
("relative", "content"),
|
|
[
|
|
("project-codex.toml", 'openai_base_url = "http://127.0.0.1:6767/v1"\n'),
|
|
(
|
|
".claude/settings.local.json",
|
|
json.dumps({"env": {"ANTHROPIC_BASE_URL": "http://127.0.0.1:6767"}}),
|
|
),
|
|
],
|
|
)
|
|
def test_names_the_live_port_only_a_project_config_routes_to(
|
|
self, runner, isolated, monkeypatch, relative, content
|
|
):
|
|
monkeypatch.setenv("HEADROOM_PORT_DISCOVERY", "1")
|
|
monkeypatch.chdir(isolated)
|
|
path = isolated / relative
|
|
path.parent.mkdir(exist_ok=True)
|
|
path.write_text(content, encoding="utf-8")
|
|
|
|
def desktop_probe(url, timeout=2.0):
|
|
return LIVEZ_OK if url == "http://127.0.0.1:6767/livez" else None
|
|
|
|
monkeypatch.setattr(doctor_mod, "probe_json", desktop_probe)
|
|
result = runner.invoke(main, ["doctor", "--json"])
|
|
proxy = next(c for c in json.loads(result.output)["checks"] if c["name"] == "proxy")
|
|
assert "a Headroom proxy IS running on port 6767" in proxy["summary"]
|
|
|
|
|
|
class TestCostTrackerBudgetKeys:
|
|
def test_stats_exposes_budget_config(self):
|
|
from headroom.proxy.cost import CostTracker
|
|
|
|
stats = CostTracker(budget_limit_usd=5.0, budget_period="monthly").stats()
|
|
assert stats["budget_limit_usd"] == 5.0
|
|
assert stats["budget_period"] == "monthly"
|
|
|
|
def test_stats_budget_none_when_unset(self):
|
|
from headroom.proxy.cost import CostTracker
|
|
|
|
assert CostTracker().stats()["budget_limit_usd"] is None
|