mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(wrap): never reuse a proxy with incompatible routing config (#3201)
## Description When `headroom wrap <tool> --port P` finds a proxy already listening, `_ensure_proxy_unlocked` decides reuse-vs-restart by comparing feature flags (`memory`, `learn`, `code_graph`, savings profile) and, only when the caller explicitly requests them, `openai_api_url`/`vertex_api_url`. Two gaps make this silently dangerous: 1. **The backend is never compared** in the shared path (`wrap copilot` has its own guard; every other tool does not). 2. **URL checks are asymmetric**: a proxy started with a non-default upstream (e.g. a Copilot tenant URL, or an augment/factory upstream from a feature-branch build) is "compatible" with every plain wrap that requests nothing, so unrelated sessions silently inherit captive routing. Observed in production: `headroom wrap codex --port 8788` reused a 21-day-old augment-routed proxy (`backend=anthropic`, `augment_api_url` set, zero clients) because nothing in the reuse check objected; the session ended up outside Headroom while the wrap reported "Proxy already running". This PR adds `_proxy_routing_mismatches` (symmetric comparison of effective backend, including `HEADROOM_BACKEND`, plus all upstream-URL keys, where "running has it set, caller did not request it" is a mismatch) and, on mismatch: - **zero live clients**: kill the proxy via the `pid` exposed in `/health` config and restart with the requested config (existing missing-flags mechanics); - **live clients attached**: never reuse and never disrupt — start a dedicated proxy on the next free port (the `wrap auggie` "different session" precedent); - **persistent deployment with routing mismatch**: leave the deployment alone (its routing comes from the manifest; killing its proxy from the ephemeral path orphans the manifest) and start a dedicated proxy on a free port; - **proxy exposes no config block at all**: never reused (it cannot be compatibility-checked and reports no PID to restart with) — dedicated port instead. Closes #3198 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/cli/wrap.py`: new `_PROXY_ROUTING_URL_KEYS` + `_proxy_routing_mismatches` helper (module-level so tests and future callers share it; URL keys absent from a running proxy's config compare equal to unrequested/`None`, so they are inert, never wrong, against builds that do not expose them). Wired into the persistent-deployment reuse gate (mismatch now diverts to a dedicated port instead of falling through to the ephemeral kill path) and into the ephemeral/shared branch (mismatch restarts when idle, port-bumps when clients are attached; configless proxies are no longer reused). - `tests/test_cli/test_wrap_persistent.py`: helper unit tests (backend via arg and `HEADROOM_BACKEND`, symmetric URL mismatch both directions, normalization tolerance, missing-key tolerance), restart-on-routing-mismatch, port-bump-with-clients, configless-no-reuse, persistent-divert, happy-path reuse. The pre-existing `..._restarts_persistent_deployment_for_feature_mismatch` test asserted the old kill-and-hijack behavior for a routing mismatch; updated to the divert semantic with an explanatory comment. - `tests/test_cli/test_wrap_helpers.py`: the dashboard-URL regression test now mocks a config-bearing proxy (configless is covered by the new dedicated-port test). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_persistent.py -q 38 passed in 0.45s $ uv run pytest tests/test_cli/test_wrap_helpers.py tests/test_cli/test_wrap_persistent.py -q 96 passed in 0.56s $ uv run pytest tests/test_cli -q -k 'wrap or unwrap or proxy' 525 passed, 1 skipped, 182 deselected in 12.27s $ uv run ruff format headroom/cli/wrap.py tests/test_cli/test_wrap_persistent.py tests/test_cli/test_wrap_helpers.py 3 files left unchanged $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_persistent.py tests/test_cli/test_wrap_helpers.py All checks passed! $ uv run mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS 26.5 arm64, Python 3.13, local source build 0.37.0-dev (integration overlay with this PR applied), live ports: 8789 (plain 0.37.0-dev proxy), 8791 free. - Exact command / steps: with the plain proxy healthy on 8789, `python -c "from headroom.cli import wrap as w; w._ensure_proxy(8789, False, agent_type='auggie', augment_api_url='https://xlb.api.augmentcode.com')"`, then `GET /health` on the resulting port. - Observed result: `Proxy on port 8789 has incompatible routing config (augment_api_url); restarting with the requested config...` — the plain proxy was killed and an augment-routed proxy came up (on 8791: the just-freed 8789 lost the bind race and 8790 was held by another stale listener; the fallback messaging named the port change). `/health` on 8791: `version 0.37.0-dev, augment_api_url https://xlb.api.augmentcode.com`. Before this PR the identical call printed "Proxy already running on port 8789" and reused the plain proxy. - Not tested: the attached-clients port-bump path against real live traffic (unit-tested with mocked markers); the persistent-deployment divert against a real `install agent` deployment (unit-tested); Windows. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: yes — wraps no longer silently reuse proxies whose backend/upstream routing differs, and no longer reuse configless proxies. Previously-correct reuse (identical config) is unchanged and still prints the dashboard URL. - Kill switch / disable path: `headroom wrap <tool> --no-proxy` bypasses the check entirely (pre-existing). - Unsafe override required: no - Qualification impact: none - Rollback path: revert the commit; the previous reuse heuristic returns. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Adversarially reviewed by a second model (glm-5.2, high reasoning); initial verdict needs-work, all accepted findings applied (configless no-reuse, persistent-divert, honest docstring about which URL keys this build's own `/health` exposes) and re-tested green. Companion PRs (same incident): `-dev` version-restart fix, orphan-proxy watchdog. Follow-up for the auggie/droid feature branches: expose `augment_api_url`/`factory_api_url` in their `/health` config payloads so same-build detection works there too (this PR already catches older/foreign builds that expose them).
This commit is contained in:
+280
-121
@@ -3668,12 +3668,12 @@ def _proxy_active_session_count(payload: dict[str, Any] | None) -> int:
|
||||
return max(counts, default=0)
|
||||
|
||||
|
||||
def _normalize_proxy_api_url(url: object) -> str | None:
|
||||
def _normalize_proxy_api_url(url: object, *, strip_provider_v1: bool = True) -> str | None:
|
||||
"""Normalize configured upstream URLs for running-proxy comparisons."""
|
||||
if not isinstance(url, str):
|
||||
return None
|
||||
normalized = url.strip().rstrip("/")
|
||||
if normalized.endswith("/v1"):
|
||||
if strip_provider_v1 and normalized.endswith("/v1"):
|
||||
normalized = normalized[:-3]
|
||||
return normalized or None
|
||||
|
||||
@@ -3700,6 +3700,95 @@ def _proxy_needs_version_restart(payload: dict[str, Any] | None) -> bool:
|
||||
)
|
||||
|
||||
|
||||
# Upstream-URL config keys that decide where a running proxy forwards
|
||||
# traffic. A running proxy is only reusable when every key matches the
|
||||
# requested value. This build's /health config payload only serializes the
|
||||
# first five keys; the augment/factory entries catch proxies built from
|
||||
# feature branches or other versions that DO expose them (exactly the
|
||||
# mixed-fleet shape of the 2026-08-22 incident, where a 0.34-era proxy
|
||||
# reported augment_api_url). Keys absent from the running config simply
|
||||
# compare equal to an unrequested (None) value, so they are inert, never
|
||||
# wrong, against proxies that do not expose them.
|
||||
_PROXY_ROUTING_URL_KEYS = (
|
||||
"openai_api_url",
|
||||
"anthropic_api_url",
|
||||
"vertex_api_url",
|
||||
"cloudcode_api_url",
|
||||
"gemini_api_url",
|
||||
"augment_api_url",
|
||||
"factory_api_url",
|
||||
)
|
||||
|
||||
|
||||
def _proxy_routing_mismatches(
|
||||
running_config: Mapping[str, Any],
|
||||
*,
|
||||
backend: str | None,
|
||||
requested_api_urls: Mapping[str, str | None] | None = None,
|
||||
) -> list[str]:
|
||||
"""Return routing-level keys where a running proxy differs from the request.
|
||||
|
||||
Unlike the feature-flag check in ``_ensure_proxy_unlocked`` (memory,
|
||||
learn, code graph), a mismatch here is never "degraded but usable": the
|
||||
proxy would forward this session's traffic to a backend or upstream the
|
||||
caller did not ask for. URL comparison is symmetric: a running proxy with
|
||||
a non-default upstream the caller did not request is a mismatch too.
|
||||
"""
|
||||
mismatches: list[str] = []
|
||||
effective_backend = backend or os.environ.get("HEADROOM_BACKEND") or "anthropic"
|
||||
running_backend = running_config.get("backend")
|
||||
if (
|
||||
isinstance(running_backend, str)
|
||||
and running_backend
|
||||
and running_backend != effective_backend
|
||||
):
|
||||
mismatches.append("backend")
|
||||
requested_api_urls = requested_api_urls or {}
|
||||
for key in _PROXY_ROUTING_URL_KEYS:
|
||||
strip_provider_v1 = key not in {"augment_api_url", "factory_api_url"}
|
||||
if _normalize_proxy_api_url(
|
||||
running_config.get(key),
|
||||
strip_provider_v1=strip_provider_v1,
|
||||
) != _normalize_proxy_api_url(
|
||||
requested_api_urls.get(key),
|
||||
strip_provider_v1=strip_provider_v1,
|
||||
):
|
||||
mismatches.append(key)
|
||||
return mismatches
|
||||
|
||||
|
||||
def _effective_requested_proxy_routing(
|
||||
*,
|
||||
backend: str | None,
|
||||
openai_api_url: str | None,
|
||||
anthropic_api_url: str | None,
|
||||
vertex_api_url: str | None,
|
||||
clear_vertex_api_url: bool,
|
||||
) -> tuple[str, dict[str, str | None]]:
|
||||
"""Resolve the routing a newly started proxy would inherit."""
|
||||
from headroom.providers.registry import resolve_api_overrides
|
||||
|
||||
overrides = resolve_api_overrides(
|
||||
anthropic_api_url=anthropic_api_url,
|
||||
openai_api_url=openai_api_url,
|
||||
gemini_api_url=None,
|
||||
cloudcode_api_url=None,
|
||||
vertex_api_url=vertex_api_url,
|
||||
)
|
||||
return (
|
||||
backend or os.environ.get("HEADROOM_BACKEND") or "anthropic",
|
||||
{
|
||||
"anthropic_api_url": overrides.anthropic,
|
||||
"openai_api_url": overrides.openai,
|
||||
"gemini_api_url": overrides.gemini,
|
||||
"cloudcode_api_url": overrides.cloudcode,
|
||||
"vertex_api_url": None if clear_vertex_api_url else overrides.vertex,
|
||||
"augment_api_url": os.environ.get("AUGMENT_TARGET_API_URL"),
|
||||
"factory_api_url": os.environ.get("FACTORY_TARGET_API_URL"),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _detect_running_proxy_backend(port: int) -> str | None:
|
||||
"""Read the backend of an already-running proxy from its health endpoint."""
|
||||
return _copilot_detect_running_proxy_backend(port)
|
||||
@@ -4161,6 +4250,18 @@ def _ensure_proxy_unlocked(
|
||||
or bool(copilot_refresh_oauth_token)
|
||||
or copilot_api_token_expires_at is not None
|
||||
)
|
||||
requested_backend, requested_api_urls = helpers._effective_requested_proxy_routing(
|
||||
backend=backend,
|
||||
openai_api_url=openai_api_url,
|
||||
anthropic_api_url=anthropic_api_url,
|
||||
vertex_api_url=vertex_api_url,
|
||||
clear_vertex_api_url=clear_vertex_api_url,
|
||||
)
|
||||
# Set True when the proxy on the requested port belongs to a persistent
|
||||
# deployment whose routing config does not match this wrap: the deployment
|
||||
# owns the port and must not be reused, killed, or restarted from the
|
||||
# shared ephemeral path; the wrap starts a dedicated proxy elsewhere.
|
||||
persistent_routing_mismatch = False
|
||||
# --no-proxy reuses an already-running proxy, so backend/region/provider
|
||||
# flags (which only apply when we start one) would be silently dropped.
|
||||
if no_proxy and (backend or anyllm_provider or region):
|
||||
@@ -4183,7 +4284,34 @@ def _ensure_proxy_unlocked(
|
||||
|
||||
if probe_ready(manifest.health_url):
|
||||
health_payload = helpers._query_proxy_health(port)
|
||||
if helpers._proxy_needs_version_restart(health_payload):
|
||||
running_config = helpers._proxy_health_config(health_payload)
|
||||
if running_config is None:
|
||||
running_config = helpers._query_proxy_config(port)
|
||||
routing_mismatches = (
|
||||
None
|
||||
if running_config is None
|
||||
else helpers._proxy_routing_mismatches(
|
||||
running_config,
|
||||
backend=requested_backend,
|
||||
requested_api_urls=requested_api_urls,
|
||||
)
|
||||
)
|
||||
if running_config is None:
|
||||
click.echo(
|
||||
f" Persistent deployment '{manifest.profile}' on port {port} "
|
||||
"did not expose routing config; starting a dedicated proxy "
|
||||
"on a different port instead."
|
||||
)
|
||||
persistent_routing_mismatch = True
|
||||
elif routing_mismatches:
|
||||
keys_str = ", ".join(routing_mismatches)
|
||||
click.echo(
|
||||
f" Persistent deployment '{manifest.profile}' on port {port} "
|
||||
f"has incompatible routing config ({keys_str}); "
|
||||
"starting a dedicated proxy on a different port instead."
|
||||
)
|
||||
persistent_routing_mismatch = True
|
||||
elif helpers._proxy_needs_version_restart(health_payload):
|
||||
running_version = helpers._proxy_version(health_payload) or "unknown"
|
||||
active_sessions = helpers._proxy_active_session_count(health_payload)
|
||||
other_wrappers = helpers._live_proxy_clients(port, exclude_self=True)
|
||||
@@ -4208,14 +4336,8 @@ def _ensure_proxy_unlocked(
|
||||
f"Persistent deployment '{manifest.profile}' on port {port} "
|
||||
f"is running stale Headroom {running_version} and could not be restarted."
|
||||
)
|
||||
# Check if the running proxy has the features we need.
|
||||
# Without this, a persistent deployment started for one use case
|
||||
# (e.g. --backend anthropic) would be silently reused for another
|
||||
# (e.g. --subscription --provider-type openai) causing auth failures.
|
||||
running_config = helpers._proxy_health_config(health_payload)
|
||||
if running_config is None:
|
||||
running_config = helpers._query_proxy_config(port)
|
||||
if running_config is not None:
|
||||
else:
|
||||
# Check if the running proxy has the features we need.
|
||||
missing = []
|
||||
if memory and not running_config.get("memory"):
|
||||
missing.append("memory")
|
||||
@@ -4223,46 +4345,18 @@ def _ensure_proxy_unlocked(
|
||||
missing.append("learn")
|
||||
if code_graph and not running_config.get("code_graph"):
|
||||
missing.append("code_graph")
|
||||
if copilot_subscription_seed_requested:
|
||||
missing.append("copilot-subscription-auth")
|
||||
if openai_api_url:
|
||||
running_openai_url = _normalize_proxy_api_url(
|
||||
running_config.get("openai_api_url")
|
||||
)
|
||||
requested_openai_url = _normalize_proxy_api_url(openai_api_url)
|
||||
if running_openai_url != requested_openai_url:
|
||||
missing.append("openai-api-url")
|
||||
if not missing:
|
||||
click.echo(f" Proxy already running on port {port}")
|
||||
click.echo(f" Dashboard: http://127.0.0.1:{port}/dashboard")
|
||||
return None, port
|
||||
# Features mismatch or config unavailable — fall through to
|
||||
# the non-persistent path which handles proxy restart.
|
||||
# Features mismatch or config unavailable — fall through to the
|
||||
# non-persistent path which handles proxy restart. A routing
|
||||
# mismatch skips that path entirely (see flag above).
|
||||
else:
|
||||
if helpers._recover_persistent_proxy(port):
|
||||
# If the caller requested feature-sensitive config (e.g.
|
||||
# openai_api_url for Copilot subscription), continue into
|
||||
# the shared running-proxy checks below so mismatch-driven
|
||||
# restart logic can run. For plain recover-only calls,
|
||||
# preserve the historical fast return.
|
||||
if not any(
|
||||
(
|
||||
memory,
|
||||
learn,
|
||||
code_graph,
|
||||
openai_api_url,
|
||||
copilot_subscription_seed_requested,
|
||||
)
|
||||
):
|
||||
return None, port
|
||||
if not helpers._check_proxy(port):
|
||||
return None, port
|
||||
|
||||
# A freshly recovered persistent proxy may not expose
|
||||
# a full config payload yet. In feature-sensitive flows
|
||||
# (e.g. Copilot subscription), treat missing or mismatched
|
||||
# config as restart-required and refresh the persistent
|
||||
# deployment directly instead of silently reusing it.
|
||||
health_payload = helpers._query_proxy_health(port)
|
||||
running_config = helpers._proxy_health_config(health_payload)
|
||||
if running_config is None:
|
||||
@@ -4271,61 +4365,93 @@ def _ensure_proxy_unlocked(
|
||||
if running_config is None:
|
||||
click.echo(
|
||||
f" Recovered persistent deployment '{manifest.profile}' "
|
||||
"did not expose config; restarting with requested features..."
|
||||
"did not expose routing config; starting a dedicated proxy "
|
||||
"on a different port instead."
|
||||
)
|
||||
if helpers._restart_persistent_proxy(manifest, port):
|
||||
return None, port
|
||||
raise click.ClickException(
|
||||
f"Persistent deployment '{manifest.profile}' on port {port} "
|
||||
"could not be restarted after recovery."
|
||||
persistent_routing_mismatch = True
|
||||
else:
|
||||
routing_mismatches = helpers._proxy_routing_mismatches(
|
||||
running_config,
|
||||
backend=requested_backend,
|
||||
requested_api_urls=requested_api_urls,
|
||||
)
|
||||
if routing_mismatches:
|
||||
keys_str = ", ".join(routing_mismatches)
|
||||
click.echo(
|
||||
f" Recovered persistent deployment '{manifest.profile}' "
|
||||
f"has incompatible routing config ({keys_str}); "
|
||||
"starting a dedicated proxy on a different port instead."
|
||||
)
|
||||
persistent_routing_mismatch = True
|
||||
else:
|
||||
missing = []
|
||||
if memory and not running_config.get("memory"):
|
||||
missing.append("memory")
|
||||
if learn and not running_config.get("learn"):
|
||||
missing.append("learn")
|
||||
if code_graph and not running_config.get("code_graph"):
|
||||
missing.append("code-graph")
|
||||
|
||||
missing = []
|
||||
if memory and not running_config.get("memory"):
|
||||
missing.append("memory")
|
||||
if learn and not running_config.get("learn"):
|
||||
missing.append("learn")
|
||||
if code_graph and not running_config.get("code_graph"):
|
||||
missing.append("code-graph")
|
||||
if copilot_subscription_seed_requested:
|
||||
missing.append("copilot-subscription-auth")
|
||||
if openai_api_url:
|
||||
running_openai_url = _normalize_proxy_api_url(
|
||||
running_config.get("openai_api_url")
|
||||
)
|
||||
requested_openai_url = _normalize_proxy_api_url(openai_api_url)
|
||||
if running_openai_url != requested_openai_url:
|
||||
missing.append("openai-api-url")
|
||||
|
||||
if missing:
|
||||
flags_str = ", ".join(f"--{f}" for f in missing)
|
||||
click.echo(
|
||||
f" Recovered persistent deployment '{manifest.profile}' is missing: "
|
||||
f"{flags_str}; restarting..."
|
||||
)
|
||||
if helpers._restart_persistent_proxy(manifest, port):
|
||||
return None, port
|
||||
raise click.ClickException(
|
||||
f"Persistent deployment '{manifest.profile}' on port {port} "
|
||||
"could not be restarted with requested features."
|
||||
)
|
||||
return None, port
|
||||
if not missing:
|
||||
return None, port
|
||||
flags_str = ", ".join(f"--{f}" for f in missing)
|
||||
click.echo(
|
||||
f" Recovered persistent deployment '{manifest.profile}' "
|
||||
f"is missing: {flags_str}; restarting..."
|
||||
)
|
||||
if helpers._restart_persistent_proxy(manifest, port):
|
||||
return None, port
|
||||
raise click.ClickException(
|
||||
f"Persistent deployment '{manifest.profile}' on port {port} "
|
||||
"could not be restarted with requested features."
|
||||
)
|
||||
elif helpers._check_proxy(port):
|
||||
raise click.ClickException(
|
||||
f"Persistent deployment '{manifest.profile}' on port {port} is not healthy."
|
||||
)
|
||||
click.echo(
|
||||
f" Warning: persistent deployment '{manifest.profile}' on port {port} "
|
||||
"is stale; starting a fresh proxy instead."
|
||||
)
|
||||
if not persistent_routing_mismatch:
|
||||
click.echo(
|
||||
f" Warning: persistent deployment '{manifest.profile}' on port {port} "
|
||||
"is stale; starting a fresh proxy instead."
|
||||
)
|
||||
|
||||
if not isolated_copilot_subscription_proxy and helpers._check_proxy(port):
|
||||
if (
|
||||
not isolated_copilot_subscription_proxy
|
||||
and not persistent_routing_mismatch
|
||||
and helpers._check_proxy(port)
|
||||
):
|
||||
# Proxy is running — check if it has the features we need
|
||||
needs_restart = False
|
||||
# Set False when the running proxy must not serve this session at
|
||||
# all (routing-level mismatch with live clients attached): fall
|
||||
# through to a fresh start on a different port.
|
||||
reuse_running = True
|
||||
health_payload = helpers._query_proxy_health(port)
|
||||
running_config = helpers._proxy_health_config(health_payload)
|
||||
if running_config is None:
|
||||
running_config = helpers._query_proxy_config(port)
|
||||
routing_mismatches = (
|
||||
None
|
||||
if running_config is None
|
||||
else helpers._proxy_routing_mismatches(
|
||||
running_config,
|
||||
backend=requested_backend,
|
||||
requested_api_urls=requested_api_urls,
|
||||
)
|
||||
)
|
||||
if running_config is None and not helpers._proxy_needs_version_restart(health_payload):
|
||||
# A proxy that exposes no config block cannot be checked for
|
||||
# feature or routing compatibility, and without a reported PID
|
||||
# it cannot be restarted either. Reusing it blindly is the
|
||||
# silent-misrouting incident class; start a dedicated proxy on
|
||||
# a free port instead (mirrors the persistent branch, which
|
||||
# already treats config-unavailable as not-reusable).
|
||||
click.echo(
|
||||
f" Proxy on port {port} did not expose its config; "
|
||||
"starting a dedicated proxy on a different port instead of "
|
||||
"reusing it unchecked."
|
||||
)
|
||||
reuse_running = False
|
||||
|
||||
if helpers._proxy_needs_version_restart(health_payload):
|
||||
running_version = helpers._proxy_version(health_payload) or "unknown"
|
||||
@@ -4345,28 +4471,37 @@ def _ensure_proxy_unlocked(
|
||||
f" Proxy on port {port} is running Headroom {running_version}; "
|
||||
f"current CLI is {_HEADROOM_VERSION}."
|
||||
)
|
||||
click.echo(
|
||||
f" Leaving it running because {detail} "
|
||||
"are still attached; it will be restarted when idle."
|
||||
)
|
||||
return None, port
|
||||
if routing_mismatches or running_config is None:
|
||||
click.echo(
|
||||
f" Not reusing it because its routing "
|
||||
f"{'cannot be verified' if running_config is None else 'is incompatible'}; "
|
||||
"starting a dedicated proxy on a different port instead."
|
||||
)
|
||||
reuse_running = False
|
||||
else:
|
||||
click.echo(
|
||||
f" Leaving it running because {detail} "
|
||||
"are still attached; it will be restarted when idle."
|
||||
)
|
||||
return None, port
|
||||
|
||||
click.echo(
|
||||
f" Proxy on port {port} is running Headroom {running_version}; "
|
||||
f"restarting with {_HEADROOM_VERSION}..."
|
||||
)
|
||||
proxy_pid = running_config.get("pid") if running_config is not None else None
|
||||
if proxy_pid is None:
|
||||
raise click.ClickException(
|
||||
f"Proxy on port {port} is stale but did not expose a PID. "
|
||||
"Stop it manually and retry."
|
||||
else:
|
||||
click.echo(
|
||||
f" Proxy on port {port} is running Headroom {running_version}; "
|
||||
f"restarting with {_HEADROOM_VERSION}..."
|
||||
)
|
||||
if not helpers._kill_proxy_by_pid(int(proxy_pid), port):
|
||||
raise click.ClickException(
|
||||
f"Failed to stop stale proxy (PID {proxy_pid}) on port {port}. "
|
||||
"Stop it manually and retry."
|
||||
)
|
||||
needs_restart = True
|
||||
proxy_pid = running_config.get("pid") if running_config is not None else None
|
||||
if proxy_pid is None:
|
||||
raise click.ClickException(
|
||||
f"Proxy on port {port} is stale but did not expose a PID. "
|
||||
"Stop it manually and retry."
|
||||
)
|
||||
if not helpers._kill_proxy_by_pid(int(proxy_pid), port):
|
||||
raise click.ClickException(
|
||||
f"Failed to stop stale proxy (PID {proxy_pid}) on port {port}. "
|
||||
"Stop it manually and retry."
|
||||
)
|
||||
needs_restart = True
|
||||
|
||||
if running_config is not None:
|
||||
missing = []
|
||||
@@ -4376,30 +4511,14 @@ def _ensure_proxy_unlocked(
|
||||
missing.append("learn")
|
||||
if code_graph and not running_config.get("code_graph"):
|
||||
missing.append("code_graph")
|
||||
if copilot_subscription_seed_requested:
|
||||
missing.append("copilot-subscription-auth")
|
||||
expected_savings_profile = helpers._wrap_agent_savings_profile(agent_type)
|
||||
if (
|
||||
expected_savings_profile is not None
|
||||
and running_config.get("savings_profile") != expected_savings_profile
|
||||
):
|
||||
missing.append("savings-profile")
|
||||
if openai_api_url:
|
||||
running_openai_url = _normalize_proxy_api_url(
|
||||
running_config.get("openai_api_url")
|
||||
)
|
||||
requested_openai_url = _normalize_proxy_api_url(openai_api_url)
|
||||
if running_openai_url != requested_openai_url:
|
||||
missing.append("openai-api-url")
|
||||
if vertex_api_url or clear_vertex_api_url:
|
||||
running_vertex_url = _normalize_proxy_api_url(
|
||||
running_config.get("vertex_api_url")
|
||||
)
|
||||
requested_vertex_url = _normalize_proxy_api_url(vertex_api_url)
|
||||
if running_vertex_url != requested_vertex_url:
|
||||
missing.append("vertex-api-url")
|
||||
|
||||
if missing:
|
||||
if missing and not routing_mismatches:
|
||||
flags_str = ", ".join(
|
||||
f if f.startswith("--") else f"--{f.replace('_', '-')}" for f in missing
|
||||
)
|
||||
@@ -4444,7 +4563,47 @@ def _ensure_proxy_unlocked(
|
||||
)
|
||||
return None, port
|
||||
|
||||
if not needs_restart:
|
||||
# Routing-level config (backend, upstream URLs) must match in
|
||||
# both directions. Unlike missing feature flags, reusing a
|
||||
# proxy that forwards to a backend/upstream the caller did not
|
||||
# request is broken, not degraded, so "reuse as-is" is never an
|
||||
# option here (cf. the factory-api-url fail-loud precedent on
|
||||
# the droid branch and the tenant check in wrap auggie).
|
||||
routing_mismatches = [] if needs_restart else (routing_mismatches or [])
|
||||
if routing_mismatches:
|
||||
keys_str = ", ".join(routing_mismatches)
|
||||
other_wrappers = helpers._live_proxy_clients(port, exclude_self=True)
|
||||
active_sessions = helpers._proxy_active_session_count(health_payload)
|
||||
if other_wrappers or active_sessions > 0:
|
||||
click.echo(
|
||||
f" Proxy on port {port} has incompatible routing config "
|
||||
f"({keys_str}), but live session(s) are attached."
|
||||
)
|
||||
click.echo(
|
||||
" Starting a dedicated proxy on a different port instead "
|
||||
"of reusing or disrupting it."
|
||||
)
|
||||
reuse_running = False
|
||||
else:
|
||||
click.echo(
|
||||
f" Proxy on port {port} has incompatible routing config "
|
||||
f"({keys_str}); restarting with the requested config..."
|
||||
)
|
||||
proxy_pid = running_config.get("pid")
|
||||
if proxy_pid is None:
|
||||
raise click.ClickException(
|
||||
f"Proxy on port {port} has incompatible routing config "
|
||||
f"({keys_str}) but did not expose a PID. "
|
||||
"Stop it manually and retry."
|
||||
)
|
||||
if not helpers._kill_proxy_by_pid(int(proxy_pid), port):
|
||||
raise click.ClickException(
|
||||
f"Failed to stop incompatible proxy (PID {proxy_pid}) "
|
||||
f"on port {port}. Stop it manually and retry."
|
||||
)
|
||||
needs_restart = True
|
||||
|
||||
if not needs_restart and reuse_running:
|
||||
click.echo(f" Proxy already running on port {port}")
|
||||
click.echo(f" Dashboard: http://127.0.0.1:{port}/dashboard")
|
||||
return None, port
|
||||
|
||||
@@ -847,12 +847,16 @@ def test_ensure_proxy_already_running_prints_dashboard_url(
|
||||
wraps (the common case) never told the user where the dashboard lives.
|
||||
"""
|
||||
port = 1234
|
||||
# A compatible running proxy exposes its config; proxies without a config
|
||||
# block are no longer reused (they cannot be compatibility-checked) and
|
||||
# are covered by test_ensure_proxy_never_reuses_configless_proxy.
|
||||
running_config = {"pid": 4321, "backend": "anthropic"}
|
||||
monkeypatch.setattr(wrap_mod, "_find_persistent_manifest", lambda _p: None)
|
||||
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda _p: True)
|
||||
monkeypatch.setattr(wrap_mod, "_query_proxy_health", lambda _p: {})
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_needs_version_restart", lambda _h: False)
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_health_config", lambda _h: None)
|
||||
monkeypatch.setattr(wrap_mod, "_query_proxy_config", lambda _p: None)
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_health_config", lambda _h: running_config)
|
||||
monkeypatch.setattr(wrap_mod, "_live_proxy_clients", lambda *a, **kw: [])
|
||||
|
||||
output = _run_in_click_context(lambda: wrap_mod._ensure_proxy(port, no_proxy=False))
|
||||
|
||||
|
||||
@@ -17,6 +17,17 @@ def _no_attached_wrappers(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
flaky. Individual tests override this to simulate attached wrappers.
|
||||
"""
|
||||
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *a, **kw: [])
|
||||
for name in (
|
||||
"ANTHROPIC_TARGET_API_URL",
|
||||
"OPENAI_TARGET_API_URL",
|
||||
"GEMINI_TARGET_API_URL",
|
||||
"CLOUDCODE_TARGET_API_URL",
|
||||
"VERTEX_TARGET_API_URL",
|
||||
"AUGMENT_TARGET_API_URL",
|
||||
"FACTORY_TARGET_API_URL",
|
||||
"HEADROOM_BACKEND",
|
||||
):
|
||||
monkeypatch.delenv(name, raising=False)
|
||||
|
||||
|
||||
class _Manifest:
|
||||
@@ -67,6 +78,18 @@ def test_ensure_proxy_recovers_persistent_deployment_when_socket_is_bound(monkey
|
||||
monkeypatch.setattr(
|
||||
"headroom.install.runtime.wait_ready", lambda manifest, timeout_seconds=45: True
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_query_proxy_health",
|
||||
lambda port: {"version": wrap_cli._HEADROOM_VERSION, "config": {"backend": "anthropic"}},
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("matching recovered proxy should be reused")
|
||||
),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
@@ -374,6 +397,694 @@ def test_ensure_proxy_restarts_ephemeral_proxy_for_openai_api_url_mismatch(monke
|
||||
assert calls[1][2]["openai_api_url"] == "https://api.individual.githubcopilot.com"
|
||||
|
||||
|
||||
def test_proxy_routing_mismatches_detects_backend_and_urls(monkeypatch) -> None:
|
||||
monkeypatch.delenv("HEADROOM_BACKEND", raising=False)
|
||||
check = wrap_cli._proxy_routing_mismatches
|
||||
|
||||
# Stock proxy vs stock request: compatible.
|
||||
assert check({"backend": "anthropic"}, backend=None) == []
|
||||
# Running non-default backend vs default request: mismatch.
|
||||
assert check({"backend": "anyllm"}, backend=None) == ["backend"]
|
||||
# Requested backend vs different running backend: mismatch.
|
||||
assert check({"backend": "anthropic"}, backend="anyllm") == ["backend"]
|
||||
# HEADROOM_BACKEND env feeds the effective requested backend.
|
||||
monkeypatch.setenv("HEADROOM_BACKEND", "anyllm")
|
||||
assert check({"backend": "anyllm"}, backend=None) == []
|
||||
monkeypatch.delenv("HEADROOM_BACKEND")
|
||||
|
||||
# Running proxy carries an upstream the caller never requested (the
|
||||
# 2026-08-22 incident: a wrap-auggie proxy reused by wrap codex).
|
||||
assert check(
|
||||
{"backend": "anthropic", "augment_api_url": "https://xlb.api.augmentcode.com"},
|
||||
backend=None,
|
||||
) == ["augment_api_url"]
|
||||
# Requested upstream the running proxy does not have: mismatch.
|
||||
assert check(
|
||||
{"backend": "anthropic"},
|
||||
backend=None,
|
||||
requested_api_urls={"openai_api_url": "https://api.githubcopilot.com"},
|
||||
) == ["openai_api_url"]
|
||||
# Same upstream on both sides, modulo normalization noise: compatible.
|
||||
assert (
|
||||
check(
|
||||
{"backend": "anthropic", "openai_api_url": "https://api.githubcopilot.com/v1/"},
|
||||
backend=None,
|
||||
requested_api_urls={"openai_api_url": "https://api.githubcopilot.com"},
|
||||
)
|
||||
== []
|
||||
)
|
||||
# Captive provider paths are semantic; /v1 is not a removable transport
|
||||
# convention for their REST catch-all routing.
|
||||
assert check(
|
||||
{"backend": "anthropic", "factory_api_url": "https://factory.example/v1"},
|
||||
backend=None,
|
||||
requested_api_urls={"factory_api_url": "https://factory.example"},
|
||||
) == ["factory_api_url"]
|
||||
# Missing backend key (older proxy): treated as unknown, not a mismatch.
|
||||
assert check({"openai_api_url": None}, backend=None) == []
|
||||
|
||||
|
||||
def test_effective_requested_proxy_routing_uses_startup_precedence(monkeypatch) -> None:
|
||||
monkeypatch.setenv("HEADROOM_BACKEND", "anyllm")
|
||||
monkeypatch.setenv("OPENAI_TARGET_API_URL", "https://env.openai.test")
|
||||
monkeypatch.setenv("VERTEX_TARGET_API_URL", "https://env.vertex.test")
|
||||
monkeypatch.setenv("AUGMENT_TARGET_API_URL", "https://env.augment.test/v1")
|
||||
monkeypatch.setenv("FACTORY_TARGET_API_URL", "https://env.factory.test/v1")
|
||||
|
||||
backend, api_urls = wrap_cli._effective_requested_proxy_routing(
|
||||
backend=None,
|
||||
openai_api_url="https://explicit.openai.test",
|
||||
anthropic_api_url=None,
|
||||
vertex_api_url=None,
|
||||
clear_vertex_api_url=True,
|
||||
)
|
||||
|
||||
assert backend == "anyllm"
|
||||
assert api_urls["openai_api_url"] == "https://explicit.openai.test"
|
||||
assert api_urls["vertex_api_url"] is None
|
||||
assert api_urls["augment_api_url"] == "https://env.augment.test/v1"
|
||||
assert api_urls["factory_api_url"] == "https://env.factory.test/v1"
|
||||
|
||||
|
||||
def test_ensure_proxy_restarts_idle_proxy_for_routing_mismatch(monkeypatch) -> None:
|
||||
"""A clientless proxy with captive routing (augment upstream) is restarted."""
|
||||
calls: list[object] = []
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": {
|
||||
"pid": "12345",
|
||||
"memory": False,
|
||||
"learn": False,
|
||||
"code_graph": False,
|
||||
"backend": "anthropic",
|
||||
"augment_api_url": "https://xlb.api.augmentcode.com",
|
||||
},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: len(calls) == 0)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *a, **kw: [])
|
||||
monkeypatch.setattr(wrap_cli, "_port_bind_error", lambda port: None)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda pid, port: calls.append(("kill", pid, port)) or True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8787
|
||||
assert calls[0] == ("kill", 12345, 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def _routing_mismatch_health(config_overrides: dict) -> dict:
|
||||
"""Shared fixture: running proxy with captive augment routing (mismatch)."""
|
||||
config = {
|
||||
"pid": "12345",
|
||||
"memory": False,
|
||||
"learn": False,
|
||||
"code_graph": False,
|
||||
"backend": "anthropic",
|
||||
"augment_api_url": "https://xlb.api.augmentcode.com",
|
||||
}
|
||||
config.update(config_overrides)
|
||||
return {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": config,
|
||||
}
|
||||
|
||||
|
||||
def test_ensure_proxy_routing_mismatch_without_pid_raises(monkeypatch) -> None:
|
||||
"""Incompatible proxy that exposes no PID cannot be restarted: fail loud."""
|
||||
health = _routing_mismatch_health({})
|
||||
del health["config"]["pid"]
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_port_bind_error", lambda port: None)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *a, **kw: (_ for _ in ()).throw(AssertionError("must not start")),
|
||||
)
|
||||
|
||||
with pytest.raises(click.ClickException, match="did not expose a PID"):
|
||||
wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
|
||||
def test_ensure_proxy_routing_mismatch_kill_failure_raises(monkeypatch) -> None:
|
||||
"""When the incompatible proxy survives the kill attempt: fail loud."""
|
||||
health = _routing_mismatch_health({})
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_port_bind_error", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_kill_proxy_by_pid", lambda pid, port: False)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *a, **kw: (_ for _ in ()).throw(AssertionError("must not start")),
|
||||
)
|
||||
|
||||
with pytest.raises(click.ClickException, match="Failed to stop incompatible proxy"):
|
||||
wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
|
||||
def test_ensure_proxy_bumps_port_for_routing_mismatch_with_attached_clients(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
"""With live clients attached, an incompatible proxy is neither reused
|
||||
nor restarted: the wrap starts a dedicated proxy on a free port."""
|
||||
calls: list[object] = []
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": {
|
||||
"pid": "12345",
|
||||
"backend": "anthropic",
|
||||
"augment_api_url": "https://xlb.api.augmentcode.com",
|
||||
},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: port == 8787)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *a, **kw: [999])
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kw: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("incompatible proxy with live clients must not be killed")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8799
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
assert calls[1][1][0] == 8799
|
||||
|
||||
|
||||
def test_ensure_proxy_reuses_proxy_with_matching_routing_config(monkeypatch) -> None:
|
||||
"""Happy path: identical routing config reuses the running proxy."""
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": {
|
||||
"pid": "12345",
|
||||
"memory": False,
|
||||
"learn": False,
|
||||
"code_graph": False,
|
||||
"backend": "anthropic",
|
||||
"openai_api_url": None,
|
||||
},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(AssertionError("must not kill")),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(AssertionError("must not start")),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8787
|
||||
|
||||
|
||||
def test_ensure_proxy_restarts_idle_proxy_for_backend_mismatch(monkeypatch) -> None:
|
||||
calls: list[object] = []
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": {"pid": "12345", "backend": "anyllm"},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: len(calls) == 0)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *a, **kw: [])
|
||||
monkeypatch.setattr(wrap_cli, "_port_bind_error", lambda port: None)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda pid, port: calls.append(("kill", pid, port)) or True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8787
|
||||
assert calls[0] == ("kill", 12345, 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def test_ensure_proxy_never_reuses_configless_proxy(monkeypatch) -> None:
|
||||
"""A proxy exposing no config block cannot be compatibility-checked; it
|
||||
must not be silently reused (the persistent branch already treats
|
||||
config-unavailable as fall-through). No PID is known, so the wrap starts
|
||||
a dedicated proxy on a free port instead of killing blindly."""
|
||||
calls: list[object] = []
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: port == 8787)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_config", lambda port: None)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kw: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("configless proxy exposes no PID; must not kill")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8799
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def test_ensure_proxy_routing_mismatch_on_persistent_deployment_uses_dedicated_port(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
"""A persistent deployment's routing comes from its manifest; a wrap with
|
||||
different routing must not reuse it AND must not let the ephemeral path
|
||||
kill it (the manifest would point at a hijacked port). The wrap starts a
|
||||
dedicated proxy on a free port and the deployment keeps running."""
|
||||
calls: list[object] = []
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": {
|
||||
"pid": "12345",
|
||||
"backend": "anthropic",
|
||||
"augment_api_url": "https://xlb.api.augmentcode.com",
|
||||
},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: _Manifest())
|
||||
monkeypatch.setattr("headroom.install.health.probe_ready", lambda url: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: port == 8787)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kw: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_restart_persistent_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("routing mismatch must not restart the deployment")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("routing mismatch must not kill the deployment's proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8799
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
assert calls[1][1][0] == 8799
|
||||
|
||||
|
||||
def test_ensure_proxy_healthy_persistent_uses_config_fallback(monkeypatch) -> None:
|
||||
config = {"pid": 12345, "backend": "anthropic"}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: _Manifest())
|
||||
monkeypatch.setattr("headroom.install.health.probe_ready", lambda url: True)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_query_proxy_health",
|
||||
lambda port: {"version": wrap_cli._HEADROOM_VERSION},
|
||||
)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_config", lambda port: config)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("matching persistent proxy should be reused")
|
||||
),
|
||||
)
|
||||
|
||||
assert wrap_cli._ensure_proxy(8787, False) == (None, 8787)
|
||||
|
||||
|
||||
def test_ensure_proxy_healthy_configless_persistent_uses_dedicated_port(monkeypatch) -> None:
|
||||
calls: list[object] = []
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: _Manifest())
|
||||
monkeypatch.setattr("headroom.install.health.probe_ready", lambda url: True)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_query_proxy_health",
|
||||
lambda port: {"version": wrap_cli._HEADROOM_VERSION},
|
||||
)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_config", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: port == 8787)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kwargs: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("configless persistent proxy must not be killed")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
assert wrap_cli._ensure_proxy(8787, False) == (None, 8799)
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("env_name", "config_key"),
|
||||
[
|
||||
("ANTHROPIC_TARGET_API_URL", "anthropic_api_url"),
|
||||
("OPENAI_TARGET_API_URL", "openai_api_url"),
|
||||
("GEMINI_TARGET_API_URL", "gemini_api_url"),
|
||||
("CLOUDCODE_TARGET_API_URL", "cloudcode_api_url"),
|
||||
("VERTEX_TARGET_API_URL", "vertex_api_url"),
|
||||
("AUGMENT_TARGET_API_URL", "augment_api_url"),
|
||||
("FACTORY_TARGET_API_URL", "factory_api_url"),
|
||||
],
|
||||
)
|
||||
def test_ensure_proxy_reuses_matching_inherited_routing_target(
|
||||
monkeypatch,
|
||||
env_name: str,
|
||||
config_key: str,
|
||||
) -> None:
|
||||
target = "https://tenant.example/api"
|
||||
for name in (
|
||||
"ANTHROPIC_TARGET_API_URL",
|
||||
"OPENAI_TARGET_API_URL",
|
||||
"GEMINI_TARGET_API_URL",
|
||||
"CLOUDCODE_TARGET_API_URL",
|
||||
"VERTEX_TARGET_API_URL",
|
||||
"AUGMENT_TARGET_API_URL",
|
||||
"FACTORY_TARGET_API_URL",
|
||||
):
|
||||
monkeypatch.delenv(name, raising=False)
|
||||
monkeypatch.setenv(env_name, target)
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": {"pid": 12345, "backend": "anthropic", config_key: target},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("matching inherited routing must reuse the proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("matching inherited routing must not kill the proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("matching inherited routing must not start another proxy")
|
||||
),
|
||||
)
|
||||
|
||||
assert wrap_cli._ensure_proxy(8787, False) == (None, 8787)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("ensure_kwargs", "config_overrides", "env_override"),
|
||||
[
|
||||
(
|
||||
{"anthropic_api_url": "https://expected.anthropic.test"},
|
||||
{"anthropic_api_url": "https://running.anthropic.test"},
|
||||
None,
|
||||
),
|
||||
(
|
||||
{"vertex_api_url": "https://expected.vertex.test"},
|
||||
{"vertex_api_url": "https://running.vertex.test"},
|
||||
None,
|
||||
),
|
||||
(
|
||||
{"backend": "anyllm"},
|
||||
{"backend": "anthropic"},
|
||||
None,
|
||||
),
|
||||
(
|
||||
{},
|
||||
{"factory_api_url": "https://running.factory.test"},
|
||||
("FACTORY_TARGET_API_URL", "https://expected.factory.test"),
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_ensure_proxy_recovered_routing_mismatch_uses_dedicated_port(
|
||||
monkeypatch,
|
||||
ensure_kwargs: dict[str, object],
|
||||
config_overrides: dict[str, object],
|
||||
env_override: tuple[str, str] | None,
|
||||
) -> None:
|
||||
calls: list[object] = []
|
||||
if env_override is not None:
|
||||
monkeypatch.setenv(*env_override)
|
||||
config = {"pid": 12345, "backend": "anthropic"}
|
||||
config.update(config_overrides)
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": config,
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: _Manifest())
|
||||
monkeypatch.setattr("headroom.install.health.probe_ready", lambda url: False)
|
||||
monkeypatch.setattr(wrap_cli, "_recover_persistent_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: port == 8787)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kwargs: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_restart_persistent_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("routing mismatch must not restart the persistent proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("routing mismatch must not kill the persistent proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
assert wrap_cli._ensure_proxy(8787, False, **ensure_kwargs) == (None, 8799)
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def test_ensure_proxy_stale_attached_persistent_routing_mismatch_uses_dedicated_port(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
calls: list[object] = []
|
||||
health = _routing_mismatch_health({})
|
||||
health["version"] = "0.0.1"
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: _Manifest())
|
||||
monkeypatch.setattr("headroom.install.health.probe_ready", lambda url: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *args, **kwargs: [999])
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: port == 8787)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kwargs: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_restart_persistent_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("active persistent proxy must not restart")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("active persistent proxy must not be killed")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
assert wrap_cli._ensure_proxy(8787, False, memory=True) == (None, 8799)
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def test_ensure_proxy_stale_attached_ephemeral_routing_mismatch_uses_dedicated_port(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
calls: list[object] = []
|
||||
health = _routing_mismatch_health({})
|
||||
health["version"] = "0.0.1"
|
||||
health["runtime"]["websocket_sessions"]["active_sessions"] = 1
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *args, **kwargs: [])
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: port == 8787)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kwargs: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("active proxy must not be killed")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
assert wrap_cli._ensure_proxy(8787, False, memory=True) == (None, 8799)
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("pid", "kill_result", "message"),
|
||||
[
|
||||
(None, True, "did not expose a PID"),
|
||||
(12345, False, "Failed to stop stale proxy"),
|
||||
],
|
||||
)
|
||||
def test_ensure_proxy_stale_idle_restart_failures_are_actionable(
|
||||
monkeypatch,
|
||||
pid: int | None,
|
||||
kill_result: bool,
|
||||
message: str,
|
||||
) -> None:
|
||||
config: dict[str, object] = {"backend": "anthropic"}
|
||||
if pid is not None:
|
||||
config["pid"] = pid
|
||||
health = {
|
||||
"version": "0.0.1",
|
||||
"runtime": {"websocket_sessions": {"active_sessions": 0, "active_relay_tasks": 0}},
|
||||
"config": config,
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_kill_proxy_by_pid", lambda found_pid, port: kill_result)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("failed stale restart must not start another proxy")
|
||||
),
|
||||
)
|
||||
|
||||
with pytest.raises(click.ClickException, match=message):
|
||||
wrap_cli._ensure_proxy(8787, False)
|
||||
|
||||
|
||||
def test_ensure_proxy_starts_isolated_ephemeral_proxy_for_copilot_subscription_seed(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
@@ -741,8 +1452,14 @@ def test_ensure_proxy_restarts_for_flags_when_no_other_wrapper(monkeypatch) -> N
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def test_ensure_proxy_restarts_persistent_deployment_for_feature_mismatch(monkeypatch) -> None:
|
||||
"""Persistent deployment should restart when requested features differ from running config."""
|
||||
def test_ensure_proxy_diverts_persistent_deployment_for_routing_mismatch(
|
||||
monkeypatch,
|
||||
) -> None:
|
||||
"""A routing-URL mismatch against a persistent deployment starts a
|
||||
dedicated proxy on a free port. Killing the deployment's proxy and
|
||||
starting an ephemeral one with the requested routing (the old behavior)
|
||||
orphaned the manifest: it still pointed at the port while the routing it
|
||||
declares no longer matched what served it."""
|
||||
calls: list[object] = []
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
@@ -761,11 +1478,17 @@ def test_ensure_proxy_restarts_persistent_deployment_for_feature_mismatch(monkey
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
# Persistent proxy is running, so _check_proxy returns True
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_port_bind_error", lambda port: None)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kw: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda pid, port: calls.append(("kill", pid, port)) or True,
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("routing mismatch must not kill the deployment's proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
@@ -781,10 +1504,12 @@ def test_ensure_proxy_restarts_persistent_deployment_for_feature_mismatch(monkey
|
||||
)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8787
|
||||
# Proxy should be killed and restarted due to openai_api_url mismatch
|
||||
assert calls[0] == ("kill", 12345, 8787)
|
||||
assert actual_port == 8799
|
||||
# Deployment left running; dedicated proxy started on the free port with
|
||||
# the requested routing.
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
assert calls[1][1][0] == 8799
|
||||
assert calls[1][2]["openai_api_url"] == "https://api.githubcopilot.com"
|
||||
|
||||
|
||||
@@ -852,17 +1577,29 @@ def test_ensure_proxy_restarts_recovered_persistent_for_openai_api_url_mismatch(
|
||||
monkeypatch.setattr(wrap_cli, "_recover_persistent_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kwargs: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_restart_persistent_proxy",
|
||||
lambda manifest, port: calls.append(("restart", manifest.profile, port)) or True,
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("routing mismatch must not restart the persistent proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("routing mismatch must not kill the persistent proxy")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("ephemeral proxy should not start")
|
||||
),
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(
|
||||
@@ -872,8 +1609,9 @@ def test_ensure_proxy_restarts_recovered_persistent_for_openai_api_url_mismatch(
|
||||
)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8787
|
||||
assert calls == [("restart", "default", 8787)]
|
||||
assert actual_port == 8799
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def test_ensure_proxy_restarts_recovered_persistent_when_config_unavailable(monkeypatch) -> None:
|
||||
@@ -885,17 +1623,29 @@ def test_ensure_proxy_restarts_recovered_persistent_when_config_unavailable(monk
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: {"version": "x"})
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_config", lambda port: None)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_find_available_port",
|
||||
lambda start_port, **kwargs: calls.append(("find_port", start_port)) or 8799,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_restart_persistent_proxy",
|
||||
lambda manifest, port: calls.append(("restart", manifest.profile, port)) or True,
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("configless recovered proxy must not restart")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_kill_proxy_by_pid",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("configless recovered proxy must not be killed")
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("ephemeral proxy should not start")
|
||||
),
|
||||
lambda *args, **kwargs: calls.append(("start", args, kwargs)),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(
|
||||
@@ -905,8 +1655,9 @@ def test_ensure_proxy_restarts_recovered_persistent_when_config_unavailable(monk
|
||||
)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8787
|
||||
assert calls == [("restart", "default", 8787)]
|
||||
assert actual_port == 8799
|
||||
assert calls[0] == ("find_port", 8787)
|
||||
assert calls[1][0] == "start"
|
||||
|
||||
|
||||
def test_ensure_proxy_reuses_persistent_deployment_when_features_match(monkeypatch) -> None:
|
||||
@@ -953,12 +1704,22 @@ def test_ensure_proxy_reuses_persistent_deployment_when_features_match(monkeypat
|
||||
assert actual_port == 8787
|
||||
|
||||
|
||||
def test_ensure_proxy_recovered_persistent_deployment_checks_feature_mismatch(monkeypatch) -> None:
|
||||
@pytest.mark.parametrize(
|
||||
"ensure_kwargs",
|
||||
[
|
||||
{"memory": True},
|
||||
{"learn": True},
|
||||
{"code_graph": True},
|
||||
],
|
||||
)
|
||||
def test_ensure_proxy_recovered_persistent_deployment_checks_feature_mismatch(
|
||||
monkeypatch,
|
||||
ensure_kwargs: dict[str, bool],
|
||||
) -> None:
|
||||
"""Recovered persistent deployments must still restart on feature mismatch.
|
||||
|
||||
Regression guard for the recover path: when wrap requests a different
|
||||
openai_api_url (Copilot subscription), do not early-return right after
|
||||
recover; run the shared mismatch checks and restart if needed.
|
||||
Regression guard for the recover path: when wrap requests memory, do not
|
||||
early-return right after recover; restart the deployment if needed.
|
||||
"""
|
||||
|
||||
calls: list[object] = []
|
||||
@@ -992,12 +1753,32 @@ def test_ensure_proxy_recovered_persistent_deployment_checks_feature_mismatch(mo
|
||||
),
|
||||
)
|
||||
|
||||
proc, actual_port = wrap_cli._ensure_proxy(
|
||||
8787,
|
||||
False,
|
||||
openai_api_url="https://api.githubcopilot.com",
|
||||
)
|
||||
proc, actual_port = wrap_cli._ensure_proxy(8787, False, **ensure_kwargs)
|
||||
|
||||
assert proc is None
|
||||
assert actual_port == 8787
|
||||
assert calls == [("restart", "default", 8787)]
|
||||
|
||||
|
||||
def test_ensure_proxy_recovered_feature_restart_failure_raises(monkeypatch) -> None:
|
||||
health = {
|
||||
"version": wrap_cli._HEADROOM_VERSION,
|
||||
"config": {"pid": 12345, "backend": "anthropic", "memory": False},
|
||||
}
|
||||
|
||||
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: _Manifest())
|
||||
monkeypatch.setattr("headroom.install.health.probe_ready", lambda url: False)
|
||||
monkeypatch.setattr(wrap_cli, "_recover_persistent_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
|
||||
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: health)
|
||||
monkeypatch.setattr(wrap_cli, "_restart_persistent_proxy", lambda manifest, port: False)
|
||||
monkeypatch.setattr(
|
||||
wrap_cli,
|
||||
"_start_proxy",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("failed persistent restart must not start an ephemeral proxy")
|
||||
),
|
||||
)
|
||||
|
||||
with pytest.raises(click.ClickException, match="could not be restarted"):
|
||||
wrap_cli._ensure_proxy(8787, False, memory=True)
|
||||
|
||||
Reference in New Issue
Block a user