mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat(wrap): add headroom wrap bob for IBM Bob CLI (#3801)
## Description Adds `headroom wrap bob` for the IBM Bob CLI as a wrap-registry target. `BOB_GATEWAY_URL` points Bob at the proxy, and the proxy forwards to `https://api.us-east.bob.ibm.com/inference/v1`. Bob's own `Authorization: apikey …` credential and `~/.bob/settings` are left untouched. **Stacked on #3800.** Until that merges, this diff also contains its commit. The Bob change is the last commit only (`feat(wrap): add headroom wrap bob …`). Bob's gateway needs three proxy seams, each declared on the target: - **`extra_chat_routes`**: Bob posts to `/inference/v1/chat/completions`. Without a real route that matches only the catch-all and is forwarded **uncompressed**. - **`origin_passthrough_prefixes`** (`/inference/`, `/admin/`, `/rag/`, `/metrics-forwarder/`): Bob builds these paths itself against the bare origin (inference and model info, profile and budget, the IBM docs-search tools, telemetry). Joining them onto the `/inference` base produces doubled or misrooted URLs (`…/inference/inference/v1/model/info`, `…/inference/admin/v1/profile`) that IBM's edge rejects with 403. - **`origin_passthrough_strip_json_keys`**: Bob 2.0.1 through 2.0.5 rewrites its gateway host from `region_domain` in the `/admin/v1/profile` response while keeping the proxy's port, so every later request goes to an unreachable `api.<region>:<proxy-port>`. The proxy removes that key. Bob bills flat per token, so the target defaults `HEADROOM_MODE` to `token`; an explicit `HEADROOM_MODE` still wins. `/health` now reports the proxy's `mode`, and `headroom wrap` warns when it reuses a proxy running a different mode than the session asked for. The warning applies to every harness. Replaces the Bob part of #3366, without its `wrap_targets.json` overlay, and supersedes #3323. Closes # ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `headroom/providers/wrap_registry.py`: `bare_origin` URL style; `openai_api_url`, `extra_chat_routes`, `origin_passthrough_prefixes`, `origin_passthrough_strip_json_keys`, `default_mode`, `preflight` fields; the `bob` target; `bob_preflight`, which aborts the wrap when a saved `~/.bob/settings/settings.json` `gatewayUrl` would override `BOB_GATEWAY_URL` (bobshell re-resolves `settings.gatewayUrl ?? policy.GatewayUrl ?? env` at startup, verified in 2.0.5); `resolve_origin_passthrough_url` / `strip_origin_passthrough_response_keys` over a host-keyed index built once from the static registry - `headroom/providers/route_specs.py`: `OPENAI_HANDLER_ROUTES` gains the registry's `extra_chat_routes`, built from static code data (no user-file read) - `headroom/proxy/handlers/openai.py`: catch-all passthrough uses the origin-rooted URL for declared paths and strips declared keys from 200 JSON responses - `headroom/proxy/server.py`: `/health` config block adds `mode` - `headroom/cli/wrap.py`: generated commands export `default_mode` and pass `openai_api_url`; new `_warn_proxy_mode_mismatch` on both proxy-reuse paths - `tests/test_cli/test_wrap_bob.py` (22 tests), including a handler-level test that fails without the `openai.py` wiring ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_cli/ tests/test_provider_route_specs.py tests/test_agent_savings.py \ tests/test_custom_base_passthrough_telemetry.py tests/test_compress_passthrough.py 878 passed, 1 skipped $ pytest tests/test_cli/test_wrap_bob.py -k handler # with openai.py reverted to the parent commit 1 failed (upstream URL was not origin-rooted) # -> the test guards the wiring $ mypy headroom/cli/wrap.py headroom/providers/wrap_registry.py headroom/providers/route_specs.py Success: no issues found in 3 source files ``` Locally, `tests/test_proxy_healthchecks.py` (where I added the `mode` assertion) and two `tests/test_proxy_openai.py` cases fail identically on `main`, because of a Starlette `TestClient` version mismatch in my environment. So I checked `/health` separately through `httpx.ASGITransport` from a loopback client: `ProxyConfig(mode="token")` → `"mode": "token"`, `ProxyConfig(mode="cache")` → `"mode": "cache"`. ## Real Behavior Proof - Environment: macOS (darwin 25.6.0), Python 3.13, this branch; the real IBM gateway, no Bob credentials - Exact command / steps: a stub `bob` on PATH reads `/health` through `BOB_GATEWAY_URL` and sends uncredentialed requests for Bob's three paths; `headroom wrap bob --port 8911 -- run "fix it"` with `HEADROOM_MODE` unset. For comparison, the same GETs sent straight to IBM at the correct URLs and at the misrooted ones. - Observed result: all three paths reach the correct IBM URLs. - The child gets `BOB_GATEWAY_URL=http://127.0.0.1:8911/p/run` and `argv=run fix it`. The proxy reports `mode: token`. - The chat POST is logged as `outbound_request … path=https://api.us-east.bob.ibm.com/inference/v1/chat/completions`. Cloudflare rejects it with 403 because there are no credentials. - `/admin/v1/profile` returns **401**, which matches the origin-rooted URL (direct GET → 401). The misrooted `/inference/admin/v1/profile` returns **403**. - Not tested: a credentialed, live Bob session on this branch (it spends IBM credits). That ran end to end on #3366, which shares these code paths except for the removed overlay. `/inference/v1/model/info` can't be told apart without credentials (403 either way); the handler and unit tests cover it. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: two changes reach every proxy: - The route `POST /inference/v1/chat/completions` is added. It previously matched the catch-all. - Catch-all passthrough changes only for requests whose OpenAI upstream is the Bob gateway host. `/health` gains a `mode` field. Other clients behave the same. - Kill switch / disable path: don't use `wrap bob`; the passthrough rules only fire for the Bob gateway host - Unsafe override required: no - Qualification impact: none - Rollback path: revert this PR; no persisted state. What to monitor: 403s on `/inference` or `/admin` paths in the proxy log for Bob sessions. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - Documentation: `headroom wrap bob --help` documents usage and the mode default. There's no separate docs page, matching the other env-var wrap targets. - Known limitations, deliberate in this PR: - The saved-`gatewayUrl` guard runs from `configure_launch`, after the proxy is up, because it must see the actual port. A misconfigured machine pays one proxy start/teardown (~2 s) before the abort; an earlier requested-port check reintroduces the "rejected too early" case. - Registry commands have no `--openai-api-url`; a non-us-east Bob account needs a hand-started `headroom proxy --openai-api-url https://api.<region>.bob.ibm.com/inference/v1` plus `--no-proxy`. The passthrough rules now key on `.bob.ibm.com`, so that setup gets the same path rooting and `region_domain` strip. - `default_mode` is delivered through `HEADROOM_MODE` in the wrapper's environment; that is the documented mode interface and what `_start_proxy` already reads. It is also visible to the child process. - `/inference/v1/chat/completions` is a static route on every proxy, not gated on the upstream; no other upstream serves that path. - After a persistent restart the mode warning queries the restarted proxy's config; the "recovered but nothing listening" return stays silent because there is no proxy to compare. - Known limitation, unchanged by this PR: one proxy has one OpenAI-family upstream and one mode. When Bob joins a shared proxy started without the IBM upstream, wrap restarts it and repoints OpenAI traffic for every attached client (kimi and vibe behave the same way). Give Bob its own `--port` when running several harnesses. - One of three PRs replacing #3366. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3801" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JD Davis
parent
c873d13fe0
commit
0b7dab47c6
+68
-12
@@ -4689,8 +4689,11 @@ def _ensure_proxy_unlocked(
|
||||
f" Leaving it running because {detail} "
|
||||
"are still attached; it will be restarted when idle."
|
||||
)
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return None, port
|
||||
if helpers._restart_persistent_proxy(manifest, port):
|
||||
# Restarted from the manifest, whose mode need not be ours.
|
||||
_warn_proxy_mode_mismatch(helpers._query_proxy_config(port))
|
||||
return None, port
|
||||
raise click.ClickException(
|
||||
f"Persistent deployment '{manifest.profile}' on port {port} "
|
||||
@@ -4708,6 +4711,7 @@ def _ensure_proxy_unlocked(
|
||||
if not missing:
|
||||
click.echo(f" Proxy already running on port {port}")
|
||||
click.echo(f" Dashboard: http://127.0.0.1:{port}/dashboard")
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return None, port
|
||||
# Features mismatch or config unavailable — fall through to the
|
||||
# non-persistent path which handles proxy restart. A routing
|
||||
@@ -4753,6 +4757,7 @@ def _ensure_proxy_unlocked(
|
||||
missing.append("code-graph")
|
||||
|
||||
if not missing:
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return None, port
|
||||
flags_str = ", ".join(f"--{f}" for f in missing)
|
||||
click.echo(
|
||||
@@ -4760,6 +4765,8 @@ def _ensure_proxy_unlocked(
|
||||
f"is missing: {flags_str}; restarting..."
|
||||
)
|
||||
if helpers._restart_persistent_proxy(manifest, port):
|
||||
# Restarted from the manifest, whose mode need not be ours.
|
||||
_warn_proxy_mode_mismatch(helpers._query_proxy_config(port))
|
||||
return None, port
|
||||
raise click.ClickException(
|
||||
f"Persistent deployment '{manifest.profile}' on port {port} "
|
||||
@@ -4859,6 +4866,7 @@ def _ensure_proxy_unlocked(
|
||||
f" Leaving it running because {detail} "
|
||||
"are still attached; it will be restarted when idle."
|
||||
)
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return None, port
|
||||
|
||||
else:
|
||||
@@ -4937,6 +4945,7 @@ def _ensure_proxy_unlocked(
|
||||
f" Please stop the proxy on port {port} manually "
|
||||
f"and rerun with {flags_str}."
|
||||
)
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return None, port
|
||||
|
||||
# Routing-level config (backend, upstream URLs) must match in
|
||||
@@ -4982,6 +4991,7 @@ def _ensure_proxy_unlocked(
|
||||
if not needs_restart and reuse_running:
|
||||
click.echo(f" Proxy already running on port {port}")
|
||||
click.echo(f" Dashboard: http://127.0.0.1:{port}/dashboard")
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return None, port
|
||||
|
||||
# Start (or restart) the proxy with the requested flags.
|
||||
@@ -5036,23 +5046,25 @@ def _ensure_proxy_unlocked(
|
||||
else:
|
||||
if not helpers._check_proxy(port):
|
||||
click.echo(f" Warning: No proxy detected on port {port}")
|
||||
elif vertex_api_url or clear_vertex_api_url:
|
||||
elif vertex_api_url or clear_vertex_api_url or os.environ.get("HEADROOM_MODE"):
|
||||
health_payload = helpers._query_proxy_health(port)
|
||||
running_config = helpers._proxy_health_config(health_payload)
|
||||
if running_config is None:
|
||||
running_config = helpers._query_proxy_config(port)
|
||||
running_vertex_url = (
|
||||
_normalize_proxy_api_url(running_config.get("vertex_api_url"))
|
||||
if running_config is not None
|
||||
else None
|
||||
)
|
||||
requested_vertex_url = _normalize_proxy_api_url(vertex_api_url)
|
||||
if running_vertex_url != requested_vertex_url:
|
||||
click.echo(
|
||||
" Warning: --no-proxy is set, but the running proxy does not "
|
||||
"advertise the requested Vertex target. Requests may still go "
|
||||
"to the proxy's existing Vertex upstream."
|
||||
if vertex_api_url or clear_vertex_api_url:
|
||||
running_vertex_url = (
|
||||
_normalize_proxy_api_url(running_config.get("vertex_api_url"))
|
||||
if running_config is not None
|
||||
else None
|
||||
)
|
||||
requested_vertex_url = _normalize_proxy_api_url(vertex_api_url)
|
||||
if running_vertex_url != requested_vertex_url:
|
||||
click.echo(
|
||||
" Warning: --no-proxy is set, but the running proxy does not "
|
||||
"advertise the requested Vertex target. Requests may still go "
|
||||
"to the proxy's existing Vertex upstream."
|
||||
)
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return None, port
|
||||
|
||||
|
||||
@@ -8824,6 +8836,28 @@ def unwrap_zcode(port: int, no_stop_proxy: bool) -> None:
|
||||
click.echo()
|
||||
|
||||
|
||||
def _warn_proxy_mode_mismatch(running_config: dict[str, Any] | None) -> None:
|
||||
"""Warn when a reused proxy runs a different mode than this session asked for.
|
||||
|
||||
Mode is fixed at proxy startup, so a requested HEADROOM_MODE (explicit, or
|
||||
a wrap target's default_mode) is silently ignored on reuse. Warning-only:
|
||||
other clients may be attached to the running proxy.
|
||||
"""
|
||||
requested = os.environ.get("HEADROOM_MODE")
|
||||
running = (running_config or {}).get("mode")
|
||||
if not requested or not isinstance(running, str):
|
||||
return
|
||||
from headroom.proxy.proxy_mode_policy import normalize_proxy_mode_decision
|
||||
|
||||
decision = normalize_proxy_mode_decision(requested, default=running)
|
||||
if not decision.unknown and decision.normalized != running:
|
||||
click.echo(
|
||||
f" Warning: this session requested {decision.normalized!r} mode but the "
|
||||
f"running proxy is in {running!r} mode (mode is fixed at proxy startup). "
|
||||
"Restart the proxy, or use --port for a separate one."
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Registry-generated wrap commands
|
||||
# =============================================================================
|
||||
@@ -8854,6 +8888,11 @@ def _make_registry_command(target: WrapTarget) -> click.Command:
|
||||
click.echo(target.install_hint)
|
||||
raise SystemExit(1)
|
||||
|
||||
# Exported before proxy startup so _start_proxy forwards it as --mode;
|
||||
# an explicit HEADROOM_MODE always wins.
|
||||
if target.default_mode and not os.environ.get("HEADROOM_MODE"):
|
||||
os.environ["HEADROOM_MODE"] = target.default_mode
|
||||
|
||||
env, env_vars_display = _build_registry_launch_env(
|
||||
target,
|
||||
port,
|
||||
@@ -8861,6 +8900,21 @@ def _make_registry_command(target: WrapTarget) -> click.Command:
|
||||
project=_project_name_from_cwd() if target.project_prefix else None,
|
||||
)
|
||||
|
||||
configure_launch = None
|
||||
if target.preflight is not None:
|
||||
preflight = target.preflight
|
||||
|
||||
def configure_launch(
|
||||
actual_port: int, args: tuple, env: dict[str, str], display: list[str]
|
||||
) -> tuple[tuple, dict[str, str], list[str]]:
|
||||
# Runs on the final env: _ensure_proxy may have moved to another
|
||||
# port, and the saved-gateway comparison must use that URL.
|
||||
# Raising here still tears the proxy down via _launch_tool's
|
||||
# cleanup.
|
||||
if problem := preflight(env):
|
||||
raise click.ClickException(problem)
|
||||
return args, env, display
|
||||
|
||||
_launch_tool(
|
||||
binary=tool_bin,
|
||||
args=tool_args,
|
||||
@@ -8876,6 +8930,8 @@ def _make_registry_command(target: WrapTarget) -> click.Command:
|
||||
backend=backend,
|
||||
anyllm_provider=anyllm_provider,
|
||||
region=region,
|
||||
openai_api_url=target.openai_api_url,
|
||||
configure_launch=configure_launch,
|
||||
)
|
||||
|
||||
_run.__doc__ = target.help_text
|
||||
|
||||
@@ -103,12 +103,23 @@ ANTHROPIC_BATCH_ROUTES: tuple[ProviderHandlerRoute, ...] = (
|
||||
)
|
||||
|
||||
|
||||
def _wrap_registry_chat_routes() -> tuple[ProviderHandlerRoute, ...]:
|
||||
"""Chat routes declared by wrap-registry targets (e.g. IBM Bob's
|
||||
``/inference/v1/chat/completions``). Without a real route that traffic
|
||||
matches only the catch-all and is forwarded uncompressed."""
|
||||
# Local import: wrap_registry pulls in provider runtime modules.
|
||||
from headroom.providers.wrap_registry import WRAP_TARGETS
|
||||
|
||||
paths = dict.fromkeys(path for t in WRAP_TARGETS.values() for path in t.extra_chat_routes)
|
||||
return tuple(ProviderHandlerRoute("POST", path, "handle_openai_chat") for path in paths)
|
||||
|
||||
|
||||
OPENAI_HANDLER_ROUTES: tuple[ProviderHandlerRoute, ...] = (
|
||||
ProviderHandlerRoute("POST", "/v1/chat/completions", "handle_openai_chat"),
|
||||
# Copilot Chat derives this unprefixed path from overrideCapiUrl. Route it
|
||||
# through the real handler; the generic catch-all would bypass compression.
|
||||
ProviderHandlerRoute("POST", "/chat/completions", "handle_openai_chat"),
|
||||
)
|
||||
) + _wrap_registry_chat_routes()
|
||||
|
||||
|
||||
OPENAI_BATCH_ROUTES: tuple[ProviderHandlerRoute, ...] = (
|
||||
|
||||
@@ -14,10 +14,13 @@ registrars, config rendering) or whose launch env is shared with
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from collections.abc import Mapping
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from headroom.providers.claude import proxy_base_url as claude_proxy_base_url
|
||||
from headroom.providers.codex import proxy_base_url as codex_proxy_base_url
|
||||
@@ -25,12 +28,17 @@ from headroom.proxy.project_context import with_project_prefix
|
||||
|
||||
# How an EnvVar's URL is shaped: ``openai_v1`` ends in ``/v1`` (OpenAI-style
|
||||
# clients append ``/chat/completions``); ``anthropic`` is a bare origin
|
||||
# (Anthropic clients append ``/v1/messages``).
|
||||
UrlStyle = Literal["openai_v1", "anthropic"]
|
||||
# (Anthropic clients append ``/v1/messages``); ``bare_origin`` is for tools
|
||||
# that append their own full path prefix (IBM Bob appends
|
||||
# ``/inference/v1/chat/completions``) — a ``/v1`` base would double it.
|
||||
UrlStyle = Literal["openai_v1", "anthropic", "bare_origin"]
|
||||
|
||||
_STYLE_BUILDERS = {
|
||||
"openai_v1": codex_proxy_base_url,
|
||||
"anthropic": claude_proxy_base_url,
|
||||
# Same URL as the anthropic base today; a distinct style documents why
|
||||
# there is no /v1 and keeps bare-origin tools right if that base changes.
|
||||
"bare_origin": claude_proxy_base_url,
|
||||
}
|
||||
|
||||
|
||||
@@ -60,6 +68,31 @@ class WrapTarget:
|
||||
# Encode the launch directory as a /p/<name> base-URL prefix so the proxy
|
||||
# can attribute savings per project (for tools that can't send headers).
|
||||
project_prefix: bool = True
|
||||
# OpenAI-family upstream handed to the proxy at startup.
|
||||
openai_api_url: str | None = None
|
||||
# Nonstandard chat-completions paths the tool posts to. Each becomes a
|
||||
# proxy route to ``handle_openai_chat`` (see ``route_specs``) so the
|
||||
# traffic is compressed instead of hitting the uncompressed catch-all.
|
||||
extra_chat_routes: tuple[str, ...] = ()
|
||||
# Non-chat gateway paths the tool builds itself against the bare origin.
|
||||
# The catch-all forwards these to the upstream ORIGIN verbatim; joining
|
||||
# them onto ``openai_api_url``'s path doubles or misroots them.
|
||||
origin_passthrough_prefixes: tuple[str, ...] = ()
|
||||
# (path, key) pairs removed from JSON responses on origin passthrough.
|
||||
# The key is removed at every nesting depth: Bob reads region_domain from
|
||||
# /admin/v1/profile under instances[].teams[], not at the top level.
|
||||
origin_passthrough_strip_json_keys: tuple[tuple[str, str], ...] = ()
|
||||
# Apply the passthrough rules to every host under this DNS suffix (leading
|
||||
# dot), not only the host in ``openai_api_url``: IBM runs one gateway per
|
||||
# region (api.<region>.bob.ibm.com) and --openai-api-url can select it.
|
||||
origin_host_suffix: str | None = None
|
||||
# Proxy mode exported as HEADROOM_MODE when the user has not set one.
|
||||
# Only affects a proxy this wrap starts; wrap warns when a reused proxy
|
||||
# runs a different mode.
|
||||
default_mode: str | None = None
|
||||
# Launch-time check on the built env; a returned message aborts the wrap.
|
||||
# For tools whose own saved config can override the env var we set.
|
||||
preflight: Callable[[Mapping[str, str]], str | None] | None = None
|
||||
|
||||
|
||||
def build_launch_env(
|
||||
@@ -81,6 +114,41 @@ def build_launch_env(
|
||||
return env, display
|
||||
|
||||
|
||||
def _origin_key(url: str) -> tuple[str, str, int | None]:
|
||||
parts = urlsplit(url.strip())
|
||||
host = (parts.hostname or "").lower()
|
||||
return parts.scheme, "127.0.0.1" if host == "localhost" else host, parts.port
|
||||
|
||||
|
||||
def bob_preflight(env: Mapping[str, str], settings_path: Path | None = None) -> str | None:
|
||||
"""Refuse to launch Bob when its saved gatewayUrl would bypass the proxy.
|
||||
|
||||
bobshell (verified 2.0.5) re-resolves its gateway at startup as
|
||||
``settings.gatewayUrl ?? policy.GatewayUrl ?? <env/flag>``, so a saved
|
||||
gatewayUrl silently overrides BOB_GATEWAY_URL and Bob runs uncompressed
|
||||
while the wrap banner claims otherwise. A policy-enforced URL cannot be
|
||||
read locally; the message names it so the user knows where else to look.
|
||||
"""
|
||||
path = settings_path or Path.home() / ".bob" / "settings" / "settings.json"
|
||||
try:
|
||||
saved = json.loads(path.read_text(encoding="utf-8")).get("gatewayUrl")
|
||||
except (OSError, ValueError, AttributeError):
|
||||
return None
|
||||
if not isinstance(saved, str) or not saved.strip():
|
||||
return None
|
||||
# Same proxy if scheme, host and port agree; a different /p/<project>
|
||||
# prefix only changes attribution, and the error below tells users to
|
||||
# point gatewayUrl at the proxy, so that must not then be rejected.
|
||||
if _origin_key(saved) == _origin_key(env.get("BOB_GATEWAY_URL", "")):
|
||||
return None
|
||||
return (
|
||||
f"Bob's saved gatewayUrl ({saved.strip()}) overrides BOB_GATEWAY_URL, so Bob "
|
||||
f"would bypass the Headroom proxy. Remove the gatewayUrl entry from {path} "
|
||||
"(or set it to the proxy URL shown by this wrap) and retry. If your "
|
||||
"organisation enforces a GatewayUrl policy, Bob cannot be wrapped."
|
||||
)
|
||||
|
||||
|
||||
WRAP_TARGETS: dict[str, WrapTarget] = {
|
||||
target.name: target
|
||||
for target in (
|
||||
@@ -137,6 +205,57 @@ WRAP_TARGETS: dict[str, WrapTarget] = {
|
||||
" headroom wrap openhands -- --task ... # Pass args to openhands"
|
||||
),
|
||||
),
|
||||
WrapTarget(
|
||||
name="bob",
|
||||
binary="bob",
|
||||
install_hint="Install IBM Bob CLI: npm install -g bobshell",
|
||||
env_vars=(
|
||||
# Bob resolves its gateway as --gateway-url ?? BOB_GATEWAY_URL
|
||||
# ?? default, so this reroutes inference without touching
|
||||
# ~/.bob/settings. Bob appends /inference/v1/... itself. A saved
|
||||
# settings.json gatewayUrl (or GatewayUrl policy) still wins over
|
||||
# all of these at startup — see bob_preflight.
|
||||
EnvVar("BOB_GATEWAY_URL", "bare_origin"),
|
||||
),
|
||||
preflight=bob_preflight,
|
||||
# Carries /inference/v1 so the proxy's _normalize_api_url (strips
|
||||
# /v1) and handle_openai_chat (re-appends /v1/chat/completions)
|
||||
# compose back into the path IBM serves.
|
||||
openai_api_url="https://api.us-east.bob.ibm.com/inference/v1",
|
||||
extra_chat_routes=("/inference/v1/chat/completions",),
|
||||
# Bob builds every gateway path itself: inference + model info,
|
||||
# profile/budget, IBM docs search (/rag/v1/search), telemetry.
|
||||
origin_passthrough_prefixes=("/inference/", "/admin/", "/rag/", "/metrics-forwarder/"),
|
||||
origin_host_suffix=".bob.ibm.com",
|
||||
# Bob 2.0.1–2.0.5 rewrites its gateway host from region_domain in the
|
||||
# /admin/v1/profile response while keeping the proxied port, so
|
||||
# every later request targets api.<region>:<proxy-port>. With the
|
||||
# key absent it keeps its configured gateway URL (the proxy).
|
||||
origin_passthrough_strip_json_keys=(("/admin/v1/profile", "region_domain"),),
|
||||
# Bob bills flat per token (no prompt-cache discount to protect),
|
||||
# so token mode converts compression 1:1 into dollars.
|
||||
default_mode="token",
|
||||
help_text=(
|
||||
"Launch IBM Bob CLI through Headroom proxy.\n"
|
||||
"\n"
|
||||
"\b\n"
|
||||
"Sets ``BOB_GATEWAY_URL`` so Bob routes inference traffic through\n"
|
||||
"Headroom while keeping its own ``Authorization: apikey ...``\n"
|
||||
"credential and its ~/.bob/settings files untouched.\n"
|
||||
"\n"
|
||||
"\b\n"
|
||||
"Bob bills flat per token, so token mode converts compression 1:1\n"
|
||||
"into dollars. Token mode is the default for bob; set HEADROOM_MODE\n"
|
||||
"to override:\n"
|
||||
" HEADROOM_MODE=cache headroom wrap bob\n"
|
||||
"\n"
|
||||
"\b\n"
|
||||
"Examples:\n"
|
||||
" headroom wrap bob # Start proxy + bob\n"
|
||||
' headroom wrap bob -- run "fix the bug" # Pass args to bob\n'
|
||||
" headroom wrap bob --port 9999 # Custom proxy port"
|
||||
),
|
||||
),
|
||||
WrapTarget(
|
||||
name="openclaude",
|
||||
binary="openclaude",
|
||||
@@ -162,3 +281,99 @@ WRAP_TARGETS: dict[str, WrapTarget] = {
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Origin passthrough
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _OriginRules:
|
||||
prefixes: tuple[str, ...]
|
||||
strip_keys: tuple[tuple[str, str], ...]
|
||||
|
||||
|
||||
# (scheme, host or ".suffix") of a target's upstream -> its passthrough rules.
|
||||
_ORIGIN_RULES: tuple[tuple[str, str, _OriginRules], ...] = tuple(
|
||||
(
|
||||
url.scheme,
|
||||
target.origin_host_suffix or url.netloc,
|
||||
_OriginRules(target.origin_passthrough_prefixes, target.origin_passthrough_strip_json_keys),
|
||||
)
|
||||
for target in WRAP_TARGETS.values()
|
||||
if target.openai_api_url
|
||||
and (target.origin_passthrough_prefixes or target.origin_passthrough_strip_json_keys)
|
||||
for url in (urlsplit(target.openai_api_url),)
|
||||
)
|
||||
|
||||
|
||||
def _origin_rules(base_url: str | None) -> tuple[str, _OriginRules] | None:
|
||||
if not base_url:
|
||||
return None
|
||||
base = urlsplit(base_url)
|
||||
host = (base.hostname or "").lower()
|
||||
for scheme, pattern, rules in _ORIGIN_RULES:
|
||||
if base.scheme != scheme:
|
||||
continue
|
||||
if base.netloc == pattern or (pattern.startswith(".") and host.endswith(pattern)):
|
||||
return f"{base.scheme}://{base.netloc}", rules
|
||||
return None
|
||||
|
||||
|
||||
def resolve_origin_passthrough_url(base_url: str | None, path: str) -> str | None:
|
||||
"""Origin-rooted upstream URL for a declared origin-passthrough path.
|
||||
|
||||
When ``base_url`` points at a registered target's gateway host and ``path``
|
||||
starts with one of its ``origin_passthrough_prefixes``, forward the path
|
||||
verbatim to that origin. None means the caller keeps its base+path join.
|
||||
"""
|
||||
found = _origin_rules(base_url)
|
||||
if found and any(path.startswith(prefix) for prefix in found[1].prefixes):
|
||||
return found[0] + path
|
||||
return None
|
||||
|
||||
|
||||
def _strip_json_key(obj: object, key: str) -> bool:
|
||||
"""Remove ``key`` from every dict in ``obj`` in place; True when removed."""
|
||||
removed = False
|
||||
if isinstance(obj, dict):
|
||||
if key in obj:
|
||||
del obj[key]
|
||||
removed = True
|
||||
for value in obj.values():
|
||||
removed = _strip_json_key(value, key) or removed
|
||||
elif isinstance(obj, list):
|
||||
for value in obj:
|
||||
removed = _strip_json_key(value, key) or removed
|
||||
return removed
|
||||
|
||||
|
||||
def strip_origin_passthrough_response_keys(
|
||||
base_url: str | None, path: str, body: bytes
|
||||
) -> bytes | None:
|
||||
"""Filtered JSON body for an origin-passthrough response, or None.
|
||||
|
||||
None when nothing is declared for this host and path, the body is not
|
||||
JSON, or no declared key was present.
|
||||
"""
|
||||
found = _origin_rules(base_url)
|
||||
if not found:
|
||||
return None
|
||||
keys = [
|
||||
key
|
||||
for declared, key in found[1].strip_keys
|
||||
if path == declared or path.startswith(declared.rstrip("/") + "/")
|
||||
]
|
||||
if not keys:
|
||||
return None
|
||||
try:
|
||||
payload = json.loads(body)
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
return None
|
||||
changed = False
|
||||
for key in keys:
|
||||
changed = _strip_json_key(payload, key) or changed
|
||||
if not changed:
|
||||
return None
|
||||
return json.dumps(payload, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
|
||||
|
||||
@@ -425,6 +425,26 @@ def _sanitize_forwarded_response_headers(
|
||||
return sanitize_forwarded_response_headers(headers, *extra_names)
|
||||
|
||||
|
||||
def _replaced_json_body(response: httpx.Response, body: bytes) -> tuple[bytes, dict[str, str]]:
|
||||
"""Headers for replaying ``response`` with ``body`` in place of its content.
|
||||
|
||||
The upstream's validators and digests describe the original bytes;
|
||||
forwarding them would let a cache or integrity check pair them with the
|
||||
rewritten body.
|
||||
"""
|
||||
headers = _sanitize_forwarded_response_headers(
|
||||
response.headers,
|
||||
"etag",
|
||||
"last-modified",
|
||||
"cache-control",
|
||||
"content-digest",
|
||||
"digest",
|
||||
"content-type",
|
||||
)
|
||||
headers["content-type"] = "application/json"
|
||||
return body, headers
|
||||
|
||||
|
||||
def _resolve_openai_handler_path(
|
||||
request_headers: dict[str, str],
|
||||
*,
|
||||
@@ -11291,7 +11311,14 @@ class OpenAIHandlerMixin:
|
||||
clean_model_id = sanitize_anthropic_model_id(unquote(raw_model_id))
|
||||
if clean_model_id != unquote(raw_model_id):
|
||||
path = "/v1/models/" + quote(clean_model_id, safe="")
|
||||
url = build_copilot_upstream_url(base_url, path)
|
||||
from headroom.providers.wrap_registry import resolve_origin_passthrough_url
|
||||
|
||||
# Wrap targets that build full gateway paths themselves (e.g. IBM Bob's
|
||||
# /inference/v1/model/info, /admin/v1/profile) declare origin
|
||||
# passthrough prefixes; joining those paths onto the base URL's own
|
||||
# path would double or misroot the upstream URL.
|
||||
origin_passthrough_url = resolve_origin_passthrough_url(base_url, path)
|
||||
url = origin_passthrough_url or build_copilot_upstream_url(base_url, path)
|
||||
|
||||
# Preserve query string parameters
|
||||
if request.url.query:
|
||||
@@ -11440,6 +11467,17 @@ class OpenAIHandlerMixin:
|
||||
response_headers = _sanitize_forwarded_response_headers(response.headers)
|
||||
response_content = response.content
|
||||
|
||||
if origin_passthrough_url is not None and response.status_code == 200:
|
||||
from headroom.providers.wrap_registry import strip_origin_passthrough_response_keys
|
||||
|
||||
# E.g. Bob's /admin/v1/profile carries region_domain, which bob
|
||||
# 2.0.1 uses to rewrite its gateway host away from the proxy while
|
||||
# keeping the proxied port — strip declared keys so the tool keeps
|
||||
# routing through the configured gateway URL.
|
||||
filtered = strip_origin_passthrough_response_keys(base_url, path, response_content)
|
||||
if filtered is not None:
|
||||
response_content, response_headers = _replaced_json_body(response, filtered)
|
||||
|
||||
if provider == "anthropic" and endpoint_name == "models":
|
||||
from headroom.providers.anthropic import sanitize_anthropic_model_metadata
|
||||
|
||||
@@ -11449,18 +11487,12 @@ class OpenAIHandlerMixin:
|
||||
except (TypeError, ValueError):
|
||||
sanitized_payload = None
|
||||
if sanitized_payload is not None and sanitized_payload != payload:
|
||||
response_content = json.dumps(
|
||||
sanitized_payload,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
response_headers = _sanitize_forwarded_response_headers(
|
||||
response.headers,
|
||||
"etag",
|
||||
"last-modified",
|
||||
"cache-control",
|
||||
response_content, response_headers = _replaced_json_body(
|
||||
response,
|
||||
json.dumps(sanitized_payload, separators=(",", ":"), ensure_ascii=False).encode(
|
||||
"utf-8"
|
||||
),
|
||||
)
|
||||
response_headers["content-type"] = "application/json"
|
||||
|
||||
# Passthrough request: forwarded upstream with no transforms.
|
||||
# Still recorded so dashboards see traffic on the passthrough
|
||||
|
||||
@@ -3687,6 +3687,9 @@ def create_app(config: ProxyConfig | None = None) -> FastAPI:
|
||||
# reports the proxy's policy, not the calling shell's.
|
||||
"tls": describe_trust_policy(),
|
||||
"backend": config.backend,
|
||||
# Boot-time proxy mode; `headroom wrap` compares it to the
|
||||
# session's requested mode when reusing this proxy.
|
||||
"mode": config.mode,
|
||||
"optimize": config.optimize,
|
||||
"cache": config.cache_enabled,
|
||||
"rate_limit": config.rate_limit_enabled,
|
||||
|
||||
@@ -0,0 +1,430 @@
|
||||
"""Tests for `headroom wrap bob` (IBM Bob CLI) and the registry seams it uses."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
import headroom.cli.wrap as wrap_mod
|
||||
from headroom.cli.wrap import _warn_proxy_mode_mismatch, wrap
|
||||
from headroom.providers.route_specs import OPENAI_HANDLER_ROUTES
|
||||
from headroom.providers.wrap_registry import (
|
||||
WRAP_TARGETS,
|
||||
bob_preflight,
|
||||
build_launch_env,
|
||||
resolve_origin_passthrough_url,
|
||||
strip_origin_passthrough_response_keys,
|
||||
)
|
||||
|
||||
BOB = WRAP_TARGETS["bob"]
|
||||
BASE = "https://api.us-east.bob.ibm.com/inference"
|
||||
|
||||
|
||||
def test_env_is_bare_origin_with_project_prefix():
|
||||
# Bob appends /inference/v1/... itself; a /v1 base would double the prefix.
|
||||
env, display = build_launch_env(BOB, 8788, environ={}, project="myproj")
|
||||
assert env["BOB_GATEWAY_URL"] == "http://127.0.0.1:8788/p/myproj"
|
||||
assert display == ["BOB_GATEWAY_URL=http://127.0.0.1:8788/p/myproj"]
|
||||
|
||||
|
||||
def test_inference_chat_route_reaches_openai_handler():
|
||||
routes = [r for r in OPENAI_HANDLER_ROUTES if r.path == "/inference/v1/chat/completions"]
|
||||
assert [(r.method, r.handler_name) for r in routes] == [("POST", "handle_openai_chat")]
|
||||
|
||||
|
||||
class TestLaunch:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _isolated_bob_home(self, monkeypatch, tmp_path):
|
||||
# The preflight reads ~/.bob/settings/settings.json; never the developer's.
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
|
||||
@staticmethod
|
||||
def _invoke(monkeypatch, expect_exit: int = 0):
|
||||
monkeypatch.setattr(wrap_mod.shutil, "which", lambda name: f"/usr/bin/{name}")
|
||||
captured: dict = {}
|
||||
|
||||
def fake_launch_tool(**kwargs):
|
||||
captured.update(kwargs, mode=os.environ.get("HEADROOM_MODE"))
|
||||
|
||||
monkeypatch.setattr(wrap_mod, "_launch_tool", fake_launch_tool)
|
||||
result = CliRunner().invoke(wrap, ["bob", "--", "run", "fix it"])
|
||||
assert result.exit_code == expect_exit, result.output
|
||||
return captured if expect_exit == 0 else result.output
|
||||
|
||||
def test_hands_proxy_the_inference_upstream(self, monkeypatch):
|
||||
monkeypatch.delenv("HEADROOM_MODE", raising=False)
|
||||
captured = self._invoke(monkeypatch)
|
||||
# /inference/v1: the proxy strips /v1 and handle_openai_chat re-appends
|
||||
# /v1/chat/completions, composing back into the path IBM serves.
|
||||
assert captured["openai_api_url"] == "https://api.us-east.bob.ibm.com/inference/v1"
|
||||
assert captured["args"] == ("run", "fix it")
|
||||
|
||||
def test_default_mode_fills_unset_headroom_mode(self, monkeypatch):
|
||||
# setenv-then-delenv registers restoration even though the command
|
||||
# writes os.environ itself.
|
||||
monkeypatch.setenv("HEADROOM_MODE", "sentinel")
|
||||
monkeypatch.delenv("HEADROOM_MODE")
|
||||
assert self._invoke(monkeypatch)["mode"] == "token"
|
||||
|
||||
def test_explicit_headroom_mode_wins(self, monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_MODE", "cache")
|
||||
assert self._invoke(monkeypatch)["mode"] == "cache"
|
||||
|
||||
|
||||
class TestSavedGatewayGuard:
|
||||
"""bobshell re-resolves settings.gatewayUrl over BOB_GATEWAY_URL at startup,
|
||||
so launching would run Bob uncompressed behind a banner saying otherwise.
|
||||
The guard must judge the URL Bob actually receives, i.e. after _ensure_proxy
|
||||
may have fallen back to another port."""
|
||||
|
||||
@pytest.fixture
|
||||
def saved(self, monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(Path, "home", lambda: tmp_path)
|
||||
settings = tmp_path / ".bob" / "settings" / "settings.json"
|
||||
settings.parent.mkdir(parents=True)
|
||||
|
||||
def _write(gateway_url: str) -> Path:
|
||||
settings.write_text(json.dumps({"gatewayUrl": gateway_url}))
|
||||
return settings
|
||||
|
||||
return _write
|
||||
|
||||
@staticmethod
|
||||
def _launch(monkeypatch, tmp_path, *, actual_port: int):
|
||||
"""Run `wrap bob` for real up to the child spawn, with the proxy stubbed
|
||||
to come up on ``actual_port`` (8787 requested)."""
|
||||
import sys
|
||||
from unittest.mock import patch
|
||||
|
||||
monkeypatch.setattr(wrap_mod.shutil, "which", lambda _name: sys.executable)
|
||||
monkeypatch.setattr(wrap_mod, "_project_name_from_cwd", lambda: "proj")
|
||||
marker = tmp_path / "child.txt"
|
||||
child = (
|
||||
"import os; from pathlib import Path; "
|
||||
f"Path({str(marker)!r}).write_text(os.environ['BOB_GATEWAY_URL'])"
|
||||
)
|
||||
with (
|
||||
patch.object(wrap_mod, "_make_cleanup", return_value=lambda: None),
|
||||
patch.object(wrap_mod.signal, "signal"),
|
||||
patch.object(wrap_mod, "_register_proxy_client"),
|
||||
patch.object(wrap_mod, "_unregister_proxy_client"),
|
||||
patch.object(wrap_mod, "_push_runtime_env"),
|
||||
patch.object(wrap_mod, "_ensure_proxy", return_value=(None, actual_port)),
|
||||
patch.object(wrap_mod, "_configure_quiet_cli_env", return_value=[]),
|
||||
):
|
||||
result = CliRunner().invoke(wrap, ["bob", "--port", "8787", "--", "-c", child])
|
||||
return result, marker
|
||||
|
||||
def test_foreign_gateway_aborts(self, monkeypatch, tmp_path, saved):
|
||||
settings = saved("https://api.eu-de.bob.ibm.com")
|
||||
result, marker = self._launch(monkeypatch, tmp_path, actual_port=8787)
|
||||
assert result.exit_code == 1
|
||||
# Exact ClickException text; a substring check on the URL reads to
|
||||
# CodeQL as URL sanitization.
|
||||
expected = bob_preflight({"BOB_GATEWAY_URL": "http://127.0.0.1:8787"}, settings)
|
||||
assert result.output.strip().endswith(f"Error: {expected}")
|
||||
assert not marker.exists(), "Bob must not be spawned"
|
||||
|
||||
def test_saved_requested_port_aborts_after_fallback(self, monkeypatch, tmp_path, saved):
|
||||
# Saved URL matches the requested port, but the proxy came up on 8899:
|
||||
# Bob would ignore BOB_GATEWAY_URL and talk to a port with no proxy.
|
||||
saved("http://127.0.0.1:8787/p/proj")
|
||||
result, marker = self._launch(monkeypatch, tmp_path, actual_port=8899)
|
||||
assert result.exit_code == 1 and "overrides BOB_GATEWAY_URL" in result.output
|
||||
assert not marker.exists()
|
||||
|
||||
def test_saved_fallback_port_launches(self, monkeypatch, tmp_path, saved):
|
||||
# The inverse: saved URL already names the port the proxy ended up on.
|
||||
saved("http://127.0.0.1:8899/p/proj")
|
||||
result, marker = self._launch(monkeypatch, tmp_path, actual_port=8899)
|
||||
assert result.exit_code == 0, result.output
|
||||
assert marker.read_text() == "http://127.0.0.1:8899/p/proj"
|
||||
|
||||
|
||||
class TestBobPreflight:
|
||||
def _settings(self, tmp_path, payload) -> Path:
|
||||
path = tmp_path / "settings.json"
|
||||
path.write_text(payload if isinstance(payload, str) else json.dumps(payload))
|
||||
return path
|
||||
|
||||
ENV = {"BOB_GATEWAY_URL": "http://127.0.0.1:8787/p/myproj"}
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"payload",
|
||||
[
|
||||
{},
|
||||
{"gatewayUrl": ""},
|
||||
{"gatewayUrl": None},
|
||||
{"gatewayUrl": "http://127.0.0.1:8787/p/myproj/"}, # already the proxy
|
||||
{"gatewayUrl": "http://127.0.0.1:8787/p/another-project"}, # attribution only
|
||||
{"gatewayUrl": "http://localhost:8787"}, # same proxy, spelled differently
|
||||
"not json",
|
||||
],
|
||||
)
|
||||
def test_passes(self, tmp_path, payload):
|
||||
assert bob_preflight(self.ENV, self._settings(tmp_path, payload)) is None
|
||||
|
||||
def test_passes_when_no_settings_file(self, tmp_path):
|
||||
assert bob_preflight(self.ENV, tmp_path / "missing.json") is None
|
||||
|
||||
def test_fails_on_same_host_other_port(self, tmp_path):
|
||||
path = self._settings(tmp_path, {"gatewayUrl": "http://127.0.0.1:9999"})
|
||||
assert bob_preflight(self.ENV, path) is not None
|
||||
|
||||
def test_fails_on_foreign_gateway(self, tmp_path):
|
||||
path = self._settings(tmp_path, {"gatewayUrl": "https://api.eu-de.bob.ibm.com"})
|
||||
message = bob_preflight(self.ENV, path)
|
||||
assert message == (
|
||||
"Bob's saved gatewayUrl (https://api.eu-de.bob.ibm.com) overrides "
|
||||
"BOB_GATEWAY_URL, so Bob would bypass the Headroom proxy. Remove the "
|
||||
f"gatewayUrl entry from {path} (or set it to the proxy URL shown by this "
|
||||
"wrap) and retry. If your organisation enforces a GatewayUrl policy, Bob "
|
||||
"cannot be wrapped."
|
||||
)
|
||||
|
||||
|
||||
class TestOriginPassthrough:
|
||||
"""Bob builds full gateway paths itself; the catch-all must not re-prefix
|
||||
them. Regression for the 403 loop: base .../inference + inbound
|
||||
/inference/v1/model/info doubled the prefix, and /admin/v1/profile was
|
||||
misrooted under /inference."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"path",
|
||||
[
|
||||
"/inference/v1/model/info",
|
||||
"/inference/v1/embeddings",
|
||||
"/admin/v1/profile",
|
||||
"/admin/v1/teams/t1/users/u1",
|
||||
"/rag/v1/search", # IBM docs tools (search_ibm_docs)
|
||||
"/metrics-forwarder/v1/codeagent/core/metrics",
|
||||
],
|
||||
)
|
||||
def test_declared_paths_are_origin_rooted(self, path):
|
||||
assert (
|
||||
resolve_origin_passthrough_url(BASE, path) == f"https://api.us-east.bob.ibm.com{path}"
|
||||
)
|
||||
|
||||
def test_other_ibm_regions_get_the_same_rules(self):
|
||||
# IBM runs one gateway per region; --openai-api-url selects it.
|
||||
base = "https://api.eu-de.bob.ibm.com/inference"
|
||||
assert (
|
||||
resolve_origin_passthrough_url(base, "/admin/v1/profile")
|
||||
== "https://api.eu-de.bob.ibm.com/admin/v1/profile"
|
||||
)
|
||||
body = b'{"instances":[{"teams":[{"id":"t","region_domain":"eu-de.bob.ibm.com"}]}]}'
|
||||
assert json.loads(
|
||||
strip_origin_passthrough_response_keys(base, "/admin/v1/profile", body)
|
||||
) == {"instances": [{"teams": [{"id": "t"}]}]}
|
||||
# The web-login host is not a gateway.
|
||||
assert resolve_origin_passthrough_url("https://bob.ibm.com", "/admin/v1/profile") is None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("base", "path"),
|
||||
[
|
||||
(BASE, "/v1/embeddings"),
|
||||
("https://api.openai.com/v1", "/inference/v1/model/info"),
|
||||
(None, "/inference/v1/model/info"),
|
||||
],
|
||||
)
|
||||
def test_everything_else_falls_back(self, base, path):
|
||||
assert resolve_origin_passthrough_url(base, path) is None
|
||||
|
||||
|
||||
class TestResponseStrip:
|
||||
"""Bob 2.0.1 rewrites its gateway host from region_domain in the proxied
|
||||
/admin/v1/profile response while keeping the proxy's port; stripping the
|
||||
key keeps it on its configured gateway URL (the proxy)."""
|
||||
|
||||
def test_strips_region_domain_from_profile(self):
|
||||
body = json.dumps(
|
||||
{"profiles": [{"id": "p1", "region": "us-east", "region_domain": "us-east.x"}]}
|
||||
).encode()
|
||||
out = strip_origin_passthrough_response_keys(BASE, "/admin/v1/profile", body)
|
||||
assert out is not None
|
||||
assert json.loads(out) == {"profiles": [{"id": "p1", "region": "us-east"}]}
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("base", "path", "body"),
|
||||
[
|
||||
(BASE, "/admin/v1/profile", b'{"id": "p1"}'), # key absent
|
||||
(BASE, "/inference/v1/model/info", b'{"region_domain": "x"}'), # undeclared path
|
||||
("https://api.openai.com/v1", "/admin/v1/profile", b'{"region_domain": "x"}'),
|
||||
(BASE, "/admin/v1/profile", b"<html>403"), # not JSON
|
||||
],
|
||||
)
|
||||
def test_none_when_nothing_to_strip(self, base, path, body):
|
||||
assert strip_origin_passthrough_response_keys(base, path, body) is None
|
||||
|
||||
|
||||
class TestModeMismatchWarning:
|
||||
@staticmethod
|
||||
def _warnings(monkeypatch, running_config, requested=None) -> list[str]:
|
||||
if requested is None:
|
||||
monkeypatch.delenv("HEADROOM_MODE", raising=False)
|
||||
else:
|
||||
monkeypatch.setenv("HEADROOM_MODE", requested)
|
||||
lines: list[str] = []
|
||||
monkeypatch.setattr("click.echo", lines.append)
|
||||
_warn_proxy_mode_mismatch(running_config)
|
||||
return lines
|
||||
|
||||
def test_warns_on_mismatch(self, monkeypatch):
|
||||
(line,) = self._warnings(monkeypatch, {"mode": "cache"}, requested="token")
|
||||
assert "'token' mode" in line and "'cache' mode" in line
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("running_config", "requested"),
|
||||
[
|
||||
({"mode": "cache"}, "cache"),
|
||||
({"mode": "cache"}, "cost_savings"), # alias of cache
|
||||
({"mode": "token"}, None), # nothing requested
|
||||
({}, "token"), # pre-upgrade proxy without the field
|
||||
(None, "token"), # config unavailable
|
||||
],
|
||||
)
|
||||
def test_silent_otherwise(self, monkeypatch, running_config, requested):
|
||||
assert self._warnings(monkeypatch, running_config, requested) == []
|
||||
|
||||
|
||||
def test_passthrough_handler_roots_profile_at_origin_and_strips_region_domain():
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
|
||||
import httpx
|
||||
|
||||
from headroom.proxy.handlers.openai import OpenAIHandlerMixin
|
||||
|
||||
class _Upstream:
|
||||
calls: list[str] = []
|
||||
|
||||
async def request(self, **kwargs):
|
||||
self.calls.append(kwargs["url"])
|
||||
return httpx.Response(
|
||||
200,
|
||||
request=httpx.Request(kwargs["method"], kwargs["url"]),
|
||||
json={"id": "p1", "region_domain": "us-east.bob.ibm.com"},
|
||||
# Validators/integrity metadata describe the unfiltered bytes.
|
||||
headers={
|
||||
"ETag": '"upstream-v1"',
|
||||
"Last-Modified": "Mon, 28 Sep 2026 00:00:00 GMT",
|
||||
"Cache-Control": "max-age=60",
|
||||
"Content-Digest": "sha-256=:dW5maWx0ZXJlZA==:",
|
||||
"Digest": "SHA-256=dW5maWx0ZXJlZA==",
|
||||
"X-Request-Id": "req-1",
|
||||
},
|
||||
)
|
||||
|
||||
class _ProfileRequest:
|
||||
method = "GET"
|
||||
headers: dict[str, str] = {}
|
||||
url = SimpleNamespace(path="/admin/v1/profile", query="")
|
||||
|
||||
async def body(self) -> bytes:
|
||||
return b""
|
||||
|
||||
handler = object.__new__(OpenAIHandlerMixin)
|
||||
handler.http_client = _Upstream()
|
||||
|
||||
response = asyncio.run(handler.handle_passthrough(_ProfileRequest(), BASE))
|
||||
|
||||
assert handler.http_client.calls == ["https://api.us-east.bob.ibm.com/admin/v1/profile"]
|
||||
assert response.status_code == 200
|
||||
assert json.loads(response.body) == {"id": "p1"}
|
||||
forwarded = {k.lower() for k in response.headers}
|
||||
stale = {"etag", "last-modified", "cache-control", "content-digest", "digest"}
|
||||
assert not forwarded & stale, "filtered body must not carry the upstream's validators"
|
||||
assert response.headers["x-request-id"] == "req-1"
|
||||
assert response.headers["content-type"] == "application/json"
|
||||
|
||||
|
||||
class TestModeWarningOnEveryReusePath:
|
||||
"""Bob requests token mode; every path that hands it an existing proxy warns
|
||||
when that proxy runs a different mode, not only the ordinary reuse return."""
|
||||
|
||||
@pytest.fixture
|
||||
def cache_mode_proxy(self, monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_MODE", "token")
|
||||
monkeypatch.setattr(wrap_mod, "_check_proxy", lambda _p: True)
|
||||
monkeypatch.setattr(wrap_mod, "_foreign_listener", lambda _p: False)
|
||||
monkeypatch.setattr(
|
||||
wrap_mod, "_query_proxy_health", lambda _p: {"config": {"mode": "cache"}}
|
||||
)
|
||||
lines: list[str] = []
|
||||
monkeypatch.setattr("click.echo", lambda msg="", *a, **k: lines.append(str(msg)))
|
||||
return lines
|
||||
|
||||
def test_no_proxy_warns(self, cache_mode_proxy):
|
||||
assert wrap_mod._ensure_proxy_unlocked(8787, True) == (None, 8787)
|
||||
assert any("'token' mode" in line for line in cache_mode_proxy)
|
||||
|
||||
def test_recovered_persistent_proxy_warns(self, monkeypatch, cache_mode_proxy):
|
||||
from types import SimpleNamespace
|
||||
|
||||
import headroom.install.health as install_health
|
||||
|
||||
manifest = SimpleNamespace(profile="p", health_url="http://127.0.0.1:8787/readyz")
|
||||
monkeypatch.setattr(wrap_mod, "_find_persistent_manifest", lambda _p: manifest)
|
||||
monkeypatch.setattr(install_health, "probe_ready", lambda _url: False)
|
||||
monkeypatch.setattr(wrap_mod, "_recover_persistent_proxy", lambda _p: True)
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_routing_mismatches", lambda *_a, **_k: [])
|
||||
|
||||
assert wrap_mod._ensure_proxy_unlocked(8787, False) == (None, 8787)
|
||||
assert any("'token' mode" in line for line in cache_mode_proxy)
|
||||
|
||||
def test_stale_version_left_running_for_attached_clients_warns(
|
||||
self, monkeypatch, cache_mode_proxy
|
||||
):
|
||||
# Shared proxy on an older Headroom with other wrappers attached is
|
||||
# deliberately left running; Bob still ends up on its cache mode.
|
||||
monkeypatch.setattr(
|
||||
wrap_mod,
|
||||
"_query_proxy_health",
|
||||
lambda _p: {"version": "0.0.1", "config": {"mode": "cache"}},
|
||||
)
|
||||
monkeypatch.setattr(wrap_mod, "_live_proxy_clients", lambda *_a, **_k: ["other-wrapper"])
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_routing_mismatches", lambda *_a, **_k: [])
|
||||
monkeypatch.setattr(wrap_mod, "_find_persistent_manifest", lambda _p: None)
|
||||
|
||||
assert wrap_mod._ensure_proxy_unlocked(8787, False) == (None, 8787)
|
||||
assert any("'token' mode" in line for line in cache_mode_proxy)
|
||||
|
||||
def test_persistent_restart_warns_with_the_restarted_config(
|
||||
self, monkeypatch, cache_mode_proxy
|
||||
):
|
||||
from types import SimpleNamespace
|
||||
|
||||
import headroom.install.health as install_health
|
||||
|
||||
# Dormant persistent deployment: recovery brings it up without
|
||||
# --memory, so wrap restarts it from the manifest, in the manifest's
|
||||
# (cache) mode.
|
||||
manifest = SimpleNamespace(profile="p", health_url="http://127.0.0.1:8787/readyz")
|
||||
monkeypatch.setattr(wrap_mod, "_find_persistent_manifest", lambda _p: manifest)
|
||||
monkeypatch.setattr(install_health, "probe_ready", lambda _url: False)
|
||||
monkeypatch.setattr(wrap_mod, "_recover_persistent_proxy", lambda _p: True)
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_routing_mismatches", lambda *_a, **_k: [])
|
||||
restarted: list[int] = []
|
||||
monkeypatch.setattr(
|
||||
wrap_mod, "_restart_persistent_proxy", lambda _m, p: restarted.append(p) or True
|
||||
)
|
||||
monkeypatch.setattr(wrap_mod, "_query_proxy_config", lambda _p: {"mode": "cache"})
|
||||
|
||||
assert wrap_mod._ensure_proxy_unlocked(8787, False, memory=True) == (None, 8787)
|
||||
assert restarted == [8787]
|
||||
assert any("'token' mode" in line for line in cache_mode_proxy)
|
||||
|
||||
def test_feature_gap_without_pid_reused_as_is_warns(self, monkeypatch, cache_mode_proxy):
|
||||
# Proxy lacks --memory and exposes no PID, so wrap reuses it unchanged.
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_needs_version_restart", lambda _p: False)
|
||||
monkeypatch.setattr(wrap_mod, "_live_proxy_clients", lambda *_a, **_k: [])
|
||||
monkeypatch.setattr(wrap_mod, "_proxy_routing_mismatches", lambda *_a, **_k: [])
|
||||
monkeypatch.setattr(wrap_mod, "_find_persistent_manifest", lambda _p: None)
|
||||
|
||||
assert wrap_mod._ensure_proxy_unlocked(8787, False, memory=True) == (None, 8787)
|
||||
assert any("Cannot restart automatically" in line for line in cache_mode_proxy)
|
||||
assert any("'token' mode" in line for line in cache_mode_proxy)
|
||||
@@ -83,6 +83,9 @@ def test_direct_handler_routes_model_endpoint_intent() -> None:
|
||||
assert OPENAI_HANDLER_ROUTES == (
|
||||
ProviderHandlerRoute("POST", "/v1/chat/completions", "handle_openai_chat"),
|
||||
ProviderHandlerRoute("POST", "/chat/completions", "handle_openai_chat"),
|
||||
# Wrap-registry targets append their declared nonstandard chat paths
|
||||
# (IBM Bob posts under its /inference/v1 gateway prefix).
|
||||
ProviderHandlerRoute("POST", "/inference/v1/chat/completions", "handle_openai_chat"),
|
||||
)
|
||||
assert GEMINI_HANDLER_ROUTES == (
|
||||
ProviderHandlerRoute(
|
||||
|
||||
@@ -88,6 +88,8 @@ def test_health_preserves_backwards_compatible_config_payload(client):
|
||||
assert config["max_items_after_crush"] == 50
|
||||
assert config["smart_crusher_with_compaction"] is None
|
||||
assert isinstance(config["pid"], int)
|
||||
# `headroom wrap` reads this on proxy reuse to warn on a mode mismatch.
|
||||
assert config["mode"] in ("token", "cache")
|
||||
|
||||
|
||||
def test_health_reports_agent_savings_config():
|
||||
|
||||
Reference in New Issue
Block a user