Files
headroom/tests/test_cli/test_wrap_droid.py
bb8d18a56a feat(wrap): add headroom wrap droid for Factory Droid (#4143)
## Description

Add `headroom wrap droid` so Factory subscription traffic, including
Droid Core, can be compressed on the existing Factory plan.

Droid ignores `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` for
Factory-served models. It honors `FACTORY_API_BASE_URL`. This wrapper
points that at the local proxy and gives the proxy a Factory upstream
(`--factory-api-url` / `FACTORY_TARGET_API_URL`, default
`https://api.factory.ai`).

With a Factory upstream configured the proxy registers:

- `POST /api/llm/a/v1/messages` — compressed with the existing Anthropic
pipeline and forwarded to `{factory}/api/llm/a/v1/messages`. A
client-sent `x-headroom-base-url` is ignored. `x-headroom-*` headers are
stripped. Operator Anthropic extra headers are withheld from the
untrusted Factory host.
- Every other `/api/*` path — forwarded verbatim to Factory, except
Hermes Studio's scoped prefixes and requests that name their own
`x-headroom-base-url`.

Nothing outside `/api/` is sent to Factory.
`select_passthrough_base_url` and `upstream_trust` are untouched, so a
leftover Codex, Claude, or Copilot config pointed at a Factory-mode
proxy keeps its own upstream and credentials. That is the deliberate
difference from https://github.com/headroomlabs-ai/headroom/pull/2352,
which is still open and which this does not close. #2352 sends every
unrouted request on a Factory-mode proxy to Factory.

`wrap droid` writes no Droid config. It threads `factory_api_url`
through the existing proxy-reuse logic. With `--no-proxy` it refuses
unless `/health` shows a Headroom proxy serving the same Factory
upstream. OpenAI-shaped `/api/llm/o/*` traffic is forwarded
uncompressed. Those routes are deferred.

This commit is `74c889fb` rebased onto current `main` (`0dd8e0b5`). The
only conflicts were `docs/content/docs/meta.json` (kept both the new
`hermes` page and `droid`) and the `--no-proxy` branch in
`headroom/cli/wrap.py` (kept upstream's OpenAI upstream check and the
Factory check).

Refs #2351

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom wrap droid` sets `FACTORY_API_BASE_URL` to the local proxy
and resolves the Factory upstream from `--factory-api-url`, then
`FACTORY_API_BASE_URL`, then `https://api.factory.ai`. Loopback,
credential, query, and fragment URLs are rejected.
- Factory inference and `/api/*` passthrough routes are registered only
when a Factory upstream is configured. The default proxy is unchanged.
- `/health` reports `factory_api_url`.
- Docs: `docs/content/docs/droid.mdx`.

## Testing

- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

Rebased tree `/home/hermes/workspace/headroom-rebase` at `0dd8e0b5`,
importing that tree via `PYTHONPATH` and the existing venv at
`/home/hermes/workspace/headroom/.venv` (Python 3.11.16).

```text
tests/test_cli/test_wrap_droid.py ............................           [ 31%]
tests/test_provider_droid_runtime.py ................................    [ 66%]
```

60 passed. Those are the wrap launch-env tests and the Factory runtime
tests.

`tests/test_proxy/test_factory_decoy_proof.py` did not complete in this
worktree. The proxy refuses to start because `headroom._core` is not
built here (`ModuleNotFoundError`, exit 78). Setting
`HEADROOM_REQUIRE_RUST_CORE=false` still did not finish within 180s. The
decoy proof (decoy upstream stays silent, Factory stand-in receives the
inference request, spoofed `x-headroom-*` headers, REST passthrough,
`--no-proxy` refusals) was written for this change and passed in the
pre-rebase review session. It was not re-run to a green result after
this rebase.

`ruff` and `mypy` were not re-run after the rebase.

## Real Behavior Proof

- Environment: Linux, Python 3.11.16, rebased commit `0dd8e0b5` on top
of `upstream/main` `8c1beb20`. No Factory token. No call to
`api.factory.ai`.
- Exact command / steps:
`PYTHONPATH=/home/hermes/workspace/headroom-rebase
/home/hermes/workspace/headroom/.venv/bin/python -m pytest
tests/test_cli/test_wrap_droid.py tests/test_provider_droid_runtime.py`
- Observed result: 60 passed.
- Not tested: a live Factory subscription, `droid` itself, the socket
decoy proof on this rebased commit (blocked locally by the missing Rust
extension), `ruff`, `mypy`, and CI on GitHub.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. Factory routes register only when
`factory_api_url` / `--factory-api-url` is set.
- Minimum rollout channel: stable. Default proxy behavior is unchanged.
- Stable/default behavior changed: no, unless a Factory upstream is
configured.
- Kill switch / disable path: do not pass `--factory-api-url` and do not
set `FACTORY_TARGET_API_URL`. `wrap droid` is opt-in.
- Unsafe override required: no.
- Qualification impact: none on the default proxy.
- Rollback path: revert this commit. No config is written into Droid.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

The pre-rebase commit `74c889fb` was approved by a fresh Claude Opus 5.5
high review with an empty blocking list. This commit is that change
rebased onto current main. The two conflict resolutions are described
above and have not had a second Opus pass.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Does not close #2352. That PR is the prior attempt. This one is a
current-main rebuild with the catch-all leak removed: only `/api/*` is
forwarded to Factory.

akbarkanso is not a Headroom maintainer. Please do not expect this
account to merge.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4143"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `e28dfe6422dfcb9b2cbd5ce0fe673196d3b6ad40`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Updated against upstream/main e4a60ed7. Resolved the OpenAI passthrough
overlap by retaining main's prepared-request/model/timing parameters and
soft-passthrough guard alongside the contributor's verbatim flag;
generic compression requires both guards to permit it. Re-reviewed
Factory direct-routing/verbatim semantics and existing discussion. All
93 wrapper/runtime/Factory passthrough/socket-decoy tests pass with
existing native extension; changed-file Ruff lint/format and whitespace
checks pass. No contributor routing policy was rewritten.

Independently reran the official Droid1.3.0 CLI with isolated home,
dummy API key, actual Headroom/Uvicorn proxy and controlled LAN HTTP
upstream on this exact combined head. The CLI consumed the controlled
Anthropic SSE reply and exited zero with HEADROOM_QUALIFICATION_OK. The
message request preserved dummy Authorization and forwarded no
x-headroom headers. This is controlled transport/runtime qualification,
not a live Factory subscription or savings benchmark, and not a fresh
native build. Applied shared test-only Gemini fixture repair for
independently established main failures; all ten cases pass. Formal
approval awaits exact-head hosted qualification.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `7dd0725a9ac4dd8103dedf78552ceca886fc3620`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Clean mechanical merge with upstream main
c01cf07d94. Re-reviewed current
Factory/Droid routing and author/Devin discussion. Factory inference
retains force_direct to bypass translating backends; Factory passthrough
retains verbatim alongside main's soft-passthrough/prepared-request
guards. Hermes scoped proxy prefixes remain excluded from Factory
dispatch; the existing standalone environment precedence is
intentionally consistent with sibling targets, while Click/wrap retain
flag precedence. No contributor routing policy rewritten.

Fresh wrapper/runtime/Factory passthrough/socket-decoy/Hermes
verification: 100 passed, with test-client/runpy warnings only. All 14
changed Python files pass Ruff lint/format; main-relative whitespace
clean and owned tree clean. Tests use the current-main native DLL for
local startup. Earlier official Droid 1.3.0 controlled-upstream runtime
proof remains scoped to its original run; no fresh official CLI replay,
Factory subscription inference, savings benchmark or branch-native build
is claimed. No new blocker found; approval awaits exact-head hosted
checks. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-07 20:48:53 -06:00

332 lines
13 KiB
Python

"""Tests for ``headroom wrap droid`` (Factory Droid).
The command points Droid's ``FACTORY_API_BASE_URL`` at the local proxy and
starts (or reuses) a proxy whose ``factory_api_url`` is the real Factory
upstream. Nothing durable is written. Every proxy/launch seam is patched, so
no proxy starts and no Droid binary runs.
"""
from __future__ import annotations
import subprocess
from typing import Any
import click
import pytest
from click.testing import CliRunner
import headroom.cli.wrap as wrap_cli
from headroom.cli.main import main
FACTORY = "https://api.factory.ai"
OTHER_FACTORY = "https://eu.factory.example"
@pytest.fixture(autouse=True)
def _hermetic(monkeypatch: pytest.MonkeyPatch, tmp_path) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path / ".headroom"))
for name in (
"FACTORY_API_BASE_URL",
"FACTORY_TARGET_API_URL",
"ANTHROPIC_TARGET_API_URL",
"OPENAI_TARGET_API_URL",
"GEMINI_TARGET_API_URL",
"CLOUDCODE_TARGET_API_URL",
"VERTEX_TARGET_API_URL",
"AUGMENT_TARGET_API_URL",
"HEADROOM_BACKEND",
):
monkeypatch.delenv(name, raising=False)
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *a, **kw: [])
monkeypatch.setattr(wrap_cli, "_find_persistent_manifest", lambda port: None)
def _boom(message: str):
def _raise(*args: Any, **kwargs: Any):
raise AssertionError(message)
return _raise
def _invoke(args: list[str], env: dict[str, str] | None = None):
return CliRunner().invoke(main, ["wrap", "droid", *args], env=env)
# ── command: env, precedence, rejection, missing binary ───────────────
def test_launch_env_points_droid_at_proxy(monkeypatch: pytest.MonkeyPatch) -> None:
captured: dict[str, Any] = {}
monkeypatch.setattr(wrap_cli.shutil, "which", lambda name: "/usr/bin/droid")
monkeypatch.setattr(wrap_cli, "_launch_tool", lambda **kw: captured.update(kw))
result = _invoke(["--port", "8787", "--", "exec", "hi"])
assert result.exit_code == 0, result.output
assert captured["binary"] == "/usr/bin/droid"
assert captured["args"] == ("exec", "hi")
assert captured["env"]["FACTORY_API_BASE_URL"] == "http://127.0.0.1:8787"
assert captured["factory_api_url"] == FACTORY
assert captured["agent_type"] == "droid"
assert captured["tool_label"] == "DROID"
def test_fallback_port_rewrites_child_env(monkeypatch: pytest.MonkeyPatch) -> None:
captured: dict[str, Any] = {}
ensure_kwargs: dict[str, Any] = {}
def _ensure(port: int, no_proxy: bool, **kwargs: Any):
ensure_kwargs.update(kwargs)
return None, 8799
def _run(cmd: list[str], env: dict[str, str]):
captured["cmd"] = cmd
captured["env"] = env
return subprocess.CompletedProcess(cmd, 0)
monkeypatch.setattr(wrap_cli.shutil, "which", lambda name: "/usr/bin/droid")
monkeypatch.setattr(wrap_cli, "_ensure_proxy", _ensure)
monkeypatch.setattr(wrap_cli, "_push_runtime_env", lambda *a, **kw: None)
monkeypatch.setattr(wrap_cli.subprocess, "run", _run)
result = _invoke(["--port", "8787"])
assert result.exit_code == 0, result.output
assert captured["env"]["FACTORY_API_BASE_URL"] == "http://127.0.0.1:8799"
assert ensure_kwargs["factory_api_url"] == FACTORY
@pytest.mark.parametrize(
("args", "env", "expected"),
[
([], {}, FACTORY),
([], {"FACTORY_API_BASE_URL": "https://EU.factory.example/"}, OTHER_FACTORY),
(
["--factory-api-url", "https://flag.factory.example"],
{"FACTORY_API_BASE_URL": OTHER_FACTORY},
"https://flag.factory.example",
),
],
)
def test_upstream_precedence(args: list[str], env: dict[str, str], expected: str) -> None:
result = _invoke([*args, "--port", "8787", "--prepare-only"], env=env)
assert result.exit_code == 0, result.output
assert "FACTORY_API_BASE_URL=http://127.0.0.1:8787" in result.output
assert f"upstream={expected}" in result.output
@pytest.mark.parametrize(
("args", "env"),
[
(["--factory-api-url", "http://127.0.0.1:8787"], {}),
([], {"FACTORY_API_BASE_URL": "http://127.0.0.1:8787"}),
(["--factory-api-url", "http://localhost:9000"], {}),
(["--factory-api-url", "https://user:pw@api.factory.ai"], {}),
(["--factory-api-url", "https://api.factory.ai?x=1"], {}),
(["--factory-api-url", "https://api.factory.ai#frag"], {}),
],
)
def test_rejected_upstreams_never_launch(
monkeypatch: pytest.MonkeyPatch, args: list[str], env: dict[str, str]
) -> None:
monkeypatch.setattr(wrap_cli.shutil, "which", lambda name: "/usr/bin/droid")
monkeypatch.setattr(wrap_cli, "_launch_tool", _boom("must not launch"))
monkeypatch.setattr(wrap_cli, "_ensure_proxy", _boom("must not start a proxy"))
result = _invoke([*args, "--port", "8787"], env=env)
assert result.exit_code != 0
assert "FACTORY_API_BASE_URL" in result.output
def test_missing_binary_exits_before_proxy_work(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(wrap_cli.shutil, "which", lambda name: None)
monkeypatch.setattr(wrap_cli, "_ensure_proxy", _boom("must not ensure a proxy"))
monkeypatch.setattr(wrap_cli, "_start_proxy", _boom("must not start a proxy"))
result = _invoke(["--port", "8787"])
assert result.exit_code == 1
assert "'droid' not found" in result.output
# ── proxy routing plumbing ────────────────────────────────────────────
def test_requested_routing_uses_explicit_factory(monkeypatch: pytest.MonkeyPatch) -> None:
kwargs = {
"backend": None,
"openai_api_url": None,
"anthropic_api_url": None,
"vertex_api_url": None,
"clear_vertex_api_url": False,
}
_, urls = wrap_cli._effective_requested_proxy_routing(**kwargs, factory_api_url=FACTORY)
assert urls["factory_api_url"] == FACTORY
monkeypatch.setenv("FACTORY_TARGET_API_URL", OTHER_FACTORY)
_, urls = wrap_cli._effective_requested_proxy_routing(**kwargs)
assert urls["factory_api_url"] == OTHER_FACTORY
_, urls = wrap_cli._effective_requested_proxy_routing(**kwargs, factory_api_url=FACTORY)
assert urls["factory_api_url"] == FACTORY
def test_start_proxy_forwards_factory_flag_and_env(monkeypatch: pytest.MonkeyPatch, tmp_path):
captured: dict[str, Any] = {}
class _Proc:
returncode = None
def poll(self):
return None
def _popen(cmd, **kwargs):
captured["cmd"] = cmd
captured["env"] = kwargs["env"]
return _Proc()
monkeypatch.setattr(wrap_cli, "_get_log_path", lambda port=None: tmp_path / "proxy.log")
monkeypatch.setattr(
wrap_cli, "_get_proxy_stdio_log_path", lambda port=None: tmp_path / "stdio.log"
)
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_cli.subprocess, "Popen", _popen)
wrap_cli._start_proxy(8787, agent_type="droid", factory_api_url=FACTORY)
cmd = captured["cmd"]
assert cmd[cmd.index("--factory-api-url") + 1] == FACTORY
assert captured["env"]["FACTORY_TARGET_API_URL"] == FACTORY
def _health(config: dict[str, Any], *, sessions: int = 0) -> dict[str, Any]:
return {
"service": "headroom-proxy",
"version": wrap_cli._HEADROOM_VERSION,
"runtime": {"websocket_sessions": {"active_sessions": sessions, "active_relay_tasks": 0}},
"config": {"pid": 4242, "backend": "anthropic", **config},
}
def test_idle_non_factory_proxy_is_restarted_in_factory_mode(
monkeypatch: pytest.MonkeyPatch,
) -> None:
calls: list[Any] = []
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: _health({}))
monkeypatch.setattr(
wrap_cli, "_kill_proxy_by_pid", lambda pid, port: calls.append(("kill", pid)) or True
)
monkeypatch.setattr(wrap_cli, "_find_available_port", lambda start, **kw: start)
monkeypatch.setattr(
wrap_cli, "_start_proxy", lambda port, **kw: calls.append(("start", port, kw))
)
assert wrap_cli._ensure_proxy(8787, False, factory_api_url=FACTORY) == (None, 8787)
assert calls[0] == ("kill", 4242)
assert calls[1][0] == "start"
assert calls[1][1] == 8787
assert calls[1][2]["factory_api_url"] == FACTORY
def test_attached_non_factory_proxy_pushes_droid_to_dedicated_port(
monkeypatch: pytest.MonkeyPatch,
) -> None:
calls: list[Any] = []
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: _health({}))
monkeypatch.setattr(wrap_cli, "_live_proxy_clients", lambda *a, **kw: [{"pid": 1}])
monkeypatch.setattr(wrap_cli, "_kill_proxy_by_pid", _boom("attached proxy must survive"))
monkeypatch.setattr(
wrap_cli,
"_find_available_port",
lambda start, **kw: calls.append(("find", start)) or 8799,
)
monkeypatch.setattr(
wrap_cli, "_start_proxy", lambda port, **kw: calls.append(("start", port, kw))
)
assert wrap_cli._ensure_proxy(8787, False, factory_api_url=FACTORY) == (None, 8799)
assert calls[0] == ("find", 8787)
assert calls[1][1] == 8799
assert calls[1][2]["factory_api_url"] == FACTORY
def test_matching_factory_proxy_is_reused(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
monkeypatch.setattr(
wrap_cli,
"_query_proxy_health",
lambda port: _health({"factory_api_url": f"{FACTORY}/"}),
)
monkeypatch.setattr(wrap_cli, "_kill_proxy_by_pid", _boom("must reuse"))
monkeypatch.setattr(wrap_cli, "_find_available_port", _boom("must reuse"))
monkeypatch.setattr(wrap_cli, "_start_proxy", _boom("must reuse"))
assert wrap_cli._ensure_proxy(8787, False, factory_api_url=FACTORY) == (None, 8787)
# ── --no-proxy ────────────────────────────────────────────────────────
_NO_PROXY_REFUSALS = {
"no-listener": (False, None),
"foreign-service": (
True,
{"service": "something-else", "config": {"factory_api_url": FACTORY}},
),
"config-less": (True, {"service": "headroom-proxy"}),
"non-factory": (True, _health({})),
"other-factory": (True, _health({"factory_api_url": OTHER_FACTORY})),
}
@pytest.mark.parametrize("case", sorted(_NO_PROXY_REFUSALS))
def test_no_proxy_refuses_unverified_listener(monkeypatch: pytest.MonkeyPatch, case: str) -> None:
listening, payload = _NO_PROXY_REFUSALS[case]
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: listening)
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: payload)
monkeypatch.setattr(wrap_cli, "_query_proxy_config", lambda port: None)
with pytest.raises(click.ClickException, match="--no-proxy requires a Headroom proxy"):
wrap_cli._ensure_proxy(8787, True, factory_api_url=FACTORY)
@pytest.mark.parametrize("case", sorted(_NO_PROXY_REFUSALS))
def test_no_proxy_refusal_never_launches_droid(monkeypatch: pytest.MonkeyPatch, case: str) -> None:
listening, payload = _NO_PROXY_REFUSALS[case]
monkeypatch.setattr(wrap_cli.shutil, "which", lambda name: "/usr/bin/droid")
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: listening)
monkeypatch.setattr(wrap_cli, "_query_proxy_health", lambda port: payload)
monkeypatch.setattr(wrap_cli, "_query_proxy_config", lambda port: None)
monkeypatch.setattr(wrap_cli.subprocess, "run", _boom("droid must not launch"))
result = _invoke(["--port", "8787", "--no-proxy"])
assert result.exit_code != 0
assert "--no-proxy requires a Headroom proxy" in result.output
def test_no_proxy_matching_factory_launches(monkeypatch: pytest.MonkeyPatch) -> None:
captured: dict[str, Any] = {}
def _run(cmd: list[str], env: dict[str, str]):
captured["env"] = env
return subprocess.CompletedProcess(cmd, 0)
monkeypatch.setattr(wrap_cli.shutil, "which", lambda name: "/usr/bin/droid")
monkeypatch.setattr(wrap_cli, "_check_proxy", lambda port: True)
monkeypatch.setattr(
wrap_cli, "_query_proxy_health", lambda port: _health({"factory_api_url": FACTORY})
)
monkeypatch.setattr(wrap_cli, "_push_runtime_env", lambda *a, **kw: None)
monkeypatch.setattr(wrap_cli, "_start_proxy", _boom("--no-proxy must not start a proxy"))
monkeypatch.setattr(wrap_cli.subprocess, "run", _run)
result = _invoke(["--port", "8787", "--no-proxy"])
assert result.exit_code == 0, result.output
assert captured["env"]["FACTORY_API_BASE_URL"] == "http://127.0.0.1:8787"