Commit Graph
22 Commits
Author SHA1 Message Date
66e0086952 feat(opencode): add repeatable --extra-model flag for custom wrap-time models (#2364)
## Description

`headroom wrap opencode` injects a hardcoded model list
(`HEADROOM_OPENCODE_MODELS`) into the `headroom` provider block of
`opencode.json`. There's no way to add custom models (for example a
free-tier DeepSeek) at wrap time. Hand-patching `opencode.json` is the
only workaround, and the next wrap overwrites it.

This PR adds a repeatable `--extra-model` flag and a
`HEADROOM_OPENCODE_EXTRA_MODELS` env var. Both take
`id[=name[=context_window[=output_limit]]]` specs. Extra models are
merged into the injected provider block alongside the built-ins, and
flag specs win over env specs on id collision. `name` defaults to the
id, `context_window` to 200000, and `output_limit` to `min(16384,
context_window)`.

Fields are separated by `=` rather than `:` because model ids routinely
contain `:`. Ollama ids are `name:tag` (`qwen2.5-coder:7b`), and
OpenRouter free-tier ids end in `:free`. With a `:` separator those were
silently cut down to a different id (Devin review, 2026-10-03).

The PR also fixes the wrap/unwrap lifecycle the flag depends on. A wrap
that starts without an `opencode.json` used to write the file with no
backup and no record of what Headroom put in it, so `headroom unwrap
opencode` left the injected models and the proxy base URL behind while
reporting that OpenCode was no longer routed through Headroom, and a
re-wrap under `--openai-api-url` kept an extra model Headroom had
injected but no longer advertised. Wrap now records what it owns in
`opencode.json.headroom-managed.json`, and unwrap uses that record: it
removes the model ids and the base URL Headroom wrote, keeps everything
else in the file, drops an extra model that is no longer requested, and
deletes the config file when Headroom created it and nothing else is
left in it.

Closes #2359

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [x] Documentation update
- [ ] Refactoring (no functional changes)

## Changes Made

- `headroom/providers/opencode/config.py`:
- New `parse_extra_model_spec()`, which splits on `=` with an optional
fourth `output_limit` field and raises `ValueError` on an empty id or a
non-integer / non-positive limit.
- New `extra_models_from_env()`, which reads comma-separated specs from
`HEADROOM_OPENCODE_EXTRA_MODELS`.
- An `extra_models` parameter threaded through
`headroom_provider_entry()`, `_render_provider_block()` and
`inject_opencode_provider_config()`.
- With `keep_user_entries` (#3125's `--openai-api-url` gate),
`--extra-model` entries win over a kept user entry with the same id.
- New managed-state sidecar (`opencode.json.headroom-managed.json`)
recording the model ids and base URL Headroom injected and whether
Headroom created the config file. `inject_opencode_provider_config()`
writes it on every injection; `record_opencode_config_origin()` marks a
config that Headroom creates because none existed.
- `remove_managed_opencode_provider()` drops the recorded model ids, the
recorded base URL, and the provider block once nothing else is left in
it. Entries the user added or edited are preserved.
- `_drop_stale_managed_models()` prunes an extra model Headroom injected
but no longer advertises, including an extra that overrode a built-in id
and is dropped again, so a re-wrap without the flag restores the
built-in defaults rather than keeping the old override.
- `snapshot_opencode_config_if_unwrapped()` no longer snapshots a config
Headroom wrote itself as pre-wrap content.
- `headroom/providers/opencode/runtime.py`: extras also reach the
launch-time `OPENCODE_CONFIG_CONTENT`.
- `headroom/cli/wrap.py`:
- A repeatable `--extra-model` option on `wrap opencode`. Env and flag
specs are parsed up front, and a malformed spec raises
`click.BadParameter`. The merged dict goes to both
`inject_opencode_provider_config` call sites (prepare-only and launch)
and to the launch env. Also adds a help text and docstring example.
- `wrap opencode` records the config origin before the first config
mutation.
- `unwrap opencode` undoes the recorded provider entries when there is
no pre-wrap backup, keeps the rest of the file, and removes a config
Headroom created once nothing Headroom-owned is left in it.
- `docs/content/docs/opencode.mdx`: new "Adding models" subsection
(format, defaults, Ollama/OpenRouter example, built-in id override, the
env var's no-comma limitation) and an env-var table row. The subsection
and the "How It Works Under The Hood" unwrap step now describe the
managed-state record and the automatic cleanup it enables.
- Tests:
- Spec parsing: full spec with output, defaults, the output cap at small
contexts, colon-bearing ids (`qwen2.5-coder:7b`,
`deepseek/deepseek-chat:free`, a name containing `:`), and invalid specs
including a bad output limit.
- Env parsing, provider-entry merge, and end-to-end inject. Also
@JerrettDavis's collision test across both `keep_user_entries` values.
- CLI: the flag reaches `opencode.json`, the env var persists, both
reach `OPENCODE_CONFIG_CONTENT` on launch, the flag wins over a kept
user entry under `--openai-api-url`, and a malformed spec exits with a
usage error.
- Managed state: the sidecar records the injected ids and base URL, a
config the user already had is not marked as Headroom-created, a re-wrap
does not snapshot Headroom's own config, a dropped extra model is pruned
(including an override of a built-in id, which returns to the built-in
defaults), a user model and a user-edited model survive, and the
provider removal keeps unrelated user entries.
- Lifecycle: a fresh wrap followed by an unwrap removes the config file
again (with and without the MCP entry), user content added after a fresh
wrap survives the unwrap, a re-wrap without the extra model drops the
stale one, and a directly injected provider entry is undone by the
unwrap callback as well.
- A user setting that is false, zero or an empty string is content, not
an empty section: such a setting survives the unwrap instead of costing
the user the whole file.

## Testing

- [x] Unit tests pass locally
- [x] Lint/format/type checks pass locally
- [x] New tests added for new functionality

The new parsing tests fail against the previous `:`-based parser:

```text
FAILED tests/test_providers_opencode_config.py::test_parse_extra_model_spec_keeps_colons_in_ids[qwen2.5-coder:7b-...]
FAILED tests/test_providers_opencode_config.py::test_parse_extra_model_spec_keeps_colons_in_ids[deepseek/deepseek-chat:free=DeepSeek Free-...]
FAILED tests/test_providers_opencode_config.py::test_parse_extra_model_spec_full
...
================ 21 failed, 2 passed, 103 deselected in 0.86s =================
```

With this branch (head 6be848942):

```text
$ pytest tests/test_providers_opencode_config.py tests/test_cli/test_wrap_opencode.py -q
179 passed, 2 skipped in 3.31s

$ pytest tests -q -k "opencode"
322 passed, 26 skipped

$ ruff check headroom/providers/opencode/config.py headroom/cli/wrap.py tests/test_providers_opencode_config.py tests/test_cli/test_wrap_opencode.py
All checks passed!
$ ruff format --check (same files)
4 files already formatted
$ mypy headroom --ignore-missing-imports --no-site-packages
headroom\fileperms.py:92: error: Module has no attribute "fchmod"; maybe "chmod"?  [attr-defined]
headroom\proxy\interceptors\astgrep.py:220: error: Module has no attribute "O_DIRECTORY"  [attr-defined]
...
Found 7 errors in 2 files (checked 561 source files)
```

The mypy errors are the Windows-only `os.fchmod` / `os.O_DIRECTORY`
false positives in files this PR does not touch. The same Windows
path-escaping failures that `test_build_launch_env_with_project` used to
hit (`tests/test_cli/test_unwrap_claude.py`,
`tests/test_wrap_code_memory.py`, `tests/test_install/test_planner.py`,
`tests/test_cli/test_wrap_stale_marker.py`) reproduce on unchanged
`main` and are unrelated to this change. `Dependency audit (pip-audit)`
passes on this head; only `Rust advisories (cargo audit)` fails, on
advisories published after upstream `main`'s last green run (2026-11-13
04:47) — `quinn-udp` `RUSTSEC-2026-0538` and `rustix`
`RUSTSEC-2026-0486`. `Cargo.lock` on this branch is byte-identical to
upstream `main` (`81818d49`), so the PR does not introduce them.

## Real Behavior Proof

- Environment: Windows 11, Python 3.13, this branch vs. the previous
`:`-based parser and vs. the previous unwrap lifecycle,
`HOME`/`USERPROFILE` pointed at a temp dir.
- Exact command / steps: `headroom wrap opencode --prepare-only
--no-serena --extra-model "qwen2.5-coder:7b" --extra-model
"deepseek/deepseek-chat:free=DeepSeek Free=65536"`, then read
`provider.headroom.models` from `~/.config/opencode/opencode.json`; then
`headroom unwrap opencode --no-stop-proxy` against a config that did not
exist before the wrap.
- Observed result: with the old parser the entries were `qwen2.5-coder`
(name `7b`) and `deepseek/deepseek-chat` (name `free=DeepSeek
Free=65536`, context 200000): wrong ids, and the paid DeepSeek model
instead of the free one. With this branch they're `qwen2.5-coder:7b`
(name defaults to the id, context 200000, output 16384) and
`deepseek/deepseek-chat:free` (name `DeepSeek Free`, context 65536,
output 16384), next to the built-in `gpt-4o` / `gpt-4.1`. After the
unwrap of a fresh wrap, `opencode.json` and its managed-state sidecar
are both gone and no `provider.headroom` block is left. After the unwrap
of a fresh wrap the user had added to, the file keeps the user's own
model id, key and `theme`, and only the injected ids and the
`http://127.0.0.1:8787/v1` base URL are gone.
- Not tested: a full (non-prepare-only) wrap launching the real OpenCode
binary against a live upstream; the OpenCode TS plugin side (extras are
user-supplied and intentionally not synced into
`plugins/opencode/src/provider.ts` `DEFAULT_MODELS`).

## Runtime Rollout Safety

- Rollout-managed feature(s): none; CLI-only opt-in flag and env var on
`headroom wrap opencode`, plus a `opencode.json.headroom-managed.json`
sidecar next to the OpenCode config
- Minimum rollout channel: stable
- Stable/default behavior changed: no. Without `--extra-model` or
`HEADROOM_OPENCODE_EXTRA_MODELS` the injected provider block is
unchanged. `unwrap opencode` now also removes the provider entries
Headroom recorded, which is what it already reported doing
- Kill switch / disable path: drop the flag and unset
`HEADROOM_OPENCODE_EXTRA_MODELS`, then re-wrap; delete the sidecar to
stop Headroom from tracking the config
- Unsafe override required: no
- Qualification impact: none; no proxy, compression or request-path code
changed
- Rollback path: revert the PR; `headroom unwrap opencode` restores the
pre-wrap `opencode.json` when a backup exists

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2364"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-17 02:38:10 +05:30
2649835a51 feat(wrap): make the Serena code-memory MCP opt-in (#4509)
## What changes

`headroom wrap claude|codex|grok|opencode` no longer registers Serena by
default.

| | Before | After |
|---|---|---|
| Default | Serena registered | No code-memory MCP |
| Opt in | (default) | `--code-memory serena` or
`HEADROOM_CODE_MEMORY=serena` (deprecated `--serena` still works) |
| Opt out | `--code-memory none` / `--no-serena` | Still accepted |
| Upgrading user, Serena installed by Headroom | Kept | Removed on next
wrap, with the message "Serena is now off by default; pass --code-memory
serena to keep it" |
| Upgrading user, Serena added by hand | Kept | Kept, no message (shown
only with `--verbose` or an explicit opt-out) |

OpenCode now accepts `--code-memory` and uses the same setup path as the
other agents. Its `--no-serena` flag is hidden and deprecated.

## Why

Measured on 2026-10-27, Serena started exactly as `wrap` registers it:

- It adds 22 tools, about 5.9K tokens of tool definitions on every
request where tools aren't deferred. Its server instruction tells the
model to call `initial_instructions`, which costs another 2K tokens.
- Each launch takes about 6 s cold via `uvx`, plus a background `serena
project index` that starts language servers. It also writes a `.serena/`
folder into the user's repo.
- Serena was registered in 6 of our own Claude Code sessions and called
0 times. Separately, an offline replay of about 108K tool calls found
that symbol-level reads have little to recover for Claude Code.

## Not in this PR

- `headroom.integrations.strands` still defaults
`enable_serena_mcp=True`. It's public integration API, so it needs a
deprecation step instead of a silent flip.
- The opt-in `--serena-instructions` block in CLAUDE.md/AGENTS.md is
never removed, by unwrap or anything else. Users who opted into it keep
instructions that point at tools no longer registered. This predates
this PR and will be fixed separately.
- For Claude Code, Serena entries are per project, so removal happens
the next time `wrap` or `unwrap` runs in each project. The docs say so.

## Tests

- `pytest tests/test_wrap_code_memory.py tests/test_cli
tests/test_mcp_registry`: 1368 passed. 5 failed, and those 5 fail the
same way on `main` in my local venv:
`test_mcp_reconcile.py::test_ordinary_install_does_not_adopt_serena` and
4 in `test_wrap_persistent.py`, all from an environment/install
mismatch.
- New tests:
- The default wrap removes a Serena entry Headroom installed and prints
how to keep it.
- The default wrap leaves a hand-added Serena alone and prints nothing.
  - `--serena --no-serena` resolves to none.
- `wrap opencode --code-memory serena` registers Serena with the `agent`
context.
- `ruff check`, `ruff format --check` and `mypy` are clean on the
changed files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

## Real Behavior Proof

- **Environment:** macOS (Darwin 27.2), Python 3.12.13, Claude Code
2.1.318, real `claude` CLI doing every `claude mcp add`/`remove`. A
temporary `HOME`, so `~/.claude.json` starts empty, and a fresh git repo
per case.
- **Builds:** "before" = `main` at `12864fc8`; "after" = this PR at
`56557dc9`.
- **Command:** `headroom wrap claude --no-proxy [flags] -- --version`.
Wrap does its full MCP setup, then launches Claude Code, which prints
its version and exits, so no model call is made. `state` is read from
`~/.claude.json` (user scope and this project's local scope) and the
project's `.mcp.json` after each run.

```
== 1. Upgrade: Serena installed by Headroom (project/local scope)
    $ headroom wrap claude --no-proxy -- --version    (cwd=projA, build=12864fc8)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projA): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projA, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projA): - | .mcp.json: -

== 2. Upgrade: Serena added by hand (claude mcp add)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projB, build=56557dc9)
    exit=0
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy --code-memory none -- --version    (cwd=projB, build=56557dc9)
    exit=0
    wrap|   Serena MCP is present but user-managed — leaving it in place (Headroom only removes entries it installed)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -

== 3. Explicit opt-in, then rollback to the default
    $ headroom wrap claude --no-proxy --code-memory serena -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy --serena -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -

== 4. User scope: installed by Headroom with --code-memory-scope user
    $ headroom wrap claude --no-proxy --code-memory-scope user -- --version    (cwd=projD, build=12864fc8)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     registered for every Claude Code session on this machine (--code-memory-scope user)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom', 'serena'] | local(projD): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projD, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projD): - | .mcp.json: -

== 5. User scope opt-in on the PR build
    $ headroom wrap claude --no-proxy --code-memory serena --code-memory-scope user -- --version    (cwd=projE, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     registered for every Claude Code session on this machine (--code-memory-scope user)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom', 'serena'] | local(projE): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projE, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projE): - | .mcp.json: -
```

What this shows:
1. A Serena entry Headroom installed on `main` is removed on the first
wrap after upgrading, with the "off by default" message. This holds for
both scopes (cases 1 and 4).
2. A hand-added Serena entry is kept, and the default wrap prints
nothing. `--code-memory none` keeps it and says it's user-managed (case
2).
3. Opting in works with `--code-memory serena`,
`HEADROOM_CODE_MEMORY=serena` and the deprecated `--serena`, at project
scope and user scope (cases 3 and 5). Running the default afterwards
rolls it back.
4. Project scope here means Claude Code's `local` scope, which is where
wrap registers. Wrap never writes `.mcp.json`.

Not tested live: `wrap codex|grok|opencode` (unit tests only), Windows,
and an interactive Claude session that calls Serena tools.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4509"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer post-merge qualification
Published `983b6bb15119a3ecb8f6e20bf014b8bed123e95d` includes upstream
main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 19 selector/Serena
tests pass,1optional skip; Ruff/format/diff pass. Clean main sync;
contributor selector/ownership code unchanged. Existing real
upgrade/configuration/launch proof requirement remains; jCodeMunch
proposal is unrelated to this source delta. Contributor correctness
logic was not rewritten. Existing Changes Requested decision retained
without duplicate findings. Scoped Windows validation uses the existing
development native extension, not a fresh build. Hosted qualification is
handled by maintainers; nothing merged.


### Maintainer current-main requalification

Exact head `e8e3011187a2ffdfbb6c9932db26b22ec911ab39`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Mechanically updated from upstream/main
ecbaf86d78 without changing contributor
selector/ownership logic. Reviewed the current discussion and
integration delta. All 29 focused code-memory/Serena-disable/Gemini
tests pass with one optional skip; Ruff0.17.3 lint/format and whitespace
checks pass. Existing Windows HOME/USERPROFILE fixture isolation remains
intact.

The prior upgrade/runtime qualification requirement remains unresolved:
post-patch real configuration/launch evidence must cover an existing
Headroom-owned Serena entry, a hand-managed entry, explicit opt-in, and
Claude project/user scopes, with commands/results and untested-agent
limits. No new author evidence was found to close it. This pass does not
claim those actual launch/upgrade flows were executed; passing unit
tests and the main sync do not establish that deployment behavior. The
hold is not an inherited-main CI defect. Maintainers handle hosted
checks; no contributor CI execution requested. Nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `0b49a79cb192b294dde7ecb1c72b75fffa2733d5`, upstream main
`8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`.

Clean mechanical sync with upstream
main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a without changing
contributor selector/ownership logic. Current source, author discussion
and Devin findings reviewed.29focused code-memory/Serena-disable/Gemini
tests pass with1optional skip; changed Python Ruff0.17.3check/format and
whitespace pass. Windows HOME/USERPROFILE fixture isolation preserved.

Existing real upgrade/runtime qualification remains missing: existing
Headroom-owned Serena versus hand-managed entry, explicit opt-in, and
Claude project/user scopes need observed configuration/launch results
with explicit untested-agent limits. No new evidence found to close it;
these launch/upgrade paths were not executed in this pass. Passing unit
tests and clean main integration do not establish deployment behavior.
No new provider/native-build claim or inherited-main pipeline
attribution. Maintainers handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `30dfb386fa7284c9817bf5d7db0648fb10830055`, upstream main
`7592e681459f14a2041eb5f524a658cb6ad1dd92`.

Clean current-main
sync7592e681459f14a2041eb5f524a658cb6ad1dd92,including HNSW re-index and
memory relation-key fixes after917release-recovery merge. Contributor
selector/ownership logic unchanged; fresh discussion
considered.37wrapper/Serena/memory-regression/Gemini tests pass
with1optional skip;whitespace passes. PriorRuff0.17.3validation applies
to unchanged PR files.

Existing real upgrade/launch evidence hold remains: Headroom-owned
versus hand-managed Serena
entries,explicitoptin,andClaudeproject/userscopes need
observedpostpatchresults. No newevidence closesit;actualupgrade/launch
pathsnotrerun. No freshnativebuild/provider claim.
MaintainershandleCI;nothingmerged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `e1778a5368626547ad958926d5570edf153182f6`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Clean integration with upstream/main
2d3ee51846 after the external 0.43.1
release merge. Exact headroom/crates/tests comparison against previously
reviewed 30dfb386f shows no changes. Re-reviewed generated metadata; 78
release/version/MCP tests pass on this tree and whitespace passes.
Earlier 37 wrapper/Serena/memory/Gemini passes and one optional skip
remain scoped to unchanged inputs.

Existing actual upgrade/launch proof hold remains: Headroom-owned versus
hand-managed Serena entries, explicit opt-in, and Claude project/user
scopes need observed postpatch results. Those runtime paths were not
rerun and no new evidence closes the hold. No contributor ownership
policy changed, no fresh native-build/provider claim. Maintainers handle
checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `173c5739d7357ca9398faa41b4d1ce3b72d6c550`, upstream main
`7877241d84c93b3ec487e07324245838ddae85cc`.

Cleanly integrated upstream/main7877241d84c93b3ec487e07324245838ddae85cc
and re-reviewed the wrapper overlap with Hermes registration and port
discovery.35 selector/Serena-disable/Hermes provider and wrapper tests
pass,1optional skip; Ruff and whitespace pass. Contributor
ownership/default policy is unchanged. Existing actual
upgrade/configuration/launch proof requirement remains open for
Headroom-owned versus user-managed entries, explicit opt-in, and Claude
project/user scopes. No runtime proof is inferred from mocks or
native-build claim made. Maintainers handle hosted checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `eb1c3d66669cf71f881f1ba06dc049d36b20da7b`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 35 passed, 1 skipped in 2.93s.
Selected real dependency overlays and native DLL paths are recorded in
final4509-fresh-native-tests.log; the DLL provenance/input hashes are in
final-native-artifact-proof.json. No redundant rerun is implied after
this Cargo-only merge. Whitespace passes. Prior source review and scoped
real-behavior evidence remain applicable to unchanged contributor
inputs.

Existing real upgrade/configuration/launch qualification remains open
for Headroom-owned versus user-managed Serena entries, explicit
opt-in/rollback, and Claude project/user scopes. No new author evidence
closes it; contributor ownership/default policy is unchanged.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `43bfea8a164c21ab69793b3905fbf1457fef7bc2`, upstream main
`da6d9736cd82074988c7d0a2360b49d9d8b6c0a0`.

Current-main requalification against
da6d9736cd. All69 integration patches
exactly match c01cf07-to-current-main delta after diff-location
normalization; contributor selector/default/Serena ownership logic
unchanged.162 focused code-memory,Serena disable,Hermes,Windows
supervisor,search-compressor tests pass;3 platform/optional skips. All31
changed Python files pass Ruff lint/format; offline lock and whitespace
pass. Native pr3982 DLL verified against complete tracked native-input
hash proof and artifact checksum; no fresh native-build claim.
Author/Devin discussion checked. Existing real
upgrade/configuration/launch qualification remains open: prior
Headroom-owned Serena entry versus user-managed entry, explicit
opt-in/rollback, Claude project/user scopes. No new author evidence
closes it; this run does not assert actual agent launch or deployment
proof. Maintainers handle hosted checks; no contributor policy rewrite
or merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `56557dc91725422257f90c3cb5c5f04a665e6043`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 19 integration patches
exactly match the prior-reviewed-main da6d9736 to current-main delta
after diff-location normalization; contributor selector/default/Serena
ownership logic is unchanged. Fresh code-memory/Serena
disable/scope/reconciliation/migration, doctor and incoming
Vertex/backend/CLI tests: 330 passed, 12 skipped, 1 warning. Fourteen
changed Python files pass Ruff lint/format; offline lock and whitespace
checks pass.

Author/Devin discussion read. Existing real upgrade/configuration/launch
qualification remains open: prior Headroom-owned entry versus
user-managed entry, explicit opt-in/rollback, Claude project/user
scopes. This run establishes no actual agent launch or deployment proof.
Static source/test scope and security inspection found no new workflow,
dependency or setup path; the inherited flagged workflow-validation job
is excluded by the exact PR's unchanged workflow paths. Native DLL
reused after complete tracked input-set/hash and artifact-checksum
verification; no fresh build claimed. Maintainers handle exact-head
hosted checks; no PR merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-09 12:23:08 -06:00
1dc7a183f5 fix(copilot): honour GitHub's advertised Business/Enterprise host by default (#4155)
## Description

Supersedes #3403 (parked, stacked on the conflicting #3402) as a
standalone change on `main`.

GitHub's token exchange advertises a per-plan chat host in
`endpoints.api`: `api.business.githubcopilot.com` for Business seats and
`api.enterprise.githubcopilot.com` for Enterprise seats, and GitHub's
own clients route with it. Since #2441 Headroom has folded both into the
generic `api.githubcopilot.com`, because the segmented hosts lagged on
newer Claude models at the time.

That fold breaks enterprises on GitHub's [subscription-based network
routing](https://docs.github.com/en/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-access/manage-network-access):
their firewall allows only the plan host, the generic host is
unreachable, and the failure looks like a random connection error. It
also made the VS Code guide's Enterprise section wrong, which said
Headroom uses the advertised URL.

This PR honours the Business and Enterprise advertised hosts by default
and adds `GITHUB_COPILOT_USE_ADVERTISED_HOST=0` (also
`false`/`no`/`off`) to restore the generic-host fold for seats where the
segmented host still lags. An explicit `GITHUB_COPILOT_API_URL` or
enterprise domain still wins over both. The individual host keeps being
folded (#610 is unchanged).

Refs #2441, #610, #3352 (an Enterprise seat behind a corporate firewall;
the inline-completions 401 there is #4144).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- `headroom/copilot_auth.py`:
`_subscription_api_url_from_user_info_payload` returns the advertised
host for the two plan hosts unless `GITHUB_COPILOT_USE_ADVERTISED_HOST`
opts out, and logs the chosen host with the opt-out next to it. That
line is emitted where the host is resolved: the `wrap` process for the
subscription lanes, which does not write to `proxy.log`. The dedicated
proxy's `/health` (`config.openai_api_url`) is where the chosen host
shows up. The generic and individual hosts still normalize to
`DEFAULT_API_URL`. `resolve_copilot_api_url`'s docstring no longer
claims the advertised host is ignored on purpose. Every subscription
lane (`wrap vscode`, `wrap copilot --subscription`, OpenCode
`--copilot-subscription`) already pins `resolution.api_url` into the
dedicated proxy as `--openai-api-url` and `GITHUB_COPILOT_API_URL`, so
they pick this up without changes; `is_copilot_api_url`,
`copilot_completions_base_url` and `_github_host` already treat
`*.githubcopilot.com` as a public Copilot host (verified by an
independent review of the diff).
- `headroom/cli/wrap.py`: `wrap copilot` help text describes the
per-plan routing for `--subscription`; the implicit OAuth lane still
uses the generic host (it performs no token exchange), which the text
now says.
- Docs: `docs/content/docs/vscode-copilot.mdx` Enterprise section is now
true and names the opt-out and the firewall case;
`docs/content/docs/corporate-networks.mdx` lists the plan hosts;
`TESTING-copilot-subscription.md` host section split by lane:
`--subscription` / `wrap vscode` / OpenCode honour the plan host, the
implicit OAuth `wrap copilot` lane is explicitly generic-host (it
performs no token exchange at launch, and `resolve_copilot_api_url` is
pinned network-free by an existing test), and its "open an issue for
auto-detection" paragraph removed.
- Tests: Business and Enterprise honoured; individual and generic still
folded; opt-out restores the fold; explicit pin wins; `wrap copilot
--subscription` pins the advertised host; the implicit OAuth lane stays
on the generic host even when user-info advertises the Enterprise host;
OpenCode subscription lane expectation updated. Three existing tests
that asserted the fold were updated to the new contract.

Follow-ups, not in this PR: `headroom doctor --network` still probes
only `api.githubcopilot.com` (it should probe the resolved plan host);
the implicit OAuth `wrap copilot` lane could resolve the plan host
through the token exchange too.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed (live Copilot Business seat, 2026-10-13;
see Live verification)

### Test Output

```text
$ pytest tests/test_copilot_auth.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_vscode.py \
    tests/test_copilot_vscode_completions_routing.py tests/test_provider_copilot_vscode_config.py \
    tests/test_cli/test_wrap_opencode.py tests/test_proxy_copilot_auth_hooks.py tests/test_provider_proxy_routes.py \
    tests/test_copilot_integration_id_hmac.py tests/test_proxy/test_anthropic_copilot_upstream_auth.py \
    tests/test_copilot_native_mode.py tests/test_copilot_subscription_smoke.py tests/test_cli/test_copilot_auth.py \
    tests/test_cli/test_wrap_persistent.py
449 passed, 4 deselected
# the 4 deselected tests (test_wrap_persistent.py::*recover*) fail identically on the base commit 976aa714
# with AttributeError: 'TaskManifest' object has no attribute 'runtime_kind'; unrelated to this change

$ uvx ruff@0.16.8 check . && uvx ruff@0.16.8 format --check .
All checks passed! / 1777 files already formatted
$ mypy headroom/copilot_auth.py headroom/cli/wrap.py --ignore-missing-imports
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, repo venv, native core loaded.
- Exact command / steps: the suites above, with the new regressions
failing when `_subscription_api_url_from_user_info_payload` is reverted
to the fold.
- Observed result: the token-exchange and user-info producers, `wrap
vscode`, `wrap copilot --subscription` and the OpenCode lane all hand
the dedicated proxy `https://api.enterprise.githubcopilot.com` (or
`.business.`) as both `--openai-api-url` and `GITHUB_COPILOT_API_URL`;
completions still resolve to `copilot-proxy.githubusercontent.com`.
- Live seat: see "Live verification" below. Not tested: an
Enterprise-plan seat (`api.enterprise.`; same code branch as Business),
and the `wrap copilot --subscription` lane (the test seat has the
Copilot CLI policy off).

### Verification needed before release

No Business or Enterprise seat is available to us. Carrier (Enterprise
seats behind a subscription-routed firewall) runs, on this branch, with
`SSL_CERT_FILE` unset and no `GITHUB_COPILOT_API_URL` exported:

```bash
headroom copilot-auth login            # device flow; approve in the browser
headroom wrap vscode                   # leave running; note the port it prints
curl -s http://127.0.0.1:<port>/health | python3 -m json.tool | grep openai_api_url
#   expected: "openai_api_url": "https://api.enterprise.githubcopilot.com"
# then send one Copilot Chat turn in VS Code (any model) and one inline completion
headroom wrap copilot --subscription -- --model claude-sonnet-4.6 -s -p "Reply with exactly: OK"
#   expected: the banner shows COPILOT_PROVIDER_API_URL=https://api.enterprise.githubcopilot.com and the reply is OK
```

`curl -s http://127.0.0.1:<port>/health` on the dedicated proxy should
report `"openai_api_url": "https://api.enterprise.githubcopilot.com"`
(the "advertised plan host" log line is printed by the wrap process, not
written to `proxy.log`). If a model that the generic host serves is
rejected on the plan host (`400 ... model is not supported`), the #2455
condition still holds for that seat: set
`GITHUB_COPILOT_USE_ADVERTISED_HOST=0`, restart the wrapper, and report
the model and the response body on this PR before it is released.

### Live verification (2026-10-13, Copilot Business seat)

Branch merged with #4154 on current `main`, isolated
`HEADROOM_WORKSPACE_DIR`, `SSL_CERT_FILE` and `GITHUB_COPILOT_API_URL`
unset, `headroom copilot-auth login` device flow.

- `/copilot_internal/user` for the seat: `copilot_plan: business`,
`endpoints.api: https://api.business.githubcopilot.com`. (For about
three hours after the seat was assigned GitHub still advertised
`api.individual.`; the plan host appeared on its own.)
- Resolution: default → `https://api.business.githubcopilot.com`;
`GITHUB_COPILOT_USE_ADVERTISED_HOST=0` →
`https://api.githubcopilot.com`; explicit `GITHUB_COPILOT_API_URL` wins.
`main`'s `copilot_auth.py` resolves the same seat to
`https://api.githubcopilot.com`.
- `headroom wrap vscode`: dedicated proxy started with `--openai-api-url
https://api.business.githubcopilot.com`; `/health` reports it.
- Model sweep through the proxy, one turn per picker model on each
endpoint it lists (plus `gpt-4o-mini`): **36/36 returned 200** on the
plan host (Claude Haiku/Sonnet/Opus 4.x–5.5 on `/v1/messages` and
`/chat/completions`, Gemini 3.7/3.8/4, GPT-5.x/6.x and Grok on
`/responses`, `gpt-5-mini`, `gpt-4o-mini`). No `model_not_supported`, so
the #2441/#2455 lag does not reproduce on Business today.
- Real VS Code (Copilot Chat 0.69): Ask mode on Auto (`gpt-6-luna`),
Claude Sonnet 5.5 chat (`/v1/messages`), and Agent mode with tool calls
(wrote and ran a pytest file) all went to
`api.business.githubcopilot.com`; every Copilot upstream call in the
proxy log used the plan host, none the generic host.
- Found while testing, not caused by this PR and present on `main`:
ghost text / next edits (`copilot-nes-*` on `/chat/completions`) are
sent to CAPI and get `400 model_not_supported` on both the plan host and
the generic host. Fixed separately in #4188.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; env-gated.
- Minimum rollout channel: stable.
- Stable/default behavior changed: yes. Business and Enterprise
subscription sessions now route to their advertised plan host instead of
`api.githubcopilot.com`. Individual seats, explicit pins and GHE domains
are unchanged.
- Kill switch / disable path: `GITHUB_COPILOT_USE_ADVERTISED_HOST=0`
restores the previous fold without a rollback; `GITHUB_COPILOT_API_URL`
pins any host.
- Unsafe override required: no.
- Qualification impact: adds plan-host routing coverage to the Copilot
auth and wrap suites.
- Rollback path: revert the commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-13 15:20:50 -07:00
1fb4cbaea0 fix(proxy): astgrep disk verify truncation (#3192)
## Description

`_detect_truncation()` currently relies on client-specific truncation
banners in `tool_result` text. This works for clients that emit a
recognizable line-count banner, but not for clients such as Cursor
(byte-based, direction-reversed truncation) and OpenCode (placeholder +
external file pointer).

Because `headroom proxy` runs alongside the client it wraps, it can
optionally verify the returned content against the actual file on disk.
If the `tool_result` is a strict prefix of the file, we can identify
truncation without depending on client-specific output formatting.

This adds disk verification as an opt-in fallback: the existing
banner-based detection always runs first, and disk verification is only
attempted when no banner is found. All disk-verification failures or
ambiguous cases return `UNKNOWN` rather than guessing.

Following review, the verifier is also constrained to a trusted,
confined workspace root rather than trusting caller-supplied input
directly — see the last four bullets under Changes Made.

**Current scope:** the session-token workspace binding this fallback
requires is wired for `headroom wrap claude` and plugin-loaded `headroom
wrap opencode` only. Cursor remains unsupported/not tested — it has no
header-injection surface today to transmit an authoritative session
token, so disk verification stays `UNKNOWN` for it; extending Cursor
support needs a separate, safe transport design.

Related to #2430
Closes #2421

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made
- Added optional disk-based Read truncation verification behind
`HEADROOM_VERIFY_TRUNCATION_ON_DISK`.
- Disk verification compares the visible `tool_result` against the file
on disk and distinguishes `COMPLETE`, `TRUNCATED`, and `UNKNOWN`.
- Disk verification only runs when the existing `_detect_truncation()`
banner check finds no truncation marker, avoiding unnecessary file I/O
when the client already provides a signal.
- Added a `current_cwd` contextvar populated from `x-headroom-cwd` so
relative Read paths can be resolved against the client's working
directory.
- Registered `HEADROOM_VERIFY_TRUNCATION_ON_DISK` as a hot-reloadable
runtime setting.
- Gated verification on a new `is_current_request_trusted` contextvar,
set from `is_loopback_host(request.client.host)` — a server-observed
fact, not the caller-supplied header — so `x-headroom-cwd` is never
sufficient authority on its own.
- Confined all path resolution strictly under the canonicalized `cwd`
(`.resolve()` + `relative_to()`), rejecting `..` traversal,
absolute-path escapes, and post-resolution symlink escapes.
- Required the resolved target to be a regular file and under a byte cap
(new `HEADROOM_VERIFY_TRUNCATION_MAX_BYTES` knob, default 5MB), both
checked via `fstat` on the open fd before any read.
- Opened with `O_NONBLOCK` so a non-regular/special file can't park the
read indefinitely; disk I/O already runs off the event loop via the
existing compression thread pool.
- Added unit and integration tests covering successful verification,
missing/unreadable paths, relative paths, content races, feature gating,
banner precedence, and the adversarial cases below.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_tool_result_interceptors.py -v
51 passed in 2.21s
(includes TestVerifyReadAgainstDisk::* [14 cases, incl. absolute-path escape, `..`
traversal, symlink escape, sibling-prefix boundary, non-regular file, oversized
file, and untrusted-request-touches-no-disk], plus the end-to-end disk-verification
and banner-precedence cases)

$ pytest tests/test_proxy_loopback_gating.py -k trusted
16 passed
(includes test_record_headroom_stack_binds_trusted_true_for_loopback_client and
..._false_for_non_loopback_client — proves a spoofed x-headroom-cwd from a
non-loopback peer does not bind trust)

$ ruff check headroom/proxy/interceptors/astgrep.py headroom/proxy/project_context.py \
    headroom/proxy/server.py headroom/proxy/runtime_env.py \
    tests/test_tool_result_interceptors.py tests/test_proxy_loopback_gating.py
All checks passed!

$ mypy headroom/proxy/interceptors/astgrep.py headroom/proxy/project_context.py \
    headroom/proxy/server.py headroom/proxy/runtime_env.py
Success: no issues found in 4 source files
```

## Real Behavior Proof

- Environment: Mac OS local dev machine, no live proxy/network involved.
Exercised the real interceptor pipeline (`apply_to_messages`) directly
against an actual file on disk, using the same fixture and cwd-binding
path a live proxy request would use.
- Exact command / steps: wrote a 9-function Python fixture to a temp
file, built a client-truncated `tool_result` (first 8 functions, cut
mid-file, **no truncation banner text anywhere**), bound
`x-headroom-cwd` to the temp directory via `set_current_cwd()`, then
called `apply_to_messages()` once with
`HEADROOM_VERIFY_TRUNCATION_ON_DISK` unset and once with it set to `1`.
- Observed result: With disk verification disabled, the existing
ast-grep interceptor produced its normal header without indicating that
the input was truncated. With HEADROOM_VERIFY_TRUNCATION_ON_DISK=1, disk
verification detected that the 50-line tool_result was a strict prefix
of the 59-line file and produced:
 ```text
=== HEADROOM_VERIFY_TRUNCATION_ON_DISK unset (default) ===
[headroom: outlined by ast-grep — 8 definition(s); bodies elided.
Re-read the file with a line range to see a specific body.]

=== HEADROOM_VERIFY_TRUNCATION_ON_DISK=1, x-headroom-cwd bound to the
real directory ===
[headroom: outlined by ast-grep — 8 definition(s) in the visible
portion; input was truncated upstream (showing through line 50 of 59
total). Bodies elided. Re-read remaining lines to see more.]
```

**Trust-gate behavior:** same fixture and disk layout, `HEADROOM_VERIFY_TRUNCATION_ON_DISK=1`, but the request context's trust flag left at its default (`False`, simulating a non-loopback caller) instead of being marked trusted.

```text
=== trusted context (loopback) ===
[headroom: outlined by ast-grep — 8 definition(s) in the visible
portion; input was truncated upstream ...]

=== untrusted context (non-loopback) — same file, same cwd header, same
tool_result ===
[headroom: outlined by ast-grep — 8 definition(s); bodies elided.
Re-read the file with a line range to see a specific body.]
(no disk read attempted — verified via a call-recording wrapper around
os.open asserting zero invocations)
```
Also verified at the HTTP-middleware level with `TestClient(..., client=("100.90.0.5", 12345))`: a request carrying a spoofed `x-headroom-cwd` header from a non-loopback client IP does not bind trust, regardless of header content.

- Not tested: a live `headroom proxy` session with a real coding-agent client; the WebSocket/Codex responses-WS path.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. The feature is gated by `HEADROOM_VERIFY_TRUNCATION_ON_DISK`
- Minimum rollout channel: N/A. `--intercept-tool-results` must already be enabled for the ast-grep interceptor to run.
- Stable/default behavior changed: no
- Kill switch / disable path: Unset `HEADROOM_VERIFY_TRUNCATION_ON_DISK`; the setting is hot-reloadable and does not require a restart.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit, or unset the env var on a running proxy (takes effect immediately, no restart).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

## Additional Notes

this branch is based on `fix/astgrep-truncation-marker` #2430 not main


<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3192" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-08 10:52:09 -05:00
1d4b357ca7 feat(opencode): support the OpenCode 2.x plugin API (#3999)
## Description

Takeover of #3719 by @laurence-diack-pk, who has been unavailable since
Sep 23 and whose branch was left conflicted with `main`. The original
commit is preserved and @laurence-diack-pk remains its author; this
branch adds follow-up commits addressing @JerrettDavis's review and the
findings from live OpenCode 2.x testing.

It makes the OpenCode plugin and `headroom wrap opencode` work on both
OpenCode 1.x and 2.x. On 2.x the 1.x function plugin was silently
dropped (the loader requires a `{ id, setup }` default export loaded
from a directory), and Effect's `FetchHttpClient` keeps its own `fetch`
reference from before plugins load — so `wrap opencode` loaded no plugin
and routed nothing.

Closes #3669. Supersedes #3719.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)

## Changes Made

Carried from #3719:
- One plugin for both versions: default export `{ id: "headroom",
server, setup }`; 1.x calls `server`, 2.x calls `setup`.
- 2.x `setup`: registers `headroom_retrieve` via `ctx.tool.transform`
(`codemode: false`), sets `HEADROOM_*` shell env through the
`create.before` hook, and returns a cleanup that releases the refcounted
transport so hot reloads do not stack interceptors.
- 2.x model routing: `ctx.model.transform` points remote OpenAI-wire
models at the proxy with `x-headroom-base-url` /
`x-headroom-original-path` headers, since 2.x keeps its own `fetch`
reference and the transport patch alone misses model traffic.
- Wheel ships the bundle as `_dist/index.js`;
`headroom_opencode_plugin_path()` returns the directory; `wrap opencode`
adds `--standalone` on 2.x so the private server sees
`OPENCODE_CONFIG_CONTENT`.

Follow-up commits (review from #3719 + live testing):
- Rebased on current `main` (0.40.0). Fixed the routing-contract
conflict: replaced `routesThroughProxy` (TS2554 against `shouldRoute`'s
3-arg signature) with `modelBaseRoutesThroughProxy`, a deliberate
model-base eligibility check that keeps the shared protocol / loopback /
proxy-origin / excluded-host contract but drops the endpoint-suffix
requirement, since 2.x hands us model base URLs (e.g. `/v1`), not
inference endpoints.
- `setup` now resolves the effective `excludeHosts` once (option over
`HEADROOM_OPENCODE_EXCLUDE_HOSTS`) and applies it to both the transport
install and the model rewrite, with new tests for both paths.
- Fixed `test_build_launch_env_with_project` to decode
`OPENCODE_CONFIG_CONTENT` instead of substring-matching the raw path,
which JSON backslash escaping makes impossible on Windows.
- `opencode_major_version` now reads the `v2` channel marker in dev
builds (`opencode v0.0.0-v2-<stamp>`), which previously parsed as major
0, so `wrap` skips `--standalone` and the private server never loads
Headroom's config.
- Rebuilt `_dist/index.js` and `hook-shim/handler.js`; drift check
clean.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .` — run on the changed paths)
- [x] Type checking passes (`mypy headroom` — run on the changed paths)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# plugins/opencode
$ npm ci && npm run typecheck && npm run build:standalone && npm test
tsc: clean (no TS2554)
vitest: Test Files 3 passed (3) | Tests 45 passed (45)
$ cmp dist-standalone/index.js ../../headroom/providers/opencode/_dist/index.js          # identical
$ cmp dist-standalone/hook-shim/handler.js ../../headroom/providers/opencode/hook-shim/handler.js  # identical

# focused Python suites (same set as #3719, plus the 2.x dev-build version case)
$ pytest tests/test_providers_opencode_plugin_path.py tests/test_providers_opencode_config.py \
    tests/test_providers_opencode_install.py tests/test_cli/test_wrap_opencode.py \
    tests/test_mcp_registry_opencode.py tests/test_learn/test_opencode_scanner.py -q
193 passed in 1.10s

$ ruff check / ruff format --check on changed paths: clean
$ mypy headroom/providers/opencode headroom/cli/wrap.py: Success: no issues found
```

## Real Behavior Proof

- Environment: macOS arm64, Node v26.10.0, Python 3.13, headroom from
this branch (0.41.0-dev). Tested against two real OpenCode 2.x clients,
each in throwaway XDG dirs with its own proxy port:
  - `opencode 2.0.19` (official 2.x binary);
- OpenCode 2.x head built from `anomalyco/opencode@v2`
(`0.0.0-v2-202610061051`, Bun 1.4.2, `bun run script/build.ts --single`
in `packages/cli`).
- Exact command / steps: throwaway XDG dirs and a dedicated proxy port;
fake key first (routing proof), then the real key (end-to-end proof):
  ```bash
  # fake key first (routing proof), then the real key (end-to-end proof)
OPENCODE_API_KEY=<key> headroom wrap opencode --port 8799 --no-serena --
\
run -m opencode-go/deepseek-v4.1-flash "reply with the single word ok"
  # exclusion check
HEADROOM_OPENCODE_EXCLUDE_HOSTS=opencode.ai OPENCODE_API_KEY=<key>
headroom wrap opencode ...
  ```
(`opencode-go` declares `OPENCODE_API_KEY` as its env credential in the
catalog, so no interactive login is needed.)
- Observed result: routed 401 with the fake key and 200 with the real
key through the proxy, `--standalone` added automatically, exclusion
honored — details:
- **Fake key** — proxy log: `proxy_inbound_request POST
/v1/chat/completions` with `user-agent: opencode/latest/2.0.19/cli` (and
`opencode/v2/0.0.0-v2-202610061051/cli` for the head), headers
`x-headroom-base-url: https://opencode.ai` and
`x-headroom-original-path: /zen/go/v1/chat/completions`; forwarded
upstream, `401` from the fake key — routed through Headroom.
- **Real key** — the model answered `ok`; proxy `status=200` with
upstream `https://opencode.ai/inference/go/openai/v1/chat/completions`.
- `wrap` added `--standalone` automatically on 2.0.19, and again on the
v2 head after the dev-build version fix (explicit `--standalone` also
works, per the kill-switch path).
- **Exclusion** — with `HEADROOM_OPENCODE_EXCLUDE_HOSTS=opencode.ai` the
model still answered while the proxy saw **zero** new chat requests: the
2.x model transform honors the exclusion list.
- Not tested: Windows; OpenAI "responses"-wire against a live 2.x
server; Anthropic-wire custom providers on 2.x; loading the plugin by
npm name (`headroom-opencode`), which needs a published release;
OpenCode 1.x older than 1.2.20.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: on OpenCode 2.x, `wrap opencode`
launches with `--standalone` (a private server per session); on 1.x
behaviour is unchanged apart from the plugin path now being a directory.
- Kill switch / disable path: pass `--server <url>` or `--standalone`
explicitly; `HEADROOM_OPENCODE_PLUGIN_PATH` still overrides the plugin
location.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR; the wheel bundle and plugin path revert
with it.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

@laurence-diack-pk — thank you for the original implementation; the
commit is preserved verbatim at the base of this branch. @JerrettDavis —
this addresses the routing-contract conflict you described in #3719 (the
missing `excludeHosts` argument and the model-base vs endpoint
mismatch), the Windows assertion, and a dev-build version-detection gap
found while validating on macOS. Live 2.x validation is done on both
2.0.19 and today's `v2` head; details above.

---------

Co-authored-by: laurence <laurence.diack@peekrobotics.co.nz>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-07 20:52:53 -05:00
Mansurshakh JaparovandJD Davis 40955b0eef fix(install): preserve OpenCode edits during reactivation (#4022)
## Description

Reapply, restart, and automatic recovery temporarily remove the Headroom
provider before reactivating it. They must not restore the original
backup and erase user configuration added after installation. Final
removal must still restore the original snapshot.

Closes #4016.

## Type of Change

- [x] Bug fix
- [x] Documentation update

## Changes Made

- Separate transient provider deactivation from final backup
restoration, and propagate the intent through the existing install
registry/manifest lifecycle.
- During reactivation and activation rollback, remove only Headroom's
provider; retain unrelated providers, MCP settings, theme, and the
original backup.
- On true removal, restore and delete the original backup as before.
- Associate backups with the actual selected `opencode.json` or
`opencode.jsonc` filename, and pin edited writes to LF.
- Persist snapshot ownership independently of active mutations,
including activation persistence failures and stop/reload/final-remove.
- Migrate historical JSONC backup names (including custom config names)
before snapshotting/restoration in installer and wrapper callers; do not
claim a sibling JSON installation's backup.
- Accept JSONC comments/trailing commas without erasing user settings;
retain quoted URLs and delimiters.
- Restore the snapshot when a successful replacement drops OpenCode, but
retain it on replacement failure.
- Document preservation during reapply/restart/recovery and
original-snapshot behavior on final removal.

## Testing

- [x] Unit tests pass (`pytest`), including JSON/JSONC and all three
reactivation paths.
- [x] Changed-file lint, configured pre-commit mypy and manual
real-filesystem checks pass.

### Test Output

Real filesystem smoke output for both config suffixes:

```json
[{"final_removal_deletes_backup": true, "final_removal_restores_original": true, "headroom_reactivated": true, "original_backup_unchanged": true, "suffix": ".json", "theme": "edited-after-install", "user_mcp_preserved": true, "user_provider_preserved": true}, {"final_removal_deletes_backup": true, "final_removal_restores_original": true, "headroom_reactivated": true, "original_backup_unchanged": true, "suffix": ".jsonc", "theme": "edited-after-install", "user_mcp_preserved": true, "user_provider_preserved": true}]
```

The mandatory pre-push `make ci-precheck` passed: workspace Rust
fmt/clippy/tests, native extension build/import verification, the
complete configured Python gate, and Commitlint. The native gate
explicitly used the installed CPU `libonnxruntime.so.1.26.0` via
`ORT_DYLIB_PATH`, with cached-model assertions enabled.

```text
[build_rust_extension] headroom._core build + install + verify: OK
================== 177 passed, 4 skipped, 1 warning in 6.31s ===================
✅ ci-precheck PASSED — safe to push.
```

```sh
python -m pytest -q tests/test_providers_opencode_install.py tests/test_cli/test_install_cli.py tests/test_install/test_providers.py
```

Original lifecycle suite: **121 passed**. After the review corrections,
the combined provider/config/wrapper/install-CLI/recovery suite passed
**240 tests in 8.32s**:

```sh
python -m pytest -q tests/test_providers_opencode_install.py tests/test_providers_opencode_config.py tests/test_cli/test_wrap_opencode.py tests/test_cli/test_install_cli.py tests/test_install/test_providers.py tests/test_install_tool_env_reconcile.py
```

The regression suite checks real original bytes and persisted recovery
ownership, not mocked call order or incidental error wording. The
obsolete JSONC-trailing-comma rejection expectation was removed.

Changed-file Ruff, formatting, installed pre-commit `mypy headroom`, and
Commitlint hooks passed. Full-tree Ruff remains blocked locally by
executable permissions on five unchanged OAuth2 plugin files (tracked
100644). A separate full dev-environment mypy run found an unchanged
optional LangChain `truthy-function` error; the configured isolated
pre-commit hook passed. Unrelated code/permissions were not modified.

## Real Behavior Proof

- Environment: Linux x86_64, CPython 3.13.14; unchanged
`dea9092e19385f286b7f67a64f4d85de2708d756` and this branch, real
temporary config files and real installer mutation helpers, no mocked
file mutation or service installed on the workstation.
- Exact command / steps: for each suffix `.json` and `.jsonc`, write an
original theme/provider/MCP config; activate a provider-scoped OpenCode
manifest with `_activate_deployment_mutations`; edit the theme and add a
user provider plus remote MCP; call `_deactivate_deployment_mutations`
then `_activate_deployment_mutations`; inspect real files and the
original backup; finally invoke `revert_mutations(...,
restore_backup=True)`.
- Before: theme reset to `original`; added provider and MCP disappeared.
JSONC's actual-name backup was absent.
- Observed result: after the fix, theme remains `edited-after-install`;
both user additions survive; Headroom is active again; both actual-name
backups remain byte-identical to the original. Final removal restores
the original bytes and deletes the backup for both suffixes.
- Review correction proof against published head `4bc8e50c`: actual
temporary files and persisted manifests cover stop/reload/final-remove
for JSON and JSONC, legacy backup migration for default and custom
filenames, JSONC trailing commas and quoted `//`/`,}` text, successful
target removal, and activation-persistence failure followed by
reload/removal. All eight scenario outcomes pass after the correction;
the published baseline leaves or consumes the wrong snapshots and loses
JSONC edits. The actual `headroom unwrap opencode --no-stop-proxy`
command also restores the historical JSONC snapshot byte-for-byte and
consumes its backup. Only external process/service management and the
explicit persistence-fault boundary are substituted; config, manifest
and snapshot IO are real.
- Second review correction against `1c0855c4`: all four new findings
reproduced on real temporary config/backup files and persisted
manifests. After the fix, failed addition consumes its newly created
snapshot (a later install restores the later user edit); changed config
selection restores the recorded original path; an existing canonical
JSONC snapshot wins without deleting the ambiguous legacy backup; and
restoration failure retains ownership in the active replacement so a
retry after reload finishes restoration. Seven permanent boundary cases
initially gave **6 failures and 1 passing control**; the complete
affected suite now gives **247 passed**. OS supervisor calls and
explicit persistence/restoration faults are the only substituted
boundaries.
- Not tested: a real launchd/systemd supervisor or Windows process on
this Linux host. CLI regression tests exercise apply/restart/recovery
boundaries while replacing service management; the filesystem mutation
smoke uses real production helpers.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; lifecycle data-preservation fix.
- Minimum rollout channel: stable; no rollout-managed feature is
enabled.
- Stable/default behavior changed: transient reactivation stops
reverting unrelated user edits. Final uninstall still restores the
original snapshot.
- Kill switch / disable path: remove the persistent OpenCode
installation using its existing removal flow; no new flag.
- Unsafe override required: no.
- Qualification impact: provider-scoped OpenCode configuration only;
other providers retain their existing removal semantics.
- Rollback path: revert the commits; no deployment manifest schema
change. Backup ownership uses the existing artifact record, and legacy
JSONC snapshots migrate to the selected config's canonical filename.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] Existing registry/lifecycle interfaces reused; persistent-install
docs updated.
- [x] Consumer-visible regressions cover edits, original-backup
invariants, both config suffixes, three reactivation paths, and final
removal.
- [x] `CHANGELOG.md` not edited; Conventional Commit title supplied.

Remote CI results are reported by GitHub, not assumed from local tests.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-07 17:46:39 -05:00
Ayush Kumar JhaandJerrett Davis 0d7d61d5cc fix(wrap): let wrap opencode target a third-party OpenAI-compatible upstream (#3125)
## Description

`headroom wrap opencode` had no way to say where OpenAI-compatible
traffic should go. `opencode()` forwarded `openai_api_url` to
`_ensure_proxy` only for `--copilot-subscription`; every other run
inherited the proxy default, `https://api.openai.com/v1`. A user
following the README's agent-wrap table with a DeepSeek key therefore
had that key presented to OpenAI, which rejected it with `401 Incorrect
API key provided` and pointed them at a `platform.openai.com` dashboard
for a key they never had. Meanwhile
`docs/content/docs/opencode-deepseek.mdx` said "Do not use `headroom
wrap`", with nothing reconciling the two.

This adds the missing plumbing (`--openai-api-url` /
`OPENAI_TARGET_API_URL`, mirroring `headroom proxy`) and the
documentation the issue asked for.

Closes #3107

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cli/wrap.py`: `wrap opencode` gains `--openai-api-url`, with
`envvar="OPENAI_TARGET_API_URL"`, mirroring `headroom proxy
--openai-api-url`. It is passed to `_ensure_proxy`, which already knows
how to start the proxy with it and how to report a mismatch against an
already-running proxy.
- `headroom/cli/wrap.py`: combining `--openai-api-url` with
`--copilot-subscription` raises a `ClickException` before anything is
mutated — the subscription resolves its own upstream, so a manual
override would silently fight it.
- `headroom/cli/wrap.py`: the `opencode` docstring documents the default
upstream and both invocation forms.
- `headroom/cli/wrap.py`: `--no-proxy` combined with an upstream
override now fails closed (review). New
`_require_no_proxy_openai_upstream(port, url)` requires a healthy
Headroom listener whose `/health` config advertises the same
`openai_api_url` after `_normalize_proxy_api_url`; an absent listener, a
listener with no Headroom config, or a mismatched upstream each raise a
`ClickException` that names the exact `headroom proxy --port N
--openai-api-url URL` command to run. The check is opted into via a new
`require_openai_api_url` keyword on `_ensure_proxy_unlocked`, set only
by `wrap opencode`, so wrappers that pass a built-in upstream (`vibe`,
`kimi`, `grok`, `zcode`) keep their existing warn-and-reuse `--no-proxy`
behaviour. On a match wrap prints `Proxy on port N already targets URL`,
warns about a proxy-mode mismatch like the other reuse paths, and
continues. `wrap opencode` runs this check before it touches OpenCode's
config or registers a client marker, so a refused upstream leaves
nothing behind. `--prepare-only` skips it, since it never uses a proxy.
- `headroom/providers/opencode/config.py`:
`inject_opencode_provider_config(port, keep_user_entries=...)` keeps the
user's own `models` and `options` (an `apiKey`, say) under
`provider.headroom` instead of replacing the entry; Headroom still owns
`npm`, `name` and `baseURL`. `wrap opencode` passes `keep_user_entries`
only when `--openai-api-url` (or `OPENAI_TARGET_API_URL`) is set.
Without it the proxy forwards to OpenAI, so a kept third-party key would
be sent there, and wrap writes Headroom's entry as before. OpenCode only
resolves `headroom/<id>` for listed ids, so without this a DeepSeek user
had no DeepSeek model to select (`ProviderModelNotFoundError`) and no
key to send.
- `docs/content/docs/opencode.mdx`: `--openai-api-url` documented on the
canonical OpenCode page (Options block, prose, env-var table), with a
two-form example — wrap starts the proxy, or `headroom proxy
--openai-api-url …` then `wrap --no-proxy --openai-api-url …` — and a
paragraph stating that `--no-proxy` verifies and refuses rather than
re-points.
- `docs/content/docs/opencode-deepseek.mdx`: new "Common issues" entry
for the `Incorrect API key provided` 401 (it comes from OpenAI, not
DeepSeek) and a "When is `headroom wrap opencode` safe?" section — safe
as-is for the default OpenAI/Anthropic upstreams, needs
`--openai-api-url` for any third-party OpenAI-compatible provider, and
that wrap keeps the step 4 `models` and `apiKey` under
`provider.headroom` (DeepSeek needs both). The "Models appear but
requests fail" entry and the "What's NOT in this guide" bullet are
reworded to match, replacing the blanket "do not use `headroom wrap`".
- `README.md`: footnote on the OpenCode row of the agent compatibility
matrix naming the default upstream and the flag/env var.
- `tests/test_cli/test_wrap_opencode.py`: ten tests — the flag, the env
var, the unset default, the `--copilot-subscription` rejection; for
`--no-proxy`: absent, non-Headroom, matching (trailing slash, proving
normalization), and mismatched listeners; that wrappers without
`require_openai_api_url` keep lenient reuse; and that the CLI forwards
the flag. With the fix reverted, 5 of the 6 new tests fail. Review
follow-ups add: `wrap opencode --help` keeps its examples unwrapped and
`wrap.py` has no raw control bytes; a refused `--no-proxy` upstream
exits before any OpenCode config edit or client marker; the
matching-upstream path still warns about a mode mismatch.
- `tests/test_providers_opencode_config.py`: with `keep_user_entries`
the user's `headroom` models and `apiKey` survive re-injection, other
providers are untouched, and Headroom's `npm`/`baseURL` win; without it
they are dropped.
- `tests/test_cli/test_wrap_opencode.py` (second round): through the CLI
the key is kept with `--openai-api-url` and dropped without it, and
`--prepare-only --no-proxy --openai-api-url` writes config with no proxy
listening.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

Four failures in the wider `tests/test_cli` run are pre-existing and
Windows-only — they reproduce identically with this branch's changes
stashed: symlink/settings-path issues in `test_recover_codex.py` and
`test_unwrap_claude.py`, and `signal.SIGHUP` (absent on Windows) in
`test_wrap_stale_marker.py`. They are deselected below.
`tests/test_cli_proxy_improvements.py` and
`tests/test_memory/test_learn_flag.py` cannot run from this source
checkout because the Rust `headroom._core` extension is not built here
(same result on the clean branch); CI builds the wheel.

### Test Output

```text
$ python -m pytest tests/test_cli/test_wrap_opencode.py -q
50 passed, 1 warning in 1.96s

$ python -m pytest tests/test_cli -q     --deselect tests/test_cli/test_recover_codex.py     --deselect tests/test_cli/test_unwrap_claude.py     --deselect "tests/test_cli/test_wrap_stale_marker.py::test_signal_handler_unwinds_so_the_restore_can_run"
706 passed, 1 skipped, 46 deselected, 1 warning in 60.41s

$ git stash push -- headroom/cli/wrap.py && python -m pytest tests/test_cli/test_wrap_opencode.py -q -k no_proxy
5 failed, 2 passed   # the new tests fail without the fix; the 2 passing pin unchanged behaviour

$ python -m ruff check .
All checks passed!

$ python -m ruff format --check .
1577 files already formatted

$ python -m mypy headroom
Success: no issues found in 532 source files

$ python -m coverage run --include=headroom/cli/wrap.py -m pytest tests/test_cli/test_wrap_opencode.py -q
50 passed   # 17/17 statements added to wrap.py executed, 0 missing
# Review follow-ups (this branch, with current main merged in)
$ pytest tests/test_cli/test_wrap_opencode.py tests/test_providers_opencode_config.py
102 passed, 1 failed  # test_build_launch_env_with_project: a known Windows-only path-escaping failure, also on main
# each new test fails on the code before its fix:
test_inject_provider_config_keeps_the_users_models_and_options -> KeyError: 'apiKey'
test_no_proxy_with_openai_api_url_still_warns_about_a_mode_mismatch -> warning missing from output
test_wrap_opencode_no_proxy_rejects_a_mismatched_upstream_before_editing_config -> "MCP retrieve tool: registered", then _ensure_proxy reached
test_wrap_opencode_keeps_the_users_headroom_key_only_for_an_explicit_upstream[no flag] -> key kept (would go to OpenAI)
test_wrap_opencode_prepare_only_does_not_need_the_no_proxy_listener -> "No Headroom proxy is listening on port 9000"
# help test: dropping either no-rewrap marker, or writing a raw 0x08 byte, fails a test (3/3)
$ ruff check / ruff format --check (0.16.8): clean
$ mypy headroom/cli/wrap.py headroom/providers/opencode/config.py --ignore-missing-imports --follow-imports=silent
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.11.9, headroom 0.35.0 (`rust_core:
loaded`), with a stub `opencode` on `PATH` that curls the proxy's
`/health` — so the wrapped agent reports the upstream the proxy actually
resolved.
- Exact command / steps: (1) baseline against released 0.35.0 without
this fix on `sys.path` — `python -m headroom.cli wrap opencode --port
8799 --no-mcp --no-serena --openai-api-url https://api.deepseek.com/v1`;
(2) same command on this branch, port 8801; (3) env-var form on this
branch — `OPENAI_TARGET_API_URL=https://api.deepseek.com/v1 python -m
headroom.cli wrap opencode --port 8802 --no-mcp --no-serena`. For the
`--no-proxy` review: `python -m headroom.cli wrap opencode --port <P>
--no-proxy --no-mcp --no-serena --openai-api-url
https://api.deepseek.com/v1` on this branch against (4) nothing on 8811;
(5) `python -m http.server 8812`; (6) a real released-0.35.0 `headroom
proxy --port 8813` (default upstream, `/health` reports
`"openai_api_url": null`); (7) a real `headroom proxy --port 8814
--openai-api-url https://api.deepseek.com/v1`.
- Observed result: step 1 reproduces the bug exactly — wrap prints
`Extra args: --openai-api-url https://api.deepseek.com/v1` (the flag is
swallowed by `ignore_unknown_options` and handed to the agent) and the
proxy it started reports `"openai_api_url": null`, i.e. it would forward
to `api.openai.com`. Step 2 prints no `Extra args` line and the proxy
reports `"openai_api_url": "https://api.deepseek.com/v1"`. Step 3
reports the same. All three runs cleaned up their proxy on exit — no
listener left on 8799/8801/8802. Steps 4–6 each exit 1 before `opencode`
is launched, with: (4) `No Headroom proxy is listening on port 8811 …
Start it separately with `headroom proxy --port 8811 --openai-api-url
https://api.deepseek.com/v1`, or drop --no-proxy …`; (5) `The listener
on port 8812 did not report a Headroom config, so wrap cannot confirm it
forwards to https://api.deepseek.com/v1 …`; (6) `The Headroom proxy on
port 8813 forwards OpenAI-compatible traffic to
https://api.openai.com/v1, not https://api.deepseek.com/v1. Restart it
with …` — and the 8813 proxy is left running, still reporting `null`.
Step 7 prints `Proxy on port 8814 already targets
https://api.deepseek.com/v1` and launches the stub (exit 0). Both
proxies were stopped afterwards; 8811–8814 verified free.
- Real OpenCode (review follow-up): OpenCode 1.18.34 installed from npm
into a scratch directory with an isolated `HOME`, and a local fake
OpenAI-compatible "DeepSeek" upstream on 8792 that records each
request's model and `Authorization` header. The global `opencode.json`
declares `provider.headroom.models.deepseek-chat` and `options.apiKey:
"{env:DEEPSEEK_API_KEY}"`. Command: `headroom wrap opencode --port 8796
--openai-api-url http://127.0.0.1:8792/v1 --no-mcp --no-serena -- run
"say hi" -m headroom/deepseek-chat`.
- before this commit: `ProviderModelNotFoundError: Model not found:
headroom/deepseek-chat.` Nothing reached the upstream, and the config
after wrap held only `gpt-4o` and `gpt-4.1` under `headroom`, with no
`apiKey`.
- after (re-run on the final head e8754b61): OpenCode printed `> build ·
deepseek-chat` and `hello from fake deepseek`. The upstream logged
`{"path": "/v1/chat/completions", "model": "deepseek-chat", "auth":
"Bearer sk-fa...", "stream": true}`. The config kept `apiKey` and
`deepseek-chat` alongside Headroom's `baseURL` and defaults.
- Not tested: a live request against `api.deepseek.com` with a real key,
since none is available here. The real-OpenCode run above uses a local
fake upstream instead, so it proves model selection, key forwarding and
routing through Headroom but not DeepSeek's own API. The
`--copilot-subscription` rejection was exercised as a unit test, not
against a real Copilot subscription.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — a CLI option and its plumbing, not
gated by the rollout-channel machinery.
- Minimum rollout channel: n/a, not rollout-gated.
- Stable/default behavior changed: no — without the flag and without
`OPENAI_TARGET_API_URL`, `openai_api_url` stays `None` and the proxy
resolves its upstream exactly as before; a test pins this.
- Kill switch / disable path: omit the flag and leave
`OPENAI_TARGET_API_URL` unset — behavior is identical to the previous
default.
- Unsafe override required: no.
- Qualification impact: none — no change to compression, routing, or the
proxy request path; the option only chooses which upstream URL the proxy
is started with.
- Rollback path: revert this commit — no config, cache, or on-disk state
is written by the new option, so a revert needs no cleanup.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

n/a — CLI and docs change.

## Additional Notes

- The `x-headroom-base-url` header workaround from the issue is
deliberately left out of the DeepSeek guide. With `--openai-api-url` the
proxy targets the right upstream directly, so the header is no longer
needed for this case. It stays documented where it belongs, in
`configuration.mdx` and `pipeline-extensions.mdx`, as a per-request
override.
- Scope is intentionally limited to `wrap opencode`, the command named
in the issue. Other `wrap` subcommands that hardcode an upstream
(`mistral`, `kimi`, `grok`) or detect one (`zcode`) are unaffected;
giving the whole wrap family a uniform upstream override is a reasonable
follow-up, not this fix.

---------

Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
2026-10-06 08:45:54 -05:00
rNoz 4227bd2419 fix(proxy): reap wrap-spawned proxies once no wrap clients remain (#3202)
## Description

Nothing in Headroom ever stops a wrap-spawned proxy whose wrappers are
gone. Two leak paths, both by-products of deliberate hardening without a
counterpart:

1. **Unclean wrapper death.** `_start_proxy` detaches the proxy
(`start_new_session` on POSIX, breakaway console flags on Windows) on
purpose, so a crashed wrapper cannot tree-kill a shared proxy out from
under other clients. But when the wrapper dies without running cleanup
(SIGKILL, crash, terminal close), its client marker goes stale,
`_live_proxy_clients` unlinks it, and the proxy itself is never stopped.
2. **Graceful last-exit leak.** `_make_cleanup` only terminates the
proxy when the exiting wrapper *started* it (`proxy_holder[0] is not
None`). When the starter exits while other clients remain (correct),
every later wrapper exits with no handle, so even a perfectly behaved
multi-wrap topology leaks the proxy at the end.

Observed in production: three `headroom proxy` processes still alive 21
days after their last session, holding ports and being silently reused
by later wraps (which then inherited their stale version and captive
routing; see the two companion PRs).

The fix: wrap-spawned proxies carry `HEADROOM_WRAP_OWNED=1` and run an
orphan-watchdog task that polls the per-port client-marker dir every 30s
and gracefully stops the process once no live markers, no active
WebSocket sessions, and no served-request activity remain for a grace
period (default 900s, floor 60s, `HEADROOM_WRAP_ORPHAN_GRACE_SECONDS`).
Standalone `headroom proxy` instances and persistent installs never get
the marker env var and are never affected; multi-worker runs are
excluded (WS sessions are per-worker, and wrap never requests multiple
workers).

Closes #3199

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/proxy/orphan_watchdog.py` (new): env config
(`WRAP_OWNED_ENV`, grace parsing with floor), marker liveness
(`pid_alive` + start-time identity, pruning stale markers), and
`orphan_watchdog_loop` (idle clock reset by live markers, active WS
sessions, or movement of the proxy's served-request counter — the last
covers direct-HTTP and SSH-forwarded clients that write no marker). Stop
is `signal.raise_signal(SIGTERM)`, which reaches uvicorn's handler on
Unix and the emulated handler table on Windows (unlike `os.kill(pid,
SIGTERM)`, which is `TerminateProcess` there).
- `headroom/proxy/server.py`: lifespan spawns the watchdog when
`orphan_watchdog_enabled()` and not a multi-worker run
(`HEADROOM_PROXY_CONFIG_JSON` absent), and cancels it in teardown like
the other periodic tasks.
- `headroom/cli/wrap.py`: `_start_proxy` sets `HEADROOM_WRAP_OWNED=1` in
the proxy subprocess env; `_proc_identity`/`_identity_mismatch` moved to
`headroom/_subprocess.py` (`proc_identity`/`identity_mismatch`) so proxy
code does not import the CLI layer — thin module-level delegates keep
the existing monkeypatch-based tests working unchanged.
- `tests/test_proxy/test_orphan_watchdog.py` (new, 13 tests): env
parsing (default/override/floor/garbage), marker pruning (live, dead
PID, recycled PID, garbage JSON, missing dir), loop semantics (exit
after grace, hold on live client, hold on active WS session, hold on
served-request movement, grace reset on returning client, missing
registry counts as idle).
- `tests/test_cli/test_wrap_claude_vertex_proxy_env.py`: `_start_proxy`
marks the subprocess env wrap-owned.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_proxy/test_orphan_watchdog.py -q
13 passed in 2.77s
$ uv run pytest tests/test_proxy tests/test_cli -q
973 passed, 1 skipped in 49.85s   # 1 pre-existing unrelated failure
                                  # (test_litellm_vertex_backend_path_...) fails
                                  # identically on clean upstream/main
$ uv run ruff format / ruff check on touched files
All checks passed!
$ uv run mypy headroom/_subprocess.py headroom/cli/wrap.py headroom/proxy/orphan_watchdog.py
Success: no issues found in 3 source files
```

## Real Behavior Proof

- Environment: macOS 26.5 arm64, Python 3.13, local source build
0.37.0-dev (integration overlay with this PR applied).
- Exact command / steps: start wrap-spawned proxies via
`wrap._ensure_proxy(...)` (which goes through `_start_proxy` and thus
`HEADROOM_WRAP_OWNED=1`), with zero registered clients; wait; inspect
`~/.headroom/logs/proxy.log`.
- Observed result: two such proxies (plain on 8789 PID 48365,
augment-mode on 8791 PID 49729) self-exited at the 900s default grace:
`event=proxy_shutdown reason=signal pid=49729` at 09:51:49 against a
~09:36 start, port released afterwards (`lsof` clean). A machine-wide
sweep beforehand showed three 21-day-old orphans (PIDs 73974, 63454,
79339); with this build they can no longer accumulate.
- Not tested: Windows signal path (reasoned from CPython/uvicorn
semantics; unit-injected `stop` callable covers the loop side); the
multi-worker exclusion with a real `--workers 2` run (guarded by env
presence only); marker pruning under active PID recycling on Linux
without psutil (`/proc` fallback is unit-tested via mocks).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: only for wrap-spawned proxies, which
now self-exit after 15 clientless minutes. Standalone and persistent
proxies are untouched.
- Kill switch / disable path: omit `HEADROOM_WRAP_OWNED` (not settable
by users of wrap; standalone proxies never have it), or set
`HEADROOM_WRAP_ORPHAN_GRACE_SECONDS` very high.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit; orphaned proxies leak again (status
quo ante).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Adversarially reviewed by a second model (glm-5.2, high reasoning);
initial verdict needs-work, all accepted findings applied (multi-worker
exclusion, Windows-safe stop signal, served-request activity hold for
marker-less clients, wider test timing margins) and re-tested green. The
reviewer independently verified the boot-race ordering, teardown
cancellation, and the kill-vs-self-exit race as already correct.
Companion PRs (same incident): `-dev` version-restart fix,
routing-config reuse check.
2026-09-28 13:31:49 -05:00
b7f342c153 fix(wrap): verify proxy deps before mutating Codex config (#1628)
## Description

\`headroom wrap codex\` now verifies that optional proxy dependencies
(\`headroom-ai[proxy]\`) are installed before mutating Codex
\`config.toml\`. If the check fails, the command exits with the same
error message as \`headroom proxy\` and leaves Codex config untouched.

Fixes #1614 (Bug 1: config mutated before proxy dependency check).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Extract \`ensure_proxy_dependencies()\` in \`headroom/cli/proxy.py\`
(shared with \`headroom proxy\`)
- Call it at the start of \`wrap codex\` when \`not no_proxy\`, before
config snapshot/injection
- Add regression tests for prepare-only abort, \`--no-proxy\` skip, and
import failure messaging

## Testing

- [x] Unit tests pass (\`pytest\`)
- [x] Linting passes (\`ruff check .\`)
- [ ] Type checking passes (\`mypy headroom\`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

\`\`\`bash
pytest
tests/test_cli/test_wrap_codex.py::test_wrap_codex_aborts_before_mutating_config_when_proxy_deps_missing
\

tests/test_cli/test_wrap_codex.py::test_wrap_codex_skips_proxy_dependency_check_with_no_proxy
\

tests/test_cli/test_wrap_codex.py::test_ensure_proxy_dependencies_exits_when_server_import_fails
-q
# 3 passed
ruff check headroom/cli/wrap.py headroom/cli/proxy.py
tests/test_cli/test_wrap_codex.py
ruff format --check headroom/cli/wrap.py headroom/cli/proxy.py
tests/test_cli/test_wrap_codex.py
\`\`\`

## Real Behavior Proof

Environment: Linux (Ubuntu), Python 3.12, local checkout with
\`PYTHONPATH\` pointed at patched sources.

Exact command / steps:
1. Created a temp \`~/.codex/config.toml\` with \`model_provider =
"openai"\`.
2. Patched \`headroom.cli.wrap.ensure_proxy_dependencies\` to raise
\`SystemExit(1)\` (simulating missing \`[proxy]\` extra).
3. Ran \`headroom wrap codex --prepare-only --no-serena --port 8787\`.

Observed result: exit code 1; \`config.toml\` unchanged; no
\`config.toml.headroom-backup\` created; no \`[mcp_servers.headroom]\`
block written.

Also verified: \`headroom wrap codex --prepare-only --no-proxy ...\`
does not invoke the dependency check.

Not tested: Windows-specific proxy selector behavior (covered separately
in #1655).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did not edit CHANGELOG.md; release notes are generated
automatically

---------

Co-authored-by: syf2211 <syf2211@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
2026-08-13 11:52:22 -05:00
Abhay Singh ae384862a4 fix(wrap/opencode): verify the opencode binary before mutating config
Verify the OpenCode executable before changing configuration.
2026-08-11 18:15:37 -07:00
Tejas Chopra e0ce4b1d48 fix: remove rtk and lean-ctx CLI context tools (#2677)
## Description

Removes both third-party CLI context tools — **rtk** and **lean-ctx** —
and with them the context-tool selector itself. Headroom no longer
downloads, installs or configures either one, and there is no
replacement.

The previous pass (#2344) gated only three entry points inside
`headroom/cli/wrap.py`. That left the feature reachable in practice:

| Gap | Effect |
|---|---|
| `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global
--auto-patch` from bash/PowerShell, **bypassing the Python gate
entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook
regardless of `HEADROOM_RTK` |
| `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was
broken by default**: `rtk_required=True` met a gate returning `None` →
`SystemExit(1)`. Invisible because all 8 openhands tests patched
`_ensure_rtk_binary` to a fake path |
| `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to
`rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker
polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) |
| No cleanup path | Nothing removed artifacts an earlier default had
installed, so a machine that once ran the old default kept rtk in the
loop forever (#1669, #1955) |

Also worth noting: the rtk binary download had **no SHA or signature
verification** — only `rtk --version` as a smoke test.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages,
`headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` /
`_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` /
`--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap
subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the
dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine
getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers,
`benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path
filters.

**Fails loudly, not silently** — `--context-tool` / `--no-context-tool`
/ `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in
shell profiles, aliases and CI jobs, and accepting them as a no-op would
read as Headroom having quietly stopped working. The installers reject
them too, which matters more than it looks: their arg parsers forward
the first unknown flag **and everything after it** to the wrapped tool,
so a leftover `--no-rtk` would have silently swallowed a following
`--port` and then been ignored downstream.

**New `headroom/context_tool_cleanup.py`** — deleting the code cannot
help a machine that already ran the old default, since the hooks,
binaries and injected guidance are durable on disk.
`purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and
removes the registered hook entries, the generated hook scripts, the
Headroom-managed `~/.local/bin` symlinks, the vendored
`~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server
entry and the marker-fenced instruction blocks. Deliberately
conservative: idempotent, **skips** a malformed config rather than
overwriting it, and only unlinks a symlink resolving inside Headroom's
own bin dir so a user's own build is untouched. It reports on
**stderr**, because `wrap/unwrap openclaw --prepare-only` emit
machine-readable JSON on stdout as their entire contract. Skipped for
`wrap selfheal` (runs from a SessionStart hook; must not race Claude
Code's writer for `~/.claude.json`) and for `--help`, which must stay
read-only.

**Client-config hardening** (discovered while investigating a "corrupted
Serena settings file" report) — `wrap.py` reset a settings file to `{}`
when an existing file would not parse, then wrote that back. One
hand-edited typo or a transient `EACCES`/`EINTR` on a valid file
destroyed the user's `permissions`, `env` and `hooks`, on **every
`headroom wrap claude`**. It now refuses to write. Separately,
`fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`),
fixing all 14 non-atomic client-config writes at once; it follows
symlinks rather than replacing them (dotfile managers) and preserves an
existing file's mode.

**Deliberately kept** — `rtk` stays in the wrapper-peel list in
`transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as
shell-command grammar, so `rtk cat f` is still classified as a file read
for anyone running their own rtk install, which the purge intentionally
leaves alone.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
All checks passed!

$ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
1255 files already formatted

$ mypy headroom/
Success: no issues found in 508 source files

$ pytest tests/test_context_tool_cleanup.py -q
11 passed

$ pytest tests/test_fsutil.py -q
12 passed

$ pytest tests/test_cli/test_wrap_codex.py -q            # 89 tests
89 passed in 431.68s
$ pytest tests/test_cli/test_wrap_opencode.py -q
39 passed in 257.46s
$ pytest tests/test_cli/test_wrap_helpers.py -q
45 passed
$ pytest tests/test_paths.py -q
75 passed
$ pytest tests/test_cli/test_unwrap_claude.py -q
14 passed
$ pytest tests/test_proxy_savings_history.py -q
39 passed
$ pytest tests/test_cli/test_wrap_copilot.py -q
27 passed
$ pytest tests/test_cli/test_wrap_zcode.py -q
20 passed
$ pytest tests/test_subscription_tracker.py -q
9 passed
$ pytest tests/test_proxy_dashboard_stats_cache.py -q
5 passed, 1 skipped
```

Repo-wide grep for 14 removed symbols (`headroom.rtk`,
`headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`,
`_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`,
`wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`,
`tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`,
`*.html`: **zero hits**.

Notable test changes: `test_wrap_openhands.py` no longer patches
`_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0
unpatched — the regression that was previously masked.
`test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed
(every test drove RTK instruction injection). A new
`test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed`
proves a pre-removal `subscription_state.json` still loads.

## Real Behavior Proof

- **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @
this branch, real `~/.headroom` and `~/.claude` on the dev machine.
- **Exact command / steps and observed result:**

```text
# 1. Retired flag fails loudly instead of silently no-op'ing
$ headroom wrap codex --prepare-only --context-tool rtk
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they
rewrote shell commands through a third-party binary Headroom no longer manages.
Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL;
`headroom wrap` uninstalls what they left behind on first run.

$ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ...

# 2. install.sh rejects the retired flags (extracted parse_wrap_args harness)
['--no-rtk', '--port', '9999']   rc=1  ERROR: CLI context tools ... Drop --no-rtk
['--context-tool=rtk']           rc=1  ERROR: CLI context tools ... Drop --context-tool
$ bash -n scripts/install.sh   # syntax OK

# 3. Purge ran against the real machine, which had all the orphaned artifacts
$ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..."
  removed ~/.headroom/bin/lean-ctx        (51 MB)
  removed ~/.headroom/bin/rtk             (7.7 MB)
  removed ~/.local/bin/rtk                (symlink into ~/.headroom/bin)
  removed ~/.claude/hooks/rtk-rewrite.sh
  removed 8 lean-ctx-* hook scripts
# ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged
# → ~59 MB reclaimed, no unrelated key touched

# 4. stdout stays machine-readable while the purge reports (planted a fake artifact)
$ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err
$ cat out
{"enabled":true,"config":{"proxyPort":8787,...}}     # parses as JSON
$ cat err
Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk

# 5. --help is inert (planted artifact survives), a real run purges
$ headroom wrap codex --help   → artifact survived: CORRECT
$ headroom wrap openclaw --prepare-only → purged: CORRECT

# 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json
top-level keys 90 -> 90;  projects 19 -> 19;  LOST keys: none
all content outside mcpServers byte-identical: True
```

Dashboard rendered via the Playwright test after the panel removal:
"Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`,
and "Token Usage" reads Before Compression → Proxy Removed → After
Compression with no "Filtered (this session)" row. Nothing below the
removed panel broke.

- **Not tested:** Windows and Linux (macOS only) — `install.ps1` is
verified by brace-balance and inspection, not executed, since no `pwsh`
is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated
but not run; it needs the Docker e2e image. `serena project index`
interaction is exercised in the stacked base PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

**Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge
that first; this PR's base should then be retargeted to `main`, or it
will read as containing that fix too.

**Breaking-change migration for users:**
- Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`,
`--context-tool`, `--no-context-tool` from any alias, script or CI job,
and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error
rather than being ignored, so the failure is immediate and
self-explaining.
- Previously-installed artifacts are purged automatically on the next
`wrap`/`unwrap`; no manual cleanup needed.
- `headroom perf --json` no longer carries a `cli_filtering` key, and
`/stats` no longer returns a `context_tool` section.

**Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from
`README.md`,
`docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`,
`docs/observability.md` and the matching `wiki/` pages.
`REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED
rather than deleted, to keep the planning record.

**Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as
dead code — its only caller was the `except KeyboardInterrupt` guarding
the binary download, so with no download there is nothing slow left to
interrupt.
2026-07-30 22:59:41 -07:00
Rod Boev 2eca5ee114 fix(copilot): normalize subscription API routing (#2441) (#2455)
## Description

PR https://github.com/headroomlabs-ai/headroom/pull/2445 added the
missing OpenCode subscription path, but the shared Copilot subscription
resolver still lets Business and Enterprise payload hosts route through
segmented `*.githubcopilot.com` domains and still drops an explicit
`GITHUB_COPILOT_API_URL` pin on two resolution paths. This follow-up
moves the final hosted-route decision back into the shared resolver,
normalizes `api.business.githubcopilot.com` and
`api.enterprise.githubcopilot.com` to the generic host by default, and
makes the explicit pin win on token exchange, explicit API token, and
Copilot-token candidate resolution. Both `headroom wrap copilot
--subscription` and `headroom wrap opencode --copilot-subscription`
inherit the same fix because they already consume the same
`CopilotSubscriptionTokenResolution.api_url`. Refs #2441.

Attribution:
https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026212395
reported and narrowed the Business or Enterprise regression, and
https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026863498
scoped the shared-resolver follow-up that this change implements.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Centralize subscription hosted-route selection so explicit
`GITHUB_COPILOT_API_URL` pins win on token exchange, explicit API token,
and Copilot-token candidate resolution.
- Normalize `api.business.githubcopilot.com` and
`api.enterprise.githubcopilot.com` to `https://api.githubcopilot.com` by
default, extending the existing individual-seat normalization.
- Extend focused auth and wrapper tests so both subscription wrappers
prove the corrected shared resolver output and the private-proxy
isolation contract stays intact.

## Testing

- [x] Unit tests pass (`uv run pytest tests/test_copilot_auth.py -q`)
- [x] Unit tests pass (`uv run pytest
tests/test_cli/test_wrap_copilot.py -q`)
- [x] Unit tests pass (`uv run pytest
tests/test_cli/test_wrap_opencode.py -q`)
- [x] Unit tests pass (`uv run pytest
tests/test_cli/test_wrap_persistent.py -q`)
- [x] Linting passes (`uv run ruff check .`)
- [x] Formatting check passes (`uv run ruff format . --check`)
- [x] New tests added for new functionality when applicable
- [ ] Manual testing performed

### Test Output

```text
uv run pytest tests/test_copilot_auth.py -q -> 84 passed
uv run pytest tests/test_cli/test_wrap_copilot.py -q -> 31 passed
uv run pytest tests/test_cli/test_wrap_opencode.py -q -> 44 passed in 143.46s
uv run pytest tests/test_cli/test_wrap_persistent.py -q -> 31 passed
uv run ruff check . -> All checks passed!
uv run ruff format . --check -> 1331 files already formatted
```

## Real Behavior Proof

- Environment: Windows
- Exact command / steps: Run the focused auth, Copilot wrapper, OpenCode
wrapper, and persistent-proxy pytest files after implementing the shared
resolver change, then ask lucasp1337 to rerun the Business or Enterprise
`--copilot-subscription` scenario from PR
https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026212395
on a real seat.
- Observed result: Focused auth and wrapper pytest runs passed locally,
including the enterprise-host exchange reproduction row, explicit-pin
precedence on all three producer paths, both subscription wrapper
routes, and the private-proxy isolation regression. Live Business or
Enterprise success stays behind reporter retest.
- Not tested: live Business or Enterprise tenant run

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

No `CHANGELOG.md` edit is needed because Headroom generates release
notes from conventional commits.

Risk for maintainers: PR
https://github.com/headroomlabs-ai/headroom/pull/641 manually validated
a Business seat against the GitHub-returned hosted domain in June on
`gpt-5.4`, so generic-by-default could affect tenants that genuinely
require a dedicated host. This follow-up keeps the documented escape
hatch intact by making `GITHUB_COPILOT_API_URL` win on every path.

Live-seat proof boundary: lucasp1337 offered to retest on a Business or
Enterprise seat in PR
https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026212395.
Keep any live success claim behind that rerun.
2026-07-20 17:15:57 -07:00
Rod Boev 9089e7f7d3 feat(opencode): support Copilot subscription backend for headroom models (#2441) (#2445)
## Description

`headroom wrap opencode` currently routes `headroom/*` models only to
ordinary Anthropic or OpenAI backends. A user with a GitHub Copilot
subscription cannot point those `headroom/*` requests at the Copilot
seat while keeping Headroom compression and stats, even though Headroom
already has the validated subscription resolver, the proxy seed path,
and the OpenCode provider route needed to do it.

This PR adds `--copilot-subscription` to the OpenCode wrap command. It
reuses the existing Copilot subscription token resolver, passes the
validated endpoint and token seed into the existing proxy startup path,
rejects unsupported runtime modes, and treats any non-empty Copilot API
token as a private session seed so token-only sessions do not reuse a
shared proxy. The generated OpenCode provider still targets the local
proxy, and subscription secrets stay out of OpenCode config,
environment, and terminal output.

Closes #2441

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `headroom wrap opencode --copilot-subscription` in
`headroom/cli/wrap.py`.
- Reuse the existing validated Copilot subscription resolver through one
small required-resolution helper shared with the dedicated Copilot
wrapper.
- Pass the resolved endpoint and token seed into `_ensure_proxy()` as
`openai_api_url`, `copilot_api_token`, `copilot_refresh_oauth_token`,
and `copilot_api_token_expires_at`.
- Reject `--copilot-subscription` with `--no-proxy`, `--prepare-only`,
and translated backends before proxy or OpenCode launch.
- Validate subscription mode before snapshotting OpenCode config, so
rejected invocations don't create stale backups.
- Scrub inherited Copilot proxy seed variables from the OpenCode child
environment.
- Treat any non-empty Copilot API token as a private session seed so
token-only sessions do not reuse shared or persistent proxies.
- Add focused OpenCode and persistent-proxy coverage for seed handoff,
guard failures, direct-token isolation, secret non-disclosure, and
unchanged non-subscription behavior.
- Leave `CHANGELOG.md` untouched because Headroom generates changelog
entries from conventional commits.

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_cli/test_wrap_opencode.py
tests/test_cli/test_wrap_persistent.py
tests/test_cli/test_wrap_copilot.py -q`, `106 passed in 136.65s`)
- [x] Linting passes (`uv run ruff check headroom/cli/wrap.py
tests/test_cli/test_wrap_opencode.py
tests/test_cli/test_wrap_persistent.py`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality when applicable
- [ ] Manual testing performed

### Test Output

```text
Targeted subscription tests pass: OpenCode `7 passed, 37 deselected in 0.36s`, persistent proxy `2 passed, 29 deselected in 0.34s`, and dedicated Copilot `11 passed, 20 deselected in 0.39s`. Coverage includes inherited resolver-input env scrubbing, HEADROOM_BACKEND rejection, no-backup-on-rejection, OpenCode-only scrub scoping, and private-proxy teardown on config-injection failure.
The full focused command `uv run pytest tests/test_cli/test_wrap_opencode.py tests/test_cli/test_wrap_persistent.py tests/test_cli/test_wrap_copilot.py -q` passed with `106 passed in 136.65s`. Ruff check and format check pass.
```

## Real Behavior Proof

- Environment: Windows, `uv` development environment, local CLI tests,
no live Copilot seat on this host
- Exact command / steps: run the focused OpenCode and persistent-proxy
tests with a mocked `CopilotSubscriptionTokenResolution`, then capture
the proof rows for seed handoff, direct-token isolation, guards, and
secret non-disclosure
- Observed result: Targeted subscription and proxy-seed tests pass,
including OpenCode-only resolver-input env scrubbing and private-proxy
teardown on config-injection failure; the full focused command passed
with `106 passed in 136.65s`; Ruff check and format check pass.
- Not tested: live Copilot subscription seat run on this host

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- Feature approval comes from the open `enhancement` label on
https://github.com/headroomlabs-ai/headroom/issues/2441.
- Keep the final live-backend claim behind manual owner proof. Local CLI
tests can prove config, guard, secret, and proxy-seed behavior, but they
cannot prove a real Copilot seat on this host.
- `CHANGELOG.md` remains untouched because Headroom's release pipeline
generates changelog entries from conventional commits.
2026-07-20 11:02:14 -07:00
Tejas ChopraandJerrettDavis 44136ed042 fix(wrap): make RTK opt-in (off by default) across wrap subcommands (#2344)
## Description

RTK CLI-command filtering was set up **by default** across ~16 `wrap`
subcommands (copilot, codex, aider, cursor, cline, continue, goose,
openhands, opencode, grok, omp, openclaude, vibe, …) via `if not
no_rtk:` — so users got rtk hooks / instruction injection without opting
in. `wrap claude` was the lone exception (already gated on
`--context-tool`).

This makes RTK **opt-in (off by default)** everywhere, so Headroom's own
savings are what's measured unless a user explicitly wants rtk.

Closes #

## Type of Change
- [x] Bug fix (behavior change: default flip)

## Changes Made
- **Central gate** `_rtk_opt_in()` — RTK runs only when explicitly
enabled via `--rtk` or `HEADROOM_RTK=1`. Guards the 3 RTK entry points
(`_setup_rtk`, `_ensure_rtk_binary`, `_inject_rtk_instructions`) so they
no-op by default — one small change instead of editing ~30 call sites.
- **`--rtk` opt-in flag** on all 18 tool subcommands via a shared
eager-callback option (`expose_value=False`, sets `HEADROOM_RTK=1`; no
subcommand signature changes).
- `wrap claude`'s legacy `--context-tool` still opts in (mirrored into
the gate).
- **`--no-rtk` kept** as an accepted, now-redundant no-op (back-compat).
- lean-ctx and all non-RTK behavior untouched.

## Testing
```text
pytest tests/test_wrap_rtk_opt_in.py   -> 4 passed
ruff check / format                    -> clean
mypy headroom                          -> Success: no issues found in 504 source files
```
Verified `--rtk` appears in `wrap {claude,codex,copilot} --help`;
`_rtk_opt_in()` is False by default, True with `HEADROOM_RTK=1`; entry
points no-op + write nothing when off.

## Real Behavior Proof
- Env: local `.venv`, click CliRunner.
- Steps: import wrap; assert gate default-off / env-on; assert
`_setup_rtk`/`_ensure_rtk_binary` return None and
`_inject_rtk_instructions` returns False + writes no file when not opted
in; assert `--rtk` in subcommand help.
- Observed: all pass. Not tested: a live end-to-end wrap launch (proxy
spawn).

## Notes
Part 2 of 3 (RTK opt-in). Separate PRs cover the code-graph MCP engine
and the proxy-option cleanup. No `CHANGELOG.md` edit — release-please
generates it from the PR title (per the changelog guard).

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-17 16:47:19 -07:00
Tanmay Garg 4e2bbfee3f fix(opencode): Use opencode.jsonc when present (#1590)
## Description

Fix OpenCode proxy injection so it respects user configurations that use
the `.jsonc` extension, preventing Headroom from creating a duplicate
`.json` file that overrides it.

Closes #1588

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Updated `opencode_config_path` in `paths.py` to check for `.jsonc`
- Updated backup creation in `config.py` to preserve the original
extension

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
N/A
```

## Real Behavior Proof

- Environment: local headroom dev
- Exact command / steps: creating a dummy
`.config/opencode/opencode.jsonc` and running `headroom wrap opencode`.
- Observed result: Headroom successfully injects into `.jsonc` and
creates a backup named `opencode.jsonc.headroom-backup`.
- Not tested: N/A

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

## Additional Notes
2026-07-16 13:51:21 -07:00
20968a4fa4 fix(wrap/opencode): unwrap removes the rtk block from AGENTS.md (#2025)
## Description

`headroom wrap opencode` injects the marker-fenced rtk guidance block —
"prefix shell commands
with `rtk`" — into **both** instruction files (`headroom/cli/wrap.py`):

```python
# wrap opencode
project_agents = Path.cwd() / "AGENTS.md"
_inject_rtk_instructions(project_agents, verbose=verbose)
global_agents = _opencode_home_dir() / "AGENTS.md"
_inject_rtk_instructions(global_agents, verbose=verbose)
```

But `unwrap_opencode` only restores the OpenCode config and cleans up
MCP servers — it never
removes that rtk block. So after `unwrap opencode`, both `AGENTS.md`
files still contain the
marker-fenced instruction, and a plain `opencode` launch keeps following
"prefix shell commands
with `rtk`" and fails once the managed rtk binary is off PATH.

`unwrap_codex` (#1421) and `unwrap_copilot` both already do this cleanup
via
`_remove_rtk_instructions`; opencode was simply never given the
equivalent — a wrap/unwrap
asymmetry.

Closes: no issue filed — found while auditing wrap/unwrap symmetry
across agents.

## Fix

In `unwrap_opencode`, after the MCP cleanup, strip the rtk block from
both files it was injected
into, mirroring `unwrap_codex`:

```python
for _agents_md in (Path.cwd() / "AGENTS.md", _opencode_home_dir() / "AGENTS.md"):
    if _remove_rtk_instructions(_agents_md):
        click.echo(f"  Removed Headroom rtk instructions from {_agents_md}.")
```

Best-effort and unconditional, matching the existing MCP cleanup and the
codex/copilot unwrap
paths. `_remove_rtk_instructions` already no-ops when the file or marker
is absent.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/cli/wrap.py`: `unwrap_opencode` removes the rtk block from
the project and global `AGENTS.md`.
- `tests/test_cli/test_wrap_opencode.py`: add
`test_unwrap_opencode_removes_rtk_from_agents_md` (wrap injects into
both, unwrap removes from both).

## Testing

- [x] New regression test added (`tests/test_cli/test_wrap_opencode.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).

```text
$ uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_opencode.py
All checks passed!
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the wrap→unwrap
round-trip through the new Click-runner test (which drives the real
command) and reasoned through the marker logic; the full pytest runs on
CI.
- Exact command / steps: the added test runs `wrap opencode --no-mcp`
(asserts `_RTK_MARKER` present in both the project and global
`AGENTS.md`), then `unwrap opencode`, and asserts the marker is gone
from both.
- Observed result (the assertions the test enforces): before the fix,
`unwrap opencode` left `_RTK_MARKER` in both files; after the fix both
are clean:

```text
after wrap:   _RTK_MARKER in project AGENTS.md  ✓   _RTK_MARKER in global AGENTS.md  ✓
after unwrap: _RTK_MARKER absent (project)      ✓   _RTK_MARKER absent (global)      ✓
```

- Not tested: launching a real `opencode` binary (mocked in the test, as
the existing wrap tests do). Full local `pytest` deferred to CI (OOM,
per above).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + the new Click-runner test path; full pytest deferred to CI (local
OOM, disclosed above)
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- Directly parallels the merged `unwrap codex` rtk cleanup (#1421); no
new dependencies.
- @JerrettDavis tagging you — same class as the codex rtk fix, just the
opencode side that was missed. Thanks!

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-13 09:53:45 -04:00
Andrew BarnesandTejas Chopra f53f720eb5 feat(wrap): allow project RTK instruction opt-out (#2078)
## Description

Add an opt-out for project-level RTK guidance when wrapping OpenCode, so
teams can preserve an existing repository `AGENTS.md` while still
installing RTK and its global OpenCode instructions.

Closes #1980

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring

## Changes Made

- add `--no-project-rtk` to `headroom wrap opencode`
- leave the repository `AGENTS.md` untouched when requested
- continue installing RTK and its global OpenCode guidance
- cover preservation of existing team instructions

## Testing

- [x] Focused unit test passes (`pytest`)
- [x] Touched-file linting passes (`ruff check`)
- [ ] Type checking passes (`mypy headroom`) — not run
- [x] New test added for new functionality
- [ ] Manual testing performed

### Test Output

```text
uv run pytest tests/test_cli/test_wrap_opencode.py::test_wrap_opencode_no_project_rtk_only_skips_project_agents_md -q
1 passed

uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_opencode.py
All checks passed!

uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_opencode.py
2 files already formatted
```

## Real Behavior Proof

- Environment: local Python test environment
- Exact command / steps: invoke the focused OpenCode wrapper test with
an existing project `AGENTS.md` and project RTK guidance disabled
- Observed result: the existing project instructions remain
byte-for-byte untouched while the global RTK guidance path still runs
- Not tested: manual OpenCode launch or the full repository test suite

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings in the scoped checks
- [x] I have added a test that proves the feature works
- [x] Relevant existing and new unit coverage passes locally
- [ ] Documentation and changelog updates — not applicable for this
self-documenting CLI option

## Additional Notes

The opt-out is deliberately narrow: it suppresses only the project
`AGENTS.md` mutation, not RTK installation or global OpenCode
configuration.

Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
2026-07-13 09:37:17 -04:00
Rudimar Ronsoni 4bd3ddfaa5 fix(opencode): use local MCP config (#1383)
## Description

Fixes OpenCode Headroom MCP configuration across wrap, MCP
install/status/uninstall, and persistent install docs/CLI.

OpenCode was being configured to use a remote HTTP MCP endpoint at
`/mcp`, but the Headroom proxy does not expose MCP there. The correct
OpenCode configuration is a local stdio MCP server that runs `headroom
mcp serve`.

Closes #1380

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [x] Documentation update
- [x] Tests

## Changes Made

- Changed OpenCode MCP registration to emit `type: "local"` with
`command: ["headroom", "mcp", "serve"]`.
- Changed OpenCode MCP environment serialization from `env` to
OpenCode's `environment` key, while still reading legacy `env` entries.
- Removed generated remote `/mcp` entries from OpenCode wrap/runtime
config.
- Made `wrap opencode --no-mcp` skip persistent `mcp.headroom`
injection.
- Kept provider-only OpenCode config injection from writing MCP; MCP
persistence is owned by the registrar path.
- Made `headroom mcp status` and `headroom mcp uninstall` use the
registrar lifecycle so OpenCode is covered.
- Added `opencode` to persistent install `--target` choices.
- Clarified OpenCode persistent install docs to use `--scope provider`
for direct `opencode.json` edits.
- Added regression coverage for registrar serialization, wrap behavior,
runtime config, provider-scope install, MCP CLI lifecycle, and install
target parsing.

## Testing

- [x] `rtk .venv/bin/python -m pytest tests/test_mcp_registry
tests/test_cli/test_mcp.py tests/test_cli/test_wrap_opencode.py
tests/test_providers_opencode_config.py
tests/test_providers_opencode_install.py tests/test_install -q`
- [x] Result after absorbing #1381 overlap: `263 passed, 1 skipped`
- [x] Targeted Ruff check passed for the changed Python/test files.
- [x] Targeted Ruff format check passed for the changed Python/test
files.
- [x] Isolated HOME smoke tests with real `opencode mcp list --pure`.

## Real Behavior Proof

- `headroom mcp install --agent opencode --proxy-url
http://127.0.0.1:9000 --force` against an isolated HOME wrote a valid
local OpenCode MCP entry with `environment.HEADROOM_PROXY_URL`.
- `opencode mcp list --pure` against that isolated HOME connected to
`headroom mcp serve`.
- `headroom wrap opencode --prepare-only --no-rtk --no-serena --port
9001` wrote local MCP plus provider config.
- `headroom wrap opencode --prepare-only --no-rtk --no-serena --no-mcp
--port 9002` wrote provider config without `mcp.headroom`.
- Generated runtime `OPENCODE_CONFIG_CONTENT` was accepted by `opencode
mcp list --pure`; `include_mcp=False` reported no MCP servers.
- `headroom mcp status` detected the isolated OpenCode config and read
the custom proxy URL.
- `headroom mcp uninstall` removed `mcp.headroom` from the isolated
OpenCode config while leaving provider config intact.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
2026-07-06 06:22:15 -07:00
Vinay Gupta 6c83790680 fix(opencode): write local MCP config (#1381)
## Description

Fixes the OpenCode config corruption reported in #1380 for wrap, MCP
registration, and provider-scope install paths.

OpenCode MCP entries are local stdio servers, not remote HTTP endpoints.
This changes Headroom's OpenCode MCP serialization to write `type:
"local"` with `command: ["headroom", "mcp", "serve"]`, uses OpenCode's
`environment` field for MCP env vars, and still reads the older `env`
key for compatibility.

This also stops provider-only OpenCode config injection from creating a
fake `http://127.0.0.1:<port>/mcp` entry, so `headroom wrap opencode
--no-mcp` no longer leaves `mcp.headroom` behind. Finally, the install
CLI/docs now accept and document `--target opencode` with provider
scope.

This does not change the broader `headroom mcp status/uninstall`
behavior from #1380; that looks like a separate follow-up.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Write OpenCode MCP entries as local stdio config instead of remote
`/mcp` config.
- Use `environment` for OpenCode MCP env vars while continuing to read
legacy `env` entries.
- Stop OpenCode provider injection/persistent provider install from
adding MCP config.
- Keep `--no-mcp` from writing `mcp.headroom` while preserving other MCP
entries such as Serena.
- Allow `headroom install apply --target opencode` at the CLI layer.
- Update OpenCode docs and changelog.

## Testing

- [x] Focused unit tests pass
- [x] Linting passes (`ruff check .`)
- [x] Formatting passes (`ruff format --check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for the fixed behavior
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_mcp_registry_opencode.py tests/test_cli/test_wrap_opencode.py tests/test_providers_opencode_config.py tests/test_providers_opencode_install.py tests/test_cli/test_install_cli.py tests/test_install/test_providers.py
Pytest: 164 passed

$ uvx ruff check .
All checks passed!

$ uvx ruff format --check .
986 files already formatted

$ uvx mypy --config-file pyproject.toml headroom
Success: no issues found in 398 source files
```

## Real Behavior Proof

- Environment: macOS local worktree at
`/Users/vinaygupta/Desktop/git/headroom-fix-opencode-mcp-config`; branch
`fix-opencode-mcp-config`; commit `aea96208`.
- Exact command / steps: ran the focused OpenCode/installer regression
suite plus Ruff lint/format checks and mypy commands shown above.
- Observed result: the focused tests pass and cover OpenCode MCP
serialization as `type: "local"`, `command: ["headroom", "mcp",
"serve"]`, `environment` env vars, `--no-mcp` not writing
`mcp.headroom`, provider-scope install not adding MCP config, and
`install apply --target opencode` being accepted.
- Not tested: full `pytest` locally, because collection requires the
native `headroom._core` extension in this worktree. Attempting the
project runner hit a local native build failure first: `esaxx-rs` failed
compiling `src/esaxx.cpp` with `fatal error: 'cstdint' file not found`.
The broader generic `headroom mcp status/uninstall` behavior from #1380
is intentionally left for a follow-up.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

Scope note: generic `mcp status/uninstall` support from #1380 is
intentionally left as a separate follow-up PR.
2026-06-26 12:23:54 -05:00
Shengbo_WangandClaude Opus 4.6 a0cb7982e3 fix(cli): add explicit UTF-8 encoding to file I/O in wrap commands (#1126) (#1164)
## Description

On Windows, `Path.read_text()` and `open()` default to the system locale
encoding (cp1252, GBK, etc.) instead of UTF-8. This causes
`UnicodeDecodeError` when reading or writing instruction files that
contain multi-byte UTF-8 characters such as smart quotes or em dashes.

The RTK instructions block itself contains an em dash (U+2014, `—`), so
`_inject_rtk_instructions` crashes on any non-UTF-8 Windows locale when
writing to `.copilot_instructions`, `.clinerules`, `AGENTS.md`, or
similar hint files.

Closes #1126

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `encoding="utf-8"` to all bare `read_text()`, `write_text()`, and
`open()` calls in `headroom/cli/wrap.py` that handle instruction or
config files (18 call sites)
- Update test assertions in `test_wrap_hintfile_agents.py`,
`test_wrap_copilot.py`, and `test_wrap_bridge.py` to read with
`encoding="utf-8"`
- Add `test_inject_rtk_handles_utf8_content` verifying that existing
hint files with smart quotes and em dashes survive RTK injection without
crashing

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_cli/test_wrap_hintfile_agents.py tests/test_cli/test_wrap_copilot.py tests/test_cli/test_wrap_bridge.py -v
47 passed in 1.28s
```

## Real Behavior Proof

- Environment: Windows 11 China (GBK locale), Python 3.11, headroom main
(f03e77b)
- Exact command / steps: python -m pytest
tests/test_cli/test_wrap_hintfile_agents.py -v (on GBK Windows)
- Observed result: Before fix,
test_prepare_only_injects_rtk_into_hintfile fails with
UnicodeDecodeError 'gbk' at position 283 (the em dash in RTK block).
After fix, all 12 hintfile tests pass including new UTF-8 round-trip
test.
- Not tested: no manual `headroom wrap copilot` run against a real
Copilot installation

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

This is the same class of bug reported in #733 (GBK config.toml
corruption). This PR fixes the `wrap.py` call sites; other modules
(`learn/analyzer.py`, `install/providers.py`) have the same pattern and
could benefit from the same treatment in a follow-up.

---------

Signed-off-by: Yiming Zeng <yzeng424@gmail.com>
Signed-off-by: RTCartist <wangshengb@buaa.edu.cn>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-26 12:07:03 -05:00
ParideboyandClaude Opus 4.8 487aa71a3c ci: restore green lint (reformat for ruff 0.15.17, fix mypy no-any-return, pin linters) (#1295)
## Description

The CI lint job (`ruff check .` → `ruff format --check .` → `mypy
headroom`) was red on `main`
and therefore on every open PR, for two unrelated reasons that the early
ruff failure was masking:

1. **ruff**: the lint job installs `ruff` unpinned, and ruff 0.15.17
began enforcing import-block
sorting (`I001`) and formatting that older ruff accepted → `ruff check
.` / `ruff format --check .`
   fail on files nobody touched.
2. **mypy**: `headroom/providers/opencode/config.py` had two `return
json.loads(...)` statements in
a function declared `-> dict[str, Any]`; `json.loads` is typed `Any`, so
`mypy headroom` fails
with `no-any-return` (reproduced on mypy 1.20.2 — not a version-specific
quirk).

This restores a green lint baseline and pins both linters so a future
release can't silently break
CI again.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `.github/workflows/ci.yml`: pin `ruff==0.15.17` and `mypy==1.20.2` in
the lint job.
- Applied `ruff check --fix .` (6 `I001` import-sort fixes) and `ruff
format .` (10 files) across
  the repo — import ordering and whitespace only, no behavior change.
- `headroom/providers/opencode/config.py`: narrow both
`_parse_json_loose` return sites with an
`isinstance(parsed, dict)` guard, so the `dict[str, Any]` annotation is
true at runtime
(non-dict JSON falls back to `{}`) and mypy's `no-any-return` is
resolved.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, `ruff format --check .`, `mypy
headroom --ignore-missing-imports`)

### Test Output

```text
$ python -m ruff check .
All checks passed!

$ python -m ruff format --check .
913 files already formatted

$ python -m mypy headroom/providers/opencode/config.py --ignore-missing-imports
Success: no issues found in 1 source file

$ python -m pytest tests/test_providers_opencode_config.py -q
37 passed
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13, ruff 0.15.17, mypy 1.20.2,
branch ci/fix-ruff-lint off
  headroomlabs-ai/main
- Exact command / steps: reproduced the red lint (latest ruff: 6 `I001`
+ 10 unformatted files;
the mypy failure was read from the #1295 CI lint log —
`config.py:125,133 no-any-return`, and
re-confirmed locally on mypy 1.20.2). Applied the ruff auto-fix/format,
added the dict guard,
  pinned both linters, and re-ran each lint step.
- Observed result: `ruff check .` → "All checks passed!"; `ruff format
--check .` → "913 files
already formatted"; `mypy` on the fixed file → "Success: no issues
found"; full `mypy headroom`
reports only Unix `fcntl` attributes that don't exist on this Windows
box (present on the Linux
CI runner, where the prior run showed exactly the two now-fixed errors).
37 opencode-config
  tests pass.
- Not tested: did not run the full OS/Python test matrix — the change is
formatting + two CI
dependency pins + a two-line type-narrowing guard, with no runtime
behavior change for dict JSON.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:14:40 -05:00
Rudimar RonsoniandRudimar Ronsoni b4571cc346 feat: headroom wrap opencode / unwrap opencode CLI (#1105)
## Summary

This PR implements transparent `headroom wrap opencode` support without
asking users to edit OpenCode provider URLs, choose an extra CLI flag,
or maintain a static provider list.

The wrapper now lives at the runtime transport boundary: OpenCode keeps
its user/provider config, while Headroom intercepts outbound provider
traffic in-process and routes it through the local Headroom proxy.

## What changed

### Transparent OpenCode wrapping

- `headroom wrap opencode` injects the `headroom-opencode` plugin
through `OPENCODE_CONFIG_CONTENT`.
- Existing OpenCode provider URLs are preserved. We do not rewrite user
config URLs to point at Headroom.
- Existing `OPENAI_BASE_URL` and `ANTHROPIC_BASE_URL` env vars are
preserved.
- Local OpenCode traffic, localhost traffic, and Headroom proxy traffic
bypass the shim to avoid loops.

### Runtime transport interception

- Added an OpenCode plugin transport shim that wraps:
  - `globalThis.fetch`
  - `http.request` / `http.get`
  - `https.request` / `https.get`
- External provider calls are routed to the local Headroom proxy.
- The original upstream origin is passed through `x-headroom-base-url`,
so the proxy can forward to the real provider without changing OpenCode
config.
- External `http2.connect` is blocked loudly instead of allowing direct
provider traffic to leak outside Headroom.

### Live provider additions

Provider coverage is no longer based on a static config scan. Because
routing happens at outbound request time, providers added mid-session
are routed through Headroom automatically as long as they use the
covered Node transport paths.

### Subagent and child-process coverage

- The parent OpenCode plugin sets a packaged Node preload shim through
`NODE_OPTIONS=--import=.../hook-shim/handler.js`.
- The transport shim patches `child_process.spawn`, `exec`, `execFile`,
and `fork` so child Node processes receive the Headroom preload even
when OpenCode passes a custom `env`.
- The child-process shim fails closed if it loads without
`HEADROOM_OPENCODE_TRANSPORT_PROXY_URL`.
- This closes the subagent leak path where a child Node process could
otherwise start without Headroom transport interception.

## Why this goes beyond PR #1089

PR #1089 improves OpenCode provider registration, but it still focuses
on provider config shape. This PR moves the enforcement boundary to
runtime transport interception.

This PR goes further because:

- No provider URL rewriting is required.
- New providers added mid-session are covered automatically.
- Subagents and child Node processes inherit the Headroom transport
shim.
- Direct external HTTP/2 paths fail loudly instead of leaking.
- The wrap remains transparent to the user's OpenCode provider config.
- The wrapper is fail-closed for unsupported child-process preload
state.

## Additional robustness fixes

While validating the change in Docker, the full Python suite exposed
unrelated Linux/container robustness issues. These are fixed in this PR
so the suite is green:

- Binary cache handling now treats cache paths under a non-writable
existing parent as unavailable, including when tests run as root in
Docker.
- `release_version.py` honors `MANUAL_VER` before git calls so direct
script execution works outside a `.git` checkout.
- Test logger isolation now resets relevant Headroom child loggers so
proxy logging setup cannot poison later `caplog` tests.
- The scanner missing-path test now uses a guaranteed missing `tmp_path`
child instead of relying on `/nonexistent/path`.

## Validation

All implementation validation was run inside Docker.

- Full Python suite from a fresh Docker copy: `6605 passed, 523
skipped`.
- Ruff on changed Python/OpenCode paths: passed.
- OpenCode plugin typecheck: passed.
- OpenCode plugin tests: `9 passed`.
- OpenCode plugin build: passed.
- Hook shim preload smoke test: passed.

## Notes

This PR intentionally does not add a CLI option. `headroom wrap
opencode` means full wrap. Either Headroom wraps OpenCode transparently,
or the path fails loudly instead of silently leaking provider traffic.

---------

Co-authored-by: Rudimar Ronsoni <6081613+rudironsoni@users.noreply.github.com>
2026-06-22 11:07:12 -05:00