Commit Graph
8 Commits
Author SHA1 Message Date
89f90bb411 fix(install): write persistent shell env values verbatim (#4289)
## Description

`headroom install apply --scope user` (and `--scope system`) writes a
managed `export` block into `~/.bashrc`, `~/.zshrc` and `~/.profile` (or
the `/etc` equivalents). Two bugs in `headroom/install/providers.py`
meant the values in that block didn't survive:

1. **No shell quoting.** `_env_block()` wrote `export NAME="value"` with
no escaping, so a `$`, `"` or backtick in a value is expanded or breaks
the line when the shell sources the file. For example, `--env
HTTPS_PROXY=http://bob:pa$$w0rd@proxy:3128` comes back as
`http://bob:pa43794w0rd@proxy:3128` (`$$` is the shell's PID), and
`{"a": "b"}` comes back as `{a: b}`.
2. **The block was used as a regex template on re-apply.** When the
block already exists, `_merge_marker_block()` calls `pattern.sub(block,
existing)`, so backslashes in the new block are parsed as escapes.
Re-running `install apply` with a value like `D:\work\headroom` or `\d+`
raises `re.error: bad escape \w` and the install aborts.

The supervisor runner script in `install/supervisors.py` already writes
the same values with `shlex.quote`, so only the shell-rc path was
affected.

Closes: N/A (no matching issue found)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/providers.py`: `_env_block()` writes `export
NAME=<shlex.quote(value)>`. Plain values such as `8787` or
`http://127.0.0.1:8787` stay unquoted; anything with shell
metacharacters is single-quoted.
- `headroom/install/providers.py`: `_merge_marker_block()` passes a
callable to `pattern.sub`, so the block is inserted as literal text.
- `tests/test_install/test_providers.py`:
`test_unix_env_block_round_trips_shell_special_values` applies a block
with `$$`, double quotes and backticks, re-applies it with a backslash
value added, then sources the file with real `bash` and checks every
value comes back byte for byte and that there is still exactly one
block.

## Testing

- [x] Unit tests pass (`pytest`): `tests/test_install/`, not the full
suite
- [x] Linting passes (`ruff check .`): on the changed files
- [ ] Type checking passes (`mypy headroom`): not run
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

New test on `main` (fix reverted, test kept) fails on the re-apply
crash:

```text
$ python -m pytest tests/test_install/test_providers.py -q -k round_trips
E   re.error: bad escape \d at position 185 (line 5, column 38)
======================= 1 failed, 54 deselected in 4.45s =======================
```

With only the `re.sub` half of the fix, it fails on the quoting half:

```text
E     At index 0 diff: 'http://bob:pa31588w0rd@proxy:3128' != 'http://bob:pa$$w0rd@proxy:3128'
======================= 1 failed, 54 deselected in 4.01s =======================
```

With the full fix:

```text
$ python -m pytest tests/test_install -q
======================= 251 passed, 13 skipped in 45.17s =======================

$ ruff check headroom/install/providers.py tests/test_install/test_providers.py
All checks passed!
$ ruff format --check headroom/install/providers.py tests/test_install/test_providers.py
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS 27.0.1 (arm64), Python 3.12.11, source tree on this
branch (native `headroom._core` from the 0.41.0 wheel), `HOME` pointed
at a fresh temp dir, bash 3.2 and zsh 5.9 from macOS
- Exact command / steps: built a real user-scope manifest with
`headroom.install.planner.build_manifest(preset="persistent-service",
scope="user", targets=["claude"], extra_env={"HTTPS_PROXY":
"http://bob:pa$$w0rd@proxy:3128", "HEADROOM_WORKSPACE_DIR":
r"D:\work\headroom"})` (what `install apply --env` produces) and ran
`headroom.install.providers.apply_mutations()` twice (first install,
then a re-run), then sourced the written `~/.bashrc` in `bash` and `zsh`
and printed both variables. Same script on `main` and on this branch.
- Observed result: on `main` the second apply fails with `bad escape \w`
and both shells read the proxy password as `pa43794w0rd`; on this branch
both applies succeed and both shells read the values exactly.

  ```text
  == main
  apply #1: ok
  apply #2: error: bad escape \w at position 356 (line 10, column 34)
  export HTTPS_PROXY="http://bob:pa$$w0rd@proxy:3128"
  bash sees: ['http://bob:pa43794w0rd@proxy:3128', 'D:\\work\\headroom']
  zsh sees: ['http://bob:pa43804w0rd@proxy:3128', 'D:\\work\\headroom']
  == this branch
  apply #1: ok
  apply #2: ok
  export HTTPS_PROXY='http://bob:pa$$w0rd@proxy:3128'
  export HEADROOM_WORKSPACE_DIR='D:\work\headroom'
  export ANTHROPIC_BASE_URL=http://127.0.0.1:8787
  bash sees: ['http://bob:pa$$w0rd@proxy:3128', 'D:\\work\\headroom']
  zsh sees: ['http://bob:pa$$w0rd@proxy:3128', 'D:\\work\\headroom']
  ```
- Not tested: the full `headroom install apply` CLI end to end (it would
also install a launchd service on this machine, so I called the same
planner + `apply_mutations` functions it uses); fish shell; system scope
(`/etc/*`); Windows (the PowerShell path is separate and unchanged).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: values in the managed shell block are
now shell-quoted (`export X=8787` instead of `export X="8787"`); what
the shell ends up with is the same for plain values and now correct for
values with `$`, `"`, backticks or backslashes. Existing blocks are
rewritten on the next apply; `install remove` matches on the markers, so
it is unaffected.
- Kill switch / disable path: N/A
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Docs unchecked: no user-facing docs describe the block's quoting.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4289"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->


### Reviewer-requested commit history correction

Reworded the merge commit as `chore: merge upstream main for aiohttp
remediation`, preserving both merge parents and the exact final source
tree (`e39ceb04133a87face651962106bc843d506d8ac`). The resulting head is
`9a4d01111e788d132b42d37f8f65a5fd88f70d37`. Commitlint passes over the
repaired contribution range. This update changes commit metadata only;
previous source validation is unchanged. The documented upload-hook
exception was used for this metadata-only repair; full native checks and
required hosted security CI are not claimed as passing.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-28 20:59:23 -05:00
Rod Boev 22def93177 fix(mcp): register managed installs with a resolvable headroom command (#1386)
## Description

Managed Headroom installs can register the MCP server with a bare
`headroom mcp serve` command even when the active runtime lives in a
venv outside `PATH`. That leaves Claude and Codex with a registration
they cannot re-launch reliably, and Claude eventually fails with `Failed
to reconnect to headroom: ENOENT`.

This PR reuses the existing runtime command resolver when building the
shared Headroom MCP spec, so the generated registration follows the
active install instead of assuming `headroom` is globally discoverable.
It also updates the shared-builder and registrar tests so the proof rows
now flow through `build_headroom_spec()` and prove the same resolved
command contract on both the Claude CLI path and the Codex TOML path. A
follow-up CI fix keeps the Docker init E2E expectation aligned with that
same resolver-backed contract.

Closes #487

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/mcp_registry/install.py`: build the Headroom MCP server spec
from the canonical runtime command resolver instead of hardcoding
`headroom mcp serve`
- `tests/test_mcp_registry/test_install.py`: cover the shared builder's
direct-binary and module-fallback command shapes
- `tests/test_mcp_registry/test_claude_registrar.py`: prove the Claude
CLI registration forwards the resolved command vector end to end
- `tests/test_mcp_registry/test_codex_registrar.py`: prove the Codex
registrar writes the same resolved command vector into TOML
- `e2e/init/run.py`: derive the Docker init E2E's expected Claude MCP
registration argv from `resolve_headroom_command()` so the CI harness
follows the same runtime contract
- `CHANGELOG.md`: note the managed-install MCP registration fix

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_mcp_registry/test_install.py -v`, `uv run pytest
tests/test_mcp_registry/test_claude_registrar.py -v`, `uv run pytest
tests/test_mcp_registry/test_codex_registrar.py -v`)
- [x] Linting passes (`uv run ruff check .`)
- [ ] Type checking passes (`uv run mypy headroom`) or explain N/A
truthfully
- [x] New tests added for new functionality when applicable
- [ ] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_mcp_registry/test_install.py -v
============================= 12 passed in 0.13s ==============================

$ uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v
============================= 24 passed in 0.18s ==============================

$ uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v
============================= 25 passed in 0.20s ==============================

$ uv run python -c "from e2e.init.run import _expected_headroom_mcp_call; print(_expected_headroom_mcp_call('http://127.0.0.1:9011'))"
['mcp', 'add', 'headroom', '-s', 'user', '-e', 'HEADROOM_PROXY_URL=http://127.0.0.1:9011', '--', '.../headroom', 'mcp', 'serve']

$ uv run ruff check e2e/init/run.py
All checks passed!

$ uv run ruff format e2e/init/run.py --check
1 file already formatted

$ uv run ruff check .
All checks passed!

$ uv run ruff format . --check
987 files already formatted
```

`uv run mypy headroom` was not run locally; this repo's focused local
gate for the touched Python registry path is the targeted pytest set
plus Ruff.

## Real Behavior Proof

- Environment: managed-install-safe MCP registration path, Python 3.11+,
no provider required
- Exact command / steps: run the focused MCP registry pytest files,
inspect the captured Claude CLI argv and rendered Codex TOML block, and
verify the Docker init E2E expectation derives its Claude MCP argv from
the same runtime helper
- Observed result: the persisted MCP registration uses a resolvable
command tied to the active Headroom runtime instead of bare `headroom`,
while `HEADROOM_PROXY_URL` handling stays unchanged
- Not tested: full live Claude reconnect against a real managed venv,
unless that is run during implementation

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- Scoped to the MCP registration slice in `#487`. The RTK hook rewriting
thread from the same issue is intentionally out of scope here.
- `@erikpr1994` isolated the managed-install `ENOENT` failure mode in
the issue thread and narrowed it to the bare-command MCP registration
path.
- If existing owned registrations with the old bare-command contract
need an in-place upgrade path, that should be handled explicitly in the
final diff rather than left implicit.
2026-06-26 23:39:00 -05:00
Logan Kang dff6a19946 fix(codex): write canonical hooks feature flag and migrate deprecated codex_hooks (#743)
## Description

`headroom init codex` writes the hooks feature flag into
`.codex/config.toml`
under the key `codex_hooks`. Codex renamed the canonical key to `hooks`
and kept
`codex_hooks` as a legacy alias (openai/codex#20522). Current Codex
builds warn
about `[features].codex_hooks` and tell users to use `[features].hooks`
instead,
so configs written by headroom should stop emitting the deprecated key.

This PR switches headroom to write the canonical `hooks` key and
**migrates
existing configs in place**. The migration is the tricky part: a config
can
already contain `codex_hooks`, `hooks`, or both, in any order, inside or
outside
headroom's marker block — and a naive replace can emit a *duplicate*
`hooks`
key, which is invalid TOML that Codex rejects outright.

The fix strips every `codex_hooks` line up front (any value, anywhere) —
mirroring the existing top-level key cleanup in `_ensure_codex_provider`
(#260)
— then guarantees `hooks` is present without ever duplicating it, and
respects a
user-managed `hooks` value that lives outside our marker block.

Fixes: N/A (no tracking issue — surfaced while aligning with Codex >=
0.129;
related upstream context: openai/codex#20522 and the warning behavior
discussed
in openai/codex#22148)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Write the canonical `hooks` key instead of the deprecated
`codex_hooks` in
  `_ensure_codex_feature_flag` (`headroom/cli/init.py`).
- Strip any `codex_hooks` line (any value, inside or outside the marker
block)
before ensuring the flag, so re-running `init` migrates a legacy config
instead
of leaving a stale key or producing a duplicate `hooks` key (invalid
TOML).
- Respect a user-managed `hooks` value found outside headroom's marker
block
  (e.g. `hooks = false`); only the deprecated alias is removed.
- Make the insert/create paths match `_replace_marker_block`'s
normalisation so
  re-running `init` is byte-idempotent.
- Extract a `_codex_feature_block()` helper to remove the 4x duplicated
marker
  block assembly.
- Add regression tests for the previously-broken edge cases.

## Testing

- [x] Unit tests pass (`pytest`) — affected module fully green (see
output)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom/cli/init.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed (reproduced each edge case against the
patched
      function via `tomllib.loads`)

## Test Output

```
$ pytest -v tests/test_cli/test_init_cli.py -k "feature_flag or hooks_feature"
tests/test_cli/test_init_cli.py::test_init_codex_merges_feature_flag_into_existing_table PASSED
tests/test_cli/test_init_cli.py::test_init_codex_creates_hooks_feature_flag_on_first_init PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_replaces_existing_marker PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_legacy_codex_hooks_key PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_when_both_keys_present PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_when_keys_reversed PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_drops_legacy_key_outside_marker PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_is_idempotent PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_creates_features_section_when_missing PASSED
======================= 9 passed, 45 deselected in 0.24s =======================

$ pytest -q tests/test_cli/test_init_cli.py
54 passed

$ ruff check .
All checks passed!

$ mypy headroom/cli/init.py
Success: no issues found in 1 source file
```

Note: the broader `tests/test_cli/` run has one unrelated failure
(`test_wrap_copilot_auto_detects_running_proxy_backend`) caused by a
real proxy
already bound to port 8787 in the local environment — it fails
identically on a
clean checkout without this change.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (none
required)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable (managed by
release-please;
      generated from the conventional commit, not edited by hand)

## Screenshots (if applicable)

N/A

## Additional Notes

- **Why the duplicate-key path matters:** TOML forbids duplicate keys,
so a
`[features]` table containing both `codex_hooks` and `hooks` (which the
old
in-place migration could produce) makes Codex reject `config.toml`
entirely.
  The new "strip then ensure" approach can never emit two `hooks` lines.
- **Version provenance:** the `codex_hooks` -> `hooks` rename landed in
openai/codex#20522, first shipped in Codex `rust-v0.129.0`.
`codex_hooks`
remains a working legacy alias, but current Codex builds can warn users
to
  move to `[features].hooks`.
- **Idempotency:** running `headroom init codex` repeatedly now produces
a
  byte-stable `config.toml`, so there is no churn on re-init.
2026-06-12 12:49:34 -05:00
Tejas Chopra 8714c64a20 test(init-e2e): expect mcp add in seq_claude_local install sequence
d9d8972 wired auto-MCP registration into ``init`` so ``[Retrieve
more: hash=…]`` markers stay live for users who never ran
``headroom mcp install`` separately, but the ``seq_claude_local``
e2e assertion was still pinned to the pre-MCP two-command sequence
and failed in docker-native-e2e on main.

The ``-e HEADROOM_PROXY_URL=…`` arg is only emitted when the proxy
port differs from the 8787 default; this case sets ``--port 9011``,
so the env arg is included in the expected argv.
2026-05-08 17:38:34 -07:00
JerrettDavisandClaude Opus 4.7 4071d57134 fix(ci): update tests to assert absence of requires_openai_auth (bug 3, #406)
- Restore build_provider_section() to headroom/providers/codex/install.py
  without requires_openai_auth (was removed entirely; pre-existing test
  test_provider_codex_install.py imports it and would fail to collect)
- Flip test_codex_provider_section_preserves_openai_oauth to assert
  requires_openai_auth is ABSENT, not present (old behavior was wrong)
- Fix test_provider_codex_runtime.py:337 same way — init config must
  NOT contain requires_openai_auth
- Fix Ruff B023 lint error in test_providers.py:492 — capture loop
  variable config_path in lambda default arg (_p=config_path)
- Fix e2e/init/run.py _verify_codex_local and _verify_codex_global to
  assert requires_openai_auth is absent, not present
- Fix e2e/wrap/run.py verify_codex_wrap same way

All unit tests pass locally (82 affected tests green).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 12:18:47 -05:00
JerrettDavisandCopilot 06428d20fd fix: preserve Codex OAuth proxy delivery
Preserve Codex OAuth-safe provider config across init, wrap, and

persistent install paths, and strengthen coverage so Codex requests

are proven to reach Headroom and the mock upstream.

The wrap e2e now sends a real chat-completions probe and checks

Headroom /stats. Runtime tests cover temporary launch env, install

env, init config, provider-scope config delivery, and the Python

3.11 ws bootstrap path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 21:03:42 -05:00
JerrettDavisandClaude Opus 4.7 48e2431510 test(init): extend Docker e2e with bare/shim/per-subcommand cases
Port e2e/init/run.py onto the shared harness and extend coverage so
issue #245 (bare ``headroom init -g`` with no agents) is locked in:

* ``seq_claude_local`` / ``seq_copilot_global`` / ``seq_codex_local`` —
  the original scenario, now expressed as a sequence of Cases sharing
  one scratch so the manifest-merge behavior (claude + codex targets)
  is still exercised end-to-end
* ``bare_init_g_no_shims`` — regression guard for issue #245: asserts
  the new guided error mentions every probed target and the concrete
  ``headroom init -g <agent>`` example
* ``bare_init_g_with_all_shims`` — complementary happy path with all
  four shims present; asserts all three configurable agents report
  ``Configured ... (user scope)`` on stdout
* ``init_g_{claude,codex,copilot}_explicit`` — one case per
  subcommand, each with only its own shim on PATH, asserting exit 0
  and the correct per-agent settings file is written
* ``init_g_openclaw_missing`` — negative path for openclaw when its
  binary isn't installed (delegates to ``headroom wrap openclaw`` which
  can't be shimmed cheaply)
* ``init_verbose_no_shims`` — smoke test for ``headroom init -v``
  ensuring ``detect_init_targets``, ``global_scope=True``, and every
  agent name appear on stderr

Dockerfile is updated to COPY e2e/__init__.py and e2e/_lib/ so the
harness is importable inside the container. A new e2e/__init__.py
marks the tree as a package.

One small harness fix rides along: ``_resolve_headroom_bin`` captures
the absolute path to headroom before ``with_clean_path`` narrows PATH.
This is required for any case run inside a venv-scoped image - the
real ``headroom`` lives outside the shim dir and would otherwise be
hidden by the scrubbed PATH. Same bug would have bitten every future
command suite, so the fix belongs in the harness rather than run.py.

Verified locally inside the Docker image: all 10 cases pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 16:12:27 -05:00
JerrettDavisandCopilot a278a7b0ba test: cover init install flows end to end
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-21 20:15:11 -05:00