Commit Graph
5 Commits
Author SHA1 Message Date
1da8930ab4 fix(wrap): match Serena's unset dashboard key instead of appending a duplicate (#4315)
## Description

`headroom wrap` calls `_ensure_serena_dashboard_disabled()`, which turns
off Serena's dashboard auto-open by editing
`web_dashboard_open_on_launch` in `~/.serena/serena_config.yml` in
place. The regex required a non-empty value (`:\s*\S+\s*$`), but
Serena's own template writes the key with **no value**:

```yaml
web_dashboard_open_on_launch:
```

That is valid YAML meaning "unset". The regex missed it, so Headroom
appended a second copy of the key at EOF. Serena loads this file with
ruamel.yaml in duplicate-key-strict mode, so the next startup dies while
constructing the config:

```text
ValueError: Error loading Serena configuration from ~/.serena/serena_config.yml: while constructing a mapping
found duplicate key "web_dashboard_open_on_launch" with value "False" (original value: "None")
  in "~/.serena/serena_config.yml", line 247, column 1

ruamel.yaml.constructor.DuplicateKeyError
```

MCP clients surface that as `Connection closed: MCP server process
exited with code 1`, and it is persistent: every launch fails until the
file is repaired by hand. This was hit in the field with Serena 2.1.0
installed by `headroom wrap opencode`.

The setting is now rewritten line-wise instead of by regex substitution,
which fixes three things:

1. An unset value is matched, so no duplicate is appended.
2. The first `#` on the line is the comment, including when it is flush
against the colon (`web_dashboard_open_on_launch:# note: reason`). It
becomes `...: false # note: reason` (boolean `False`, comment intact,
colons kept) instead of the string `"false note: reason"` or a second
copy of the key.
3. Redundant copies of the key at the root mapping's indent are dropped,
so a config an older Headroom already broke comes back valid instead of
staying broken.
4. The root indent is the shortest indent of a non-comment line,
ignoring `---`, `...`, and `%` directives. A root mapping indented as a
whole, including one under a document marker, is rewritten in place at
that indent; a deeper key (a nested mapping that reuses the name) is
left alone. A missing key is inserted before a trailing `...` so it
stays in the one document Serena loads. Trailing tokens that are not a
comment, such as `!!bool true`, are dropped so the value stays boolean
`False`.

No tracking issue; reported from the live failure described below.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cli/wrap.py`: `_ensure_serena_dashboard_disabled` rewrites
the setting line-wise at the root mapping's indent
(`_serena_root_indent`), which is column 0 for a normal file and the
shared indent when the whole mapping is indented. Document markers and
`%` directives are not mapping lines, so they do not pull that indent
back to column 0. A missing key is inserted before a trailing `...`.
`_serena_inline_comment` keeps the first `#` (a space is inserted when
it was glued to the colon or the value) and drops every other token. The
first root occurrence is kept and redundant root copies are dropped. A
deeper key is left alone. Docstring documents the ruamel failure this
prevents.
- `tests/test_wrap_code_memory.py`: regression tests for the unset key,
a comment with a colon, a `#` flush against the colon, a populated value
with a comment, an already-duplicated config, a uniformly indented root
mapping, an indented root under `---` and under `%YAML`, a missing key
before `...`, a nested key under both a column-0 root and an indented
root, and a YAML type tag. Each asserts the parsed value is boolean
`False`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

Type checking was not run for this change; `ruff check` and `ruff format
--check` pass on both touched files.

### Test Output

```text
# headroom/cli/wrap.py reverted to the previous commit, new tests only
$ uv run pytest tests/test_wrap_code_memory.py -k "comment_on_unset_value or repairs_duplicate_keys" -q
FF                                                                       [100%]
E   AssertionError: assert 'web_dashboard_open_on_launch: false # note: leave unset for now' in
    '...web_dashboard_open_on_launch: false note: leave unset for now...'
E   AssertionError: duplicate key breaks Serena
E   assert 2 == 1
2 failed, 15 deselected in 0.30s

$ uv run pytest tests/test_wrap_code_memory.py -q
16 passed, 1 skipped in 0.24s

$ uv run ruff check headroom/cli/wrap.py tests/test_wrap_code_memory.py
All checks passed!

$ uv run ruff format --check headroom/cli/wrap.py tests/test_wrap_code_memory.py
2 files already formatted

# follow-up in 9949c4bc, including the flush-# and nested-key tests
$ .venv/bin/python -m pytest tests/test_wrap_code_memory.py -q
19 passed, 1 skipped in 0.35s

$ .venv/bin/ruff check headroom/cli/wrap.py tests/test_wrap_code_memory.py
All checks passed!

$ .venv/bin/ruff format --check headroom/cli/wrap.py tests/test_wrap_code_memory.py
2 files already formatted

# follow-up in e62338ab, indented root mapping
$ .venv/bin/python -m pytest tests/test_wrap_code_memory.py -q
21 passed, 1 skipped in 0.28s

$ .venv/bin/ruff check headroom/cli/wrap.py tests/test_wrap_code_memory.py
All checks passed!

$ .venv/bin/ruff format --check headroom/cli/wrap.py tests/test_wrap_code_memory.py
2 files already formatted

# follow-up in 8fb014f2, document markers. The three new tests fail on e62338ab.
$ .venv/bin/python -m pytest tests/test_wrap_code_memory.py -q
24 passed, 1 skipped in 0.40s

$ .venv/bin/ruff check headroom/cli/wrap.py tests/test_wrap_code_memory.py
All checks passed!

$ .venv/bin/ruff format --check headroom/cli/wrap.py tests/test_wrap_code_memory.py
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS 26.6.2 arm64, Python 3.13.12, Serena 2.1.0 (`uvx
--from serena-agent`), headroom `main` at `fbf7bc32` plus this branch;
released Headroom 0.35.0 used for the "before" case; each case in its
own temp `$HOME`, so the real user config was never touched.
- Exact command / steps: run the old and patched
`_ensure_serena_dashboard_disabled` on temp homes, then load the results
with Serena's own loader and start the real MCP server (details below).
- Observed result: old code appends a duplicate and Serena refuses to
load; patched code keeps one key, preserves comments, and the MCP
handshake completes (output below).
- Not tested: Windows and Linux, and a fresh live Serena process for the
document-marker case. Nested keys, a uniformly indented root, a `---`
marker, a `%YAML` directive, and a trailing `...` are covered by the
unit tests and by running the helper on temp configs. Quoted values that
contain a `#` are not specially handled; the value is still rewritten to
a plain boolean.

Details:

- Steps: (1) write a minimal faithful slice of the config Serena 2.1.0
generates into `$HOME/.serena/serena_config.yml` for throwaway homes;
(2) run `_ensure_serena_dashboard_disabled` (0.35.0 for "before", this
branch for "after"); (3) load each file with `python3 -c "from
serena.config.serena_config import SerenaConfig;
SerenaConfig.from_config_file()"`; (4) start the real server with
`printf '' | uvx --from serena-agent serena start-mcp-server
--project-from-cwd --context agent --open-web-dashboard False`.
- Before: file ends with an appended `web_dashboard_open_on_launch:
false` (2 occurrences) and Serena's loader raises `ValueError: Error
loading Serena configuration ... while constructing a mapping` — the
reported crash.
- After, unset key with comment: input `web_dashboard_open_on_launch: #
note: leave unset for now` becomes `web_dashboard_open_on_launch: false
# note: leave unset for now`; Serena's loader prints `loaded OK;
web_dashboard_open_on_launch = False`; a second call is byte-identical.
- After, already-broken duplicate config: 2 occurrences collapse to 1
with the comment block and `projects` intact; Serena's loader prints
`loaded OK; web_dashboard_open_on_launch = False`; the MCP server prints
`Starting MCP server …`, `MCP server lifetime setup complete` and exits
0.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no (same end state; only the
duplicate-key bug, comment corruption and blank-line churn are gone)
- Kill switch / disable path: `--no-serena` /
`HEADROOM_CODE_MEMORY=none` still skip Serena setup entirely
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit; the edit is idempotent either way

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — config file behaviour only.

## Additional Notes

Documentation checkbox: only the function docstring changed; there is no
user-facing docs page describing this internal line edit. The `mypy` box
is unchecked because it was not run. Duplicate repair keeps the first
occurrence at the root indent and its comment; comments attached to a
dropped copy are not merged. A key deeper than the root indent is a
nested mapping and is left as written. `---`, `...`, and `%` directives
are not part of that indent.


### Maintainer current-main requalification

Exact head `66ad70f02bc4994b6002e2ec2f766e8dd64fa7ae`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Requalified against upstream main
c01cf07d94 after a clean mechanical merge.
Maintainer test-only adjustment isolates Path.home() to tmp_path on
Windows and POSIX; original tests set HOME, which Windows Path.home()
ignores. Before that fixture correction the local run had 15
fixture-path failures; after correction 24 tests pass and one optional
real-Serena test skips because SERENA_SRC is unavailable. Ruff
lint/format and main-relative whitespace checks pass. Contributor
YAML-rewrite logic is unchanged.

The existing document-terminator finding still reproduces through the
actual combined-branch helper in an isolated temporary home: valid
`---\nprojects: []\n... # end of config\n` receives the new setting
after the terminator and becomes ParserError. A bare terminator followed
by a comment-only line also reproduces. Please recognize valid end
markers with trailing whitespace/comments and preserve additions inside
the document even when trailing comments follow it. Cover parsing and
idempotence for both cases. The ordinary unset-key/comment/duplicate-key
and indented-root improvements remain acknowledged; the request concerns
incomplete handling in the PR's document-marker support, not inherited
pipeline failures.

Local mocked-home config probes and current-main native DLL for Python
test startup only; no fresh real Serena, inference or branch-native
build claimed. Maintainers handle hosted checks; no contributor CI
action requested. Nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-09 23:09:09 -06:00
2649835a51 feat(wrap): make the Serena code-memory MCP opt-in (#4509)
## What changes

`headroom wrap claude|codex|grok|opencode` no longer registers Serena by
default.

| | Before | After |
|---|---|---|
| Default | Serena registered | No code-memory MCP |
| Opt in | (default) | `--code-memory serena` or
`HEADROOM_CODE_MEMORY=serena` (deprecated `--serena` still works) |
| Opt out | `--code-memory none` / `--no-serena` | Still accepted |
| Upgrading user, Serena installed by Headroom | Kept | Removed on next
wrap, with the message "Serena is now off by default; pass --code-memory
serena to keep it" |
| Upgrading user, Serena added by hand | Kept | Kept, no message (shown
only with `--verbose` or an explicit opt-out) |

OpenCode now accepts `--code-memory` and uses the same setup path as the
other agents. Its `--no-serena` flag is hidden and deprecated.

## Why

Measured on 2026-10-27, Serena started exactly as `wrap` registers it:

- It adds 22 tools, about 5.9K tokens of tool definitions on every
request where tools aren't deferred. Its server instruction tells the
model to call `initial_instructions`, which costs another 2K tokens.
- Each launch takes about 6 s cold via `uvx`, plus a background `serena
project index` that starts language servers. It also writes a `.serena/`
folder into the user's repo.
- Serena was registered in 6 of our own Claude Code sessions and called
0 times. Separately, an offline replay of about 108K tool calls found
that symbol-level reads have little to recover for Claude Code.

## Not in this PR

- `headroom.integrations.strands` still defaults
`enable_serena_mcp=True`. It's public integration API, so it needs a
deprecation step instead of a silent flip.
- The opt-in `--serena-instructions` block in CLAUDE.md/AGENTS.md is
never removed, by unwrap or anything else. Users who opted into it keep
instructions that point at tools no longer registered. This predates
this PR and will be fixed separately.
- For Claude Code, Serena entries are per project, so removal happens
the next time `wrap` or `unwrap` runs in each project. The docs say so.

## Tests

- `pytest tests/test_wrap_code_memory.py tests/test_cli
tests/test_mcp_registry`: 1368 passed. 5 failed, and those 5 fail the
same way on `main` in my local venv:
`test_mcp_reconcile.py::test_ordinary_install_does_not_adopt_serena` and
4 in `test_wrap_persistent.py`, all from an environment/install
mismatch.
- New tests:
- The default wrap removes a Serena entry Headroom installed and prints
how to keep it.
- The default wrap leaves a hand-added Serena alone and prints nothing.
  - `--serena --no-serena` resolves to none.
- `wrap opencode --code-memory serena` registers Serena with the `agent`
context.
- `ruff check`, `ruff format --check` and `mypy` are clean on the
changed files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

## Real Behavior Proof

- **Environment:** macOS (Darwin 27.2), Python 3.12.13, Claude Code
2.1.318, real `claude` CLI doing every `claude mcp add`/`remove`. A
temporary `HOME`, so `~/.claude.json` starts empty, and a fresh git repo
per case.
- **Builds:** "before" = `main` at `12864fc8`; "after" = this PR at
`56557dc9`.
- **Command:** `headroom wrap claude --no-proxy [flags] -- --version`.
Wrap does its full MCP setup, then launches Claude Code, which prints
its version and exits, so no model call is made. `state` is read from
`~/.claude.json` (user scope and this project's local scope) and the
project's `.mcp.json` after each run.

```
== 1. Upgrade: Serena installed by Headroom (project/local scope)
    $ headroom wrap claude --no-proxy -- --version    (cwd=projA, build=12864fc8)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projA): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projA, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projA): - | .mcp.json: -

== 2. Upgrade: Serena added by hand (claude mcp add)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projB, build=56557dc9)
    exit=0
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy --code-memory none -- --version    (cwd=projB, build=56557dc9)
    exit=0
    wrap|   Serena MCP is present but user-managed — leaving it in place (Headroom only removes entries it installed)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -

== 3. Explicit opt-in, then rollback to the default
    $ headroom wrap claude --no-proxy --code-memory serena -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy --serena -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -

== 4. User scope: installed by Headroom with --code-memory-scope user
    $ headroom wrap claude --no-proxy --code-memory-scope user -- --version    (cwd=projD, build=12864fc8)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     registered for every Claude Code session on this machine (--code-memory-scope user)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom', 'serena'] | local(projD): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projD, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projD): - | .mcp.json: -

== 5. User scope opt-in on the PR build
    $ headroom wrap claude --no-proxy --code-memory serena --code-memory-scope user -- --version    (cwd=projE, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     registered for every Claude Code session on this machine (--code-memory-scope user)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom', 'serena'] | local(projE): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projE, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projE): - | .mcp.json: -
```

What this shows:
1. A Serena entry Headroom installed on `main` is removed on the first
wrap after upgrading, with the "off by default" message. This holds for
both scopes (cases 1 and 4).
2. A hand-added Serena entry is kept, and the default wrap prints
nothing. `--code-memory none` keeps it and says it's user-managed (case
2).
3. Opting in works with `--code-memory serena`,
`HEADROOM_CODE_MEMORY=serena` and the deprecated `--serena`, at project
scope and user scope (cases 3 and 5). Running the default afterwards
rolls it back.
4. Project scope here means Claude Code's `local` scope, which is where
wrap registers. Wrap never writes `.mcp.json`.

Not tested live: `wrap codex|grok|opencode` (unit tests only), Windows,
and an interactive Claude session that calls Serena tools.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4509"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer post-merge qualification
Published `983b6bb15119a3ecb8f6e20bf014b8bed123e95d` includes upstream
main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 19 selector/Serena
tests pass,1optional skip; Ruff/format/diff pass. Clean main sync;
contributor selector/ownership code unchanged. Existing real
upgrade/configuration/launch proof requirement remains; jCodeMunch
proposal is unrelated to this source delta. Contributor correctness
logic was not rewritten. Existing Changes Requested decision retained
without duplicate findings. Scoped Windows validation uses the existing
development native extension, not a fresh build. Hosted qualification is
handled by maintainers; nothing merged.


### Maintainer current-main requalification

Exact head `e8e3011187a2ffdfbb6c9932db26b22ec911ab39`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Mechanically updated from upstream/main
ecbaf86d78 without changing contributor
selector/ownership logic. Reviewed the current discussion and
integration delta. All 29 focused code-memory/Serena-disable/Gemini
tests pass with one optional skip; Ruff0.17.3 lint/format and whitespace
checks pass. Existing Windows HOME/USERPROFILE fixture isolation remains
intact.

The prior upgrade/runtime qualification requirement remains unresolved:
post-patch real configuration/launch evidence must cover an existing
Headroom-owned Serena entry, a hand-managed entry, explicit opt-in, and
Claude project/user scopes, with commands/results and untested-agent
limits. No new author evidence was found to close it. This pass does not
claim those actual launch/upgrade flows were executed; passing unit
tests and the main sync do not establish that deployment behavior. The
hold is not an inherited-main CI defect. Maintainers handle hosted
checks; no contributor CI execution requested. Nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `0b49a79cb192b294dde7ecb1c72b75fffa2733d5`, upstream main
`8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`.

Clean mechanical sync with upstream
main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a without changing
contributor selector/ownership logic. Current source, author discussion
and Devin findings reviewed.29focused code-memory/Serena-disable/Gemini
tests pass with1optional skip; changed Python Ruff0.17.3check/format and
whitespace pass. Windows HOME/USERPROFILE fixture isolation preserved.

Existing real upgrade/runtime qualification remains missing: existing
Headroom-owned Serena versus hand-managed entry, explicit opt-in, and
Claude project/user scopes need observed configuration/launch results
with explicit untested-agent limits. No new evidence found to close it;
these launch/upgrade paths were not executed in this pass. Passing unit
tests and clean main integration do not establish deployment behavior.
No new provider/native-build claim or inherited-main pipeline
attribution. Maintainers handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `30dfb386fa7284c9817bf5d7db0648fb10830055`, upstream main
`7592e681459f14a2041eb5f524a658cb6ad1dd92`.

Clean current-main
sync7592e681459f14a2041eb5f524a658cb6ad1dd92,including HNSW re-index and
memory relation-key fixes after917release-recovery merge. Contributor
selector/ownership logic unchanged; fresh discussion
considered.37wrapper/Serena/memory-regression/Gemini tests pass
with1optional skip;whitespace passes. PriorRuff0.17.3validation applies
to unchanged PR files.

Existing real upgrade/launch evidence hold remains: Headroom-owned
versus hand-managed Serena
entries,explicitoptin,andClaudeproject/userscopes need
observedpostpatchresults. No newevidence closesit;actualupgrade/launch
pathsnotrerun. No freshnativebuild/provider claim.
MaintainershandleCI;nothingmerged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `e1778a5368626547ad958926d5570edf153182f6`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Clean integration with upstream/main
2d3ee51846 after the external 0.43.1
release merge. Exact headroom/crates/tests comparison against previously
reviewed 30dfb386f shows no changes. Re-reviewed generated metadata; 78
release/version/MCP tests pass on this tree and whitespace passes.
Earlier 37 wrapper/Serena/memory/Gemini passes and one optional skip
remain scoped to unchanged inputs.

Existing actual upgrade/launch proof hold remains: Headroom-owned versus
hand-managed Serena entries, explicit opt-in, and Claude project/user
scopes need observed postpatch results. Those runtime paths were not
rerun and no new evidence closes the hold. No contributor ownership
policy changed, no fresh native-build/provider claim. Maintainers handle
checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `173c5739d7357ca9398faa41b4d1ce3b72d6c550`, upstream main
`7877241d84c93b3ec487e07324245838ddae85cc`.

Cleanly integrated upstream/main7877241d84c93b3ec487e07324245838ddae85cc
and re-reviewed the wrapper overlap with Hermes registration and port
discovery.35 selector/Serena-disable/Hermes provider and wrapper tests
pass,1optional skip; Ruff and whitespace pass. Contributor
ownership/default policy is unchanged. Existing actual
upgrade/configuration/launch proof requirement remains open for
Headroom-owned versus user-managed entries, explicit opt-in, and Claude
project/user scopes. No runtime proof is inferred from mocks or
native-build claim made. Maintainers handle hosted checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `eb1c3d66669cf71f881f1ba06dc049d36b20da7b`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 35 passed, 1 skipped in 2.93s.
Selected real dependency overlays and native DLL paths are recorded in
final4509-fresh-native-tests.log; the DLL provenance/input hashes are in
final-native-artifact-proof.json. No redundant rerun is implied after
this Cargo-only merge. Whitespace passes. Prior source review and scoped
real-behavior evidence remain applicable to unchanged contributor
inputs.

Existing real upgrade/configuration/launch qualification remains open
for Headroom-owned versus user-managed Serena entries, explicit
opt-in/rollback, and Claude project/user scopes. No new author evidence
closes it; contributor ownership/default policy is unchanged.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `43bfea8a164c21ab69793b3905fbf1457fef7bc2`, upstream main
`da6d9736cd82074988c7d0a2360b49d9d8b6c0a0`.

Current-main requalification against
da6d9736cd. All69 integration patches
exactly match c01cf07-to-current-main delta after diff-location
normalization; contributor selector/default/Serena ownership logic
unchanged.162 focused code-memory,Serena disable,Hermes,Windows
supervisor,search-compressor tests pass;3 platform/optional skips. All31
changed Python files pass Ruff lint/format; offline lock and whitespace
pass. Native pr3982 DLL verified against complete tracked native-input
hash proof and artifact checksum; no fresh native-build claim.
Author/Devin discussion checked. Existing real
upgrade/configuration/launch qualification remains open: prior
Headroom-owned Serena entry versus user-managed entry, explicit
opt-in/rollback, Claude project/user scopes. No new author evidence
closes it; this run does not assert actual agent launch or deployment
proof. Maintainers handle hosted checks; no contributor policy rewrite
or merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `56557dc91725422257f90c3cb5c5f04a665e6043`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 19 integration patches
exactly match the prior-reviewed-main da6d9736 to current-main delta
after diff-location normalization; contributor selector/default/Serena
ownership logic is unchanged. Fresh code-memory/Serena
disable/scope/reconciliation/migration, doctor and incoming
Vertex/backend/CLI tests: 330 passed, 12 skipped, 1 warning. Fourteen
changed Python files pass Ruff lint/format; offline lock and whitespace
checks pass.

Author/Devin discussion read. Existing real upgrade/configuration/launch
qualification remains open: prior Headroom-owned entry versus
user-managed entry, explicit opt-in/rollback, Claude project/user
scopes. This run establishes no actual agent launch or deployment proof.
Static source/test scope and security inspection found no new workflow,
dependency or setup path; the inherited flagged workflow-validation job
is excluded by the exact PR's unchanged workflow paths. Native DLL
reused after complete tracked input-set/hash and artifact-checksum
verification; no fresh build claimed. Maintainers handle exact-head
hosted checks; no PR merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-09 12:23:08 -06:00
Tejas Chopra 759209cff3 fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676)
## Description

`_ensure_serena_dashboard_disabled()` wrote a one-key bootstrap config
(`web_dashboard_open_on_launch: false`) into
`~/.serena/serena_config.yml` when the file was absent, assuming Serena
fills in any key it omits.

Verified against **Serena 1.6.2.dev0**
(`serena/config/serena_config.py`), that holds for every field except
one. Serena autogenerates its own complete config **only when the path
does not exist**:

```python
if not os.path.exists(config_file_path):
    cls._generate_config_file(config_file_path)
```

Once any file is present it validates instead. Every other field falls
back to a dataclass default via `get_value_or_default`, but a missing
`projects` key is fatal (~line 1064):

```
SerenaConfigError: `projects` key not found in Serena configuration.
```

So Headroom's own bootstrap file killed Serena on **every machine
without a pre-existing Serena config**. The MCP server exited during
handshake — surfacing as `connection closed: initialize response` on
Codex and a bare `MCP error -32000: Connection closed` on OpenCode
(#2674) — and `serena project index` failed identically.

Headroom now leaves that file to Serena. That is immune to Serena adding
required keys later; guessing the schema is what caused the outage. The
popup never needed the file anyway: `build_serena_spec` passes
`--open-web-dashboard False`, which Serena applies *after* loading the
config (`serena/mcp.py:361` — `config.web_dashboard_open_on_launch =
open_web_dashboard`), so the flag wins regardless of what is on disk.

An **existing** config is still edited in place — dashboard key flipped,
`projects: []` backfilled to repair machines an affected version already
wrote — preserving a populated `projects` list, other keys and comments.

Closes #2674

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- `_ensure_serena_dashboard_disabled()` never creates
`serena_config.yml`; it only edits an existing one, and backfills
`projects: []` there to repair already-broken machines.
- Dropped `_scope_serena_languages` + `_detect_repo_languages` +
`_EXT_TO_SERENA_LANGUAGE` (**−133 lines**). Dead weight: Serena
determines languages itself in `ProjectConfig.autogenerate`
(`_determine_project_language_servers`) and records them under
`language_servers` — `languages`, which Headroom wrote, is a legacy name
Serena migrates via `RENAMED_FIELDS`. Serena's generated file uses a
block-style list, so our single-line-flow regex never matched it: on any
Serena-generated `project.yml` the function was a **verified no-op**.
The only case where it acted was creating the file — the same
partial-config trap — which also skipped the `project.local.yml` sidecar
Serena writes alongside.
- **Test isolation:** the MCP install ledger defaults to
`~/.headroom/mcp_installs.json`, so any test registering a server wrote
into the developer's real ledger (observed adding a live `claude/serena`
entry during a local run). `conftest.py` now redirects it per-test.
- **Repo config:** `.serena/project.yml` carried a stale `project_name`
(`"feature-opencode-wrap"`) and listed only `typescript`, so Serena's
symbol index skipped 1331 Python and 194 Rust files for every
contributor.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_serena_boost.py \
         tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py -q
35 passed, 1 skipped in 0.84s

$ SERENA_SRC=<serena checkout> pytest tests/test_wrap_code_memory.py -q
13 passed in 0.62s        # the skipped test runs when a Serena source tree is available

$ ruff check headroom/ tests/ --exclude headroom/dashboard/templates
All checks passed!

$ mypy headroom/cli/wrap.py
Success: no issues found in 1 source file
```

New tests. The key one asserts the invariant rather than our own key
list, so it stays correct even if Serena adds a required key — a test
pinning `projects: []` would keep passing while users broke again:

- `test_serena_config_is_never_created_by_headroom` — Headroom must not
pre-empt Serena's bootstrap
- `test_serena_dashboard_disabled_repairs_config_missing_projects` —
heals a config an affected version wrote
- `test_serena_dashboard_disabled_preserves_registered_projects` — never
clobbers the real registry; comments kept, no duplicate key
- `test_serena_dashboard_disabled_is_idempotent`
- `test_serena_config_required_keys_match_serena_source` — reads
Serena's real source and pins the two facts this fix rests on
(bootstrap-only-when-absent, `projects` is the sole fatal omission).
Skipped unless `SERENA_SRC` is set; deliberately **not** named
`HEADROOM_*` because `conftest.py` scrubs that namespace, which would
make it silently always-skip.

## Real Behavior Proof

- **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Serena
1.6.2.dev0 via `uvx --from git+https://github.com/oraios/serena`, Codex
CLI 0.146.0.
- **Exact command / steps:** a probe doing a real JSON-RPC `initialize`
handshake against the exact command `headroom wrap` registers — i.e.
what Codex/OpenCode actually do — in a throwaway `HOME` per case. (A)
pre-seeded with the one-line config an affected version wrote; (B) no
config; (C) real `headroom wrap codex --prepare-only`, then handshake.
- **Observed result:**

```text
=== A. BROKEN: single-key config (Headroom 0.33.0) ===
  MCP handshake: FAIL — no initialize response (exit=1). stderr tail:
    File ".../serena/config/serena_config.py", line 1064, in from_config_file
      raise SerenaConfigError("`projects` key not found in Serena configuration. ...")
    serena.config.serena_config.SerenaConfigError: `projects` key not found ...
  config after run: 1 lines, has 'projects': False

=== B. FIXED: no config, Serena bootstraps it ===
  MCP handshake: PASS — initialize OK — serverInfo.name='Serena'
  config after run: 213 lines, has 'projects': True

=== C. FULL FLOW: real `headroom wrap codex` then handshake ===
    Serena: no serena_config.yml yet — letting Serena generate it
    Serena MCP: registered (restart OpenAI Codex CLI if it was already running)
    Serena: project pre-indexed (symbol cache warmed)
    serena_config.yml: 213 lines, written by Serena (correct)
    MCP handshake: PASS — initialize OK — serverInfo.name='Serena'

--- verdict ---
  A (broken config)     started: False   <- expected False
  B (fixed, no config)  started: True   <- expected True
  C (after real wrap)   started: True   <- expected True
```

A second `wrap` in the same HOME flips the dashboard without damage:
`true` → `false`, `projects` intact, all 153 comment lines intact. The
writer was isolated against a pristine 213-line Serena config: **delta 0
newlines**.

- **Not tested:** Windows and Linux (macOS only); Serena versions other
than 1.6.2.dev0; the JetBrains language backend. The probe needs network
+ `uvx` (~2 min) so it is a manual verification tool, not wired into CI.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- **Docs:** N/A — no user-facing docs described the `serena_config.yml`
bootstrap or the language scoping.
- Also fixes the OpenCode report (#2674). The Codex-side report of the
same root cause quotes the `SerenaConfigError` verbatim; OpenCode only
surfaces the generic `-32000`, which is why it read as two different
bugs.
- Users already broken by an affected version are repaired automatically
on their next `headroom wrap` — no manual `serena_config.yml` edit
needed.
- A stacked PR removing the rtk/lean-ctx CLI context tools is based on
this branch; this one is deliberately small so it can land first.
2026-07-30 20:55:47 -07:00
Tejas Chopra 5d23a0aec2 refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499)
## Description

`tokensave` was a **downloaded third-party Rust binary**
(`aovestdipaperino/tokensave`) that `headroom wrap` registered as a
code-graph MCP server. This removes it entirely and standardises on
**Serena** as the code-memory MCP — which was already the default in
`wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads
or executes a binary of its own for code memory.

The change is a *removal + safe transition*, not a behaviour flip:
Serena was already the default, so existing users move over
automatically. This PR also folds in a small README repositioning
(Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx
are third-party tools we don't control), since it's the same
tooling-stack story.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

- **Removed the external tool:** `headroom/graph/tokensave_installer.py`
(the download-and-execute path), `build_tokensave_spec`, and the
`_ensure_tokensave_binary` / `_index_tokensave_project` /
`_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`.
- **`--code-memory`** now offers `serena` (default) or `none` — the
`tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as
its (default-on) code-memory MCP.
- **Tombstone / transition (ledger-verified):** `headroom wrap` **and**
`headroom unwrap` remove a previously Headroom-installed `tokensave` MCP
entry so upgrading users stop launching it, and print that the leftover
`~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to
delete. A user-managed `tokensave` entry is left untouched. Mirrors the
existing `codebase-memory-mcp` retirement.
- **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and
`--no-tokensave` resolve to Serena instead of erroring; `--no-serena`
now means "no code memory". **No state migration needed** — both tools'
indexes are regenerable caches of the source, so Serena simply
re-indexes.
- **Docs/README:** replaced the "tokensave binary trust model" section
with a Serena note + an "Upgrading from tokensave?" callout; retired the
RTK "first-class part of our stack" framing.
- **Tests:** deleted the tokensave-only test files
(`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`,
`test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py`
for the new resolver/dispatch, fixed a codex test that patched a removed
symbol.

Net: **+102 / −1187 lines.**

## Testing

- [x] Unit tests pass (`pytest`) — targeted to the affected areas
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py
All checks passed!

$ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py
Success: no issues found in 12 source files

$ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \
         tests/test_cli/test_wrap_claude_vertex_proxy_env.py \
         tests/test_cli/test_wrap_claude_finally_unbound.py -q
======================== 120 passed in 97.86s (0:01:37) ========================

$ pytest tests/test_cli --collect-only -q
========================= 713 tests collected in 1.82s =========================   # no import errors after symbol removal
```

## Real Behavior Proof

- **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on
branch `tejas/remove-tokensave`.
- **Exact command / steps:**
- `python -c "from headroom.integrations.strands.bundle import
HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False,
enable_serena_mcp=False); print(len(b.tools))"` → confirms the module
imports after `build_tokensave_spec` removal (the import that my change
would otherwise break).
- CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`)
writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context",
"codex"`) to the codex config and **no** tokensave entry.
- `_resolve_code_memory` unit tests confirm: default → `serena`;
`HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`;
`--code-memory bogus` → `ClickException`.
- **Observed result:** import OK (`tools: 0`); Serena registered,
tokensave absent; resolver behaves as above; 120/120 tests pass.
- **Not tested:** a live `headroom wrap` against a real agent on a
machine with a *previously-installed* tokensave MCP entry — the
tombstone-removal path is covered by unit tests with a fake
registrar/ledger, not an end-to-end run.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title

## Additional Notes

- `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden,
deprecated flags (no-ops or mapped) so existing scripts don't break.
- `--code-graph` is unchanged — it's the proxy's live file-watcher flag
and was never the tokensave MCP; only the dead tokensave hook behind it
was removed.
- `CHANGELOG.md` intentionally left untouched.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-22 20:59:24 -07:00
Tejas Chopra 6e4425a6bd feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413)
## What
Two commits:
1. **Unify code-memory MCP selection behind `--code-memory
{tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the
`--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector.
Old flags remain as hidden deprecated aliases that map into it. Shared
across the code-memory-capable subcommands (claude/codex/grok).
2. **Default the engine to Serena**, with its **dashboard browser
suppressed**.

## Why Serena as default
Serena is a mature, offline, symbol-level code-navigation MCP with broad
language coverage (LSP-backed) — the strongest zero-account default for
reducing tokens by letting the agent query
symbols/definitions/references instead of reading whole files. It
attacks the *protected-reads* volume the proxy deliberately doesn't
compress, so it's complementary to the pipeline compressors.

## Browser suppression (in Serena's own settings)
`_ensure_serena_dashboard_disabled()` sets
`web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml`
when Serena is set up, so wrapped sessions don't spawn a browser tab.
The dashboard backend stays reachable manually at `localhost:24282`.
This lives in Serena's config (authoritative), not just a startup flag.

## Schema-overhead note
Serena injects tool schemas per request; that cost is deferred by the
tool-search deferral the coding profile already enables
(`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation
benefit without a standing schema tax on turns that don't navigate.

## Selection / escape hatches
`--code-memory serena` (default) · `tokensave` (lighter/faster) · `none`
(disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still
work.

## Testing
Updated the primary/backup policy test to the serena-primary default;
code-memory selector + serena disable/migrate tests pass. Local: 21
passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
2026-07-18 22:39:20 -07:00