Commit Graph
15 Commits
Author SHA1 Message Date
02c4e922f7 fix(cli): resolve active proxy port for read commands (#3522)
## Description

`headroom wrap copilot` can intentionally start a dedicated proxy on
port 8788
(or fall forward when 8787 is busy), but read-only CLI commands
previously
defaulted directly to 8787. As a result, `headroom dashboard` could open
a dead
or unrelated dashboard even though the wrapped session reported its
proxy on
8788.

This change extends the shared `headroom/cli/port_discovery.py` resolver
(introduced by #3855) to include live wrap-client markers, preferring
the
newest healthy session while preserving explicit `--port` and
`HEADROOM_PORT`
overrides.

Closes: N/A (no matching open issue found)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Extended the shared port discovery module from #3855 with live
wrap-client
marker validation (PID liveness and reuse checks), newest-session
ordering,
  and `/health` verification.
- Updated `dashboard`, `inspect`, and the runtime probe used by `learn`
to
  follow the newest healthy live wrap session after explicit and
  `HEADROOM_PORT` overrides.
- Kept `doctor`'s requested-port behavior; it now reports a healthy
proxy
  discovered from wrap-client markers elsewhere, with a `--port` hint.
- Added a warning when `dashboard` cannot reach the resolved port.
- Exported the actual proxy port as `HEADROOM_PORT` into every wrapped
agent
  environment.
- Reused #3855's port-discovery module rather than introducing a
parallel
  resolver.
- Added regression coverage for stale/recycled markers, multiple
sessions,
  explicit/environment overrides, dashboard output, doctor diagnostics,
  inspect/learn routing, and wrapped child environments.
- Documented automatic discovery and wrong-port troubleshooting.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_cli/test_proxy_port_discovery.py \
    tests/test_cli/test_dashboard_port_discovery.py \
    tests/test_cli/test_port_alignment.py \
    tests/test_cli/test_wrap_helpers.py \
    tests/test_cli_doctor.py tests/test_cli_inspect.py tests/test_cli_learn.py -q
267 passed in 5.49s

$ uv run ruff check .
All checks passed!

$ uv run ruff format --check .
1677 files already formatted

$ uv run mypy headroom/cli/port_discovery.py headroom/cli/proxy.py \
    headroom/cli/doctor.py headroom/cli/inspect.py headroom/cli/learn.py
Success: no issues found in 5 source files

$ uv run pre-commit run --all-files
Sync plugin versions.....................................................Passed
Verify Ruff version alignment............................................Passed
check for merge conflicts................................................Passed
ruff (legacy alias)......................................................Passed
ruff format..............................................................Passed
mypy.....................................................................Passed
```

After the first rebase, `make ci-precheck` completed `cargo fmt --check`
and
`cargo clippy --workspace -- -D warnings`, then stalled in the existing
Rust
Magika/detection tests (multiple tests exceeded 60 seconds). The same
stall
recurred during the push hook; it was stopped after confirming the
affected
suite. The final rebase onto #3883 was followed by the Python/CLI
regression
suite and all repository pre-commit gates above, but not another Rust
test run.

## Real Behavior Proof

- Environment: Ubuntu 22.04.5 LTS x86_64; Python 3.10.12; uv 0.12.10;
Headroom proxy 0.40.0-dev, started on isolated port 18788 with an
isolated live-client marker.
- Exact command / steps: Started an isolated process that wrote its live
PID marker and launched `headroom proxy --port 18788` under a temporary
workspace, with `HEADROOM_PORT` unset; then ran `env -u HEADROOM_PORT -u
HEADROOM_PORT_DISCOVERY
HEADROOM_WORKSPACE_DIR=/tmp/headroom-port-discovery-proof-20260930/workspace
./.venv/bin/headroom dashboard --no-open` and curled
`http://127.0.0.1:18788/dashboard`.

```text
$ env -u HEADROOM_PORT -u HEADROOM_PORT_DISCOVERY \
    HEADROOM_WORKSPACE_DIR=/tmp/headroom-port-discovery-proof-20260930/workspace \
    ./.venv/bin/headroom dashboard --no-open
  Detected live proxy on port 18788
  Dashboard: http://127.0.0.1:18788/dashboard

$ curl --max-time 5 -sS -o /dev/null -w '%{http_code}\n' \
    http://127.0.0.1:18788/dashboard
200

# Proxy health before discovery returned:
service=headroom-proxy status=healthy ready=true
```

- Observed result: With `HEADROOM_PORT` unset, the CLI selected the
marker-associated healthy proxy on 18788, printed the matching dashboard
URL, and the endpoint returned HTTP 200. The earlier manual run with a
real wrapped Copilot session also confirmed discovery on 8788 before
this rebase.
- Not tested: Starting a new interactive `wrap copilot` session after
this rebase (avoided launching an agent); Windows/macOS PID-identity
behavior; multiple simultaneous real proxies; browser UI launch. Unit
tests cover marker ordering, PID reuse, and explicit/environment
overrides.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: `dashboard`, `inspect`, and `learn`
now follow
the newest healthy live wrap proxy when no port override is set;
`doctor`
remains on the requested/default port and reports a live proxy
elsewhere.
- Kill switch / disable path: Pass `--port`, set `HEADROOM_PORT`, or set
  `HEADROOM_PORT_DISCOVERY=0` to bypass automatic discovery.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit; commands return to the fixed 8787
default.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from
  my Conventional Commit PR title

## Screenshots (if applicable)

N/A — terminal output above captures the behavior.

## Additional Notes

Port discovery is integrated into #3855's shared `port_discovery.py`;
candidates
are accepted only when `/health` identifies `service: headroom-proxy`,
so an
unrelated local JSON service cannot be mistaken for a Headroom proxy.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3522"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `e66a6393f83278aa07e9965f5b87fed820603677`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Re-reviewed latest integration head 420c2d636 and current author
response, including isolated workspace/port fixtures and live-marker
ordering/pruning regressions. Clean integration with upstream/main
2d3ee51846. Explicit CLI ports and
HEADROOM_PORT retain precedence; discovery bounds candidate probes,
verifies Headroom health identity, prefers newest live wrap markers, and
preserves configured-agent discovery. No new PR-specific correctness
blocker found in scope.

All 436
dashboard/port/discovery/wrap/doctor/inspect/learn/release/version/MCP
tests pass. All 14 changed Python files pass Ruff check/format;
whitespace passes. Previous cached-read freeze-boundary failure was
independently reproduced on main and is not a contributor-code blocker.
Maintainers handle fresh exact-head qualification and distinguish
inherited failures; no contributor CI execution requested. Existing
native extension supported startup, not a fresh native build. No
contributor discovery logic rewritten; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-05 21:10:06 -06:00
71663b44e2 fix(learn): pin the learned verbosity level on a cache-mode proxy (#4033)
## Description

`headroom learn --verbosity --apply` hot-enables the output shaper on a
running proxy and prints `level N is live now`. On a proxy in **cache
mode**, which is the default, that claim is false. The proxy never reads
`verbosity.json` there: `resolve_verbosity_level` returns the startup
level (`cache_mode_default`, L2) unless `HEADROOM_VERBOSITY_LEVEL` pins
one. A level that can change mid-conversation would bust the prefix
cache (#3773). So after `--apply` the shaper is on, but at L2 instead of
the learned level, while the CLI reports the learned level as live.

This PR makes `--apply` either do what it says or say what it did:

- **Cache mode, no pin:** the learned level is pinned through
`HEADROOM_VERBOSITY_LEVEL` in the same `/admin/runtime-env` update. That
is exactly the stable-pin path #3773 allows in cache mode.
- **An existing `HEADROOM_VERBOSITY_LEVEL`:** left alone, and the CLI
says which level stays in effect. The one exception is a pin this CLI
provably set. When the proxy confirms a pin (its `applied` map),
`--apply` records `{port, pid, started_at, level}` in
`verbosity_pin.json`. `started_at` is `timestamp - uptime_seconds` from
`/health`, because pids are reused. A later `--apply` replaces the pin
only while that same proxy process still holds that exact value. Hot
overrides die with the process, so a restarted proxy's pin, a reused
pid, or a value changed since stays the operator's.
- **Mode unreadable:** the shaper is still enabled, but the CLI no
longer claims the level is live.
- **Token mode:** unchanged. The proxy reads `verbosity.json` per
request, so no pin is added (a pin would also shadow the verbosity
autotune controller).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cli/learn.py`:
- New `_query_proxy_verbosity()` reads `config.mode` and
`config.runtime_env.HEADROOM_VERBOSITY_LEVEL` from the local proxy's
`/health`. The proxy includes that config block for loopback callers.
The level is read the way `OutputShaperSettings.from_env` reads it
(parse, default on garbage, clamp to 0–4), so `9` or `04` compare and
print as the level the proxy steers at.
- `_proxy_process()`, `_record_pin()` and `_pin_is_ours()` keep the
ownership record described above. Recording is best-effort: an `OSError`
no longer aborts `--apply`; the CLI reports the activation and notes the
pin wasn't recorded. The record is written with `newline="\n"`.
- `_activate_output_shaper()` returns `(status, port, applied)`, so a
pin the proxy applied while reporting the shaper "blocked" is still
recorded.
- `_activate_output_shaper()` takes an optional `verbosity_level` and
sends it in the same runtime-env request.
- The `--apply` messages distinguish the four cases above. The restart
hints now include `HEADROOM_VERBOSITY_LEVEL=<level>` for cache mode.
- The "learned level is used automatically" wording on the blocked and
no-proxy paths no longer claims cache mode reads the level.
- Those hints no longer prescribe `HEADROOM_ROLLOUT_CHANNEL=beta`:
`proxy_output_shaper` is available on every channel, and beta would also
enable unrelated features. "Blocked" now points at
`HEADROOM_DISABLE_FEATURES`, the only thing that refuses the hot enable.
- `tests/test_cli_learn.py`:
- CLI tests run `learn --verbosity --apply` against a stubbed local
proxy: cache-mode pin, token mode, an operator pin, an unreadable mode,
recording and then replacing this CLI's own pin, and keeping pins it
cannot prove it set. That last case covers an operator pin equal to the
previously learned level, a record from a restarted proxy, and a value
changed since. Also covered: out-of-range pins, re-pinning the same
level, and a proxy that disables the shaper. Each asserts both the
runtime-env body sent and the message printed.
- Further cases: a reused pid, a record without a start time, a pin
applied while the shaper is blocked, a failed record write, and a proxy
without a pid.
- Fallback branches: a non-numeric pin (read as the default level), a
`/health` reply without a config object, no usable proxy start time
(zero or missing uptime, missing or bad timestamp), and a record that
isn't a JSON object. Every new line and branch in
`headroom/cli/learn.py` is covered (`pytest --cov=headroom.cli.learn
--cov-branch`).
- A `windows_newline` test asserts that the ownership record is written
with `newline="\n"`.
- `tests/test_offline_egress_chokepoint.py`: `cli/learn.py` now has two
loopback egress sites (`/health` and `/admin/runtime-env`), recorded in
the allowlist with the same wording as `cli/wrap.py`.
- The existing `--all --apply` test made a real POST to whatever
listened on `127.0.0.1:8787`, which on a developer machine is their own
proxy. It now runs against a stubbed `urlopen`.
- `README.md`, `wiki/learn.md`: state that cache mode does not read the
learned level, and what `--apply` does there. `wiki/learn.md` no longer
says the shaper needs the beta channel.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Before: origin/main's headroom/cli/learn.py with the new tests
$ pytest tests/test_cli_learn.py -k verbosity_apply -q
FAILED tests/test_cli_learn.py::test_verbosity_apply_pins_the_level_on_a_cache_mode_proxy
FAILED tests/test_cli_learn.py::test_verbosity_apply_leaves_a_token_mode_proxy_reading_the_profile
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_override_an_explicit_level
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_claim_live_when_the_mode_is_unknown
4 failed, 22 deselected in 1.04s

# Review round 3: the previous head (e10141c3) with this round's tests
$ pytest tests/test_cli_learn.py -q
FAILED tests/test_cli_learn.py::test_activate_output_shaper_reports_effective_rollout_decision[True-live]
FAILED tests/test_cli_learn.py::test_activate_output_shaper_reports_effective_rollout_decision[False-blocked]
FAILED tests/test_cli_learn.py::test_activate_output_shaper_handles_malformed_response
FAILED tests/test_cli_learn.py::test_verbosity_apply_records_the_pin_it_sets
FAILED tests/test_cli_learn.py::test_verbosity_apply_replaces_the_pin_an_earlier_apply_set
FAILED tests/test_cli_learn.py::test_verbosity_apply_keeps_a_pin_it_cannot_prove_it_set[no-start-time]
FAILED tests/test_cli_learn.py::test_verbosity_pin_record_is_written_with_lf
FAILED tests/test_cli_learn.py::test_verbosity_apply_records_a_pin_the_proxy_applied_while_the_shaper_is_disabled
FAILED tests/test_cli_learn.py::test_verbosity_apply_reports_activation_when_the_pin_cannot_be_recorded
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_record_a_pin_without_a_process_identity
10 failed, 31 passed in 0.74s

# After: this branch
$ pytest tests/test_cli_learn.py -q
50 passed in 0.71s

# Offline-egress coverage (failed on e10141c3 in CI shard 3)
$ pytest tests/test_offline_egress_chokepoint.py -k test_every_egress_site_is_guarded_or_allowlisted -q
1 passed, 142 deselected in 2.63s

# Related suites (learn, verbosity, output shaper, runtime-env), with branch coverage of the changed file
$ pytest tests/test_*learn*.py tests/test_*verbosity*.py tests/test_*output_shaper*.py tests/test_*runtime_env*.py -q --cov=headroom.cli.learn --cov-branch
207 passed, 3 warnings in 7.83s
# lines/branches added by this PR in headroom/cli/learn.py that are not covered: none

$ ruff check headroom/cli/learn.py tests/test_cli_learn.py
All checks passed!
$ ruff format --check headroom/cli/learn.py tests/test_cli_learn.py
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS (Apple Silicon), Python 3.11. A real `headroom
proxy --port 8799` in default cache mode from this checkout, with an
isolated `HEADROOM_WORKSPACE_DIR` and `--anthropic-api-url` pointed at a
closed local port, so no request left the machine. `headroom learn
--verbosity --apply` ran over real Claude Code transcripts and learned
level 3. Before = `origin/main`'s `headroom/cli/learn.py`; after = this
branch. A fresh proxy was used for each run.
- Exact command / steps: Start the proxy. Run `HEADROOM_PORT=8799
headroom learn --verbosity --apply --project <repo>`. Send one
`/v1/messages` request through the proxy. Read `/health`
`config.runtime_env` and the proxy's `OutputShaper(...)` log line.
- Observed result: before, the CLI says level 3 is live while the proxy
steers at L2. After, the level is pinned and the proxy steers at L3:

  ```text
  ===== BEFORE (origin/main learn.py), proxy in default cache mode
  >> Recommended verbosity level: 3 (confidence: high)
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (while HEADROOM_VERBOSITY_LEVEL is unset).
To keep it on across restarts: export HEADROOM_ROLLOUT_CHANNEL=beta and
HEADROOM_OUTPUT_SHAPER=1 before `headroom wrap ...`.
proxy /health: {"mode": "cache", "HEADROOM_OUTPUT_SHAPER": "1",
"HEADROOM_VERBOSITY_LEVEL": null}
  proxy log: OutputShaper(L2/cache_mode_default)

  ===== AFTER (this branch), proxy in default cache mode
  >> Recommended verbosity level: 3 (confidence: high)
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (pinned with HEADROOM_VERBOSITY_LEVEL: cache mode does not read
the learned level).
Conversations already open re-cache their prompt once, because the
steering text changed.
To keep it on across restarts: export HEADROOM_OUTPUT_SHAPER=1
HEADROOM_VERBOSITY_LEVEL=3 before `headroom wrap ...` or `headroom
proxy`.
  --- verbosity_pin.json: {"port": 8799, "pid": 25345, "level": 3}
  --- second --apply against the same proxy:
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (pinned with HEADROOM_VERBOSITY_LEVEL: cache mode does not read
the learned level).
proxy /health: {"mode": "cache", "HEADROOM_OUTPUT_SHAPER": "1",
"HEADROOM_VERBOSITY_LEVEL": "3"}
  proxy log: OutputShaper(L3/env_pinned)
  ```
- Not tested: a real token-mode proxy, one with an operator-set
`HEADROOM_VERBOSITY_LEVEL`, and a second `--apply` that learns a
*different* level (all covered by the CLI tests against a stubbed proxy;
the real run above re-learns the same level 3); multi-worker proxies,
which reject `/admin/runtime-env` before and after this change; Windows
(the record write is covered by a `windows_newline` test).

## Runtime Rollout Safety

- Rollout-managed feature(s): `proxy_output_shaper` (unchanged). This PR
only changes what the `headroom learn` CLI sends to and prints about a
running proxy.
- Minimum rollout channel: stable.
- Stable/default behavior changed: only `learn --verbosity --apply`
against a running cache-mode proxy with no `HEADROOM_VERBOSITY_LEVEL`.
It now also pins that variable to the learned level, which is what the
CLI already claimed. The proxy itself is unchanged.
- Kill switch / disable path: run `learn --verbosity` without `--apply`;
or POST `{"HEADROOM_VERBOSITY_LEVEL": ""}` / restart the proxy to drop
the hot override.
- Unsafe override required: no.
- Qualification impact: none expected.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Type checking was not run locally.
- Pin ownership: `/health` cannot say whether `HEADROOM_VERBOSITY_LEVEL`
came from startup env or a hot override, so `--apply` keeps its own
record. The process start time is derived from `/health` rather than
adding an instance id to the proxy, which keeps this PR CLI-only. The
one remaining ambiguity is an operator hot-POSTing the *same* value to
the *same* proxy process after `--apply` pinned it. That pin would be
treated as ours, but its value is the one `--apply` set anyway.
- Locally, 5 OTLP/Langfuse tests in `test_offline_egress_chokepoint.py`
fail on both `origin/main` and this branch because the dev venv lacks
`opentelemetry-exporter`; they pass in CI.


## Maintainer follow-up at 8c3eba8919

Ownership now compares the exact raw `HEADROOM_VERBOSITY_LEVEL` value
from the same health snapshot as process identity. A changed operator
value is preserved even when it clamps to the same effective level as a
previous CLI pin (for example, CLI4 followed by operator8). The display
still reports the effective clamped level. The new regression fails on
the preceding head and passes here; existing CLI ownership/restart/error
behavior remains covered.

Fresh validation:140 CLI/verbosity/output-shaper/runtime-env tests pass
with one existing warning; Ruff lint/format and Linux-platform mypy
over566files pass. Actual proxy/CLI proof additionally covers learning3,
re-learning1, preserving an operator8 pin, and the
changed-raw/same-clamped-level ownership case. The last case fails
before this follow-up and passes afterward.

Exact commands and full runnable Windows/Python3.12 script: [maintainer
evidence](https://github.com/headroomlabs-ai/headroom/pull/4033#issuecomment-6063120923).
Save the script as `verify_learn_pin.py` in the checkout root and run
`python verify_learn_pin.py` with proxy dependencies installed.
Synthetic Claude transcripts are processed by the real analyzer; no live
LLM judge or provider request is made. Hosted CI and fresh review must
qualify this new head before approval.

---------

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-08 10:48:59 -05:00
3fdf18eded fix(learn): merge git worktree sessions into the repo's project (#3854)
## Description

`headroom learn` treats every git worktree as its own project. Claude
Code stores each working directory's sessions in a separate
`~/.claude/projects/<encoded-cwd>/` folder, and
`ClaudeCodePlugin.discover_projects` turns each folder into one
`ProjectInfo`. So a linked worktree (a Conductor workspace,
`.claude/worktrees/*`, any `git worktree add` checkout) shows up as a
separate project with a few sessions:

- Each worktree rarely has enough sessions to cross a pattern threshold,
so the repo's learnings never add up.
- Learnings are written into the worktree: `CLAUDE.local.md` in a
checkout that is deleted when the workspace is archived, and `MEMORY.md`
in that worktree's own memory folder.
- For a repo worked on only through worktrees (the usual Conductor
setup), `headroom learn --project <repo>` finds nothing: `No matching
project. Try --all or --project <path>.`
- The traffic learner uses the same roots, so live patterns that mention
a worktree path go to the worktree too.

A user reported this: their project list was a row of Conductor
workspace names, each "not scanned yet".

This PR merges each linked worktree into the project of its main
checkout.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `learn/plugins/claude.py`: new `_main_worktree_root(path)` resolves a
linked worktree to its main checkout by reading git's own pointer files:
the `.git` file's `gitdir:` line, then that gitdir's `commondir`. It
does not run git. It returns None for a main checkout, a submodule (its
gitdir has no `commondir`), a bare repo, a relative path, or anything
outside git.
- `discover_projects` now ends in `_merge_worktrees`, which groups
projects by that root. The merged project:
  - keeps the main checkout's `project_path` and session folder;
  - writes to the main checkout's `CLAUDE.local.md`;
- writes `MEMORY.md` to the main checkout's memory folder, the same
folder Claude Code's own auto-memory uses for every worktree of a repo.
If only worktrees have sessions, the merged project uses the folder
Claude Code would give the main checkout. Projects outside worktrees are
unchanged.
- `ProjectInfo` gains `worktree_paths` and `extra_data_paths` (both
default to empty lists). `scan_project` and the `--verbosity` path read
sessions from `data_path` plus `extra_data_paths`.
- `memory/traffic_learner.py`: `_project_for_pattern` also matches paths
under a project's `worktree_paths` and routes those patterns to the
project. Longest match still wins.
- A worktree whose checkout has been deleted has no `.git` file left to
resolve, so it stays a separate project, as before.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

`mypy` passes on the four changed modules. `mypy headroom` reports one
existing error in `headroom/integrations/langchain/chat_model.py:394`
(`truthy-function`), a file this PR does not touch.

### Test Output

```text
$ uv run --extra dev pytest tests/test_learn/ tests/test_memory/ tests/test_cli_learn.py tests/test_learn_grok_plugin.py tests/test_verbosity_learn.py -q
966 passed, 4 skipped, 79 warnings in 44.31s

$ uv run --extra dev ruff check . && uv run --extra dev ruff format --check .
All checks passed!
1671 files already formatted

$ uv run --extra dev mypy headroom/learn/plugins/claude.py headroom/learn/models.py headroom/memory/traffic_learner.py headroom/cli/learn.py
Success: no issues found in 4 source files
```

With the source reverted to `main`,
`tests/test_learn/test_worktree_merge.py` fails (it imports
`_main_worktree_root`).

## Real Behavior Proof

- Environment: macOS 15 (Darwin 24.6), Python 3.12, git 2.x; a scratch
repo plus a real `git worktree add` checkout; a scratch
`CLAUDE_CONFIG_DIR` with three Claude Code session files stored under
the worktree's encoded folder name.
- Exact command / steps: `git init repo && git -C repo commit
--allow-empty -m init && git -C repo worktree add ../ws/san-salvador -b
feat`, then `CLAUDE_CONFIG_DIR=<scratch>/claude headroom learn
--verbosity --agent claude --project <scratch>/repo`, run once on `main`
and once on this branch.
- Observed result: on `main`: `No matching project. Try --all or
--project <path>.` On this branch: `Verbosity — repo`, `Path:
<scratch>/repo`, `Sessions: 3`. The real git layout matches the test
fixtures: the worktree's `.git` is `gitdir:
<repo>/.git/worktrees/san-salvador`, and `commondir` is `../..`.
- Not tested: Windows worktrees on a real machine (the resolver uses
only `pathlib`); the full LLM analysis path of `headroom learn --apply`
(it shares `discover_projects`/`scan_project` with the verbosity path
shown above).

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes, for users with git worktrees
only: `headroom learn` and the traffic learner now treat a linked
worktree as part of its main checkout's project instead of as a separate
project. Repos without worktrees are unchanged.
- Kill switch / disable path: Not applicable.
- Unsafe override required: No.
- Qualification impact: None. Learn and the memory writer only; the
proxy request path and compression are untouched.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

- No documentation change: the learn docs don't describe how projects
are discovered.
- Open question for maintainers: Conductor-style workspaces are fresh
checkouts, so they never see the main checkout's `CLAUDE.local.md`. Only
`MEMORY.md` is shared across a repo's worktrees. For a repo worked on
only through worktrees, CONTEXT_FILE rules may be better routed to
`MEMORY.md`. I've left that out of this PR to keep it to one behavior
change.
- Follow-up idea, also not included: a worktree whose checkout has been
deleted could be attributed through its session folder name. Today it
stays a separate project, as before.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3854"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:25:53 -05:00
TrîggeredandJD Davis 4ba231ab3d feat(learn): add agy (Antigravity CLI) as an analysis backend (#3939)
## Description

Adds the Antigravity CLI (`agy`) as a keyless LLM backend for `headroom
learn`, next to `claude`, `gemini` and `codex`. Antigravity subscription
users with no raw API key can now run the analysis through `agy`.

`agy` cannot use the existing stdin path as is. `agy -p` takes the
prompt as a flag argument and never reads stdin, so a large digest would
hit OS argument-length limits. The only stdin path is `--input-format
stream-json`, which requires `--output-format stream-json`. The backend
therefore sends the prompt as a single NDJSON user event on stdin and
reads the answer from `result.response` in the final `result` event.
Everything else (wall-clock timeout, Windows shim fallback, failure
detail, fenced-JSON parsing) reuses the existing `run()` path that
`gemini` and `codex` use.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/learn/analyzer.py`: add an `agy` / `agy-cli` entry to
`_CLI_BACKENDS` (`agy --input-format stream-json --output-format
stream-json`). `agy` is never auto-detected, and it only runs when
`HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` is set: `agy` has no zero-tool mode,
so transcript text could drive its tools. Without the opt-in,
`HEADROOM_LEARN_CLI=agy` and `--model agy-cli` fail before any
subprocess starts; with it, a warning is logged on every run.
- `headroom/learn/analyzer.py`: in `_call_cli_llm`, wrap the prompt as
`{"event": "user", "message": {"content": ...}}` for `agy-cli`, and
extract `result.response` from the stream with a new
`_agy_result_text()` helper before the existing `_strip_fenced_json()`
parsing. A missing `result` event surfaces as the existing "unparseable
output" error, with both ends of stdout.
- `tests/test_learn/test_analyzer.py`: tests that auto-detection never
selects `agy` (even when it is the only CLI on PATH and the opt-in is
set), that `HEADROOM_LEARN_CLI=agy` and `_call_cli_llm(..., "agy-cli")`
fail without the opt-in (no subprocess call), `HEADROOM_LEARN_CLI=agy`
with the opt-in, existing CLIs still taking precedence, the exact
command and stdin payload, result-event parsing (including a fenced
response and a literal U+2028), and a missing `result` event.
- `headroom/cli/learn.py`: check the agy opt-in during model resolution,
so `HEADROOM_LEARN_CLI=agy` and `--model agy-cli` without it exit 1 with
an `Error:` line before any sessions are scanned.
- `tests/test_cli_learn.py`: a CLI test for both paths without the
opt-in.
- `wiki/learn.md`: list `agy` among valid `HEADROOM_LEARN_CLI` values
and document the `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` opt-in and its
risk.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --frozen --extra dev pytest tests/test_learn/ tests/test_cli_learn.py -q
5 failed, 345 passed, 4 skipped in 10.78s
# The same 5 fail on unmodified upstream/main on this Windows machine (5 failed, 340 passed),
# all path/git related and unrelated to this change:
#   test_analyzer.py::TestDigestBuilder::test_includes_project_info
#   test_scanner.py::TestDecodeProjectPath::test_home_dir_username_stays_single_component
#   test_writer.py::TestClaudeLocalMdStaysOutOfGit::{test_apply_adds_exclude_entry,
#     test_second_run_does_not_duplicate_the_entry, test_tracked_file_warns_instead_of_excluding}

$ uv run --frozen --extra dev pytest tests/test_learn/test_analyzer.py -q -k agy   # analyzer.py reverted
FAILED TestDetectDefaultModel::test_env_var_selects_agy_with_unsafe_opt_in
FAILED TestDetectDefaultModel::test_env_var_agy_without_unsafe_opt_in_raises
FAILED TestCallCliLlm::test_agy_cli_sends_stream_json_and_parses_result_event
FAILED TestCallCliLlm::test_agy_cli_result_with_unicode_line_separator
FAILED TestCallCliLlm::test_agy_cli_without_unsafe_opt_in_does_not_run
FAILED TestCallCliLlm::test_agy_cli_missing_result_event_raises
6 failed, 2 passed, 97 deselected
# All 8 pass with the change.

$ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
All checks passed!
$ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/learn/analyzer.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, `agy` 1.2.14 signed in via subscription, no
API keys set.
- Exact command / steps: `HEADROOM_LEARN_CLI=agy
HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1 ANTHROPIC_API_KEY= OPENAI_API_KEY=
GEMINI_API_KEY= uv run --frozen --extra dev headroom learn` in this
repository (dry run).
- Observed result: `Analyzing with agy-cli...`, then `Sessions: 3 |
Calls: 109 | Failures: 19 (17.4%)`, `Recommendations: 8`, with `[WOULD
WRITE]` blocks for `CLAUDE.local.md` and `MEMORY.md`; exit 0. Before
wiring the backend I confirmed the protocol by hand: `agy -p -` answers
the literal prompt `-` and ignores stdin, while the stream-json
invocation returns the answer in `{"event": "result", "result":
{"status": "SUCCESS", "response": "..."}}`.
- Not tested: macOS/Linux `agy` installs; `agy` under a permission mode
that prompts for tool approval (my config is `always-proceed`), where a
tool request in print mode would be bounded by the existing
`HEADROOM_LEARN_CLI_TIMEOUT_SECS` wall-clock cap.

## Runtime Rollout Safety

- Rollout-managed feature(s): None (offline `headroom learn` CLI)
- Minimum rollout channel: dev
- Stable/default behavior changed: No. `agy` is never auto-detected; it
is only used when selected with `HEADROOM_LEARN_CLI=agy` / `--model
agy-cli` together with `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1`.
- Kill switch / disable path: Set an API key, `HEADROOM_LEARN_CLI`, or
`--model` to pick another backend.
- Unsafe override required: None
- Qualification impact: None
- Rollback path: Revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A (CLI change).

## Additional Notes

- No tracking issue exists for an `agy` analysis backend, so `Closes #`
is left empty. Related Antigravity work (#3428 `headroom init -g agy`,
#3610 Antigravity transcripts in `headroom learn`) is independent of
this change.
- Unlike `claude-cli` (#3892), `agy` has no flag to start a session with
zero tools. I deliberately did not add `--dangerously-skip-permissions`:
the digest contains untrusted transcript content, and the backend should
run under the user's own permission settings.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3939"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-04 13:56:26 -07:00
4c6bd3e8fc fix(learn): echo live progress during claude-cli analysis (#3158)
## Description

`headroom learn` already streams claude-cli's `stream-json` output
internally, using it to drive an idle-timeout watchdog, but none of that
liveness reaches stdout. Wrapper UIs that tail the CLI output (the
Headroom desktop app included) see nothing print between "Analyzing with
claude-cli..." and the final result, sometimes for minutes, which makes
the scan look hung. This closes #3105 by surfacing that liveness as
short, throttled progress lines.

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)

## Changes Made

- Added an optional `on_progress: typing.Callable[[str], None] | None =
None` callback, threaded through `SessionAnalyzer.analyze` ->
`_call_llm` -> `_call_cli_llm` -> `_call_claude_cli_streaming`. It
defaults to `None` at every layer, so no existing caller changes
behavior.
- Inside the streaming event loop, added `_progress_detail()`, which
maps each non-terminal `stream-json` event
(`system`/`assistant`/`stream_event`/`user`) to a short phrase like
`"assistant responding, 12s"`. The terminal `result` event is handled
separately and is never echoed as progress.
- Throttled the callback to at most once per 3 seconds
(`_PROGRESS_THROTTLE_SECS`) via the same `time.monotonic()` idiom the
function already used for its idle-timeout watchdog, so a burst of
partial-message events doesn't spam the console.
- The callback is invoked defensively: any exception it raises is caught
and logged via `logger.debug`, never propagated, so a broken UI pipe
can't abort an otherwise-successful analysis.
- `headroom/cli/learn.py` supplies the one concrete callback, reusing
the exact existing `" Analyzing with {model}..."` prefix with `(detail)`
appended, so any wrapper UI that whitelists that prefix keeps working
unchanged.
- Added `--include-partial-messages` to the claude-cli invocation, as
suggested in the issue. Without it, claude only emits about three events
for an entire run (session init, one assistant message, result), which
is too sparse for the throttle to produce useful incremental updates.

## Testing

- [x] Added new tests
- [x] All tests pass locally

```
$ .venv/Scripts/python.exe -m pytest tests/test_learn/test_analyzer.py tests/test_cli_learn.py -q
tests\test_learn\test_analyzer.py .......................................
......................................................                  [ 84%]
tests\test_cli_learn.py .................                                [100%]
109 items collected, 108 passed, 1 failed

FAILED tests/test_learn/test_analyzer.py::TestDigestBuilder::test_includes_project_info
  (pre-existing, unrelated: a Windows-path-separator assertion that also
  fails identically on a clean upstream/main checkout, before this change)

$ .venv/Scripts/ruff.exe check headroom/learn/analyzer.py headroom/cli/learn.py tests/test_learn/test_analyzer.py tests/test_cli_learn.py
All checks passed!

$ .venv/Scripts/ruff.exe format --check headroom/learn/analyzer.py headroom/cli/learn.py tests/test_learn/test_analyzer.py tests/test_cli_learn.py
4 files already formatted

$ .venv/Scripts/mypy.exe headroom/learn/analyzer.py headroom/cli/learn.py
Success: no issues found in 2 source files
```

Three new tests were added covering: a burst of events within one
throttle window only echoing the first one, spaced-out events each
mapping to the correct phrase in order (with the terminal `result` event
never echoed), and a raising `on_progress` callback not aborting a
successful analysis. A fourth new test at the CLI layer asserts the `"
Analyzing with ... (detail)"` lines appear in `headroom learn` output.

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, local `.venv`, no live
`claude` CLI invoked (tests mock `subprocess.Popen`, matching the
existing test style in this file)
- Exact command / steps: `.venv/Scripts/python.exe -m pytest
tests/test_learn/test_analyzer.py tests/test_cli_learn.py -v -k
progress`
- Observed result: all 4 new progress-related tests pass, confirming the
throttling, event-to-phrase mapping, defensive-callback behavior, and
the CLI's rendered output line all work as designed
- Not tested: an actual live `claude` CLI process (no `claude` binary
available in this sandboxed environment); the fix only changes what data
is threaded through an existing, already-live streaming loop, so this is
low risk, but a manual check against the real CLI before/after would be
good follow-up validation

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no — `on_progress` defaults to `None`
everywhere, and when `None` the code path is unchanged from before this
PR
- Kill switch / disable path: none needed; the only new default-on
behavior is the extra progress lines in `headroom learn` CLI output,
which is additive to existing output (same prefix, one line appended per
event) and does not change any parsed/machine-readable output
- Unsafe override required: no
- Qualification impact: none — no changes to routing, caching, or
backend selection logic
- Rollback path: revert this commit; nothing persists state or changes
on-disk formats

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
2026-09-09 22:00:36 -05:00
JD Davis 632cb81dbe fix(learn): surface Codex analysis failures (#3016)
## Description

`headroom learn` could invoke Codex CLI from a non-Git working directory
without Codex’s required bypass flag. The resulting backend error was
then swallowed by the analyzer and rendered as “No actionable patterns
found” with exit code 0. This fixes both coupled defects so Codex can
run from discovered project locations and genuine analysis failures
remain visible and machine-detectable.

Closes #3008

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `--skip-git-repo-check` to the Codex CLI analysis backend
command.
- Added an explicit `analysis_error` result field instead of conflating
backend failure with an empty recommendation set.
- Kept multi-project analysis best-effort, while returning exit code 1
after any project analysis fails.
- Prevented failed analysis from printing a misleading no-pattern
success message.
- Added analyzer and CLI regression coverage for the command and
failure-propagation contracts.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run pytest -q tests/test_learn/test_analyzer.py tests/test_cli_learn.py
102 passed in 2.34s

uv run pytest -q tests/test_learn tests/test_cli_learn.py
257 passed, 7 skipped in 3.11s

uv run mypy headroom
Success: no issues found in 520 source files

uv run ruff check <changed files>
All checks passed!
uv run ruff format --check <changed files>
5 files already formatted

uv run pytest tests scripts/tests --splits 4 --group N --tb=short -q
shard 1: 2766 passed, 140 skipped in 174.08s
shard 2: 2699 passed, 207 skipped in 60.00s
shard 3: 2822 passed, 84 skipped in 76.10s
shard 4: 2734 passed, 172 skipped in 80.29s
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.13, Codex CLI 0.147.0-compatible
command surface, current `main` including #2996.
- Exact command / steps: verified `codex exec --help`; exercised
`_call_cli_llm` with a captured subprocess command; invoked the Click
command with a simulated Codex nonzero backend result.
- Observed result: the subprocess command is `codex exec
--skip-git-repo-check`; backend failure text is printed as `Analysis
failed`, the misleading no-pattern message is absent, and the CLI exits
1.
- Not tested: live paid Codex analysis against production account
credentials; subprocess and CLI behavior are covered deterministically.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; this is CLI-only failure handling.
- Minimum rollout channel: normal patch release after exact-head CI is
entirely green.
- Stable/default behavior changed: failed LLM analysis now exits nonzero
instead of reporting success; successful and genuinely empty analyses
are unchanged.
- Kill switch / disable path: select another backend with
`HEADROOM_LEARN_CLI` or `--model` if Codex CLI is unavailable.
- Unsafe override required: none.
- Qualification impact: all four Python CI shards, static checks,
security checks, and command-level regression tests must pass.
- Rollback path: fix forward through a human-reviewed corrective PR; no
persisted data or migration is involved.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project’s style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation — inline
result-contract documentation; no separate user guide change is required
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable; command-line backend and exit semantics only.

## Additional Notes

Human review only. No merge or auto-merge is configured. This corrects
the root failure and exit semantics without extending any timeout.
2026-08-25 21:40:12 -05:00
3077ac81e8 feat: add deterministic runtime rollout controls (#1490)
## Description

Establish one centrally resolved, observable, deterministic, versioned
runtime rollout-control mechanism for Headroom. Runtime rollout controls
which behaviors an already-built artifact may expose; it does not select
or qualify a Headroom release/version.

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Bug fix (non-breaking change that fixes rollout enforcement
regressions)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

- Added `RolloutChannel`, `HEADROOM_ROLLOUT_CHANNEL`,
`--rollout-channel`, and a versioned immutable `RolloutSnapshot` shared
by Python configuration boundaries.
- Added schema/policy versions, canonical registry and snapshot SHA-256
identities, per-feature decision reasons, disable precedence, unsafe
qualification poisoning, strict CLI validation, and fail-closed
environment handling.
- Added `headroom rollout status --json`, Python `/stats.rollout`, and
Rust `/rollout/status` runtime provenance.
- Added equivalent Rust snapshot semantics and shared Python/Rust policy
vectors while retaining language-specific feature registries.
- Enforced rollout policy at alternate Python server composition roots
so `HEADROOM_READ_MATURATION=1` cannot bypass its beta gate.
- Preserved typed rollout snapshots across multi-worker serialization
with schema, policy, registry, snapshot-digest, type, and feature-name
validation.
- Made loopback runtime output-shaper updates replace the immutable
snapshot atomically for request readers, retain explicit request/disable
provenance, preserve channel and kill-switch precedence, invalidate
cached stats, and return the effective rollout decision.
- Made `headroom learn --verbosity --apply` report a channel-blocked
update instead of claiming the shaper is live.
- Made explicit CLI feature flags fail loudly when their current channel
blocks them.
- Made persistent interceptor installation select canary automatically,
or reject an explicitly insufficient channel unless the break-glass
override is set.
- Updated architecture, proxy, rollout, learn, and output-shaper
documentation with required channels and hot-reload semantics.

## Testing

- [x] Unit tests pass
- [x] Linting passes (`ruff check .` and `ruff format --check .`)
- [x] Type checking passes (`mypy headroom --ignore-missing-imports`)
- [x] New regression tests added for every corrected behavior
- [x] Rust tests and production-target Clippy pass
- [x] Documentation build passes

### Test Output

```text
Focused rollout coverage suite
57 passed; headroom.rollout + rollout CLI: 98% coverage

Affected proxy/rollout/transform/governance suites
222 passed; 0 failed

Final changed regression suites
100 passed; 0 failed

Cross-module hot-reload isolation regression
6 passed; 0 failed

cargo test -p headroom-core -p headroom-proxy --quiet
headroom-core: 924 passed; 1 ignored
headroom-proxy and integration suites: all passed

cargo clippy -p headroom-core -p headroom-proxy --lib --bins -- -D warnings
cargo fmt --all -- --check
ruff check .
ruff format --check .
mypy headroom --ignore-missing-imports
git diff --check
All passed

cd docs && npm run build
Compiled successfully; 164 static pages generated
```

The unsharded Windows-only CI selection exposed unrelated baseline
failures, principally the existing `sqlite:///C:\\...` URL parser
producing an invalid `\\C:\\...` path. At commit `8e793a80`, all 52
completed GitHub checks passed; the only other conclusions are expected
skips and superseded governance jobs.

## Real Behavior Proof

- **Environment:** Windows checkout on Python 3.13.3 and the current
Rust workspace, based on upstream `main` at `93f2d7a2`.
- **Exact command / steps:** Exercised canary and beta feature requests
through CLI status, Python `/stats.rollout`, Rust `/rollout/status`,
multi-worker payload round trips, loopback `/admin/runtime-env`, real
proxy request shaping before/after hot reload, installer manifest
generation, and shared Python/Rust policy vectors.
- **Observed result:** Stable blocks unstable requests; disable wins
over explicit/default/legacy/unsafe paths; unsafe state reports
`qualification_eligible=false`; worker handoff rejects tampering;
running output shaping changes only when the effective beta policy
permits it; explicit blocked flags fail with actionable diagnostics.
- **Not tested:** Live production traffic requiring provider
credentials, or future artifact qualification/promotion automation
(intentionally out of scope).

## Runtime Rollout Safety

- **Rollout-managed features:** Python `tool_result_interceptors`,
`proxy_output_shaper`, `read_maturation`; Rust `native_bedrock`,
`openai_responses_streaming`, `canary_probe`.
- **Minimum rollout channel:** Registry-defined per feature; process
default is `stable`.
- **Stable/default behavior changed:** No unstable feature becomes
enabled by default. Explicit blocked CLI flags now fail instead of
silently doing nothing.
- **Kill switch / disable path:**
`HEADROOM_DISABLE_FEATURES=<comma-separated feature names>`; explicit
disable has highest precedence, including over the unsafe override.
- **Unsafe override required:** No.
`HEADROOM_UNSAFE_ALLOW_UNSTABLE_FEATURES=1` is break-glass only and
makes qualification evidence ineligible.
- **Qualification impact:** Adds machine-readable policy/snapshot
identities and eligibility; does not implement qualification itself.
- **Rollback path:** Set the named disable list for operational
rollback, lower the channel, or revert this PR.

## Review Readiness

- [x] I have performed a full diff review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have made corresponding documentation changes
- [x] My changes generate no new warnings
- [x] I added tests that reproduce and prevent every regression fixed
during review
- [x] New and existing affected tests pass locally
- [x] I did **not** edit `CHANGELOG.md`; release-please generates it
from the Conventional Commit PR title

## Additional Notes

Out of scope: artifact candidates, benchmark orchestration,
qualification manifests/gates, promotion automation, release branches,
publication guards, and release-risk classification. Those workflows can
consume the rollout registry digest, runtime snapshot digest, decision
reasons, and qualification eligibility through supported black-box
interfaces.

---------

Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Co-authored-by: JD Davis <jd@JDH-AIR-00.local>
2026-08-12 23:16:54 -05:00
27a5468349 fix(learn): aggregate verbosity baselines across projects instead of overwriting (#1288)
## Description

`headroom learn --verbosity --apply --all` was building the
output-shaper's savings baseline from only **one** project.
`_run_verbosity` wrote the savings ledger *inside* the per-project loop
(`ledger.baseline = baseline; ledger.save(...)`), so each project
replaced the previous baseline and only the last project processed
survived — frequently a near-empty one. The synthetic-control estimate
that `/stats` exposes (`savings.by_layer.output_shaping`) was then
computed against a tiny, unrepresentative sample.

This PR makes `--all` aggregate across every targeted project and write
the ledger **once**, after the loop.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `BaselineModel.merge()` / `_Accum.merge()`
(`headroom/proxy/output_savings.py`): fold one baseline into another.
The accumulators hold additive online stats (`n` / `sum` / `sumsq`), so
merging is element-wise and order-independent — identical to having
observed both corpora against a single model.
- `_run_verbosity` (`headroom/cli/learn.py`): accumulate a single
`BaselineModel` across all targeted projects and persist it once after
the loop, instead of overwriting per project. The applied verbosity
level now comes from the project with the most samples (strongest
signal) rather than whichever sorted last. Single-project runs are
unchanged (an aggregate of one). When no transcripts are found, it
prints a clear message and writes nothing.
- Tests: unit test for `BaselineModel.merge`; CLI test that `--all
--apply` across two projects aggregates both strata (totals summed, not
last-wins) and applies the busier project's level.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_output_savings.py tests/test_cli_learn.py tests/test_verbosity_learn.py -q
tests/test_output_savings.py ...............................             [ 54%]
tests/test_cli_learn.py ...........                                      [ 73%]
tests/test_verbosity_learn.py ...............                            [100%]
============================== 57 passed in 0.51s ==============================

$ uv run ruff check headroom/cli/learn.py headroom/proxy/output_savings.py
All checks passed!

$ uv run mypy headroom/cli/learn.py headroom/proxy/output_savings.py
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12.13, this branch off `upstream/main`.
- Exact command / steps: `headroom learn --verbosity --apply --all` (run
across a multi-project transcript corpus), then inspect
`~/.headroom/output_savings.json` (`baseline.glob.n`); compared against
`headroom learn --verbosity --apply` for a single busy project.
- Symptom (pre-fix, installed build): `headroom learn --verbosity
--apply --all` across a multi-project transcript corpus wrote
`~/.headroom/output_savings.json` with `baseline.glob.n = 2` (the last
project processed was a near-empty `…/venv/bin` dir), while targeting a
single busy project gave `baseline.glob.n = 15658`.
- With this change: the new CLI test
(`test_verbosity_all_apply_aggregates_baselines_across_projects`) drives
`--all --apply` over two projects (3 samples + 1 sample) and asserts the
persisted ledger has `total_samples == 4` with both strata present, plus
the busier project's level applied.
- Observed result: aggregated baseline persisted once; both strata
retained; level taken from the higher-sample project.
- Not tested: re-running the patched `--all` end-to-end on a live
multi-project machine (covered instead by the unit merge-math test and
the faked-`analyze` CLI test).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A — CLI/behavioral change.

## Additional Notes

- No linked issue (`Closes #` left blank intentionally).
- Documentation checklist item is N/A — no user-facing docs describe the
per-project overwrite behavior.
- Level-selection note: for `--all`, the applied verbosity level is now
deterministic (most-samples project) instead of last-processed; this is
the intended improvement, not a behavior to preserve.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-06-30 08:37:37 -05:00
Focused Instability ced75e4718 feat(learn): write per-project learnings to CLAUDE.local.md by default (#1115)
## Description

`headroom learn` wrote per-project learnings into the project's
`CLAUDE.md`, which Claude Code treats as team-shared and git-tracked.
That meant machine-specific absolute paths and tool-discovery byproducts
polluted the shared file for every teammate. This switches the default
to the personal, gitignored `CLAUDE.local.md`, adds a `--target`
override, and migrates any stale block out of `CLAUDE.md`.

Closes #1072.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `ClaudeCodeWriter` now writes CONTEXT_FILE recommendations to
`CLAUDE.local.md` by default instead of `CLAUDE.md` (the home-directory
case still uses `~/.claude/CLAUDE.md`, which is personal global memory).
- Added a `--target` flag (Claude Code only) and `set_context_target()`
to override the destination — e.g. `--target CLAUDE.md` to opt back into
the shared file, or any relative/absolute path.
- On first run after upgrade, a stale Headroom block left in `CLAUDE.md`
is moved into `CLAUDE.local.md` and stripped from `CLAUDE.md`, with a
warning surfaced by the CLI. If `CLAUDE.md` held nothing but the block,
the empty file is removed.
- `WriteResult` carries `warnings`; the `learn` CLI prints them.
- Updated docs (`failure-learning.mdx`) and `CHANGELOG.md`.

This implements the maintainer's stated preference order from the issue
(default → `CLAUDE.local.md`, plus a `--target` flag), scoped to the
Claude writer only — `AGENTS.md`/`GEMINI.md` have no `.local` convention
and are untouched.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_learn/ tests/test_cli_learn.py -q
196 passed, 2 skipped in 17.80s

$ ruff check headroom/learn/writer.py headroom/cli/learn.py
All checks passed!

$ mypy headroom/learn/writer.py headroom/cli/learn.py
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS (Darwin), Python 3.11, headroom on rebased
upstream/main
- Exact command / steps: ran ClaudeCodeWriter against a temp project
whose `CLAUDE.md` held hand-written content plus a legacy Headroom
block, then `writer.write([...], dry_run=False)`
- Observed result: `CLAUDE.md` kept its hand-written content with the
block removed; `CLAUDE.local.md` gained both the migrated `### Old`
section and the new `### Env` section; `result.warnings` contained the
"Moved Headroom learnings out of …" notice. A block-only `CLAUDE.md` was
deleted and a "Removed …" warning emitted.
- Not tested: live end-to-end `headroom learn --apply` against real LLM
analysis (writer + CLI plumbing covered by unit/CLI tests with mocked
analysis)

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated CHANGELOG.md if applicable

## Additional Notes

Scoped to the Claude Code writer per the issue. After migration,
`discover_projects` may briefly re-surface a section the LLM re-derives,
but the write-side merge dedups by section name so the file stays
correct.
2026-06-22 15:05:06 -05:00
Tejas ChopraandJerrettDavis 0ddd4ed9e9 fix(learn): scan subagent and workflow transcripts (#1045)
## Description

`headroom learn` only scanned top-level main Claude Code sessions
(`<project>/<uuid>.jsonl`). Nested subagent and workflow transcripts
under `<project>/<uuid>/subagents/**` were not opened, which hid a large
amount of tool-call failure and token-spend activity from failure mining
and downstream analysis.

This change makes the Claude scanner descend into nested transcripts by
default and tag each `SessionData` with its source. `--main-only`
restores the previous top-level-only scan scope.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Updated `ClaudeCodePlugin.scan_project` to discover nested subagent
and workflow transcripts by default.
- Added source tagging for `main`, `subagent`, and `workflow` sessions.
- Added `--main-only` and `include_subagents` plumbing so callers can
opt back into top-level-only scanning.
- Added the `include_subagents` scanner parameter to Codex/Gemini as a
documented no-op because those scanners use flat session layouts.
- Added regression tests for nested discovery, source tagging, parallel
scanning, and CLI flag threading.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality

### Test Output

```text
Full learn + CLI suite:
# 186 passed, 2 skipped

GitHub Actions CI for this PR:
# build, build-wheel, lint, tests, e2e, CodeQL, and native wrapper checks passed
```

## Real Behavior Proof

- Environment: local Claude Code corpus with nested subagent/workflow
transcripts.
- Exact command / steps: Scanned the corpus with the previous
top-level-only behavior and then with nested transcript discovery
enabled.
- Observed result: The scanner saw 24 sessions before and 306 sessions
after descending into nested transcripts.
- Not tested: Codex/Gemini nested transcript discovery, because those
providers currently use flat session layouts and treat
`include_subagents` as a no-op.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-06-16 12:20:37 -07:00
Tejas Chopra 22e750c78f Fix learn CLI worker expectation in CI 2026-05-09 22:11:42 -07:00
Tejas Chopra 155c069176 fix: stabilize learn write-failure worker test
Pin os.cpu_count in the write-failure regression test so the expected default worker cap is independent of the GitHub runner CPU count.
2026-05-09 16:00:10 -07:00
Tejas Chopra 5ceca13c65 fix: harden learn path handling across platforms 2026-05-09 15:45:26 -07:00
GarmandClaude Opus 4.7 efd2ac1ca4 chore: renormalize line endings to LF
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.

Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.

Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 15:33:30 +02:00
JerrettDavisandCopilot 38bf3e639c test: expand coverage across helper slices
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-23 07:39:52 -05:00