mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat/https-compose-caddy
75
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
cfdeb1c466 |
fix(learn): render non-string Bash commands without raising (#4826)
## Description
Fixes a crash in `headroom learn` when a scanned session stores a
non-string Bash `command`.
`ToolCall.input_summary` sliced `input_data["command"]` directly,
assuming a string. A real opencode v2 database contained eight Bash
calls with `{"command": true}`, which raised:
```
TypeError: object of type 'bool' has no len()
```
Both the digest formatter (`analyzer._format_tool_call`) and the loop
sampler (`loops.detect_loops`) read `input_summary`, so a single
malformed call aborted the entire `headroom learn --all --apply` run.
No linked issue: this was found while validating the opencode v2 scanner
change in #4199 (now merged). The crash is in shared `ToolCall` code, so
the fix is submitted separately.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `ToolCall.input_summary` now coerces a non-string `command` with
`str()` instead of slicing the raw value; `None` renders as an empty
string.
- `input_summary` treats a non-mapping `input_data` as `{}` before
calling `.get()`, matching the defensive shape already used in
`loops._identity_input`.
- Added `TestInputSummaryToleratesMalformedInput` with six cases:
bool/int/`None`/list commands, a non-mapping input, and an end-to-end
`detect_loops` + `_build_digest` case.
- No behavior change for valid string commands; the other summary
branches already use `str()`/f-strings.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality
- [x] Manual testing performed
- [ ] Type checking passes (`mypy headroom`) — not run; the touched
module was not type-checked as part of this change.
### Test Output
```text
$ .venv/bin/python -m pytest tests/test_learn/test_loop_weighting.py -k TestInputSummaryToleratesMalformedInput -q
6 passed, 89 deselected, 1 warning in 0.78s
$ .venv/bin/python -m pytest tests/test_learn/ tests/test_cli_learn.py -q
455 passed, 7 skipped, 2 warnings in 32.88s
$ uvx ruff@0.16.8 check headroom/learn/models.py tests/test_learn/test_loop_weighting.py
All checks passed!
$ uvx ruff@0.16.8 format --check headroom/learn/models.py tests/test_learn/test_loop_weighting.py
2 files already formatted
```
All six new tests fail against the unpatched `models.py`.
## Real Behavior Proof
- Environment: Linux x86_64; Python 3.14.8; `uv sync --frozen`; PR head
`22e6a6f8fb05f79dcc82ea57a24bfbd6615305b8`. No provider credentials,
network, or private data required.
- Exact command / steps: save the script below as
`proof_malformed_opencode.py` at the repo root, then run
`.venv/bin/python proof_malformed_opencode.py`. It creates a temporary
opencode v2 SQLite database (`project` / `session_v2` /
`session_message`) containing three Bash calls whose input is
`{"command": true}`, then runs `OpenCodePlugin.scan_project` →
`detect_loops` → `_build_digest`.
- Observed result: with this PR, `sessions: 1 calls: 3`, `summaries:
['True', 'True', 'True']`, `loops: [('Bash', 3, 'True')]`, `digest
chars: 684`, `digest contains malformed call: True`. With
`headroom/learn/models.py` reverted to `origin/main`, the same script
exits with `TypeError: object of type 'bool' has no len()` at
`models.py:67` (`return cmd[:100] + "..." if len(cmd) > 100 else cmd`).
The original private database scan (28 sessions / 7,882 calls, 8
malformed commands, 29 loops, 319,885-char digest) is additional
evidence but is not reproducible outside that machine.
- Not tested: a real LLM analysis call (requires provider credentials),
Windows path handling, other agents' databases, and `mypy`. The scanner
and analyzer logic are unchanged; only the shared display property is
guarded.
<details>
<summary>proof_malformed_opencode.py</summary>
```python
"""Reproduce the malformed-command crash with a synthetic opencode v2 DB."""
import json
import sqlite3
import tempfile
from pathlib import Path
from headroom.learn.analyzer import _build_digest
from headroom.learn.loops import detect_loops
from headroom.learn.plugins.opencode import OpenCodePlugin
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
project_path = tmp_path / "project"
project_path.mkdir()
db_path = tmp_path / "opencode.db"
conn = sqlite3.connect(db_path)
conn.executescript(
"""
CREATE TABLE project (id TEXT PRIMARY KEY, worktree TEXT NOT NULL, name TEXT);
CREATE TABLE session_v2 (
id TEXT PRIMARY KEY, project_id TEXT NOT NULL, time_created INTEGER NOT NULL
);
CREATE TABLE session_message (
id TEXT PRIMARY KEY, session_id TEXT NOT NULL, type TEXT NOT NULL,
seq INTEGER NOT NULL, time_created INTEGER NOT NULL,
time_updated INTEGER NOT NULL, data TEXT NOT NULL
);
"""
)
conn.execute("INSERT INTO project VALUES (?, ?, ?)", ("p", str(project_path), "proof"))
conn.execute("INSERT INTO session_v2 VALUES (?, ?, ?)", ("s", "p", 1))
content = [
{
"type": "tool",
"id": f"call-{i}",
"name": "bash",
"state": {
"status": "completed",
"input": {"command": True},
"content": [{"type": "text", "text": "exit code 0"}],
},
}
for i in range(3)
]
conn.execute(
"INSERT INTO session_message VALUES (?, ?, ?, ?, ?, ?, ?)",
(
"m",
"s",
"assistant",
1,
1,
1,
json.dumps({"content": content}, separators=(",", ":")),
),
)
conn.commit()
conn.close()
plugin = OpenCodePlugin(db_path=db_path)
project = plugin.discover_projects()[0]
sessions = plugin.scan_project(project)
print("sessions:", len(sessions), "calls:", sum(len(s.tool_calls) for s in sessions))
print("summaries:", [c.input_summary for c in sessions[0].tool_calls])
loops = detect_loops(sessions)
print("loops:", [(lp.tool, lp.count, lp.sample_input) for lp in loops])
digest = _build_digest(project, sessions, loops=loops)
print("digest chars:", len(digest))
print("digest contains malformed call:", "Bash: True" in digest)
```
</details>
## Runtime Rollout Safety
- Rollout-managed feature(s): `headroom learn` local session scanning
and digest formatting only; no proxy, provider, or runtime request path.
- Minimum rollout channel: N/A — library bug fix, not gated by a rollout
channel.
- Stable/default behavior changed: No. Valid string commands render
exactly as before; only malformed inputs that previously raised now
render.
- Kill switch / disable path: N/A — revert the commit to restore the
prior behavior.
- Unsafe override required: No.
- Qualification impact: None; no network, model, or rollout behavior is
touched.
- Rollback path: revert this commit/PR.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation — N/A, no
public API or docs change
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A — no UI change.
## Additional Notes
The malformed value is real session data, not a synthetic edge case:
session `ses_e6bcaf904fd4blosS7bPwIvQ1f` in a local opencode v2 DB
stores `{"command": true}` for 8 Bash calls. The fix only affects
display/summary rendering, so loop identity, failure classification, and
recommendations are unchanged.
Co-authored-by: JavierLeon9966 <javier@lumineproxy.org>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
d2fb350a34 |
fix(learn): only fold real linked worktrees into their main checkout (#4839)
## Description `headroom learn` folds a linked git worktree's sessions into its main checkout's project via `_main_worktree_root()` (`headroom/learn/plugins/claude.py`). It read the checkout's `.git` file (`gitdir: ...`) and that dir's `commondir`, and returned the parent of whatever directory they named. Both files come from the checkout itself, so a checkout whose pointers named an unrelated directory made learn use that directory as the project root, and `learn --apply` wrote `CLAUDE.local.md` there. The function now accepts only git's own linked-worktree layout: - the admin dir (the `gitdir:` target) must be `<common>/worktrees/<name>`, and - its `gitdir` file must point back at this checkout's `.git` (git writes this for every `git worktree add`). Anything else returns `None`, so the session folder stays its own project at its own path (the existing fallback). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/learn/plugins/claude.py`: layout and back-pointer checks in `_main_worktree_root()`. - `tests/test_learn/test_worktree_merge.py`: the worktree fixture now writes the `gitdir` back-pointer like real git; new tests for pointers outside the `worktrees/<name>` layout and for an admin dir that does not point back at the checkout. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # new tests on origin/main (fix stashed) $ pytest -q tests/test_learn/test_worktree_merge.py FAILED tests/test_learn/test_worktree_merge.py::test_main_worktree_root_ignores_pointers_outside_gits_worktree_layout FAILED tests/test_learn/test_worktree_merge.py::test_main_worktree_root_needs_the_admin_dir_to_point_back ========================= 2 failed, 7 passed in 0.96s ========================== # with the fix $ pytest -q tests/test_learn ======================= 402 passed, 4 skipped in 32.06s ======================== $ ruff check / ruff format --check (0.17.3) on changed files All checks passed! 2 files already formatted $ mypy headroom/learn/plugins/claude.py --ignore-missing-imports Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS, git 2.x, Python 3.12 venv. - Exact command / steps: `git init m && git -C m commit --allow-empty -m x && git -C m worktree add ../w`; separately a directory `scanned/` whose `.git` file is `gitdir: <abs>/elsewhere/.git` with `elsewhere/.git/commondir` = `.`. Then `_main_worktree_root(Path(...))` for both. - Observed result: - main: `w -> .../gwt/m`, `scanned -> .../gwt/elsewhere` - this branch: `w -> .../gwt/m` (real worktree still folds), `scanned -> None` - Not tested: a full `learn --apply` run against live Claude Code session folders. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: worktrees created by `git worktree add` fold exactly as before. A worktree whose admin dir was moved or whose back-pointer is stale (git would report it as prunable) is no longer folded and stays its own project. - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert this PR ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes No docs change needed. Fix adapted from a proposal by HetCreep (this version adds the back-pointer check, since the `worktrees/<name>` shape alone can be reproduced inside a checkout). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4839" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
12864fc8f0 |
fix(learn): harden analyzer JSON parsing against malformed LLM output (#2471)
## Description
Two spots in the `learn` analyzer crash on JSON a model can
realistically emit. Both parse fully model-controlled output.
**1. `_strip_fenced_json` returns a non-object instead of raising.** It
is documented to return a dict and to raise `json.JSONDecodeError` when
no candidate is an object, but its final fallback was `result: dict =
json.loads(text); return result`. `json.loads` raises for invalid JSON,
but for valid JSON that is not an object (a bare array, string, or
number) it returns a non-dict. The `-> dict`-annotated value is then a
list/str, and all three callers do `.get(...)` on it, raising
`AttributeError` instead of the clean `JSONDecodeError` they already
handle.
**2. `_parse_llm_response` crashes on null rule lists/fields.** It read
`raw.get("context_file_rules", [])` / `raw.get("memory_file_rules", [])`
and `rule.get("section", "").strip()` / `rule.get("content",
"").strip()`. `.get(key, default)` only falls back when the key is
absent, so a present-but-null `"context_file_rules": null` made `for
rule in None` raise `TypeError`, and a null `section`/`content` made
`None.strip()` raise `AttributeError`.
## Fix
- `_strip_fenced_json`: reject a non-dict fallback result explicitly
(`raise json.JSONDecodeError(...)`), restoring the documented contract.
The lenient first-brace..last-brace slice still recovers a single object
a model wrapped in an array; only genuinely object-less JSON raises.
- `_parse_llm_response`: coerce the rule lists and the
`section`/`content` fields with `or` so a null is treated like an absent
value. Valid rules parse unchanged.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/analyzer.py`: raise `JSONDecodeError` when
`_strip_fenced_json`'s fallback parse yields a non-dict; `or`-coerce
null rule lists and null `section`/`content` in `_parse_llm_response`.
- `tests/test_learn/test_analyzer.py`: regressions for non-object JSON
raising (and array-wrapped-object still extracted), and for null rule
lists / null fields being tolerated with valid rules still parsed.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_learn/test_analyzer.py::TestStripFencedJson tests/test_learn/test_analyzer.py::TestLLMResponseParser -q
# all pass; each new test fails with the corresponding fix reverted
# (JSONDecodeError contract / 'NoneType' object is not iterable / has no attribute 'strip')
$ uvx ruff@0.15.17 check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/learn/analyzer.py
Success: no issues found in 1 source file
```
Note: one unrelated digest-builder test
(`TestDigestBuilder::test_includes_project_info`) fails in my local
Windows venv because it hardcodes a `/tmp/...` path; it fails
identically on a clean `main` and is not touched by this change.
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called the real `_strip_fenced_json` with
object-less JSON (`[1,2,3]`, `"str"`, `42`, ...) and valid/extractable
objects, and the real `_parse_llm_response` with `{"context_file_rules":
null}`, a rule with `section: null`, and a valid rule; then reverted
each fix and re-ran.
- Observed result: with the fixes the object-less inputs raise
`JSONDecodeError`, the null shapes yield an empty recommendation list,
and valid inputs parse as before; with each fix reverted the respective
input raises (`AttributeError: 'list' object has no attribute 'get'`,
`TypeError: 'NoneType' object is not iterable`, `AttributeError:
'NoneType' object has no attribute 'strip'`). Ran against the actual
module.
- Not tested: a live `headroom learn` run where the backing model
returns these shapes.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2471"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
### Maintainer current-main requalification
Exact head `27fa31eac507413415a00b54e29679bef08eabe9`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.
Clean current upstream/main0ddaa56c integration; production
_strip_fenced_json/_parse_llm_response/_as_str ASTs are identical to
contributor published head. Main-relative scope remains two files. Small
maintainer test-only repair uses str(project.project_path) instead of a
POSIX-only literal in existing digest assertion; source and test fixture
data unchanged. Initial full analyzer run109pass/1fail and separate
clean-main failure independently establish inherited Windows path
expectation before repair. After repair399tests pass,11skip,1warning
(all110analyzer cases plus289incoming-main regressions). Two changed
files pass Rufflint/format,candidate offline lock and whitespace pass.
Complete native inputset/allrecordedhashes/artifactchecksum verified
before existing3982DLL reuse; no fresh build.
Independent paired parser probes: branch passes both truthy non-string
section and objectless JSON contract cases; clean main fails with
AttributeError and missing JSONDecodeError respectively. These are
controlled parser executions, not live LLM quality/provider behavior.
Outstanding Devin truthy non-list rule-container report also
independently reproduced on BOTH branch and clean main:
context_file_rules:42 and memory_file_rules:42 raise TypeError in both.
This is an inherited limitation, not a PR-introduced regression or
grounds to withhold this narrower fix. Do not claim arbitrary-container
safety. The later author suggestion to join list-valued rule content
changes intended normalization semantics; not selected or implemented by
maintainer. Existing _as_str drops non-string fields, preserving
contributor intent.
Read author replies/current discussions and unresolved thread. No new
unreviewed external findings at prepublication audit. Source/test diff
contains no workflow/setup/dependency/native/CLI paths; central
publication gate passed. Exact published-head Actions/rollup, current
main ancestry/mergeability and fresh discussion remain before final
approval. Maintainers handle CI; no PR merged.
Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
c05466a469 |
fix(learn): support opencode v2 session schema in learn scanner (#4199)
## Description
`headroom learn --agent opencode` crashes on current opencode releases
(v2.x, e.g. opencode v2.0.18). The plugin only understood the v1
database layout (`message` + `part` tables, `session` table), so
discovery fails with:
```
sqlite3.OperationalError: no such table: session
```
Current opencode stores conversations in `session_v2` +
`session_message`, where each message row holds JSON `data` with a
`content` list and tool entries shaped as `{"type": "tool", "id",
"name", "state": {"status", "input", "content": [{"type": "text",
"text"}], "error"}}`.
This PR makes the scanner schema-aware while keeping v1 databases
working unchanged:
- Detect the session table (`session_v2` preferred, `session` fallback);
return [] with a warning when neither exists
- Scan `session_message` rows for v2 (tolerant `LIKE` prefilter,
per-item tool parsing) and keep the legacy `part`/`message` path for v1
- Parse both tool shapes: `name`/`id` (v2) and `tool`/`callID` (v1);
extract output from `state.content[]` text blocks, `state.output`
(legacy), and surface `state.error` so classification still sees failure
messages
- Fall back to the worktree basename when `project.name` is NULL (v2
leaves it NULL) and tolerate DBs without a `name` column
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/learn/plugins/opencode.py`: schema detection + v2 scanning +
dual-shape output extraction
- `tests/test_learn/test_opencode_scanner.py`: v2 regression tests
(discover/scan + NULL-name fallback)
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
tests/test_learn/test_opencode_scanner.py: 12 passed (10 pre-existing + 2 new)
tests/test_learn/: 357 passed, 7 skipped
tests/test_cli_learn.py: 51 passed
ruff check on both touched files: All checks passed!
```
## Real Behavior Proof
- Environment: Linux, headroom 0.40.0, opencode v2.0.18,
`~/.local/share/opencode/opencode.db` with
`session_v2`/`session_message` tables
- Exact command / steps: `headroom learn --apply --all --agent opencode`
(with the patched plugin)
- Observed result: 11 projects discovered (previously crashed on first),
e.g. portal 72 calls/6 failures, proxy 1093 calls/47 failures;
recommendations written to per-project `AGENTS.md`
- Not tested: legacy v1 database against a live install (covered by
pre-existing unit tests instead)
## Runtime Rollout Safety
- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no (v1 path preserved; v2 previously
crashed)
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Local end-to-end verification also needed an LLM backend (this env has
no API keys and the codex CLI auth is expired), so I ran analysis via an
ad-hoc local `opencode run` backend patch — deliberately **not**
included here since `opencode run` has full tool access (same safety
concern as `agy-cli`, which needs an opt-in). Happy to follow up with an
opt-in-gated `opencode-cli` backend PR if maintainers want it.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4199"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
### Maintainer current-main requalification
Exact head `1ac78ad8755f34af5e5531b1f4de446588cbb234`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.
Re-reviewed contributor fix
|
||
|
|
45dacc1f94 |
fix(learn): anchor the managed block on its last start marker, append when no end marker (#4342)
## Description `_merge_into_file` (`headroom/learn/writer.py`) located the managed block with `block_pattern` (first occurrence of the start marker to the nearest end marker) whenever `_MARKER_START in existing`. Two consequences, both reported in #4339: 1. Hand-written prose that quotes the start-marker string became the start of the match, so `--apply` replaced everything from that mention through the real block's end marker, deleting the hand-written text in between. 2. A file holding the start marker but no end marker matched nothing: `sub()` returned the file unchanged and every recommendation of the run was silently dropped while the CLI still printed `[WROTE]`. The block is now identified by the shape `_build_section` writes: a line that is **exactly** the start marker through the next line that is **exactly** the end marker. A marker quoted inside prose (sharing its line with other text, or indented) or inside a fenced code block (a Markdown example) is not a boundary, and a start marker with no end marker before the next start marker is skipped. One helper, `_find_managed_block()`, is used by every reader and writer of the block in this module, so the block that is parsed is the block that is replaced. The first commit ( |
||
|
|
506ba16c4a |
fix(learn): keep the digest's loop section inside its budget (#4269)
## Description
`headroom learn` can build a prompt larger than the CLI it runs will
accept, and the prompt-too-long retry cannot shrink it.
`_build_digest` budgets the per-session event stream (`max_tokens * 4`
chars), but it renders the "Detected Loops" section ahead of that stream
without any bound: every detected loop, each with every message index it
repeated at. Codex learn scans all of a machine's Codex sessions as one
project, so loops accumulate across every session the user ever ran. On
one Windows machine this produced a 1,786,526-char prompt, and `codex
exec` refused it on every scan:
```text
Error: turn/start: turn/start failed: Input exceeds the maximum length of 1048576 characters. (code -32602), data: {"input_error_code":"input_too_large","max_chars":1048576,"actual_chars":1786526}
```
Halving `max_tokens` on a prompt-too-long error does not help here: it
only trims the event stream, never the loop section.
This PR bounds the rendered loop section by the same budget, and lets a
Codex overflow take the smaller-digest retry.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `learn/loops.py`: `format_loops_for_digest` takes `max_chars` and
counts the rendered text, header and newlines included. It adds loops in
the given order (`detect_loops` sorts by measured waste) and stops at
the first one whose line would pass `max_chars`, so the costliest loops
are kept. Each line shows its sample input cut to 120 chars (the same
cut `detect_loops` makes) and its first 20 message indices beside the
full repeat count. It returns "" when no loop fits.
- `learn/analyzer.py`: `_build_digest` gives the loop section a quarter
of the `max_tokens * 4` char budget, less whatever the header and the
prior learned patterns leave short, so header + loops + prior patterns
stay inside the budget and the event stream fills the rest. The prior
patterns are never cut: a re-emitted section replaces its prior one
wholesale, so a partial baseline would drop bullets from the user's
file.
- `learn/analyzer.py`: the event stream counts every line with the
newline that joins it, and a session header, event or tool-call line
goes in only if it leaves 100 chars for the truncation markers that can
follow it. Once a session is cut, no later session is started. So the
whole digest stays inside `max_tokens * 4`; the only exception is a
prior-patterns section that leaves no room on its own, since it is never
cut.
- `learn/analyzer.py`: `_is_prompt_too_long` also matches Codex's "Input
exceeds the maximum length" refusal, so a Codex overflow is retried at
half the budget like the other backends.
- The caller's `LoopPattern`s are not touched (slices only), and
`apply_loop_weighting` still receives every loop.
- Tests: `test_loop_section_stays_inside_the_digest_budget` is
parametrized over a 120-char Bash input, a 2,000-char Grep pattern and a
2,000-char Glob path, at the 80k and 10k (retry floor) budgets. New
`test_loops_fit_beside_a_large_prior_patterns_section` (80k, 10k). New
`test_whole_digest_stays_inside_the_budget`: 5 sessions of 3,000 failing
calls each, at 80k and 10k, as tool calls and as events, with and
without loops, asserts `len(digest) <= max_tokens * 4`. New
`test_many_tiny_sessions_stay_inside_the_budget`: 20,000 one-call
sessions at 80k and 10k, as tool calls and as events (614 to 4,862
sessions kept), so the blank line closing each session is counted too.
`test_retries_smaller_digest_on_prompt_too_long` adds the real Codex
error text.
- `pyproject.toml` / `uv.lock` (review: Dependency audit): the `aiohttp`
constraint floor is now `>=3.14.4` (CVE-2026-111063, CVE-2026-111064 and
GHSA-2g87-pp6c-29x5, all fixed in 3.14.4), relocked with `uv lock
--upgrade-package aiohttp==3.14.4`. Only aiohttp (3.14.3 -> 3.14.4) and
yarl (1.22.0 -> 1.25.1, which 3.14.4 requires) move. Main still resolves
3.14.3, so the same commit is on #4299, #4286, #4285 and #4269.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
`mypy` passes on the changed module.
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_learn/ -q
384 passed, 4 skipped in 34.16s
$ uvx ruff@0.17.0 check headroom/learn/analyzer.py headroom/learn/loops.py tests/test_learn/test_loop_weighting.py tests/test_learn/test_analyzer.py
All checks passed!
$ uvx ruff@0.17.0 format --check headroom/learn/analyzer.py headroom/learn/loops.py tests/test_learn/test_loop_weighting.py tests/test_learn/test_analyzer.py
4 files already formatted
$ uv run --frozen --extra dev mypy headroom/learn/analyzer.py headroom/learn/loops.py
Success: no issues found in 2 source files
```
With `headroom/learn/` reverted to
|
||
|
|
4e6e955cad |
fix(memory): log storage and SQLite failures that currently make retrieval miss silently (#4111)
## Description Several storage paths turn a real failure into a quiet miss. They either log nothing, or log a line that does not say which operation or file failed. When CCR or memory retrieval misses, an operator cannot tell corruption or a broken plugin from a genuine "not found". This PR adds the missing log lines. Return values and control flow do not change. Refs #4136 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/storage/__init__.py` (`create_storage`): - A `headroom.storage_backend` plugin that raised on load was swallowed by `except Exception: pass`. So was a scheme with no plugin. Both then fell back to `SQLiteStorage(store_url)`, which opens a SQLite file named after the URL. - Both cases now log a warning that names the scheme. The URL itself is not logged, because it can carry credentials. - A plugin that raises is logged with the exception type only, because its text can echo the URL. A separate DEBUG line gives `log_safety.describe_exception` (exception types and code locations, never the message), so no level carries the URL or its password. - The module gets a `logger`. - `headroom/cache/backends/sqlite.py` (CCR store): - `_entry_from_json` returned `None` with no log for a row that fails to decode or rebuild. So `get()` reported a plain miss for an entry that exists but is unreadable. - It now logs a warning with the hash, the db path and the exception type. `items()` decodes every row on each eviction pass, so the warning fires once per hash. - The once-per-hash guard is `log_safety.WarnOnce`. Within a one-hour window it warns for at most 1,000 hashes, then logs one overflow warning, and later rows log at DEBUG only, so repeated eviction scans over many bad rows cannot flood the log. When the window ends, a row that is still broken warns again, about once an hour. A hash is used up only while WARNING is enabled. - CCR keys come from model-issued retrieve calls, so they are logged through `log_safety.safe_id`: a repr (newlines escaped) capped at 80 characters. - Failed `delete`/`exists`/`clear`/`count`/`keys`/`items`/`get_stats` all logged as `"op"`. Each now logs its real name, with the key for `get`/`set`/`delete`/`exists`. - The transient-error warning now includes the db path. - `headroom/memory/traffic_learner.py`: the two read-only SQLite reads returned `[]` on `OperationalError`/`DatabaseError` with no log: the hydrate path and `_load_persisted_patterns_from_sqlite`. Both now warn with the db path. Also: - dropped a redundant `conn.close()` in an `except` whose `finally` already closes; - the two best-effort `conn.close()` calls now catch `sqlite3.Error` and log at debug, instead of `except Exception: pass`. - `headroom/memory/backends/direct_mem0.py`: `get_memory`/`delete_memory` turned any Qdrant/Neo4j exception into "not found". They now warn with the memory id and the exception type first. - The exception text is not logged at any level, because backend errors can carry stored memory or connection details. A DEBUG line gives `describe_exception`. - Memory ids come from the model, so they are logged through `log_safety.safe_id`, which escapes newlines and caps the length. - mem0 2.x `Memory.delete` raises exactly `ValueError(f"Memory with id {memory_id} not found")` for an unknown id. Only that message, for the requested id, logs at DEBUG; the delete is model-driven, so it is expected. Any other `ValueError`, such as a missing collection, still warns. - When a corrupt CCR store is recreated, the unreadable-row guard starts fresh, because the new file has no rows. - `headroom/learn/registry.py`: a built-in learn plugin that fails to import was logged at debug, so `headroom learn --agent <name>` only said "Unknown agent". It is now a warning with the traceback. Built-in plugins use only the stdlib, so this never fires on a healthy install. - `headroom/cache/compression_store.py`: the "Failed to load CCR backend" warning now says it falls back to the in-process store, and what that costs. This matches the SQLite branch next to it. It logs the exception type only, because the backend's text can echo `HEADROOM_REDIS_URL`. DEBUG gets `describe_exception`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest -q -p no:cacheprovider tests/test_storage_backends.py tests/test_storage tests/test_ccr_sqlite_backend.py tests/test_compression_store.py tests/test_learn/test_registry.py tests/test_memory/test_traffic_learner.py tests/test_memory/test_direct_mem0.py tests/test_private_state_files.py tests/test_adapter_hooks.py tests/test_cache/test_backends.py tests/test_learn/test_opencode_scanner.py tests/test_openai_responses_traffic_learner.py tests/test_learn/test_section_lifecycle.py tests/test_learn/test_worktree_merge.py tests/test_memory_handler_native_ops.py ================= 537 passed, 1 skipped, 5 warnings in 34.37s ================== # The three new tests against origin/main (product code reverted): FAILED tests/test_storage_backends.py::test_create_storage_warns_when_a_backend_plugin_fails_to_load FAILED tests/test_ccr_sqlite_backend.py::TestSQLiteBackend::test_unreadable_row_is_logged_once_and_healthy_rows_are_not FAILED tests/test_learn/test_registry.py::TestBuiltinLoadFailure::test_broken_builtin_plugin_is_logged_as_a_warning $ uvx ruff@0.16.8 check <changed files> All checks passed! # Added in review: the hydration-failure warning; mem0 delete/get paths (only mem0's exact missing-memory message is quiet; a missing collection warns); newline and 100,000-char model ids; two scans over more bad rows than the cap; and storage-plugin, CCR-backend and mem0 failures whose records are rendered with logging.Formatter at DEBUG and WARNING and checked for credential and content canaries. # Those tests against the previous head |
||
|
|
abbf36f052 |
refactor(memory,learn,cache): remove unreferenced classes, methods and constants (#4112)
## Description This removes nine symbols in `headroom/memory`, `headroom/learn` and `headroom/cache` that nothing references outside their own definition. One is a legacy alias whose only user was its own test. Evidence: for each name I ran `grep -rnw <name>` across `headroom crates plugins sdk docs wiki scripts benchmarks examples e2e .github pyproject.toml tests` and the org's consumer repos. The only hit was the definition, plus the `FailureAnalyzer` alias test removed here. None of the names is public API. None is in an `__all__`, a re-export or lazy export map, an entry-point group or the docs, and no consumer imports it. Refs #4136 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `memory/inline_extractor.py`: remove `InlineMemoryWrapper`, with `chat` and `chat_with_response`. `with_memory` (`memory/wrapper.py`) uses only the module's functions, `inject_memory_instruction` and `parse_response_with_memory`, and those stay. - `memory/traffic_learner.py`: remove these: - `AgentType`; - `_COMMAND_RE` and `_FILE_PATH_RE`. `_MODULE_RE` and `_COMMAND_NF_RE` next to them are used and stay; - `TrafficLearner.get_learned_patterns`. Its docstring said "retained for backwards compatibility", but `TrafficLearner` is internal to the proxy. - `memory/core.py`: remove the private `HierarchicalMemory._scope_level_value`. - `cache/compression_feedback.py`: remove the private `CompressionFeedback._truncate_strategy_dicts`. - `learn/analyzer.py`: remove `FailureAnalyzer`. Its docstring said "used by existing CLI code", but `cli/learn.py` uses `SessionAnalyzer` directly. Also remove `TestFailureAnalyzerCompat` from `tests/test_learn/test_analyzer.py`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest -q -p no:cacheprovider tests/test_memory_wrapper.py tests/test_inline_extractor_inject.py tests/test_memory/test_traffic_learner.py tests/test_openai_responses_traffic_learner.py tests/test_memory/test_core_operations.py tests/test_learn/test_analyzer.py tests/test_ccr_feedback.py tests/test_memory/test_hierarchical.py tests/test_toin_fixes.py ================ 417 passed, 8 skipped, 772 warnings in 49.26s ================= $ mypy <5 changed product files> --ignore-missing-imports Success: no issues found in 5 source files $ uvx ruff@0.16.8 check <changed files> All checks passed! ``` ## Real Behavior Proof - Environment: macOS, Python 3.12.13, repo .venv, branch checkout at |
||
|
|
ec1ba70c97 |
fix(learn): decode sanitized pattern-id markers and keep visible-text dedup keys (#3956)
## Description **Problem:** `test_category_losing_its_last_pattern_drops_its_heading` fails on main (reproduced independently at |
||
|
|
3fdf18eded |
fix(learn): merge git worktree sessions into the repo's project (#3854)
## Description `headroom learn` treats every git worktree as its own project. Claude Code stores each working directory's sessions in a separate `~/.claude/projects/<encoded-cwd>/` folder, and `ClaudeCodePlugin.discover_projects` turns each folder into one `ProjectInfo`. So a linked worktree (a Conductor workspace, `.claude/worktrees/*`, any `git worktree add` checkout) shows up as a separate project with a few sessions: - Each worktree rarely has enough sessions to cross a pattern threshold, so the repo's learnings never add up. - Learnings are written into the worktree: `CLAUDE.local.md` in a checkout that is deleted when the workspace is archived, and `MEMORY.md` in that worktree's own memory folder. - For a repo worked on only through worktrees (the usual Conductor setup), `headroom learn --project <repo>` finds nothing: `No matching project. Try --all or --project <path>.` - The traffic learner uses the same roots, so live patterns that mention a worktree path go to the worktree too. A user reported this: their project list was a row of Conductor workspace names, each "not scanned yet". This PR merges each linked worktree into the project of its main checkout. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `learn/plugins/claude.py`: new `_main_worktree_root(path)` resolves a linked worktree to its main checkout by reading git's own pointer files: the `.git` file's `gitdir:` line, then that gitdir's `commondir`. It does not run git. It returns None for a main checkout, a submodule (its gitdir has no `commondir`), a bare repo, a relative path, or anything outside git. - `discover_projects` now ends in `_merge_worktrees`, which groups projects by that root. The merged project: - keeps the main checkout's `project_path` and session folder; - writes to the main checkout's `CLAUDE.local.md`; - writes `MEMORY.md` to the main checkout's memory folder, the same folder Claude Code's own auto-memory uses for every worktree of a repo. If only worktrees have sessions, the merged project uses the folder Claude Code would give the main checkout. Projects outside worktrees are unchanged. - `ProjectInfo` gains `worktree_paths` and `extra_data_paths` (both default to empty lists). `scan_project` and the `--verbosity` path read sessions from `data_path` plus `extra_data_paths`. - `memory/traffic_learner.py`: `_project_for_pattern` also matches paths under a project's `worktree_paths` and routes those patterns to the project. Longest match still wins. - A worktree whose checkout has been deleted has no `.git` file left to resolve, so it stays a separate project, as before. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed `mypy` passes on the four changed modules. `mypy headroom` reports one existing error in `headroom/integrations/langchain/chat_model.py:394` (`truthy-function`), a file this PR does not touch. ### Test Output ```text $ uv run --extra dev pytest tests/test_learn/ tests/test_memory/ tests/test_cli_learn.py tests/test_learn_grok_plugin.py tests/test_verbosity_learn.py -q 966 passed, 4 skipped, 79 warnings in 44.31s $ uv run --extra dev ruff check . && uv run --extra dev ruff format --check . All checks passed! 1671 files already formatted $ uv run --extra dev mypy headroom/learn/plugins/claude.py headroom/learn/models.py headroom/memory/traffic_learner.py headroom/cli/learn.py Success: no issues found in 4 source files ``` With the source reverted to `main`, `tests/test_learn/test_worktree_merge.py` fails (it imports `_main_worktree_root`). ## Real Behavior Proof - Environment: macOS 15 (Darwin 24.6), Python 3.12, git 2.x; a scratch repo plus a real `git worktree add` checkout; a scratch `CLAUDE_CONFIG_DIR` with three Claude Code session files stored under the worktree's encoded folder name. - Exact command / steps: `git init repo && git -C repo commit --allow-empty -m init && git -C repo worktree add ../ws/san-salvador -b feat`, then `CLAUDE_CONFIG_DIR=<scratch>/claude headroom learn --verbosity --agent claude --project <scratch>/repo`, run once on `main` and once on this branch. - Observed result: on `main`: `No matching project. Try --all or --project <path>.` On this branch: `Verbosity — repo`, `Path: <scratch>/repo`, `Sessions: 3`. The real git layout matches the test fixtures: the worktree's `.git` is `gitdir: <repo>/.git/worktrees/san-salvador`, and `commondir` is `../..`. - Not tested: Windows worktrees on a real machine (the resolver uses only `pathlib`); the full LLM analysis path of `headroom learn --apply` (it shares `discover_projects`/`scan_project` with the verbosity path shown above). ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Stable. - Stable/default behavior changed: Yes, for users with git worktrees only: `headroom learn` and the traffic learner now treat a linked worktree as part of its main checkout's project instead of as a separate project. Repos without worktrees are unchanged. - Kill switch / disable path: Not applicable. - Unsafe override required: No. - Qualification impact: None. Learn and the memory writer only; the proxy request path and compression are untouched. - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable. ## Additional Notes - No documentation change: the learn docs don't describe how projects are discovered. - Open question for maintainers: Conductor-style workspaces are fresh checkouts, so they never see the main checkout's `CLAUDE.local.md`. Only `MEMORY.md` is shared across a repo's worktrees. For a repo worked on only through worktrees, CONTEXT_FILE rules may be better routed to `MEMORY.md`. I've left that out of this PR to keep it to one behavior change. - Follow-up idea, also not included: a worktree whose checkout has been deleted could be attributed through its session folder name. Today it stays a separate project, as before. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3854" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
4ba231ab3d |
feat(learn): add agy (Antigravity CLI) as an analysis backend (#3939)
## Description
Adds the Antigravity CLI (`agy`) as a keyless LLM backend for `headroom
learn`, next to `claude`, `gemini` and `codex`. Antigravity subscription
users with no raw API key can now run the analysis through `agy`.
`agy` cannot use the existing stdin path as is. `agy -p` takes the
prompt as a flag argument and never reads stdin, so a large digest would
hit OS argument-length limits. The only stdin path is `--input-format
stream-json`, which requires `--output-format stream-json`. The backend
therefore sends the prompt as a single NDJSON user event on stdin and
reads the answer from `result.response` in the final `result` event.
Everything else (wall-clock timeout, Windows shim fallback, failure
detail, fenced-JSON parsing) reuses the existing `run()` path that
`gemini` and `codex` use.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/analyzer.py`: add an `agy` / `agy-cli` entry to
`_CLI_BACKENDS` (`agy --input-format stream-json --output-format
stream-json`). `agy` is never auto-detected, and it only runs when
`HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` is set: `agy` has no zero-tool mode,
so transcript text could drive its tools. Without the opt-in,
`HEADROOM_LEARN_CLI=agy` and `--model agy-cli` fail before any
subprocess starts; with it, a warning is logged on every run.
- `headroom/learn/analyzer.py`: in `_call_cli_llm`, wrap the prompt as
`{"event": "user", "message": {"content": ...}}` for `agy-cli`, and
extract `result.response` from the stream with a new
`_agy_result_text()` helper before the existing `_strip_fenced_json()`
parsing. A missing `result` event surfaces as the existing "unparseable
output" error, with both ends of stdout.
- `tests/test_learn/test_analyzer.py`: tests that auto-detection never
selects `agy` (even when it is the only CLI on PATH and the opt-in is
set), that `HEADROOM_LEARN_CLI=agy` and `_call_cli_llm(..., "agy-cli")`
fail without the opt-in (no subprocess call), `HEADROOM_LEARN_CLI=agy`
with the opt-in, existing CLIs still taking precedence, the exact
command and stdin payload, result-event parsing (including a fenced
response and a literal U+2028), and a missing `result` event.
- `headroom/cli/learn.py`: check the agy opt-in during model resolution,
so `HEADROOM_LEARN_CLI=agy` and `--model agy-cli` without it exit 1 with
an `Error:` line before any sessions are scanned.
- `tests/test_cli_learn.py`: a CLI test for both paths without the
opt-in.
- `wiki/learn.md`: list `agy` among valid `HEADROOM_LEARN_CLI` values
and document the `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` opt-in and its
risk.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_learn/ tests/test_cli_learn.py -q
5 failed, 345 passed, 4 skipped in 10.78s
# The same 5 fail on unmodified upstream/main on this Windows machine (5 failed, 340 passed),
# all path/git related and unrelated to this change:
# test_analyzer.py::TestDigestBuilder::test_includes_project_info
# test_scanner.py::TestDecodeProjectPath::test_home_dir_username_stays_single_component
# test_writer.py::TestClaudeLocalMdStaysOutOfGit::{test_apply_adds_exclude_entry,
# test_second_run_does_not_duplicate_the_entry, test_tracked_file_warns_instead_of_excluding}
$ uv run --frozen --extra dev pytest tests/test_learn/test_analyzer.py -q -k agy # analyzer.py reverted
FAILED TestDetectDefaultModel::test_env_var_selects_agy_with_unsafe_opt_in
FAILED TestDetectDefaultModel::test_env_var_agy_without_unsafe_opt_in_raises
FAILED TestCallCliLlm::test_agy_cli_sends_stream_json_and_parses_result_event
FAILED TestCallCliLlm::test_agy_cli_result_with_unicode_line_separator
FAILED TestCallCliLlm::test_agy_cli_without_unsafe_opt_in_does_not_run
FAILED TestCallCliLlm::test_agy_cli_missing_result_event_raises
6 failed, 2 passed, 97 deselected
# All 8 pass with the change.
$ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
All checks passed!
$ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/learn/analyzer.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, `agy` 1.2.14 signed in via subscription, no
API keys set.
- Exact command / steps: `HEADROOM_LEARN_CLI=agy
HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1 ANTHROPIC_API_KEY= OPENAI_API_KEY=
GEMINI_API_KEY= uv run --frozen --extra dev headroom learn` in this
repository (dry run).
- Observed result: `Analyzing with agy-cli...`, then `Sessions: 3 |
Calls: 109 | Failures: 19 (17.4%)`, `Recommendations: 8`, with `[WOULD
WRITE]` blocks for `CLAUDE.local.md` and `MEMORY.md`; exit 0. Before
wiring the backend I confirmed the protocol by hand: `agy -p -` answers
the literal prompt `-` and ignores stdin, while the stream-json
invocation returns the answer in `{"event": "result", "result":
{"status": "SUCCESS", "response": "..."}}`.
- Not tested: macOS/Linux `agy` installs; `agy` under a permission mode
that prompts for tool approval (my config is `always-proceed`), where a
tool request in print mode would be bounded by the existing
`HEADROOM_LEARN_CLI_TIMEOUT_SECS` wall-clock cap.
## Runtime Rollout Safety
- Rollout-managed feature(s): None (offline `headroom learn` CLI)
- Minimum rollout channel: dev
- Stable/default behavior changed: No. `agy` is never auto-detected; it
is only used when selected with `HEADROOM_LEARN_CLI=agy` / `--model
agy-cli` together with `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1`.
- Kill switch / disable path: Set an API key, `HEADROOM_LEARN_CLI`, or
`--model` to pick another backend.
- Unsafe override required: None
- Qualification impact: None
- Rollback path: Revert this PR.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A (CLI change).
## Additional Notes
- No tracking issue exists for an `agy` analysis backend, so `Closes #`
is left empty. Related Antigravity work (#3428 `headroom init -g agy`,
#3610 Antigravity transcripts in `headroom learn`) is independent of
this change.
- Unlike `claude-cli` (#3892), `agy` has no flag to start a session with
zero tools. I deliberately did not add `--dangerously-skip-permissions`:
the digest contains untrusted transcript content, and the backend should
run under the user's own permission settings.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3939"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
|
f00d425614 |
fix(learn): drop the echoed prompt from a failed cli's error (#3928)
## Description
`codex exec` echoes the whole stdin prompt to stderr ahead of its own
error, and a failed CLI's error message keeps the first 2000 chars of
stderr. The analyzer's system prompt alone is 4116 chars, so every
codex-cli failure reported Codex's banner and the start of our system
prompt and never the reason (a 401, an unknown model, a rate limit). The
echo also carries the session digest, which does not belong in an error
message.
`_failure_detail` now takes the prompt that was sent on stdin. When
stderr contains it, the echo is replaced with `[prompt omitted]` and the
end of what the CLI printed after it is kept, cut at a line boundary,
inside the same 2000-char cap. Stderr from a CLI that does not echo the
prompt is handled exactly as before.
Closes #3927
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/analyzer.py`: `_failure_detail` gets a keyword-only
`prompt` argument; when the prompt (newlines normalized the way a
text-mode pipe reads them back) appears in stderr, it is replaced with a
marker and the text after it is tailed to fit the cap, whole lines where
possible.
- Both callers pass the prompt they wrote to stdin: the `run(...)` path
used by codex-cli and gemini-cli, and the claude-cli streaming path
(claude does not echo today, so nothing changes there).
- `tests/test_learn/test_analyzer.py`:
`test_codex_failure_drops_prompt_echo_and_keeps_reason` feeds stderr in
the shape codex-cli 0.156.1 prints (banner, `user`, the prompt, 100
reconnect lines, the final error) and checks the message ends with the
real reason, carries the marker, has no digest text, and stays inside
the cap.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
Ran on the changed files only (`ruff check`/`ruff format --check` on the
two changed files, `mypy headroom/learn/analyzer.py`), not the whole
tree.
### Test Output
Before the fix (new test against upstream/main's analyzer):
```text
$ pytest tests/test_learn/test_analyzer.py -q -k prompt_echo
E AssertionError: assert False
E + where False = <built-in method endswith of str object at 0x77522a6400>('ERROR: unexpected status 401 Unauthorized: Missing bearer')
E + where <built-in method endswith of str object at 0x77522a6400> = '`codex exec --skip-git-repo-check` failed (exit 1):\nReading prompt from stdin...\nOpenAI Codex v0.156.1\n--------\nm...endation (how many tokens would be saved per session if this rule existed)\n- Separate stable project facts (CONTEXT_F'.endswith
```
After:
```text
$ pytest tests/test_learn/test_analyzer.py tests/test_learn/test_error_classification.py -q
============================= 102 passed in 4.73s ==============================
$ ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
All checks passed!
$ ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
2 files already formatted
$ mypy headroom/learn/analyzer.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS 27, Python 3.12, codex-cli 0.156.1 (Homebrew),
upstream/main
|
||
|
|
2a4d34c476 |
fix(learn): run the claude-cli analysis with hooks disabled (#3926)
## Description
`headroom learn`'s claude-cli backend runs `claude -p` as the user, so
the user's hooks (from settings files and installed plugins) run inside
the analysis session. #3892 took the session's tools away; this takes
its hooks away. A blocking Stop hook makes Claude take a second turn
whose reply becomes the final `result`, replacing the analysis, so learn
finds no rules and prints "No actionable patterns found." with exit 0
and no error.
The fix passes `--settings '{"disableAllHooks":true}'`. `--settings` is
merged on top of the user's own settings, which still load, so env
routing (`ANTHROPIC_BASE_URL` in settings) and OAuth keep working.
`--bare` would also skip hooks but never reads OAuth or the keychain.
Closes #3925
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/analyzer.py`: add `--settings
'{"disableAllHooks":true}'` to the claude-cli entry in `_CLI_BACKENDS`,
with a comment on why hooks must be off and why `--bare` is not used.
- `tests/test_learn/test_analyzer.py`:
`test_claude_cli_streams_and_parses_result_event` pins the new argv.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
Ran on the changed files only (`ruff check`/`ruff format --check` on the
two changed files, `mypy headroom/learn/analyzer.py`), not the whole
tree. The existing argv test was updated rather than a new one added.
### Test Output
Before the fix (updated test against upstream/main's analyzer):
```text
$ pytest tests/test_learn/test_analyzer.py -q -x -k test_claude_cli_streams_and_parses_result_event
E AssertionError: assert ['claude', '-...essages', ...] == ['claude', '-...essages', ...]
E Right contains 2 more items, first extra item: '--settings'
FAILED tests/test_learn/test_analyzer.py::TestCallCliLlm::test_claude_cli_streams_and_parses_result_event
======================= 1 failed, 96 deselected in 3.35s =======================
```
After:
```text
$ pytest tests/test_learn/test_analyzer.py tests/test_learn/test_error_classification.py -q
============================= 101 passed in 4.19s ==============================
$ ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
All checks passed!
$ ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
2 files already formatted
$ mypy headroom/learn/analyzer.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS 27, Python 3.12, Claude Code 2.1.287 logged in with
a subscription (OAuth), upstream/main
|
||
|
|
2e74f06ece |
fix(learn): recognize escaped persisted pattern IDs (#3951)
## Description Fix the main CI shard-2 failure introduced by #2293. The writer saves pattern-ID comments as escaped text, but its ID parser only recognized raw comments. After a real write/read cycle, expired categories remained forever, active unbatched items could disappear, and repeated IDs could keep stale text. Recognize both raw incoming annotations and the escaped annotations already persisted by the writer. Content remains sanitized; no HTML comments are restored and managed-block boundary protection is unchanged. Failing main job: https://github.com/headroomlabs-ai/headroom/actions/runs/37097682536/job/111131105546 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - Recognize persisted escaped pattern-ID annotations in the existing parser. - Add real write/read regressions for retaining active unbatched items, removing expired items, and replacing stale text for an existing ID without a lifecycle signal. - Keep the existing category-removal regression and managed-block injection defenses intact. - Explicitly test LF and CRLF saved files and mark those cases for Windows newline coverage. ## Testing - [x] Focused unit tests pass - [x] Linting passes on changed files - [x] Type checking passes on the changed writer module - [x] New tests added for the reproduced failures - [x] Manual reproduction performed ### Test Output ```text Before: test_section_lifecycle.py: 3 failed, 4 passed (original main failure plus both new round-trip regressions) After: lifecycle + writer + traffic learner + managed-block injection: 231 passed, 2 deselected (two confirmed pre-existing Windows path tests excluded) Broader tests/test_learn + managed-block injection: 354 passed, 1 skipped, 2 failed (both failures reproduced on unchanged main; details below) ruff check: All checks passed! ruff format --check: 2 files already formatted mypy headroom/learn/writer.py --follow-imports=silent: Success: no issues found in 1 source file git diff --check: clean ``` All four unrelated Windows failures were reproduced in an isolated unchanged-main checkout at |
||
|
|
1631cde686 |
fix(learn): preserve stable traffic pattern items (#2293)
## Description Fixes #2294. Follow-up to #231. TrafficLearner can re-surface a learned section with only a subset of previously written patterns. The writer currently replaces a same-named section as one unit, so a valid prior preference, environment fact, or architecture rule can disappear even though it was not contradicted. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add opt-in same-section markdown-item preservation to Recommendation. - Render stable pattern IDs for TrafficLearner preference, environment, and architecture bullets. - Prefer new text when a stable ID reappears and deduplicate legacy ID-less bullets by visible content. - Keep error-recovery sections authoritative so expiry and refinement still remove stale entries. - Leave every other Headroom Learn analyzer on the existing same-section replacement behavior. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text 170 focused writer and TrafficLearner tests passed full tests/test_learn plus TrafficLearner run completed without failures ruff check: passed ruff format --check: passed ``` ## Real Behavior Proof - Environment: macOS, Python 3.13, current Headroom main. - Exact command / steps: write a prior Learned: preference section containing a stable-ID item and a legacy ID-less item, merge a new recommendation with the same section and stable ID, then inspect the rebuilt marker block. - Observed result: new text wins for the repeated stable ID, the unrelated prior bullet remains, and the legacy duplicate appears only once. Error-recovery recommendations retain replacement behavior. - Not tested: full repository suite, live proxy traffic, or migration of arbitrary non-list markdown sections. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [x] I have commented code where the behavior is not self-explanatory - [x] I have made corresponding documentation changes - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused unit tests pass locally - [x] I have updated CHANGELOG.md if applicable ## Screenshots (if applicable) Not applicable. ## Additional Notes Draft for maintainer feedback on the opt-in merge boundary and stable-ID representation. Already persisted incorrect rules are intentionally not deleted automatically. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
db90b93da9 |
fix(learn): run the claude-cli analysis with no tools (#3892)
## Description `headroom learn`'s claude-cli backend runs the analysis as `claude -p`, and a headless `claude -p` is a full Claude Code agent. With Claude Code 2.1.285, its init event lists 127 tools from 7 MCP servers: Bash, Edit, Task, cron, every claude.ai connector. They run under the user's own permissions, hooks and skills. The analysis needs none of them, because the digest is the whole input and the answer is one JSON object. When the model decides to go read or verify things, the run turns multi-turn. Each tool call emits stream events, so the idle cap never fires, and the run is killed at the hard cap with nothing written. We saw this in the field as `claude -p --output-format stream-json --verbose exceeded the 900s hard cap` (Headroom Desktop raises the hard cap to 900s). A normal analysis finishes in seconds to under a minute in a single turn. This adds `--tools ""` and `--strict-mcp-config` to the claude-cli command, so the session has zero tools and answers in one turn. You need both flags: `--tools ""` alone removes the built-in tools but still leaves every MCP server's tools. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/analyzer.py`: the `claude-cli` entry of `_CLI_BACKENDS` now also passes `--tools "" --strict-mcp-config`, with a comment explaining why. - `tests/test_learn/test_analyzer.py`: `test_claude_cli_streams_and_parses_result_event` pins the new command. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_learn/ ======================= 320 passed, 4 skipped in 22.78s ======================== $ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! $ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py 2 files already formatted $ uv run --frozen --extra dev mypy headroom/learn/analyzer.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS 15 (Darwin 24.6), Claude Code 2.1.285 logged in with OAuth, user settings with plugins, hooks and claude.ai MCP connectors, this branch run through `uv run`. - Exact command / steps: (1) Piped `hi` into the old and the new claude-cli command and read the `system`/`init` stream event. (2) Called `_call_cli_llm(<small two-call digest>, "claude-cli")` against the real CLI. - Observed result: (1) The old command's init event listed 127 tools and 7 MCP servers; the new command's listed 0 tools and 0 MCP servers. (2) The analysis returned parsed JSON with `context_file_rules` and `memory_file_rules` in 8.5s. - Not tested: Windows (the command is still a list passed to Popen, and the empty `--tools` value is a separate argv entry); Claude Code versions without `--tools`, which will fail with `unknown option '--tools'` and need a `claude update`. ## Runtime Rollout Safety - Rollout-managed feature(s): none; this only changes the learn claude-cli command. - Minimum rollout channel: N/A, not rollout-managed. - Stable/default behavior changed: yes, the learn analysis session runs with no tools. The recommendations come from the digest alone, as the prompt already asks. - Kill switch / disable path: `--model <litellm-model-name>`, or the gemini-cli or codex-cli backends, skip this command entirely. - Unsafe override required: no. - Qualification impact: none; learn is not in the proxy request path. - Rollback path: revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Documentation: no docs describe the claude-cli argv, so there was nothing to update. - The existing command test was updated rather than a new one added; it asserts the exact argv. - Stop hooks that block can still make a `claude -p` run multi-turn. Only Claude Code's `--safe-mode` disables hooks under OAuth (`--bare` needs an API key), and that flag is too new to pass unconditionally. - Headroom Desktop carries this as a sitecustomize patch until a wheel with it ships. The patch does nothing when the wheel already passes either flag. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
c072251c10 |
fix(learn): keep CLAUDE.local.md out of git via .git/info/exclude (#3108)
## Description
`headroom learn` writes per-project learnings to `CLAUDE.local.md`
rather than the team-shared `CLAUDE.md` (#1115, for #1072). The premise
is that `CLAUDE.local.md` is personal and gitignored — `writer.py` says
so in its own docstring — but nothing makes that true. Git ships no
default rule for the name, Claude Code does not add one, and no ignore
rule is written at learn time. The file is therefore swept up by the
next `git add -A`, and the machine-specific absolute paths inside it
reach teammates anyway. The pollution #1072 set out to prevent just
moved one filename over.
This is item 3 of #1070 ("Add `*.local.md` to the gitignore template"),
the half that never shipped. The codebase already half-acknowledges the
gap: `writer.py:317` builds a `gitignore_hint` telling users to add the
rule by hand, but it is only attached to the legacy-`CLAUDE.md`
migration warning, so users who never had a legacy block never see it.
Closes #1070
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Add `_ensure_git_ignored()`, which registers the written filename in
`.git/info/exclude` after a non-dry-run write.
- Use `.git/info/exclude` rather than the repo's `.gitignore`: the
exclude file is per-clone and untracked, so Headroom never leaves an
unexpected diff in a team-shared file. Writing to `.gitignore` would
trade one kind of pollution for another.
- Resolve the exclude path through `git rev-parse --git-common-dir` so
linked worktrees share the entry.
- Gate on `git check-ignore` so a rule the user already has (or one we
wrote on a previous run) is left alone and never duplicated.
- Emit a `WriteResult.warnings` entry instead when the file is **already
tracked**: git honors no ignore rule for tracked paths, so only `git rm
--cached` fixes it, and running that here would silently stage a
deletion in the user's repo.
- Gate the whole thing on `target_path.name.endswith(".local.md")`, so
an explicit `--target CLAUDE.md` (a deliberate opt-in to the shared
file) and the home-directory `~/.claude/CLAUDE.md` case are untouched.
The latter matters because `~/.claude` is often itself a dotfiles repo.
- Best-effort by design: no git on PATH, not a repo, or a read-only
`.git` all no-op rather than failing the write.
The entry is deliberately unanchored (`CLAUDE.local.md`, not
`/CLAUDE.local.md`). A project may be a subdirectory of a larger repo,
so anchoring would require the path relative to the repo root; and a
`CLAUDE.local.md` at any depth is personal by the same argument.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
New tests drive real `git` in a real temp repo rather than mocking it —
the behavior under test *is* git's, and a mocked `check-ignore` would
assert nothing.
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_learn/test_writer.py
tests/test_learn/test_writer.py::TestClaudeLocalMdStaysOutOfGit::test_apply_adds_exclude_entry PASSED [ 83%]
tests/test_learn/test_writer.py::TestClaudeLocalMdStaysOutOfGit::test_second_run_does_not_duplicate_the_entry PASSED [ 86%]
tests/test_learn/test_writer.py::TestClaudeLocalMdStaysOutOfGit::test_existing_gitignore_rule_is_left_alone PASSED [ 89%]
tests/test_learn/test_writer.py::TestClaudeLocalMdStaysOutOfGit::test_tracked_file_warns_instead_of_excluding PASSED [ 91%]
tests/test_learn/test_writer.py::TestClaudeLocalMdStaysOutOfGit::test_dry_run_does_not_touch_exclude PASSED [ 94%]
tests/test_learn/test_writer.py::TestClaudeLocalMdStaysOutOfGit::test_explicit_shared_target_is_never_excluded PASSED [ 97%]
tests/test_learn/test_writer.py::TestClaudeLocalMdStaysOutOfGit::test_outside_a_git_repo_is_a_no_op PASSED [100%]
============================== 37 passed in 2.53s ==============================
$ uvx ruff check headroom/learn/writer.py tests/test_learn/test_writer.py
All checks passed!
$ uvx ruff format --check headroom/learn/writer.py tests/test_learn/test_writer.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/learn/writer.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS 15.6 (darwin 24.6.0), Python 3.12, git 2.51.0,
worktree off `upstream/main` @
|
||
|
|
b10dd8db6e |
fix(memory): pin LF on memory writers and guard the learn-writer newline contract (#3706)
Closes #3698. Follow-up to #3594, which fixed `headroom/learn/writer.py` correctly but left three gaps. ## 1. The write half had no regression guard `test_apply_does_not_accumulate_carriage_returns_on_crlf_file` passes on `main` with the fix reverted — its assertion (`raw.count(b"\r") == raw.count(b"\r\n")`) holds for a CRLF-preserving file too. So the `newline="\n"` pins, which *are* the fix, were unguarded. This cannot be caught by asserting on file bytes on POSIX: CPython's C `TextIOWrapper` picks the newline translation target at compile time (`#ifdef MS_WINDOWS`), so monkeypatching `os.linesep` does not simulate Windows. The new test spies on `Path.write_text` and drives the real writer `.write()` methods. Stripping the kwargs from all seven sites: ``` tests/test_learn/test_writer.py ................................F E AssertionError: learn writers wrote without newline='\n': [ E .../data/memory/MEMORY.md, .../instructions.md, .../myproject/AGENTS.md, E .../myproject/CLAUDE.local.md, .../myproject/CLAUDE.md, E .../myproject/GEMINI.md, .../myproject/GROK.md] =================== 1 failed, 32 passed =================== ``` All seven named — and note the other 32 pass, including the old test, confirming it was never a guard. ## 2. The same bug was still live in the memory writers Ten call sites, all nine from the issue plus **`headroom/memory/sync.py:137`** (`_save_sync_state`), which the issue missed: `writers/base.py:168` · `writers/claude_writer.py:119` · `writers/cursor_writer.py:130` · `bridge.py:126, 394, 650` · `sync.py:137` · `sync_adapters/codex_agent.py:124` · `sync_adapters/claude_code.py:196, 239` These write files that **overlap** with the learn writers (`~/.claude/CLAUDE.md`, `AGENTS.md`), so on Windows a file flipped between LF and CRLF depending on which subsystem wrote last. **No live accumulation bug:** every reader of these files uses `read_text()`, which normalizes — that is the key difference from `learn/writer.py`'s `read_bytes().decode()`, which is what let `\r` pile up. Checked all of them (`bridge.py:114, 222, 617`, `writers/base.py:193`, `writers/cursor_writer.py:103`, `sync_adapters/claude_code.py:106, 163, 178, 216`, `sync_adapters/codex_agent.py:48, 83`, `sync.py:127`). **But the churn is more than cosmetic**, which the issue did not note: `claude_code.py:252` and `codex_agent.py:135` read via `read_bytes()` for `fingerprint()` change-detection hashing. An EOL flip changes the hash, so a CRLF/LF flip reads as a content change and triggers a spurious re-sync. Three new tests cover the writer, bridge and sync-adapter paths; all three fail with the pins stripped. ## 3. No unit test had ever run on Windows The sharded `test` job is ubuntu-only, and `build-wheel-windows` builds without testing — which is how a Windows-only bug shipped twice. Added `test-windows-newline`: consumes the existing `headroom-wheel-windows` artifact and runs only tests marked `windows_newline`. Marker registered in `pyproject.toml`. **Known limitation, documented in the job comment:** it is gated on the `packaging` paths-filter (that is what builds the wheel), so a pure-Python PR skips it. The ubuntu shards still run the same marked tests — they assert the kwarg, which fails on any platform — so this is the Windows-side backstop, not the only guard. Making it fire on every writer change would mean gating the Windows Rust wheel build on `code`, which is too expensive to do unasked. ## Verification - 4 guards proven to fail before, pass after. - `tests/test_learn/ tests/test_memory/ tests/test_memory_sync.py` → **950 passed, 4 skipped**. - Adjacent suites → 168 passed. `pytest tests -m windows_newline` → 4 passed, no marker warning. - `actionlint .github/workflows/*.yml` → exit 0. Action versions match the file's conventions (`checkout@v7`, `setup-python@v7`, `download-artifact@v8`). `CONTRIBUTING.md` states the invariant once under Coding standards: Headroom writes LF and normalizes on read, on every platform. ## Not in this PR Issue section 3's release-note line about the one-time EOL churn on Windows (the first `--apply` rewrites the whole file to LF). Harmless under `autocrlf=true`, but worth a release note; `CHANGELOG.md` edits are blocked by `changelog-guard.yml`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
ff60d5764c |
fix(learn): treat an unreadable project memory dir as absent instead of crashing (#3710)
## Description
`headroom learn` aborts with a traceback when `~/.claude/projects/`
contains a project directory the user cannot traverse. Field report
(Headroom desktop 0.9.18, macOS):
```text
File ".../headroom/learn/plugins/claude.py", line 91, in discover_projects
memory_file = memory_dir / "MEMORY.md" if memory_dir.exists() else None
PermissionError: [Errno 13] Permission denied: '~/.claude/projects/-/memory'
```
The `-` entry is the project dir for `/`, left behind by a session
Claude Code ran as root. The entry itself stats fine (its parent is the
user's own `projects/`), so `entry.is_dir()` passes, but nothing under
it is traversable and the first stat below it raises. #2446 guarded the
decoded project path and `CLAUDE.md` probes with `path_exists` for
exactly this reason; the `memory` probe two lines later was still a bare
`Path.exists()`.
This PR routes the two memory probes through the same `path_exists`
helper, so an unreadable project dir contributes no memory file and
discovery continues, instead of taking the whole scan down.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/plugins/claude.py`: `discover_projects` probes
`<entry>/memory` and `<entry>/memory/MEMORY.md` with `_path_exists`
instead of `Path.exists()`, matching the project-path and `CLAUDE.md`
probes above it.
- `tests/test_learn/test_plugin_permission_errors.py`:
`test_claude_discover_projects_survives_unreadable_memory_dir` builds a
`projects/-/` entry whose `memory` subtree raises `PermissionError` on
stat and asserts discovery still returns the project with
`memory_file=None`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_learn/test_plugin_permission_errors.py -q
collected 6 items
tests/test_learn/test_plugin_permission_errors.py ...... [100%]
6 passed in 0.75s
# With the fix reverted (git show HEAD~1:headroom/learn/plugins/claude.py), the new test fails:
E PermissionError: [Errno 13] Permission denied: '.../projects/-/memory'
1 failed, 5 passed in 0.26s
$ uvx ruff check headroom/learn/plugins/claude.py tests/test_learn/test_plugin_permission_errors.py
All checks passed!
$ uvx ruff format --check headroom/learn/plugins/claude.py tests/test_learn/test_plugin_permission_errors.py
2 files already formatted
```
## Real Behavior Proof
- Environment: macOS 15, Python 3.12, worktree of `main` at
|
||
|
|
abde3e6e70 |
fix(learn): prevent Windows CRLF accumulation in context writers (#3594)
## Description On Windows, context file updates in `headroom.learn.writer` cause carriage return accumulation. Python's `Path.write_text()` defaults to `newline=None`, which converts `\n` to `\r\n`. Because `_read_text_tolerant()` decodes bytes directly without normalizing existing `\r\n` endings, the `\r` remains in the in-memory string. On subsequent writes, `\n` is converted to `\r\n` again, resulting in `\r\r\n` -> `\r\r\r\n`. Over multiple learning iterations, this leads to thousands of orphan `\r` characters, causing markdown context and memory files (such as `MEMORY.md`) to balloon into tens of thousands of blank lines. This PR normalizes line breaks in `_read_text_tolerant()` and specifies `newline="\n"` on all `write_text()` calls in context writers so line endings are preserved cleanly regardless of host platform. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Normalizes input text in `_read_text_tolerant()` (`\r\n` and standalone `\r` -> `\n`). - Passes `newline="\n"` to all `write_text()` calls across `ClaudeCodeWriter`, `CodexWriter`, `GeminiWriter`, and `GrokWriter`. - Adds regression tests in `tests/test_learn/test_writer.py` verifying CRLF files do not accumulate carriage returns on repeated applies. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text tests/test_learn/test_writer.py::TestEncodingResilience::test_read_text_tolerant_normalizes_crlf_and_cr PASSED tests/test_learn/test_writer.py::TestEncodingResilience::test_apply_does_not_accumulate_carriage_returns_on_crlf_file PASSED ======================== 32 passed, 1 warning in 1.28s ======================== ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.14 - Exact command / steps: Run writer repeatedly over an existing context file with Windows CRLF endings. - Observed result: Prior to fix, orphan `\r` count doubled on every rewrite. With this fix, byte count and line count remain stable and clean LF across consecutive runs. - Not tested: N/A ## Runtime Rollout Safety - Rollout-managed feature(s): None - Minimum rollout channel: dev - Stable/default behavior changed: No - Kill switch / disable path: N/A - Unsafe override required: No - Qualification impact: None - Rollback path: Revert commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
f302a38e8f |
fix(learn): distinguish Grep and Glob searches by path (#3455)
## Description Fixes #3453. `headroom learn` grouped native Grep and Glob calls by pattern alone, so the same pattern in different directories appeared as one loop. This includes the search path in the existing summary used for loop signatures and samples, so separate searches remain separate while repeated searches in one directory still form a loop. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Include `path` alongside `pattern` for Grep and Glob input summaries. - Add regression coverage for separate paths, repeated same-path searches, and the generic summary fallback. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_learn/test_loop_weighting.py -q 16 passed in 0.12s uv run ruff check headroom/learn/models.py tests/test_learn/test_loop_weighting.py All checks passed! uv run ruff format --check headroom/learn/models.py tests/test_learn/test_loop_weighting.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS on Apple Silicon, Python 3.13.14. - Exact command / steps: created one `SessionData` session with three native Grep or Glob calls using one pattern and `/srv/ingest`, `/srv/billing`, and `/srv/web`, then ran `detect_loops`. - Observed result: before, each tool produced one signature and one three-call loop with 20,000 attributed tokens. After, each produces three path-qualified signatures and no loop; three calls with the same path still produce one loop. - Not tested: full suite and scanning a retained production transcript. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: Standard release. - Stable/default behavior changed: Grep and Glob searches with the same pattern in different directories no longer merge into one loop. - Kill switch / disable path: Revert this change. - Unsafe override required: No. - Qualification impact: Existing `headroom learn` loop reports become path-specific for native Grep and Glob calls. - Rollback path: Revert this change; pathless grouping resumes. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas (N/A: the summary format is direct) - [ ] I have made corresponding changes to the documentation (N/A: no public API or documentation change) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) |
||
|
|
f6b9fc09e3 |
fix(learn): derive loop signature from tool input identity, not display summary (#3461)
## Description
`headroom learn`'s loop detector groups tool calls by
`ToolCall.input_summary`, a helper
whose own docstring says it is a *"Short summary of tool input for
display."* It truncates
Bash commands at 100 characters (`models.py:67`) and renders any
non-builtin tool as
`str(input_data)[:80]` (`models.py:76`). Using a deliberately lossy
display value as an
*identity* key means calls that differ only after the cutoff compare
equal and merge into
one phantom loop, so `learn` reports loops that never happened and
attributes real tokens
to them.
A second, narrower form of the same collapse: `normalize_tool_name()`
renames provider
tools onto the builtin names — `codebase_search` becomes `Grep`,
`read_many_files` becomes
`Read` — without normalizing their *input*. `input_summary` then looks
for `pattern` or
`file_path`, finds neither, and returns its `"?"` placeholder for all of
them, so N
unrelated searches share the signature `grep::?` and merge into one
loop. This one needs no
long input; the calls collide at any length.
The estimation math is not at fault — it sums real `output_bytes`
correctly. The fault is in
the partition it sums over.
This is one of two PRs splitting #3440. The replay double-counting
originally filed with it
is a different mechanism and ships separately against #3452; the two are
independent and
either can land first. They conflict textually in `detect_loops` — this
PR renames the merge
loop's variable, that one rewrites its body — so whichever lands second
needs a ~6-line
rebase. Happy to do it either way round.
Closes #3440
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/loops.py`: added `_IDENTITY_FIELDS` and
`_identity_input()`. The loop key
now comes from the untruncated input — the identifying field per builtin
tool
(`command` / `file_path` / `pattern`), and key-sorted `json.dumps` for
everything else —
instead of `input_summary`.
- `headroom/learn/loops.py`: `_identity_input()` falls back to the
key-sorted JSON
whenever the per-tool fields yield *nothing*, not only when the tool is
absent from
`_IDENTITY_FIELDS`. Without this, a builtin name carrying a foreign
schema keys on an
empty string and every such call merges into one loop — the same defect,
reintroduced in
the new code. `Grep` keyed on a *present* `pattern` still ignores
`path`, so the separate
defect now filed as #3453 is left exactly as found. The same branch
means an unexpected
input shape returns a signature rather than raising: identity is derived
from files the
user did not write.
- `headroom/learn/loops.py`: identity and the *stored* signature are
kept separate.
Groups are keyed on a `blake2b` digest of the full signature, so the
grouping tables
stay a fixed size per loop no matter how large the input is.
`LoopPattern.signature`
keeps its body capped at the width `input_summary` previously imposed.
That cap is
load-bearing: `apply_loop_weighting` requires a *majority* of the
signature's tokens to
appear in a recommendation, so an untruncated signature puts the
threshold out of reach
for long inputs and silently costs a real loop its measured-waste boost.
I hit this and
added regression tests for it — see Additional Notes.
- `headroom/learn/loops.py`: whitespace collapsing switched from
`re.sub(r"\s+", " ", raw)`
to `" ".join(raw.split())`. Same output, roughly 4x cheaper on the
now-untruncated input.
See Performance for where the remaining cost actually sits — it is not
here.
- The existing pagination / bare-int collapsing still runs on top of the
new key, so
re-fetch variants of one command keep collapsing exactly as before.
- `tests/test_learn/test_loop_weighting.py`: 11 new tests, 17 cases with
parametrization.
I ran them against `upstream/main` rather than assume the split: **7
fail there and pass
here**; the other 10 pin behavior that must survive — pagination and
>100-char command
variants must still collapse, whitespace variants must still fold
together, detection
must not depend on call order, and weighting must hold at four input
sizes. The weighting
and signature-size ones guard against a mistake I made building this
(see Additional
Notes) rather than against the reported bug, so they pass on `main` too.
`ToolCall.input_summary` itself is untouched and still used for display
(`sample_input`,
digest lines), so digest output formatting does not change.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest -q tests/test_learn/
264 passed, 4 skipped
$ pytest -q tests/test_learn/test_loop_weighting.py
30 passed # 13 pre-existing + 17 added by this PR
$ ruff check .
All checks passed!
$ ruff format --check headroom/learn/ tests/test_learn/
30 files already formatted
$ mypy headroom
Found 1 error in 1 file (checked 530 source files)
# The one error is pre-existing and in a file this PR does not touch:
# headroom/integrations/langchain/chat_model.py:394 [truthy-function]
# Zero errors in headroom/learn/loops.py.
$ pytest -q tests/test_learn/ \
--cov=headroom.learn.loops --cov=headroom.learn.fixtures --cov-report=term-missing
Name Stmts Miss Branch BrPart Cover Missing
headroom/learn/fixtures.py 27 3 10 1 84% 33-36
headroom/learn/loops.py 104 0 36 1 99% 296->288
# fixtures.py is untouched by this PR; its uncovered lines predate it.
# Every statement in loops.py is covered.
$ pytest -q # full suite, this branch
21 failed, 11916 passed, 583 skipped, 6044 warnings in 791.19s (0:13:11)
$ pytest -q # same suite, unpatched upstream/main @
|
||
|
|
4088a65bb7 |
fix(learn): count each tool call once per tool_call_id when detecting loops (#3462)
## Description
`headroom learn` counts the same tool call once per transcript file it
appears in.
`detect_loops()` merges each session's qualifying groups into a global
view keyed by
signature and never dedups by `tool_call_id`:
```python
for sig, calls in per_session.items():
if len(calls) >= min_occurrences:
groups.setdefault(sig, []).extend(calls) # <- no id dedup
```
When a provider resumes a conversation by writing a **new** transcript
that replays prior
history, every replayed call is scanned again and counted again. Three
reads become six and
the measured waste rises with them, though nothing about the agent's
behavior changed —
only the number of files on disk describing it. `ToolCall.tool_call_id`
already carries the
provider-assigned id and is already parsed by every scanner, so the
information needed to
tell a replay from a repetition is present and unused.
The inflated counts propagate: `detect_loops()` sorts by
`wasted_tokens`, and
`apply_loop_weighting()` scales by that same measured waste, so a loop
that is merely
replayed can outrank one that is real.
This is one of two PRs splitting #3440. The grouping-key truncation
originally filed with it
is a different mechanism — that one merges *distinct* calls because the
key is lossy, this
one over-counts calls that are correctly grouped and genuinely identical
— and it ships
separately against #3440. The two are independent and either can land
first. They conflict
textually in `detect_loops` — this PR rewrites the merge loop's body,
that one renames its
variable — so whichever lands second needs a ~6-line rebase. Happy to do
it either way round.
Closes #3452
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/loops.py`: `detect_loops()` counts a call once per
`tool_call_id` when
merging per-session groups into the global view. Calls carrying **no**
id are always
counted — nothing identifies them as replays, and inventing an identity
for them would
drop real repetitions.
- `headroom/learn/loops.py`: `min_occurrences` is re-checked **after**
dedup. The existing
check runs on the per-session group, so a group whose repetitions are
entirely replays of
one call clears the bar on replay count alone. Deduping without the
re-check would convert
a double-counted loop into a sub-threshold phantom rather than removing
it — the two
changes are not separable, which is why they are in one commit.
- `headroom/learn/loops.py`: the docstring now states that
`tool_call_id` must be unique
*across* sessions, not merely within one. This is a new constraint on
every current and
future scanner: where a plugin synthesizes ids, two unrelated sessions
running the same
loop would otherwise look like one session replayed twice — the same
defect in the
opposite direction. I checked the existing plugins rather than assume
it; both that
synthesize ids already scope them by session (`gemini.py:280`,
`opencode.py:250`), so no
scanner needed changing. The docstring records the requirement they
already satisfy.
- `headroom/learn/fixtures.py`: `_tc()` took `msg_index` only and built
`tool_call_id` as
`f"tc_{msg_index}"` — unique within a session, colliding across them.
Under the new dedup
that made two fixture sessions running the same loop dedup into one, so
the helper now
takes `session_id` and scopes the id. This is the cross-session contract
above, applied to
the one place in-tree that was violating it; two tests pin that the
fixtures satisfy it.
- `tests/test_learn/test_loop_weighting.py`: 10 new tests, 13 cases with
parametrization.
I ran them against `upstream/main` rather than assume the split: **9
fail there and pass
here**. The rest pin the boundaries the fix must not cross — a resume
that *continues* a
loop still accumulates, calls with no id stay counted, and the fixture
contract holds.
No public API, CLI flag, config key or digest format changes.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest -q tests/test_learn/
260 passed, 4 skipped
$ pytest -q tests/test_learn/test_loop_weighting.py
26 passed # 13 pre-existing + 13 added by this PR
$ ruff check .
All checks passed!
$ ruff format --check headroom/learn/ tests/test_learn/
30 files already formatted
$ mypy headroom
Found 1 error in 1 file (checked 530 source files)
# The one error is pre-existing and in a file this PR does not touch:
# headroom/integrations/langchain/chat_model.py:394 [truthy-function]
# Zero errors in headroom/learn/loops.py.
$ pytest -q tests/test_learn/ \
--cov=headroom.learn.loops --cov=headroom.learn.fixtures --cov-report=term-missing
Name Stmts Miss Branch BrPart Cover Missing
headroom/learn/fixtures.py 27 3 10 1 84% 39-42
headroom/learn/loops.py 92 0 40 1 99% 243->235
# The 4 uncovered fixtures.py lines are an unrelated branch that predates this PR.
# Every statement in loops.py is covered.
$ pytest -q # full suite, this branch
21 failed, 11912 passed, 583 skipped, 6044 warnings in 666.10s (0:11:06)
$ pytest -q # same suite, unpatched upstream/main @
|
||
|
|
cb3ab61d4d |
fix(learn): show both ends of CLI output on a parse failure (#3517)
## Description
`_call_cli_llm` and `_call_claude_cli_streaming` attach a HEAD-only
excerpt of
the CLI's output to their parse failures ("returned unparseable output.
First
2000 chars:", "did not emit a final `result` event"). For those two
failures
specifically, the head is the one part that is never diagnostic: a
model's JSON
answer that does not parse looks like valid JSON for its first 2000
chars in
every case. What separates a payload truncated mid-value from one with
prose
after the closing fence is the END of it, and that is exactly what gets
cut.
This is not hypothetical. Downstream (the Headroom desktop app) these
RuntimeErrors are reported with the CLI's stderr attached, and a real
report
arrived with 2000 characters of well-formed ```json ... and no way to
tell which
of the two shapes it was, so the defect could not be acted on at all.
`_failure_detail` already makes this argument for the non-zero-exit path
("stdout
is tailed rather than headed because CLI backends emit the error last").
This
extends the same reasoning to the parse failures, which need both ends:
the head
is where a wrapper (fence, preamble) shows up, the tail is where the
break is.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- Added `_output_snippet(text, limit=_MAX_SNIPPET_LEN)`: returns short
output
unchanged, otherwise `head + "...[N chars omitted]..." + tail`, still
bounded
by the same `_MAX_SNIPPET_LEN` budget.
- Used it at the three sites that excerpt CLI OUTPUT for a parse
failure: the
non-streaming unparseable stdout, the missing final `result` event, and
the
unparseable `result` text.
- Reworded those messages from "First 2000 chars:" to "Head and tail of
the
output:". Deliberately no counts in the marker line: downstream groups
these
errors by that line, so a length in it would open a new group per
failure.
- Left the stderr excerpts and `_failure_detail` alone.
## Testing
- [x] Unit tests pass (`pytest`)
### Test Output
```text
uv run --frozen --extra dev pytest tests/test_learn/ -q
260 passed, 4 skipped in 11.02s
```
## Real Behavior Proof
- Environment: macOS 15, Python 3.12, uv --frozen, headroom main @
|
||
|
|
60bfe8be93 |
fix(learn): guard session-derived path checks against PermissionError in gemini/grok plugins (#2522)
## Description Issue #2443 fixed a crash where `headroom learn` aborted with a raw traceback because `Path.exists()` on a decoded candidate path raised `PermissionError` (`os.stat` on a parent directory the user can't stat) instead of returning `False`. That fix added a local `_path_exists` guard to the **claude** plugin only. The **gemini**, **grok**, and **opencode** plugins have the exact same shape on paths recorded in session files / the opencode DB: ```python # gemini: _detect_project_path (outside the json-load try) if project_path and Path(project_path).exists(): # recorded projectPath / cwd ... # gemini: discovery if project_path and project_path.exists(): candidate = project_path / "GEMINI.md" if candidate.exists(): ... # grok: discovery context_file=grok_md if grok_md.exists() else agents_md if agents_md.exists() else None # opencode: discovery (worktree recorded in the SQLite DB) agents_md = Path(worktree) / "AGENTS.md" context_file=agents_md if agents_md.exists() else None ``` These paths come from session files that may have been written on another machine or by another user, so a recorded `cwd`/`projectPath` behind a restricted mount (or a decoded path colliding with another user's home) makes `.exists()` raise `PermissionError`, which nothing catches — aborting the whole `learn` run, exactly like #2443. ## Fix Promote the claude plugin's local `_path_exists` into a shared `headroom.learn.plugins._paths.path_exists` and use it for every session-derived existence check across the claude, gemini, grok, and opencode plugins. `path_exists` treats any `OSError` (including `PermissionError`) as "does not exist", so discovery skips the path instead of crashing. The claude plugin keeps its behavior (its `_path_exists` name is now an import alias of the shared helper, so the existing `TestDecodePermissionError` still passes). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/plugins/_paths.py` (new): shared `path_exists` helper. - `headroom/learn/plugins/claude.py`: import `path_exists` as `_path_exists` and drop the local copy (behavior unchanged). - `headroom/learn/plugins/gemini.py`: guard the `_detect_project_path` `projectPath`/`cwd` checks and the discovery `GEMINI.md` probe. - `headroom/learn/plugins/grok.py`: guard the `GROK.md` / `AGENTS.md` probes. - `headroom/learn/plugins/opencode.py`: guard the `AGENTS.md` probe under the DB-recorded worktree path. - `tests/test_learn/test_plugin_permission_errors.py` (new): regressions that `path_exists` swallows `PermissionError`, that gemini `_detect_project_path` returns `None` instead of raising, that grok `discover_projects` completes with `context_file=None`, and the same for opencode `discover_projects` over its SQLite DB. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_learn/test_plugin_permission_errors.py tests/test_learn/test_scanner.py::TestDecodePermissionError -q 7 passed # with the gemini/grok/opencode source reverted, those plugin tests fail with # PermissionError: [Errno 13] Permission denied $ uvx ruff@0.15.17 check headroom/learn/plugins/ tests/test_learn/test_plugin_permission_errors.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/learn/plugins/_paths.py headroom/learn/plugins/gemini.py headroom/learn/plugins/grok.py headroom/learn/plugins/claude.py Success: no issues found in 4 source files ``` One pre-existing, unrelated failure in `tests/test_learn/test_scanner.py::TestDecodeProjectPath::test_home_dir_username_stays_single_component` fails identically on clean `main` on Windows (it decodes a `/home/...` POSIX path and asserts a non-`None` result); it is not touched by this change. ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: monkeypatched `Path.exists` to raise `PermissionError` and called `path_exists`, `GeminiPlugin(gemini_dir=tmp)._detect_project_path(session_with_projectPath)`, `GrokPlugin(grok_dir=tmp).discover_projects()`, and `OpenCodePlugin(db_path=db).discover_projects()` (each with a real session tree / DB on disk); then reverted `gemini.py` / `grok.py` / `opencode.py` and re-ran. - Observed result: with the fix `path_exists` returns `False`, gemini `_detect_project_path` returns `None`, and grok/opencode `discover_projects` return one project with `context_file=None`; with the fix reverted the gemini, grok, and opencode cases raise `PermissionError: [Errno 13] Permission denied`. Ran against the actual modules via `tests/test_learn/test_plugin_permission_errors.py`. - Not tested: a real cross-user / restricted-mount filesystem end to end (the PermissionError is injected). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable --------- Co-authored-by: Jerrett Davis <mxjerrett@gmail.com> |
||
|
|
4c6bd3e8fc |
fix(learn): echo live progress during claude-cli analysis (#3158)
## Description `headroom learn` already streams claude-cli's `stream-json` output internally, using it to drive an idle-timeout watchdog, but none of that liveness reaches stdout. Wrapper UIs that tail the CLI output (the Headroom desktop app included) see nothing print between "Analyzing with claude-cli..." and the final result, sometimes for minutes, which makes the scan look hung. This closes #3105 by surfacing that liveness as short, throttled progress lines. ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) ## Changes Made - Added an optional `on_progress: typing.Callable[[str], None] | None = None` callback, threaded through `SessionAnalyzer.analyze` -> `_call_llm` -> `_call_cli_llm` -> `_call_claude_cli_streaming`. It defaults to `None` at every layer, so no existing caller changes behavior. - Inside the streaming event loop, added `_progress_detail()`, which maps each non-terminal `stream-json` event (`system`/`assistant`/`stream_event`/`user`) to a short phrase like `"assistant responding, 12s"`. The terminal `result` event is handled separately and is never echoed as progress. - Throttled the callback to at most once per 3 seconds (`_PROGRESS_THROTTLE_SECS`) via the same `time.monotonic()` idiom the function already used for its idle-timeout watchdog, so a burst of partial-message events doesn't spam the console. - The callback is invoked defensively: any exception it raises is caught and logged via `logger.debug`, never propagated, so a broken UI pipe can't abort an otherwise-successful analysis. - `headroom/cli/learn.py` supplies the one concrete callback, reusing the exact existing `" Analyzing with {model}..."` prefix with `(detail)` appended, so any wrapper UI that whitelists that prefix keeps working unchanged. - Added `--include-partial-messages` to the claude-cli invocation, as suggested in the issue. Without it, claude only emits about three events for an entire run (session init, one assistant message, result), which is too sparse for the throttle to produce useful incremental updates. ## Testing - [x] Added new tests - [x] All tests pass locally ``` $ .venv/Scripts/python.exe -m pytest tests/test_learn/test_analyzer.py tests/test_cli_learn.py -q tests\test_learn\test_analyzer.py ....................................... ...................................................... [ 84%] tests\test_cli_learn.py ................. [100%] 109 items collected, 108 passed, 1 failed FAILED tests/test_learn/test_analyzer.py::TestDigestBuilder::test_includes_project_info (pre-existing, unrelated: a Windows-path-separator assertion that also fails identically on a clean upstream/main checkout, before this change) $ .venv/Scripts/ruff.exe check headroom/learn/analyzer.py headroom/cli/learn.py tests/test_learn/test_analyzer.py tests/test_cli_learn.py All checks passed! $ .venv/Scripts/ruff.exe format --check headroom/learn/analyzer.py headroom/cli/learn.py tests/test_learn/test_analyzer.py tests/test_cli_learn.py 4 files already formatted $ .venv/Scripts/mypy.exe headroom/learn/analyzer.py headroom/cli/learn.py Success: no issues found in 2 source files ``` Three new tests were added covering: a burst of events within one throttle window only echoing the first one, spaced-out events each mapping to the correct phrase in order (with the terminal `result` event never echoed), and a raising `on_progress` callback not aborting a successful analysis. A fourth new test at the CLI layer asserts the `" Analyzing with ... (detail)"` lines appear in `headroom learn` output. ## Real Behavior Proof - Environment: Windows 11, Python 3.13.11, local `.venv`, no live `claude` CLI invoked (tests mock `subprocess.Popen`, matching the existing test style in this file) - Exact command / steps: `.venv/Scripts/python.exe -m pytest tests/test_learn/test_analyzer.py tests/test_cli_learn.py -v -k progress` - Observed result: all 4 new progress-related tests pass, confirming the throttling, event-to-phrase mapping, defensive-callback behavior, and the CLI's rendered output line all work as designed - Not tested: an actual live `claude` CLI process (no `claude` binary available in this sandboxed environment); the fix only changes what data is threaded through an existing, already-live streaming loop, so this is low risk, but a manual check against the real CLI before/after would be good follow-up validation ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: no — `on_progress` defaults to `None` everywhere, and when `None` the code path is unchanged from before this PR - Kill switch / disable path: none needed; the only new default-on behavior is the extra progress lines in `headroom learn` CLI output, which is additive to existing output (same prefix, one line appended per event) and does not change any parsed/machine-readable output - Unsafe override required: no - Qualification impact: none — no changes to routing, caching, or backend selection logic - Rollback path: revert this commit; nothing persists state or changes on-disk formats ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Jerrett Davis <mxjerrett@gmail.com> |
||
|
|
d329a04b4b |
fix(learn): retry LLM analysis with a smaller digest when the prompt overflows the context window (#3404)
## Description
`headroom learn` dies with zero recommendations when the analysis prompt
overflows the model's context window. Field report (desktop telemetry):
a 4-session / 443-call scan failed with ``LLM analysis failed: `claude
-p --output-format stream-json --verbose` failed (exit 1): Prompt is too
long``, and the whole project's analysis was lost.
The root cause is that `_build_digest` budgets `_MAX_DIGEST_TOKENS`
(80k) for the digest as if the prompt had the window to itself. CLI
backends do not work that way: `claude -p` loads the user's own context
on top - CLAUDE.md files, MCP tool definitions, skills. The digest
budget cannot know how big that share is, and the 4-chars-per-token
estimate also runs hot on dense tool-call lines. On installs with heavy
context the combined prompt overflows a 200k window and the analysis
fails outright, every scan, until the user's sessions age out.
The fix makes the budget adaptive instead of guessing a universally safe
constant: when the backend reports a context overflow, rebuild the
digest at half the budget and retry, down to a 10k-token floor (80k ->
40k -> 20k -> 10k, then give up). A truncated digest that produces
recommendations beats a full digest that produces an error. Non-overflow
failures are re-raised on the first attempt exactly as before.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `SessionAnalyzer.analyze` wraps the LLM call in a shrinking-budget
retry loop; overflow detection via new `_is_prompt_too_long(exc)`, which
matches the known phrasings across backends: claude CLI ("Prompt is too
long"), Anthropic via LiteLLM ("prompt is too long: N tokens > M
maximum"), OpenAI ("context_length_exceeded" / "maximum context
length"), Gemini ("input token count exceeds the maximum").
- `_build_digest` gains a `max_tokens` parameter (default
`_MAX_DIGEST_TOKENS`, so all other callers are unchanged); the char
budget derives from it.
- New `_MIN_DIGEST_TOKENS = 10_000` floor so a persistent overflow
terminates after 4 attempts instead of looping.
- Tests: retry produces a strictly shorter second digest and succeeds;
persistent overflow stops after the 10k floor (4 calls) and returns
stats-only; non-overflow failures still fail on the first call with no
retry.
## Testing
- [x] Unit tests pass (`pytest`)
### Test Output
```text
uv run --frozen --extra dev pytest tests/test_learn/ -q
216 passed, 3 skipped in 8.18s
```
## Real Behavior Proof
- Environment: macOS (arm64), Python 3.12 via uv, this branch's
worktree, real `claude` CLI installed.
- Exact command / steps: Drove the real CLI through `_call_cli_llm` with
oversized digests to verify the trigger end-to-end.
- Observed result: Could not force the overflow on this machine - the
account resolves to a 1M-context model, so even a ~950k-token prompt
succeeds (confirmed via the stream-json `result` event's `contextWindow:
1000000`). The failure is real on 200k-window installs: the field
telemetry's stderr carries `Prompt is too long` verbatim, which is the
exact string `_is_prompt_too_long` matches and the exact `RuntimeError`
message the retry test injects through the real `analyze` -> `_call_llm`
path.
- Not tested: a live end-to-end overflow against a 200k-window account
(none at hand); the gemini/codex CLI phrasings are covered by the
pattern list, not live runs.
## Runtime Rollout Safety
- Rollout-managed feature(s): None; `learn` is not rollout-gated.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Only on the previously-failing path:
an overflow now retries with a smaller digest instead of aborting the
analysis. Successful analyses are byte-identical (same budget, same
digest).
- Kill switch / disable path: Not needed; a persistent overflow degrades
to today's behavior (stats-only result, warning logged) after 4
attempts.
- Unsafe override required: No.
- Qualification impact: Heavy-context installs (large CLAUDE.md, many
MCP servers) go from 100% learn failure to a truncated-digest analysis.
- Rollback path: Revert the commit; no persisted formats touched.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
632cb81dbe |
fix(learn): surface Codex analysis failures (#3016)
## Description `headroom learn` could invoke Codex CLI from a non-Git working directory without Codex’s required bypass flag. The resulting backend error was then swallowed by the analyzer and rendered as “No actionable patterns found” with exit code 0. This fixes both coupled defects so Codex can run from discovered project locations and genuine analysis failures remain visible and machine-detectable. Closes #3008 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `--skip-git-repo-check` to the Codex CLI analysis backend command. - Added an explicit `analysis_error` result field instead of conflating backend failure with an empty recommendation set. - Kept multi-project analysis best-effort, while returning exit code 1 after any project analysis fails. - Prevented failed analysis from printing a misleading no-pattern success message. - Added analyzer and CLI regression coverage for the command and failure-propagation contracts. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest -q tests/test_learn/test_analyzer.py tests/test_cli_learn.py 102 passed in 2.34s uv run pytest -q tests/test_learn tests/test_cli_learn.py 257 passed, 7 skipped in 3.11s uv run mypy headroom Success: no issues found in 520 source files uv run ruff check <changed files> All checks passed! uv run ruff format --check <changed files> 5 files already formatted uv run pytest tests scripts/tests --splits 4 --group N --tb=short -q shard 1: 2766 passed, 140 skipped in 174.08s shard 2: 2699 passed, 207 skipped in 60.00s shard 3: 2822 passed, 84 skipped in 76.10s shard 4: 2734 passed, 172 skipped in 80.29s ``` ## Real Behavior Proof - Environment: macOS arm64, Python 3.13, Codex CLI 0.147.0-compatible command surface, current `main` including #2996. - Exact command / steps: verified `codex exec --help`; exercised `_call_cli_llm` with a captured subprocess command; invoked the Click command with a simulated Codex nonzero backend result. - Observed result: the subprocess command is `codex exec --skip-git-repo-check`; backend failure text is printed as `Analysis failed`, the misleading no-pattern message is absent, and the CLI exits 1. - Not tested: live paid Codex analysis against production account credentials; subprocess and CLI behavior are covered deterministically. ## Runtime Rollout Safety - Rollout-managed feature(s): none; this is CLI-only failure handling. - Minimum rollout channel: normal patch release after exact-head CI is entirely green. - Stable/default behavior changed: failed LLM analysis now exits nonzero instead of reporting success; successful and genuinely empty analyses are unchanged. - Kill switch / disable path: select another backend with `HEADROOM_LEARN_CLI` or `--model` if Codex CLI is unavailable. - Unsafe override required: none. - Qualification impact: all four Python CI shards, static checks, security checks, and command-level regression tests must pass. - Rollback path: fix forward through a human-reviewed corrective PR; no persisted data or migration is involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project’s style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation — inline result-contract documentation; no separate user guide change is required - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable; command-line backend and exit semantics only. ## Additional Notes Human review only. No merge or auto-merge is configured. This corrects the root failure and exit semantics without extending any timeout. |
||
|
|
5d25abd356 |
test: repair three suite failures that are red on main (#3196)
## Summary
Three tests fail on a clean `main` full-suite run. None is a product
defect — all three are tests that stopped describing reality, and they
will noise up or block the 0.36.4 release.
| Test | Why it fails | Fix |
|---|---|---|
| `test_release_workflows::test_no_native_tls_in_wheel_build_tree` |
Shells out to `cargo`; raises `FileNotFoundError` wherever the Rust
toolchain is absent | Copied the skip guards its own dual already had |
|
`test_learn/test_integration::TestCodexIntegration::test_full_pipeline`
| Asserts `"Bash" in all_tools` against **real local Codex data**; Codex
renamed its shell tool | Assert what the test is for, across Codex
versions |
|
`test_graceful_shutdown::test_run_server_installs_cancelled_error_filter`
| Counts installs on the **process-global** `uvicorn.error` logger;
order-dependent | Isolate the global state; assert the real contract |
## 1. native-tls / cargo
The `openssl-sys` gate 30 lines above is described in-code as this
test's dual. It already skips when `cargo` is missing, **and** when
cargo fails for a reason other than `"package did not match"` (the Linux
wheel target not being installed locally). The native-tls test never
copied either guard.
Not disabled: CI installs the toolchain via `dtolnay/rust-toolchain`, so
the check still executes there. The skip only applies where cargo is
genuinely absent.
## 2. Codex tool vocabulary
This test runs against whatever Codex sessions the machine actually has
(gated by `HAS_CODEX_DATA`), and asserted:
```python
# Codex has only Bash tool (shell)
assert "Bash" in all_tools
```
Codex has since renamed its shell tool (`Bash` → `shell` → `exec`), and
0.149.0 added agent tools (`spawn_agent`, `send_message`, `wait`) beside
it. The assertion pinned one release's vocabulary, so it fails on any
current install.
It now asserts what the pipeline is actually being tested for — that
tool calls were extracted, including a shell-execution tool under any of
its known names — and names the remedy in the failure message for the
next rename.
**Still discriminating** (verified, not assumed):
| Scenario | Result |
|---|---|
| pipeline parsed nothing | fails ✓ |
| tool names garbled | fails ✓ |
| agent tools only, no shell tool | fails ✓ |
| real current Codex data | passes ✓ |
## 3. Global logger state
```python
if not any(isinstance(item, _SuppressCancelledErrorFilter) for item in uvicorn_error_logger.filters):
uvicorn_error_logger.addFilter(_SuppressCancelledErrorFilter())
```
`run_server` is deliberately idempotent and `uvicorn.error` is a
process-global logger, so any earlier test in the session that reached
`run_server` leaves the filter attached — and this test then observes
**zero** installs against its `== 1` assertion. It passes alone and
fails in a full run, which is exactly the symptom.
The test now clears and restores that global state around itself, and
additionally asserts the idempotence guard that is the real contract:
calling `run_server` twice must not stack a duplicate filter. The test
got stronger, not just quieter.
## Scope
Tests only — no product code is touched.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
c5563d3a7d |
fix(learn): include stdout in CLI failure messages, not just stderr (#3080)
## Description `headroom learn` reports CLI backend failures using **stderr only**. `claude -p --output-format stream-json --verbose` writes *nothing* to stderr when the run fails at the API layer, so the failure a user actually sees is a message that stops at the colon: ```text LLM analysis failed: `claude -p --output-format stream-json --verbose` failed (exit 1): ``` The reason is not missing, it is discarded. Claude Code still emits a final `result` event on stdout whose `result` field is the human-readable cause, and the streaming path has already parsed it into `final_result` one line above the `raise`. This makes a whole class of failures undiagnosable for users and maintainers alike: a usage limit, an unreachable local proxy, and an expired login all render identically as an empty message. Reported by a desktop user who could only tell us "sometimes i have this LLM analysis failed" with nothing after the colon. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `_failure_detail(stderr, stdout, *, result_text=None)` in `headroom/learn/analyzer.py`. Prefers the already-parsed `result` text, falls back to the **tail** of stdout (CLI backends emit the error last, after their whole event log), keeps stderr when present, and returns `"(no output captured)"` so the message is never a dangling colon. - Use it in `_call_claude_cli_streaming` (streaming claude-cli path) and in `_call_cli_llm` (the `subprocess.run` backends, gemini-cli / codex-cli), so the same blind spot is closed for every CLI backend rather than only the one that was reported. - Existing truncation behaviour is unchanged: each stream is still capped at `_MAX_SNIPPET_LEN`. Complements #3016, which makes an analysis failure propagate instead of being swallowed as success; that PR fixes *whether* the user learns a failure happened, this one fixes *what* the failure says. No overlapping lines. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_learn/ -q 247 passed, 4 skipped in 27.08s $ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! $ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py 2 files already formatted $ uv run --frozen --extra dev mypy headroom/learn/analyzer.py Success: no issues found in 1 source file ``` New tests: `test_claude_cli_nonzero_exit_includes_api_error_from_stdout`, `test_claude_cli_nonzero_exit_with_no_output_says_so`, `test_claude_cli_nonzero_exit_keeps_stderr_when_present`, `test_codex_nonzero_exit_includes_stdout_when_stderr_empty`. ## Real Behavior Proof - Environment: macOS 15.6 (Darwin 24.6.0), Claude Code 2.1.228, Python 3.10.18, headroom on this branch. - Exact command / steps: forced an API-layer failure in the exact command the analyzer runs, capturing the streams separately: `echo "say hi" | claude -p --output-format stream-json --verbose --settings '{"env":{"ANTHROPIC_BASE_URL":"http://127.0.0.1:9"}}' > out.txt 2> err.txt; echo "EXIT=$?"; wc -c err.txt; tail -c 400 out.txt` - Observed result: `EXIT=1`, `err.txt` is **0 bytes**, and the reason appears only in the last stdout line: `"terminal_reason":"api_error", ..., "result":"API Error: Connection refused — a firewall or proxy may be blocking it (ConnectionRefused)"`. A second run with `--bare` produced the same shape with `"result":"Not logged in · Please run /login"`. Before this change both surface as `failed (exit 1):` with nothing after the colon; after it, the `result` text is in the message. The unit tests encode this exact stream shape (stdout `result` event, empty stderr, exit 1). - Not tested: real usage-limit and 429 responses, which I cannot provoke on demand. They travel the same code path as the reproduced `api_error` case (final `result` event on stdout, empty stderr), so they are covered by construction rather than by observation. Windows and the gemini-cli backend were not exercised manually; the shared helper is covered by unit tests for both the streaming and `subprocess.run` paths. ## Runtime Rollout Safety - Rollout-managed feature(s): None. This touches only the error text raised by `headroom learn`'s CLI backends; no rollout-gated feature, flag, or runtime component is involved. - Minimum rollout channel: N/A, not rollout-gated. Ships with the package like any other library fix. - Stable/default behavior changed: Yes, narrowly. The message text of an existing `RuntimeError` on a non-zero CLI exit now includes the stdout/`result` reason alongside stderr. No control flow, exit code, public API, or return value changes: the same exception is raised in the same cases. - Kill switch / disable path: None needed. Nothing is enabled or newly executed, so there is nothing to switch off; the only behavioral surface is the string inside an exception that was already being raised. - Unsafe override required: No. - Qualification impact: None. No qualification-gated path, model, or provider behavior is touched. Callers that pattern-match this message on `"failed (exit N)"` still match, since that prefix is unchanged. - Rollback path: Revert this commit. The previous stderr-only message returns with no migration, state, or config to undo. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
a24fe7dcbf |
fix(learn): stop classifying a successful exit code 0 as an error (#2289)
## Description
`is_error_content` classifies successful shell commands as errors,
inflating the failure stats that `headroom learn` reports.
The heuristic flags a tool result as an error when it contains any of a
list of substrings, one of which is the bare `"exit code"`:
```python
indicators = [
..., "timed out", "exit code", "FileNotFoundError",
]
return any(ind in snippet for ind in indicators)
```
But agent harnesses (Codex, Grok, opencode, ...) append `exit code 0` to
the output of every **successful** shell command. `"exit code" in
snippet` is `True` for `exit code 0`, so those successes are counted as
failures.
That is not cosmetic: `is_error_content` sets `ToolCall.is_error`, which
feeds:
- the per-project failure rate the digest shows the LLM
(`_build_digest`: "N failures (X%)"), and
- loop classification (`detect_loops` treats a group as an *error loop*
when ≥ half its calls are errors),
so a project where most shell commands succeed can read as one riddled
with failures, biasing the learned recommendations.
## Fix
Match a **nonzero** exit code instead of the bare substring:
```python
_NONZERO_EXIT_RE = re.compile(r"exit code:?\s*(?!0\b)\d", re.IGNORECASE)
...
if any(ind in snippet for ind in indicators):
return True
return bool(_NONZERO_EXIT_RE.search(snippet))
```
`exit code 0` no longer matches. A nonzero code still does — and, as a
small bonus, the case-insensitive regex now also catches `Exit code: 1`
(colon + capitalized), which the old case-sensitive lowercase substring
missed.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/_shared.py`: replace the `"exit code"` substring
indicator with a nonzero-exit-code regex (`_NONZERO_EXIT_RE`) checked
after the other indicators.
- `tests/test_learn/test_integration.py`: new tests that `exit code 0`
is not an error and a nonzero code (any casing / with a colon) still is.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/learn/_shared.py tests/test_learn/test_integration.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/learn/_shared.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the classifier with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: ran a successful output ending `Process
finished with exit code 0`, plus several nonzero-code failures (`exit
code 1`, `Exit code: 127`, `exit code 137`) and control strings, through
the OLD substring form and the NEW regex form.
- Observed result: OLD flags `exit code 0` as an error; NEW returns
`False` for it, still returns `True` for every nonzero code (including
the colon/capitalized form the old lowercase substring missed), and
leaves the other indicators unchanged.
- Not tested: a full `learn` run over a real history; full local
`pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new tests live
alongside the existing `is_error_content` false-positive/true-positive
tests in `tests/test_learn/test_integration.py`, so they run under the
normal CI pytest job; behaviour is additionally verified by the
standalone proof above.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
29d8a5e563 |
fix(learn/gemini): stop double-counting session tokens (#2230)
## Description
The Gemini `learn` scanner inflates every session's token totals by
double-counting.
In `_parse_messages` the per-message usage accumulation is:
```python
usage = msg.get("usageMetadata", msg.get("usage", {}))
if isinstance(usage, dict):
total_input_tokens += usage.get("promptTokenCount", 0)
total_input_tokens += usage.get("cachedContentTokenCount", 0)
total_output_tokens += usage.get("candidatesTokenCount", 0)
total_output_tokens += (
usage.get("totalTokenCount", 0) - usage.get("promptTokenCount", 0)
if usage.get("totalTokenCount")
else 0
)
```
Both additions on each side double-count, per Gemini's `usageMetadata`
semantics:
- `cachedContentTokenCount` is the cached **subset** of
`promptTokenCount`, not tokens on top of it. Adding both counts the
cached input twice.
- `totalTokenCount == promptTokenCount + candidatesTokenCount`, so
`totalTokenCount - promptTokenCount` is just `candidatesTokenCount`
again. Adding it on top of `candidatesTokenCount` counts the output
twice.
For a turn with 1000 prompt tokens (300 cached) and 500 output tokens
(`totalTokenCount` 1500), the scanner records input 1300 and output 1000
instead of 1000 / 500 — so both totals are materially inflated for any
Gemini session that carries usage metadata.
## Fix
Count the prompt as input and the candidates as output, once each:
```python
total_input_tokens += usage.get("promptTokenCount", 0)
total_output_tokens += usage.get("candidatesTokenCount", 0)
```
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/plugins/gemini.py`: drop the `cachedContentTokenCount`
and `totalTokenCount - promptTokenCount` additions in `_parse_messages`.
- `tests/test_learn/test_gemini_scanner.py`: new test asserting the
input/output totals equal `promptTokenCount` / `candidatesTokenCount`.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/learn/plugins/gemini.py tests/test_learn/test_gemini_scanner.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/learn/plugins/gemini.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the arithmetic with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: fed a usage dict of `promptTokenCount=1000,
cachedContentTokenCount=300, candidatesTokenCount=500,
totalTokenCount=1500` through the OLD accumulation and the NEW one.
- Observed result: OLD → input 1300, output 1000 (cached and candidates
both counted twice); NEW → input 1000, output 500 (the true figures).
- Not tested: a full `learn` run over a real Gemini history; full local
`pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new test uses the
existing `GeminiScanner` harness in
`tests/test_learn/test_gemini_scanner.py` so it runs under the normal CI
pytest job; behaviour is additionally verified by the standalone proof
above.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
7e83b8da3c |
fix(learn/gemini): detect the project path for JSONL sessions (#2229)
## Description
The Gemini `learn` plugin can't detect the project path for JSONL
sessions, so it writes its insights to the wrong project.
`discover_projects` globs both `session-*.json` and `session-*.jsonl`
and calls `_detect_project_path`, which reads the file with a single
whole-file `json.load`:
```python
def _detect_project_path(self, session_path: Path) -> Path | None:
try:
with open(session_path, encoding="utf-8", errors="replace") as f:
data = json.load(f)
except (OSError, json.JSONDecodeError):
return None
...
```
A `.jsonl` session is one JSON object per line, so `json.load` on the
whole file raises `json.JSONDecodeError` ("Extra data") on the second
line. The method swallows that and returns `None`, and the caller falls
back to `Path.cwd()`:
```python
project_path = self._detect_project_path(session_files[0])
...
ProjectInfo(
name=project_path.name if project_path else project_dir.name,
project_path=project_path or Path.cwd(), # wrong project
context_file=gemini_md, # None: GEMINI.md never found
...
)
```
So for the JSONL format (Gemini CLI's newer session format — the one
that carries `type: "session_metadata"` records), detection never works:
the learned tool/verbosity insights are attributed to the current
working directory instead of the real project, and the project's
`GEMINI.md` is never located. The sibling `_scan_jsonl_session` already
reads this format line-by-line, and the Claude plugin recovers the
project path from session `cwd` the same way.
## Fix
Route `.jsonl` sessions through a line-by-line reader and share the
field extraction (`projectPath` / `project_path` / `cwd` /
`workingDirectory`) between both formats:
```python
if session_path.suffix == ".jsonl":
return self._detect_project_path_jsonl(session_path)
```
`_detect_project_path_jsonl` parses each line (skipping blanks and
unparseable lines, exactly like `_scan_jsonl_session`) and returns the
first record that yields an existing path. The JSON path is unchanged.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/plugins/gemini.py`: dispatch `.jsonl` sessions to a
new line-by-line `_detect_project_path_jsonl`; factor the field
extraction into `_project_path_from_entry` shared by both paths.
- `tests/test_learn/test_gemini_scanner.py`: new test asserting a JSONL
session's `cwd` is recovered.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/learn/plugins/gemini.py tests/test_learn/test_gemini_scanner.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/learn/plugins/gemini.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the behavior with a dependency-free script mirroring both
detection paths and left the full pytest to CI.
- Exact command / steps: wrote a `.jsonl` session whose first record is
`{"type":"session_metadata","cwd":"<project>"}`, then ran the OLD
whole-file `json.load` reader and the NEW line-by-line reader; also
checked a single-object `.json` session still resolves under both.
- Observed result: OLD returns `None` for the JSONL file (the caller
would fall back to cwd); NEW returns the project path; the `.json` case
resolves identically under both.
- Not tested: a full `learn` run over a real Gemini history; full local
`pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new test uses the
existing `GeminiScanner` harness in
`tests/test_learn/test_gemini_scanner.py` so it runs under the normal CI
pytest job; behaviour is additionally verified by the standalone proof
above.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
e0ce4b1d48 |
fix: remove rtk and lean-ctx CLI context tools (#2677)
## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt. |
||
|
|
85e8699451 |
fix(learn): keep traceback tail in tool-error digest preview (#2596)
## Description `_format_tool_call` in `headroom/learn/analyzer.py` built the error preview with a head-only slice — `tc.output[:200]`. For tracebacks the root cause (`ExceptionType: message`) is at the **tail**, so the digest showed only `Traceback (most recent call last):` plus the first frame and dropped the actual diagnosis. The issue reports 46% of 715 measured errors were truncated past the 200-char head. Closes #2590 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `_truncate_head_tail()` helper that collapses newlines and, when over budget, keeps both the head and the tail joined by `…`. - `_format_tool_call` now uses it for error output so the exception line survives truncation. Short errors are returned unchanged (no marker). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ uv run pytest tests/test_learn/test_analyzer.py::TestDigestBuilder -q 9 passed in 1.47s $ uv run ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! ``` ## Real Behavior Proof - Environment: headroom @ main, Python 3.14, uv - Exact command / steps: added a long synthetic traceback (`KeyError: 'the-actual-root-cause'` at the tail) as a failing tool call and built the digest. - Observed result: digest now contains both `Traceback` and `KeyError: 'the-actual-root-cause'`, separated by `…`; short errors have no `…`. - Not tested: mypy not run locally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Truncation budget stays at 200 chars (now split head/tail). mypy not run locally; happy to adjust if CI flags anything. |
||
|
|
f74d874777 |
fix(learn): detect the active OpenCode database (#2587)
## Description `headroom learn --agent opencode` can silently mine a frozen conversation corpus. `OpenCodePlugin` hardcodes `~/.local/share/opencode/opencode.db`, but source-built OpenCode writes `opencode-local.db` in the same directory. When both files exist, learn still succeeds against the stale packaged DB and ignores the live source-built corpus. This follows the report in https://github.com/headroomlabs-ai/headroom/issues/2581 and builds on the existing OpenCode learn path introduced in https://github.com/headroomlabs-ai/headroom/pull/559. This change keeps explicit constructor paths authoritative, honors `HEADROOM_OPENCODE_DB` when it is set, and otherwise selects the newest existing database between `opencode.db` and `opencode-local.db`, preferring canonical `opencode.db` on exact ties. It also updates the OpenCode learn docs line so the documented behavior matches the landed resolver. Closes #2581. The branch also carries one narrow CI repair requested during review: `headroom/cli/wrap.py` now binds the `unwrap claude` Click command back to `unwrap_claude` instead of the leak-warning helper, which restores the existing unwrap test surface and leaves the helper as an internal warning function. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - add a private OpenCode DB resolver in `headroom/learn/plugins/opencode.py` with precedence `db_path` then `HEADROOM_OPENCODE_DB` then newest existing default filename then canonical fallback - preserve canonical `opencode.db` for exact mtime ties and for canonical-only installs - add focused regression coverage for newer-local, explicit-path, canonical-only, equal-tie, missing-override, and end-to-end scanning cases - sync the OpenCode learn docs paragraph so it no longer claims `opencode.db` is the only supported default path - restore the `unwrap claude` Click command binding in `headroom/cli/wrap.py` and apply the repo formatter so the branch passes the existing unwrap test and lint gates ## Testing - [x] Unit tests pass (`uv run pytest tests/test_learn/test_opencode_scanner.py -q`) - [x] Linting passes (`uv run ruff check headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py`) - [x] Type checking passes (`uv run mypy headroom/learn/plugins/opencode.py`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_learn/test_opencode_scanner.py -q -k "newer_local_database" 1 passed, 9 deselected in 0.26s uv run pytest tests/test_learn/test_opencode_scanner.py -q 10 passed in 0.50s uv run ruff check headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py All checks passed! uv run ruff format headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py --check 2 files already formatted uv run mypy headroom/learn/plugins/opencode.py Success: no issues found in 1 source file rg -n "opencode-local\.db|HEADROOM_OPENCODE_DB|opencode\.db" docs/content/docs/opencode.mdx 78:`headroom learn` supports OpenCode as a scan target. It reads past sessions from the newer of `~/.local/share/opencode/opencode-local.db` and `~/.local/share/opencode/opencode.db`, or from `HEADROOM_OPENCODE_DB` when you set an explicit override, and writes corrections to your project's `AGENTS.md`. uv run pytest tests/test_cli/test_unwrap_claude.py -q -k "removes_mcp_rtk_and_stops_proxy or preserves_user_managed_serena or removes_headroom_installed_serena or keep_flags_skip_cleanup or restores_all_base_url_modes or stops_claude_owned_persistent_deployment or reports_ambiguous_same_port_persistent_deployment or warns_about_same_port_inherited_env or ignores_malformed_inherited_env_port" 9 passed, 5 deselected in 0.40s uv run ruff check . All checks passed! uv run ruff format --check . 1340 files already formatted ``` ## Real Behavior Proof - Environment: temporary SQLite databases exercised through the production `OpenCodePlugin()` constructor - Exact command / steps: run `uv run pytest tests/test_learn/test_opencode_scanner.py -q -k "newer_local_database"` against `origin/main` with the new regression test overlaid, then run the same command and the full `uv run pytest tests/test_learn/test_opencode_scanner.py -q` suite on the branch head - Observed result: the base reproduction fails with `AssertionError: assert 'Canonical' == 'Local'`, proving current main still selects the stale canonical DB; the branch head passes the reproduction row and the full 10-test scanner suite - Not tested: live user OpenCode corpus ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - `CHANGELOG.md` stays untouched because Headroom generates release notes from conventional commits. - The automatic chooser is intentionally limited to the two known default filenames, `opencode.db` and `opencode-local.db`. Other layouts can use `HEADROOM_OPENCODE_DB`. - The fix stays inside `headroom/learn/plugins/opencode.py`; no provider-neutral learn or pipeline code changes are planned. |
||
|
|
a09ba6c087 |
fix(learn): treat unreadable candidate paths as absent in project decode (#2446)
## Description `headroom learn` crashes with an uncaught `PermissionError` when the current user's username contains a dash. `_decode_project_path` (in `headroom/learn/plugins/claude.py`) probes speculative candidate paths when reconstructing an original filesystem path from a Claude Code encoded project directory name. When the username is e.g. `marco-rocha`, one candidate becomes `/home/marco/rocha`, which can collide with another user's home directory whose parent isn't stat-able. `Path.exists()` calls `os.stat` internally, raising `PermissionError` instead of returning `False`, so the whole `learn` command crashes before returning any recommendations. Fixes #2443 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `_path_exists()` to `headroom/learn/plugins/claude.py` — a thin wrapper around `Path.exists()` that returns `False` on any `OSError` (including `PermissionError`), mirroring the existing `OSError` handling already used in `_greedy_path_decode`. - Route every speculative candidate-path existence check in the decode path through `_path_exists()`: the Windows drive/path probes in `_decode_windows_path`, the `simple` POSIX candidate and greedy-branch bases in `_decode_project_path`/`_greedy_path_decode`, and the decoded `project_path`/`CLAUDE.md` checks in `discover_projects`. - Add regression tests covering the exact issue shape (`PermissionError` on `/home/marco/rocha`) and the `_path_exists` helper directly. - Leave `CHANGELOG.md` untouched — release-please generates it from conventional commits. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_learn/test_scanner.py::TestDecodePermissionError -q`) - [x] Linting passes (`ruff check`, `ruff format --check` on the two changed files) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_learn/test_scanner.py::TestDecodePermissionError -q collected 2 items tests\test_learn\test_scanner.py .. [100%] 2 passed in 1.86s $ ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, local dev checkout of headroom on branch off upstream/main - Exact command / steps: Simulated the issue by monkeypatching `Path.exists` to raise `PermissionError` for the colliding candidate `/home/marco/rocha`, then calling `_decode_project_path("-home-marco-rocha-butterfly-sylphina")` - Observed result: Before the fix the call propagates `PermissionError` (crash, matching the reported traceback); after the fix it returns without raising and the unreadable candidate is treated as non-existent. Both regression tests pass. - Not tested: End-to-end `headroom learn --apply` on a real Linux multi-user box with an actually unreadable `/home/<prefix>` — reproduced via the documented minimal logic instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
eed80dd4ba |
fix(learn/claude): don't abort the whole scan on a null message line (#2299)
## Description
A single Claude session-log line with an explicit `{"message": null}`
crashes the entire `headroom learn` run.
`ClaudeCodePlugin._scan_session` reads the message object in four
places:
```python
usage = d.get("message", {}).get("usage", {}) # assistant line
...
msg = d.get("message", {}) # _extract_tool_uses
msg = d.get("message", {}) # _extract_tool_results
msg = d.get("message", {}) # _extract_user_events
```
`dict.get("message", {})` only substitutes `{}` for a **missing** key. A
present-but-null `{"type": "assistant", "message": null}` yields `None`,
and `None.get(...)` raises `AttributeError`.
The per-file guard only catches I/O errors:
```python
try:
with open(jsonl_path, ...) as f:
for line in f:
...
except (OSError, UnicodeDecodeError) as e:
...
return None
```
so the `AttributeError` propagates out of `_scan_session`, past
`scan_project` (which has no try/except around the scan), and aborts the
whole `learn` invocation — every project, not just the one bad line. One
malformed line takes down the entire run.
## Fix
Coalesce the message with `or {}` at all four sites, so a null (or any
falsy) value collapses to `{}`:
```python
usage = (d.get("message") or {}).get("usage", {})
msg = d.get("message") or {}
```
The malformed line is now skipped and scanning continues; valid lines
are parsed exactly as before.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/plugins/claude.py`: coalesce `d.get("message")` with
`or {}` in `_scan_session` and the three `_extract_*` helpers.
- `tests/test_learn/test_subagent_scanning.py`: new test that a session
containing `{"message": null}` lines scans without crashing and still
parses the valid tool call.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/learn/plugins/claude.py tests/test_learn/test_subagent_scanning.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/learn/plugins/claude.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the per-line handling with a dependency-free script and left
the full pytest to CI.
- Exact command / steps: ran an assistant line `{"message": null}` (plus
a real assistant line and a missing-message line) through the OLD
`get("message", {})` and NEW `get("message") or {}` logic.
- Observed result: OLD raises `AttributeError` on the null message; NEW
returns `0` for it and still counts `42` input tokens for the real line
and `0` for a missing-message line.
- Not tested: a full `learn` run over a real history containing such a
line; full local `pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new test reuses the
existing `ClaudeCodePlugin` scanner harness in
`tests/test_learn/test_subagent_scanning.py`, so it runs under the
normal CI pytest job; behaviour is additionally verified by the
standalone proof above.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
|
c7b5a24b4f |
fix(learn): don't shadow TIMEOUT/CONNECTION with the generic RUNTIME_ERROR (#2099)
## Description `classify_error` (used by `headroom learn` to categorize failed tool calls) miscategorizes timeouts and connection failures as generic runtime errors. The pattern list is checked in order, first match wins, and it puts the generic catch-all *before* the specific categories: ```python (re.compile(r"Traceback \(most recent|Exception:|Error:", re.I), ErrorCategory.RUNTIME_ERROR), (re.compile(r"timed? ?out|TimeoutError|deadline exceeded", re.I), ErrorCategory.TIMEOUT), ... (re.compile(r"ConnectionError|ConnectionRefused|ECONNREFUSED|network", re.I), ErrorCategory.CONNECTION_ERROR), ``` Every Python exception repr is `XxxError: ...` (or `Exception: ...`), so the generic `Error:`/`Exception:` pattern matches first. A tool result of `"TimeoutError: timed out after 30s"` is classified `RUNTIME_ERROR` instead of `TIMEOUT`; `"ConnectionError: [Errno 111] Connection refused"` is classified `RUNTIME_ERROR` instead of `CONNECTION_ERROR`. The dedicated `TIMEOUT` and `CONNECTION_ERROR` categories — which explicitly list `TimeoutError` and `ConnectionError` — are therefore unreachable for the most common (colon-repr) message shape; they only fire for tokenless phrasings like `deadline exceeded`. That mislabels the learn digest's per-category error stats. ## Fix Check the two specific categories (`TIMEOUT`, `CONNECTION_ERROR`) before the generic `RUNTIME_ERROR` catch-all. A generic exception repr with no timeout/connection token still classifies as `RUNTIME_ERROR`, so existing behavior for those is unchanged (including the opencode scanner's `"Error: command failed with exit code 1"` → `RUNTIME_ERROR`). Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/_shared.py`: move the `TIMEOUT` and `CONNECTION_ERROR` patterns above the generic `RUNTIME_ERROR` pattern, with a comment explaining the ordering. - `tests/test_learn/test_error_classification.py`: new tests asserting `TimeoutError:`/`ConnectionError:` reprs classify specifically, a generic `Error:` stays `RUNTIME_ERROR`, and non-error text is `UNKNOWN`. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/learn/_shared.py tests/test_learn/test_error_classification.py All checks passed! $ python -m py_compile headroom/learn/_shared.py tests/test_learn/test_error_classification.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the ordering with a dependency-free script that replicates the pattern list under both the old and new orderings, and left the full pytest to CI. - Exact command / steps: classified `"ConnectionError: [Errno 111] Connection refused"` and `"TimeoutError: timed out after 30s"` under the old order (RUNTIME before TIMEOUT/CONNECTION) and the new order (TIMEOUT/CONNECTION before RUNTIME), plus the opencode scanner's `"Error: command failed with exit code 1"` as a regression guard. - Observed result: old order classifies both as `RUNTIME_ERROR`; new order classifies them as `CONNECTION_ERROR` and `TIMEOUT` respectively; the guard string stays `RUNTIME_ERROR` under both orderings, so the existing opencode scanner test is unaffected. - Not tested: a full `headroom learn` digest run; full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" and "type checking" boxes are unchecked because the full suite imports the ML stack, which I can't run in this environment; the change is a reordering of two entries in a pure pattern list, verified by the standalone proof (which also confirms the one existing test that touches this path stays green) and the new regression tests for CI. Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
|
d2170b1922 |
fix(learn): parse fenced JSON even with a prose preamble (#1988)
## Description `_strip_fenced_json` only stripped a markdown fence when the string *started with* ```` ``` ````. When the model prefixed prose before the fence (e.g. `Here is the JSON:\n\n```json ...`) despite being told to return JSON only, the guard was skipped and `json.loads` ran on the prose, raising `JSONDecodeError`. The claude-cli streaming path surfaced this as `returned unparseable output`, and `headroom learn` silently discarded the LLM analysis, degrading to "No actionable patterns found". This is the parsing-side cousin of the silent-degradation issue fixed in #373. Closes #1989. Related: #373. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/analyzer.py`: rewrote `_strip_fenced_json` to locate the fenced block wherever it appears, then fall back to the whole text, then to a first-`{` / last-`}` slice, only re-raising `JSONDecodeError` if nothing parses as a JSON object. Preserves the prior "first opening / last closing fence" behaviour and triple-backtick content inside the payload. Fixes all three call sites (non-streaming CLI, claude-cli streaming, litellm). - `tests/test_learn/test_analyzer.py`: added regression cases to `TestStripFencedJson` for preamble-before-fence, prose around a bare object, and triple-backticks inside the payload. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) — scoped to the changed module (see Additional Notes) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_learn/test_analyzer.py -q ........................................................................ [ 86%] ........... [100%] 83 passed, 1 warning in 2.18s $ ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! $ ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py 2 files already formatted $ mypy --ignore-missing-imports --follow-imports=silent headroom/learn/analyzer.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS, Python 3.13.12, headroom-ai at this branch (runtime deps from an installed 0.30.0 env). - Exact command / steps: ran the old vs new `_strip_fenced_json` on the exact failing model output (a prose preamble followed by a ```json fence), then applied the fix over an installed 0.30.0 and re-ran the previously failing `headroom learn --apply`. Input sample: `'The JSON is my deliverable for this analysis task. Here it is:\n\n```json\n{"context_file_rules": [], "memory_file_rules": []}\n```'` - Observed result: OLD raised `JSONDecodeError: Expecting value: line 1 column 1 (char 0)`; NEW returned `{'context_file_rules': [], 'memory_file_rules': []}`. The real `headroom learn --apply` run that had been failing with `returned unparseable output` then completed and consumed the LLM analysis instead of dropping it. Full transcript: ```text OLD: JSONDecodeError -> Expecting value: line 1 column 1 (char 0) NEW: {'context_file_rules': [], 'memory_file_rules': []} ``` - Not tested: full end-to-end `headroom learn --apply` was not re-run inside CI here (it shells out to a live `claude` CLI); the parser is exercised deterministically by the added unit tests and the before/after repro above. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A — updated the function docstring only; no external docs affected) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (N/A — no CHANGELOG entry convention observed for this fix; happy to add if maintainers prefer) ## Additional Notes - `mypy` was run against the changed module in isolation (`--ignore-missing-imports --follow-imports=silent`) rather than the full project, because I validated in an ad-hoc environment; the change keeps the existing `-> dict` signature and annotations, so it is type-neutral. - Not addressed here (possible follow-up): the failure is swallowed as a warning in `analyze()`, so users only see "No actionable patterns found" with no signal the LLM pass produced nothing — the same silent-degradation class as #373, on the parsing side. |
||
|
|
e3b45e402b |
fix(learn): handle Windows UTF-8, drive-letter paths, and CLI shim fallback (#1895)
## Description `headroom learn --verbosity` is broken on Windows in three related ways: - Transcript/profile reads can use the platform default codec, so non-ASCII content can raise `UnicodeDecodeError` and collapse learning signals to empty output. - `--project <path>` can miss real Claude project directories because Windows profile junctions can raise `PermissionError` during directory walks, and escaped Claude project folder names cannot always distinguish `vibe-remote` from `vibe\remote`. - `headroom learn --agent codex` can fail with `` `claude` not found in PATH `` even when the npm-installed CLI exists, because Windows `.cmd` shims require `PATHEXT` resolution. Refs https://github.com/headroomlabs-ai/headroom/issues/1624 for the Windows learn failures. The dashboard-hint UX and third-party-provider-auth items in that issue are unrelated and out of scope for this PR. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/verbosity.py`: read and write verbosity transcripts/profiles with `encoding="utf-8"` so non-ASCII content works regardless of the Windows locale codec. - `headroom/learn/plugins/claude.py`: skip inaccessible siblings one entry at a time during greedy project path decoding, so one Windows junction no longer hides valid project directories. - `headroom/learn/plugins/claude.py`: prefer a valid `cwd` found in Claude session JSONL when discovering project paths, which resolves ambiguous escaped folder names such as `vibe-remote` versus `vibe\remote`. - `headroom/learn/analyzer.py`: resolve Windows CLI shim paths through `shutil.which()` after `FileNotFoundError`, then retry once for streaming and non-streaming CLI calls. - `CHANGELOG.md`: document the Windows learn fixes under `Unreleased`. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_verbosity_learn.py::TestWindowsEncoding tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_permission_denied_sibling_does_not_abort_the_walk tests/test_learn/test_analyzer.py::TestWindowsCliShimFallback tests/test_learn/test_scanner.py::TestDecodeProjectPath::test_discover_project_prefers_session_cwd_over_ambiguous_folder_name -q`) - [x] Linting passes (`uv run ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py`) - [x] Formatting passes (`uv run ruff format --check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py`) - [ ] Type checking passes (`uv run mypy headroom`) not run; no new public type surface - [x] New tests added for the Windows `cwd` disambiguation regression - [x] Manual testing performed ### Test Output ```text uv run ruff format headroom/learn/plugins/claude.py 1 file reformatted uv run ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py All checks passed! uv run ruff format --check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py 2 files already formatted uv run pytest tests/test_verbosity_learn.py::TestWindowsEncoding tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_permission_denied_sibling_does_not_abort_the_walk tests/test_learn/test_analyzer.py::TestWindowsCliShimFallback tests/test_learn/test_scanner.py::TestDecodeProjectPath::test_discover_project_prefers_session_cwd_over_ambiguous_folder_name -q 9 passed in 0.25s ``` CI on current head `c6dbac40` is green. A prior `test (1)` run hit an unrelated timing-sensitive scheduler assertion; GitHub did not permit direct rerun without admin rights, so the empty commit `c6dbac40` retriggered CI and the shard passed. ## Real Behavior Proof - Environment: Windows 11, Python 3.12, real filesystem for the path-decoding reproduction. - Exact command / steps: Ran `uv run pytest tests/test_verbosity_learn.py::TestWindowsEncoding tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_permission_denied_sibling_does_not_abort_the_walk tests/test_learn/test_analyzer.py::TestWindowsCliShimFallback tests/test_learn/test_scanner.py::TestDecodeProjectPath::test_discover_project_prefers_session_cwd_over_ambiguous_folder_name -q`, `uv run ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py`, and `uv run ruff format --check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py`; the path tests create real Windows-style project directories, inaccessible siblings, ambiguous `vibe\remote` versus `vibe-remote` folders, and Claude session JSONL with `cwd` pointing at the intended project. - Observed result: The focused test command returned `9 passed in 0.25s`, Ruff check passed, and Ruff format check passed. The decoder skips the inaccessible sibling and reaches `vibe-remote`; the session-`cwd` test returns `vibe-remote` instead of trusting the ambiguous escaped folder name; the UTF-8 tests round-trip non-ASCII transcript/profile content under a non-UTF-8 Windows-style codec; the CLI shim tests retry once through `shutil.which()` after `FileNotFoundError`. - Not tested: real npm-installed `claude`/`codex` CLI shims, dashboard-hint UX, and third-party-provider auth. ## Review Readiness - [x] I have performed a self-review - [x] Retrospective review completed after opening; it found one missing `cwd` disambiguation case, now fixed in this PR - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The post-open retrospective review concluded this needed targeted rework rather than only a retrospective sign-off. - The `cwd` recovery commit is `ee720bb1`; `0905be7b` contains the required formatter cleanup; current head `c6dbac40` is an empty CI-rerun commit after GitHub denied direct rerun without admin rights. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
4f22cbb05c |
fix(learn): decode Windows drive-style project dirs with dotted usernames (#1855)
## Description Fixes #1849. On Windows, `headroom learn --all --apply` failed to write recommendations for every project when the username contains a dot (e.g. `pradipe.yoggi`), reporting `[WinError 161] The specified path is invalid: '\\\Users\...'`. Root cause: Claude Code encodes `C:\Users\first.last\proj` as `C--Users-first-last-proj` — **no leading dash** (the path starts with the drive letter), and `:` + `\` each collapse to `-`, producing a double dash after the drive letter. Two defects followed: 1. `_decode_project_path()` required `escaped_name.startswith("-")` and returned `None` for every real Windows encoding, so the greedy filesystem-walking decoder (which correctly rejoins dotted components like `first.last`) was unreachable. 2. The `discover_projects()` fallback blindly stripped the first character (`entry.name[1:]`), turning `C--Users-...` into `--Users-...`, whose dash→slash replacement yields the invalid `\\\Users\first\last\proj` seen in the issue. ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation update - [ ] Refactoring (no functional changes) ## Changes Made - `headroom/learn/plugins/claude.py` - New `_decode_windows_path(drive, parts)` helper: drops empty split tokens (so separators are never doubled), checks the literal path, greedy-decodes from the drive root (so `Users` → `first.last` is rejoined from the real filesystem via the existing `_component_tokenizations` dot-split), and keeps the trust-`Users` literal fallback. - `_decode_project_path()` now matches both the real drive-style encoding `C--Users-...` (no leading dash) and the legacy `-C-Users-...` form via `^-?([A-Za-z])--?(.+)$`, routing both through the helper; POSIX logic unchanged. - `discover_projects()` fallback applies the same normalization instead of stripping the first character, so nonexistent projects still get a *valid* `C:\Users\...` path instead of `\\\Users\...`. - `tests/test_learn/test_scanner.py`: three new tests — double-dash encoding decodes without doubled separators; dotted username rejoined via greedy decode on a real directory tree (Windows-only); `discover_projects` fallback produces a valid path for a nonexistent `C--Users-...` project. ## Testing - [x] Existing tests pass locally - [x] Added new tests covering the change ``` $ python -m pytest tests/test_learn -q 3 failed, 211 passed, 5 skipped in 7.22s # The 3 failures (test_home_dir_username_stays_single_component, # test_includes_project_info, test_double_write_replaces_not_appends) are # pre-existing Windows-local failures, verified identical on a clean # upstream/main checkout via git stash — none introduced by this change. $ ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py && ruff format --check ... All checks passed! $ mypy headroom --ignore-missing-imports Success: no issues found ``` ## Real Behavior Proof - Environment: Windows 11 Pro, PowerShell, Python 3.14, headroom built from this branch (Rust core built locally) - Exact command / steps: `python -c "from headroom.learn.plugins.claude import _decode_project_path as d; print(d('G--Programmi-Aggiuntivi-headroom')); print(d('C--Users-esiri-AppData-Local-Temp'))"` — decoding this machine's own real `~/.claude/projects` directory names (which use the drive-style encoding this PR fixes; note `Programmi Aggiuntivi` contains a space, exercising the greedy multi-token rejoin just like a dotted username) - Observed result: `G:\Programmi Aggiuntivi\headroom` and `C:\Users\esiri\AppData\Local\Temp` — both correct real paths. On upstream/main the same call returns `None` for both, which is what pushed `learn --all` into the mangling fallback. - Not tested: an actual Active Directory `first.last` account end-to-end (no such account available); covered instead by the Windows-only greedy-decode test against a real `john.doe` directory tree and by the space-in-path live decode above, which exercises the identical code path. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c600e314b3 |
fix(learn): honor CLAUDE_CONFIG_DIR when locating Claude logs and memory (#1642)
## Description `headroom learn` ignored `CLAUDE_CONFIG_DIR`. `ClaudeCodePlugin.__init__` resolved the Claude config directory as `~/.claude`, and the memory writer wrote the global `CLAUDE.md` to `~/.claude/CLAUDE.md`. A user who relocates their Claude config with that env var had `learn` scan the wrong directory and detect no projects. Other parts of the codebase already honor the override (`subscription/client.py`, `subscription/session_tracking.py`, `mcp_registry/claude.py`); the `learn` path was the outlier. Closes #1630 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `claude_config_dir()` to `headroom/learn/_shared.py` — returns `$CLAUDE_CONFIG_DIR` when set, else `~/.claude` (via `Path.home()`, matching the existing override elsewhere). - `ClaudeCodePlugin.__init__` now defaults `claude_dir` to `claude_config_dir()` instead of a hardcoded `~/.claude` (an explicit `claude_dir=` argument still wins). - `ClaudeCodeWriter._resolve_context_path` writes the home-directory global memory to `claude_config_dir() / "CLAUDE.md"` instead of `~/.claude/CLAUDE.md`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_learn/test_claude_config_dir.py tests/test_learn/test_writer.py -q 35 passed, 1 warning in 0.18s $ ruff check headroom/learn/ tests/test_learn/test_claude_config_dir.py All checks passed! $ mypy headroom/learn/_shared.py headroom/learn/plugins/claude.py headroom/learn/writer.py Success: no issues found in 3 source files ``` ## Real Behavior Proof - Environment: macOS (arm64), Python 3.14 venv, editable install of this branch. - Exact command / steps: ran `python -c "from headroom.learn.plugins.claude import ClaudeCodePlugin; print(ClaudeCodePlugin().projects_dir)"` with and without `CLAUDE_CONFIG_DIR=/tmp/altclaude` set, then `pytest tests/test_learn/ tests/test_cli_learn.py`. - Observed result: default prints `/Users/<me>/.claude/projects`; with `CLAUDE_CONFIG_DIR=/tmp/altclaude` it prints `/tmp/altclaude/projects` (before this change the second still printed `~/.claude/projects`). Test suite: 226 passed, 3 skipped. New regression tests cover the plugin scan dir, explicit-arg precedence, and the writer's home-memory path. - Not tested: end-to-end `headroom learn` against a real relocated log tree with live Claude Code transcripts — verified at the plugin/writer resolution layer plus the existing scanner suite. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes The identical hardcode also exists at `headroom/cli/mcp.py:21` (`CLAUDE_CONFIG_DIR = Path.home() / ".claude"`), but that is a separate command outside this issue's scope, so I left it for a follow-up to keep this PR to one issue. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
91cd2102d7 |
feat: add first-class OpenCode support (wrap, learn, mcp install) (#559)
## Summary Adds full OpenCode support to headroom — wrap, learn, and mcp install — on par with the existing Claude Code and Codex integrations. ## Changes ### Provider slice (`headroom/providers/opencode/`) - **runtime.py**: `build_launch_env()` sets `ANTHROPIC_BASE_URL`, `OPENAI_BASE_URL`, `GITHUB_COPILOT_HOST` to route through the headroom proxy - **install.py**: `apply_provider_scope()` patches `~/.config/opencode/opencode.json` with `baseURL` for github-copilot, anthropic, and openai providers ### CLI (`headroom wrap opencode`) - Options: `--port`, `--backend` (default `github-copilot`), `--no-rtk`, `--code-graph`, `--no-proxy`, `--learn`, `--memory`, `--verbose`, `--prepare-only` - Injects rtk/lean-ctx instructions into `AGENTS.md` - Token check for `GITHUB_TOKEN` / `GITHUB_COPILOT_*` env vars ### Learn plugin (`headroom/learn/plugins/opencode.py`) - Reads `~/.local/share/opencode/opencode.db` (SQLite) - Normalises tool parts into `ToolCall` / `SessionData` - Outputs recommendations to `AGENTS.md` via `CodexWriter` ### MCP registrar (`headroom/mcp_registry/opencode.py`) - Reads/writes `~/.config/opencode/opencode.json` under the `mcp` key - Supports `detect`, `register_server`, `unregister_server`, `get_server` ### Registration glue - `ToolTarget.OPENCODE` in `install/models.py` - `opencode_config_path()` in `install/paths.py` - Registered in `providers/install_registry.py` and `mcp_registry/install.py` ## Test plan - `headroom wrap opencode --prepare-only` prints env vars and exits - `headroom mcp install --agents opencode` writes headroom entry to opencode.json - `headroom learn opencode` mines sessions and appends to AGENTS.md <!-- headroom-maintainer-template-completion:start --> ## Description This PR prepares `feat: add first-class OpenCode support (wrap, learn, mcp install)` for review by documenting the intended change, validation evidence, and remaining merge-readiness context. Linked issues: None declared. ## Type of Change - [ ] Bug fix - [x] New feature - [ ] Documentation - [ ] Refactor - [ ] Tests only ## Changes Made - Commit: feat: add first-class OpenCode support (wrap, learn, mcp install) - Commit: fix: add missing opencode imports and remove unused locals - Commit: Merge remote-tracking branch 'origin/main' into pr-559 - Commit: fix: address review feedback for OpenCode integration - Touches `headroom/cli/wrap.py` - Touches `headroom/install/models.py` - Touches `headroom/install/paths.py` - Touches `headroom/learn/plugins/opencode.py` - Touches `headroom/mcp_registry/__init__.py` - Touches `headroom/mcp_registry/install.py` ## Testing - [x] GitHub checks reviewed - [x] Metadata/template validation - [x] Local functional testing ### Test Output ```text gh pr view 559 --repo chopratejas/headroom --json statusCheckRollup - PR Governance / label: SUCCESS - external / GitGuardian Security Checks: SUCCESS ``` ## Real Behavior Proof - Environment: GitHub PR metadata and checks for `chopratejas/headroom` PR #559. - Exact command / steps: Reviewed PR title, commits, changed files, linked issues, labels, and check rollup; appended this maintainer template completion block without replacing the author's original description. - Observed result: PR body now contains all required governance sections, checked readiness fields, and a non-placeholder validation evidence block. - Not tested: This pass updated PR metadata only; code validation remains represented by the linked GitHub checks and any author-provided evidence above. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review <!-- headroom-maintainer-template-completion:end --> --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
14e8dc4c84 |
feat(learn): weight loops in Headroom Learn + RTK-loop eval (#1160)
## Description `headroom learn` ranked recommendations by a single LLM-guessed `estimated_tokens_saved` with a flat hardcoded `confidence`, and had **no notion of a loop**. So (1) RTK re-fetch loops were invisible - RTK truncates a command's output, the agent re-runs larger-limit variants, those calls *succeed* (`is_error=False`), and `analyze()` even early-returned when a session had no failures and no events - and (2) even when surfaced, a loop ranked no higher than a one-off mistake. This adds loop-aware weighting plus the eval that reproduces an RTK loop, runs it through Learn, and checks the guardrail prevents re-triggering. Closes #1159 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - New `headroom/learn/loops.py`: `detect_loops()` (canonical signature collapses RTK pagination/limit variants; classifies error vs rtk-refetch loops; **measured** wasted tokens), `format_loops_for_digest()`, `apply_loop_weighting()`. - `analyzer.py`: detect loops up front (fixes the no-failure early-return), lead the digest with them, prioritize loops in the system prompt, re-sort after weighting. - `models.py`: `Recommendation.is_loop_guardrail` / `loop_occurrences`. - `benchmarks/rtk_loop_learn_eval.py` + `headroom/learn/fixtures.py`: the two-phase RTK-loop eval and its session fixtures. - Tests, `docs/rtk-loop-weighting.md`, CHANGELOG entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - not run (mypy not in my minimal env; see Not tested) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_learn/ -q 190 passed, 3 skipped, 1 warning in 5.85s $ ruff check <changed files> All checks passed! ``` ## Real Behavior Proof - Environment: macOS (Darwin 25.0), Python 3.10.18, fresh venv (`pip install -e` minus the optional `hnswlib`/proxy extras, which are unrelated to `learn`); real LLM via the analyzer's claude CLI backend (`HEADROOM_LEARN_CLI=claude`, claude-cli 2.1.158) — no API key used. - Exact command / steps: `HEADROOM_LEARN_CLI=claude python -c "from benchmarks.rtk_loop_learn_eval import run_eval; c=run_eval(use_real_llm=True); print(c.render())"` - Observed result: the analyzer shelled out to a real model and produced the "Commands" guardrail quoted below, naming the looping command. The digest reports the measured 5,005-token waste and asks the model to rank loops first, so the model emitted that figure; in this run the guardrail ranked **#1** and the scorecard was all-PASS (below). Caveat — real-mode is run-dependent: the rule's wording, and whether the post-hoc `apply_loop_weighting` fuzzy match fires, vary across runs (in one run it did not tag the rule). The **deterministic CI eval** (stub LLM) is the stable, reproducible artifact; this real run corroborates it. - Not tested: the analyzer's API-key path (ANTHROPIC/OPENAI/GEMINI) — exercised the equivalent claude CLI backend instead; `mypy`; a live agent *obeying* the written rule end-to-end (Phase 2 is a non-recurrence check, not a live agent — called out in the doc). Real model output from this run, ranked #1 at the measured 5,005-token weight: > **Commands** — When grepping logs (or any large file), never loop with increasing `| head -N` limits — tool output is capped at ~4 KB regardless of N, so repeated attempts return identical bytes. Instead: redirect to a temp file (`grep ... > /tmp/out.txt`) then read it, or use `grep -c` first… ```text [PASS] loop_detected (1 loop(s), ~5,005 tok wasted) [PASS] guardrail_produced [PASS] ranked_first [PASS] names_command [PASS] prescribes_fix [PASS] weight_reflects_waste [PASS] guardrail_holds RESULT: PASS ``` (One real-mode run via the claude CLI backend. The deterministic `pytest` eval above is the stable artifact; see the run-dependence caveat under Observed result.) The real run also caught an over-brittle check: an earlier `names_command` required the literal "TimeoutError"; the real model wrote a *more general* rule (grep + `head -N`) without it, so I fixed the check to verify the looping **command** is named, not an incidental literal. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - No new dependencies. No network, no user/assistant content dropped — operates on already-captured session digests. - Kept as one logical change. mypy not run locally (minimal env); happy to address anything CI's mypy flags. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
ced75e4718 |
feat(learn): write per-project learnings to CLAUDE.local.md by default (#1115)
## Description `headroom learn` wrote per-project learnings into the project's `CLAUDE.md`, which Claude Code treats as team-shared and git-tracked. That meant machine-specific absolute paths and tool-discovery byproducts polluted the shared file for every teammate. This switches the default to the personal, gitignored `CLAUDE.local.md`, adds a `--target` override, and migrates any stale block out of `CLAUDE.md`. Closes #1072. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `ClaudeCodeWriter` now writes CONTEXT_FILE recommendations to `CLAUDE.local.md` by default instead of `CLAUDE.md` (the home-directory case still uses `~/.claude/CLAUDE.md`, which is personal global memory). - Added a `--target` flag (Claude Code only) and `set_context_target()` to override the destination — e.g. `--target CLAUDE.md` to opt back into the shared file, or any relative/absolute path. - On first run after upgrade, a stale Headroom block left in `CLAUDE.md` is moved into `CLAUDE.local.md` and stripped from `CLAUDE.md`, with a warning surfaced by the CLI. If `CLAUDE.md` held nothing but the block, the empty file is removed. - `WriteResult` carries `warnings`; the `learn` CLI prints them. - Updated docs (`failure-learning.mdx`) and `CHANGELOG.md`. This implements the maintainer's stated preference order from the issue (default → `CLAUDE.local.md`, plus a `--target` flag), scoped to the Claude writer only — `AGENTS.md`/`GEMINI.md` have no `.local` convention and are untouched. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_learn/ tests/test_cli_learn.py -q 196 passed, 2 skipped in 17.80s $ ruff check headroom/learn/writer.py headroom/cli/learn.py All checks passed! $ mypy headroom/learn/writer.py headroom/cli/learn.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: macOS (Darwin), Python 3.11, headroom on rebased upstream/main - Exact command / steps: ran ClaudeCodeWriter against a temp project whose `CLAUDE.md` held hand-written content plus a legacy Headroom block, then `writer.write([...], dry_run=False)` - Observed result: `CLAUDE.md` kept its hand-written content with the block removed; `CLAUDE.local.md` gained both the migrated `### Old` section and the new `### Env` section; `result.warnings` contained the "Moved Headroom learnings out of …" notice. A block-only `CLAUDE.md` was deleted and a "Removed …" warning emitted. - Not tested: live end-to-end `headroom learn --apply` against real LLM analysis (writer + CLI plumbing covered by unit/CLI tests with mocked analysis) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated CHANGELOG.md if applicable ## Additional Notes Scoped to the Claude Code writer per the issue. After migration, `discover_projects` may briefly re-surface a section the LLM re-derives, but the write-side merge dedups by section name so the file stays correct. |
||
|
|
b4571cc346 |
feat: headroom wrap opencode / unwrap opencode CLI (#1105)
## Summary This PR implements transparent `headroom wrap opencode` support without asking users to edit OpenCode provider URLs, choose an extra CLI flag, or maintain a static provider list. The wrapper now lives at the runtime transport boundary: OpenCode keeps its user/provider config, while Headroom intercepts outbound provider traffic in-process and routes it through the local Headroom proxy. ## What changed ### Transparent OpenCode wrapping - `headroom wrap opencode` injects the `headroom-opencode` plugin through `OPENCODE_CONFIG_CONTENT`. - Existing OpenCode provider URLs are preserved. We do not rewrite user config URLs to point at Headroom. - Existing `OPENAI_BASE_URL` and `ANTHROPIC_BASE_URL` env vars are preserved. - Local OpenCode traffic, localhost traffic, and Headroom proxy traffic bypass the shim to avoid loops. ### Runtime transport interception - Added an OpenCode plugin transport shim that wraps: - `globalThis.fetch` - `http.request` / `http.get` - `https.request` / `https.get` - External provider calls are routed to the local Headroom proxy. - The original upstream origin is passed through `x-headroom-base-url`, so the proxy can forward to the real provider without changing OpenCode config. - External `http2.connect` is blocked loudly instead of allowing direct provider traffic to leak outside Headroom. ### Live provider additions Provider coverage is no longer based on a static config scan. Because routing happens at outbound request time, providers added mid-session are routed through Headroom automatically as long as they use the covered Node transport paths. ### Subagent and child-process coverage - The parent OpenCode plugin sets a packaged Node preload shim through `NODE_OPTIONS=--import=.../hook-shim/handler.js`. - The transport shim patches `child_process.spawn`, `exec`, `execFile`, and `fork` so child Node processes receive the Headroom preload even when OpenCode passes a custom `env`. - The child-process shim fails closed if it loads without `HEADROOM_OPENCODE_TRANSPORT_PROXY_URL`. - This closes the subagent leak path where a child Node process could otherwise start without Headroom transport interception. ## Why this goes beyond PR #1089 PR #1089 improves OpenCode provider registration, but it still focuses on provider config shape. This PR moves the enforcement boundary to runtime transport interception. This PR goes further because: - No provider URL rewriting is required. - New providers added mid-session are covered automatically. - Subagents and child Node processes inherit the Headroom transport shim. - Direct external HTTP/2 paths fail loudly instead of leaking. - The wrap remains transparent to the user's OpenCode provider config. - The wrapper is fail-closed for unsupported child-process preload state. ## Additional robustness fixes While validating the change in Docker, the full Python suite exposed unrelated Linux/container robustness issues. These are fixed in this PR so the suite is green: - Binary cache handling now treats cache paths under a non-writable existing parent as unavailable, including when tests run as root in Docker. - `release_version.py` honors `MANUAL_VER` before git calls so direct script execution works outside a `.git` checkout. - Test logger isolation now resets relevant Headroom child loggers so proxy logging setup cannot poison later `caplog` tests. - The scanner missing-path test now uses a guaranteed missing `tmp_path` child instead of relying on `/nonexistent/path`. ## Validation All implementation validation was run inside Docker. - Full Python suite from a fresh Docker copy: `6605 passed, 523 skipped`. - Ruff on changed Python/OpenCode paths: passed. - OpenCode plugin typecheck: passed. - OpenCode plugin tests: `9 passed`. - OpenCode plugin build: passed. - Hook shim preload smoke test: passed. ## Notes This PR intentionally does not add a CLI option. `headroom wrap opencode` means full wrap. Either Headroom wraps OpenCode transparently, or the path fails loudly instead of silently leaking provider traffic. --------- Co-authored-by: Rudimar Ronsoni <6081613+rudironsoni@users.noreply.github.com> |
||
|
|
6129808462 |
Fix headroom learn crashing/no-op on Windows from missing UTF-8 encoding (#1239)
## Description Fixes #1202. On a Windows (cp1252) locale, `headroom learn` cannot complete a run: the whole pipeline opens transcript files and pipes analyzer prompts without `encoding="utf-8"`, so any non-ASCII byte (em-dashes, arrows — ubiquitous in code and prose) breaks it. Same bug class already fixed for `headroom wrap` (#65, #1126) and the dashboard (#533), never swept through `learn`. Three independent failure points, each hidden behind the previous: 1. **Reading transcripts** — six bare `open()` calls in the learn plugins. The **Codex** JSONL scanner caught only `OSError`, so a `UnicodeDecodeError` propagated and **aborted the whole cross-agent run**; the **Claude** scanner caught it and **silently dropped the session**. `analyzer.py` also read the user's own CLAUDE.md/MEMORY.md with no encoding. 2. **Analyzer subprocess** — `subprocess.run`/`Popen(..., text=True)` with no encoding raised `UnicodeEncodeError` on the piped prompt; it was swallowed, so the run produced **0 recommendations** with no obvious failure. 3. **`--apply` merge** — `writer.py` read the existing context file with strict `encoding="utf-8"`, which aborts on a single stray legacy byte. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `learn/plugins/{claude,codex,gemini}.py`: add `encoding="utf-8", errors="replace"` to the six transcript `open()` calls. - `learn/analyzer.py`: same on the `read_text` of the user's context files and on both analyzer subprocess calls (`subprocess.run` and `Popen`). - `learn/writer.py`: add `_read_text_tolerant` — decode the to-be-rewritten context file as UTF-8, falling back to UTF-8-with-replacement on a stray byte (a whole-file cp1252 fallback is wrong: it mojibakes genuine UTF-8 em-dashes); the subsequent `write_text(encoding="utf-8")` self-heals the file. - `cli/learn.py`: wrap `plugin.scan_project` so one unreadable agent/project is skipped with a warning instead of aborting the whole run. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_learn/test_writer.py tests/test_learn/test_plugin_encoding.py -q 22 passed $ ruff check headroom/learn/plugins/*.py headroom/learn/analyzer.py \ headroom/learn/writer.py headroom/cli/learn.py tests/test_learn/test_*.py All checks passed! ``` New tests are **red on the old code, green with the fix**: - `test_plugin_encoding.py` — a transcript with a stray `0x9d` byte (undefined in cp1252 *and* an invalid UTF-8 start byte, so a bare `open()` fails on any locale): the Codex scanner no longer raises, the Claude scanner now recovers the session instead of dropping it. - `test_writer.py::TestEncodingResilience` — `_read_text_tolerant` preserves valid UTF-8 (no mojibake) and `--apply` merges over a file with a stray byte. ## Real Behavior Proof - Environment: Windows 11, Python 3.10, against the real learn plugins/writer (no live LLM backend; the decode failures occur before any backend call). - Exact command / steps: write a Claude transcript and a Codex rollout JSONL containing a valid em-dash/arrow line plus a stray `0x9d` byte, then call `ClaudeCodePlugin._scan_session` / `CodexPlugin._scan_jsonl_session`; for the writer, `write_bytes` an `AGENTS.md` with a stray `0x97` and run `_merge_into_file`. - Observed result: **before** the fix → `CodexPlugin._scan_jsonl_session` raises `UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9d` (aborts the run) and `ClaudeCodePlugin._scan_session` returns `None` (session dropped); **after** → Codex completes, Claude returns the `SessionData` (`total_input_tokens == 5`), and `_merge_into_file` keeps `Notes — existing` with no mojibake. - Not tested: a full end-to-end `headroom learn --apply` against live agent histories + a real LLM backend (verified at the plugin/writer level, which is where the decode failures live). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review |
||
|
|
0ddd4ed9e9 |
fix(learn): scan subagent and workflow transcripts (#1045)
## Description `headroom learn` only scanned top-level main Claude Code sessions (`<project>/<uuid>.jsonl`). Nested subagent and workflow transcripts under `<project>/<uuid>/subagents/**` were not opened, which hid a large amount of tool-call failure and token-spend activity from failure mining and downstream analysis. This change makes the Claude scanner descend into nested transcripts by default and tag each `SessionData` with its source. `--main-only` restores the previous top-level-only scan scope. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Updated `ClaudeCodePlugin.scan_project` to discover nested subagent and workflow transcripts by default. - Added source tagging for `main`, `subagent`, and `workflow` sessions. - Added `--main-only` and `include_subagents` plumbing so callers can opt back into top-level-only scanning. - Added the `include_subagents` scanner parameter to Codex/Gemini as a documented no-op because those scanners use flat session layouts. - Added regression tests for nested discovery, source tagging, parallel scanning, and CLI flag threading. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text Full learn + CLI suite: # 186 passed, 2 skipped GitHub Actions CI for this PR: # build, build-wheel, lint, tests, e2e, CodeQL, and native wrapper checks passed ``` ## Real Behavior Proof - Environment: local Claude Code corpus with nested subagent/workflow transcripts. - Exact command / steps: Scanned the corpus with the previous top-level-only behavior and then with nested transcript discovery enabled. - Observed result: The scanner saw 24 sessions before and 306 sessions after descending into nested transcripts. - Not tested: Codex/Gemini nested transcript discovery, because those providers currently use flat session layouts and treat `include_subagents` as a no-op. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
2d3701b59e |
fix(learn): decode directory names with spaces in Windows project paths (#997) (#1027)
## Description `headroom learn --apply` crashes with `FileNotFoundError` when the project lives in a Windows directory whose name contains spaces (e.g. `C:\Users\user\Desktop\Claude Code Projects`). Claude Code encodes that path as `-C-Users-user-Desktop-Claude-Code-Projects`, using `-` for both path separators *and* spaces. The greedy path decoder walks the real filesystem to reconstruct the original components, but `_component_tokenizations()` never tried splitting on spaces — so it couldn't match `Claude Code Projects` against tokens `["Claude", "Code", "Projects"]`. Closes #997 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `" "` (space) to the explicit separator list in `_component_tokenizations()` - Updated the catch-all regex from `[-._]` to `[-.\s_]` so the combined split also covers whitespace - Same change in the hidden-component (dotfile) branch ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality ### Test Output ```text tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_single_space_in_dirname PASSED tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_multiple_spaces_in_dirname PASSED tests/test_learn/test_scanner.py::TestGreedyPathDecode::test_space_nested_path PASSED tests/test_learn/test_scanner.py::TestDecodeProjectPath::test_windows_path_with_spaces_decoded_via_greedy PASSED 4 passed in 0.64s ``` ## Real Behavior Proof - Environment: Windows 11 Home 10.0.26200, Python 3.10.18 - Exact command / steps: Ran `python -m pytest tests/test_learn/test_scanner.py -v` on Windows after applying the fix. Also verified `_component_tokenizations("Claude Code Projects")` returns `[['Claude Code Projects'], ['Claude', 'Code', 'Projects']]`. The integration test creates a real temp directory with spaces and asserts `_decode_project_path()` resolves it correctly. - Observed result: All 4 new tests pass on Windows. All 34 scanner tests pass. Ruff check clean. - Not tested: No manual `headroom learn --apply` end-to-end run, but the integration test exercises the same `_decode_project_path` code path with a real temp directory on disk. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes ## Additional Notes The fix follows the exact same pattern used for underscores (issue #159) and dots (issue #47) — extending the separator list. Spaces are the last common character that Claude Code flattens to `-` but the decoder didn't know about. |