fix(learn): merge git worktree sessions into the repo's project (#3854)

## Description

`headroom learn` treats every git worktree as its own project. Claude
Code stores each working directory's sessions in a separate
`~/.claude/projects/<encoded-cwd>/` folder, and
`ClaudeCodePlugin.discover_projects` turns each folder into one
`ProjectInfo`. So a linked worktree (a Conductor workspace,
`.claude/worktrees/*`, any `git worktree add` checkout) shows up as a
separate project with a few sessions:

- Each worktree rarely has enough sessions to cross a pattern threshold,
so the repo's learnings never add up.
- Learnings are written into the worktree: `CLAUDE.local.md` in a
checkout that is deleted when the workspace is archived, and `MEMORY.md`
in that worktree's own memory folder.
- For a repo worked on only through worktrees (the usual Conductor
setup), `headroom learn --project <repo>` finds nothing: `No matching
project. Try --all or --project <path>.`
- The traffic learner uses the same roots, so live patterns that mention
a worktree path go to the worktree too.

A user reported this: their project list was a row of Conductor
workspace names, each "not scanned yet".

This PR merges each linked worktree into the project of its main
checkout.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `learn/plugins/claude.py`: new `_main_worktree_root(path)` resolves a
linked worktree to its main checkout by reading git's own pointer files:
the `.git` file's `gitdir:` line, then that gitdir's `commondir`. It
does not run git. It returns None for a main checkout, a submodule (its
gitdir has no `commondir`), a bare repo, a relative path, or anything
outside git.
- `discover_projects` now ends in `_merge_worktrees`, which groups
projects by that root. The merged project:
  - keeps the main checkout's `project_path` and session folder;
  - writes to the main checkout's `CLAUDE.local.md`;
- writes `MEMORY.md` to the main checkout's memory folder, the same
folder Claude Code's own auto-memory uses for every worktree of a repo.
If only worktrees have sessions, the merged project uses the folder
Claude Code would give the main checkout. Projects outside worktrees are
unchanged.
- `ProjectInfo` gains `worktree_paths` and `extra_data_paths` (both
default to empty lists). `scan_project` and the `--verbosity` path read
sessions from `data_path` plus `extra_data_paths`.
- `memory/traffic_learner.py`: `_project_for_pattern` also matches paths
under a project's `worktree_paths` and routes those patterns to the
project. Longest match still wins.
- A worktree whose checkout has been deleted has no `.git` file left to
resolve, so it stays a separate project, as before.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

`mypy` passes on the four changed modules. `mypy headroom` reports one
existing error in `headroom/integrations/langchain/chat_model.py:394`
(`truthy-function`), a file this PR does not touch.

### Test Output

```text
$ uv run --extra dev pytest tests/test_learn/ tests/test_memory/ tests/test_cli_learn.py tests/test_learn_grok_plugin.py tests/test_verbosity_learn.py -q
966 passed, 4 skipped, 79 warnings in 44.31s

$ uv run --extra dev ruff check . && uv run --extra dev ruff format --check .
All checks passed!
1671 files already formatted

$ uv run --extra dev mypy headroom/learn/plugins/claude.py headroom/learn/models.py headroom/memory/traffic_learner.py headroom/cli/learn.py
Success: no issues found in 4 source files
```

With the source reverted to `main`,
`tests/test_learn/test_worktree_merge.py` fails (it imports
`_main_worktree_root`).

## Real Behavior Proof

- Environment: macOS 15 (Darwin 24.6), Python 3.12, git 2.x; a scratch
repo plus a real `git worktree add` checkout; a scratch
`CLAUDE_CONFIG_DIR` with three Claude Code session files stored under
the worktree's encoded folder name.
- Exact command / steps: `git init repo && git -C repo commit
--allow-empty -m init && git -C repo worktree add ../ws/san-salvador -b
feat`, then `CLAUDE_CONFIG_DIR=<scratch>/claude headroom learn
--verbosity --agent claude --project <scratch>/repo`, run once on `main`
and once on this branch.
- Observed result: on `main`: `No matching project. Try --all or
--project <path>.` On this branch: `Verbosity — repo`, `Path:
<scratch>/repo`, `Sessions: 3`. The real git layout matches the test
fixtures: the worktree's `.git` is `gitdir:
<repo>/.git/worktrees/san-salvador`, and `commondir` is `../..`.
- Not tested: Windows worktrees on a real machine (the resolver uses
only `pathlib`); the full LLM analysis path of `headroom learn --apply`
(it shares `discover_projects`/`scan_project` with the verbosity path
shown above).

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes, for users with git worktrees
only: `headroom learn` and the traffic learner now treat a linked
worktree as part of its main checkout's project instead of as a separate
project. Repos without worktrees are unchanged.
- Kill switch / disable path: Not applicable.
- Unsafe override required: No.
- Qualification impact: None. Learn and the memory writer only; the
proxy request path and compression are untouched.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

- No documentation change: the learn docs don't describe how projects
are discovered.
- Open question for maintainers: Conductor-style workspaces are fresh
checkouts, so they never see the main checkout's `CLAUDE.local.md`. Only
`MEMORY.md` is shared across a repo's worktrees. For a repo worked on
only through worktrees, CONTEXT_FILE rules may be better routed to
`MEMORY.md`. I've left that out of this PR to keep it to one behavior
change.
- Follow-up idea, also not included: a worktree whose checkout has been
deleted could be attributed through its session folder name. Today it
stays a separate project, as before.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3854"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
gglucass
2026-10-05 08:25:53 -05:00
committed by GitHub
co-authored by Claude Opus 5.5 JD Davis
parent 4d27d024b0
commit 3fdf18eded
6 changed files with 325 additions and 21 deletions
+19 -7
View File
@@ -13,6 +13,16 @@ if TYPE_CHECKING:
from .main import main
def _projects_at(projects: list[Any], path: Path) -> list[Any]:
"""Projects checked out at ``path``, including linked worktrees merged into one.
``getattr``: external plugins may return duck-typed projects without the field.
"""
return [
p for p in projects if p.project_path == path or path in getattr(p, "worktree_paths", ())
]
class _AgentChoice(click.ParamType):
"""Dynamic Click type that validates against the plugin registry."""
@@ -257,15 +267,15 @@ def learn(
targets = all_projects
elif project:
resolved = project.resolve()
targets = [p for p in all_projects if p.project_path == resolved]
targets = _projects_at(all_projects, resolved)
if not targets:
continue
else:
cwd = Path.cwd().resolve()
targets = [p for p in all_projects if p.project_path == cwd]
targets = _projects_at(all_projects, cwd)
if not targets:
for parent in cwd.parents:
targets = [p for p in all_projects if p.project_path == parent]
targets = _projects_at(all_projects, parent)
if targets:
break
if not targets and len(agent_configs) == 1:
@@ -496,13 +506,13 @@ def _run_verbosity(
targets = all_projects
elif project:
resolved = project.resolve()
targets = [p for p in all_projects if p.project_path == resolved]
targets = _projects_at(all_projects, resolved)
else:
cwd = Path.cwd().resolve()
targets = [p for p in all_projects if p.project_path == cwd]
targets = _projects_at(all_projects, cwd)
if not targets:
for parent in cwd.parents:
targets = [p for p in all_projects if p.project_path == parent]
targets = _projects_at(all_projects, parent)
if targets:
break
if not targets:
@@ -521,7 +531,9 @@ def _run_verbosity(
analyzed_count = 0
for proj in targets:
session_paths = sorted(proj.data_path.glob("*.jsonl"))
session_paths = sorted(
f for d in (proj.data_path, *proj.extra_data_paths) for f in d.glob("*.jsonl")
)
if not session_paths:
continue
profile, baseline = analyze(session_paths, str(proj.project_path), llm_judge=judge)
+4
View File
@@ -134,6 +134,10 @@ class ProjectInfo:
data_path: Path # Where conversation logs are stored
context_file: Path | None = None # CLAUDE.md / .cursorrules / AGENTS.md
memory_file: Path | None = None # MEMORY.md or equivalent
# Linked git worktrees folded into this project: their checkouts, and the
# extra folders their conversation logs are stored in.
worktree_paths: list[Path] = field(default_factory=list)
extra_data_paths: list[Path] = field(default_factory=list)
# =============================================================================
+109 -9
View File
@@ -5,6 +5,7 @@ Reads conversation logs from ~/.claude/projects/ (JSONL format).
from __future__ import annotations
import dataclasses
import json
import logging
import re
@@ -115,7 +116,70 @@ class ClaudeCodePlugin(LearnPlugin, ConversationScanner):
)
)
return projects
return self._merge_worktrees(projects)
def _merge_worktrees(self, projects: list[ProjectInfo]) -> list[ProjectInfo]:
"""Fold each linked git worktree into the project of its main checkout.
Claude Code files every working directory's sessions under its own
folder, so each worktree (a Conductor workspace, ``.claude/worktrees/*``)
would otherwise be its own project: a handful of sessions, too thin to
cross a pattern threshold, with learnings written into a checkout that
is deleted with the workspace. The merged project keeps the main
checkout's path and memory folder, which Claude Code's auto-memory
shares across a repo's worktrees, and scans every member's sessions.
A worktree whose checkout is gone has nothing to resolve it from and
stays separate.
"""
groups: dict[Path, list[tuple[ProjectInfo, Path | None]]] = {}
for project in projects:
root = _main_worktree_root(project.project_path)
try:
key = root or project.project_path.resolve()
except OSError:
key = project.project_path
groups.setdefault(key, []).append((project, root))
merged: list[ProjectInfo] = []
for key, members in groups.items():
if all(root is None for _, root in members):
merged.extend(project for project, _ in members)
continue
main = next((project for project, root in members if root is None), None)
# A repo worked on only through worktrees has no session folder of
# its own; use the one Claude Code would give its main checkout.
data_path = (
main.data_path
if main
else self.projects_dir / re.sub(r"[^A-Za-z0-9]", "-", str(key))
)
project_path = main.project_path if main else key
claude_md = project_path / "CLAUDE.md"
memory_file = data_path / "memory" / "MEMORY.md"
merged.append(
dataclasses.replace(
main or members[0][0],
name=_project_display_name(project_path, data_path.name),
project_path=project_path,
data_path=data_path,
context_file=claude_md if _path_exists(claude_md) else None,
memory_file=memory_file if _path_exists(memory_file) else None,
# The session folder can be a subdirectory of the checkout;
# record the checkout root too so selecting the worktree
# itself, or a path elsewhere in it, still finds this project.
worktree_paths=list(
dict.fromkeys(
path
for p, root in members
if root is not None
for path in (p.project_path, _checkout_root(p.project_path))
if path is not None
)
),
extra_data_paths=[p.data_path for p, _ in members if p is not main],
)
)
return merged
@staticmethod
def _project_path_from_session_cwd(jsonl_files: list[Path]) -> Path | None:
@@ -150,15 +214,16 @@ class ClaudeCodePlugin(LearnPlugin, ConversationScanner):
by default we descend into them. Pass ``include_subagents=False`` to
restrict to top-level main sessions only.
"""
data_path = project.data_path
if include_subagents:
jsonl_files = sorted(data_path.rglob("*.jsonl"))
else:
jsonl_files = sorted(data_path.glob("*.jsonl"))
if not jsonl_files:
file_sources: list[tuple[Path, str]] = []
for data_path in (project.data_path, *project.extra_data_paths):
if include_subagents:
jsonl_files = sorted(data_path.rglob("*.jsonl"))
else:
jsonl_files = sorted(data_path.glob("*.jsonl"))
file_sources.extend((f, self._classify_source(data_path, f)) for f in jsonl_files)
if not file_sources:
return []
file_sources = [(f, self._classify_source(data_path, f)) for f in jsonl_files]
jsonl_files = [f for f, _ in file_sources]
if max_workers <= 1 or len(jsonl_files) <= 1:
return [
@@ -436,6 +501,41 @@ def _decode_project_path(escaped_name: str) -> Path | None:
return None
def _checkout_root(path: Path) -> Path | None:
"""The nearest directory at or above ``path`` holding a ``.git`` entry."""
try:
return next((d for d in (path, *path.parents) if _path_exists(d / ".git")), None)
except OSError:
return None
def _main_worktree_root(path: Path) -> Path | None:
"""Return the main checkout of the linked git worktree ``path`` is in.
Returns None for a main checkout, a submodule, a bare repo, or anything
outside git. Reads git's own pointer files rather than running git.
"""
# A relative path (a Windows path decoded on POSIX) would walk up into the
# process's working directory and match whatever repo that sits in.
if not path.is_absolute():
return None
try:
checkout = _checkout_root(path)
if checkout is None:
return None
# A main checkout's .git is a directory, so this read fails there.
pointer = (checkout / ".git").read_text(encoding="utf-8").strip()
if not pointer.startswith("gitdir:"):
return None
gitdir = checkout / pointer[len("gitdir:") :].strip()
# Only a linked worktree's gitdir has `commondir` (a submodule's does not).
common = (gitdir / (gitdir / "commondir").read_text(encoding="utf-8").strip()).resolve()
except (OSError, UnicodeDecodeError):
return None
# A bare repo has no checkout to merge into.
return common.parent if common.name == ".git" else None
def _project_display_name(project_path: Path, fallback: str) -> str:
"""Return a human project name for POSIX and Windows-style decoded paths."""
rendered = str(project_path)
+8 -3
View File
@@ -1729,12 +1729,17 @@ def _project_for_pattern(pattern: ExtractedPattern, roots: list[ProjectInfo]) ->
if not candidates:
return None
# A path inside a worktree folded into a project belongs to that project.
# Longest root first — most specific wins
roots_sorted = sorted(roots, key=lambda p: len(str(p.project_path)), reverse=True)
roots_sorted = sorted(
((path, root) for root in roots for path in (root.project_path, *root.worktree_paths)),
key=lambda item: len(str(item[0])),
reverse=True,
)
for cand in candidates:
for root in roots_sorted:
root_str = str(root.project_path).rstrip("/\\")
for path, root in roots_sorted:
root_str = str(path).rstrip("/\\")
if not root_str:
continue
if (
+34 -2
View File
@@ -208,6 +208,34 @@ def test_learn_project_lookup_and_apply_flow(
assert plugin.writer.calls[0][2] is False
@pytest.mark.parametrize("from_cwd", [False, True])
def test_learn_selects_the_project_a_worktree_was_merged_into(
monkeypatch: pytest.MonkeyPatch, runner: CliRunner, tmp_path: Path, from_cwd: bool
) -> None:
main_checkout, worktree = tmp_path / "repo", tmp_path / "workspaces" / "ws"
(worktree / "src").mkdir(parents=True)
worktree = worktree.resolve()
merged = SimpleNamespace(
name="repo", project_path=main_checkout, worktree_paths=[worktree], extra_data_paths=[]
)
plugin = FakePlugin("claude", "Claude Code", [merged])
analyzer = FakeAnalyzer()
monkeypatch.setattr("headroom.learn.analyzer._detect_default_model", lambda: "gpt-4o")
monkeypatch.setattr("headroom.learn.registry.get_plugin", lambda name: plugin)
monkeypatch.setattr("headroom.learn.analyzer.SessionAnalyzer", lambda model=None: analyzer)
args = ["learn", "--agent", "claude"]
if from_cwd:
monkeypatch.chdir(worktree / "src")
else:
args += ["--project", str(worktree)]
result = runner.invoke(main, args, catch_exceptions=False)
assert result.exit_code == 0, result.output
assert [call[0] for call in plugin.scan_calls] == [merged]
class ProgressEchoingAnalyzer(FakeAnalyzer):
def analyze(self, project, sessions, on_progress=None): # noqa: ANN001, ANN201
self.calls.append((project, sessions))
@@ -263,8 +291,12 @@ def test_verbosity_all_apply_aggregates_baselines_across_projects(
for d in (proj_a_dir, proj_b_dir):
d.mkdir()
(d / "s.jsonl").write_text("{}")
proj_a = SimpleNamespace(name="a", project_path=tmp_path / "src-a", data_path=proj_a_dir)
proj_b = SimpleNamespace(name="b", project_path=tmp_path / "src-b", data_path=proj_b_dir)
proj_a = SimpleNamespace(
name="a", project_path=tmp_path / "src-a", data_path=proj_a_dir, extra_data_paths=[]
)
proj_b = SimpleNamespace(
name="b", project_path=tmp_path / "src-b", data_path=proj_b_dir, extra_data_paths=[]
)
plugin = FakePlugin("claude", "Claude Code", [proj_a, proj_b])
# Per-project synthetic baselines. Project A has more samples, so its level
+151
View File
@@ -0,0 +1,151 @@
"""A linked git worktree's sessions belong to its repo's learn project.
Claude Code files each working directory's sessions under its own
``~/.claude/projects`` folder, so a worktree (a Conductor workspace,
``.claude/worktrees/*``) used to surface as its own project.
"""
from __future__ import annotations
import json
import re
from pathlib import Path
from headroom.learn.plugins.claude import ClaudeCodePlugin, _main_worktree_root
from headroom.memory.traffic_learner import (
ExtractedPattern,
PatternCategory,
_project_for_pattern,
)
def _repo_with_worktree(base: Path) -> tuple[Path, Path]:
main, worktree = base / "repo", base / "workspaces" / "san-salvador"
gitdir = main / ".git" / "worktrees" / "san-salvador"
gitdir.mkdir(parents=True)
(gitdir / "commondir").write_text("../..\n")
worktree.mkdir(parents=True)
(worktree / ".git").write_text(f"gitdir: {gitdir}\n")
return main, worktree
def _folder(claude_dir: Path, cwd: Path) -> Path:
return claude_dir / "projects" / re.sub(r"[^A-Za-z0-9]", "-", str(cwd))
def _write_session(claude_dir: Path, cwd: Path, name: str) -> None:
call = {"type": "tool_use", "id": "u1", "name": "Bash", "input": {"command": "ls"}}
result = {"type": "tool_result", "tool_use_id": "u1", "content": "ok"}
lines = [
{"type": "assistant", "cwd": str(cwd), "message": {"content": [call]}},
{"type": "user", "cwd": str(cwd), "message": {"content": [result]}},
]
folder = _folder(claude_dir, cwd)
folder.mkdir(parents=True, exist_ok=True)
(folder / f"{name}.jsonl").write_text("\n".join(json.dumps(x) for x in lines) + "\n")
def test_main_worktree_root_resolves_linked_worktrees_only(tmp_path: Path) -> None:
base = tmp_path.resolve()
main, worktree = _repo_with_worktree(base)
(worktree / "src").mkdir()
submodule, sub_gitdir = main / "vendor" / "lib", main / ".git" / "modules" / "lib"
submodule.mkdir(parents=True)
sub_gitdir.mkdir(parents=True)
(submodule / ".git").write_text("gitdir: ../../.git/modules/lib\n")
assert _main_worktree_root(worktree) == main
assert _main_worktree_root(worktree / "src") == main
assert _main_worktree_root(main) is None
assert _main_worktree_root(submodule) is None
assert _main_worktree_root(base) is None
def test_relative_path_never_resolves_through_the_working_directory(
tmp_path: Path, monkeypatch
) -> None:
_, worktree = _repo_with_worktree(tmp_path.resolve())
monkeypatch.chdir(worktree)
assert _main_worktree_root(Path("C:\\Users\\dev\\work")) is None
assert _main_worktree_root(Path("src")) is None
def test_worktree_sessions_merge_into_the_main_checkout(tmp_path: Path) -> None:
base = tmp_path.resolve()
main, worktree = _repo_with_worktree(base)
claude_dir = base / "claude"
_write_session(claude_dir, main, "a")
_write_session(claude_dir, worktree, "b")
plugin = ClaudeCodePlugin(claude_dir=claude_dir)
[project] = plugin.discover_projects()
assert project.project_path == main
assert project.data_path == _folder(claude_dir, main)
assert project.worktree_paths == [worktree]
assert project.extra_data_paths == [_folder(claude_dir, worktree)]
assert len(plugin.scan_project(project)) == 2
def test_worktree_only_repo_gets_the_main_checkouts_memory_folder(tmp_path: Path) -> None:
# Conductor shape: every session runs in a workspace, none in the repo.
base = tmp_path.resolve()
main, worktree = _repo_with_worktree(base)
claude_dir = base / "claude"
_write_session(claude_dir, worktree, "a")
plugin = ClaudeCodePlugin(claude_dir=claude_dir)
[project] = plugin.discover_projects()
assert project.project_path == main
assert project.name == "repo"
assert project.data_path == _folder(claude_dir, main)
assert project.extra_data_paths == [_folder(claude_dir, worktree)]
assert len(plugin.scan_project(project)) == 1
def test_projects_outside_worktrees_are_untouched(tmp_path: Path) -> None:
base = tmp_path.resolve()
one, two = base / "one", base / "two"
one.mkdir()
two.mkdir()
claude_dir = base / "claude"
_write_session(claude_dir, one, "a")
_write_session(claude_dir, two, "b")
projects = ClaudeCodePlugin(claude_dir=claude_dir).discover_projects()
assert sorted(p.project_path for p in projects) == [one, two]
assert all(not p.worktree_paths and not p.extra_data_paths for p in projects)
def test_traffic_pattern_in_a_worktree_routes_to_its_repo(tmp_path: Path) -> None:
base = tmp_path.resolve()
main, worktree = _repo_with_worktree(base)
claude_dir = base / "claude"
_write_session(claude_dir, main, "a")
_write_session(claude_dir, worktree, "b")
[project] = ClaudeCodePlugin(claude_dir=claude_dir).discover_projects()
pattern = ExtractedPattern(
category=PatternCategory.PREFERENCE,
content=f"Edit `{worktree}/src/app.py` rather than the generated copy.",
importance=0.5,
)
assert _project_for_pattern(pattern, [project]) is project
def test_a_session_started_in_a_worktree_subfolder_records_the_checkout(tmp_path: Path) -> None:
"""Selecting the worktree itself must find a project whose only session
started in one of its subfolders."""
base = tmp_path.resolve()
main, worktree = _repo_with_worktree(base)
(worktree / "src").mkdir()
claude_dir = base / "claude"
_write_session(claude_dir, worktree / "src", "a")
[project] = ClaudeCodePlugin(claude_dir=claude_dir).discover_projects()
assert project.project_path == main
assert project.worktree_paths == [worktree / "src", worktree]