Merge pull request #439 from Tencent/worktree-issue-374-p1-3-auto-migration

feat(data-layout): auto-migrate a legacy .teamai into the partition (#374 P1-3)
This commit is contained in:
jeff
2026-09-08 20:48:24 +08:00
committed by GitHub
6 changed files with 928 additions and 23 deletions
+79 -6
View File
@@ -1,7 +1,8 @@
# Design: teamai data directory layout — global home + per-project partitioning
> Status: **P0 implemented** (this doc ships with the P0 PR for issue #374).
> P1–P3 are follow-up phases, tracked below.
> Status: **P0 + P1 implemented** (issue #374). P1 shipped as PRs #397 / #402 /
> #406 / #414 / #417 (partition routing) and the P1-3 auto-migration below.
> P2–P3 are follow-up phases, tracked below.
## Problem
@@ -114,12 +115,84 @@ is a P1 concern. This keeps P0 independently reviewable (issue R7).
deploy to the repo root; run inside a worktree, resources land in the worktree and
the main checkout is untouched (`src/__tests__/detect-subdir.test.ts` + manual run).
## P1-3 — automatic migration (implemented)
An install created before partitioning keeps its machine data in the business repo
at `<workspaceRoot>/.teamai/`. P1-2 routed NEW installs to the partition and reads
old installs through a legacy fallback; P1-3 moves a real legacy `.teamai/` INTO the
partition on the next write command, so the workspace ends up with zero residue.
**Trigger** (`src/migrate.ts`, wired into the global `preAction` hook in `index.ts`):
- Only `init` / `pull` / `push`. Read-only commands (`status`, `recall`, …) keep using
the double-read fallback and never move data.
- `hook-dispatch` is excluded outright (via `TEAMAI_HOOK_SUBCOMMANDS`): it is a
high-frequency silent path and must never move 12 MB.
- `--dry-run` (the existing global flag) previews without writing.
**Gate** (`planMigration`, deliberately NOT `detectProjectConfig` — that
short-circuits on an existing partition and runs the self-heal bootstrap as a side
effect, both of which would mask the raw legacy state). Act iff:
- in a git repo (the partition only exists for git repos), AND
- `<workspaceRoot>/.teamai/config.yaml` exists, AND
- the legacy config is `scope: project` (user data never lives under `.teamai/`), AND
- the legacy config is NOT `kind: self` — **self mode is a hard no-op**: its `.teamai/`
is team knowledge committed to main, and `init --self` already retires any partition,
so moving it would break "knowledge on main".
The plan's **mode** then depends on the partition: a full copy when
`<partition>/config.yaml` does not exist yet, or **retire-only** when it does (a prior
run built the partition but was interrupted before retiring the source — see Interrupt
recovery). retire-only never re-copies onto the authoritative partition; it only cleans
up the leftover legacy dir.
**Steps** (`runMigration`) — copy → verify → atomic rename, so an interruption never
leaves data half-in-both-places:
```
0. Acquire <legacyDir>/.sync-lock (the exact lock an un-migrated pull/push contends
on, since their getDataHome still resolves to the legacy dir pre-migration).
Contention → skip this attempt (idempotent; the next write command retries).
1. Copy legacyDir → <partition>.staging (raw fse.copy, NOT copyDir — copyDir filters
out `.git` and would corrupt the team-repo clone). Skip reports-wt/knowledge-wt
(disposable worktrees with absolute gitdirs — rebuilt on demand) and lock files.
2. Verify staging: config.yaml parses; if the source has team-repo/.git the copy must
too; every migratable top-level entry is present. Failure → discard staging, abort,
source untouched.
3. Atomic switch: fse.rename(staging → partition) (same-filesystem, atomic).
4. Write <partition>/anchor with the projectAnchor path — the slug is a one-way
sha256, so this file is the only reverse lookup; it lives off the workspace.
5. Release the lock, then retire the source:
a. Drop a self-contained `.gitignore` (`*`) INTO legacyDir first. An old
install's `.teamai/` was often protected only by a repo-root rule matching
`.teamai/`, which does NOT match `.teamai.bak/` — so without this the rename
would expose the plaintext env/token to the next `git add`. Written before
the rename so the credentials are never in a non-ignored directory.
b. Rename legacyDir → the first FREE `.teamai.bak[.N]` name. An existing backup
(a prior migration's, or the user's own) is NEVER removed — it may hold
irreplaceable data — so we pick `.teamai.bak`, else `.teamai.bak.1`, …
The backup is NEVER auto-deleted: it is the manual rollback path.
```
Interrupt recovery: staging is a separate sibling dir, so a crash before step 3 leaves
the partition absent and the source intact — a rerun discards `.staging/` and starts
clean. A crash between steps 3 and 5 leaves the partition built with the legacy dir
still present; the next write command's `planMigration` sees "partition exists AND
legacy lingers" and returns a **retire-only** plan that finishes the job — it retires
the leftover legacy dir to `.teamai.bak/` WITHOUT re-copying onto the now-authoritative
partition. This closes the gap where the legacy dir (including its plaintext `env`)
would otherwise linger in the workspace forever, breaking the zero-residue guarantee.
The staged team-repo clone is smoke-checked (`git rev-parse HEAD`) before the rename,
so a partial/corrupt copy aborts with the source untouched rather than promoting a
broken clone. If a write command's migration fails, teamai prints a clean error and
exits non-zero (the source is intact, so a rerun retries safely) instead of surfacing
a raw async-hook rejection.
**Downgrade is not supported** — an older teamai treats a partitioned install as
uninitialized; `.teamai.bak/` is the manual rollback. Flag prominently in release notes.
## Follow-up phases (not in this PR)
- **P1** — `slug(projectAnchor) = <basename>-<sha256 prefix>`, `projectDataHome(anchor)`,
partition-level `.sync-lock` for shared clones, `status` partition print, and the
automatic migration (copy → verify → atomic rename → keep `.teamai.bak/` backup;
triggered only on write commands, never on read/hook paths).
- **P2** — self (single-repo) mode slimming: only team knowledge (class B) stays in
the repo.
- **P3** — functionize module-load-time path constants (so tests that swap `$HOME`
+26 -8
View File
@@ -115,16 +115,34 @@ teamai init https://github.com/yourorg/yourrepo
Resulting directory structure:
```
/path/to/my-project/
├── .teamai/ # Project-level config (with an auto-generated .gitignore)
│ ├── config.yaml
│ └── team-repo/
/path/to/my-project/ # your business repo — ZERO teamai residue
├── .claude/skills/ # Project-level skills (auto-synced)
├── .claude/rules/ # Project-level rules (auto-synced)
└── src/
~/.teamai/projects/my-project-<hash>/ # this project's machine-data partition
├── config.yaml
├── state.json
└── team-repo/ # clone of the team repo
```
`teamai init` writes `.teamai/` only. Per-agent project roots (`.claude/`, `.cursor/`, `.codebuddy/`, …) are created on **SessionStart** for the tool that just opened. For example, opening Claude Code creates `.claude/`, then pull writes into it. A bare `teamai pull` still skips tools whose project root does not exist, so it never invents agent directories for tools you have not opened in this project.
Project machine-data (config, state, the team-repo clone, search index, MCP
manifests, resource cache) lives in a per-project partition under
`~/.teamai/projects/<slug>/`, **not** in the business repo, so your workspace has no
teamai residue and a `git worktree` of the same repo shares one partition. Per-agent
project roots (`.claude/`, `.cursor/`, `.codebuddy/`, …) are still created inside the
workspace on **SessionStart** for the tool that just opened. For example, opening
Claude Code creates `.claude/`, then pull writes into it. A bare `teamai pull` still
skips tools whose project root does not exist, so it never invents agent directories
for tools you have not opened in this project.
> **Upgrading from an older teamai?** The first `teamai init` / `pull` / `push` after
> upgrading automatically migrates an existing `<repo>/.teamai/` into the partition
> (copy → verify → atomic switch), then leaves the old directory as `<repo>/.teamai.bak/`
> for you to delete once you've confirmed everything works. Read-only commands and the
> `hook-dispatch` path never migrate; `teamai --dry-run pull` previews the move.
> **Downgrading afterwards is not supported** — an older teamai would treat the project
> as uninitialized; `.teamai.bak/` is the manual rollback path.
If the repo has role-based skills enabled (i.e. `manifest/roles.yaml` exists), `teamai init` will also interactively ask you to choose:
@@ -140,7 +158,7 @@ teamai init https://github.com/yourorg/yourrepo --scope project --role hai_dev -
| Flag | Description |
|------|------|
| `[repo]` / `--repo <url>` | Team repo URL (positional preferred; `--repo` is a permanent alias) |
| `--scope <project\|user>` | Install scope, defaults to `project` (`<cwd>/.teamai`). Use `user` for `~/` |
| `--scope <project\|user>` | Install scope, defaults to `project` (machine-data in `~/.teamai/projects/<slug>/`, resources in `<cwd>`). Use `user` for `~/` |
| `--inherit-user-scope` | Project scope only: also sync safe user resources and search user knowledge |
| `--no-inherit-user-scope` | Disable previously configured user-scope inheritance for this project |
| `--role <id>` | Directly specify the primary role, skipping the interactive role prompt |
@@ -196,11 +214,11 @@ Example local config:
```yaml
repo:
localPath: /path/to/my-project/.teamai/team-repo
localPath: ~/.teamai/projects/my-project-<hash>/team-repo
remote: https://github.com/group/repo.git
username: alice
scope: project
projectRoot: /path/to/my-project
projectRoot: /path/to/my-project # where resources land (this checkout)
inheritUserScope: true # optional; project scope only
primaryRole: hai
additionalRoles:
+21 -8
View File
@@ -114,16 +114,29 @@ teamai init https://github.com/yourorg/yourrepo
生成的目录结构:
```
/path/to/my-project/
├── .teamai/ # 项目级配置(含自动生成的 .gitignore)
│ ├── config.yaml
│ └── team-repo/
/path/to/my-project/ # 你的业务仓库 —— 零 teamai 残留
├── .claude/skills/ # 项目级 skills(自动同步)
├── .claude/rules/ # 项目级 rules(自动同步)
└── src/
~/.teamai/projects/my-project-<hash>/ # 本项目的机器数据分区
├── config.yaml
├── state.json
└── team-repo/ # 团队仓库克隆
```
`teamai init` 只写入 `.teamai/`。各 Agent 的项目根目录(`.claude/`、`.cursor/`、`.codebuddy/` 等)会在 **SessionStart** 时按刚打开的工具创建。例如,打开 Claude Code 时会创建 `.claude/`,再由 pull 写入。单独执行 `teamai pull` 仍会跳过项目里还不存在根目录的工具,因此不会给尚未在本项目打开过的 Agent 凭空建目录。
项目的机器数据(config、state、team-repo 克隆、搜索索引、MCP manifest、资源缓存)
存放在 `~/.teamai/projects/<slug>/` 下的按项目分区里,**不再**放进业务仓库,因此工作区
无 teamai 残留,且同一仓库的 `git worktree` 共享同一分区。各 Agent 的项目根目录
(`.claude/`、`.cursor/`、`.codebuddy/` 等)仍在工作区内、于 **SessionStart** 时按刚打开的
工具创建。例如,打开 Claude Code 时会创建 `.claude/`,再由 pull 写入。单独执行 `teamai pull`
仍会跳过项目里还不存在根目录的工具,因此不会给尚未在本项目打开过的 Agent 凭空建目录。
> **从旧版 teamai 升级?** 升级后首次执行 `teamai init` / `pull` / `push` 会自动把已有的
> `<repo>/.teamai/` 迁移进分区(复制 → 校验 → 原子切换),并把旧目录保留为
> `<repo>/.teamai.bak/`,待你确认一切正常后自行删除。只读命令与 `hook-dispatch` 路径
> 永不触发迁移;`teamai --dry-run pull` 可预演。**迁移后不支持降级**——旧版会把项目判定为
> 未初始化;`.teamai.bak/` 是人工回滚路径。
如果仓库启用了角色化 skills(存在 `manifest/roles.yaml`),`teamai init` 还会交互式要求你选择:
@@ -139,7 +152,7 @@ teamai init https://github.com/yourorg/yourrepo --scope project --role hai_dev -
| 参数 | 说明 |
|------|------|
| `[repo]` / `--repo <url>` | 团队仓库地址(推荐位置参数;`--repo` 为永久别名) |
| `--scope <project\|user>` | 安装作用域,默认 `project`(`<cwd>/.teamai`)。需要装到 `~/` 时用 `user` |
| `--scope <project\|user>` | 安装作用域,默认 `project`(机器数据在 `~/.teamai/projects/<slug>/`,资源落在 `<cwd>`)。需要装到 `~/` 时用 `user` |
| `--inherit-user-scope` | 仅 project scope:同时同步安全的 user 资源并检索 user 知识 |
| `--no-inherit-user-scope` | 关闭当前项目先前配置的 user scope 继承 |
| `--role <id>` | 直接指定 primaryRole,跳过角色交互选择 |
@@ -189,11 +202,11 @@ projects:
```yaml
repo:
localPath: /path/to/my-project/.teamai/team-repo
localPath: ~/.teamai/projects/my-project-<hash>/team-repo
remote: https://github.com/yourorg/yourrepo.git
username: alice
scope: project
projectRoot: /path/to/my-project
projectRoot: /path/to/my-project # 资源落地位置(当前 checkout)
inheritUserScope: true # 可选,仅 project scope
primaryRole: hai
additionalRoles:
+383
View File
@@ -0,0 +1,383 @@
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import { execFileSync } from 'node:child_process';
import fs from 'node:fs';
import { realpathSync } from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import fse from 'fs-extra';
import YAML from 'yaml';
vi.mock('../utils/logger.js', () => ({
log: {
info: vi.fn(),
success: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
debug: vi.fn(),
dim: vi.fn(),
},
}));
import { planMigration, runMigration, maybeMigrate } from '../migrate.js';
import { projectDataHome } from '../utils/partition.js';
// ─── Real-git migration tests (issue #374 P1-3) ─────────────────────────────
//
// A legacy install kept machine data in `<repo>/.teamai/`, including a real git
// team-repo clone. Migration copies it into the partition, verifies, atomically
// renames, and retires the source to `.teamai.bak/`. These tests build a REAL
// legacy layout (real anchors + a real nested git clone) rather than an empty
// fixture, so the load-bearing details — anchor resolution, `.git` survival,
// atomicity — are genuinely exercised.
function git(cwd: string, ...args: string[]): void {
execFileSync('git', args, { cwd, stdio: 'pipe' });
}
function gitOut(cwd: string, ...args: string[]): string {
return execFileSync('git', args, { cwd, stdio: 'pipe' }).toString();
}
let base: string;
let repoRoot: string;
let homeDir: string;
let legacyDir: string;
/** Write a minimal but schema-valid legacy project config into legacyDir. */
async function writeLegacyConfig(overrides: Record<string, unknown> = {}): Promise<void> {
const cfg = {
repo: {
localPath: path.join(legacyDir, 'team-repo'),
remote: 'git@example.com:team/repo.git',
kind: 'git',
},
username: 'tester',
scope: 'project',
...overrides,
};
await fse.ensureDir(legacyDir);
await fse.writeFile(path.join(legacyDir, 'config.yaml'), YAML.stringify(cfg));
}
/** Build a real, non-empty legacy `.teamai/` with a genuine git team-repo clone. */
async function seedLegacyLayout(): Promise<void> {
await writeLegacyConfig();
await fse.writeJson(path.join(legacyDir, 'state.json'), { lastSync: 'x' });
await fse.writeFile(path.join(legacyDir, 'env'), 'TEAM_TOKEN=s3cret\n');
await fse.writeJson(path.join(legacyDir, 'search-index.json'), { docs: [] });
// A real git clone under team-repo/ — the `.git` dir is what the copyDir filter
// would silently drop, so the test must assert it survives.
const teamRepo = path.join(legacyDir, 'team-repo');
await fse.ensureDir(teamRepo);
git(teamRepo, 'init', '-q');
git(teamRepo, 'config', 'user.email', 'test@example.com');
git(teamRepo, 'config', 'user.name', 'Test');
await fse.writeFile(path.join(teamRepo, 'README'), 'team\n');
git(teamRepo, 'add', '.');
git(teamRepo, 'commit', '-q', '-m', 'seed');
// A per-worktree managed-mcp subtree (P1-2C layout).
const wsDir = path.join(legacyDir, 'workspaces', 'abc123def456');
await fse.ensureDir(wsDir);
await fse.writeJson(path.join(wsDir, 'managed-mcp.json'), { 'claude:project': {} });
}
beforeEach(() => {
base = realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'teamai-migrate-')));
repoRoot = path.join(base, 'business-repo');
fs.mkdirSync(repoRoot);
git(repoRoot, 'init', '-q');
git(repoRoot, 'config', 'user.email', 'test@example.com');
git(repoRoot, 'config', 'user.name', 'Test');
git(repoRoot, 'commit', '--allow-empty', '-q', '-m', 'init');
homeDir = path.join(base, 'home');
fs.mkdirSync(homeDir);
legacyDir = path.join(repoRoot, '.teamai');
vi.stubEnv('HOME', homeDir);
});
afterEach(() => {
vi.unstubAllEnvs();
try {
fs.rmSync(base, { recursive: true, force: true });
} catch {
/* best effort */
}
});
describe('planMigration', () => {
it('plans a migration for a legacy git-mode project install', async () => {
await seedLegacyLayout();
const plan = await planMigration(repoRoot);
expect(plan).not.toBeNull();
expect(plan!.legacyDir).toBe(legacyDir);
expect(plan!.partitionDir).toBe(projectDataHome(repoRoot));
expect(plan!.anchor).toBe(repoRoot);
});
it('skips when no legacy config.yaml exists', async () => {
expect(await planMigration(repoRoot)).toBeNull();
});
it('plans a retire-only cleanup when a partition exists but legacy lingers', async () => {
// Interrupted prior run: partition built, source never retired. Instead of
// skipping (which would leave the legacy dir — incl. plaintext env — forever),
// planMigration must return a retire-only plan to finish the cleanup.
await seedLegacyLayout();
const partition = projectDataHome(repoRoot);
await fse.ensureDir(partition);
await fse.writeFile(path.join(partition, 'config.yaml'), 'repo: {}\n');
const plan = await planMigration(repoRoot);
expect(plan).not.toBeNull();
expect(plan!.mode).toBe('retire-only');
});
it('skips a user-scope legacy config', async () => {
await writeLegacyConfig({ scope: 'user' });
expect(await planMigration(repoRoot)).toBeNull();
});
it('skips a self-mode legacy config (its .teamai is committed knowledge)', async () => {
await writeLegacyConfig({ repo: { localPath: legacyDir, remote: '', kind: 'self' } });
expect(await planMigration(repoRoot)).toBeNull();
});
it('skips outside a git repository', async () => {
const plain = path.join(base, 'plain');
fs.mkdirSync(plain);
await fse.ensureDir(path.join(plain, '.teamai'));
await fse.writeFile(
path.join(plain, '.teamai', 'config.yaml'),
YAML.stringify({ repo: { localPath: '', remote: '', kind: 'git' }, username: 'x', scope: 'project' }),
);
expect(await planMigration(plain)).toBeNull();
});
it('skips a malformed legacy config rather than throwing', async () => {
await fse.ensureDir(legacyDir);
await fse.writeFile(path.join(legacyDir, 'config.yaml'), ':::not yaml:::\n');
expect(await planMigration(repoRoot)).toBeNull();
});
});
describe('runMigration', () => {
it('migrates into the partition, keeps the git clone intact, and retires the source', async () => {
await seedLegacyLayout();
const plan = await planMigration(repoRoot);
const result = await runMigration(plan!);
expect(result).toBe('migrated');
const partition = projectDataHome(repoRoot);
// Machine data landed in the partition.
expect(await fse.pathExists(path.join(partition, 'config.yaml'))).toBe(true);
expect(await fse.pathExists(path.join(partition, 'state.json'))).toBe(true);
expect(await fse.pathExists(path.join(partition, 'search-index.json'))).toBe(true);
expect(await fse.pathExists(path.join(partition, 'workspaces', 'abc123def456', 'managed-mcp.json'))).toBe(true);
// The anchor reverse-lookup file was written.
expect((await fse.readFile(path.join(partition, 'anchor'), 'utf-8')).trim()).toBe(repoRoot);
// team-repo/.git SURVIVED — the clone is still a working repo (proves raw
// fse.copy was used, not the .git-filtering copyDir).
const migratedRepo = path.join(partition, 'team-repo');
expect(await fse.pathExists(path.join(migratedRepo, '.git'))).toBe(true);
expect(() => git(migratedRepo, 'status')).not.toThrow();
expect(() => git(migratedRepo, 'rev-parse', 'HEAD')).not.toThrow();
// Source retired to .bak, original gone → workspace zero-residue.
expect(await fse.pathExists(legacyDir)).toBe(false);
expect(await fse.pathExists(`${legacyDir}.bak`)).toBe(true);
expect(await fse.pathExists(path.join(`${legacyDir}.bak`, 'config.yaml'))).toBe(true);
});
it('rebases repo.localPath from the legacy dir onto the partition', async () => {
await seedLegacyLayout();
const plan = await planMigration(repoRoot);
await runMigration(plan!);
// The migrated config must name the team-repo INSIDE the partition, not the
// now-retired legacy path — otherwise the next pull reads the wrong clone.
const partition = projectDataHome(repoRoot);
const migrated = YAML.parse(
await fse.readFile(path.join(partition, 'config.yaml'), 'utf-8'),
);
expect(migrated.repo.localPath).toBe(path.join(partition, 'team-repo'));
expect(migrated.repo.localPath).not.toContain('.teamai/team-repo');
});
it('leaves a localPath that is not inside the legacy dir untouched', async () => {
// e.g. an install whose team-repo clone lives elsewhere entirely.
const external = path.join(base, 'external-clone');
await writeLegacyConfig({
repo: { localPath: external, remote: 'git@example.com:t/r.git', kind: 'git' },
});
const plan = await planMigration(repoRoot);
await runMigration(plan!);
const migrated = YAML.parse(
await fse.readFile(path.join(projectDataHome(repoRoot), 'config.yaml'), 'utf-8'),
);
expect(migrated.repo.localPath).toBe(external);
});
it('does not carry a live sync-lock into the backup', async () => {
await seedLegacyLayout();
const plan = await planMigration(repoRoot);
await runMigration(plan!);
// The lock lived in legacyDir and must be released before the .bak rename,
// so neither the partition nor the backup keeps a stale lock.
expect(await fse.pathExists(path.join(`${legacyDir}.bak`, '.sync-lock'))).toBe(false);
expect(await fse.pathExists(path.join(projectDataHome(repoRoot), '.sync-lock'))).toBe(false);
});
it('does not copy disposable worktrees or lock files', async () => {
await seedLegacyLayout();
await fse.ensureDir(path.join(legacyDir, 'reports-wt'));
await fse.writeFile(path.join(legacyDir, 'reports-wt', 'x'), 'stale\n');
await fse.writeFile(path.join(legacyDir, '.update-lock'), '{}');
const plan = await planMigration(repoRoot);
await runMigration(plan!);
const partition = projectDataHome(repoRoot);
expect(await fse.pathExists(path.join(partition, 'reports-wt'))).toBe(false);
expect(await fse.pathExists(path.join(partition, '.update-lock'))).toBe(false);
});
it('is idempotent: a second run stands down once the partition exists', async () => {
await seedLegacyLayout();
await runMigration((await planMigration(repoRoot))!);
// Legacy is now .bak; planMigration returns null (nothing to migrate).
expect(await planMigration(repoRoot)).toBeNull();
});
it('retire-only mode retires a leftover legacy dir without re-copying (finishes an interrupted run)', async () => {
// Simulate a crash between the partition rename and the source retire: the
// partition is already built AND the legacy dir still lingers.
await seedLegacyLayout();
const partition = projectDataHome(repoRoot);
await fse.ensureDir(partition);
await fse.writeFile(path.join(partition, 'config.yaml'), 'repo:\n kind: git\n');
await fse.writeFile(path.join(partition, 'sentinel'), 'authoritative\n');
const plan = await planMigration(repoRoot);
expect(plan!.mode).toBe('retire-only');
const result = await runMigration(plan!);
expect(result).toBe('migrated');
// Legacy retired → workspace zero-residue (the plaintext env no longer lingers).
expect(await fse.pathExists(legacyDir)).toBe(false);
expect(await fse.pathExists(`${legacyDir}.bak`)).toBe(true);
// The authoritative partition was NOT overwritten by a re-copy.
expect(await fse.pathExists(path.join(partition, 'sentinel'))).toBe(true);
// A follow-up plan is now null — the workspace is clean.
expect(await planMigration(repoRoot)).toBeNull();
});
it('aborts without touching the source when the staged clone is corrupt', async () => {
// A team-repo whose .git is present but not a real repo → verify's rev-parse
// smoke-check must fail, leaving the source intact and no partition/.bak.
await writeLegacyConfig();
await fse.writeJson(path.join(legacyDir, 'state.json'), {});
const tr = path.join(legacyDir, 'team-repo');
await fse.ensureDir(path.join(tr, '.git')); // a .git dir that is NOT a valid repo
await fse.writeFile(path.join(tr, 'README'), 'x\n');
const plan = await planMigration(repoRoot);
await expect(runMigration(plan!)).rejects.toThrow(/not a usable git repository/);
// Source untouched; nothing half-migrated.
expect(await fse.pathExists(path.join(legacyDir, 'config.yaml'))).toBe(true);
expect(await fse.pathExists(`${legacyDir}.bak`)).toBe(false);
expect(await fse.pathExists(projectDataHome(repoRoot))).toBe(false);
expect(await fse.pathExists(`${projectDataHome(repoRoot)}.staging`)).toBe(false);
});
it('never overwrites an existing .teamai.bak — picks a fresh name instead (no data loss)', async () => {
await seedLegacyLayout();
// The user (or a prior migration) already has a .teamai.bak with data.
const existingBak = `${legacyDir}.bak`;
await fse.ensureDir(existingBak);
await fse.writeFile(path.join(existingBak, 'only-copy.txt'), 'irreplaceable');
await runMigration((await planMigration(repoRoot))!);
// The pre-existing backup is untouched...
expect(await fse.readFile(path.join(existingBak, 'only-copy.txt'), 'utf-8')).toBe('irreplaceable');
// ...and the migration's own backup went to a fresh name.
expect(await fse.pathExists(path.join(`${legacyDir}.bak.1`, 'config.yaml'))).toBe(true);
});
it('keeps the retired backup git-ignored so a `git add -A` cannot leak its secrets', async () => {
// Precondition that makes this dangerous: the legacy .teamai is protected
// ONLY by a repo-root `.gitignore` rule for `.teamai/`, which does not match
// `.teamai.bak/`. Without an in-dir .gitignore the rename would expose the
// plaintext env/token to the next commit.
await fse.writeFile(path.join(repoRoot, '.gitignore'), '.teamai/\n');
await seedLegacyLayout();
await fse.writeFile(path.join(legacyDir, 'token'), 'api-key-xyz\n');
// sanity: env IS ignored pre-migration
expect(gitOut(repoRoot, 'status', '--porcelain', '--ignored')).toContain('.teamai/');
await runMigration((await planMigration(repoRoot))!);
git(repoRoot, 'add', '-A');
const staged = gitOut(repoRoot, 'diff', '--cached', '--name-only');
expect(staged.split('\n').filter((l) => l.includes('.teamai.bak'))).toHaveLength(0);
// The secrets are unreadable via git but still on disk (rollback intact).
expect(() => git(repoRoot, 'show', ':.teamai.bak/env')).toThrow();
expect(() => git(repoRoot, 'show', ':.teamai.bak/token')).toThrow();
expect(await fse.pathExists(path.join(`${legacyDir}.bak`, 'env'))).toBe(true);
});
it('migrates an http-mode install (no team-repo clone)', async () => {
await writeLegacyConfig({
repo: { localPath: legacyDir, remote: 'https://team.example/api', kind: 'http', url: 'https://team.example/api' },
});
await fse.writeFile(path.join(legacyDir, 'token'), 'api-key-xyz\n');
await fse.writeJson(path.join(legacyDir, 'state.json'), {});
const plan = await planMigration(repoRoot);
expect(plan!.mode).toBe('full');
const result = await runMigration(plan!);
expect(result).toBe('migrated');
const partition = projectDataHome(repoRoot);
expect(await fse.pathExists(path.join(partition, 'config.yaml'))).toBe(true);
expect(await fse.pathExists(path.join(partition, 'token'))).toBe(true);
expect(await fse.pathExists(legacyDir)).toBe(false);
});
it('recovers from a leftover staging dir (interrupted prior run)', async () => {
await seedLegacyLayout();
const partition = projectDataHome(repoRoot);
const staging = `${partition}.staging`;
// Simulate a crash mid-copy: a partial staging dir is left behind.
await fse.ensureDir(staging);
await fse.writeFile(path.join(staging, 'garbage'), 'partial\n');
const result = await runMigration((await planMigration(repoRoot))!);
expect(result).toBe('migrated');
// Stale staging content was discarded, not merged.
expect(await fse.pathExists(path.join(partition, 'garbage'))).toBe(false);
expect(await fse.pathExists(path.join(partition, 'config.yaml'))).toBe(true);
});
it('dry-run writes nothing', async () => {
await seedLegacyLayout();
const plan = await planMigration(repoRoot);
const result = await runMigration(plan!, { dryRun: true });
expect(result).toBe('dry-run');
const partition = projectDataHome(repoRoot);
expect(await fse.pathExists(partition)).toBe(false);
// Source untouched.
expect(await fse.pathExists(legacyDir)).toBe(true);
expect(await fse.pathExists(`${legacyDir}.bak`)).toBe(false);
});
});
describe('maybeMigrate', () => {
it('is a no-op when there is nothing to migrate', async () => {
// No legacy layout; must not throw.
const spy = vi.spyOn(process, 'cwd').mockReturnValue(repoRoot);
try {
await expect(maybeMigrate()).resolves.toBeUndefined();
} finally {
spy.mockRestore();
}
});
});
+28 -1
View File
@@ -2,8 +2,14 @@ import { createRequire } from 'node:module';
import { Command, Option } from 'commander';
import { setVerbose, setSilent, log } from './utils/logger.js';
import type { GlobalOptions } from './types.js';
import { TEAMAI_HOOK_SUBCOMMANDS } from './hooks.js';
import { registerPackagesCommand } from './pkg/register-command.js';
// Commands that migrate a legacy `<repo>/.teamai/` into the partition on first
// run (issue #374 P1-3). Only write commands trigger it; read-only commands rely
// on the double-read fallback, and hook-dispatch is excluded outright (see below).
const MIGRATION_TRIGGER_COMMANDS = new Set(['init', 'pull', 'push']);
const require = createRequire(import.meta.url);
const { version } = require('../package.json');
@@ -15,9 +21,30 @@ program
.version(version)
.option('--dry-run', 'Preview mode, no changes made')
.option('-v, --verbose', 'Verbose output')
.hook('preAction', (thisCommand) => {
.hook('preAction', async (thisCommand, actionCommand) => {
const opts = thisCommand.opts();
if (opts.verbose) setVerbose(true);
// Auto-migrate a legacy `<repo>/.teamai/` into the partition before the
// command runs, so init/pull/push (and every path resolver they call) see
// the migrated layout. Narrowed twice: hook-dispatch is a high-frequency
// silent path that must never move 12MB, and only write commands trigger a
// move (read-only commands use the double-read fallback). Dry-run previews.
const name = actionCommand.name();
if (TEAMAI_HOOK_SUBCOMMANDS.includes(name as (typeof TEAMAI_HOOK_SUBCOMMANDS)[number])) return;
if (!MIGRATION_TRIGGER_COMMANDS.has(name)) return;
const { maybeMigrate } = await import('./migrate.js');
try {
await maybeMigrate({ dryRun: !!opts.dryRun });
} catch (e) {
// A failed migration must not proceed into the command on stale/partial
// state. Surface a clean message and exit — the copy→verify→rename design
// leaves the source intact, so a rerun retries safely. (Without this the
// async-hook rejection would surface as a raw unhandled-rejection stack.)
log.error(`Auto-migration failed: ${(e as Error).message}`);
log.error('Your original .teamai data is unchanged. Re-run the command to retry.');
process.exit(1);
}
});
program
+391
View File
@@ -0,0 +1,391 @@
import path from 'node:path';
import fse from 'fs-extra';
import YAML from 'yaml';
import { LocalConfigSchema, SYNC_LOCK_FILENAME } from './types.js';
import { resolveAnchors } from './utils/git.js';
import { projectDataHome } from './utils/partition.js';
import { realpath } from 'node:fs/promises';
import { expandHome, pathExists, readFileSafe, remove, writeFile } from './utils/fs.js';
import { acquireLock, releaseLock } from './update.js';
import { log } from './utils/logger.js';
/**
* P1-3 automatic migration (issue #374).
*
* Old installs kept teamai's project-scope machine data (config, state, the
* team-repo clone, search index, per-worktree managed-mcp + resource cache …)
* inside the business repo at `<workspaceRoot>/.teamai/`. P1-2 flipped NEW
* installs to the partition `~/.teamai/projects/<slug>/` and reads old installs
* via a legacy fallback. This module moves a real legacy `.teamai/` INTO the
* partition the first time a write command (`init`/`pull`/`push`) runs, so the
* business workspace ends up with zero teamai residue.
*
* Safety model (issue R2): copy → verify → atomic rename, so an interruption
* never leaves the data half-in-both-places. The source is only renamed to
* `.teamai.bak/` AFTER the partition is fully in place; we never delete it.
*
* Trigger is narrowed by the caller (the global preAction hook): hook-dispatch
* and read-only commands never reach here. self mode is a hard no-op (its
* `.teamai/` is team knowledge committed to main, not machine data).
*/
/** Directories/files under a legacy `.teamai/` that must NOT be copied. */
const SKIP_ENTRIES = new Set([
// Disposable git worktrees: their gitdir records an ABSOLUTE path, so moving
// them breaks the linkage. They are rebuilt on demand (git.ts calls them
// "disposable worktrees"). Self-mode only, but skip defensively either way.
'reports-wt',
'knowledge-wt',
// Lock files: transient, and a stale one copied into the partition would be
// mistaken for a live lock.
SYNC_LOCK_FILENAME,
'.update-lock',
]);
export interface MigrationPlan {
legacyDir: string;
partitionDir: string;
anchor: string;
/**
* 'full': copy legacy → partition, then retire the source.
* 'retire-only': the partition is already built (e.g. a prior run crashed
* between the partition rename and the source retire), so just clean up the
* leftover legacy dir. Without this, planMigration would return null on the
* "partition exists" check and the legacy dir — including its plaintext `env`
* — would linger in the workspace forever, breaking the zero-residue promise.
*/
mode: 'full' | 'retire-only';
}
/**
* Decide whether the current working directory is a legacy install that needs
* migration, WITHOUT going through detectProjectConfig (which short-circuits on
* an existing partition and runs the self-heal bootstrap as a side effect — both
* would mask the raw "legacy exists, partition doesn't" state we must observe).
*
* Returns the plan when migration should run, or null to skip. Skip when:
* - not a git repo (the partition only exists for git repos; a non-git
* `.teamai/` is already at its final location),
* - no legacy config.yaml (nothing to migrate),
* - the legacy config is user scope (user data never lives under `.teamai/`),
* - the legacy config is self mode (its `.teamai/` is committed team knowledge).
*
* When a partition config already exists AND a legacy dir still lingers, returns
* a 'retire-only' plan to finish an interrupted migration instead of skipping.
*/
export async function planMigration(cwd?: string): Promise<MigrationPlan | null> {
const anchors = await resolveAnchors(cwd ?? process.cwd());
if (!anchors) return null;
const legacyDir = path.join(anchors.workspaceRoot, '.teamai');
const legacyConfig = path.join(legacyDir, 'config.yaml');
if (!(await pathExists(legacyConfig))) return null;
// Read the legacy config directly to gate on scope/kind. A malformed config is
// treated as "nothing to migrate" rather than crashing a write command.
const content = await readFileSafe(legacyConfig);
if (!content) return null;
let scope: string | undefined;
let kind: string | undefined;
try {
const parsed = LocalConfigSchema.parse(YAML.parse(content));
scope = parsed.scope;
kind = parsed.repo.kind;
} catch {
return null;
}
if (scope !== 'project') return null;
if (kind === 'self') return null;
const partitionDir = projectDataHome(anchors.projectAnchor);
// If the partition is already built, the copy is done (or was done by a prior
// run that crashed before retiring the source). Don't re-copy onto the
// authoritative partition — just finish the job by retiring the leftover
// legacy dir, so the workspace really does end up residue-free.
const mode: MigrationPlan['mode'] =
(await pathExists(path.join(partitionDir, 'config.yaml'))) ? 'retire-only' : 'full';
return { legacyDir, partitionDir, anchor: anchors.projectAnchor, mode };
}
/**
* Run migration for a decided plan.
*
* Locking (the load-bearing part): a concurrent pull/push from any worktree of
* the same repo races the shared team-repo clone. Before migration those
* processes lock `<legacyDir>/.sync-lock` (their `getDataHome` still resolves to
* the legacy dir until the partition exists); after migration they lock
* `<partitionDir>/.sync-lock`. To be mutually exclusive with the PRE-migration
* side — the only side that can run concurrently, since planMigration stands
* down once the partition exists — migration takes `<legacyDir>/.sync-lock`, the
* exact path an un-migrated pull/push contends on. The lock lives INSIDE
* legacyDir, which is renamed to `.bak` at the very end; we release it BEFORE
* that rename so the lock path stays valid for release and no live lock is
* carried into the backup.
*
* dryRun previews without touching disk.
*/
export async function runMigration(
plan: MigrationPlan,
opts: { dryRun?: boolean } = {},
): Promise<'migrated' | 'skipped' | 'dry-run'> {
const { legacyDir, partitionDir, anchor, mode } = plan;
if (opts.dryRun) {
if (mode === 'retire-only') {
log.info(
`[dry-run] partition already built at ${partitionDir}; would retire the ` +
`leftover ${legacyDir} to ${legacyDir}.bak`,
);
} else {
const entries = await listMigratableEntries(legacyDir);
log.info(
`[dry-run] would migrate ${entries.length} item(s) from ${legacyDir} ` +
`to ${partitionDir}, then rename the old directory to ${legacyDir}.bak`,
);
}
return 'dry-run';
}
const lockPath = path.join(legacyDir, SYNC_LOCK_FILENAME);
if (!(await acquireLock(lockPath))) {
log.debug('migration skipped: a concurrent pull/push holds the sync lock');
return 'skipped';
}
const staging = `${partitionDir}.staging`;
let lockReleased = false;
try {
// 'retire-only': a prior run already built the partition but crashed before
// retiring the source. The partition is authoritative — do NOT re-copy onto
// it — just finish by retiring the leftover legacy dir.
if (mode === 'retire-only') {
await releaseLock(lockPath);
lockReleased = true;
const backup = await retireLegacy(legacyDir);
log.success(`Finished an interrupted migration: retired ${legacyDir} to ${backup}`);
return 'migrated';
}
// Re-check under the lock: a sibling worktree may have migrated while we
// waited (TOCTOU). If the partition config now exists, retire our leftover
// legacy dir rather than copying onto the authoritative partition.
if (await pathExists(path.join(partitionDir, 'config.yaml'))) {
await releaseLock(lockPath);
lockReleased = true;
const backup = await retireLegacy(legacyDir);
log.debug(`partition built by a concurrent process; retired ${legacyDir} to ${backup}`);
return 'migrated';
}
// 1. Copy into a sibling staging dir (NOT the partition itself) so an
// interrupted copy never looks like a built partition. Use raw fse.copy
// (NOT copyDir): copyDir filters out `.git`, which would corrupt the
// team-repo clone. Skip disposable worktrees and lock files.
await remove(staging);
await fse.ensureDir(path.dirname(partitionDir));
await fse.copy(legacyDir, staging, {
overwrite: true,
filter: (src) => {
const rel = path.relative(legacyDir, src);
if (!rel) return true; // the root itself
const top = rel.split(path.sep)[0];
return !SKIP_ENTRIES.has(top);
},
});
// 2. Verify the staged copy before making it authoritative.
await verifyStaging(legacyDir, staging);
// 2b. Rebase absolute paths persisted in config.yaml that pointed INTO the
// legacy dir (chiefly repo.localPath → <legacyDir>/team-repo) onto the
// partition. Without this, the migrated config would still name the old
// team-repo location, so the next pull would read/clone the wrong path.
// Done in staging (pre-rename) so it stays inside the atomic window.
await rebaseConfigPaths(path.join(staging, 'config.yaml'), legacyDir, partitionDir);
// 3. Atomic switch: same-filesystem rename of the staged dir onto the final
// partition path. partitionDir does not exist yet (planMigration + the
// under-lock re-check both gate on its config.yaml, and nothing else
// creates it), so the rename lands on a clean name.
await remove(partitionDir);
await fse.rename(staging, partitionDir);
// 4. Write the anchor reverse-lookup file. The slug is a one-way sha256, so
// the original projectAnchor is only recoverable from this file — which
// lives inside the partition, off the workspace, preserving zero-residue.
await writeFile(path.join(partitionDir, 'anchor'), `${anchor}\n`);
// 5. Release the lock BEFORE renaming legacyDir away, so releaseLock finds
// the lock at its original path and no live lock is buried in the backup.
await releaseLock(lockPath);
lockReleased = true;
const backup = await retireLegacy(legacyDir);
log.success(`Migrated teamai data to ${partitionDir}`);
log.info(
`Old data preserved at ${backup} — remove it once you've confirmed ` +
`everything works (downgrading to an older teamai is not supported).`,
);
return 'migrated';
} catch (e) {
// Any failure before the rename leaves the source untouched; discard the
// partial staging dir so a rerun starts clean.
await remove(staging).catch(() => {});
throw e;
} finally {
if (!lockReleased) await releaseLock(lockPath);
}
}
/**
* Convenience entry point for the preAction hook: plan + run, swallowing the
* "nothing to do" case. Migration failures are surfaced (a write command should
* not silently proceed on stale legacy data), but never crash a dry-run preview.
*/
export async function maybeMigrate(opts: { dryRun?: boolean } = {}): Promise<void> {
const plan = await planMigration();
if (!plan) return;
await runMigration(plan, opts);
}
/**
* Rewrite absolute paths in the staged config.yaml that pointed into the legacy
* dir so they name the partition instead. Only `repo.localPath` is persisted as
* an absolute path today (the team-repo clone at `<legacyDir>/team-repo`); a path
* NOT inside legacyDir (e.g. an http install whose localPath sits elsewhere) is
* left untouched. Preserves every other field verbatim via YAML round-trip.
*/
async function rebaseConfigPaths(
stagedConfig: string,
legacyDir: string,
partitionDir: string,
): Promise<void> {
const content = await readFileSafe(stagedConfig);
if (!content) return;
let doc: Record<string, unknown>;
try {
doc = YAML.parse(content);
} catch {
return; // verifyStaging already validated parseability; be defensive anyway
}
const repo = doc?.repo as { localPath?: string } | undefined;
const rebased = await rebasePath(repo?.localPath, legacyDir, partitionDir);
if (repo && rebased !== undefined && rebased !== repo.localPath) {
repo.localPath = rebased;
await writeFile(stagedConfig, YAML.stringify(doc));
}
}
/**
* If `p` is inside `fromDir`, return the equivalent path inside `toDir`;
* otherwise return `p` unchanged (undefined stays undefined).
*
* `fromDir` is realpath-normalized (it comes from resolveAnchors), but the
* persisted `p` may use a symlinked spelling (e.g. macOS `/tmp` → `/private/tmp`)
* or a `~` prefix, so a raw string compare would miss the match. We expand `~`
* and realpath `p` first — the old location still exists at this point in the
* migration (the source is renamed to `.bak` only afterwards) — so both sides are
* canonical before path.relative decides containment. A `..` result means `p`
* escapes fromDir and is left alone (e.g. an external clone).
*/
async function rebasePath(
p: string | undefined,
fromDir: string,
toDir: string,
): Promise<string | undefined> {
if (!p) return p;
const expanded = expandHome(p);
const canonical = await realpath(expanded).catch(() => expanded);
const rel = path.relative(fromDir, canonical);
if (rel === '') return toDir;
if (rel.startsWith('..') || path.isAbsolute(rel)) return p;
return path.join(toDir, rel);
}
/**
* Retire the source dir to a `.bak` sibling (same-fs → atomic rename). Never
* auto-deleted: it is the manual rollback path (downgrading to an older teamai
* is not supported — see release notes / design doc R6). Returns the backup path.
*
* Two safety measures beyond a plain rename:
* - **Never overwrite an existing backup.** A pre-existing `.teamai.bak/` (from a
* prior migration, or the user's own) may hold irreplaceable data, so we pick
* the first FREE name (`.teamai.bak`, `.teamai.bak.1`, …) instead of removing
* whatever is there.
* - **Keep the backup git-ignored.** An old install's `.teamai/` was often
* protected only by a repo-root `.gitignore` rule matching `.teamai/`, which
* does NOT match `.teamai.bak/` — so after the rename a `git add -A` would
* stage the plaintext `env`/`token` in the backup. We drop a self-contained
* `.gitignore` (`*`) INTO the dir BEFORE renaming, so the backup ignores its
* own contents regardless of its final name or the repo's ignore rules.
*/
async function retireLegacy(legacyDir: string): Promise<string> {
// Make the backup ignore everything it contains, independent of repo rules and
// the backup's eventual name. Written before the rename so there is never a
// window in which the credentials sit in a non-ignored directory.
await writeFile(path.join(legacyDir, '.gitignore'), '# teamai migration backup — ignore everything\n*\n');
let backup = `${legacyDir}.bak`;
for (let n = 1; await pathExists(backup); n++) {
backup = `${legacyDir}.bak.${n}`;
}
await fse.rename(legacyDir, backup);
return backup;
}
/** Top-level entries under a legacy `.teamai/` that migration will copy. */
async function listMigratableEntries(legacyDir: string): Promise<string[]> {
const names = await fse.readdir(legacyDir);
return names.filter((n) => !SKIP_ENTRIES.has(n));
}
/**
* Verify a staged copy is complete enough to become authoritative:
* - config.yaml parses as a LocalConfig,
* - if the source has a team-repo git clone, the staged copy has its `.git`
* AND `git rev-parse HEAD` works on it (proves the copy did NOT drop `.git`
* — the copyDir-vs-fse.copy trap — and the clone is actually usable, not just
* present-but-corrupt),
* - every migratable top-level entry made it across.
* Runs on the STAGING copy, before the atomic rename, so any shortfall aborts
* with the source untouched and the partial staging discarded.
*/
async function verifyStaging(legacyDir: string, staging: string): Promise<void> {
const stagedConfig = path.join(staging, 'config.yaml');
const content = await readFileSafe(stagedConfig);
if (!content) throw new Error(`migration verify: ${stagedConfig} missing after copy`);
try {
LocalConfigSchema.parse(YAML.parse(content));
} catch (e) {
throw new Error(`migration verify: staged config.yaml is invalid (${(e as Error).message})`);
}
const legacyGit = path.join(legacyDir, 'team-repo', '.git');
if (await pathExists(legacyGit)) {
const stagedRepo = path.join(staging, 'team-repo');
if (!(await pathExists(path.join(stagedRepo, '.git')))) {
throw new Error('migration verify: team-repo/.git missing after copy (clone would be broken)');
}
// Smoke-check the clone: a working rev-parse proves the .git is intact, not
// just present. Catches a partial/corrupt copy that a mere existence check
// would wave through.
try {
const { execFile } = await import('node:child_process');
const { promisify } = await import('node:util');
await promisify(execFile)('git', ['rev-parse', 'HEAD'], { cwd: stagedRepo });
} catch (e) {
throw new Error(
`migration verify: the staged team-repo clone is not a usable git ` +
`repository (${(e as Error).message})`,
);
}
}
const expected = await listMigratableEntries(legacyDir);
for (const name of expected) {
if (!(await pathExists(path.join(staging, name)))) {
throw new Error(`migration verify: ${name} missing after copy`);
}
}
}