mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-11-12 10:35:06 +08:00
igooch
596cfb8ab0
Phase 2 of #463 (image-cache GC + observability): this is the eviction engine itself — the root set, the eviction pass, and startup orphan recovery — assembled from the foundations merged in #650 (sizes/last-use), #656 (spec digest), #658 (retirement primitives), and #717 (record-first pull). What remains after this is the watermark loop + flags that turn it on, then metrics and e2e. **Inert in production**: nothing calls `EvictUnused` yet — the watermark GC loop and its flags come in the next PR. The only behavior `New` gains is the startup orphan scan. ## What - **`Store.InUse()`** — the root set: scans bundle overlay specs under the actors dir (opt-in via `WithActorsDir`) for image digests, layer dirs, and exact layer-set signatures. The exact-set rule roots the multi-arch twin record and digestless (pre-`imageDigest`) specs' records without making every subset image unevictable. Returns `(RootSet, error)`: a non-nil error means the root set may be incomplete, and deleting callers must do nothing. - **`Store.EvictUnused(ctx, targetBytes, dryRun)`** — LRU eviction of unprotected images and the layers their removal leaves unreferenced. Min-age vetoes; per-victim re-checks under `hitMu` held exclusive against the cache-hit touch; two-phase deletion via `retireLayer`, with the slow `RemoveAll` outside all locks. If any layer of a deleted record must be kept (still referenced, rooted, fresh, or failed to retire), the record is atomically restored from bytes captured at listing time — a kept layer is never left unreachable. - **`Store.RecoverOrphans`**, called once from `New` — reclaims layers no record references. Startup is the one moment this scan is race-free (no pull can be in flight); a failure never fails `New`. - **Everything fails toward retention.** Both passes skip entirely, at ERROR naming the culprit, when their inputs can't be fully enumerated: an unreadable/undecodable image record, a garbled diffID inside a valid record, or an unenumerable root set (actors dir, bundles dir, or bundle spec). Refcounts and roots from partial data would retire layers that unread records still reference or running actors still mount — and for a long-running actor, the bundle spec is the only protection (its record mtime can be arbitrarily old, so min-age is no backstop). Missing dirs/specs remain legitimate emptiness, not errors. - **Eviction never writes into the live pool.** Retired layers are credited from the size file that rides along with the rename-aside (O(1), read-only, consistent with `CacheSize`); the size-file backfill exists only on the `CacheSize` path, where it bumps the dir mtime rather than restoring it (restoring could rewind a concurrent reuse-touch). - Per-item log lines are Debug behind an `Enabled` gate (level control via #677); pass summaries, evictions, and restores are Info. - `internal/imagecache/README.md` now documents the GC: pass structure, protection rules, failure direction, startup recovery, and two-phase deletion, plus the on-disk layout entries (`size`, `.tmp-*`, `.rm-*`) that shipped with #650/#658. ## Notes for reviewers - The gc tests reuse the helpers merged with #650 (`backdate`, `mustEnsure`) and #717's gated-registry harness — no duplicated symbols. The slow-pull gated test gains the eviction-veto assertion deferred from #717. - Dry-run mutates nothing, including size-file backfills (`layerSizeReadOnly`). - A `Store` built without `WithActorsDir` has an empty root set (deliberate for tools/tests). - Carried to the wiring PR: pass `WithActorsDir` in atelet; handle `EvictUnused`'s error before any shortfall accounting (a gated pass returns zero stats plus an error and must not be logged as a shortfall); and surface skipped passes as a counter in the observability PR so a wedged node alerts rather than scrolling ERROR lines. ## Testing - `go test -race ./internal/imagecache/`: every veto, shared-layer survival, both restore-on-keep shapes, orphan reclaimed at startup and ignored by the periodic pass, all enumeration gates — unreadable record (shared-layer scenario), undecodable record, garbled diffID, and the three unreadable root-set shapes (actors dir / bundles dir / spec, each against a rooted image with an arbitrarily old record) — dry-run purity, 25-iteration ensure-vs-evict race loop. - Kind, via the integration branch that stacks the watermark loop on this PR (the loop and flags are **not** part of this PR): full gVisor e2e suite green; a planted backdated orphan layer (no size file) was reclaimed at atelet startup with the expected log lines, its size credited read-only from the retired dir. - GKE (6 nodes), same integration branch: demo lifecycle + snapshot-continuity suites green; under aggressive watermarks the idle pool was evicted and subsequent actor resumes cold-pulled cleanly. No enumeration-gate false positives on either cluster.