Commit Graph
4 Commits
Author SHA1 Message Date
dberkov 610e3916ad Add node-local OCI image layer cache (imagecache), replacing memorypullcache
Content-addressed pool of unpacked image layers, shared by every actor on
the node; actor rootfs becomes an overlayfs mount (cached layers as read-
only lowers, bundle-local upper) instead of a full re-untar per run.
atelet (no capabilities) pulls and unpacks; the privileged ateoms finalize
whiteouts and mount. Tag refs resolve via one HEAD and become cacheable;
pull memory is O(stream buffers); the cache survives restarts.

Phase 1 of #463. Fixes #437, #166, #228.

Validated: kind + GKE counter demos (gvisor and microvm), suspend/resume
(oci_unpack ~3ms vs ~15-20s), 411 SWE-bench-scale images pulled and
unpacked with 0 failures, root-gated unit tests for the privileged paths.
Known gaps: no GC yet (Phase 2, see internal/imagecache/README.md);
upgrade ordering — deploy new ateoms before/with the new atelet.
2026-07-22 21:15:39 -07:00
Dmitry Berkovich 228fde4949 Bump go-containerregistry to v0.21.7 to fix OOM during image extract (#464)
## Summary

Bumps `github.com/google/go-containerregistry` v0.21.5 → v0.21.7. Phase
0 of #463.

v0.21.6 fixed the memory spike in `mutate.Extract` described in #120
(upstream fix: google/go-containerregistry#2190, merged as
google/go-containerregistry@38d6e4087c): extraction is now refactored
into a per-layer `extractLayer` function, so each layer's decompressor
and HTTP response body are closed as soon as that layer is processed,
instead of all being held open via deferred `Close` calls until the
entire (multi-GB) extraction completes. atelet's pull path goes through
`memorypullcache.Fetch` → `mutate.Extract`, so this directly caps its
extract-time memory.

Note this is only interim relief: #437's unbounded cache retention is
untouched and is addressed by the redesign in #463.

## Changes

- `go-containerregistry` v0.21.5 → v0.21.7, plus required transitive
bumps: `docker/cli`, `klauspost/compress`, `golang.org/x/sync`,
`golang.org/x/sys`
- `go mod tidy` + `go mod vendor`; the
`containerd/stargz-snapshotter/estargz` vendor tree drops out (upstream
removed the estargz integration; nothing in substrate used it), along
with `vbatts/tar-split` and `mitchellh/go-homedir`

## Testing

- `go build ./...` passes
- Full `go test ./...` passes
- Verified the layer-close fix is present in the vendored
`pkg/v1/mutate/mutate.go`

Fixes #120

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-07-20 10:25:47 -04:00
Eitan Yarmush b923428f46 Fix ateom cleanup failure handling 2026-06-10 22:37:09 -07:00
Tim Hockin e10ae9cafb Update/verify dep vendoring (#132)
This adds an update/verify script pair to check `go mod vendor` in the
main module. Should we do this in sub-modules too?

Fixes #19
2026-06-02 20:02:51 -07:00