ate-setup: add flags and functionality to use pre-built images (#1388)

Fixes #1387

Also tested manually on GKE cluster and kind cluster both existing build
on demand pathways and relying on prebuilt images from my own registry.

- [X] Tests pass
- [X] Appropriate changes to documentation are included in the PR

---------

Co-authored-by: Anna Pendleton <13793293+annapendleton@users.noreply.github.com>

cmd/ate-setup/README.md

@@ -10,6 +10,22 @@ make build-ate-setup    # builds bin/ate-setup
 `ate-setup` is a Go port of `hack/install-ate.sh` and the scripts it sources.
 Both work today and can be used against the same cluster.
 
+## Installing a release
+
+By default every image is built from the checkout with `ko`, which is what a
+developer wants and what the shell installer always did. To install published
+images instead, name the registry they were pushed to:
+
+```
+ate-setup deploy ate-system \
+  --image-repo registry.example.com/substrate \
+  --image-tag v0.0.0
+```
+
+Nothing is built and no registry is pushed to; the manifests still come from the
+checkout. Each reference is pinned to the digest its tag names, so the registry
+has to be readable from here as well as from the cluster.
+
 - [`commands.md`](commands.md) — every command with its `hack/install-ate.sh`
   equivalent.
 - [`differences.md`](differences.md) — where the port deliberately behaves

cmd/ate-setup/commands.md

@@ -31,6 +31,34 @@ a pre-scan pass, so they may appear anywhere on its command line.
 | `--kubeconfig PATH` | `KUBECONFIG=PATH` | Explicit kubeconfig path |
 | `--no-dev-env` | `NO_DEV_ENV=1` | Skip `.ate-dev-env.sh` at the repository root |
 | `--version` / `-v` | — | New; the shell installer had no version |
+| `--image-repo REPO` | — | New. Install pre-built images from `REPO` instead of building them with `ko` |
+| `--image-tag TAG` | — | New. The tag those images carry. Each of the two requires the other |
+
+Both have an environment equivalent, read when the flag is absent:
+`ATE_IMAGE_REPO` and `ATE_IMAGE_TAG`.
+
+## Installing a release
+
+Without `--image-repo`, `ate-setup` builds every image from the checkout with
+`ko` and pushes it to `KO_DOCKER_REPO`. That is the developer install and is
+unchanged.
+
+```
+ate-setup deploy ate-system \
+  --image-repo registry.example.com/substrate \
+  --image-tag v0.0.0
+```
+
+installs published images instead, and never invokes `ko`. The manifests still
+come from the checkout, so this needs one; what it removes is the build, the Go
+toolchain, and write access to a registry.
+
+`REPO` has to hold every component image the manifests reference, all under the
+same tag, which is how a release publishes them. A release that adds a component
+has to publish it alongside the others before a pre-built install can use it.
+Each reference is then pinned to the digest its tag names, which takes one HEAD
+request per image, so the installer needs read access to `REPO` and not only the
+cluster does.
 
 ## Deploy
 

cmd/ate-setup/differences.md

@@ -73,9 +73,9 @@ Still required, and why:
 
 | Binary | Used for |
 |---|---|
-| `ko` | building and publishing images (`ko resolve` only) |
-| `go` | locating the pinned `ko` tool, exactly as `hack/run-tool.sh` does |
-| `git` | the `git describe` version stamp passed to ko |
+| `ko` | building and publishing images (`ko resolve` only); not used with `--image-repo` |
+| `go` | locating the pinned `ko` tool, exactly as `hack/run-tool.sh` does; not used with `--image-repo` |
+| `git` | the `git describe` version stamp passed to ko; not used with `--image-repo` |
 | `docker` | the kind CSI setup (`docker exec` into the node) and the claude-code-multiplex workload build |
 | `gcloud` | GKE `get-credentials`, only when `PROJECT_ID` is set and no context was given |
 | `bash` | sourcing `.ate-dev-env.sh` |
@@ -92,6 +92,56 @@ made ko shell out to kubectl and forced the awkward `-- --context=` special case
 them). Now ko only resolves, and the resulting manifest is applied through
 client-go.
 
+## Image sources
+
+The shell installer had exactly one: build every image from the checkout with
+`ko` and push it to `KO_DOCKER_REPO`. That is still the default and is
+unchanged. `--image-repo` adds a second, in which nothing is built.
+
+Manifests carry `ko://<import path>` references either way. With `--image-repo`
+each is looked up in `images.Components` and rewritten to
+`<repo>/<base of the import path>:<tag>`, which is ko's own
+`--base-import-paths` naming and so is how the release images are already
+published — `ateapi`, `atelet`, `atenet`, `ateom-gvisor`, and the rest.
+`--image-tag` supplies the tag. Each of the two requires the other: a tag with
+no repository to pull from would be dropped, leaving a build from source that
+looks like the release the tag names.
+
+The rewritten reference then keeps that tag and adds the digest it names, as
+`<repo>/<image>:<tag>@sha256:...`, which is the shape ko itself produces for a
+tagged release. Resolving the digest is one HEAD request per image, cached for
+the install, so `--image-repo` needs read access to the registry from wherever
+`ate-setup` runs rather than only from the cluster. It is not optional: an
+ActorTemplate's container image, an image volume's reference, and a
+SandboxConfig's `pauseImage` each carry the CEL rule `self.contains('@')`, so an
+unpinned reference is rejected by admission and every demo fails to deploy.
+Pinning also gives a pre-built install the property a `ko` install had for free,
+that a tag moving afterwards cannot change what is running.
+
+That tag also becomes the substrate version, which names the atelet DaemonSet
+and sets the node label partitioning nodes across coexisting versions. It has
+to: that label must describe the atelet actually running, which came from the
+image, not the checkout `git describe` happens to be sitting on. `VERSION`
+overrides it, as it always has.
+
+The rewrite is textual, one token at a time, matching `SubstituteVersion` above.
+A reference is replaced wherever it appears, so the code needs no list of which
+fields hold images — `workerImage:` in a CRD spec is rewritten like any other —
+and the applied manifest differs from the checked-in YAML only in the image
+references.
+
+Every reference must be in the list. A reference outside the module path, one
+carrying an unexpanded `${PLACEHOLDER}`, and one naming a package that is not
+installable all fail the same way — an error naming every unmapped reference at
+once, rather than an unpullable image reaching the cluster and failing as
+`ImagePullBackOff` twenty minutes later. Matching whole references rather than
+replacing the listed ones is what makes that hold: a reference extending a
+listed package would otherwise inherit its image. `images.Components` is checked
+against the real manifests by a test, so adding a component fails a test rather
+than an install — though only the list is checked that way. The release also has
+to publish the new image under the same repository and tag as the rest, and
+until it does, the component can only be installed by building it.
+
 ## Cluster access
 
 **Server-side apply.** `kubectl apply -f -` is a client-side apply by default.
@@ -219,4 +269,4 @@ outcome.
 The shell installer had no tests. `cmd/ate-setup` has unit tests for template
 rendering, overlay selection, config resolution, the authentication config, the
 apiserver environment ConfigMap, delegated script arguments, manifest deletion,
-and per-demo rendering.
+per-demo rendering, and image reference rewriting.

cmd/ate-setup/internal/cmd/root.go

@@ -95,6 +95,11 @@ func init() {
 	f.StringVar(&opts.AdditionalEgressExtprocService, "experimental-additional-egress-extproc-service", "", "Run an additional ext_proc authorization filter served by NS/SVC:PORT (requires --experimental-use-sdsmint)")
 	f.BoolVar(&opts.NoDevEnv, "no-dev-env", false, "Do not source .ate-dev-env.sh")
 
+	f.StringVar(&opts.ImageRepo, "image-repo", "",
+		"Install pre-built images from this registry path instead of building them from source (e.g. registry.example.com/substrate)")
+	f.StringVar(&opts.ImageTag, "image-tag", "",
+		"Tag the pre-built images carry (required with --image-repo)")
+
 	// Cobra's default completion command would run PersistentPreRunE and
 	// require a cluster; the tree is not deep enough to justify that.
 	rootCmd.CompletionOptions.DisableDefaultCmd = true

cmd/ate-setup/internal/config/config.go

@@ -27,6 +27,8 @@ import (
 	"strconv"
 	"strings"
 	"time"
+
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/images"
 )
 
 // Enumerated values for the install-shaping flags.
@@ -85,6 +87,10 @@ type Config struct {
 	// KODefaultPlatforms constrains ko's build platforms.
 	KODefaultPlatforms string
 
+	// Images selects where container images come from. Its zero value builds
+	// them from source with ko, which is what a developer install does.
+	Images images.Source
+
 	// Router selects the atenet router dataplane.
 	Router string
 	// PostgresConnectionString is the apiserver's store connection string.
@@ -134,6 +140,10 @@ type Options struct {
 	ExperimentalUseSDSMint         bool
 	AdditionalEgressExtprocService string
 
+	// Image source selection.
+	ImageRepo string
+	ImageTag  string
+
 	// NoDevEnv skips sourcing .ate-dev-env.sh even when it exists.
 	NoDevEnv bool
 }
@@ -202,6 +212,7 @@ func Load(opts Options) (*Config, error) {
 		BucketName:                     env["BUCKET_NAME"],
 		KODockerRepo:                   env["KO_DOCKER_REPO"],
 		KODefaultPlatforms:             env["KO_DEFAULTPLATFORMS"],
+		Images:                         loadImageSource(opts, env),
 		PostgresConnectionString:       env["ATE_API_POSTGRES_CONNECTION_STRING"],
 		RolloutTimeout:                 rolloutTimeout,
 		rolloutTimeoutSet:              timeoutStr != "",
@@ -226,6 +237,14 @@ func Load(opts Options) (*Config, error) {
 	return cfg, nil
 }
 
+// loadImageSource resolves where images come from.
+func loadImageSource(opts Options, env map[string]string) images.Source {
+	return images.Source{
+		Repo: strings.TrimSuffix(firstNonEmpty(opts.ImageRepo, env["ATE_IMAGE_REPO"]), "/"),
+		Tag:  firstNonEmpty(opts.ImageTag, env["ATE_IMAGE_TAG"]),
+	}
+}
+
 func applyKindDefaults(cfg *Config) {
 	cfg.ProjectID = ""
 	cfg.ClusterLocation = ""
@@ -237,6 +256,9 @@ func applyKindDefaults(cfg *Config) {
 }
 
 func validate(cfg *Config) error {
+	if err := cfg.Images.Validate(); err != nil {
+		return err
+	}
 	switch cfg.Router {
 	case RouterEnvoy, RouterAgentgateway:
 	default:

cmd/ate-setup/internal/config/config_test.go

@@ -22,6 +22,8 @@ import (
 	"strings"
 	"testing"
 	"time"
+
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/images"
 )
 
 // loadEnv isolates Load from the ambient environment.
@@ -41,6 +43,8 @@ func loadEnv(t *testing.T) {
 		"ATE_API_POSTGRES_CONNECTION_STRING",
 		"ATE_ATENET_ROUTER",
 		"ATE_EXPERIMENTAL_USE_SDSMINT",
+		"ATE_IMAGE_REPO",
+		"ATE_IMAGE_TAG",
 		"ATE_INSTALL_PODCERT_WORKERS_PER_SIGNER",
 		"ATE_INSTALL_ROLLOUT_TIMEOUT",
 		"ATE_OTLP_ENDPOINT",
@@ -341,3 +345,89 @@ func TestSourceShellEnvReportsFailure(t *testing.T) {
 		t.Fatal("sourceShellEnv() succeeded, want an error")
 	}
 }
+
+func TestLoadImageSource(t *testing.T) {
+	tests := []struct {
+		name      string
+		env       map[string]string
+		opts      Options
+		want      images.Source
+		wantError string
+	}{
+		{
+			name: "unset builds from source",
+			want: images.Source{},
+		},
+		{
+			name: "flags",
+			opts: Options{ImageRepo: "example.com/substrate", ImageTag: "v1.2.3"},
+			want: images.Source{Repo: "example.com/substrate", Tag: "v1.2.3"},
+		},
+		{
+			name: "environment",
+			env:  map[string]string{"ATE_IMAGE_REPO": "example.com/env", "ATE_IMAGE_TAG": "v9"},
+			want: images.Source{Repo: "example.com/env", Tag: "v9"},
+		},
+		{
+			name: "flags beat the environment",
+			env:  map[string]string{"ATE_IMAGE_REPO": "example.com/env", "ATE_IMAGE_TAG": "v9"},
+			opts: Options{ImageRepo: "example.com/flag", ImageTag: "v1"},
+			want: images.Source{Repo: "example.com/flag", Tag: "v1"},
+		},
+		{
+			// A repo pasted from a registry UI often carries one, and it would
+			// otherwise produce a double slash in every reference.
+			name: "trailing slash is trimmed",
+			opts: Options{ImageRepo: "example.com/substrate/", ImageTag: "v1"},
+			want: images.Source{Repo: "example.com/substrate", Tag: "v1"},
+		},
+		{
+			name:      "a repo needs a tag",
+			opts:      Options{ImageRepo: "example.com/substrate"},
+			wantError: "--image-repo (or ATE_IMAGE_REPO) requires --image-tag",
+		},
+		{
+			name:      "a tag needs a repo",
+			opts:      Options{ImageTag: "v1"},
+			wantError: "--image-tag (or ATE_IMAGE_TAG) requires --image-repo",
+		},
+		{
+			// The environment reaches validation the same way the flags do.
+			name:      "a tag from the environment needs a repo",
+			env:       map[string]string{"ATE_IMAGE_TAG": "v1"},
+			wantError: "--image-tag (or ATE_IMAGE_TAG) requires --image-repo",
+		},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			loadEnv(t)
+			for name, value := range tc.env {
+				t.Setenv(name, value)
+			}
+
+			cfg, err := Load(tc.opts)
+			if tc.wantError != "" {
+				if err == nil {
+					t.Fatalf("Load() = nil error, want one containing %q", tc.wantError)
+				}
+				if !strings.Contains(err.Error(), tc.wantError) {
+					t.Errorf("Load() error = %v, want it to contain %q", err, tc.wantError)
+				}
+				return
+			}
+			if err != nil {
+				t.Fatalf("Load() error = %v", err)
+			}
+			if cfg.Images.Repo != tc.want.Repo {
+				t.Errorf("Images.Repo = %q, want %q", cfg.Images.Repo, tc.want.Repo)
+			}
+			if cfg.Images.Tag != tc.want.Tag {
+				t.Errorf("Images.Tag = %q, want %q", cfg.Images.Tag, tc.want.Tag)
+			}
+			if cfg.Images.IsPrebuilt() != (tc.want.Repo != "") {
+				t.Errorf("Images.IsPrebuilt() = %v, want %v", cfg.Images.IsPrebuilt(), tc.want.Repo != "")
+			}
+		})
+	}
+}

cmd/ate-setup/internal/demos/simple.go

@@ -110,7 +110,7 @@ func (d *Simple) DeployWorkload(ctx context.Context, e *steps.Env) error {
 	if err != nil {
 		return err
 	}
-	return e.KoApplyBytes(ctx, manifest)
+	return e.ResolveAndApplyBytes(ctx, manifest)
 }
 
 // WaitReady blocks until the demo's Deployments are rolled out.

cmd/ate-setup/internal/demos/substrate.go

@@ -116,7 +116,7 @@ func (d *Substrate) Deploy(ctx context.Context, e *steps.Env) error {
 	if err != nil {
 		return err
 	}
-	if err := e.KoApplyBytes(ctx, manifest); err != nil {
+	if err := e.ResolveAndApplyBytes(ctx, manifest); err != nil {
 		return err
 	}
 	for _, ref := range d.Deployments {
@@ -142,10 +142,10 @@ func (d *Substrate) Deploy(ctx context.Context, e *steps.Env) error {
 		if err != nil {
 			return err
 		}
-		// ko resolve replaces the ko:// image references with pushed digests
-		// before the manifest is parsed; manifests without ko:// references
-		// pass through unchanged.
-		resolved, err := e.KoResolveBytes(ctx, manifest)
+		// The ko:// image references become digest-pinned ones before the
+		// manifest is parsed, which is what the ActorTemplate schema requires;
+		// manifests without ko:// references pass through unchanged.
+		resolved, err := e.ResolveManifestBytes(ctx, manifest)
 		if err != nil {
 			return err
 		}

cmd/ate-setup/internal/images/digest.go

@@ -0,0 +1,50 @@
+// Copyright 2026 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package images
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/google/go-containerregistry/pkg/authn"
+	"github.com/google/go-containerregistry/pkg/name"
+	"github.com/google/go-containerregistry/pkg/v1/remote"
+)
+
+// Digester resolves a tagged image reference to the digest its tag currently
+// names, in "sha256:..." form. Prebuilt takes one so tests can rewrite
+// manifests without a registry.
+type Digester func(ctx context.Context, ref string) (string, error)
+
+// RemoteDigest asks ref's registry which manifest the tag names.
+//
+// One HEAD request per image, needing only read access, and authenticated the
+// way docker and ko are: from the docker config file and the credential helpers
+// it names. A localhost registry is contacted over plain HTTP, which is what
+// the Kind local registry serves.
+func RemoteDigest(ctx context.Context, ref string) (string, error) {
+	parsed, err := name.ParseReference(ref)
+	if err != nil {
+		return "", fmt.Errorf("%s is not a valid image reference: %w", ref, err)
+	}
+	desc, err := remote.Head(parsed,
+		remote.WithContext(ctx),
+		remote.WithAuthFromKeychain(authn.DefaultKeychain),
+	)
+	if err != nil {
+		return "", fmt.Errorf("resolving %s to a digest: %w", ref, err)
+	}
+	return desc.Digest.String(), nil
+}

cmd/ate-setup/internal/images/images.go

@@ -0,0 +1,107 @@
+// Copyright 2026 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package images decides where ate-setup gets container images from.
+//
+// The manifests name images as ko:// import paths, which ko builds and
+// publishes on every install. That needs a checkout, a Go toolchain, and a
+// registry to push to. Installing an already-published release instead means
+// mapping those same references onto images that exist, which is what Source
+// describes and Prebuilt performs.
+package images
+
+import (
+	"fmt"
+	"path"
+)
+
+// ModulePath is this repository's module path. A ko:// reference is it joined
+// with a component's package.
+const ModulePath = "github.com/agent-substrate/substrate"
+
+// Components is every package the installer can deploy a published image for,
+// as an import path within the module.
+//
+// The list is closed on purpose. A reference that is not in it names something
+// this build has no published image for, and failing beats guessing at a name
+// that was never pushed. TestComponentsCoverTheManifests keeps it in step with
+// the manifests, which is half of it: a new component also has to be published
+// under the same repository and tag as the rest before a pre-built install can
+// use it.
+var Components = []string{
+	"cmd/ateapi",
+	"cmd/atecontroller",
+	"cmd/atelet",
+	"cmd/atenet",
+	"cmd/ateom-gvisor",
+	"cmd/ateom-microvm",
+	"cmd/podcertcontroller",
+	"demos/counter",
+	"demos/egress",
+	"demos/multi-template/fspersist",
+	"demos/sandbox",
+}
+
+// KoReference is the ko:// reference the manifests name pkg by.
+//
+// ko resolves a manifest reference as written, so a "./"-relative import path
+// works there too. Naming packages in full is this repository's convention
+// rather than ko's rule, and only the full form maps to an image here:
+// TestComponentsCoverTheManifests holds the convention, and a reference spelled
+// any other way fails the install rather than resolving.
+func KoReference(pkg string) string { return "ko://" + ModulePath + "/" + pkg }
+
+// ImageName is the image name pkg is published under: the last element of the
+// import path, which is ko's --base-import-paths naming. ko lowercases it as
+// well, which changes nothing while every package here already is.
+func ImageName(pkg string) string { return path.Base(pkg) }
+
+// Source describes where images come from.
+//
+// The zero value builds and publishes them from source with ko. Setting Repo
+// switches to images already published in a registry, and ko is never invoked.
+type Source struct {
+	// Repo is the registry path holding the component images, without a
+	// trailing slash, e.g. "registry.example.com/substrate".
+	Repo string
+	// Tag is the tag every component image carries. Releases tag all
+	// components together, so one tag covers the set.
+	Tag string
+}
+
+// IsPrebuilt reports whether images should be pulled rather than built.
+func (s Source) IsPrebuilt() bool { return s.Repo != "" }
+
+// Validate checks a Source is usable before anything reaches the cluster.
+//
+// Both halves are checked. A tag on its own would otherwise be discarded in
+// silence, leaving a build from source that looks like the release the tag
+// names.
+func (s Source) Validate() error {
+	switch {
+	case s.IsPrebuilt() && s.Tag == "":
+		return fmt.Errorf("--image-repo (or ATE_IMAGE_REPO) requires --image-tag (or ATE_IMAGE_TAG)")
+	case !s.IsPrebuilt() && s.Tag != "":
+		return fmt.Errorf("--image-tag (or ATE_IMAGE_TAG) requires --image-repo (or ATE_IMAGE_REPO); without one, images are built from source and the tag names nothing")
+	}
+	return nil
+}
+
+// Describe renders the source for the install log.
+func (s Source) Describe() string {
+	if !s.IsPrebuilt() {
+		return "built from source with ko"
+	}
+	return fmt.Sprintf("pre-built from %s, tag %s", s.Repo, s.Tag)
+}

cmd/ate-setup/internal/images/images_test.go

@@ -0,0 +1,165 @@
+// Copyright 2026 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// The component list is checked against the real manifests, so this is an
+// external test package: config imports images, and only the test needs config.
+package images_test
+
+import (
+	"io/fs"
+	"os"
+	"path/filepath"
+	"regexp"
+	"slices"
+	"strings"
+	"testing"
+
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/config"
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/images"
+)
+
+// installTrees are the directories the installer reads manifests from.
+//
+// benchmarking/ and internal/e2e/ are left out deliberately: neither is
+// installed through the image resolvers -- the benchmark workloads go through
+// a shell script that runs ko itself, and the e2e fixtures are built by the
+// test suite.
+var installTrees = []string{"manifests", "demos"}
+
+var koRefPattern = regexp.MustCompile(`ko://[^\s"',\]}]+`)
+
+// componentsInTree collects the package of every ko:// reference under dir.
+//
+// A reference that does not carry the module path is recorded whole, so it
+// shows up in the failure rather than being silently dropped.
+func componentsInTree(t *testing.T, dir string) []string {
+	t.Helper()
+	var found []string
+	err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if d.IsDir() {
+			return nil
+		}
+		switch filepath.Ext(strings.TrimSuffix(p, ".tmpl")) {
+		case ".yaml", ".yml", ".json":
+		default:
+			return nil
+		}
+		body, err := os.ReadFile(p)
+		if err != nil {
+			return err
+		}
+		for _, match := range koRefPattern.FindAllString(string(body), -1) {
+			pkg, ok := strings.CutPrefix(match, "ko://"+images.ModulePath+"/")
+			if !ok {
+				pkg = match
+			}
+			if !slices.Contains(found, pkg) {
+				found = append(found, pkg)
+			}
+		}
+		return nil
+	})
+	if err != nil {
+		t.Fatalf("walking %s: %v", dir, err)
+	}
+	return found
+}
+
+// images.Components is a closed list, so a component added to a manifest but
+// not to the list would install fine from source and fail only when someone
+// tried to install a release. Catch it here instead.
+//
+// The same comparison holds the spelling of the references. ko accepts a
+// "./"-relative import path as well as a full one, and images.KoReference only
+// builds the full form, so a relative reference would resolve to no image. It
+// lands here as an entry the list cannot contain.
+func TestComponentsCoverTheManifests(t *testing.T) {
+	root, err := config.RepoRoot()
+	if err != nil {
+		t.Fatalf("resolving repo root: %v", err)
+	}
+
+	var found []string
+	for _, tree := range installTrees {
+		for _, pkg := range componentsInTree(t, filepath.Join(root, tree)) {
+			if !slices.Contains(found, pkg) {
+				found = append(found, pkg)
+			}
+		}
+	}
+	slices.Sort(found)
+
+	want := slices.Clone(images.Components)
+	slices.Sort(want)
+
+	if !slices.Equal(found, want) {
+		t.Errorf("the components referenced by %s and images.Components disagree:\n"+
+			" in the manifests: %v\n"+
+			" in images.Components: %v\n"+
+			"add the missing entries to images.Components, or remove the stale ones; "+
+			"an entry still carrying a ko:// prefix is a reference that does not name "+
+			"its package by the full import path %s/...",
+			strings.Join(installTrees, "/ and "), found, want, images.ModulePath)
+	}
+}
+
+func TestSourceValidate(t *testing.T) {
+	tests := []struct {
+		name  string
+		src   images.Source
+		error string
+	}{
+		{
+			name: "build from source",
+			src:  images.Source{},
+		},
+		{
+			name: "prebuilt",
+			src:  images.Source{Repo: "example.com/substrate", Tag: "v1"},
+		},
+		{
+			name:  "no tag",
+			src:   images.Source{Repo: "example.com/substrate"},
+			error: "--image-repo (or ATE_IMAGE_REPO) requires --image-tag",
+		},
+		{
+			// A tag with nowhere to pull from would otherwise be dropped, and
+			// the source build that followed would look like the release.
+			name:  "no repo",
+			src:   images.Source{Tag: "v1"},
+			error: "--image-tag (or ATE_IMAGE_TAG) requires --image-repo",
+		},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			err := tc.src.Validate()
+			if tc.error == "" {
+				if err != nil {
+					t.Fatalf("Validate() = %v, want nil", err)
+				}
+				return
+			}
+			if err == nil {
+				t.Fatalf("Validate() = nil, want an error containing %q", tc.error)
+			}
+			if !strings.Contains(err.Error(), tc.error) {
+				t.Errorf("Validate() error = %v, want it to contain %q", err, tc.error)
+			}
+		})
+	}
+}

cmd/ate-setup/internal/images/prebuilt.go

@@ -0,0 +1,161 @@
+// Copyright 2026 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package images
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"maps"
+	"regexp"
+	"slices"
+	"strings"
+
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/kube"
+)
+
+// koRef matches a whole ko:// reference: the prefix plus everything up to the
+// first delimiter that can end a YAML scalar.
+//
+// Matching the whole reference is what makes the rewrite safe. Replacing the
+// table's literals directly would rewrite a prefix of a longer reference, so
+// ko://<module>/cmd/atelet-sidecar would silently become the atelet image with
+// "-sidecar" left dangling on the tag. Requiring a character after the prefix
+// also keeps the "# ko:// image reference" prose in the demo templates from
+// matching.
+var koRef = regexp.MustCompile(`ko://[^\s"',\]}]+`)
+
+// imageByRef maps a ko:// reference to the image name it publishes as.
+// Rewriting is a lookup in this table.
+var imageByRef = func() map[string]string {
+	m := make(map[string]string, len(Components))
+	for _, pkg := range Components {
+		m[KoReference(pkg)] = ImageName(pkg)
+	}
+	return m
+}()
+
+// Prebuilt rewrites ko:// references to point at already-published images.
+//
+// It is the drop-in counterpart to ko.Runner: same two methods, no builds, no
+// registry writes, and no repository checkout beyond the manifests themselves.
+// What it does need is read access to the registry, to pin each tag to a
+// digest.
+type Prebuilt struct {
+	src    Source
+	digest Digester
+
+	// pinned caches a resolved reference by its tagged form, so a component
+	// several manifests name costs one registry lookup per install.
+	pinned map[string]string
+}
+
+// NewPrebuilt returns a resolver for an already-published image set. digest
+// turns a tag into the image it names; callers outside tests pass RemoteDigest.
+func NewPrebuilt(src Source, digest Digester) *Prebuilt {
+	return &Prebuilt{src: src, digest: digest, pinned: make(map[string]string)}
+}
+
+// ResolvePath rewrites the manifests a path covers.
+func (p *Prebuilt) ResolvePath(ctx context.Context, path string) ([]byte, error) {
+	manifest, err := kube.ReadPath(path)
+	if err != nil {
+		return nil, err
+	}
+	out, err := p.rewrite(ctx, manifest)
+	if err != nil {
+		return nil, fmt.Errorf("in %s: %w", path, err)
+	}
+	return out, nil
+}
+
+// ResolveBytes rewrites an in-memory manifest, such as kustomize output.
+func (p *Prebuilt) ResolveBytes(ctx context.Context, manifest []byte) ([]byte, error) {
+	return p.rewrite(ctx, manifest)
+}
+
+// pin returns the reference to install an image by: its tag, and the digest
+// that tag currently names.
+//
+// The digest is not decoration. An ActorTemplate's container image, an image
+// volume's reference, and a SandboxConfig's pauseImage each carry the CEL rule
+// self.contains('@'), so a tag on its own is rejected by admission and every
+// demo would fail to deploy. Pinning also restores what ko gave the control
+// plane deployments for free: an install that cannot shift under a tag someone
+// moves later. Keeping the tag alongside the digest is ko's own output shape,
+// and leaves the reference legible in `kubectl get`.
+func (p *Prebuilt) pin(ctx context.Context, image string) (string, error) {
+	tagged := p.src.Repo + "/" + image + ":" + p.src.Tag
+	if ref, ok := p.pinned[tagged]; ok {
+		return ref, nil
+	}
+	digest, err := p.digest(ctx, tagged)
+	if err != nil {
+		return "", err
+	}
+	ref := tagged + "@" + digest
+	p.pinned[tagged] = ref
+	return ref, nil
+}
+
+// rewrite replaces every ko:// reference with the image it maps to.
+//
+// Substitution is textual, the same way SubstituteVersion fills
+// ${SUBSTRATE_VERSION}. References appear in CRD fields as well as pod specs (a
+// WorkerPool's spec.workerImage, an ActorTemplate's image), so a schema walk
+// would need to know every such field, and rewriting the bytes leaves
+// everything else -- the multi-kilobyte Envoy configuration blocks in
+// particular -- exactly as committed.
+func (p *Prebuilt) rewrite(ctx context.Context, manifest []byte) ([]byte, error) {
+	// Collect the failures rather than stopping at the first. Diagnosing them
+	// one install attempt at a time is slow, and they are usually related: a
+	// registry that is unreachable or a tag that was never pushed fails for
+	// every component at once.
+	unknown := make(map[string]bool)
+	unpinned := make(map[string]error)
+
+	out := koRef.ReplaceAllFunc(manifest, func(match []byte) []byte {
+		image, ok := imageByRef[string(match)]
+		if !ok {
+			unknown[string(match)] = true
+			return match
+		}
+		ref, err := p.pin(ctx, image)
+		if err != nil {
+			unpinned[image] = err
+			return match
+		}
+		return []byte(ref)
+	})
+
+	var errs []error
+	if len(unknown) > 0 {
+		// One line per reference, then the package list once. Repeating the
+		// list per reference buries the references themselves.
+		for _, ref := range slices.Sorted(maps.Keys(unknown)) {
+			errs = append(errs, fmt.Errorf("%s has no published image", ref))
+		}
+		errs = append(errs, fmt.Errorf("installable packages under %s are %s; a reference that is "+
+			"templated has to be rendered before it can be resolved",
+			ModulePath, strings.Join(Components, ", ")))
+	}
+	for _, image := range slices.Sorted(maps.Keys(unpinned)) {
+		errs = append(errs, unpinned[image])
+	}
+	if len(errs) > 0 {
+		return nil, errors.Join(errs...)
+	}
+	return out, nil
+}

cmd/ate-setup/internal/images/prebuilt_test.go

@@ -0,0 +1,340 @@
+// Copyright 2026 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package images_test
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"maps"
+	"os"
+	"path/filepath"
+	"regexp"
+	"strings"
+	"testing"
+
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/config"
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/images"
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/kube"
+)
+
+// testSource is the repo/tag every rewrite test resolves against.
+var testSource = images.Source{Repo: "example.com/substrate", Tag: "v1.2.3"}
+
+// What a resolved reference carries beyond its tag. Fixed rather than derived
+// from the reference so the expected output below reads as a literal.
+const (
+	testDigest   = "sha256:1111111111111111111111111111111111111111111111111111111111111111"
+	digestSuffix = "@" + testDigest
+)
+
+// stubRegistry stands in for the registry the installer would query: it answers
+// every lookup with testDigest, or with err, and records what was asked for.
+type stubRegistry struct {
+	calls map[string]int
+	err   error
+}
+
+func newStubRegistry() *stubRegistry { return &stubRegistry{calls: make(map[string]int)} }
+
+// digest is the [images.Digester] a test passes to [images.NewPrebuilt].
+func (s *stubRegistry) digest(_ context.Context, ref string) (string, error) {
+	s.calls[ref]++
+	if s.err != nil {
+		return "", fmt.Errorf("resolving %s to a digest: %w", ref, s.err)
+	}
+	return testDigest, nil
+}
+
+func TestPrebuiltResolveBytes(t *testing.T) {
+	tests := []struct {
+		name  string
+		src   images.Source
+		in    string
+		want  string
+		error string
+	}{
+		{
+			name: "bare scalar",
+			in:   "        image: ko://github.com/agent-substrate/substrate/cmd/ateapi\n",
+			want: "        image: example.com/substrate/ateapi:v1.2.3" + digestSuffix + "\n",
+		},
+		{
+			name: "double quoted",
+			in:   `image: "ko://github.com/agent-substrate/substrate/cmd/atelet"`,
+			want: `image: "example.com/substrate/atelet:v1.2.3` + digestSuffix + `"`,
+		},
+		{
+			name: "single quoted",
+			in:   `image: 'ko://github.com/agent-substrate/substrate/cmd/atenet'`,
+			want: `image: 'example.com/substrate/atenet:v1.2.3` + digestSuffix + `'`,
+		},
+		{
+			name: "flow sequence",
+			in:   `images: [ko://github.com/agent-substrate/substrate/demos/counter, other]`,
+			want: `images: [example.com/substrate/counter:v1.2.3` + digestSuffix + `, other]`,
+		},
+		{
+			// The reference is a plain CRD field here, not a pod spec, which is
+			// why the rewrite cannot be driven off a Kubernetes schema.
+			name: "CRD field",
+			in:   "spec:\n  workerImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor\n",
+			want: "spec:\n  workerImage: example.com/substrate/ateom-gvisor:v1.2.3" + digestSuffix + "\n",
+		},
+		{
+			name: "nested package maps to its image name",
+			in:   "image: ko://github.com/agent-substrate/substrate/demos/multi-template/fspersist\n",
+			want: "image: example.com/substrate/fspersist:v1.2.3" + digestSuffix + "\n",
+		},
+		{
+			name: "two references on one line",
+			in:   "a: ko://github.com/agent-substrate/substrate/cmd/ateapi, b: ko://github.com/agent-substrate/substrate/demos/egress\n",
+			want: "a: example.com/substrate/ateapi:v1.2.3" + digestSuffix + ", b: example.com/substrate/egress:v1.2.3" + digestSuffix + "\n",
+		},
+		{
+			// The demo templates carry "# ko:// image reference" as prose. The
+			// regex requires a character after the prefix so those survive.
+			name: "prose mention is left alone",
+			in:   "        # ko:// image reference for the workload\n",
+			want: "        # ko:// image reference for the workload\n",
+		},
+		{
+			name: "nothing to rewrite",
+			in:   "apiVersion: v1\nkind: ConfigMap\n",
+			want: "apiVersion: v1\nkind: ConfigMap\n",
+		},
+		{
+			// benchmarking/workloads/manifests templates the sandbox class into
+			// the import path. It has to fail rather than resolve to something.
+			//
+			// The reported reference stops at the "}", which ends a YAML flow
+			// mapping and so terminates the match. That costs a character in the
+			// message and nothing else: the truncation cannot collide with a
+			// listed package, because every one of those ends before the "$".
+			name:  "unexpanded placeholder",
+			in:    "image: ko://github.com/agent-substrate/substrate/cmd/ateom-${SANDBOX_CLASS}\n",
+			error: "cmd/ateom-${SANDBOX_CLASS has no published image",
+		},
+		{
+			// A reference extending a known one must not inherit its image. A
+			// literal replacement would leave "-sidecar" dangling on the tag.
+			name:  "reference extending a known one",
+			in:    "image: ko://github.com/agent-substrate/substrate/cmd/atelet-sidecar\n",
+			error: "cmd/atelet-sidecar has no published image",
+		},
+		{
+			// ko accepts a "./"-relative import path in a manifest as readily as
+			// a full one, and would publish this as "atelet". Nothing in the tree
+			// spells a reference this way, so it is rejected rather than mapped:
+			// only the packages named in full can be known to have been published.
+			name:  "relative import path",
+			in:    "image: ko://./cmd/atelet\n",
+			error: "ko://./cmd/atelet has no published image",
+		},
+		{
+			// ko would map this onto its own "atelet" image, because it names
+			// images by the last path element alone. Matching the whole reference
+			// is what keeps another module's package out of this repo's registry
+			// path.
+			name:  "another module ending in a known name",
+			in:    "image: ko://github.com/example/other/cmd/atelet\n",
+			error: "ko://github.com/example/other/cmd/atelet has no published image",
+		},
+		{
+			name:  "outside the module",
+			in:    "image: ko://github.com/example/other/cmd/thing\n",
+			error: "ko://github.com/example/other/cmd/thing has no published image",
+		},
+		{
+			name:  "unknown component",
+			in:    "image: ko://github.com/agent-substrate/substrate/cmd/notacomponent\n",
+			error: "cmd/notacomponent has no published image",
+		},
+		{
+			// The e2e fixtures are not part of an install, so a manifest
+			// reaching the installer with one is a mistake worth reporting.
+			name:  "e2e fixture is not installable",
+			in:    "image: ko://github.com/agent-substrate/substrate/internal/e2e/fixtures/probe\n",
+			error: "e2e/fixtures/probe has no published image",
+		},
+		{
+			// Every unmappable reference is named, not just the first.
+			name:  "all failures are reported",
+			in:    "a: ko://github.com/agent-substrate/substrate/cmd/one\nb: ko://github.com/agent-substrate/substrate/cmd/two\n",
+			error: "cmd/two has no published image",
+		},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			src := tc.src
+			if src.Repo == "" {
+				src = testSource
+			}
+			got, err := images.NewPrebuilt(src, newStubRegistry().digest).
+				ResolveBytes(context.Background(), []byte(tc.in))
+
+			if tc.error != "" {
+				if err == nil {
+					t.Fatalf("ResolveBytes() = %q, want an error containing %q", got, tc.error)
+				}
+				if !strings.Contains(err.Error(), tc.error) {
+					t.Errorf("ResolveBytes() error = %v, want it to contain %q", err, tc.error)
+				}
+				return
+			}
+			if err != nil {
+				t.Fatalf("ResolveBytes() error = %v", err)
+			}
+			if string(got) != tc.want {
+				t.Errorf("ResolveBytes() =\n%q\nwant\n%q", got, tc.want)
+			}
+		})
+	}
+}
+
+// Every unmappable reference is reported at once, so a manifest with several
+// problems takes one install attempt to diagnose rather than several.
+func TestPrebuiltReportsEveryFailure(t *testing.T) {
+	in := "a: ko://github.com/agent-substrate/substrate/cmd/ateom-${SANDBOX_CLASS}\n" +
+		"b: ko://github.com/example/other/cmd/thing\n" +
+		"c: ko://github.com/agent-substrate/substrate/cmd/nope\n"
+
+	_, err := images.NewPrebuilt(testSource, newStubRegistry().digest).
+		ResolveBytes(context.Background(), []byte(in))
+	if err == nil {
+		t.Fatal("ResolveBytes() = nil error, want failures for all three references")
+	}
+	for _, want := range []string{"SANDBOX_CLASS", "github.com/example/other", "cmd/nope"} {
+		if !strings.Contains(err.Error(), want) {
+			t.Errorf("error is missing %q:\n%v", want, err)
+		}
+	}
+}
+
+// A repeated bad reference is one problem, not one per occurrence.
+func TestPrebuiltDeduplicatesFailures(t *testing.T) {
+	ref := "image: ko://github.com/agent-substrate/substrate/cmd/nope\n"
+	_, err := images.NewPrebuilt(testSource, newStubRegistry().digest).
+		ResolveBytes(context.Background(), []byte(ref+ref+ref))
+	if err == nil {
+		t.Fatal("ResolveBytes() = nil error, want a failure")
+	}
+	if n := strings.Count(err.Error(), "cmd/nope has no published image"); n != 1 {
+		t.Errorf("reported the same reference %d times, want 1:\n%v", n, err)
+	}
+}
+
+// A tag is resolved once per image, however many manifests name it, and the
+// answer is reused across calls: an install applies several manifests through
+// one resolver, and a registry round trip each time would be waste.
+func TestPrebuiltLooksUpEachImageOnce(t *testing.T) {
+	registry := newStubRegistry()
+	resolver := images.NewPrebuilt(testSource, registry.digest)
+
+	in := "a: ko://github.com/agent-substrate/substrate/cmd/ateapi\n" +
+		"b: ko://github.com/agent-substrate/substrate/cmd/atelet\n" +
+		"c: ko://github.com/agent-substrate/substrate/cmd/ateapi\n"
+	for range 2 {
+		if _, err := resolver.ResolveBytes(context.Background(), []byte(in)); err != nil {
+			t.Fatalf("ResolveBytes() error = %v", err)
+		}
+	}
+
+	want := map[string]int{
+		"example.com/substrate/ateapi:v1.2.3": 1,
+		"example.com/substrate/atelet:v1.2.3": 1,
+	}
+	if !maps.Equal(registry.calls, want) {
+		t.Errorf("registry lookups = %v, want %v", registry.calls, want)
+	}
+}
+
+// A registry that cannot answer fails the install, and says so for every image
+// it was asked about. Falling back to the bare tag would produce manifests that
+// admission rejects, and a partially pinned manifest would be worse still.
+func TestPrebuiltReportsDigestFailures(t *testing.T) {
+	registry := newStubRegistry()
+	registry.err = errors.New("UNAUTHORIZED")
+
+	in := "a: ko://github.com/agent-substrate/substrate/cmd/ateapi\n" +
+		"b: ko://github.com/agent-substrate/substrate/cmd/atelet\n"
+	got, err := images.NewPrebuilt(testSource, registry.digest).
+		ResolveBytes(context.Background(), []byte(in))
+	if err == nil {
+		t.Fatalf("ResolveBytes() = %q, want an error", got)
+	}
+	for _, want := range []string{
+		"resolving example.com/substrate/ateapi:v1.2.3 to a digest: UNAUTHORIZED",
+		"resolving example.com/substrate/atelet:v1.2.3 to a digest: UNAUTHORIZED",
+	} {
+		if !strings.Contains(err.Error(), want) {
+			t.Errorf("error is missing %q:\n%v", want, err)
+		}
+	}
+}
+
+// resolvedRef matches a rewritten reference in testSource's repository.
+var resolvedRef = regexp.MustCompile(regexp.QuoteMeta(testSource.Repo) + `/[^\s"',\]}]+`)
+
+// Resolving the control plane manifests must produce a manifest that still
+// parses and holds no unresolved reference. This is the closest a unit test
+// gets to the install itself.
+func TestResolveEveryInstallManifest(t *testing.T) {
+	root, err := config.RepoRoot()
+	if err != nil {
+		t.Fatalf("resolving repo root: %v", err)
+	}
+	installDir := filepath.Join(root, "manifests", "ate-install")
+
+	entries, err := os.ReadDir(installDir)
+	if err != nil {
+		t.Fatalf("listing %s: %v", installDir, err)
+	}
+
+	resolver := images.NewPrebuilt(testSource, newStubRegistry().digest)
+
+	// The directory as a whole is what a plain GKE install applies.
+	paths := []string{installDir}
+	for _, entry := range entries {
+		if !entry.IsDir() && filepath.Ext(entry.Name()) == ".yaml" {
+			paths = append(paths, filepath.Join(installDir, entry.Name()))
+		}
+	}
+
+	for _, p := range paths {
+		t.Run(filepath.Base(p), func(t *testing.T) {
+			out, err := resolver.ResolvePath(context.Background(), p)
+			if err != nil {
+				t.Fatalf("ResolvePath() = %v", err)
+			}
+			if refs := koRefPattern.FindAllString(string(out), -1); len(refs) > 0 {
+				t.Errorf("unresolved references survived: %v", refs)
+			}
+			// Every reference has to name a digest. ActorTemplate images,
+			// image volume references, and a SandboxConfig's pauseImage all
+			// carry the CEL rule self.contains('@'), so an unpinned reference
+			// is not applied at all, it is rejected by admission.
+			for _, ref := range resolvedRef.FindAllString(string(out), -1) {
+				if !strings.Contains(ref, "@sha256:") {
+					t.Errorf("%s is not pinned to a digest", ref)
+				}
+			}
+			if _, err := kube.DecodeManifestBytes(out); err != nil {
+				t.Errorf("the resolved manifest no longer parses: %v", err)
+			}
+		})
+	}
+}

cmd/ate-setup/internal/kube/manifest.go

@@ -75,12 +75,63 @@ func DecodeManifestBytes(data []byte) ([]*unstructured.Unstructured, error) {
 // comments there call out specific filename-ordering hazards, so recursing or
 // reordering here would change install behavior.
 func LoadPath(path string) ([]*unstructured.Unstructured, error) {
+	files, err := manifestFiles(path)
+	if err != nil {
+		return nil, err
+	}
+
+	var objs []*unstructured.Unstructured
+	for _, file := range files {
+		fileObjs, err := loadFile(file)
+		if err != nil {
+			return nil, err
+		}
+		objs = append(objs, fileObjs...)
+	}
+	return objs, nil
+}
+
+// ReadPath concatenates the same files LoadPath would read into a single
+// multi-document stream, leaving them unparsed.
+//
+// This is the input the pre-built image resolver takes, and sharing
+// manifestFiles keeps it reading exactly what an apply applies. ko is handed
+// the path and enumerates it itself, accepting only .yaml and .json where this
+// also accepts .yml; nothing in the tree uses that extension, and a manifest
+// that did would reach an apply with its ko:// references unresolved.
+func ReadPath(path string) ([]byte, error) {
+	files, err := manifestFiles(path)
+	if err != nil {
+		return nil, err
+	}
+
+	var out bytes.Buffer
+	for _, file := range files {
+		data, err := os.ReadFile(file)
+		if err != nil {
+			return nil, fmt.Errorf("while reading %s: %w", file, err)
+		}
+		// A separator before every document, including the first: a leading
+		// empty document decodes to nothing, whereas a file that does not end
+		// in a newline would otherwise run into the next one.
+		out.WriteString("---\n")
+		out.Write(data)
+		if !bytes.HasSuffix(data, []byte("\n")) {
+			out.WriteString("\n")
+		}
+	}
+	return out.Bytes(), nil
+}
+
+// manifestFiles lists the files a manifest path covers: the file itself, or
+// every manifest directly inside a directory, in lexical order.
+func manifestFiles(path string) ([]string, error) {
 	info, err := os.Stat(path)
 	if err != nil {
 		return nil, fmt.Errorf("while reading %s: %w", path, err)
 	}
 	if !info.IsDir() {
-		return loadFile(path)
+		return []string{path}, nil
 	}
 
 	entries, err := os.ReadDir(path)
@@ -95,19 +146,10 @@ func LoadPath(path string) ([]*unstructured.Unstructured, error) {
 		if !manifestExtensions[strings.ToLower(filepath.Ext(entry.Name()))] {
 			continue
 		}
-		names = append(names, entry.Name())
+		names = append(names, filepath.Join(path, entry.Name()))
 	}
 	sort.Strings(names)
-
-	var objs []*unstructured.Unstructured
-	for _, name := range names {
-		fileObjs, err := loadFile(filepath.Join(path, name))
-		if err != nil {
-			return nil, err
-		}
-		objs = append(objs, fileObjs...)
-	}
-	return objs, nil
+	return names, nil
 }
 
 func loadFile(path string) ([]*unstructured.Unstructured, error) {

cmd/ate-setup/internal/kube/manifest_test.go

@@ -0,0 +1,184 @@
+// Copyright 2026 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package kube
+
+import (
+	"errors"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+// writeFiles populates a temporary directory and returns its path.
+func writeFiles(t *testing.T, files map[string]string) string {
+	t.Helper()
+	dir := t.TempDir()
+	for name, body := range files {
+		path := filepath.Join(dir, name)
+		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+			t.Fatal(err)
+		}
+		if err := os.WriteFile(path, []byte(body), 0o600); err != nil {
+			t.Fatal(err)
+		}
+	}
+	return dir
+}
+
+// manifest returns a minimal object whose name identifies it in assertions.
+func manifest(name string) string {
+	return "apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: " + name + "\n"
+}
+
+func names(t *testing.T, data []byte) []string {
+	t.Helper()
+	objs, err := DecodeManifestBytes(data)
+	if err != nil {
+		t.Fatalf("DecodeManifestBytes() = %v", err)
+	}
+	out := make([]string, 0, len(objs))
+	for _, obj := range objs {
+		out = append(out, obj.GetName())
+	}
+	return out
+}
+
+func equal(a, b []string) bool {
+	if len(a) != len(b) {
+		return false
+	}
+	for i := range a {
+		if a[i] != b[i] {
+			return false
+		}
+	}
+	return true
+}
+
+// ReadPath feeds the pre-built image resolver, and its output is applied in
+// place of what LoadPath would have applied. The two must therefore cover the
+// same documents in the same order: the install depends on that ordering, and
+// a divergence would be a silently different install rather than an error.
+func TestReadPathMatchesLoadPath(t *testing.T) {
+	tests := []struct {
+		name  string
+		files map[string]string
+		want  []string
+	}{
+		{
+			name:  "single file",
+			files: map[string]string{"a.yaml": manifest("only")},
+			want:  []string{"only"},
+		},
+		{
+			name: "directory is lexical, not filesystem order",
+			files: map[string]string{
+				"20-second.yaml": manifest("second"),
+				"10-first.yaml":  manifest("first"),
+				"30-third.yaml":  manifest("third"),
+			},
+			want: []string{"first", "second", "third"},
+		},
+		{
+			name: "non-manifest extensions are skipped",
+			files: map[string]string{
+				"keep.yaml":     manifest("keep"),
+				"keep2.yml":     manifest("keep2"),
+				"keep3.json":    `{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"keep3"}}`,
+				"skip.tmpl":     manifest("skip"),
+				"skip.md":       "# not a manifest\n",
+				"skip.yaml.bak": manifest("skip"),
+			},
+			want: []string{"keep", "keep2", "keep3"},
+		},
+		{
+			name: "nested directories are not recursed into",
+			files: map[string]string{
+				"top.yaml":         manifest("top"),
+				"nested/deep.yaml": manifest("deep"),
+			},
+			want: []string{"top"},
+		},
+		{
+			name: "a file with no trailing newline does not merge into the next",
+			files: map[string]string{
+				"a.yaml": "apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: a",
+				"b.yaml": manifest("b"),
+			},
+			want: []string{"a", "b"},
+		},
+		{
+			name: "an empty file contributes no document",
+			files: map[string]string{
+				"a.yaml":     manifest("a"),
+				"empty.yaml": "",
+			},
+			want: []string{"a"},
+		},
+		{
+			name: "multi-document files keep their internal order",
+			files: map[string]string{
+				"a.yaml": manifest("one") + "---\n" + manifest("two"),
+			},
+			want: []string{"one", "two"},
+		},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			dir := writeFiles(t, tc.files)
+
+			// A single-file case addresses the file directly, which is the
+			// shape the manifest-path call sites use.
+			path := dir
+			if len(tc.files) == 1 {
+				for name := range tc.files {
+					path = filepath.Join(dir, name)
+				}
+			}
+
+			data, err := ReadPath(path)
+			if err != nil {
+				t.Fatalf("ReadPath() = %v", err)
+			}
+			got := names(t, data)
+			if !equal(got, tc.want) {
+				t.Errorf("ReadPath() objects = %v, want %v", got, tc.want)
+			}
+
+			objs, err := LoadPath(path)
+			if err != nil {
+				t.Fatalf("LoadPath() = %v", err)
+			}
+			loaded := make([]string, 0, len(objs))
+			for _, obj := range objs {
+				loaded = append(loaded, obj.GetName())
+			}
+			if !equal(loaded, got) {
+				t.Errorf("LoadPath() objects = %v, ReadPath() objects = %v; want identical", loaded, got)
+			}
+		})
+	}
+}
+
+// The manifest-path call sites hand ReadPath a path that may not exist on a
+// teardown, and DeletePath distinguishes that from a real failure by checking
+// for fs.ErrNotExist. Keep the wrapping transparent to errors.Is.
+func TestReadPathMissing(t *testing.T) {
+	if _, err := ReadPath(filepath.Join(t.TempDir(), "absent.yaml")); !errors.Is(err, fs.ErrNotExist) {
+		t.Errorf("ReadPath(absent) = %v, want a not-exist error", err)
+	}
+}

cmd/ate-setup/internal/steps/csi.go

@@ -75,7 +75,7 @@ func (e *Env) SetupCSI(ctx context.Context) error {
 	if err := e.EnsurePodCertificateCAs(ctx); err != nil {
 		return err
 	}
-	if err := e.KoApply(ctx, e.Cfg.Manifest("pod-certificate-controller.yaml")); err != nil {
+	if err := e.ResolveAndApply(ctx, e.Cfg.Manifest("pod-certificate-controller.yaml")); err != nil {
 		return err
 	}
 	if err := e.applyPodcertWorkersOverride(ctx); err != nil {

cmd/ate-setup/internal/steps/deploy.go

@@ -85,7 +85,7 @@ func (e *Env) DeployAteSystem(ctx context.Context, opts DeployOptions) error {
 
 	// The podcertificate controller goes first so it starts signing and
 	// publishing trust bundles immediately.
-	if err := e.KoApply(ctx, e.Cfg.Manifest("pod-certificate-controller.yaml")); err != nil {
+	if err := e.ResolveAndApply(ctx, e.Cfg.Manifest("pod-certificate-controller.yaml")); err != nil {
 		return err
 	}
 	if err := e.applyPodcertWorkersOverride(ctx); err != nil {
@@ -228,7 +228,7 @@ func (e *Env) DeployAteAPIServer(ctx context.Context) error {
 	if err := e.applyOtelConfig(ctx); err != nil {
 		return err
 	}
-	if err := e.KoApply(ctx, e.Cfg.Manifest("ate-api-server.yaml")); err != nil {
+	if err := e.ResolveAndApply(ctx, e.Cfg.Manifest("ate-api-server.yaml")); err != nil {
 		return err
 	}
 	return e.Kube.RolloutStatus(ctx, kube.KindDeployment, NamespaceAteSystem, "ate-api-server", e.Cfg.RolloutTimeout)
@@ -247,7 +247,7 @@ func (e *Env) DeployAteController(ctx context.Context) error {
 	if err := e.applyOtelConfig(ctx); err != nil {
 		return err
 	}
-	if err := e.KoApply(ctx, e.Cfg.Manifest("ate-controller.yaml")); err != nil {
+	if err := e.ResolveAndApply(ctx, e.Cfg.Manifest("ate-controller.yaml")); err != nil {
 		return err
 	}
 	return e.Kube.RolloutStatus(ctx, kube.KindDeployment, NamespaceAteSystem, "ate-controller", e.Cfg.RolloutTimeout)
@@ -276,7 +276,7 @@ func (e *Env) DeployAtelet(ctx context.Context) error {
 		// The kind overlay patches the DaemonSet for the local node layout.
 		manifest, err = e.KustomizeResolve(ctx, installDir+"/kind/atelet")
 	} else {
-		manifest, err = e.KoResolve(ctx, e.Cfg.Manifest("atelet.yaml"))
+		manifest, err = e.ResolveManifest(ctx, e.Cfg.Manifest("atelet.yaml"))
 	}
 	if err != nil {
 		return err
@@ -322,7 +322,7 @@ func (e *Env) DeployAtenet(ctx context.Context) error {
 	if err := e.applyAtenetEgress(ctx); err != nil {
 		return err
 	}
-	if err := e.KoApply(ctx, e.Cfg.Manifest("atenet-dns.yaml")); err != nil {
+	if err := e.ResolveAndApply(ctx, e.Cfg.Manifest("atenet-dns.yaml")); err != nil {
 		return err
 	}
 
@@ -359,7 +359,7 @@ func (e *Env) EnsureCRDs(ctx context.Context) error {
 // DeployCRDs applies the generated CRDs and RBAC, waiting for them to reach Established condition.
 func (e *Env) DeployCRDs(ctx context.Context) error {
 	log.Step("deploy_crds")
-	if err := e.KoApply(ctx, e.Cfg.Manifest("generated")); err != nil {
+	if err := e.ResolveAndApply(ctx, e.Cfg.Manifest("generated")); err != nil {
 		return err
 	}
 	for _, name := range ateCRDs {

cmd/ate-setup/internal/steps/env.go

@@ -23,9 +23,11 @@ import (
 	"time"
 
 	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/config"
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/images"
 	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/ko"
 	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/kube"
 	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/kustomize"
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/log"
 )
 
 // Timeouts carried over from the --timeout values in the shell installer.
@@ -48,15 +50,24 @@ const (
 	NamespacePodCert   = "podcertificate-controller-system"
 )
 
+// imageResolver turns a manifest whose image references are ko:// import paths
+// into one whose references are pullable. *ko.Runner builds and publishes them
+// from source; *images.Prebuilt maps them onto an already-published release.
+type imageResolver interface {
+	ResolvePath(ctx context.Context, path string) ([]byte, error)
+	ResolveBytes(ctx context.Context, manifest []byte) ([]byte, error)
+}
+
 // Env is the shared execution context for every step: resolved configuration
 // plus the clients needed to act on the cluster.
 type Env struct {
 	Cfg  *config.Config
 	Kube *kube.Client
 
-	// ko is created lazily. Steps that only apply static manifests must work
-	// on a machine with no container registry credentials configured.
-	ko *ko.Runner
+	// resolver is created lazily. Steps that only apply static manifests must
+	// work on a machine with no container registry credentials configured, and
+	// the build-from-source resolver fails at construction without them.
+	resolver imageResolver
 
 	// substrateVersion caches the derived build version and its object-name
 	// suffix; see SubstrateVersion.
@@ -73,51 +84,59 @@ func NewEnv(cfg *config.Config) (*Env, error) {
 	return &Env{Cfg: cfg, Kube: client}, nil
 }
 
-// Ko returns the ko runner, creating it on first use.
-func (e *Env) Ko() (*ko.Runner, error) {
-	if e.ko != nil {
-		return e.ko, nil
+// imageResolver returns the image resolver, creating it on first use.
+func (e *Env) imageResolver() (imageResolver, error) {
+	if e.resolver != nil {
+		return e.resolver, nil
 	}
+
+	log.Stepf("images: %s", e.Cfg.Images.Describe())
+	if e.Cfg.Images.IsPrebuilt() {
+		e.resolver = images.NewPrebuilt(e.Cfg.Images, images.RemoteDigest)
+		return e.resolver, nil
+	}
+
 	runner, err := ko.New(e.Cfg.Root, e.Cfg.KoEnv())
 	if err != nil {
 		return nil, err
 	}
-	e.ko = runner
-	return e.ko, nil
+	e.resolver = runner
+	return e.resolver, nil
 }
 
-// KoApply resolves a manifest path with ko and applies the result. This is the
-// run_ko apply of the shell scripts, split into its two real steps.
-func (e *Env) KoApply(ctx context.Context, path string) error {
-	manifest, err := e.KoResolve(ctx, path)
+// ResolveAndApply resolves the images in a manifest path and applies the
+// result. This is the run_ko apply of the shell scripts, split into its two
+// real steps.
+func (e *Env) ResolveAndApply(ctx context.Context, path string) error {
+	manifest, err := e.ResolveManifest(ctx, path)
 	if err != nil {
 		return err
 	}
 	return e.Kube.ApplyBytes(ctx, manifest)
 }
 
-// KoResolve builds and publishes the images in a manifest path, returning the
-// digest-pinned manifest.
-func (e *Env) KoResolve(ctx context.Context, path string) ([]byte, error) {
-	runner, err := e.Ko()
+// ResolveManifest turns the ko:// references in a manifest path into pullable
+// image references.
+func (e *Env) ResolveManifest(ctx context.Context, path string) ([]byte, error) {
+	resolver, err := e.imageResolver()
 	if err != nil {
 		return nil, err
 	}
-	return runner.ResolvePath(ctx, path)
+	return resolver.ResolvePath(ctx, path)
 }
 
-// KoResolveBytes resolves an in-memory manifest, such as kustomize output.
-func (e *Env) KoResolveBytes(ctx context.Context, manifest []byte) ([]byte, error) {
-	runner, err := e.Ko()
+// ResolveManifestBytes resolves an in-memory manifest, such as kustomize output.
+func (e *Env) ResolveManifestBytes(ctx context.Context, manifest []byte) ([]byte, error) {
+	resolver, err := e.imageResolver()
 	if err != nil {
 		return nil, err
 	}
-	return runner.ResolveBytes(ctx, manifest)
+	return resolver.ResolveBytes(ctx, manifest)
 }
 
-// KoApplyBytes resolves an in-memory manifest with ko and applies the result.
-func (e *Env) KoApplyBytes(ctx context.Context, manifest []byte) error {
-	resolved, err := e.KoResolveBytes(ctx, manifest)
+// ResolveAndApplyBytes resolves an in-memory manifest and applies the result.
+func (e *Env) ResolveAndApplyBytes(ctx context.Context, manifest []byte) error {
+	resolved, err := e.ResolveManifestBytes(ctx, manifest)
 	if err != nil {
 		return err
 	}
@@ -129,14 +148,14 @@ func (e *Env) Kustomize(overlay string) ([]byte, error) {
 	return kustomize.Build(e.Cfg.Path(overlay))
 }
 
-// KustomizeResolve renders an overlay and pipes it through ko, the
+// KustomizeResolve renders an overlay and resolves its images, the
 // `kubectl kustomize ... | run_ko resolve -f -` pipeline.
 func (e *Env) KustomizeResolve(ctx context.Context, overlay string) ([]byte, error) {
 	built, err := e.Kustomize(overlay)
 	if err != nil {
 		return nil, err
 	}
-	return e.KoResolveBytes(ctx, built)
+	return e.ResolveManifestBytes(ctx, built)
 }
 
 // EnsureAteSystemNamespace applies the ate-system namespace manifest and waits

cmd/ate-setup/internal/steps/overlay.go

@@ -51,7 +51,7 @@ func (e *Env) renderSystemManifests(ctx context.Context) ([]byte, error) {
 	if overlay := SystemOverlay(e.Cfg); overlay != "" {
 		return e.KustomizeResolve(ctx, overlay)
 	}
-	return e.KoResolve(ctx, e.Cfg.Manifest())
+	return e.ResolveManifest(ctx, e.Cfg.Manifest())
 }
 
 // renderAtenetRouterManifest produces the atenet router manifest for the
@@ -60,7 +60,7 @@ func (e *Env) renderAtenetRouterManifest(ctx context.Context) ([]byte, error) {
 	if e.Cfg.Router == config.RouterAgentgateway {
 		return e.KustomizeResolve(ctx, installDir+"/agentgateway-router")
 	}
-	return e.KoResolve(ctx, e.Cfg.Manifest("atenet-router.yaml"))
+	return e.ResolveManifest(ctx, e.Cfg.Manifest("atenet-router.yaml"))
 }
 
 // atenetEgressManifestPath returns the egress manifest path based on configuration.
@@ -85,10 +85,10 @@ func (e *Env) renderAtenetEgressManifest(ctx context.Context) ([]byte, error) {
 		if err != nil {
 			return nil, err
 		}
-		return e.KoResolveBytes(ctx, patched)
+		return e.ResolveManifestBytes(ctx, patched)
 	}
 
-	return e.KoResolve(ctx, e.atenetEgressManifestPath())
+	return e.ResolveManifest(ctx, e.atenetEgressManifestPath())
 }
 
 func (e *Env) patchAtenetEgressManifest() ([]byte, error) {

cmd/ate-setup/internal/steps/postgres.go

@@ -37,7 +37,7 @@ func (e *Env) DeployPostgres(ctx context.Context) error {
 	// The StatefulSet's projected serving certificate is issued by this
 	// controller. Applying it here makes `deploy postgres` usable on a fresh
 	// cluster as well as after `deploy ate-system`.
-	if err := e.KoApply(ctx, e.Cfg.Manifest("pod-certificate-controller.yaml")); err != nil {
+	if err := e.ResolveAndApply(ctx, e.Cfg.Manifest("pod-certificate-controller.yaml")); err != nil {
 		return err
 	}
 	if err := e.applyPodcertWorkersOverride(ctx); err != nil {

cmd/ate-setup/internal/steps/version.go

@@ -17,6 +17,7 @@ package steps
 import (
 	"context"
 	"fmt"
+	"os"
 	"regexp"
 	"strings"
 	"time"
@@ -31,11 +32,28 @@ import (
 
 // SubstrateVersion returns the build version and the object-name suffix for atelet.
 // The version is the same one ko stamps into the binaries.
+//
+// With --image-repo nothing is built, so `git describe` would report the
+// checkout rather than the images being installed. The image tag is the version
+// in that case. It has to be: the version names the atelet DaemonSet and sets
+// the node label that partitions nodes across coexisting versions, so it must
+// describe the atelet that is actually running, which came from the image.
+// VERSION still wins over both, so a tag that is not a valid label value can be
+// worked around the same way.
 func (e *Env) SubstrateVersion() (version, suffix string, err error) {
 	if e.substrateVersion != "" {
 		return e.substrateVersion, e.substrateVersionSuffix, nil
 	}
-	raw := ko.BuildVersion(e.Cfg.Root)
+	var raw string
+	if e.Cfg.Images.IsPrebuilt() {
+		raw = os.Getenv("VERSION")
+		if raw == "" {
+			raw = e.Cfg.Images.Tag
+		}
+	} else {
+		// ko.BuildVersion consults VERSION itself before `git describe`.
+		raw = ko.BuildVersion(e.Cfg.Root)
+	}
 	if v := versionlabel.Value(raw); v != raw {
 		return "", "", fmt.Errorf("build version %q is not a valid label value (it would sanitize to %q); pin a label-safe one with the VERSION env var", raw, v)
 	}

cmd/ate-setup/internal/steps/version_test.go

@@ -14,7 +14,49 @@
 
 package steps
 
-import "testing"
+import (
+	"testing"
+
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/config"
+	"github.com/agent-substrate/substrate/cmd/ate-setup/internal/images"
+)
+
+// TestSubstrateVersionPrebuilt covers the version a prebuilt install stamps.
+// Root points at a directory that is not a checkout, so `git describe` has
+// nothing to say and only the image tag can supply it.
+func TestSubstrateVersionPrebuilt(t *testing.T) {
+	t.Setenv("VERSION", "")
+
+	cases := []struct {
+		name       string
+		source     images.Source
+		want       string
+		wantSuffix string
+	}{{
+		name:       "tag is the version",
+		source:     images.Source{Repo: "example.com/substrate", Tag: "v0.0.0-503-4b3423c0"},
+		want:       "v0.0.0-503-4b3423c0",
+		wantSuffix: "v0-0-0-503-4b3423c0",
+	}, {
+		name:       "built from source falls back to git",
+		source:     images.Source{},
+		want:       "dev",
+		wantSuffix: "dev",
+	}}
+
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			e := &Env{Cfg: &config.Config{Root: t.TempDir(), Images: tc.source}}
+			got, suffix, err := e.SubstrateVersion()
+			if err != nil {
+				t.Fatalf("SubstrateVersion: %v", err)
+			}
+			if got != tc.want || suffix != tc.wantSuffix {
+				t.Fatalf("SubstrateVersion = %q, %q; want %q, %q", got, suffix, tc.want, tc.wantSuffix)
+			}
+		})
+	}
+}
 
 func TestSubstituteVersion(t *testing.T) {
 	e := &Env{substrateVersion: "v1.2.3", substrateVersionSuffix: "v1-2-3"}