Nested virtualization exposes `/dev/kvm`, which microVMs need. The new
`--enable-nested-virtualization` flag turns it on for the node pool
`setup-gcp` creates. It defaults to on, so pass
`--enable-nested-virtualization=false` for a cluster that does not need
it.
Also, rename the env var `GVISOR_NODE_MACHINE_TYPE` to
`NODE_MACHINE_TYPE` to be consistent with other env vars and to avoid
confusion for microVMs. The old env var still works with a warning.
- [x] Tests pass
- [x] Appropriate changes to documentation are included in the PR
Component authors had the metric registry and the cardinality rules but
no single place that says how to add an instrument: which kind to pick,
how to name it, which labels are allowed, how the Go is shaped, how to
test it with a ManualReader, and how to register it. Each of those
conventions existed only as a pattern in one of the metrics.go files.
The guide sits beside tracing.md and otel-collector.md. It uses the
instruments already in the tree as the examples, ends with a worked
example of a node-local snapshot cache (a counter by outcome, a fill
histogram, a size gauge and an eviction counter, and what is
deliberately not a label). The registry snippets in the guide pass
`weaver registry check` when appended to the real registry.
observability.md and AGENTS.md now point at the guide from the place
that already told contributors to update the registry.
Fixes#1552
> It's a good idea to open an issue first for discussion.
- [x] Tests pass
- [x] Appropriate changes to documentation are included in the PR
This adds a test that serves as a guardrail to prevent changes to the
atespace-scoped tables that would make it difficult or impossible to
partition it in the future.
- [x] Tests pass
- [x] Appropriate changes to documentation are included in the PR
cc @bowei @BenTheElder @juli4n @shrutiyam-glitch @thockin
## What
Now that the ActorTemplate CRD has been deleted and its resources moved
to the substrate gRPC API and the control-plane store (created/managed
with `kubectl-ate`, persisted in PostgreSQL), several documents still
describe ActorTemplate as a Kubernetes CRD, or describe namespace/RBAC
relationships that no longer exist. This sweeps the docs for those stale
references.
Fixes#368 (docs side).
> This change was prepared with AI assistance; I have reviewed and
tested it.
- [x] Docs and comment-only change; no functional code changed, no tests
affected
### Description
This PR introduces native, secure support for using Cloud SQL as the
PostgreSQL store backend for `ate-api-server`.
To ensure the highest level of security and ease of use in GCP
environments, this integration leverages the Cloud SQL Auth Proxy
sidecar with automatic IAM database authentication. This means transport
security (TLS 1.3 tunnel) is handled automatically, and database
sessions are authenticated using Workload Identity via short-lived OAuth
tokens, completely eliminating the need for database passwords.
### Key Changes
* **Cloud SQL Auth Proxy Sidecar:** Added
`manifests/ate-install/cloudsql-proxy-patch.yaml` to patch the sidecar
into the `ate-api-server` deployment when a Cloud SQL instance is
configured.
* **Automated Provisioning:** Extended `tools/setup-gcp` with a new
`cloudsql` command. This handles the idempotent creation of the Cloud
SQL instance, Google Service Accounts (GSA), IAM bindings, and Workload
Identity bindings.
* **Installation Script Updates:** Updated `hack/install-ate.sh` to
parse new environment variables (e.g.,
`ATE_API_POSTGRES_CLOUDSQL_INSTANCE`, `ATE_API_POSTGRES_CLOUDSQL_GSA`)
and correctly synthesize the passwordless DSN and ConfigMaps for the
proxy.
* **Security & Documentation:**
* Added extensive documentation in `tools/setup-gcp/cloud-sql.md`
covering provisioning, schema privileges, deployment, and database
scaling.
* Updated `docs/threat-model.md` to reflect the new Cloud SQL egress
flows and Auth Proxy tunnel mechanics.
* **Dependencies:** Vendored required Google API clients (`sqladmin/v1`,
`servicenetworking/v1`, `iam/v1`) for the GCP setup tool.
- [X] Tests pass
- [X] Appropriate changes to documentation are included in the PR
> [!WARNING]
> Recreate PostgreSQL databases from earlier development builds.
## Summary
This change replaces startup schema setup with embedded, versioned SQL
migrations.
`ateapi` uses Goose to apply migrations before readiness. Goose stores
one ledger record for each applied migration.
Goose runs each migration and inserts its ledger record in one
PostgreSQL transaction.
Closes#901.
Based on this design:
https://docs.google.com/document/d/13ixDKRoAIFXeLxS-_1nikNcAy76ca8m0eVgobfib93E/edit?usp=sharing
## Migration behavior
`ateapi` gets a session advisory lock for the configured schema before
it applies pending migrations.
One replica applies migrations while other replicas wait. Goose reads
the ledger again after it gets the lock.
If a migration fails, PostgreSQL rolls back its SQL and ledger record.
Earlier successful migrations remain applied and recorded.
Kubernetes restarts the failed replica. The next startup resumes from
the first migration without a ledger record.
## Changes
- Add Goose and a per-migration ledger.
- Replace the initial up and down files with one transactional, up-only
migration.
- Keep migration 1 aligned with the current schema, including actor
egress policy storage.
- Remove existence guards and explicit transaction statements from the
baseline migration.
- Apply all pending migrations before `ateapi` becomes ready.
- Serialize each migration run with a PostgreSQL session advisory lock.
- Start without changes when the database schema is current or ahead.
- Reject application tables that do not have a migration ledger.
- Log the starting, current, and latest versions.
- Log the applied migration count and duration.
- Retry only initial database connection failures.
- Return schema and migration errors without a retry.
- Add `--postgres-schema` and `ATE_API_POSTGRES_SCHEMA`.
- Use `public` as the default PostgreSQL schema.
- Use the configured schema for the main and watch pools.
- Restrict outbox partition maintenance to the configured schema.
- Let the installer use an external PostgreSQL database.
- Add the migration design and recovery policy to the repository.
## Migration file policy
Migration files use sequential versions and contain exactly one Goose
`Up` section.
CI rejects down migrations, nontransactional migrations, environment
substitution, explicit transaction control, and `IF NOT EXISTS` guards.
Before the first stable v1 release, developers can change or squash
migrations. Developers must recreate databases after migration history
changes.
After that release, CI rejects changes or deletions against the latest
stable release tag that contains migrations.
Goose does not store migration checksums. The binary embeds each
migration file, and release-tag checks protect released migration
history.
## Compatibility
No release includes PostgreSQL support. The `v0.0.0` release predates
the PostgreSQL backend.
Users must recreate databases from earlier PostgreSQL development
builds.
Every committed migration prefix must work with the current and previous
`ateapi` releases. This rule supports rolling upgrades and temporary
binary rollback.
A binary rollback does not roll back the database schema.
## Testing
Tests cover:
- Fresh database migration.
- Concurrent startup.
- Advisory lock waits.
- Current and ahead database schemas.
- Rejection of application tables without a migration ledger.
- Atomic rollback of a failed migration.
- Retention of earlier successful migrations.
- Resume from the failed migration after restart.
- Configured schema isolation.
- Outbox partition isolation.
- Migration file policy checks.
- Stable release migration immutability.
This PR is very large since it updates all existing demos and benchmark
workloads to use the new ActorTemplate substrate proto.
Please use the "Commits" tab to review individual commits.
Verifications done:
* Used this script: gpaste/5143788763348992 to verify that the change
from CRD -> proto are equivalent.
* The e2e tests are using the new susbtrate resources.
* Picked the parking demo to run e2e manually: gpaste/6193361380311040
This is the companion to #496
In order to grant access to `/dev/kvm` we have to either:
- use a device plugin
- use a DRA driver
- use an NRI plugin
An NRI plugin is highly privileged in it's own right for all pods on the
host and is difficult to ship portably at the moment.
DRA is promising, but not enough functionality is GA yet at our current
1.35+ target.
Device plugin fits reasonably well. We do wind up publishing an
~arbitrarily high limit, which has some cost in kubelet memory, but
otherwise is relatively clean.
This approach is what kata uses currently. Their device plugin is not
available unbundled, and we anyhow have a per-node daemonset.
atlet is taught to sniff if /dev/kvm appears on the host at all, so we
can also stop using the manually labeled nodes for microVM class and
instead schedule to the KVM + TUN devices on nodes that advertise them.
Later we can migrate to device plugin by using
I implemented that already, but I don't think it's worth merging at the
moment. We would want consumable capacity to be on by default. We can
migrate later without changing the pod spec by using
`extendedResourceName`.
https://github.com/agent-substrate/substrate/compare/main...BenTheElder:substrate:ateom-microvm-dra
NOTE: I confirmed with upstream that device plugin is not going anywhere
despite being "v1beta1", it's GA in all but name. It won't receive new
features but we don't really need anyhow. We'll move to DRA down the
line.
---
By doing this, we can drop `privileged: true` from the uVM ateom pods.
We can also drop the `ate.dev/sandboxClass` node label hacks, reducing
friction to deploy.
## Summary
A follow up to #640 where we introduced PostgreSQL as an alternative
storage backend, selected conditionally in ateapi.
- Deleted ateredis, its tests, and its dependencies
- Removed Redis backend selection and configuration so ateapi always
connects to Postgres
- Replaced Valkey resources with Postgres in the standard and Kind
deployment paths and simplified install script
- Replaced miniredis fixtures with isolated Postgres testcontainers and
added centralized helpers for seeding resources
- Renamed Redis-specific debug flush command to backend-neutral
`debug-clear-store` in CLI
- Updated comments and docs where applicable
## Benchmarking
Extensive benchmarking have been performed to evaluate Redis vs
Postgres, and results can be found in these two documents:
-
https://docs.google.com/document/d/10K0wB6aTeFkJCL4HN3NbLJCdFGoLYdhIcqkFnqFHkKc/edit?usp=sharing
-
https://docs.google.com/document/d/12-ko_BFHcBo_nJkx9f4B7zMbiiWKC2saGhMhZG3aQ-s/edit?usp=sharing
---------
Signed-off-by: Jet Chiang <pokyuen.jetchiang-ext@solo.io>
Actor logs used `ate.dev/actor_*` while spans and metrics use `ate.*`
registry in internal/ateattr. This PR makes ateattr the single source of
truth for everything telemetry-related.
- Renamed the six actor log labels onto the registry:
- `ate.atespace`, `ate.actor.name`, `ate.actor.uid`,
`ate.template.namespace`, `ate.template.name`,
`ate.actor.container.name`
- Logs join traces now. Records set `trace_id`, `span_id` and
`trace_flags`, so you can go from Actor restored to the resume that
caused it. Our own lines only, not an actor's stdout: one goroutine
forwards a whole container stream and can't know which request produced
a given line. Per line correlation comes with #853.
- Actors can't fake platform labels. They already couldn't overwrite
ours, but they could invent new ones like `ate.tenant` that look
platform issued downstream. Anything under `ate.` from an actor is now
dropped.
- Fixed the asymmetry that was actually left: actor supplied label
values weren't stringified, and one non string value makes Cloud Logging
discard the labels for that whole entry.
- Note: the actor_uid bullet in the issue is stale, #841 fixed it
earlier. Lifecycle records still set five labels rather than six, on
purpose as they're about the actor, so no container produced them.
Fixes#886
- [x] Tests pass
- [x] Appropriate changes to documentation are included in the PR
---------
Signed-off-by: krisztianfekete <git@krisztianfekete.org>
Part of #563
Substrate had no way to measure how much telemetry it sends. This PR
adds one, and builds it from the parts the benchmark suite already has.
The managed collector sends its self-metrics to Cloud Monitoring only,
and gives no data for each source. Thus you cannot see which component
makes the volume. A second collector, the telemetry meter, counts the
spans and the datapoints by `service.name`.
## Contents
- **`benchmarking/telemetry/meter.yaml`** — the meter. A tee: it counts
the telemetry by service, then sends it on to the managed collector.
Thus one run gives the volume for each service and the cost of the load
together.
- **`benchmarking/telemetry/README.md`** — the method: install, measure,
read, remove.
- **`hack/install-ate.sh --otlp-endpoint`** — sends all control plane
telemetry to the meter. Each component reads the endpoint from the
`ate-otel-config` ConfigMap through `envFrom`, and `ate-controller`
copies the value to the ateom worker pods, thus one patch is sufficient
for all of them.
- **`benchmarking/workloads/deploy.sh --otlp-endpoint`** — the same for
actor containers, which substrate configures separately.
- **`benchmarking/monitoring.yaml`** — scrapes the meter and cAdvisor.
- **`benchmarking/observability.md`** — prerequisites and the scenario
ladder.
- **`docs/dev/best-practices/otel-collector.md`** — the volume model and
the size guidance, corrected for the `ParentBased` defaults from #711.
## Correction
The `glutton` ActorTemplate had no `env`. Substrate puts no OTLP
configuration in an actor container, thus the exporter used the SDK
default `localhost:4317`, no process listens on that address in the
sandbox, and substrate discarded each actor span with no error message.
The template now sets `OTEL_EXPORTER_OTLP_ENDPOINT` and
`OTEL_RESOURCE_ATTRIBUTES`.
`ActorTemplate.spec` is immutable, thus `deploy.sh --deploy` deletes the
templates before it applies them.
## How I verified this
On a GKE cluster with managed OTel. The sequence:
```bash
# 1. Start from a clean state.
./hack/install-ate.sh --delete-benchmarks
./hack/install-ate.sh --delete-ate-system
# 2. Install substrate from this branch.
./hack/install-ate.sh --deploy-ate-system
# 3. Install the meter and Prometheus.
kubectl apply -f benchmarking/telemetry/meter.yaml
kubectl apply -f benchmarking/monitoring.yaml
# 4. Send the control plane to the meter.
METER=http://telemetry-meter.benchmarking.svc.cluster.local:4317
./hack/install-ate.sh --deploy-ate-system --otlp-endpoint "${METER}"
# 5. Install the workloads and locust, with the actors on the meter also.
./benchmarking/deploy_locust.sh --deploy --worker-count 10 \
--otlp-endpoint "${METER}"
# 6. Read the volume.
kubectl port-forward -n benchmarking svc/prometheus 9090:9090
```
## Result
The idle floor, from one measurement on 10 workers:
| Service | datapoints/min |
|---|---|
| `ate-controller` | 234.8 |
| `ateapi` | 54.3 |
| `atelet` | 44.9 |
| `atenet-router` | 10.4 |
| `ateom-gvisor` | 5.2 |
| `glutton-actor` | 4.2 |
| **Total** | **353.7** |
The `glutton-actor` row is the correction above. Before it, actor
telemetry could not arrive at all.
Positive control for the same window:
`otelcol_receiver_accepted_metric_points` 10.5/s, and each
`otelcol_receiver_refused_*` counter at zero. Thus the zero values in
the table are true zeros, and not data that did not arrive.
Two values differ from the earlier manual measurement. `ate-controller`
is now the largest source, and `ateom-gvisor` is no longer zero. The new
imagecache counters from #831 are a probable cause. This difference is
the reason to keep results out of the repository.
### Traces
Trace volume follows the root sample decision, not the rate of each
component. `atelet` and `ateom-gvisor` sent zero spans in each run,
because they inherit the decision that `ateapi` makes and start no trace
of their own. At idle each component is near zero: a trace needs an
operation, but a metric pushes on a timer.
This run gives no span volume for a usual load. The actors crashed on
suspend, and the failed retries made an artificial rate of 13.7k
spans/s. That rate is not a measurement of substrate. It does show that
the meter accepted the spans with no refusals.
## Follow-up
A separate PR adds the ladder to `automation/tests.yaml` and writes the
results as artifacts of each run. The open items are at the end of
`observability.md`.
- [x] Tests pass
- [x] Appropriate changes to documentation are included in the PR
---------
Co-authored-by: Krisztian F <103492698+krisztianfekete@users.noreply.github.com>
### Why
The README Quickstart covers getting a local Substrate cluster running
with
the default gVisor runtime, but there's no documentation for the
micro-VM
runtime, where the friction can't be scripted away: it requires
`/dev/kvm`
(bare metal, nested virtualization, or Lima on macOS), and on Apple
Silicon
the Lima configuration and asset assembly are non-obvious. New
contributors
have to reverse-engineer the `hack/` scripts.
### What
Adds `docs/dev/microvm-local.md` — "Running the microVM runtime locally"
— a
focused guide covering only the microVM delta, based on notes from real
onboarding runs. General setup is deferred to the README Quickstart
(listed
as the guide's prerequisite) rather than duplicated:
- **Option A: Linux host with KVM** — verifying `/dev/kvm` and CPU virt
support, the rootless-Docker caveat for the KVM probe, cluster creation,
and the one-shot `run-microvm-demo-kind.sh` bring-up.
- **Option B: Apple Silicon macOS via Lima** — nested virtualization
with the
guest image pinned to Ubuntu 25.10 (until the kernel issue in the
default
image is fixed), `vzNAT` networking, and assembling the arm64 assets
inside
the Lima guest (`assemble.sh` requires a Linux host of the target arch).
- **Trying it out** — defers to the next steps the demo script prints
and
links the counter demo's micro-VM variant, instead of duplicating those
commands.
- **Troubleshooting** — symptom → root cause → fix entries actually hit
during onboarding (rootless-Docker KVM probe failures, the `aws` CLI
staging requirement until #804 lands, arm64 `virtiofsd` build deps, M1
lacking FEAT_NV2).
OTEL_EXPORTER_OTLP_ENDPOINT is written in 9 places: hardcoded inline in
4 base workload manifests, then re-patched in 5 spots across the kind
overlays. Changing the collector address means editing all 9 and knowing
which install path renders which.
Replace them with one checked-in ConfigMap per environment, consumed by
every component via envFrom:
manifests/ate-install/ate-otel-config.yaml (GKE)
manifests/ate-install/kind/ate-otel-config.yaml (kind, same name)
The GKE copy is listed in manifests/ate-install/base, so token-client,
agentgateway and agentgateway-token-client all inherit it; the kind
overlay lists its own copy of the same ConfigMap name. No overlay builds
on both, so the two never collide.
A kustomize-only fix does not work here. The GKE path applies the base
directory raw and hack/install-ate.sh's targeted redeploys apply single
files with no Kustomize, so the mechanism has to survive `kubectl apply
-f <one-file>` -- which rules out configMapGenerator (hash-suffixed
names) and replacements. envFrom on a stable name reaches every path.
deploy_ate_system applies the ConfigMap before the rendered bundle, the
same way it already applies the namespace. A container whose envFrom
target is missing does not start, and a raw directory apply is ordered
by filename, so ate-api-server.yaml and ate-controller.yaml would
otherwise be created first and sit in CreateContainerConfigError until
the ConfigMap caught up.
This also fixes a latent bug in the targeted redeploys.
deploy_ate_apiserver, deploy_atelet and deploy_atenet apply raw files
even in kind mode, silently reverting the endpoint to the GKE value.
They now apply the environment's ConfigMap via a new apply_otel_config
helper, which picks the file by ATE_INSTALL_KIND rather than applying
the base copy unconditionally -- applying the base one on kind would
break telemetry for every component at once.
atenet-router needs no special handling: since a98f85b1 its
--otlp-collector-address defaults to $OTEL_EXPORTER_OTLP_ENDPOINT, so
Envoy's own spans follow the ConfigMap along with everything else.
OTEL_TRACES_SAMPLER deliberately stays an inline kind patch on
ate-api-server rather than moving into the ConfigMap. The ConfigMap is
shared by every component via envFrom, so putting parentbased_always_on
there would pin the router and the rest of the control plane to 100% and
undo the per-component ratios from 15eecd02.
Note that a ConfigMap edit does not roll the consuming pods the way an
inline env change did, since the pod template is unchanged. Callers must
follow a change with `kubectl rollout restart`. This is documented in
both ConfigMaps, in docs/observability.md, and in the tracing best
practices, which previously told authors to hardcode the variable.
Fixes#745
> It's a good idea to open an issue first for discussion.
- [ ] Tests pass
- [ ] Appropriate changes to documentation are included in the PR
Today everything outside ateapi hardcodes `ParentBased(NeverSample)`,
and setting `OTEL_TRACES_SAMPLER` does nothing because an explicit
sampler silences the SDK's env handling. This makes troubleshooting
production issues impossible via traces.
This PR gives every component a sane default and makes the standard OTel
env vars work:
- Control plane (ateapi, atelet, ateom) defaults to
`parentbased_traceidratio` 0.1, the router (data plane root) to 0.01,
per the discussion on #584.
- `OTEL_TRACES_SAMPLER` / `OTEL_TRACES_SAMPLER_ARG` override any of this
without a rebuild. Invalid values keep the component default and log a
warning instead of inheriting the SDK's fall-open-to-100% behavior.
- Envoy's `RandomSampling` is derived from the router's resolved policy,
so the two root decisions cannot drift.
- kubectl-ate without `--trace` no longer installs a tracer provider at
all: the old `NeverSample` provider injected `sampled=0`, which pinned
every parent based sampler downstream and would have defeated the server
side ratios. `--trace` still forces a full end to end trace.
- ate-controller propagates the two env vars to the ateom worker pods it
creates, same as the metric export vars.
- kind pins ateapi to `parentbased_always_on`, so the local Jaeger flow
keeps showing every API call.
- The agentgateway integration follows what we have above. its
`randomSampling` changes from `true` to `0.01` to match the data plane
default, though as static config it does not follow
`OTEL_TRACES_SAMPLER` overrides, so the two need adjusting together.
Verified on a kind cluster e2e manually. It resolved samplers logged at
startup, a `--trace` resume produced one trace across ateapi, atelet,
and ateom, an unsampled CLI call got picked up server side, Envoy
continued a sampled traceparent while sampling 0 of 30 parentless
requests at the 1% default, and an invalid env value fell back to the
component default.
Gating who may use `--trace` stays a separate follow-up (and a
discussion), and the more sophisticaed tracing policies belongs in a
collector, not in substrate.
Fixes#584
cc. @git286
- [x] Tests pass
- [x] Appropriate changes to documentation are included in the PR
Substrate documents how to instrument a service but not how to stand up
the collector it exports to, so pointing it at your own collector meant
reading six manifests.
Covers both topologies: GKE Managed OpenTelemetry as the default on GKE,
with why its gateway Deployment suits Substrate's elastic actor
workload, and a DaemonSet manifest for operators who need per-node
isolation, full config control, or a non-GKE cluster.
Part of #563.
> It's a good idea to open an issue first for discussion.
- [ ] Tests pass
- [x] Appropriate changes to documentation are included in the PR
- `docs/observability.md`: log label keys were wrong. The code emits
`ate.dev/actor_id` and `ate.dev/actor_template_name`; the doc had
`actor_id` and `actor_template`. The Cloud Logging queries would return
no results.
- `docs/dev/best-practices/tracing.md`: container ports entry used
Docker Compose syntax (`"443:443"`). Changed to `containerPort: 443`.
- `docs/dev/valkey-direct-access.md`: the `kubectl exec` command had a
line break inside `--cert`, making it fail on copy-paste.
- `code-of-conduct.md`: linked to Contributor Covenant v1.4. Updated to
v2.1.
The tracing best practices doc points to
cmd/ateapi/ateapi.go:initTracing() as the example, but that
file/function does not exist. Servers now call
internal/serverboot.InitTracing(), used in cmd/ateapi/main.go.
Verified internal/serverboot.InitTracing exists and is called from
cmd/ateapi/main.go:80.
Assemble + stage the micro-VM runtime assets, an ateom-base image (debian-slim +
e2fsprogs for mkfs.ext4), and run-microvm-demo.sh to build + deploy the
counter-microvm demo end to end (overriding the worker base via KO_CONFIG_PATH so
no committed file is edited). Document the micro-VM sandbox class.
Document the rationale behind each directory and guidelines on how to
decide where to put new code in a dedicated markdown file.
Update AGENTS.md with a summary and a reference to the new markdown
file.
This commit was automatically generated by the tools/reorg tool.
```
go run ./tools/reorg
go generate ./...
bash benchmarking/locust/generate_protos.sh
```
This is the initial release of the Agent Substrate.
Agent substrate is a system built on top of Kubernetes which manages agent-like
workloads to achieve higher scale and efficiency than Kubernetes alone can
offer, with lower latency. It builds on top of Kubernetes features like
Pods and Pod autoscaling, but takes the Kubernetes control-plane out of the
critical path to achieve lower latency.
It can run on any Kubernetes cluster and does not inhibit “regular” use of
Kubernetes in any way. Kubernetes provides the infrastructure provisioning and
management for all types of workloads, while Agent Substrate provides
agent-specific scheduling and control.
At its core, Agent Substrate maps a larger set of “actors” (applications such
as agents) onto a smaller set of ready “workers” (Kubernetes Pods), relying on
the fact that agent-like applications tend to be idle most of the time to
achieve heavy multiplexing. It provides functionality to manage an actor’s
lifecycle (e.g. create/destroy, suspend/resume), to assign actors to workers in real
time, and to route incoming traffic to them.
Agent Substrate is intended to be a low-opinion system. The workloads it
manages don't have to be literal AI agents, but those are the best example of
the kind of applications it is designed for. It is not an SDK for building
agents, but rather a system for running them at scale.
Agent Substrate is currently in VERY early development. It is not ready for
production use, and the APIs are almost guaranteed to change. We are not
making any guarantees about backward compatibility at this stage, and
everything in this project may be changed.
Co-authored-by: Alex Bulankou <alexbu@google.com>
Co-authored-by: Benjamin Elder <bentheelder@google.com>
Co-authored-by: Bowei Du <bowei@google.com>
Co-authored-by: Dmitry Berkovich <dberkov@google.com>
Co-authored-by: Fabricio Voznika <fvoznika@google.com>
Co-authored-by: Francisco Cabrera <fclieutier@google.com>
Co-authored-by: Haven Xia <haoyuxia@google.com>
Co-authored-by: Julian Gutierrez Oschmann <juliangut@google.com>
Co-authored-by: Kevin Steuer <ksteuer@google.com>
Co-authored-by: Max Smythe <smythe@google.com>
Co-authored-by: Maya Wang <mymaya@google.com>
Co-authored-by: Michael Taufen <mtaufen@google.com>
Co-authored-by: Shruti Nair <shrutinair@google.com>
Co-authored-by: Taahir Ahmed <taahm@google.com>
Co-authored-by: Tim Hockin <thockin@google.com>
Co-authored-by: Zoe Zhao <zoezhao@google.com>
Huy Pham