trending/substrate

Fork 0

mirror of https://github.com/agent-substrate/substrate.git synced 2026-11-10 10:13:50 +08:00

Files

T

af3c65088e Initial commit of Agent Substrate

This is the initial release of the Agent Substrate.

Agent substrate is a system built on top of Kubernetes which manages agent-like
workloads to achieve higher scale and efficiency than Kubernetes alone can
offer, with lower latency.  It builds on top of Kubernetes features like
Pods and Pod autoscaling, but takes the Kubernetes control-plane out of the
critical path to achieve lower latency.

It can run on any Kubernetes cluster and does not inhibit “regular” use of
Kubernetes in any way. Kubernetes provides the infrastructure provisioning and
management for all types of workloads, while Agent Substrate provides
agent-specific scheduling and control.

At its core, Agent Substrate maps a larger set of “actors” (applications such
as agents) onto a smaller set of ready “workers” (Kubernetes Pods), relying on
the fact that agent-like applications tend to be idle most of the time to
achieve heavy multiplexing.  It provides functionality to manage an actor’s
lifecycle (e.g. create/destroy, suspend/resume), to assign actors to workers in real
time, and to route incoming traffic to them.

Agent Substrate is intended to be a low-opinion system.  The workloads it
manages don't have to be literal AI agents, but those are the best example of
the kind of applications it is designed for.  It is not an SDK for building
agents, but rather a system for running them at scale.

Agent Substrate is currently in VERY early development.  It is not ready for
production use, and the APIs are almost guaranteed to change.  We are not
making any guarantees about backward compatibility at this stage, and
everything in this project may be changed.

Co-authored-by: Alex Bulankou <alexbu@google.com>
Co-authored-by: Benjamin Elder <bentheelder@google.com>
Co-authored-by: Bowei Du <bowei@google.com>
Co-authored-by: Dmitry Berkovich <dberkov@google.com>
Co-authored-by: Fabricio Voznika <fvoznika@google.com>
Co-authored-by: Francisco Cabrera <fclieutier@google.com>
Co-authored-by: Haven Xia <haoyuxia@google.com>
Co-authored-by: Julian Gutierrez Oschmann <juliangut@google.com>
Co-authored-by: Kevin Steuer <ksteuer@google.com>
Co-authored-by: Max Smythe <smythe@google.com>
Co-authored-by: Maya Wang <mymaya@google.com>
Co-authored-by: Michael Taufen <mtaufen@google.com>
Co-authored-by: Shruti Nair <shrutinair@google.com>
Co-authored-by: Taahir Ahmed <taahm@google.com>
Co-authored-by: Tim Hockin <thockin@google.com>
Co-authored-by: Zoe Zhao <zoezhao@google.com>

2026-05-19 16:57:14 -07:00

3.2 KiB

Raw Blame History

Substrate Benchmarking

This is the nascent suite for benchmarking Substrate's performance at scale. It has two components:

A metrics scraping stack that leverages Prometheus and Grafana to collect and visualize metrics (currently only from the locust workers)
A locust test suite that generates load against Substrate

Rebuilding gRPC Python clients

Make sure you have a virtual environment created (python3 -m venv venv) and activated (source venv/bin/activate).

Install project requirements: pip install -r requirements.txt

Then run generate_protos.sh to generate the Python proto clients.

Install monitoring

This must be done first to create the monitoring namespace

kubectl apply -f monitoring.yaml

Deploy Workloads and Demos

Before running any benchmarks, the corresponding workloads or actor templates must be deployed to the cluster. If you run the benchmarks without deploying their dependent templates first, the Locust tasks will fail to create or locate the required actors.

Important

Caveat: Workloads/demos must be deployed before the associated benchmarks can be run. Make sure you have sourced the environment configuration file (e.g., source .ate-dev-env.sh) in your active terminal to ensure all environment variables (like BUCKET_NAME and PROJECT_ID) are properly defined.

1. Deploying Scale Workloads (For `ate-api` or `all` benchmarks)

The scale benchmarking tasks (such as kernelmem, sleep, and usermem) require deploying the scale workload templates.

Deploy Workloads:

./benchmarking/workloads/deploy.sh --deploy

Remove Workloads:

./benchmarking/workloads/deploy.sh --delete

2. Deploying the Counter Demo (For `counter` benchmark)

The counter load testing scenario uses the stateful counter actor template.

Deploy Counter Demo:

./hack/install-ate.sh --deploy-demo-counter

Remove Counter Demo:

./hack/install-ate.sh --delete-demo-counter

Build/Deploy locust

First, make sure you have sourced .ate-dev-env.sh to set up the necessary environment variables:

Run build_and_push.sh to build and push the image
Run deploy_locust.sh to deploy the locust worker
- You can control what load test is deployed by setting the LOAD_TYPE environment variable:
  - LOAD_TYPE=all (Default) to deploy all tests with the Locust web UI enabled.
  - LOAD_TYPE=ate-api to deploy the standalone ate-api load test (headless).
  - LOAD_TYPE=counter to deploy the standalone counter demo load test (headless).

Viewing the results

Grafana Dashboards

Run kubectl port-forward svc/grafana -n monitoring 3000:3000
Visit http://localhost:3000 in your browser.

Locust Web UI (for `LOAD_TYPE=all`)

Run kubectl port-forward svc/locust-all -n monitoring 8089:8089
Visit http://localhost:8089 in your browser to configure and start the load test.

Future work

Running discrete load tests and storing the results in a database.

This will require setting up a Service Account to push results to a GCS bucket and a controller script to strictly orchestrate the locust invocation(s), make observations against Prometheus, then push the results.

3.2 KiB Raw Blame History