trending/substrate
1
0
Fork 0
mirror of https://github.com/agent-substrate/substrate.git synced 2026-10-02 03:24:42 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add request-parking demo

Browse Source 
An oversubscribed WorkerPool (2 workers, several actors) that exercises the router parking path: requests to a saturated pool park and retry instead of failing fast, and are served once capacity frees up. Includes load.sh and --deploy-demo-parking / --delete-demo-parking wiring in hack/install-ate.sh.
This commit is contained in:
Omer Yahud
2026-07-28 23:55:51 -07:00
committed by Bowei Du
parent 542679ce57
commit 8f64f67f82
6 changed files with 429 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
demos/parking/README.md
+167
View File
@@ -0,0 +1,167 @@
# Request Parking Demo
This demo shows the **request parking** feature of the `atenet` router: when the
`WorkerPool` is momentarily saturated, the router *holds* (parks) an inbound
request and retries the resume until a worker frees up — instead of failing fast
with a `503`.
The setup is deliberately **oversubscribed**: a 2-worker pool with several
actors. The workload is the same `counter` binary used by the counter demo; its
reply includes the worker pod IP, so you can see which worker served a request.
See [docs/request-parking.md](../../docs/request-parking.md) for the design.
## Prerequisites
- A k8s cluster with Agent Substrate installed (`./hack/install-ate.sh --deploy-ate-system`).
- `ko` installed for building images.
- A GCS bucket for storing snapshots (configured via `BUCKET_NAME` env var).
## How to Run on Agent Substrate
### 1. Build and Deploy
> [!NOTE]
> Do not manually edit `demos/parking/parking.yaml.tmpl`. The installation script
> automatically injects your `${BUCKET_NAME}` environment variable during deployment.
```bash
./hack/install-ate.sh --deploy-demo-parking
```
This command will:
- Build the `counter` workload image using `ko`.
- Create the `ate-demo-parking` namespace.
- Create a **2-replica** `WorkerPool` (`parking`) and the `parking` `ActorTemplate`.
- Wait until the pool is rolled out and the template is `Ready`.
### 2. Create more actors than workers
Actors live in an **atespace**, and their DNS names embed it
(`<id>.<atespace>.actors.resources.substrate.ate.dev`), so create one first:
```bash
# Install the CLI as a kubectl plugin if not already installed
go install ./cmd/kubectl-ate
kubectl ate create atespace parking
# 4 actors share a 2-worker pool -> oversubscribed.
for id in p1 p2 p3 p4; do
kubectl ate create actor "$id" --atespace parking --template ate-demo-parking/parking
done
```
### 3. Port-forward the atenet router
```bash
kubectl port-forward -n ate-system svc/atenet-router 8000:80
```
## How to Use
Parking is **on by default** (`--parking-enabled=true`, `--parking-max-wait=30s`,
`--parking-max-parked=2048`), so the cluster you just deployed already parks.
### A. Watch a 503 become a served request
Fill both workers by requesting two actors, leaving them `RUNNING`:
```bash
curl -s -H "Host: p1.parking.actors.resources.substrate.ate.dev" http://localhost:8000
curl -s -H "Host: p2.parking.actors.resources.substrate.ate.dev" http://localhost:8000
kubectl ate get workers # both workers are now bound to p1 and p2
kubectl ate get actors # p1,p2 RUNNING; p3,p4 SUSPENDED
```
Now request **p3** with timing. The pool is full, so this request **parks** —
the `curl` hangs while the router retries the resume:
```bash
curl -s -w '\n-> HTTP %{http_code} in %{time_total}s\n' \
-H "Host: p3.parking.actors.resources.substrate.ate.dev" http://localhost:8000
```
While that is hanging, in a **second terminal** free a worker by suspending p1
(within the 30s park budget):
```bash
kubectl ate suspend actor p1 --atespace parking
```
Back in the first terminal, the parked request now completes with **`HTTP 200`**,
and `time_total` shows how long it waited for the worker. With parking disabled,
that same request would have returned **`503`** immediately (see section D).
### B. See it under load
`load.sh` drives one concurrent request→suspend loop per actor. Because there are
more actors than workers, the pool stays saturated; the suspend at the end of each
loop frees a worker for a competitor (standing in for an actor going idle). The
tally shows parking absorbing the contention:
```bash
./demos/parking/load.sh # 30s, actors p1 p2 p3 p4
# ==> results
# total requests : 142
# 200 OK : 142
# 503 unavailable: 0
# 200 latency : avg 0.43s, slowest 6.12s <- parked requests sit here
# => 0 failures under saturation: parking absorbed the contention.
```
### C. Observe parking state
The router's `/statusz` page has a **Request Parking** card. Port-forward the
status port and read it (run this while `load.sh` is generating load to see a
non-zero `active`):
```bash
kubectl -n ate-system port-forward deployment/atenet-router 4040:4040
curl -s 'http://localhost:4040/statusz?format=json' | jq .parking
# { "enabled": true, "active": 3, "max_parked": 2048, "max_wait": "30s" }
```
The parking metrics are also exported on the router's metrics endpoint
(`--metrics-listen-addr`, container port `9090`): `atenet.router.parking.active`,
`atenet.router.parking.wait.duration` (labeled by `outcome`), and
`atenet.router.parking.rejected`.
### D. Compare with parking disabled
Turn parking off to see the old fail-fast behavior. Add the flag to the router
container's args:
```bash
kubectl -n ate-system patch deployment atenet-router --type=json \
-p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--parking-enabled=false"}]'
kubectl -n ate-system rollout status deployment/atenet-router
```
Re-run the load test — now transient saturation surfaces as `503`s:
```bash
./demos/parking/load.sh
# 503 unavailable: 37
# => 37 requests were shed with 503 (parking off, ...).
```
Re-enable parking by removing that flag again:
```bash
kubectl -n ate-system rollout undo deployment/atenet-router
```
> [!TIP]
> You can tune parking instead of disabling it: add `--parking-max-wait=10s` or
> `--parking-max-parked=512` to the same args list.
## How to Uninstall
Remove the demo — this deletes the demo's actors (suspending running ones
first) and then the template, pool, and namespace:
```bash
./hack/install-ate.sh --delete-demo-parking
```
demos/parking/load.sh
Executable
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# load.sh -- oversubscription load generator for the request-parking demo.
#
# It drives one concurrent request->suspend loop per actor against a small
# WorkerPool. Because there are more actors than workers, the pool is constantly
# saturated; the suspend at the end of each loop is what frees a worker for a
# competitor (standing in for an actor going idle, since auto-suspend-on-idle
# isn't implemented yet). The result tally shows how parking turns transient
# saturation into (slightly slower) 200s instead of 503s.
#
# Compare two runs:
# * parking ON (default router config) -> ~all 200, some elevated latency
# * parking OFF (--parking-enabled=false) -> a burst of 503s
# See README.md for how to flip the router flag.
#
# Usage:
# ./load.sh [-d duration_secs] [-r router_url] [-a atespace] [actor_id ...]
#
# Examples:
# ./load.sh # 30s, actors p1 p2 p3 p4, http://localhost:8000
# ./load.sh -d 60 p1 p2 p3 p4 p5 p6
#
# Prerequisites:
# * `kubectl ate` plugin installed (go install ./cmd/kubectl-ate)
# * router port-forwarded: kubectl port-forward -n ate-system svc/atenet-router 8000:80
set -uo pipefail
DURATION=30
ROUTER="http://localhost:8000"
TEMPLATE="ate-demo-parking/parking"
ATESPACE="parking"
SUFFIX="actors.resources.substrate.ate.dev"
usage() {
cat <<'EOF'
load.sh -- oversubscription load generator for the request-parking demo.
Usage: ./load.sh [-d duration_secs] [-r router_url] [-a atespace] [actor_id ...]
-d load duration in seconds (default 30)
-r router base URL (default http://localhost:8000)
-a atespace for the actors (default parking)
args actor IDs (default: p1 p2 p3 p4)
Prereqs: `kubectl ate` installed and the router port-forwarded
(kubectl port-forward -n ate-system svc/atenet-router 8000:80).
EOF
}
while getopts ":d:r:a:h" opt; do
case "${opt}" in
d) DURATION="${OPTARG}" ;;
r) ROUTER="${OPTARG}" ;;
a) ATESPACE="${OPTARG}" ;;
h) usage; exit 0 ;;
*) echo "unknown option -${OPTARG}; use -h for help" >&2; exit 2 ;;
esac
done
shift $((OPTIND - 1))
ACTORS=("$@")
if [[ ${#ACTORS[@]} -eq 0 ]]; then
ACTORS=(p1 p2 p3 p4)
fi
TMP="$(mktemp -d)"
pids=()
cleanup() { rm -rf "${TMP}"; }
abort() { [[ ${#pids[@]} -gt 0 ]] && kill "${pids[@]}" 2>/dev/null; cleanup; exit 130; }
trap cleanup EXIT
trap abort INT TERM
echo "==> request-parking load test"
echo " router: ${ROUTER}"
echo " duration: ${DURATION}s"
echo " actors: ${ACTORS[*]} (${#ACTORS[@]}) vs a 2-worker pool -> oversubscribed"
echo
# Preflight: make sure the atespace and actors exist (idempotent; ignore
# "already exists").
echo "==> ensuring atespace ${ATESPACE} and actors exist (template ${TEMPLATE})"
kubectl ate create atespace "${ATESPACE}" >/dev/null 2>&1 || true
for a in "${ACTORS[@]}"; do
kubectl ate create actor "${a}" --atespace "${ATESPACE}" --template "${TEMPLATE}" >/dev/null 2>&1 || true
done
# One worker per actor: hammer it with request->suspend until the deadline.
worker() {
local actor="$1" host="$1.${ATESPACE}.${SUFFIX}" log="${TMP}/$1.log"
local deadline=$(( $(date +%s) + DURATION ))
while [[ $(date +%s) -lt ${deadline} ]]; do
# %{http_code} lets us tally outcomes; %{time_total} reveals parking waits.
curl -s -o /dev/null -w '%{http_code} %{time_total}\n' \
-H "Host: ${host}" "${ROUTER}" >>"${log}" 2>/dev/null
# Free the worker so a parked competitor can proceed (simulate going idle).
kubectl ate suspend actor "${actor}" --atespace "${ATESPACE}" >/dev/null 2>&1 || true
done
}
echo "==> generating load for ${DURATION}s ..."
for a in "${ACTORS[@]}"; do
worker "${a}" &
pids+=($!)
done
wait
# ---- aggregate -------------------------------------------------------------
echo
echo "==> results"
cat "${TMP}"/*.log 2>/dev/null >"${TMP}/all" || true
total=$(wc -l <"${TMP}/all" | tr -d ' ')
if [[ "${total}" -eq 0 ]]; then
echo " no responses recorded -- is the router port-forwarded at ${ROUTER}?"
exit 1
fi
ok=$(awk '$1==200' "${TMP}/all" | wc -l | tr -d ' ')
busy=$(awk '$1==503' "${TMP}/all" | wc -l | tr -d ' ')
other=$(awk '$1!=200 && $1!=503' "${TMP}/all" | wc -l | tr -d ' ')
slowest_ok=$(awk '$1==200{print $2}' "${TMP}/all" | sort -rn | head -1)
avg_ok=$(awk '$1==200{s+=$2;n++} END{if(n)printf "%.3f", s/n; else print "n/a"}' "${TMP}/all")
printf " total requests : %s\n" "${total}"
printf " 200 OK : %s\n" "${ok}"
printf " 503 unavailable: %s\n" "${busy}"
printf " other : %s\n" "${other}"
printf " 200 latency : avg %ss, slowest %ss <- parked requests sit here\n" "${avg_ok}" "${slowest_ok:-n/a}"
echo
if [[ "${busy}" -eq 0 ]]; then
echo " => 0 failures under saturation: parking absorbed the contention."
echo " Re-run with the router started --parking-enabled=false to see 503s."
else
echo " => ${busy} requests were shed with 503 (parking off, or lot full / budget exceeded)."
fi
demos/parking/parking.yaml.tmpl
+62
View File
@@ -0,0 +1,62 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This demo exercises request parking in the atenet router. The WorkerPool is
# deliberately small (2 workers) so that creating and requesting more than two
# actors at once oversubscribes the pool. When AssignWorker can't find a free
# worker it returns FailedPrecondition ("no free workers available"); with
# parking enabled (the default) the router holds such requests and retries the
# resume until a worker frees up (e.g. another actor suspends), instead of
# failing fast with 503. The workload is the same counter binary used by the
# counter demo -- its reply includes the worker pod IP, so you can see which
# worker served a (possibly parked) request.
apiVersion: v1
kind: Namespace
metadata:
name: ate-demo-parking
---
apiVersion: ate.dev/v1alpha1
kind: WorkerPool
metadata:
name: parking
namespace: ate-demo-parking
labels:
workload: parking
spec:
# Intentionally small: 2 workers, so 3+ concurrently-active actors saturate
# the pool and exercise the parking path.
replicas: 2
ateomImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor
---
apiVersion: ate.dev/v1alpha1
kind: ActorTemplate
metadata:
name: parking
namespace: ate-demo-parking
spec:
pauseImage: "registry.k8s.io/pause:3.10.2@sha256:f548e0e8e3dc1896ca956272154dde3314e8cc4fde0a57577ee9fa1c63f5baf4"
containers:
- name: counter
image: ko://github.com/agent-substrate/substrate/demos/counter
command: ["/ko-app/counter"]
workerSelector:
matchLabels:
workload: parking
snapshotsConfig:
location: gs://${BUCKET_NAME}/ate-demo-parking/
docs/request-parking.md
-18
View File
@@ -7,10 +7,6 @@ whose target actor cannot be served *yet* because of transient worker-pool
saturation, retrying the resume until the actor becomes routable or a bounded
wait elapses — instead of immediately returning `503` to the client.
This realizes the behavior described in the architecture doc ("the Gateway
pauses the request and asks the Control Plane for the actor's location") and is
the router-side complement to control-plane worker preemption.
## Motivation
When a request arrives for a suspended actor, the router resumes it before
@@ -98,17 +94,3 @@ within the historical `15s` budget.
**Status page** (`/statusz`): a "Request Parking" card shows whether parking is
enabled, the current vs. maximum parked count, and the max wait.
## Implementation
All in `cmd/atenet/internal/app/router`:
- `parking.go` — `parkingConfig` and `parkingLot`, a bounded, non-blocking,
nil-safe admission gate (`enter`/`release`, atomic slot accounting).
- `resumer.go` — `ActorResumer` gains a `retryable` predicate and a configurable
retry budget; on budget exhaustion it returns the underlying capacity error
rather than a generic timeout so the HTTP boundary maps it faithfully.
- `extproc.go` — `handleRequestHeaders` admits each request to the lot around the
resume call.
- `metrics.go` — the three parking instruments (nil-safe bundle).
- `errors.go` — `parkingFullErr` (503).
- `router.go` — flags and wiring.
hack/install-ate.sh
+1
View File
@@ -43,6 +43,7 @@ source "${ROOT}"/hack/install-demo-counter.sh
source "${ROOT}"/hack/install-demo-sandbox.sh
source "${ROOT}"/hack/install-demo-claude-code-multiplex.sh
source "${ROOT}"/hack/install-demo-multi-template.sh
source "${ROOT}"/hack/install-demo-parking.sh
# ANSI color codes for prettier output
COLOR_CYAN='\033[1;36m'
hack/install-demo-parking.sh
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This is sourced as part of install-ate.sh. Do not run directly.
ATE_DEMOS+=(demo-parking) # register demo-parking
demo-parking_cmdline() {
case "${1}" in
--deploy-demo-parking) demo-parking_deploy ;;
--delete-demo-parking) demo-parking_delete ;;
*)
return 1
;;
esac
return 0
}
demo-parking_deploy() {
log_step "demo-parking_deploy"
ensure_crds
sed "s|\${BUCKET_NAME}|${BUCKET_NAME}|g" demos/parking/parking.yaml.tmpl \
| run_ko apply -f -
# Wait for the demo to be fully ready before returning: the small WorkerPool
# must be rolled out and the ActorTemplate's golden snapshot built.
log_step "Waiting for parking demo to be ready..."
run_kubectl rollout status deployment/parking -n ate-demo-parking --timeout=300s
run_kubectl wait --for=condition=Ready actortemplate/parking -n ate-demo-parking --timeout=300s
}
demo-parking_delete() {
log_step "demo-parking_delete"
delete_demo_actors ate-demo-parking parking
sed "s|\${BUCKET_NAME}|${BUCKET_NAME}|g" demos/parking/parking.yaml.tmpl \
| run_kubectl delete --ignore-not-found -f -
}