mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-10-02 03:24:42 +08:00
Da Huang
cdd86eba1c
The atelet Deployment is always named "atelet-<suffix>" (see cmd/ate-setup/internal/steps/version.go), never plain "atelet", so every dashboard widget filtering on top_level_controller_name="atelet" renders empty even when the underlying metrics exist. The Snapshot Size & QPS dashboard was entirely blank on a fresh install because of this; the gRPC and E2E-latency dashboards had the same dead filter on their atelet panels. Match the deployment name prefix with a regex instead. Verified against a live install (GMP PromQL): the equality filter returns 0 series while =~"atelet-.*" returns all 12 atelet.snapshot.size series. Fixes #1453 > It's a good idea to open an issue first for discussion. - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR
Cloud Monitoring dashboards
Google Cloud Monitoring dashboard definitions for ATE. They turn the raw
prometheus.googleapis.com/... metrics that ATE emits into readable
per-method / per-stage latency / throughput / error views.
| File | Shows |
|---|---|
ate-grpc-dashboard.json |
ateapi & atelet gRPC latency (p50/p95/p99), request rate, and error rate, by method |
ate-e2e-latency-dashboard.json |
"Substrate Routing & E2E Latency". Substrate routing P50/P95/P99, routing P99 by stage (routing / ateapi ResumeActor / atelet Restore), routing P99 by ActorTemplate, routing QPS by status, plus the E2E full round-trip from Envoy (ms — includes actor compute + response, so it's context, not our overhead): P99 and QPS by response class. Needs the atenet-router-envoy PodMonitoring for the round-trip lines. |
ate-snapshot-dashboard.json |
Substrate snapshot image size and throughput ("Substrate Snapshot Size & QPS"): snapshot (pages) image size P99 by ActorTemplate, P50/P95/P99 by image kind, and snapshot QPS. Checkpoint/Restore latency is not here — it's the atelet gRPC Checkpoint/Restore methods in ate-grpc-dashboard.json. |
Applying
Dashboards are created/updated (idempotently) by setup:
go run ./tools/setup-gcp create dashboards # also part of: bootstrap
Or apply any single file by hand:
gcloud monitoring dashboards create --config-from-file=tools/setup-gcp/dashboards/<file>.json