Haven Xia a325d4118b Add Substrate routing-latency metric and Cloud Monitoring dashboard (#157) ...

This PR introduces a new OpenTelemetry histogram
`"atenet.router.route.duration"` from `atenet-router` over the existing
OTLP path. It measures substrate overhead -- from envoy receiving a
request to envoy sending the request to the the resolved worker
endpoint, excluding actor compute and the response. This is the
`Substrate E2E latency` that under our control.

Add a well-defined dashboard for E2E metrics that contains 6 charts.
- Substrate routing latency — P50/P95/P99
- Substrate routing latency P99 - by stages
- Substrate routing latency P99 — by ActorTemplate
- Substrate routing QPS - by Status
- Substrate E2E (full round-trip) Latency — P50/P95/P99
- Substrate E2E (full round-trip) QPS — by Response Code

<img width="1601" height="894" alt="image"
src="https://github.com/user-attachments/assets/be6fc572-dfd8-4ef0-a0a9-a77f85f2ecf8"
/>


Fixes b/508613998

> It's a good idea to open an issue first for discussion.

- [ ] Tests pass
- [x] Appropriate changes to documentation are included in the PR

2026-06-05 10:42:06 -07:00

..

setup-gcp

Add Substrate routing-latency metric and Cloud Monitoring dashboard (#157)

2026-06-05 10:42:06 -07:00