mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-11-10 10:13:50 +08:00
Angela
1f8f842a64
Fixes #564 (Part 2) - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR ## Description This PR implements the second part of #564 by adding telemetry counter instrumentation for `ate.actor.crashes` in `ateapi` and establishing bounded label validation for operations and crash failure causes. ## Key Changes: - Updated `maybeCrashActor` and `crashActor` to reference `ateattr.OperationName*` constants and normalize `opName`. - Passed explicit failure reasons (`ReasonCorruptedAssignment`, `ReasonWorkerPodGone`, `ReasonWorkerReassigned`) across `workflow_resume.go`, `workflow_suspend.go`, and `workflow_pause.go`. - Instrumented `WorkerPoolSyncer.releaseActorOnDeadWorker` in `syncer.go` to record `recordActorCrash` when background pod deletion crashes an actor. - Instrumented `FinalizePausedStep` in `workflow_pause.go` to record `recordActorCrash` when `nodeName` is missing during pause finalization. - Added `ate_actor_crashes` to `PlatformMetricPrefixes` in `collector_metrics.go`. - Added pre-creation resource cleanup in `metrics_test.go` to prevent `AlreadyExists` errors on test reruns. - Updated `metrics_test.go` to trigger an actor crash via `UpdateActor` and `ResumeActor`, asserting `ate_actor_crashes` emission and label presence in OTel collector scrape outputs. ## Testing * `go test -buildvcs=false ./cmd/ateapi/internal/controlapi/...` * `go test -buildvcs=false ./cmd/atenet/internal/router/...` * `make test` ## E2E Test * `./hack/create-kind-cluster.sh` * `./hack/install-ate-kind.sh --deploy-ate-system --deploy-demo-counter` * `./hack/run-e2e.sh ./internal/e2e/suites/metrics/...`