Fixes#564 (Part 2)
- [x] Tests pass
- [x] Appropriate changes to documentation are included in the PR
## Description
This PR implements the second part of #564 by adding telemetry counter
instrumentation for `ate.actor.crashes` in `ateapi` and establishing
bounded label validation for operations and crash failure causes.
## Key Changes:
- Updated `maybeCrashActor` and `crashActor` to reference
`ateattr.OperationName*` constants and normalize `opName`.
- Passed explicit failure reasons (`ReasonCorruptedAssignment`,
`ReasonWorkerPodGone`, `ReasonWorkerReassigned`) across
`workflow_resume.go`, `workflow_suspend.go`, and `workflow_pause.go`.
- Instrumented `WorkerPoolSyncer.releaseActorOnDeadWorker` in
`syncer.go` to record `recordActorCrash` when background pod deletion
crashes an actor.
- Instrumented `FinalizePausedStep` in `workflow_pause.go` to record
`recordActorCrash` when `nodeName` is missing during pause finalization.
- Added `ate_actor_crashes` to `PlatformMetricPrefixes` in
`collector_metrics.go`.
- Added pre-creation resource cleanup in `metrics_test.go` to prevent
`AlreadyExists` errors on test reruns.
- Updated `metrics_test.go` to trigger an actor crash via `UpdateActor`
and `ResumeActor`, asserting `ate_actor_crashes` emission and label
presence in OTel collector scrape outputs.
## Testing
* `go test -buildvcs=false ./cmd/ateapi/internal/controlapi/...`
* `go test -buildvcs=false ./cmd/atenet/internal/router/...`
* `make test`
## E2E Test
* `./hack/create-kind-cluster.sh`
* `./hack/install-ate-kind.sh --deploy-ate-system --deploy-demo-counter`
* `./hack/run-e2e.sh ./internal/e2e/suites/metrics/...`
* Add `args` field in addition to `command`.
* Implement logic in atelet to resolve both argv from the actor template
fields and the OCI image config (`entrypoint` and `cmd` fields).
* Update demos to not hardcode command as now we honor the image
default, which makes it not necessary.
Fixes#456
Related to #292 and #119
Changes:
1. ResumeActor:
1. During `ResumeActor` we'll put the actor in CRASHED state if the
Resume failed due to any errors related to the snapshot itself, which
includes:
1. Missing or corrupted snapshot when downloading from GCS
2. Runsc command fails.
2. All other types of errors will not put the Actor in CRASHED state.
2. SuspendActor or PauseActor
1. During `SuspendActor` or `PauseActor` we'll put the actor in CRASHED
state if we cannot produce a valid snapshot, which includes:
1. cannot connect to Ateom after retries,
2. runsc command fails.
3. cannot upload the snapshot to GCS (during suspend) ; or cannot save
snapshot locally (during pause).
2. All other types of errors will not put the Actor in CRASHED state.
Angela