mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-11-10 10:13:50 +08:00
60073ecd577cfdfa85d2904af492e5aa8bbb00d5
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jet Chiang
|
60073ecd57 |
Replace ateredis with atepg (#940)
## Summary A follow up to #640 where we introduced PostgreSQL as an alternative storage backend, selected conditionally in ateapi. - Deleted ateredis, its tests, and its dependencies - Removed Redis backend selection and configuration so ateapi always connects to Postgres - Replaced Valkey resources with Postgres in the standard and Kind deployment paths and simplified install script - Replaced miniredis fixtures with isolated Postgres testcontainers and added centralized helpers for seeding resources - Renamed Redis-specific debug flush command to backend-neutral `debug-clear-store` in CLI - Updated comments and docs where applicable ## Benchmarking Extensive benchmarking have been performed to evaluate Redis vs Postgres, and results can be found in these two documents: - https://docs.google.com/document/d/10K0wB6aTeFkJCL4HN3NbLJCdFGoLYdhIcqkFnqFHkKc/edit?usp=sharing - https://docs.google.com/document/d/12-ko_BFHcBo_nJkx9f4B7zMbiiWKC2saGhMhZG3aQ-s/edit?usp=sharing --------- Signed-off-by: Jet Chiang <pokyuen.jetchiang-ext@solo.io> |
||
|
Tim Hockin
|
87361ea519 |
Rename controlapi.Service -> RpcService (#1148)
Having been reading this code for a while, and in light of changes we want to make, I think something like this will help clarify. I am not particularly attached to the name or style `RpcService` - it's not usual Go convention but it is protobuf style. If anyone feels strongly it could be `RPCService` or `ControlService` or something else. @juli4n @laoj2 |
||
|
Luiz Oliveira
|
31a2e3850a |
Simplify Update methods to do a whole object replace (#1108)
* Removed field_mask from the API * Added a new protoupdate package to handle replacing mutable fields. This makes sure that unknown fields in the server are not dropped by an update from a stale/old client. #1011 - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR |
||
|
Jet Chiang
|
f1e5d8e4e3 |
Harden PostgreSQL concurrency, integrity, and error handling (#1023)
Part of #731 and follow up to #640. This PR makes improvements to the Postgres store by addressing review comments on #940 and optimizing Postgres updates as suggested in #988. - Replaces Postgres row-locking updates for actors, templates, and snapshot tags with bounded optimistic concurrency control using UID/version CAS checks. - Strengthens relational integrity with missing FKs and indexes, and replace snapshot-tag creation pre-check with FK - Removes unnecessary UID uniqueness constraints - Standardize pagination and validation across stores, map malformed tokens and invalid sizes to appropriate gRPC errors - Improves actor resume race handling when workers disappear or concurrent assignments exhaust their retry budget - Improves reliability with startup retries, expired-lease cleanup, worker-watch reconnection after malformed msg - Expand Postgres, Redis, Contract, and Control API tests to cover new constraints, error semantics, concurrency --------- Signed-off-by: Jet Chiang <pokyuen.jetchiang-ext@solo.io> |
||
|
Benjamin Elder
|
ecca2cac00 |
ateapi: move the ActorTemplate functional test into functionaltest (fix test build) (#1141)
The test calls `setupTest`, `namespaceForTest` and `assertGrpcError`, which are defined in the `functionaltest` package, but the file landed in `controlapi` declaring package `controlapi`, so the `controlapi` test binary did not build. This code passed on the PR branch, but it logically conflicted with a previous refactor without merge conflict. > It's a good idea to open an issue first for discussion. - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR |
||
|
Sneha-at
|
b7080602c6 |
Allow deleting an actor from any state. (#788)
Fixes #643 This feature introduces the ability to delete Substrate actors from any lifecycle state (e.g., RUNNING, PAUSED, PENDING), rather than requiring them to be hibernated into SUSPENDED (or CRASHED) beforehand. This is crucial for cleaning up stuck or failed actors. #### 1. Control Plane & gRPC API (pkg/proto/ateapipb, ateapi) • DeleteActorRequest.any_state: Added a new boolean field any_state to DeleteActorRequest. • When false (default): enforces the existing behavior where only actors in ACTOR_STATE_SUSPENDED or ACTOR_STATE_CRASHED (or already ACTOR_STATE_DELETING) can be deleted. • When true: allows deleting an actor in any state (e.g., RUNNING, PAUSED). • Orchestrated Cleanup Workflow (workflow_delete.go): 1. Transitions actor state to ACTOR_STATE_DELETING. 2. Calls atelet.Terminate to stop live workloads on the node. 3. Detaches external volumes from the worker node (with fallback logic using actor status if the ActorTemplate was deleted). 4. Releases the assigned physical worker Pod back to the WorkerPool. 5. Deletes external storage volumes through the storage plugins. 6. Finalizes removal of the actor from the persistence store. #### 2. Node Herder Agent (atelet) • Terminate RPC (main.go): Added a new Terminate method to AteomHerder: • Calls ateom.TerminateWorkload on the target worker pod. • Unmounts external volumes mounted for the actor on the node. • Cleans up and resets actor runtime host directories (OCI bundles, checkpoints, pid files). #### 3. In-Pod Sandboxes (ateom-gvisor & ateom-microvm) • TerminateWorkload RPC: • gVisor (main.go): Stops and deletes active runsc containers, unmounts bundle rootfs overlays, and tears down the actor's interior network namespace. • Micro-VM (checkpoint.go): Shuts down the Cloud Hypervisor VMM, unmounts bundle rootfs overlays, and cleans up the network namespace. • Emits an "Actor terminated" lifecycle log and clears the active actor attribution so the worker can host new workloads. #### 4. CLI (kubectl-ate) • --any-state Flag (delete_actor.go): kubectl-ate delete actor <actor-name> -a <atespace> --any-state ────── ## Verification [x] Changes tested in local with ``` • go build ./...: Passed go test ./...: Passed (all unit and functional tests pass) make lint: Passed (no lint errors) ``` Detailed manual tests ``` ### Scenario 1: Force Delete a Running Actor with --any-state 1. Create the actor: kubectl ate create actor manual-test-1 --template=ate-demo-counter-microvm/counter-microvm -a demo 2. Resume the actor: kubectl ate resume actor manual-test-1 -a demo 3. Force delete the running actor: kubectl ate delete actor manual-test-1 -a demo --any-state actor "manual-test-1" deleted ``` ────── ### Scenario 2: Standard Delete a Suspended Actor ``` 1. Create the actor: kubectl ate create actor manual-test-2 --template=ate-demo-counter-microvm/counter-microvm -a demo 2. Resume the actor: kubectl ate resume actor manual-test-2 -a demo 3. Suspend the actor: kubectl ate suspend actor manual-test-2 -a demo 4. Standard delete the suspended actor: kubectl ate delete actor manual-test-2 -a demo actor "manual-test-2" deleted ``` ────── ### Unit & E2E Tests # Unit & Functional Tests go test ./cmd/ateapi/internal/controlapi -run "TestDeleteActorWorkflow|TestEnsureMarkedDeleting" # E2E Tests E2E_TEMPLATE_NAMESPACE=ate-demo-counter-microvm E2E_TEMPLATE_NAME=counter-microvm ./hack/run-e2e-kind.sh ./internal/e2e/suites/demo -run TestActorLifecycle ./hack/run-e2e-kind.sh ./internal/e2e/suites/demo -run TestForceDeleteActorWithExternalVolume ``` - [ x] Appropriate changes to documentation are included in the PR |
||
|
Jeff Luo
|
bc59db783a |
ateapi: stamp the worker pool on successful suspend and pause (#1075)
Fixes #957 `ate.actor.lifecycle.operation.duration` carried the pool pair on suspend and pause only when they failed, so per-pool dashboards saw those two operations exclusively as failures. Both workflows record the histogram from a defer that reads the `actor` variable, and the happy path reassigns it to the finalized record. The finalize step commits the new state and the cleared `WorkerAssignment` in one update, so the defer found no assignment and dropped both keys. A failure returns the pre-finalize record, which still names the worker. Both now snapshot `lifecycleOpAttrs(...)` just before the finalize step — the same snapshot-before-clear crash.go does for the crash counter. Paths that end earlier keep the current computation. `delete` stays without a pool: it only runs from SUSPENDED or CRASHED, which already released the worker, so there is none to name. TestLifecycleOpPoolAttributesOnSuccess drives a real suspend and pause through the gRPC service; both subtests fail without the fix. - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR |
||
|
Luiz Oliveira
|
fc02127bfe |
ateapi - Require preconditions in Update methods (#1014)
https://github.com/agent-substrate/substrate/issues/954 - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR |
||
|
sfunkenhauser
|
43fa6ccbc7 |
Introduce the worker API (#1000)
Make workers a gobal resource, identified by a unique name. Worker APIs will be implemented in a follow up change. Part of #730 - [ x ] Tests pass - [ x ] Appropriate changes to documentation are included in the PR |
||
|
Luiz Oliveira
|
71a58cbf37 |
split controlapi functional tests into their own package (#918)
functional_test.go was also split into one file per resource, to match what we're doing for the RPC handlers too. See #891 I also needed to make some changes to move the tests to a separate package: * `NewAteletDialer` now takes `options`, and `WithDialCredentials` lets a test build its own transport credentials. The fake atelet is reached over insecure transport. Didn't change existing callers. This is the only non-test change. * Moved a few unit tests that are actually functional tests to the respective file under functionaltest/ Fixes #891 - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR |