mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-11-10 10:13:50 +08:00
Eitan Yarmush
02eef7e82b
Actor template discovery currently uses a fixed 20-second resync interval. This PR adds `--template-resync-interval` so deployments can tune that delay, preserving the `20s` default and rejecting nonpositive values. The setting also controls the reconciler's existing fallback retry delay. This is intentionally a small change to start a discussion about how template builds should be dispatched as the template catalog grows. The current resync fetches and decodes every template, including completed ones. At 100,000 templates, a 20-second interval implies roughly 5,000 template rows read per second per replica, assuming scans finish quickly. This is an estimate from the code, not a benchmark. Possible follow-ups: - **Immediate enqueue:** start work after creation, retaining a slower recovery scan for crashes between persistence and enqueue. - **Outbox/watch:** consume changes instead of scanning the catalog. The existing worker outbox still polls every 50 ms and broadcasts events to each subscriber; write overhead and recovery scans need consideration. - **Durable pending-build queue:** atomically record work, claim due jobs using short `FOR UPDATE SKIP LOCKED` transactions, and recover expired leases. This still polls, but queries pending work rather than the full catalog. `LISTEN/NOTIFY` is not the proposed default because it serializes notifying commits. - **Imperative build / long-running operation:** give callers explicit build control or a progress/completion handle. Either still needs reliable execution underneath. The main question is whether template builds need a broadcast change feed or a queue where replicas claim different jobs. Neither alternative is implemented here. [Full research: database costs, execution options, recovery requirements, and sources](https://gist.github.com/EItanya/0a1d6893ede34f2f0e9d9d1929ecb823). Validation: all Go race tests and repository verifiers passed using module mode with `NO_COLOR` unset. CLI checks confirmed the default and rejection of zero/negative intervals; the existing reconciliation test checks a custom interval. Fixes #<issue_number_goes_here> > It's a good idea to open an issue first for discussion. - [ ] Tests pass - [ ] Appropriate changes to documentation are included in the PR --------- Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>