mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-10-02 03:24:42 +08:00
Haven Xia
9df8ca6abe
A rolling upgrade needs two substrate versions running in one cluster, split by node: `atelet` and `ateom` speak a node-local protocol with no cross-version guarantees, so everything on a node has to come from one build. Today nothing records which build a node runs, and the `atelet` DaemonSet is a single fixed-name object, so a second version can only replace the first in place, instead of our porposed node by node rolling ([design](https://docs.google.com/document/d/1JduAyGZFyqdNp4UhKiv0EN-is3iW5tf5BwGTWbt2ouI/edit?usp=sharing&resourcekey=0-MXG12QCkleIxhY7cOB7g6A)). This PR is the basis for upgrade. Nodes and the `atelet` DaemonSet get keyed by an `ate.dev/substrate-version` label whose value is the build version stamped into the binaries. ### One derivation for the version label `internal/versionlabel` turns the build version into its two forms in one place: | form | grammar | used for | |---|---|---| | label value | k8s label value | node labels, DaemonSet labels, nodeSelectors | | name suffix | DNS-1123 | the DaemonSet name `atelet-<suffix>` | A version that is not a valid label value is rejected, because the label has to match what the `ldflags` stamp put into the binaries through MAKEFILE and `ko`. It's also exposed shell, so the install script can use it. ### A DaemonSet per version `atelet-<suffix>` as the name, the version label on metadata, selector, and pod template, and a pod nodeSelector on the same label. Two versions run side by side on disjoint old/new node sets. ### The install becomes version-aware - `install-ate.sh` reads the version from the same `make ldflags` output it stamps the binaries with, then fills the manifest placeholders. - It labels the nodes that exist at install time, nodes come after it carry no version label yet (so add notes in README). Re-running the install is idempotent, this is the basis for upgrade runbook later. - The `README` documents the invariant, how to read the installed version off the DaemonSet, and that a node added later hosts no workers until it carries the label. On GKE the node pool label is the birth default for new nodes - added in `tools/setup-gcp/README.md`. ### Follow-ups - `ate-setup` need same modification - minimize the affect for daily developer (pin the `VERSION` as `{USER}-dev` instead of from git - the manual rolling-upgrade runbook. Fix #1270 Ref [design doc](https://docs.google.com/document/d/1JduAyGZFyqdNp4UhKiv0EN-is3iW5tf5BwGTWbt2ouI/edit?usp=sharing&resourcekey=0-MXG12QCkleIxhY7cOB7g6A)