Files
NB 2183eec8ec Plan 012: archive ended pool servers in B2; prove export and import on a pool machine (#12789)
* Plan 012: archive ended pool servers in B2; prove export and import on a pool machine

Plan 012 moves the data of a Leaseweb pool server whose plan ended to an
encrypted archive in Backblaze B2, so the machine goes back to the pool within
a day. It follows #12721 and the dispute rule: 14 days of data, and a refunded
or early disputed server is deleted with no archive.

Item 1 is done. move-from-boat.sh export and import already work on a pool
machine; no pool mode is needed. scripts/hosting/pool-archive-e2e.ts proves it
on the test VPS pool-test-01: prepare, seed, export, reinstall, import, start.
6 of 6 checks pass, with 446 file facts the same after the import.

- export copied the cron spool file with its "DO NOT EDIT" header, and the
  import kept those lines as customer text. It now uses `crontab -l`. Moves
  from boat had the same bug.
- leaseweb-pool.ts packs the hosting folder with --no-xattrs: macOS tar added
  headers that GNU tar on the VPS warned about.
- apps/auth-api/.env.production: encrypted B2 keys for the EU bucket
  (operator and Worker). The Worker secrets come in item 2.

* Plan 012: encrypt pool archives with age and upload them to B2 from the machine

The E2E now runs the full archive path on a pool machine: sync makes an age
key and presigns B2 multipart part URLs, the machine encrypts the export and
uploads each part, sync checks each part MD5 and the size, then the machine is
reinstalled, downloads with a presigned URL, decrypts and imports.

leaseweb-bootstrap.sh installs age. The plan records the developer decisions
of 2026-11-19: one EU bucket for all locations, and age encryption.

A 5.75 GB server passes 9 of 9 checks (54,094 file facts). Upload 661 s, download
and decrypt 269 s, import 349 s. The import time is the disk speed of the VPS:
the same extract alone took 47 to 67 s.

Bash ignores PS4 from the environment as root, so the import trace sets it in
the shell.

* Remove a code comment and an old plan dependency

AGENTS.md forbids code comments. The reason for crontab -l stays here:
the spool file starts with the "DO NOT EDIT" header of cron, and crontab
on the VPS would keep those lines as text of the customer. Plan 012 no
longer waits for a B2 US account or a Stripe decision.

* Share the pool machine setup between the pool script and the E2E

Move the hosting copy and the prepare and upgrade scripts to pool-machine.ts.
The pool script and the archive E2E use this module. Remove the copies.

* Give fetch a null body when the B2 request has no body

The types ratchet does not allow an undefined value for an optional property.
2026-11-19 19:38:46 +01:00

38 lines
1.2 KiB
TypeScript

import { execFileSync } from "node:child_process";
import { resolve } from "node:path";
export const MACHINE_HOSTING = "/root/openbot-hosting";
export interface PoolRelease {
url: string;
sha256: string;
}
export function hostingCopy() {
const archive = execFileSync(
"tar",
["-czf", "-", "--no-xattrs", "--exclude=*.ts", "-C", resolve(import.meta.dirname, ".."), "hosting"],
{ maxBuffer: 16 * 1024 * 1024 },
);
return `set -euo pipefail
rm -rf ${MACHINE_HOSTING}
install -d -m 0700 ${MACHINE_HOSTING}
base64 -d <<'OPENBOT_HOSTING' | tar -xz --no-same-owner --strip-components=1 -C ${MACHINE_HOSTING}
${archive.toString("base64")}
OPENBOT_HOSTING
`;
}
export function prepareScript(machineId: string, release: PoolRelease, authApi: string, keys: string) {
return `${hostingCopy()}
bash ${MACHINE_HOSTING}/move-from-boat.sh prepare lsw-${machineId} 1000 1000 '${release.url}' ${release.sha256} '${authApi}'
bash ${MACHINE_HOSTING}/leaseweb-bootstrap.sh '${keys}'
`;
}
export function upgradeScript(release: PoolRelease, authApi: string) {
return `${hostingCopy()}
bash ${MACHINE_HOSTING}/move-from-boat.sh upgrade '${release.url}' ${release.sha256} '${authApi}'
`;
}