mirror of
https://github.com/akitaonrails/ai-memory.git
synced 2026-10-02 03:24:46 +08:00
fix(restore): stage and verify the tarball before replacing the live data dir
`ai-memory restore --force` deleted the live `wiki/` and `db/` before the tarball had been opened, validated or extracted, and before the restored store had been opened. A truncated or corrupt archive, an entry outside the allowed layout, or a snapshot the current binary could not open (a backup taken by a newer release, a torn file) therefore left an empty or half-extracted data dir with nothing to fall back to — at the one moment the operator has no other copy. The tarball is now extracted and validated into a staging directory beside the live data, the staged store is opened there so pending migrations run and the snapshot is verified, and only then are the live `wiki/`, `db/` and (when the archive carries one) `config.toml` renamed aside, the staged copies renamed into place, and the previous data deleted. Every move is a same-filesystem rename; a failed move reverses the moves already made, and a failed reversal reports the directory that still holds the pre-restore data. Scratch directories are removed in every outcome. A successful restore behaves as before: `wiki/` and `db/` become what the archive holds, `config.toml` is replaced only when the archive has one, `logs/`, `models/` and `raw/` are never touched, `--force` is still required for a populated data dir, and the sibling-process guard runs first. Regression tests drive the new `restore_data_dir` directly: the four "keeps live data" cases fail under the previous order of operations and pass here; success-path cases cover the populated, empty and config-less archives and reopen the swapped-in store.
This commit is contained in:
Vinícius Andrade
@@ -55,6 +55,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
unchanged — this only makes the existing warning reliably visible. (#903)
|
||||
|
||||
### Fixed
|
||||
- `ai-memory restore --force` no longer deletes the live `wiki/` and `db/`
|
||||
before the tarball has been read. The archive is now extracted and
|
||||
validated into a staging directory beside the data, the restored store is
|
||||
opened there so pending migrations run and the snapshot is verified, and
|
||||
only then are the live directories swapped out by rename (reversed if a
|
||||
move fails). A truncated or corrupt tarball, an entry outside the allowed
|
||||
layout, or a snapshot the current binary cannot open — a backup taken by
|
||||
a newer release, say — previously left an empty or half-extracted data
|
||||
dir with nothing to fall back to; it now leaves the existing data exactly
|
||||
as it was. (#923)
|
||||
- OMP (OpenClaw) tool calls are recorded again. OMP was missing from the
|
||||
closed-tool-agent set, so its tool events fell through the OpenCode-only
|
||||
legacy body reader and produced an empty excerpt — nothing reached session
|
||||
|
||||
@@ -1,9 +1,17 @@
|
||||
//! `ai-memory restore --from <tarball>` — restore a backup tarball.
|
||||
//!
|
||||
//! Refuses to overwrite a non-empty data dir unless `--force` is given.
|
||||
//! Refuses while another `ai-memory` process is alive. After extraction,
|
||||
//! re-opens the store so any pending migrations run (and a corrupt
|
||||
//! snapshot fails loudly).
|
||||
//! Refuses while another `ai-memory` process is alive.
|
||||
//!
|
||||
//! The live data is never touched before the archive has proven usable:
|
||||
//! the tarball is extracted and validated into a staging directory beside
|
||||
//! the live `wiki/` and `db/`, the restored store is opened there so any
|
||||
//! pending migrations run (and a corrupt snapshot fails loudly), and only
|
||||
//! then are the live directories swapped out by rename. A truncated
|
||||
//! archive, an entry outside the allowed layout, or a snapshot the current
|
||||
//! binary cannot open therefore leaves the existing data exactly as it was
|
||||
//! — the moment a restore fails is the moment the operator has no other
|
||||
//! copy, so the previous state must survive it.
|
||||
//!
|
||||
//! # Exception to invariant §16
|
||||
//!
|
||||
@@ -18,18 +26,29 @@ use ai_memory_store::Store;
|
||||
use anyhow::{Context, Result, bail};
|
||||
use flate2::read::GzDecoder;
|
||||
use std::path::{Component, Path};
|
||||
use tracing::info;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::cli::RestoreArgs;
|
||||
use crate::config::Config;
|
||||
use crate::process_guard::{busy_message, sibling_processes};
|
||||
|
||||
/// Data-dir directories a restore replaces wholesale: whatever the archive
|
||||
/// holds for each takes the live one's place, and a directory the archive
|
||||
/// lacks is retired rather than merged with the archive's state. `logs/`,
|
||||
/// `models/`, `raw/` and anything else beside them are never touched.
|
||||
const REPLACED_DIRS: &[&str] = &["wiki", "db"];
|
||||
|
||||
/// The one file a restore replaces, and only when the archive carries it;
|
||||
/// otherwise the live copy stays.
|
||||
const CONFIG_FILE: &str = "config.toml";
|
||||
|
||||
/// Run the `restore` subcommand.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns an error if another `ai-memory` process is running, the
|
||||
/// data dir is non-empty without `--force`, the tarball cannot be
|
||||
/// extracted, or the restored store fails to open.
|
||||
/// extracted, or the restored store fails to open. In every one of those
|
||||
/// cases the live data dir is left as it was.
|
||||
pub fn run(config: &Config, args: RestoreArgs) -> Result<()> {
|
||||
let siblings = sibling_processes();
|
||||
if !siblings.is_empty() {
|
||||
@@ -40,37 +59,7 @@ pub fn run(config: &Config, args: RestoreArgs) -> Result<()> {
|
||||
bail!("source tarball {} not found", args.from.display());
|
||||
}
|
||||
|
||||
let wiki = config.data_dir.join("wiki");
|
||||
let db = config.data_dir.join("db").join("memory.sqlite");
|
||||
if (wiki.is_dir() && std::fs::read_dir(&wiki)?.next().is_some()) || db.is_file() {
|
||||
if !args.force {
|
||||
bail!(
|
||||
"refusing to restore: data dir at {} is non-empty (pass --force to overwrite)",
|
||||
config.data_dir.display(),
|
||||
);
|
||||
}
|
||||
// Force path: drop the existing wiki + db so the tarball can
|
||||
// populate them cleanly. Keep config.toml, logs/, models/.
|
||||
for sub in ["wiki", "db"] {
|
||||
let path = config.data_dir.join(sub);
|
||||
if path.exists() {
|
||||
std::fs::remove_dir_all(&path)?;
|
||||
}
|
||||
}
|
||||
}
|
||||
std::fs::create_dir_all(&config.data_dir)?;
|
||||
|
||||
let file = std::fs::File::open(&args.from)
|
||||
.with_context(|| format!("opening {}", args.from.display()))?;
|
||||
let decoder = GzDecoder::new(file);
|
||||
let mut archive = tar::Archive::new(decoder);
|
||||
unpack_checked_archive(&mut archive, &config.data_dir)
|
||||
.with_context(|| format!("extracting into {}", config.data_dir.display()))?;
|
||||
info!(from = %args.from.display(), into = %config.data_dir.display(), "tarball extracted");
|
||||
|
||||
// Open + drop the store so refinery applies any pending migrations
|
||||
// and the SQLite file is validated.
|
||||
let _store = Store::open(&config.data_dir).context("opening restored store")?;
|
||||
restore_data_dir(&config.data_dir, &args.from, args.force)?;
|
||||
info!("restore complete");
|
||||
println!(
|
||||
"restored {} -> {}",
|
||||
@@ -80,6 +69,170 @@ pub fn run(config: &Config, args: RestoreArgs) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Stage, validate, then swap. Nothing under `data_dir` changes until the
|
||||
/// archive has been fully extracted into a staging directory and the
|
||||
/// staged store has opened; the live `wiki/`, `db/` and (when the archive
|
||||
/// carries one) `config.toml` are then exchanged for the staged copies by
|
||||
/// rename and rolled back if any step of the exchange fails.
|
||||
fn restore_data_dir(data_dir: &Path, from: &Path, force: bool) -> Result<()> {
|
||||
let wiki = data_dir.join("wiki");
|
||||
let db = data_dir.join("db").join("memory.sqlite");
|
||||
let occupied = (wiki.is_dir() && std::fs::read_dir(&wiki)?.next().is_some()) || db.is_file();
|
||||
if occupied && !force {
|
||||
bail!(
|
||||
"refusing to restore: data dir at {} is non-empty (pass --force to overwrite)",
|
||||
data_dir.display(),
|
||||
);
|
||||
}
|
||||
std::fs::create_dir_all(data_dir)?;
|
||||
|
||||
// Both scratch directories live inside the data dir so every move below
|
||||
// is a rename on one filesystem, never a copy that could half-complete.
|
||||
let stamp = format!(
|
||||
"{}-{}",
|
||||
jiff::Timestamp::now().strftime("%Y%m%d-%H%M%S"),
|
||||
std::process::id()
|
||||
);
|
||||
let staging = data_dir.join(format!(".restore-staging-{stamp}"));
|
||||
let previous = data_dir.join(format!(".restore-previous-{stamp}"));
|
||||
|
||||
let outcome = stage_then_swap(data_dir, from, &staging, &previous);
|
||||
// The staging dir is scratch in every outcome: on success its contents
|
||||
// were moved into place, on failure the live data was never touched.
|
||||
if staging.exists()
|
||||
&& let Err(e) = std::fs::remove_dir_all(&staging)
|
||||
{
|
||||
warn!(path = %staging.display(), error = %e, "could not remove restore staging dir");
|
||||
eprintln!(
|
||||
"warning: could not remove staging dir {} ({e}); delete it by hand",
|
||||
staging.display()
|
||||
);
|
||||
}
|
||||
outcome
|
||||
}
|
||||
|
||||
fn stage_then_swap(data_dir: &Path, from: &Path, staging: &Path, previous: &Path) -> Result<()> {
|
||||
std::fs::create_dir(staging)
|
||||
.with_context(|| format!("creating staging dir {}", staging.display()))?;
|
||||
|
||||
// 1. Extract into staging, validating every entry on the way. A
|
||||
// truncated gzip stream, an unreadable member or a path outside the
|
||||
// allowed layout fails here, with the live data still untouched.
|
||||
let file = std::fs::File::open(from).with_context(|| format!("opening {}", from.display()))?;
|
||||
let decoder = GzDecoder::new(file);
|
||||
let mut archive = tar::Archive::new(decoder);
|
||||
unpack_checked_archive(&mut archive, staging)
|
||||
.with_context(|| format!("extracting {} into {}", from.display(), staging.display()))?;
|
||||
info!(from = %from.display(), into = %staging.display(), "tarball extracted into staging");
|
||||
|
||||
// 2. Open + drop the staged store so refinery applies any pending
|
||||
// migrations and the SQLite file is validated — still before anything
|
||||
// live is touched. Dropping the store joins the writer thread and
|
||||
// closes every connection, so the directory can be renamed afterwards
|
||||
// on Windows as well.
|
||||
drop(Store::open(staging).context("opening restored store")?);
|
||||
|
||||
// 3. Exchange the live directories for the staged ones.
|
||||
swap_into_place(data_dir, staging, previous)?;
|
||||
info!(into = %data_dir.display(), "restored data swapped into place");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Move the live entries aside into `previous`, move the staged entries
|
||||
/// into place, then discard `previous`. Every step is a same-filesystem
|
||||
/// rename; if one fails, the moves already made are reversed so the data
|
||||
/// dir ends up as it started.
|
||||
fn swap_into_place(data_dir: &Path, staging: &Path, previous: &Path) -> Result<()> {
|
||||
std::fs::create_dir(previous).with_context(|| format!("creating {}", previous.display()))?;
|
||||
|
||||
// Entries moved from the data dir into `previous`, and staged entries
|
||||
// already placed live: the two lists a rollback has to undo.
|
||||
let mut moved_aside: Vec<&str> = Vec::new();
|
||||
let mut placed: Vec<&str> = Vec::new();
|
||||
|
||||
let exchange = (|| -> Result<()> {
|
||||
for name in REPLACED_DIRS {
|
||||
let live = data_dir.join(name);
|
||||
if live.exists() {
|
||||
std::fs::rename(&live, previous.join(name))
|
||||
.with_context(|| format!("moving {} aside", live.display()))?;
|
||||
moved_aside.push(name);
|
||||
}
|
||||
}
|
||||
let live_config = data_dir.join(CONFIG_FILE);
|
||||
if staging.join(CONFIG_FILE).is_file() && live_config.exists() {
|
||||
std::fs::rename(&live_config, previous.join(CONFIG_FILE))
|
||||
.with_context(|| format!("moving {} aside", live_config.display()))?;
|
||||
moved_aside.push(CONFIG_FILE);
|
||||
}
|
||||
for name in REPLACED_DIRS.iter().chain(std::iter::once(&CONFIG_FILE)) {
|
||||
let staged = staging.join(name);
|
||||
if staged.exists() {
|
||||
std::fs::rename(&staged, data_dir.join(name))
|
||||
.with_context(|| format!("moving {} into place", staged.display()))?;
|
||||
placed.push(name);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})();
|
||||
|
||||
if let Err(e) = exchange {
|
||||
if let Err(rollback_err) = roll_back(data_dir, previous, &placed, &moved_aside) {
|
||||
bail!(
|
||||
"INCONSISTENT STATE: restore swap failed ({e:#}) and moving the previous data \
|
||||
back also failed ({rollback_err:#}); the pre-restore wiki/ and db/ are under {} \
|
||||
— move them back into {} by hand",
|
||||
previous.display(),
|
||||
data_dir.display(),
|
||||
);
|
||||
}
|
||||
return Err(e.context("restore swap failed; the previous data was moved back into place"));
|
||||
}
|
||||
|
||||
// The previous data is only discarded once the restored copy is live —
|
||||
// the documented `--force` semantics, now applied last instead of first.
|
||||
if let Err(e) = std::fs::remove_dir_all(previous) {
|
||||
warn!(path = %previous.display(), error = %e, "could not remove pre-restore data");
|
||||
eprintln!(
|
||||
"warning: restore succeeded but the pre-restore data under {} could not be \
|
||||
removed ({e}); delete it by hand",
|
||||
previous.display()
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Undo a partial swap: remove whatever staged entries were already placed
|
||||
/// live, then move the previous entries back. Placed entries are removed
|
||||
/// before their predecessors return so a rename never finds its target
|
||||
/// occupied.
|
||||
fn roll_back(
|
||||
data_dir: &Path,
|
||||
previous: &Path,
|
||||
placed: &[&str],
|
||||
moved_aside: &[&str],
|
||||
) -> Result<()> {
|
||||
for name in placed {
|
||||
let live = data_dir.join(name);
|
||||
remove_path(&live).with_context(|| format!("removing half-placed {}", live.display()))?;
|
||||
}
|
||||
for name in moved_aside {
|
||||
let parked = previous.join(name);
|
||||
std::fs::rename(&parked, data_dir.join(name))
|
||||
.with_context(|| format!("moving {} back", parked.display()))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove_path(path: &Path) -> std::io::Result<()> {
|
||||
match std::fs::symlink_metadata(path) {
|
||||
Ok(meta) if meta.is_dir() => std::fs::remove_dir_all(path),
|
||||
Ok(_) => std::fs::remove_file(path),
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
fn unpack_checked_archive<R: std::io::Read>(
|
||||
archive: &mut tar::Archive<R>,
|
||||
data_dir: &Path,
|
||||
@@ -354,4 +507,284 @@ mod tests {
|
||||
.is_file()
|
||||
);
|
||||
}
|
||||
|
||||
// ---- stage → validate → swap: a failed restore leaves the live data alone ----
|
||||
|
||||
/// A populated data dir as an operator has it: a wiki page, a database
|
||||
/// file (never opened by these tests, so any bytes do), the config, and
|
||||
/// the neighbours `logs/` and `raw/` that a restore must never touch.
|
||||
fn seed_live_data(dir: &Path) {
|
||||
std::fs::create_dir_all(dir.join("wiki/default/project/notes")).unwrap();
|
||||
std::fs::write(dir.join("wiki/default/project/notes/old.md"), b"old page").unwrap();
|
||||
std::fs::create_dir_all(dir.join("db")).unwrap();
|
||||
std::fs::write(dir.join("db/memory.sqlite"), b"old database bytes").unwrap();
|
||||
std::fs::write(dir.join("config.toml"), b"# old config\n").unwrap();
|
||||
std::fs::create_dir_all(dir.join("logs")).unwrap();
|
||||
std::fs::write(dir.join("logs/app.log"), b"log").unwrap();
|
||||
std::fs::create_dir_all(dir.join("raw")).unwrap();
|
||||
std::fs::write(dir.join("raw/segment.jsonl"), b"{}").unwrap();
|
||||
}
|
||||
|
||||
/// Scratch directories a restore leaves behind only when its cleanup
|
||||
/// failed: none may survive a run, successful or not.
|
||||
fn restore_scratch_dirs(data_dir: &Path) -> Vec<std::path::PathBuf> {
|
||||
std::fs::read_dir(data_dir)
|
||||
.unwrap()
|
||||
.filter_map(Result::ok)
|
||||
.map(|e| e.path())
|
||||
.filter(|p| {
|
||||
p.file_name()
|
||||
.and_then(|n| n.to_str())
|
||||
.is_some_and(|n| n.starts_with(".restore-"))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn assert_live_data_untouched(dir: &Path) {
|
||||
assert_eq!(
|
||||
std::fs::read(dir.join("wiki/default/project/notes/old.md")).unwrap(),
|
||||
b"old page"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(dir.join("db/memory.sqlite")).unwrap(),
|
||||
b"old database bytes"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(dir.join("config.toml")).unwrap(),
|
||||
b"# old config\n"
|
||||
);
|
||||
assert_eq!(std::fs::read(dir.join("logs/app.log")).unwrap(), b"log");
|
||||
assert_eq!(std::fs::read(dir.join("raw/segment.jsonl")).unwrap(), b"{}");
|
||||
let scratch = restore_scratch_dirs(dir);
|
||||
assert!(scratch.is_empty(), "scratch dirs left behind: {scratch:?}");
|
||||
}
|
||||
|
||||
/// Bytes of a real, migrated `memory.sqlite` — what a `backup` tarball
|
||||
/// carries.
|
||||
fn migrated_sqlite_bytes() -> Vec<u8> {
|
||||
let tmp = tempfile::TempDir::new().unwrap();
|
||||
drop(Store::open(tmp.path()).unwrap());
|
||||
std::fs::read(tmp.path().join("db/memory.sqlite")).unwrap()
|
||||
}
|
||||
|
||||
/// A gzipped tarball holding the given regular files.
|
||||
fn gz_tarball(entries: &[(&str, &[u8])]) -> Vec<u8> {
|
||||
use std::io::Write as _;
|
||||
|
||||
let mut tar_bytes = Vec::new();
|
||||
{
|
||||
let mut builder = tar::Builder::new(&mut tar_bytes);
|
||||
for (path, body) in entries {
|
||||
let mut header = tar::Header::new_gnu();
|
||||
header.set_path(path).unwrap();
|
||||
header.set_size(body.len() as u64);
|
||||
header.set_mode(0o644);
|
||||
header.set_cksum();
|
||||
builder.append(&header, *body).unwrap();
|
||||
}
|
||||
builder.finish().unwrap();
|
||||
}
|
||||
let mut gz = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::default());
|
||||
gz.write_all(&tar_bytes).unwrap();
|
||||
gz.finish().unwrap()
|
||||
}
|
||||
|
||||
fn good_archive() -> Vec<u8> {
|
||||
let db = migrated_sqlite_bytes();
|
||||
gz_tarball(&[
|
||||
("wiki/default/project/notes/new.md", b"new page"),
|
||||
("db/memory.sqlite", db.as_slice()),
|
||||
("config.toml", b"# restored config\n"),
|
||||
])
|
||||
}
|
||||
|
||||
fn write_tarball(dir: &Path, name: &str, bytes: &[u8]) -> std::path::PathBuf {
|
||||
let path = dir.join(name);
|
||||
std::fs::write(&path, bytes).unwrap();
|
||||
path
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_force_keeps_live_data_when_the_tarball_is_not_a_gzip_stream() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
seed_live_data(data.path());
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
let from = write_tarball(scratch.path(), "bad.tar.gz", b"this is not a gzip stream");
|
||||
|
||||
let err = restore_data_dir(data.path(), &from, true).unwrap_err();
|
||||
|
||||
assert!(
|
||||
format!("{err:#}").contains("extracting"),
|
||||
"unexpected error: {err:#}"
|
||||
);
|
||||
assert_live_data_untouched(data.path());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_force_keeps_live_data_when_the_tarball_is_truncated() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
seed_live_data(data.path());
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
let whole = good_archive();
|
||||
let from = write_tarball(scratch.path(), "cut.tar.gz", &whole[..whole.len() / 2]);
|
||||
|
||||
let err = restore_data_dir(data.path(), &from, true).unwrap_err();
|
||||
|
||||
assert!(
|
||||
format!("{err:#}").contains("extracting"),
|
||||
"unexpected error: {err:#}"
|
||||
);
|
||||
assert_live_data_untouched(data.path());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_force_keeps_live_data_when_an_entry_is_outside_the_allowed_layout() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
seed_live_data(data.path());
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
// Valid entries first, so extraction is already under way when the
|
||||
// stray one is met — the failure must still leave nothing behind.
|
||||
let db = migrated_sqlite_bytes();
|
||||
let from = write_tarball(
|
||||
scratch.path(),
|
||||
"stray.tar.gz",
|
||||
&gz_tarball(&[
|
||||
("wiki/default/project/notes/new.md", b"new page"),
|
||||
("db/memory.sqlite", db.as_slice()),
|
||||
("db/extra.sqlite", b"stray"),
|
||||
]),
|
||||
);
|
||||
|
||||
let err = restore_data_dir(data.path(), &from, true).unwrap_err();
|
||||
|
||||
assert!(
|
||||
format!("{err:#}").contains("unexpected path"),
|
||||
"unexpected error: {err:#}"
|
||||
);
|
||||
assert_live_data_untouched(data.path());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_force_keeps_live_data_when_the_restored_store_cannot_open() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
seed_live_data(data.path());
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
let from = write_tarball(
|
||||
scratch.path(),
|
||||
"notadb.tar.gz",
|
||||
&gz_tarball(&[
|
||||
("wiki/default/project/notes/new.md", b"new page"),
|
||||
("db/memory.sqlite", &[0xFF; 4096]),
|
||||
]),
|
||||
);
|
||||
|
||||
let err = restore_data_dir(data.path(), &from, true).unwrap_err();
|
||||
|
||||
assert!(
|
||||
format!("{err:#}").contains("opening restored store"),
|
||||
"unexpected error: {err:#}"
|
||||
);
|
||||
assert_live_data_untouched(data.path());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_refuses_a_populated_data_dir_without_force() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
seed_live_data(data.path());
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
let from = write_tarball(scratch.path(), "good.tar.gz", &good_archive());
|
||||
|
||||
let err = restore_data_dir(data.path(), &from, false).unwrap_err();
|
||||
|
||||
assert!(
|
||||
err.to_string().contains("pass --force"),
|
||||
"unexpected error: {err:#}"
|
||||
);
|
||||
assert_live_data_untouched(data.path());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_force_replaces_the_live_data_with_the_archive() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
seed_live_data(data.path());
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
let from = write_tarball(scratch.path(), "good.tar.gz", &good_archive());
|
||||
|
||||
restore_data_dir(data.path(), &from, true).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
std::fs::read(data.path().join("wiki/default/project/notes/new.md")).unwrap(),
|
||||
b"new page"
|
||||
);
|
||||
assert!(
|
||||
!data
|
||||
.path()
|
||||
.join("wiki/default/project/notes/old.md")
|
||||
.exists(),
|
||||
"the previous wiki must not be merged into the restored one"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(data.path().join("config.toml")).unwrap(),
|
||||
b"# restored config\n"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(data.path().join("logs/app.log")).unwrap(),
|
||||
b"log"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(data.path().join("raw/segment.jsonl")).unwrap(),
|
||||
b"{}"
|
||||
);
|
||||
let scratch_dirs = restore_scratch_dirs(data.path());
|
||||
assert!(
|
||||
scratch_dirs.is_empty(),
|
||||
"scratch dirs left behind: {scratch_dirs:?}"
|
||||
);
|
||||
// The swapped-in store is the migrated one and opens cleanly in place.
|
||||
drop(Store::open(data.path()).unwrap());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_keeps_the_live_config_when_the_archive_carries_none() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
seed_live_data(data.path());
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
let db = migrated_sqlite_bytes();
|
||||
let from = write_tarball(
|
||||
scratch.path(),
|
||||
"noconfig.tar.gz",
|
||||
&gz_tarball(&[
|
||||
("wiki/default/project/notes/new.md", b"new page"),
|
||||
("db/memory.sqlite", db.as_slice()),
|
||||
]),
|
||||
);
|
||||
|
||||
restore_data_dir(data.path(), &from, true).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
std::fs::read(data.path().join("config.toml")).unwrap(),
|
||||
b"# old config\n"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(data.path().join("wiki/default/project/notes/new.md")).unwrap(),
|
||||
b"new page"
|
||||
);
|
||||
assert!(restore_scratch_dirs(data.path()).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restore_into_an_empty_data_dir_needs_no_force() {
|
||||
let data = tempfile::TempDir::new().unwrap();
|
||||
let scratch = tempfile::TempDir::new().unwrap();
|
||||
let from = write_tarball(scratch.path(), "good.tar.gz", &good_archive());
|
||||
|
||||
restore_data_dir(data.path(), &from, false).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
std::fs::read(data.path().join("wiki/default/project/notes/new.md")).unwrap(),
|
||||
b"new page"
|
||||
);
|
||||
assert!(data.path().join("db/memory.sqlite").is_file());
|
||||
assert!(restore_scratch_dirs(data.path()).is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
+23
-5
@@ -19,7 +19,7 @@ on a homelab box where mistakes are harder to undo.
|
||||
| `backup --to` | ✅ yes | no | n/a | Streams a gzipped tarball from the server's online `sqlite3 .backup` plus the wiki tree. Safe alongside the live writer. |
|
||||
| `checkpoints` | ✅ yes | no | n/a | Lists recent wiki git checkpoints. Read-only. |
|
||||
| `restore-page --path --from` | ✅ yes | overwrites one markdown page version | yes (restore another checkpoint) | Restores one page from wiki git history, reindexes it into SQLite, and writes a post-restore checkpoint. Does not restore DB-only state. |
|
||||
| `restore --from <tarball>` | ❌ **stop the server first** | overwrites the data dir | no (without prior backup) | Refuses if any sibling `ai-memory` process is alive (sysinfo guard). |
|
||||
| `restore --from <tarball>` | ❌ **stop the server first** | overwrites the data dir | no (without prior backup) | Refuses if any sibling `ai-memory` process is alive (sysinfo guard). Stages and verifies the archive before swapping it in, so a failed restore leaves `wiki/` and `db/` as they were. |
|
||||
| `reset --confirm` | ❌ **stop the server first** | yes, all data | no | Refuses if any sibling `ai-memory` process is alive (sysinfo guard). |
|
||||
| `reindex` | ❌ **stop the server first** | no wiki wipe; requires a clean DB | only with prior DB backup | Rebuilds pages/links/FTS from `wiki/` using `_meta.md` manifests. Refuses if SQLite already has rows so stale DB-only state cannot survive silently. |
|
||||
|
||||
@@ -715,17 +715,35 @@ alive (uses `sysinfo` to scan the process table).
|
||||
Order of operations:
|
||||
|
||||
1. Check the data dir is empty (or the user passed `--force`).
|
||||
2. Extract the tarball into the data dir.
|
||||
3. Restore the SQLite snapshot in place.
|
||||
4. Print a one-line summary.
|
||||
2. Extract the tarball into a staging directory beside the live data
|
||||
(`<data_dir>/.restore-staging-<stamp>/`), validating every entry.
|
||||
3. Open the staged store so pending migrations run and the SQLite
|
||||
snapshot is verified — still without touching the live data.
|
||||
4. Swap: rename the live `wiki/` and `db/` (and `config.toml`, when the
|
||||
archive carries one) aside, rename the staged copies into place, then
|
||||
delete the previous data. Each move is a same-filesystem rename; if
|
||||
one fails, the moves already made are reversed.
|
||||
5. Print a one-line summary.
|
||||
|
||||
The live data is therefore untouched until the archive has proven usable.
|
||||
A restore that fails in steps 2–3 leaves `wiki/` and `db/` exactly as they
|
||||
were, which matters because a restore is usually attempted when no other
|
||||
copy exists.
|
||||
|
||||
Failure modes:
|
||||
|
||||
- **Server still running** → exits with "another ai-memory process is
|
||||
alive (pid X); stop it before restoring" - same wording as `reset`.
|
||||
- **`--confirm` omitted** → exits with usage hint.
|
||||
- **Data dir not empty + no `--force`** → exits with "data dir not
|
||||
empty; pass `--force` to overwrite".
|
||||
- **Truncated or corrupt tarball, an entry outside the allowed layout, or
|
||||
a snapshot the current binary cannot open** (for example a backup taken
|
||||
by a newer release) → exits with the error; the existing `wiki/` and
|
||||
`db/` are left as they were.
|
||||
- **A rename in the swap fails and cannot be reversed** → exits with an
|
||||
`INCONSISTENT STATE` message naming the
|
||||
`<data_dir>/.restore-previous-<stamp>/` directory that still holds the
|
||||
pre-restore `wiki/` and `db/`, to be moved back by hand.
|
||||
|
||||
### `reset`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user