Files
ai-memory/crates/ai-memory-mcp/tests/suite/admin_write_page.rs
T
gb 3efb5b0b89 perf(dev): a self-contained build and a two-tier test loop on every platform
The edit-to-result loop was ~380s for the workspace on macOS and needed an
environment variable on every command. This makes `cargo t` the whole story
on macOS, Linux, and Windows, with numbers measured along the way.

Build
- `[profile.dev]` keeps only line tables (full debuginfo put ~190 MB of
  DWARF in each test binary and made the build linker-bound); dependencies
  build at opt-level 1 with no debuginfo; proc macros and build scripts at
  opt-level 3, since they are run once per dependent crate.
- Test binaries: 78 to 11 in the everyday loop (13 under `--workspace`).
  Each one is a link and, on macOS (Gatekeeper) and Windows (Defender), a
  first-run malware scan of the whole file, paid serially by nextest's list
  phase before the first test starts. Integration tests now live in
  `tests/suite/` and compile into their crate's own test harness (`mod.rs`,
  included from `src/lib.rs` under `#[cfg(test)]`, with `extern crate self`
  so they keep addressing the public API by crate name). Only the CLI keeps
  a separate `suite` target, because its tests run the built executable.
  The evals harness leaves `default-members`, so a bare `cargo t` skips its
  two binaries while `--workspace` (CI, the hook, `cargo tf`) still builds
  them. A repo-layout test fails on an undeclared suite file, a stray
  top-level `tests/*.rs`, or a `mod.rs` that `lib.rs` never includes.
- `ai-memory-cli` gains a lib target; `main.rs` is a shim. 806 tests that
  lived in the bin are reachable, and `--lib` runs skip the 127 MB binary.
- The web crate's vendored `static/tailwind.css` is the default on every
  build, so nothing needs `TAILWIND_SKIP=1` any more: every release, Docker,
  and CI path already used the vendored file, and the download branch only
  ever ran for developers who forgot the flag (and then rewrote the source
  tree as a side effect). `TAILWIND_BUILD=1 cargo build -p ai-memory-web`
  regenerates it explicitly. CI runs that on Linux and fails if the
  committed file is stale, a check that did not exist before; the committed
  file reproduces byte for byte today.
- `tokenizers` aligned on one version instead of the 0.21 pin plus the 0.22
  candle pulled in.

Test tiers
- `.config/nextest.toml`: the `default` profile skips any test whose module
  path has a segment starting with `slow` or `stress` (`packaging::slow::*`
  drives real wrapper scripts and fake container engines at 10-20s each;
  `stress_*` modules hammer concurrency), reports every failure in one run,
  and marks anything over 5s in its summary so a new slow test is visible
  the day it lands. `full` runs everything. `ci` keeps its retries and
  writes JUnit.
- `.cargo/config.toml` holds two aliases and nothing else: `cargo t` (default
  members) and `cargo tf` (`--workspace -P full`). `cargo t -p <crate>`
  builds just that crate. Neither passes `--all-targets`: there are no
  examples or benches, and it only added harnesses for two `test = false`
  targets.
- `scripts/install-git-hooks.sh` installs an opt-in pre-push hook that runs
  the full tier, touching only its own marked block. Two independent things
  run the skipped tier: that hook, and CI, which uses `cargo test` and never
  reads the nextest config.

Slow tests fixed rather than tiered
- `project_observations` in the consolidator trimmed an over-budget
  projection one observation at a time, re-rendering the whole text and
  re-scoring every remaining candidate after each removal. Each score scans
  the body, so 256 observations of 4k chars cost ~65k body scans per prompt:
  14s in production consolidation, exactly as in the unit test. Scores and
  per-block sizes are now computed once and the prune subtracts; output is
  unchanged and pinned by the existing tests. 13.9s to 0.18s.
- Windows takes ~2s to refuse a loopback connect, so every hook test that
  posted to a closed port paid 2s per request. `dead_http_endpoint()` in the
  new `ai-memory-test-support` crate accepts and closes instead, with a
  fallback to the closed port where binding is denied. devin hook tests:
  4.2s to 0.15s each.
- The store unit fixture opened a file-backed SQLite with the default
  rollback journal and synchronous=FULL, so ~120 parallel fixtures fsynced
  every transaction. journal_mode=MEMORY + synchronous=OFF: 242s to 89s of
  test time, p90 1.6s to 0.5s.
- Windows-only tests resolve `powershell.exe` or `pwsh.exe` once per process
  and the auto-improve eval fixtures are `.ps1` scripts instead of cmd.exe
  batch files; a post-bind settle sleep is gone; the two unpinned
  multi-thread tokio tests pin `worker_threads = 4`. The four copies of the
  PowerShell resolver and the mcp suite's duplicated `post`/`get` helpers are
  now one each.

Not done, with the numbers in AGENTS.md: nextest vs in-process libtest is a
wash per crate and a rout for the workspace (20s vs 309s); the
`local-embeddings` default feature costs ~50s of cold build and ~27 MB per
binary but under a second per relink, so it stays a product default.

Measured: workspace loop ~380s to ~150s on macOS; on a 32-thread Windows box
the warm everyday run is 20s of test time across 2919 tests in 11 binaries,
and the rebuild after a core edit is 13s of cargo with lld plus the
first-run scans.
2026-09-05 08:27:08 -04:00

321 lines
9.8 KiB
Rust

//! Integration tests for `POST /admin/write-page`.
//!
//! Exercises the route through the axum router: post a synthetic page,
//! verify it appears in `/admin/search` results. Also tests that an
//! unknown tier returns 422.
use ai_memory_mcp::{AdminState, admin_router};
use ai_memory_store::{DecayParams, Store};
use ai_memory_wiki::Wiki;
use axum::body::Body;
use axum::http::{Request, StatusCode};
use serde_json::json;
use tempfile::TempDir;
use tower::ServiceExt;
async fn make_state(tmp: &TempDir) -> AdminState {
let store = Store::open(tmp.path()).unwrap();
let wiki = Wiki::new(tmp.path(), store.writer.clone()).unwrap();
let db_path = store.db_path().to_path_buf();
AdminState {
ingest_metrics: std::sync::Arc::new(ai_memory_core::IngestMetrics::default()),
writer: store.writer.clone(),
reader: store.reader.clone(),
wiki,
llm: None,
auto_improve_require_approval: false,
auto_improve_review_config: Default::default(),
embedder: None,
provider_health: ai_memory_llm::ProviderHealth::default(),
decay_params: DecayParams::default(),
data_dir: tmp.path().to_path_buf(),
db_path,
bind: "127.0.0.1:0".to_string(),
home_dir: None,
bootstrap_lock: std::sync::Arc::new(tokio::sync::Mutex::new(())),
token_pepper: None,
active_project: ai_memory_core::ActiveProject::new(),
scope_invalidator: None,
trusted_proxy_identity: false,
}
}
async fn post_json(
state: AdminState,
uri: &str,
body: serde_json::Value,
) -> axum::response::Response {
let router = admin_router(state);
let req = Request::builder()
.method("POST")
.uri(uri)
.header("content-type", "application/json")
.body(Body::from(serde_json::to_vec(&body).unwrap()))
.unwrap();
router.oneshot(req).await.unwrap()
}
async fn body_json(resp: axum::response::Response) -> serde_json::Value {
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
serde_json::from_slice(&bytes).unwrap_or(serde_json::Value::Null)
}
#[tokio::test]
async fn write_page_returns_page_id_and_path() {
let tmp = TempDir::new().unwrap();
let state = make_state(&tmp).await;
let resp = post_json(
state,
"/admin/write-page",
json!({
"workspace": "default",
"project": "scratch",
"path": "notes/test-write.md",
"body": "This is a test page written via the admin route.",
"tier": "semantic",
}),
)
.await;
assert_eq!(resp.status(), StatusCode::OK, "write-page must succeed");
let body = body_json(resp).await;
assert!(
body["page_id"].is_string(),
"response must have page_id: {body}"
);
assert_eq!(
body["path"].as_str().unwrap(),
"notes/test-write.md",
"response path must match request: {body}"
);
}
#[tokio::test]
async fn write_page_appears_in_search() {
let tmp = TempDir::new().unwrap();
let state = make_state(&tmp).await;
// Write a page with a distinctive term.
let write_resp = post_json(
state.clone(),
"/admin/write-page",
json!({
"workspace": "default",
"project": "scratch",
"path": "notes/unique-term.md",
"body": "The xyloquartz pattern enables distributed widget fusion.",
"tier": "semantic",
}),
)
.await;
assert_eq!(write_resp.status(), StatusCode::OK);
// Now search for the distinctive term.
let router = admin_router(state);
let search_req = Request::builder()
.method("GET")
.uri("/admin/search?q=xyloquartz&limit=10")
.body(Body::empty())
.unwrap();
let search_resp = router.oneshot(search_req).await.unwrap();
assert_eq!(search_resp.status(), StatusCode::OK);
let hits: Vec<serde_json::Value> = serde_json::from_slice(
&axum::body::to_bytes(search_resp.into_body(), usize::MAX)
.await
.unwrap(),
)
.unwrap();
assert_eq!(
hits.len(),
1,
"written page must appear in search results: {hits:?}"
);
assert_eq!(hits[0]["path"].as_str().unwrap(), "notes/unique-term.md");
}
#[tokio::test]
async fn write_page_invalid_tier_returns_422() {
let tmp = TempDir::new().unwrap();
let state = make_state(&tmp).await;
let resp = post_json(
state,
"/admin/write-page",
json!({
"workspace": "default",
"project": "scratch",
"path": "notes/bad-tier.md",
"body": "Some content.",
"tier": "legendary",
}),
)
.await;
assert_eq!(
resp.status(),
StatusCode::UNPROCESSABLE_ENTITY,
"unknown tier must return 422"
);
let body = body_json(resp).await;
assert!(
body["error"].as_str().unwrap_or("").contains("legendary"),
"error must mention the unknown tier name: {body}"
);
}
#[tokio::test]
async fn write_page_two_projects_same_path_no_collision() {
// Two projects can hold pages with the same `pages.path` without
// colliding on disk — the per-project UUID-keyed layout (CLAUDE.md
// §15) guarantees structural isolation. The wiki crate already
// exercises this at the `Wiki::write_page` level; this test makes
// sure the invariant survives through the full
// `POST /admin/write-page` handler path.
let tmp = TempDir::new().unwrap();
let state = make_state(&tmp).await;
// Body 1: alpha project.
let resp = post_json(
state.clone(),
"/admin/write-page",
json!({
"workspace": "default",
"project": "alpha",
"path": "decisions/0001.md",
"body": "Page from project alpha — fingerprint AAAA-alpha.",
"tier": "semantic",
}),
)
.await;
assert_eq!(resp.status(), StatusCode::OK);
let alpha_id = body_json(resp).await["page_id"]
.as_str()
.expect("alpha page_id")
.to_string();
// Body 2: beta project, SAME page path.
let resp = post_json(
state.clone(),
"/admin/write-page",
json!({
"workspace": "default",
"project": "beta",
"path": "decisions/0001.md",
"body": "Page from project beta — fingerprint BBBB-beta.",
"tier": "semantic",
}),
)
.await;
assert_eq!(resp.status(), StatusCode::OK);
let beta_id = body_json(resp).await["page_id"]
.as_str()
.expect("beta page_id")
.to_string();
// Distinct page rows.
assert_ne!(
alpha_id, beta_id,
"the two writes must produce distinct page_ids"
);
// FTS5 search: both fingerprints findable, exactly one hit each.
let resp = post_json(state.clone(), "/admin/search?q=AAAA-alpha", json!(null)).await;
// /admin/search is a GET, not POST — re-route via the router.
drop(resp);
let router = admin_router(state.clone());
let resp = router
.clone()
.oneshot(
Request::builder()
.method("GET")
.uri("/admin/search?q=fingerprint")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let hits: serde_json::Value = body_json(resp).await;
let hits_arr = hits.as_array().expect("array");
assert_eq!(
hits_arr.len(),
2,
"both projects' pages must be searchable: {hits}"
);
// Same wiki path appears on both hits.
assert!(
hits_arr
.iter()
.all(|h| h["path"].as_str() == Some("decisions/0001.md")),
"both hits should share the same relative path: {hits}"
);
// Files on disk: both exist at their per-project namespaced paths.
// The two project_id UUIDs differ, so two distinct files live on
// disk even though pages.path is identical. Recursive walk via
// std::fs (no walkdir dep) — collect every `decisions/0001.md`
// we find under wiki/.
let wiki_dir = tmp.path().join("wiki");
let mut on_disk: Vec<std::path::PathBuf> = Vec::new();
collect_files_named(&wiki_dir, "decisions/0001.md", &mut on_disk);
assert_eq!(
on_disk.len(),
2,
"expected two physical files for the same page path; found: {on_disk:?}"
);
}
/// Recurse into `dir`, collecting every file whose relative path
/// (from `dir`) ends with `suffix`. Test helper kept inline because
/// it's only used here and we don't want a walkdir dep.
fn collect_files_named(dir: &std::path::Path, suffix: &str, out: &mut Vec<std::path::PathBuf>) {
let Ok(entries) = std::fs::read_dir(dir) else {
return;
};
for entry in entries.flatten() {
let p = entry.path();
let ft = match entry.file_type() {
Ok(ft) => ft,
Err(_) => continue,
};
if ft.is_dir() {
collect_files_named(&p, suffix, out);
} else if ft.is_file() && p.to_string_lossy().replace('\\', "/").ends_with(suffix) {
out.push(p);
}
}
}
#[tokio::test]
async fn write_page_with_tags_and_pinned() {
let tmp = TempDir::new().unwrap();
let state = make_state(&tmp).await;
let resp = post_json(
state,
"/admin/write-page",
json!({
"workspace": "default",
"project": "scratch",
"path": "notes/tagged.md",
"body": "Tagged and pinned content.",
"tier": "procedural",
"tags": ["rust", "memory"],
"pinned": true,
}),
)
.await;
assert_eq!(
resp.status(),
StatusCode::OK,
"tagged+pinned page must succeed"
);
let body = body_json(resp).await;
assert!(body["page_id"].is_string(), "must have page_id: {body}");
}