Commit Graph
9 Commits
Author SHA1 Message Date
Martin Vogel 1b84943000 Separate perf tests from CI, fix cross-platform build issues
- Add CBM_SKIP_PERF=1 env var to skip incremental/perf test suite
- CI and Docker test targets skip perf by default (run.sh perf for manual)
- Convert all perf assertions to warnings (log timing, never block)
- Fix store.h anonymous enum in struct (GCC rejects, clang accepts)
- Fix test_store_search.c mkstemp on non-template path
- Add ca-certificates to Docker test image for git HTTPS
- Add cbm_gmtime_r shim in compat.h (Windows gmtime_s wrapper)
- Fix compat.c missing constants.h include (Windows build)
- Fix platform.c _environ redeclaration on mingw
- Rename trace_call_path -> trace_path in smoke/soak/fuzz scripts
2026-04-02 14:52:14 +02:00
Martin Vogel 0c81278777 Raise soak query latency threshold to 60s (MSYS2 overhead) 2026-03-26 11:14:57 +01:00
Martin Vogel d8f168cd39 Fix Windows smoke 14e + soak latency threshold
- Phase 14: copy binary with .exe suffix on Windows (was creating
  non-.exe copy that uninstall didn't remove)
- Phase 14e: check both .exe and non-.exe paths
- Soak latency: exclude index_repository from max query latency
  (indexing 377 files is legitimately slow on Windows/MSYS2)
2026-03-26 10:34:00 +01:00
Martin Vogel b568884e4a Fix MSYS2 python3 path issue: pipe files via cat instead of open()
MinGW python3 is a native Windows binary that doesn't understand
POSIX paths like /tmp/foo.json. Piping file content through cat
(which runs in MSYS2 bash and handles path translation) to python's
stdin avoids the issue entirely.

Fixes Windows smoke test 8a and soak baseline collection.
2026-03-25 22:48:31 +01:00
Martin Vogel adb14ee245 Soak: enforce RSS slope only for 30+ min runs (10-min too noisy)
Investigation: macOS arm64 grew 1MB (43→44MB) but slope was 1123
KB/hr. Linux amd64 grew 3MB (25→28MB) but slope was only 424 KB/hr.
The linear regression fits noise on short runs — small timing
fluctuations produce wildly different slopes on nearly flat data.

For 10-min quick soak: use RSS ceiling (200MB) + ratio (3.0x) +
FD drift + idle CPU. Slope is still reported but not enforced.
For 30+ min runs: slope enforcement at 500 KB/hr (enough samples
for reliable regression).
2026-03-25 21:17:56 +01:00
Martin Vogel 47592c6927 Soak: enhanced RSS leak detection with first/last ratio check 2026-03-25 16:16:07 +01:00
Martin Vogel aa25f73adb Soak test gaps: 200-file fixture, Windows, snapshots, ASan build
- Test project: 377 files (80 Python + 40 Go + 40 TSX + configs)
  instead of 5 files. RSS baseline now ~47MB (real workload).
- Snapshots every 10s (was 30s) — 7+ data points in 1-min run.
- Reindex every 2min compressed (was 5min) — more cycles per run.
- heap_committed fallback: use RSS when mimalloc reports 0.
- Windows soak: added to quick soak matrix (MSYS2 + python3 + git).
- ASan soak: builds with -fsanitize=address (was building release).
- Collect snapshot with single python3 call (was 6 separate calls).
2026-03-25 15:52:28 +01:00
Martin Vogel b6801295f2 Fix soak test: route queries through MCP server (not CLI) for real metrics 2026-03-25 15:33:54 +01:00
Martin Vogel dbd543a966 Add soak test suite: diagnostics + compressed workload + CI jobs
CBM_DIAGNOSTICS=1 writes /tmp/cbm-diagnostics-<pid>.json every 5s.
scripts/soak-test.sh: compressed workload + crash recovery + analysis.
CI: soak_level input (full/quick/none) on dry-run + release.
MCP query timing via cbm_diag_record_query().
2026-03-25 12:38:01 +01:00