337 Commits
Author SHA1 Message Date
Martin Vogel b31d177089 VT gate: all files block equally, no warnings — scripts can contain harmful code too v0.5.7 2026-03-26 13:26:57 +01:00
Martin Vogel 4aa1690022 VT gate: increase timeout to 120 min (was 20 min) 2026-03-26 13:24:53 +01:00
Martin Vogel 3087c2ab27 VT gate: warn (not block) when script files timeout (low VT priority) 2026-03-26 13:21:05 +01:00
Martin Vogel 061ec32fe3 Fix: normalize path separators on ALL platforms + require soak-asan for release 2026-03-26 12:13:48 +01:00
jimpark b23445bdea fix: normalize mixed path separators to forward slashes on Windows (#146)
Adds `cbm_normalize_path_sep()` to platform.h — converts backslash to
forward slash on Windows, no-op on POSIX. Applied at entry points:
- `cbm_get_home_dir()` — HOME/USERPROFILE paths
- `handle_index_repository()` — repo_path argument
- `build_snippet_response()` — _fullpath results
- FQN computation — replaces inline loops

Thanks @jimpark for the clean, focused fix!
2026-03-26 12:02:39 +01:00
Martin Vogel f52376b982 Update: 2586 tests, 66 languages everywhere, tre vendored hash 2026-03-26 11:53:35 +01:00
Martin Vogel 0c81278777 Raise soak query latency threshold to 60s (MSYS2 overhead) 2026-03-26 11:14:57 +01:00
Martin Vogel de2b55bfe1 Fix Windows smoke 12a: normalize MSYS2 uname to 'windows' for archive name 2026-03-26 10:47:07 +01:00
Martin Vogel d8f168cd39 Fix Windows smoke 14e + soak latency threshold
- Phase 14: copy binary with .exe suffix on Windows (was creating
  non-.exe copy that uninstall didn't remove)
- Phase 14e: check both .exe and non-.exe paths
- Soak latency: exclude index_repository from max query latency
  (indexing 377 files is legitimately slow on Windows/MSYS2)
2026-03-26 10:34:00 +01:00
Martin Vogel 988d975813 Fix uninstall .exe path on Windows + add Windows smoke/soak to local test infra 2026-03-26 00:41:40 +01:00
Martin Vogel 5ca369f313 Fix Windows smoke: skip OpenCode/Aider when binary not on PATH 2026-03-25 23:55:39 +01:00
Martin Vogel 14a197c165 Fix ALL MSYS2 python3 path issues: pipe via cat for every JSON read 2026-03-25 23:39:56 +01:00
Martin Vogel 331b4c4dfb Fix all Windows path comparisons: path_match helper with basename fallback 2026-03-25 23:22:57 +01:00
Martin Vogel 1ad2568d66 Fix smoke 8o: OpenCode command array + update picomatch (dependabot) 2026-03-25 23:08:02 +01:00
Martin Vogel c0de9ff5a4 Weekly soak: run Sunday 2am UTC instead of nightly 2026-03-25 22:58:05 +01:00
Martin Vogel b568884e4a Fix MSYS2 python3 path issue: pipe files via cat instead of open()
MinGW python3 is a native Windows binary that doesn't understand
POSIX paths like /tmp/foo.json. Piping file content through cat
(which runs in MSYS2 bash and handles path translation) to python's
stdin avoids the issue entirely.

Fixes Windows smoke test 8a and soak baseline collection.
2026-03-25 22:48:31 +01:00
Chitral Verma 5743e1a1f3 fix: opencode MCP config format (#134)
Fixes OpenCode config to match their documented format:
- `"enabled": true` 
- `"type": "local"`
- `"command": ["path"]` (array, not string)

Thanks @chitralverma for the fix and test coverage!
2026-03-25 22:47:37 +01:00
Martin Vogel 6d4d3246ad Fix Windows smoke 8a: basename comparison + debug output for path mismatch 2026-03-25 22:30:54 +01:00
Martin Vogel 18298d287e Add --skip-config to install.ps1 too (#145) 2026-03-25 22:26:32 +01:00
Martin Vogel 0c35f152a0 Add --skip-config flag to install.sh (#145)
Allows skipping automatic agent configuration after binary install.
Useful for custom setups or CI environments.

Usage: curl ... | bash -s -- --skip-config
2026-03-25 22:24:02 +01:00
Martin Vogel 38f699b72e Add nightly 4h soak test (scheduled 2am UTC, manual trigger) 2026-03-25 22:09:49 +01:00
Martin Vogel 7074dc2f1b Fix Windows: taskkill self-kill + soak missing build deps
1. taskkill /IM killed the install process itself — now uses
   /FI "PID ne <self>" to exclude own process
2. Windows soak jobs missing clang, zlib, make packages — added
   to all soak-quick-windows and soak-asan-windows MSYS2 installs
3. Both dry-run.yml and release.yml fixed
2026-03-25 21:53:58 +01:00
Martin Vogel 5f9da0ee4a Fix 3 CI failures: Windows .exe path, tre ssize_t, UI archive name
1. Windows install/update: append .exe to binary path on _WIN32
   (was writing empty command to .claude.json)
2. Vendored tre: add ssize_t typedef for Windows (basetsd.h SSIZE_T)
3. Smoke server: copy UI archive under standard name too, so
   install.sh and update --standard work in UI variant smoke tests
2026-03-25 21:33:01 +01:00
Martin Vogel adb14ee245 Soak: enforce RSS slope only for 30+ min runs (10-min too noisy)
Investigation: macOS arm64 grew 1MB (43→44MB) but slope was 1123
KB/hr. Linux amd64 grew 3MB (25→28MB) but slope was only 424 KB/hr.
The linear regression fits noise on short runs — small timing
fluctuations produce wildly different slopes on nearly flat data.

For 10-min quick soak: use RSS ceiling (200MB) + ratio (3.0x) +
FD drift + idle CPU. Slope is still reported but not enforced.
For 30+ min runs: slope enforcement at 500 KB/hr (enough samples
for reliable regression).
2026-03-25 21:17:56 +01:00
Martin Vogel 0d4aa3fc19 Fix skip cascade: add !cancelled() to all smoke/soak job conditions 2026-03-25 16:52:41 +01:00
Martin Vogel 6a26f52e23 Fix workflow: use explicit != true for boolean inputs (unset != false) 2026-03-25 16:38:23 +01:00
Martin Vogel 47592c6927 Soak: enhanced RSS leak detection with first/last ratio check 2026-03-25 16:16:07 +01:00
Martin Vogel 6ab88f9cc3 Add Windows ASan soak (no LeakSan, catches buffer overflows + UAF) 2026-03-25 16:10:06 +01:00
Martin Vogel 817a1fc3ce Fix soak CI: parallel with smoke, all platforms, ASan on macOS
- soak-quick + soak-asan now need build-* (not smoke-*) — parallel
- Added macOS Intel (amd64) to both soak-quick and soak-asan matrices
- Added macOS arm64 + Intel to soak-asan (was Linux-only)
- Windows soak: separate job (matrix.shell not supported)
- ASan not on Windows (LeakSanitizer unsupported there)
2026-03-25 16:06:07 +01:00
Martin Vogel 4a49097ba9 Fix workflow: split Windows soak into own job (matrix.shell not supported) 2026-03-25 15:58:50 +01:00
Martin Vogel aa25f73adb Soak test gaps: 200-file fixture, Windows, snapshots, ASan build
- Test project: 377 files (80 Python + 40 Go + 40 TSX + configs)
  instead of 5 files. RSS baseline now ~47MB (real workload).
- Snapshots every 10s (was 30s) — 7+ data points in 1-min run.
- Reindex every 2min compressed (was 5min) — more cycles per run.
- heap_committed fallback: use RSS when mimalloc reports 0.
- Windows soak: added to quick soak matrix (MSYS2 + python3 + git).
- ASan soak: builds with -fsanitize=address (was building release).
- Collect snapshot with single python3 call (was 6 separate calls).
2026-03-25 15:52:28 +01:00
Martin Vogel 3a457d9458 Fix smoke: Windows .exe binary name, UI variant archive fallback 2026-03-25 15:38:36 +01:00
Martin Vogel b6801295f2 Fix soak test: route queries through MCP server (not CLI) for real metrics 2026-03-25 15:33:54 +01:00
Martin Vogel dbd543a966 Add soak test suite: diagnostics + compressed workload + CI jobs
CBM_DIAGNOSTICS=1 writes /tmp/cbm-diagnostics-<pid>.json every 5s.
scripts/soak-test.sh: compressed workload + crash recovery + analysis.
CI: soak_level input (full/quick/none) on dry-run + release.
MCP query timing via cbm_diag_record_query().
2026-03-25 12:38:01 +01:00
Martin Vogel 7d640081e0 UI smoke: verify HTML content in GET / and JSON-RPC in POST /rpc 2026-03-25 10:48:40 +01:00
Martin Vogel 6300feb70c Fix smoke tests: variant detection, Windows deps, UI reachability
- Phase 14: detect UI vs standard variant from HTTP server contents
  instead of hardcoding --standard (fixes UI smoke failure)
- Windows smoke: install zip + coreutils (sha256sum) in MSYS2
  (fixes exit code 127)
- Phase 15: UI HTTP server reachability test — verifies root returns
  200 and /rpc accepts POST (skips gracefully for non-UI builds)
- Fix switch fallthrough in release.yml (same MSYS2 deps)
2026-03-25 10:45:58 +01:00
Martin Vogel 65a38be478 Fix Windows test failures: backslash allowed, skip shell-dependent test 2026-03-25 10:12:44 +01:00
Martin Vogel 4c2e74dc99 Fix switch fallthrough: return false before #ifndef _WIN32 backslash case 2026-03-25 09:47:59 +01:00
Martin Vogel 8f56f3d81a Fix Windows build: add missing stdint.h include in test_dyn_array 2026-03-25 09:28:11 +01:00
Martin Vogel e6e9c58e80 Fix all memory leaks + Docker test infra gaps
Leaks fixed (5506 bytes → 0):
- sqlite3_open_v2 allocates handle even on CANTOPEN — must call
  sqlite3_close before freeing struct (4x 792 bytes, 32 indirect)
- graph_buffer node_by_id: reuse existing heap key on hash table
  replace to avoid leak (2 bytes)

Docker test infrastructure:
- python3 (full) instead of python3-minimal (missing json module)
- git installed (19 tests were silently skipped)
- sqlite3_close_v2 + sqlite3_shutdown at test exit
2026-03-25 01:59:36 +01:00
Martin Vogel b7e076c5d5 Fix Linux build: guard cbm_macos_adhoc_sign with #ifdef __APPLE__ 2026-03-25 00:12:10 +01:00
Martin Vogel fa97ea5b09 Fix CodeQL TOCTOU: use fchmod before fclose on gate script
CodeQL cpp/toctou-race-condition #32: fopen then chmod on the same
path allows a race where the file could be swapped between write
and chmod. Fix: use fchmod(fileno(f)) before fclose on POSIX.
Windows falls back to chmod (no fchmod).
2026-03-25 00:04:28 +01:00
Martin Vogel aa5237b43b Include install scripts in release archives + VirusTotal scan
tar.gz archives now contain install.sh alongside the binary.
zip archives now contain install.ps1.

Users who download an archive get everything needed to install
without fetching scripts separately from the repo.

VirusTotal scan also covers the scripts since they're extracted
from archives into the scan directory.
2026-03-24 23:55:49 +01:00
Martin Vogel 7f579f91ad Add 480+ tests, fix 3 bugs discovered by new test coverage
New tests (6481 lines across 23 files):
- Arena: reset invariants, growth, sized init, destroy safety
- Dynamic array: capacity, remove, clear, grow patterns
- FQN: module/package/class name computation (592 lines)
- Graph buffer: node/edge CRUD, dedup, merge, concurrent access
- Hash table: collision, resize, iteration, edge cases
- MCP: tool dispatch, argument parsing, error handling
- Memory: allocation tracking, collection, limits
- Pipeline: test detection (path + func name patterns)
- Store: node/edge CRUD, search, batch ops, persistence
- String: intern dedup, util edge cases, YAML parsing (1105 lines)
- Traces: ingestion, dedup, query
- Watcher: baseline, polling, change detection
- Worker pool: task distribution, completion

Bugs found and fixed:
1. Arena reset overflow: block_size not reset to block_sizes[0]
   after growth — subsequent allocations could overflow blocks[0]
2. Test path detection: missing __tests__/, tests/, spec/ directory
   patterns and _spec.rb suffix
3. Test func detection: "Testable" falsely matched as Go test —
   now requires uppercase or end-of-string after "Test" prefix
2026-03-24 23:48:23 +01:00
Martin Vogel cbac7be1e0 Fix #139 #133 #137: thread stack, Windows paths, detect_changes
#139 Stack overflow in autoindex: all threads now get 8MB stack
by default (CBM_DEFAULT_STACK_SIZE). macOS ARM64 default was
512KB, too small for deep pipeline passes like configlink.

#133 search_code rejects Windows backslash paths: allow backslash
in cbm_validate_shell_arg on _WIN32 builds.

#137 detect_changes fails on paths with spaces: replace cd with
git -C flag, use double quotes on Windows.

Also fix arena test: allocations must exceed 64KB block size to
force multiple blocks (test was always wrong, never caught because
it wasn't in the test suite until now).
2026-03-24 23:14:33 +01:00
Martin Vogel eaf9c83066 Eliminate rename() — reindex deletes old DB, index writes directly
Architecture change to prevent concurrent write corruption:

  Indexing (new project): cbm_write_db writes project.db directly
  Reindexing (existing): delete project.db, then index as new
  Incremental: upsert changes into existing project.db (unchanged)

Root cause: rename(project.db.tmp, project.db) while watcher/MCP
had the old file open via SQLite caused WAL corruption — stale
connections wrote to deleted inodes.

Changes:
- cbm_gbuf_dump_to_sqlite: write to final path, no .tmp + rename
- pipeline.c: if DB exists but not eligible for incremental, delete
  it before proceeding with full index (reindex = delete + index)
- Pipeline lock (atomic spinlock) serializes concurrent runs:
  MCP/autoindex block, watcher skips if busy
- delete_project acquires pipeline lock before unlinking
- BEGIN IMMEDIATE for all write transactions (respects busy_timeout)
- cbm_store_check_integrity: detects corrupt DBs (>5 project rows
  or invalid root_path), auto-deletes with ERROR log
- cbm_store_get_db accessor for test corruption simulation

Tests: 5 integrity tests + 4 concurrency lock tests
2026-03-24 23:14:33 +01:00
Martin Vogel 68cc19ebf2 Auto-detect and clean corrupt index databases
Add cbm_store_check_integrity() that validates:
- projects table has <= 5 rows (1 project per .db file)
- root_path starts with / or drive letter (not numeric garbage)

Wire into resolve_store (MCP queries) and pipeline routing:
- MCP: corrupt DB auto-deleted with ERROR log, returns "not indexed"
- Pipeline: corrupt DB deleted, falls through to full reindex

Root cause: user-reported corrupt DBs had 48K rows in projects table
with node IDs in text columns. Exact write path TBD, but detection
+ auto-clean prevents the symptom from persisting.

5 unit tests: clean DB, empty DB, bad path, too many rows, NULL.
2026-03-24 23:14:33 +01:00
Martin Vogel abe319b46c Real E2E update/uninstall + CBM_DOWNLOAD_URL support
Add CBM_DOWNLOAD_URL env var to cbm_cmd_update and checksum
verification — allows E2E testing against local HTTP server.

Phase 14: runs actual update --standard -y against local server,
verifies binary replaced, agent configs refreshed (stale path
updated), then real uninstall -y verifies binary removed and
configs cleaned.

Phase 13: add PATH setup verification. Non-interactive stdin:
isatty() prevents silent hangs.
2026-03-24 23:14:33 +01:00
Martin Vogel da7638c651 Prevent silent hangs on non-interactive stdin
Add isatty() checks to prompt_yn() and update variant chooser.
When stdin is not a terminal and no -y/-n or --standard/--ui flag
is provided, print a clear error and exit instead of hanging on
fgets(). Prevents agents from silently blocking when running
install/update programmatically.

Add smoke test 9b-9: verify non-interactive update fails cleanly.
2026-03-24 23:14:33 +01:00
Martin Vogel 8951647d3e Fix remaining install/update gaps + in-memory zip extraction
Install command:
- Add cbm_kill_other_instances() to kill stale MCP servers
- Add cbm_macos_adhoc_sign() to sign binary if placed unsigned

Update command:
- Replace skills-only reinstall with full cbm_install_agent_configs()
- Replace external unzip with cbm_extract_binary_from_zip() via zlib

Refactor: extract 10-agent config loop into cbm_install_agent_configs()
called by both install and update.

New: cbm_extract_binary_from_zip() — in-memory zip extraction with
stored + deflate support, path traversal rejection, bounds checks.
4 unit tests. Smoke: add install.ps1 E2E for Windows.
2026-03-24 23:14:33 +01:00