Commit Graph
61 Commits
Author SHA1 Message Date
Martin Vogel f52376b982 Update: 2586 tests, 66 languages everywhere, tre vendored hash 2026-03-26 11:53:35 +01:00
Martin Vogel 0c81278777 Raise soak query latency threshold to 60s (MSYS2 overhead) 2026-03-26 11:14:57 +01:00
Martin Vogel de2b55bfe1 Fix Windows smoke 12a: normalize MSYS2 uname to 'windows' for archive name 2026-03-26 10:47:07 +01:00
Martin Vogel d8f168cd39 Fix Windows smoke 14e + soak latency threshold
- Phase 14: copy binary with .exe suffix on Windows (was creating
  non-.exe copy that uninstall didn't remove)
- Phase 14e: check both .exe and non-.exe paths
- Soak latency: exclude index_repository from max query latency
  (indexing 377 files is legitimately slow on Windows/MSYS2)
2026-03-26 10:34:00 +01:00
Martin Vogel 5ca369f313 Fix Windows smoke: skip OpenCode/Aider when binary not on PATH 2026-03-25 23:55:39 +01:00
Martin Vogel 14a197c165 Fix ALL MSYS2 python3 path issues: pipe via cat for every JSON read 2026-03-25 23:39:56 +01:00
Martin Vogel 331b4c4dfb Fix all Windows path comparisons: path_match helper with basename fallback 2026-03-25 23:22:57 +01:00
Martin Vogel c0de9ff5a4 Weekly soak: run Sunday 2am UTC instead of nightly 2026-03-25 22:58:05 +01:00
Martin Vogel b568884e4a Fix MSYS2 python3 path issue: pipe files via cat instead of open()
MinGW python3 is a native Windows binary that doesn't understand
POSIX paths like /tmp/foo.json. Piping file content through cat
(which runs in MSYS2 bash and handles path translation) to python's
stdin avoids the issue entirely.

Fixes Windows smoke test 8a and soak baseline collection.
2026-03-25 22:48:31 +01:00
Martin Vogel 6d4d3246ad Fix Windows smoke 8a: basename comparison + debug output for path mismatch 2026-03-25 22:30:54 +01:00
Martin Vogel adb14ee245 Soak: enforce RSS slope only for 30+ min runs (10-min too noisy)
Investigation: macOS arm64 grew 1MB (43→44MB) but slope was 1123
KB/hr. Linux amd64 grew 3MB (25→28MB) but slope was only 424 KB/hr.
The linear regression fits noise on short runs — small timing
fluctuations produce wildly different slopes on nearly flat data.

For 10-min quick soak: use RSS ceiling (200MB) + ratio (3.0x) +
FD drift + idle CPU. Slope is still reported but not enforced.
For 30+ min runs: slope enforcement at 500 KB/hr (enough samples
for reliable regression).
2026-03-25 21:17:56 +01:00
Martin Vogel 47592c6927 Soak: enhanced RSS leak detection with first/last ratio check 2026-03-25 16:16:07 +01:00
Martin Vogel aa25f73adb Soak test gaps: 200-file fixture, Windows, snapshots, ASan build
- Test project: 377 files (80 Python + 40 Go + 40 TSX + configs)
  instead of 5 files. RSS baseline now ~47MB (real workload).
- Snapshots every 10s (was 30s) — 7+ data points in 1-min run.
- Reindex every 2min compressed (was 5min) — more cycles per run.
- heap_committed fallback: use RSS when mimalloc reports 0.
- Windows soak: added to quick soak matrix (MSYS2 + python3 + git).
- ASan soak: builds with -fsanitize=address (was building release).
- Collect snapshot with single python3 call (was 6 separate calls).
2026-03-25 15:52:28 +01:00
Martin Vogel 3a457d9458 Fix smoke: Windows .exe binary name, UI variant archive fallback 2026-03-25 15:38:36 +01:00
Martin Vogel b6801295f2 Fix soak test: route queries through MCP server (not CLI) for real metrics 2026-03-25 15:33:54 +01:00
Martin Vogel dbd543a966 Add soak test suite: diagnostics + compressed workload + CI jobs
CBM_DIAGNOSTICS=1 writes /tmp/cbm-diagnostics-<pid>.json every 5s.
scripts/soak-test.sh: compressed workload + crash recovery + analysis.
CI: soak_level input (full/quick/none) on dry-run + release.
MCP query timing via cbm_diag_record_query().
2026-03-25 12:38:01 +01:00
Martin Vogel 7d640081e0 UI smoke: verify HTML content in GET / and JSON-RPC in POST /rpc 2026-03-25 10:48:40 +01:00
Martin Vogel 6300feb70c Fix smoke tests: variant detection, Windows deps, UI reachability
- Phase 14: detect UI vs standard variant from HTTP server contents
  instead of hardcoding --standard (fixes UI smoke failure)
- Windows smoke: install zip + coreutils (sha256sum) in MSYS2
  (fixes exit code 127)
- Phase 15: UI HTTP server reachability test — verifies root returns
  200 and /rpc accepts POST (skips gracefully for non-UI builds)
- Fix switch fallthrough in release.yml (same MSYS2 deps)
2026-03-25 10:45:58 +01:00
Martin Vogel abe319b46c Real E2E update/uninstall + CBM_DOWNLOAD_URL support
Add CBM_DOWNLOAD_URL env var to cbm_cmd_update and checksum
verification — allows E2E testing against local HTTP server.

Phase 14: runs actual update --standard -y against local server,
verifies binary replaced, agent configs refreshed (stale path
updated), then real uninstall -y verifies binary removed and
configs cleaned.

Phase 13: add PATH setup verification. Non-interactive stdin:
isatty() prevents silent hangs.
2026-03-24 23:14:33 +01:00
Martin Vogel da7638c651 Prevent silent hangs on non-interactive stdin
Add isatty() checks to prompt_yn() and update variant chooser.
When stdin is not a terminal and no -y/-n or --standard/--ui flag
is provided, print a clear error and exit instead of hanging on
fgets(). Prevents agents from silently blocking when running
install/update programmatically.

Add smoke test 9b-9: verify non-interactive update fails cleanly.
2026-03-24 23:14:33 +01:00
Martin Vogel 8951647d3e Fix remaining install/update gaps + in-memory zip extraction
Install command:
- Add cbm_kill_other_instances() to kill stale MCP servers
- Add cbm_macos_adhoc_sign() to sign binary if placed unsigned

Update command:
- Replace skills-only reinstall with full cbm_install_agent_configs()
- Replace external unzip with cbm_extract_binary_from_zip() via zlib

Refactor: extract 10-agent config loop into cbm_install_agent_configs()
called by both install and update.

New: cbm_extract_binary_from_zip() — in-memory zip extraction with
stored + deflate support, path traversal rejection, bounds checks.
4 unit tests. Smoke: add install.ps1 E2E for Windows.
2026-03-24 23:14:33 +01:00
Martin Vogel 006e6dbf20 Add install scripts, CI pre-signing, download E2E smoke tests
install.sh: one-liner for macOS/Linux — detects OS/arch (Rosetta-
aware), downloads release, verifies checksum, extracts, signs on
macOS, runs install -y for all 10 agents. Supports --ui flag and
CBM_DOWNLOAD_URL env var for testing.

install.ps1: one-liner for Windows — Invoke-WebRequest + Expand-
Archive + Unblock-File (strips MOTW), installs to %LOCALAPPDATA%,
adds to user PATH via [Environment]::SetEnvironmentVariable.

CI pre-signing: add codesign --sign - step for macOS builds in
both dry-run.yml and release.yml, before archiving. Release
binaries now ship pre-signed.

Phase 12 smoke tests: real HTTP download via local artifact server,
checksum verification, archive extraction, binary verification.
Runs only when SMOKE_DOWNLOAD_URL is set (CI provides it).

Phase 13 smoke tests: install.sh E2E — runs full script with local
URL + isolated HOME, verifies binary placed, signed, runs, and
agent configs created.

CI HTTP server: smoke jobs start python3 HTTP server serving the
built binary as a tar.gz/zip archive + checksums.txt. Enables
Phases 12-13 in CI on all platforms.

Update security allowlist: remove system() entry (eliminated),
add cbm_popen for pgrep. Update README with one-liner Quick Start.
2026-03-24 23:14:33 +01:00
Martin Vogel d2451c4fd6 Add comprehensive E2E smoke tests for install/update/uninstall
Phase 8: Agent config install E2E — creates stub environment with
all 10 agent detection dirs, runs install -y with isolated HOME,
verifies exact JSON structure, TOML content, hook format, skills,
and merge-not-overwrite behavior for every agent config.

Phase 9: Agent config uninstall E2E — verifies complete removal of
all MCP entries, hooks, instructions while preserving user's existing
config keys. Adversarial tests: idempotent double install, uninstall
without prior install, corrupt JSON/TOML handling, double uninstall.

Phase 10: Platform binary security — macOS: codesign verify, strip
signature, SIGKILL on unsigned arm64 (exit 137), re-sign + verify.
Linux/Windows: verify unsigned binary runs without signing.

Phase 11: Process kill — start MCP server, verify running, kill,
verify gone.

Phase 14: Update flow — binary replacement with signing, read-only
binary edge case, platform-specific verification.

Fix cbm_find_cli: add S_IXUSR check for hardcoded path search
(was missing, only checked for PATH-based search).
2026-03-24 23:14:33 +01:00
Martin Vogel c00723e5c8 Require project param on all MCP tool calls
Make project a required parameter for all query tools (search_graph,
query_graph, trace_call_path, get_code_snippet, get_graph_schema,
get_architecture, search_code, index_status, detect_changes,
manage_adr, ingest_traces). Removes implicit fallback to session
project or last-opened store.

When project is missing or not found, return error with list of
available indexed projects so agents can self-correct.

Rename delete_project param from project_name to project for
consistency. Fix smoke test trace_call_path depth param name
(max_depth -> depth).
2026-03-24 13:38:25 +01:00
Martin Vogel f24475ca4a Add simulated binary replacement to smoke tests (Phase 6e)
Tests the update command's replacement flow without network calls:
copy binary as "downloaded" version, unlink-then-replace the
"installed" version, verify replaced binary runs --version.
Also tests read-only binary replacement edge case (#114).
2026-03-23 22:02:51 +01:00
Martin Vogel 20814e8b27 Add smoke tests to Docker test infra + CI pipeline
Expand smoke-test.sh with Phase 5 (MCP stdio transport), Phase 6
(CLI subcommands: install/uninstall/update --dry-run), and Phase 7
(MCP advanced tool calls: search_code v2, get_code_snippet).

Add smoke/smoke-amd64 services to Docker compose that build then
run all 7 smoke test phases. Include in run.sh full/all flows.

Add python3-minimal to Dockerfile for smoke test JSON parsing.
Fix Phase 4a shutdown test to use portable background+kill pattern
instead of `timeout` (not available on macOS).

Add --dry-run, --standard, --ui flags to update command. Fix
clang-tidy readability-implicit-bool-conversion in dry_run ternary.
2026-03-23 21:56:33 +01:00
Martin Vogel 85da0908f6 Add MCP stdio transport smoke tests (Phase 5)
Tests the actual agent handshake path that Claude Code, OpenCode, etc. use:
- 5a-d: initialize → notifications/initialized → tools/list via bare JSONL
- 5e: Full tool call round-trip (index + search) via JSON-RPC
- 5f: Content-Length framing (OpenCode compatibility)

Uses portable background process + kill instead of timeout command
(not available on all macOS configurations).
2026-03-23 18:33:27 +01:00
Martin Vogel de322d98a3 Merge branch 'main' into fix/mcp-stdio-buffering 2026-03-23 14:20:15 +01:00
Martin Vogel c38853f999 Remove Go-era cruft, update THIRD_PARTY.md for pure C project
- Delete cmd/codebase-memory-mcp/assets/ — skills and codex instructions
  are embedded as C string literals in src/cli/cli.c since v0.5.0
- Move BENCHMARK.md to docs/ to reduce root clutter
- Fix setup scripts: replace stale go build with scripts/build.sh
- THIRD_PARTY.md: replace Go module dependencies with vendored C
  libraries (sqlite3, mimalloc, mongoose, yyjson, xxhash, tre),
  fix CGo reference to static compilation
2026-03-22 01:09:12 +01:00
Martin Vogel 0888726073 Add security defense tests + eliminate system() command injection
TDD: 31 security tests covering shell injection prevention, SQLite
authorizer (ATTACH/DETACH blocked), SQL injection via Cypher, path
containment, and shell-free subprocess execution.

- Add cbm_exec_no_shell() in compat_fs: fork+execvp (POSIX), _spawnvp
  (Windows) — executes commands without shell interpretation
- Replace system() with cbm_exec_no_shell() for unzip extraction and
  version verification in update command — eliminates CodeQL
  command-line-injection alerts
- CodeQL: switch to build-mode manual for 100% source file coverage
- CodeQL gate: fix race condition between scan completion and alert API
  propagation (60s settle + double-check polling)
- Dismiss TOCTOU in pass_envscan.c (benign read-only directory walk)
2026-03-21 23:02:34 +01:00
Martin Vogel aa2b60be7d Fix 29 CodeQL alerts: command injection, snprintf overflow, TOCTOU races
- Command injection (CRITICAL): validate shell args before system() in
  update command's unzip and version-check calls
- TOCTOU cli.c: use open(O_CREAT, 0755) + fdopen() to set permissions
  atomically instead of fopen() + chmod() after close
- TOCTOU pass_envscan.c: open file first, then fstat() on fd to check
  size, eliminating stat-then-open race window
- Overflowing snprintf (11 locations): clamp offset after each append
  to prevent unsigned underflow on truncation in cypher.c, store.c,
  http_server.c, test_c_lsp.c
- Add CBM_SNPRINTF_APPEND macro in str_util.h for future safe appends
- CodeQL: remove pull_request trigger (only scan push to main)
- CodeQL gate: increase timeout from 30 to 45 minutes
- Add fuzz testing script (random JSON-RPC + Cypher mutations)
- 12 Scorecard governance alerts dismissed (not code vulnerabilities)
2026-03-21 22:18:31 +01:00
Shane McCarronandClaude Sonnet 4.6 5ca5f3291f fix(mcp): address QA round 1 findings
- Use O_NONBLOCK + clearerr() in Phase 2 fgetc probe to preserve the
  60s idle eviction timeout when both kernel fd and FILE* buffer are
  empty (fgetc on a blocking fd would otherwise block indefinitely,
  bypassing Phase 3 poll timeout and preventing cbm_mcp_server_evict_idle)
- Add #include <fcntl.h> for fcntl()/O_NONBLOCK
- Fix comment: "two-phase" → "three-phase" (implementation has 3 phases)
- Improve Python integration test: verify id:1 (initialize) and id:2
  (tools/list) response IDs are both present, not just "tools" substring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 12:37:28 -05:00
Shane McCarron 16552befdd test(mcp): add rapid-init integration test for poll/getline buffering fix 2026-03-21 12:03:52 -05:00
Martin Vogel dd30132e2d Pin Actions to SHA + enforce via security audit + Dependabot
- All GitHub Actions pinned to immutable commit SHAs (prevents
  tag-poisoning attacks like tj-actions/changed-files incident)
- Security audit (Layer 1) now blocks unpinned Actions in CI
- Dependabot configured to auto-propose SHA updates weekly
- Pre-commit hooks tracked in scripts/hooks/ for contributors
- Time-bomb detection + MCP file read audit added to Layer 1
2026-03-21 16:57:17 +01:00
Martin Vogel c371e9f07d Add time-bomb + MCP file read audit, tracked pre-commit hooks
Security audit (Layer 1) new checks:
- Time-bomb scan: flags time()/sleep()/clock() near dangerous calls
- MCP file read audit: tracks fopen/fread count in mcp.c against
  expected max (detects data exfiltration through tool responses)

Pre-commit hooks now tracked in scripts/hooks/:
- Contributors activate with: git config core.hooksPath scripts/hooks
- Runs: lint → security audit → build + test
- setup.sh already configures this automatically
2026-03-21 16:43:48 +01:00
Martin Vogel f7315b9bc5 Robust UI audit: skip inline URL scan on dist/, keep structural checks
Minified JS in dist/ contains hundreds of string-constant URLs from
bundled npm libraries (React error pages, W3C namespace URIs, CDN
references, OSS credits) that are never fetched at runtime. Scanning
them produced an endless stream of false positives requiring per-URL
allowlisting.

Fix: split the A1 URL check by directory:
- src/ (our code): strict — any external URL is blocked
- dist/ (npm build output): skip inline URL scan entirely

Structural checks (A2-A6) still apply to dist/:
- External <script>/<link> loads in HTML
- Tracking/analytics identifiers
- Iframes, eval/Function, external WebSockets
2026-03-20 20:56:52 +01:00
Martin Vogel a9382ce110 Fix UI audit: allow truncated CDN URL in dist/ scan 2026-03-20 20:52:58 +01:00
Martin Vogel 93d332d1e0 Fix security audit false positives in CI
- UI audit (L6): allowlist bundled framework URLs in dist/ (React error
  URLs, W3C namespace URIs, Three.js credits, Google Fonts, Tailwind).
  These are embedded by npm deps during Vite build, not our code.
- Binary strings (L2): skip URLs shorter than 15 chars — Windows binary
  has byte sequences that strings(1) interprets as "https://H9" etc.
- Allow Google Fonts <link> in HTML (loaded by index.html for Inter/
  JetBrains Mono fonts).
2026-03-20 18:47:55 +01:00
Martin Vogel cd1417427c Add 8-layer security test suite + hardening
Code-level defenses:
- cbm_validate_shell_arg(): reject shell metacharacters before popen/system
- SQLite authorizer: block ATTACH/DETACH at engine level
- CORS localhost-only origin reflection (replaces wildcard *)
- Path containment: realpath() check in get_code_snippet
- process-kill restricted to server-spawned PIDs
- SHA256 checksum verification in update command

Security audit scripts (8 layers):
- L1: Static allow-list for dangerous calls + URLs
- L2: Binary string audit (URLs, payloads, credentials)
- L3: Network egress monitoring via strace (Linux)
- L4: Install output path + content validation
- L5: Smoke test hardening (clean shutdown, residual procs)
- L6: Graph UI audit (external domains, CORS, binding)
- L7: MCP robustness (23 adversarial JSON-RPC payloads)
- L8: Vendored integrity (checksums + dangerous call scan)

CI: parallel security-static job (no build needed), binary
layers in smoke jobs per-platform. Cleanup of test fixture
dirs in clean.sh + .gitignore.
2026-03-20 18:19:15 +01:00
Martin Vogel 0f3c772014 Fix Windows smoke: convert MSYS2 POSIX paths for native binary
MSYS2 mktemp returns /tmp/tmp.XXX but the Windows binary can't resolve
POSIX paths. Use cygpath -m to convert to C:/msys64/tmp/tmp.XXX which
works in both MSYS2 shell and Windows file APIs.
2026-03-19 12:29:54 +01:00
Martin Vogel a71a65f68e Fix smoke tests: CLANG64 runtime for Windows, debug stderr on error
1. smoke-windows: Use CLANG64 (not UCRT64) so runtime matches the
   build environment. Binary built with CLANG64 needs CLANG64 DLLs.
2. smoke-unix: Add fail-fast: false so one platform failure doesn't
   cancel all 7 other smoke jobs.
3. smoke-test.sh: Capture and display stderr on indexing failure to
   help debug smoke-unix errors.
4. release.yml: Same CLANG64 smoke fix.
2026-03-19 11:31:12 +01:00
Martin Vogel 0e9a47177b Fix Windows build: add zip pkg, portable embed (od replaces xxd)
1. Add zip to MSYS2 build packages (both dry-run and release).
2. embed-frontend.sh: replace xxd with od+sed for hex dump — xxd
   requires vim which isn't in MSYS2 CLANG64 by default.
3. embed-frontend.sh: flip IS_MACOS→IS_LINUX — Linux gets ld -r -b
   binary (ELF-only), everything else (macOS, Windows) gets the
   portable C-array approach via od.
4. release.yml: same Windows archive fix (msys2 shell + .exe detect).
2026-03-19 10:59:14 +01:00
Martin Vogel 89dd3bb7c6 Add skip_tests/skip_builds flags, fix build-stage failures
1. dry-run.yml: Add skip_tests and skip_builds workflow_dispatch inputs
   for faster iteration. Build jobs use always() to run even when tests
   are skipped.

2. Windows archive: Switch from PowerShell Copy-Item to MSYS2 shell
   with .exe extension detection. MSYS2 Clang produces .exe suffix
   that Copy-Item didn't account for.

3. Linux embed: Use absolute path for ld output when cd'd into dist
   directory. Relative path build/c/embedded/*.o was invalid from
   inside graph-ui/dist/.

4. release.yml: Same Windows archive fix as dry-run.
2026-03-19 10:43:32 +01:00
Martin Vogel 0f83f08653 Auto-detect WIN32_LIBS in Makefile, remove shell quoting entirely 2026-03-19 00:00:19 +01:00
Martin Vogel b5f14f115d Fix WIN32_LIBS quoting via env vars, add skip_lint to dry-run
- Use env: block instead of shell quoting for WIN32_LIBS in CI
- Forward WIN32_LIBS from environment in test.sh and build.sh
- Add skip_lint input to dry-run workflow for faster iteration
2026-03-18 23:35:35 +01:00
Martin Vogel 30505ec596 Switch Windows CI to CLANG64 for ASan/UBSan support
MinGW GCC does not ship ASan runtime on Windows. MSYS2 CLANG64
provides full sanitizer support via compiler-rt. All platforms
now run tests with ASan + UBSan — no exceptions.
2026-03-18 21:21:30 +01:00
Martin Vogel c8bdf924e5 Fix production build errors, add build step to Docker test infra
- Fix GCC -O2 warnings: calloc for cf (maybe-uninitialized),
  suppress stringop-truncation/alloc-size-larger-than (false positives),
  guard louvain against negative node_count
- Default compiler: cc on macOS, gcc on Linux/Windows (fixes gcc-14
  not found on MSYS2)
- Docker test infra now runs BOTH test (ASan) and build (-O2 -Werror)
2026-03-18 17:15:46 +01:00
Martin Vogel 8f225a37dd Use cc (Apple Clang) on macOS, gcc on Linux — unified CI/local parity
- Add scripts/env.sh: detects true hardware arch (even under Rosetta),
  verifies compiler supports target arch, adds arch prefix on macOS
- macOS uses cc/c++ everywhere (tests + builds) — ASan works natively
- Linux uses gcc/g++ everywhere — full sanitizer support
- Remove broken CC ?= gcc-14 from Makefile (GNU Make built-in overrides it)
- Scripts accept --arch flag and CC=/CXX= overrides
2026-03-18 13:08:33 +01:00
Martin Vogel fd1640ec29 Add build scripts, fix GCC warnings, unify CI workflows
- Add scripts/clean.sh, test.sh, build.sh, lint.sh as single source
  of truth for local and CI builds (always clean, always parallel)
- Fix -Wmissing-braces: replace {{0}} with memset for struct arrays
- Fix -Wrestrict: use intermediate buffer for same-struct snprintf
- Update dry-run.yml and release.yml to use the new scripts
2026-03-18 12:31:53 +01:00
Martin Vogel ce96ecc085 Add comprehensive smoke test: version + index + search + trace + folder integrity 2026-03-17 22:25:26 +01:00