68 Commits
Author SHA1 Message Date
Martin Vogel 0888726073 Add security defense tests + eliminate system() command injection
TDD: 31 security tests covering shell injection prevention, SQLite
authorizer (ATTACH/DETACH blocked), SQL injection via Cypher, path
containment, and shell-free subprocess execution.

- Add cbm_exec_no_shell() in compat_fs: fork+execvp (POSIX), _spawnvp
  (Windows) — executes commands without shell interpretation
- Replace system() with cbm_exec_no_shell() for unzip extraction and
  version verification in update command — eliminates CodeQL
  command-line-injection alerts
- CodeQL: switch to build-mode manual for 100% source file coverage
- CodeQL gate: fix race condition between scan completion and alert API
  propagation (60s settle + double-check polling)
- Dismiss TOCTOU in pass_envscan.c (benign read-only directory walk)
2026-03-21 23:02:34 +01:00
Martin Vogel aa2b60be7d Fix 29 CodeQL alerts: command injection, snprintf overflow, TOCTOU races
- Command injection (CRITICAL): validate shell args before system() in
  update command's unzip and version-check calls
- TOCTOU cli.c: use open(O_CREAT, 0755) + fdopen() to set permissions
  atomically instead of fopen() + chmod() after close
- TOCTOU pass_envscan.c: open file first, then fstat() on fd to check
  size, eliminating stat-then-open race window
- Overflowing snprintf (11 locations): clamp offset after each append
  to prevent unsigned underflow on truncation in cypher.c, store.c,
  http_server.c, test_c_lsp.c
- Add CBM_SNPRINTF_APPEND macro in str_util.h for future safe appends
- CodeQL: remove pull_request trigger (only scan push to main)
- CodeQL gate: increase timeout from 30 to 45 minutes
- Add fuzz testing script (random JSON-RPC + Cypher mutations)
- 12 Scorecard governance alerts dismissed (not code vulnerabilities)
2026-03-21 22:18:31 +01:00
Martin Vogel 4322116349 CodeQL gate: wait for scan on current commit (max 30 min) 2026-03-21 20:51:35 +01:00
Martin Vogel f3e4eeafb1 Add CodeQL SAST + Scorecard + badges + LICENSE in archives 2026-03-21 20:44:31 +01:00
Martin Vogel 6bc40c42e3 Include LICENSE file in all release archives 2026-03-21 20:23:42 +01:00
Martin Vogel 785748cbdb Add OpenSSF Scorecard workflow (weekly + push to main) 2026-03-21 20:14:55 +01:00
Martin Vogel 6e0ad6bc61 Scorecard: threshold gate (>=4.0) instead of inline report 2026-03-21 20:11:02 +01:00
Martin Vogel d974b50304 VirusTotal: scan extracted binaries, not archives
Archives >3MB may not be unpacked by VirusTotal, meaning only the
container gets scanned (meaningless). Now extracts the actual
executables from tar.gz/zip before uploading to VirusTotal.
Users still download archives (preserves Unix permissions).
Also fixes analysis ID parsing (base64 format, not SHA256).
Also fixes SBOM format (SPDX 2.3 instead of CycloneDX).
2026-03-21 18:48:51 +01:00
Martin Vogel 88f8908c41 Fix VirusTotal: use /analyses/ endpoint with base64 analysis ID 2026-03-21 18:44:37 +01:00
Martin Vogel 1d0889e5f9 Fix SBOM: use SPDX 2.3 format (CycloneDX failed schema validation) 2026-03-21 18:15:42 +01:00
Martin Vogel b6c32a4f13 Skip SBOM attestation (schema validation fails), keep as release asset 2026-03-21 18:10:55 +01:00
Martin Vogel 77b9e7def7 Fix SBOM: use python3 for valid JSON (heredoc indentation broke format) 2026-03-21 17:46:08 +01:00
Martin Vogel e2b7797d47 Fix ClamAV macOS: suppress X509 SSL errors in freshclam 2026-03-21 17:18:35 +01:00
Martin Vogel dd30132e2d Pin Actions to SHA + enforce via security audit + Dependabot
- All GitHub Actions pinned to immutable commit SHAs (prevents
  tag-poisoning attacks like tj-actions/changed-files incident)
- Security audit (Layer 1) now blocks unpinned Actions in CI
- Dependabot configured to auto-propose SHA updates weekly
- Pre-commit hooks tracked in scripts/hooks/ for contributors
- Time-bomb detection + MCP file read audit added to Layer 1
2026-03-21 16:57:17 +01:00
Martin Vogel e0bce845eb Fix SBOM: valid JSON format for attest-sbom action 2026-03-21 16:34:03 +01:00
Martin Vogel 6881812b48 VirusTotal gate: wait for 100% scan completion, no compromises
- Wait until at least 60 engines have actually scanned (not just
  reported type-unsupported/failure/timeout)
- 20 minute timeout per binary (was 5 minutes)
- If scan doesn't complete in time → FAIL (incomplete scan is not clean)
- Engines that can't scan (type-unsupported, failure, timeout) are
  excluded from the completion count
2026-03-21 16:10:54 +01:00
Martin Vogel b5490cf9b9 Clarify native AV scans passed in release notes 2026-03-21 15:51:00 +01:00
Martin Vogel e5b6ee5733 Fix ClamAV freshclam config: ensure signatures are downloaded
freshclam requires DatabaseMirror set in config. Without it, ClamAV
scans with empty signatures (catches nothing). Now properly configures
freshclam.conf on both Linux (apt) and macOS (Homebrew) before updating.
2026-03-21 15:45:54 +01:00
Martin Vogel f8862144bf Add native AV scans on all 3 platforms + update SECURITY.md
Every platform now has a real antivirus gate that fails the build:
- Windows: Windows Defender with ML heuristics (MpCmdRun.exe)
- Linux: ClamAV with freshclam signature update
- macOS: ClamAV with freshclam signature update

Any detection on any platform blocks the release-draft from being
created. This is on top of the VirusTotal zero-tolerance gate in
the verify job.

SECURITY.md updated with complete security architecture.
2026-03-21 15:39:45 +01:00
Martin Vogel 75b5dcb897 Fix VirusTotal output parsing + use file report API
- Output is comma-separated, not newline — use tr ',' '\n' to split
- URLs are /gui/file/<sha256> permalinks — extract SHA256 hash and
  query /api/v3/files/<sha256> for last_analysis_stats
- Release notes table builder uses same comma-split fix
- Gate uses /tmp flag file to propagate failure from subshell
2026-03-21 15:22:11 +01:00
Martin Vogel 398ef68691 Add VirusTotal zero-tolerance gate: any detection blocks publish
The verify job polls VirusTotal API for completed scan results.
If ANY engine flags ANY binary as malicious or suspicious, the
release stays as draft and the pipeline fails. Zero tolerance —
investigate and fix before retrying.
2026-03-21 15:00:08 +01:00
Martin Vogel e1b106cb2b Release pipeline: draft → verify → publish flow
Restructured release to prevent unverified binaries from being public:
1. release-draft: create draft release (not visible to users)
2. verify: VirusTotal scan + OpenSSF Scorecard on draft binaries
3. If verification passes: append results to notes + publish
4. If verification fails: draft stays unpublished
2026-03-21 14:51:18 +01:00
Martin Vogel 2e21a0133b Release security hardening: VirusTotal + attestations + cosign + SBOM + OpenSSF + SECURITY.md
Release pipeline (step 5 + 6) now includes:
- SLSA build provenance attestations for all binaries
- Sigstore cosign keyless signing (.bundle files)
- SBOM (CycloneDX) with all vendored dependencies
- VirusTotal scan of all binaries (70+ engines)
- OpenSSF Scorecard (repo security health score)
- All results appended to release notes automatically

Also:
- SECURITY.md: vulnerability reporting + security measures
- Removed DLL API strings from test binary (issue #89)
2026-03-21 14:34:22 +01:00
Martin Vogel f43774f975 Add VirusTotal scanning to release pipeline + CHANGELOG.md
- VirusTotal: scans all release binaries (tar.gz + zip) via
  crazy-max/ghaction-virustotal after publishing. Scan result
  links are appended to the release notes automatically.
- CHANGELOG.md: local copy of all release notes so history
  is preserved in the repo, not just on GitHub.
2026-03-21 14:05:53 +01:00
Martin Vogel cd1417427c Add 8-layer security test suite + hardening
Code-level defenses:
- cbm_validate_shell_arg(): reject shell metacharacters before popen/system
- SQLite authorizer: block ATTACH/DETACH at engine level
- CORS localhost-only origin reflection (replaces wildcard *)
- Path containment: realpath() check in get_code_snippet
- process-kill restricted to server-spawned PIDs
- SHA256 checksum verification in update command

Security audit scripts (8 layers):
- L1: Static allow-list for dangerous calls + URLs
- L2: Binary string audit (URLs, payloads, credentials)
- L3: Network egress monitoring via strace (Linux)
- L4: Install output path + content validation
- L5: Smoke test hardening (clean shutdown, residual procs)
- L6: Graph UI audit (external domains, CORS, binding)
- L7: MCP robustness (23 adversarial JSON-RPC payloads)
- L8: Vendored integrity (checksums + dangerous call scan)

CI: parallel security-static job (no build needed), binary
layers in smoke jobs per-platform. Cleanup of test fixture
dirs in clean.sh + .gitignore.
2026-03-20 18:19:15 +01:00
Martin Vogel 5890aecd48 Fix smoke-windows: skip self-copy for standard variant, fail-fast off
The cp command fails when SUFFIX is empty because source and destination
are the same file. Only copy when SUFFIX is non-empty (ui variant).
Also add fail-fast: false to smoke-windows so both variants always run.
2026-03-19 13:11:23 +01:00
Martin Vogel 796782e08a Fix smoke-windows (ui): rename extracted binary to match expected name
The UI zip contains codebase-memory-mcp-ui.exe but the smoke test
always runs ./codebase-memory-mcp.exe. Copy the extracted binary
to the expected name so both standard and ui variants work.
2026-03-19 12:50:45 +01:00
Martin Vogel a71a65f68e Fix smoke tests: CLANG64 runtime for Windows, debug stderr on error
1. smoke-windows: Use CLANG64 (not UCRT64) so runtime matches the
   build environment. Binary built with CLANG64 needs CLANG64 DLLs.
2. smoke-unix: Add fail-fast: false so one platform failure doesn't
   cancel all 7 other smoke jobs.
3. smoke-test.sh: Capture and display stderr on indexing failure to
   help debug smoke-unix errors.
4. release.yml: Same CLANG64 smoke fix.
2026-03-19 11:31:12 +01:00
Martin Vogel 0e9a47177b Fix Windows build: add zip pkg, portable embed (od replaces xxd)
1. Add zip to MSYS2 build packages (both dry-run and release).
2. embed-frontend.sh: replace xxd with od+sed for hex dump — xxd
   requires vim which isn't in MSYS2 CLANG64 by default.
3. embed-frontend.sh: flip IS_MACOS→IS_LINUX — Linux gets ld -r -b
   binary (ELF-only), everything else (macOS, Windows) gets the
   portable C-array approach via od.
4. release.yml: same Windows archive fix (msys2 shell + .exe detect).
2026-03-19 10:59:14 +01:00
Martin Vogel 89dd3bb7c6 Add skip_tests/skip_builds flags, fix build-stage failures
1. dry-run.yml: Add skip_tests and skip_builds workflow_dispatch inputs
   for faster iteration. Build jobs use always() to run even when tests
   are skipped.

2. Windows archive: Switch from PowerShell Copy-Item to MSYS2 shell
   with .exe extension detection. MSYS2 Clang produces .exe suffix
   that Copy-Item didn't account for.

3. Linux embed: Use absolute path for ld output when cd'd into dist
   directory. Relative path build/c/embedded/*.o was invalid from
   inside graph-ui/dist/.

4. release.yml: Same Windows archive fix as dry-run.
2026-03-19 10:43:32 +01:00
Martin Vogel 0f83f08653 Auto-detect WIN32_LIBS in Makefile, remove shell quoting entirely 2026-03-19 00:00:19 +01:00
Martin Vogel b5f14f115d Fix WIN32_LIBS quoting via env vars, add skip_lint to dry-run
- Use env: block instead of shell quoting for WIN32_LIBS in CI
- Forward WIN32_LIBS from environment in test.sh and build.sh
- Add skip_lint input to dry-run workflow for faster iteration
2026-03-18 23:35:35 +01:00
Martin Vogel 5c9f97b5b4 Switch to llvm-mingw for local Windows testing, fix CI quoting
- Replace mingw-w64 with llvm-mingw Docker image (mstorsjo/llvm-mingw)
  for local cross-compilation — includes Wine for future test execution
- Fix WIN32_LIBS quoting: use single quotes so -Wl,--allow-multiple-
  definition actually reaches the linker
- Install MSYS2 zlib into llvm-mingw sysroot for cross-compilation
2026-03-18 23:12:47 +01:00
Martin Vogel fa6aa581aa Fix Windows linker: allow-multiple-definition for MinGW CRT symbols 2026-03-18 22:32:18 +01:00
Martin Vogel 30505ec596 Switch Windows CI to CLANG64 for ASan/UBSan support
MinGW GCC does not ship ASan runtime on Windows. MSYS2 CLANG64
provides full sanitizer support via compiler-rt. All platforms
now run tests with ASan + UBSan — no exceptions.
2026-03-18 21:21:30 +01:00
Martin Vogel 9b22c1180c Add Windows to test matrix, drop libsqlite3-dev dependency
- test-windows job runs scripts/test.sh under MSYS2 (same as build)
- Remove libsqlite3-dev from all CI steps (sqlite3 is vendored now)
- Both dry-run.yml and release.yml updated
2026-03-18 16:49:42 +01:00
Martin Vogel 8f225a37dd Use cc (Apple Clang) on macOS, gcc on Linux — unified CI/local parity
- Add scripts/env.sh: detects true hardware arch (even under Rosetta),
  verifies compiler supports target arch, adds arch prefix on macOS
- macOS uses cc/c++ everywhere (tests + builds) — ASan works natively
- Linux uses gcc/g++ everywhere — full sanitizer support
- Remove broken CC ?= gcc-14 from Makefile (GNU Make built-in overrides it)
- Scripts accept --arch flag and CC=/CXX= overrides
2026-03-18 13:08:33 +01:00
Martin Vogel fd1640ec29 Add build scripts, fix GCC warnings, unify CI workflows
- Add scripts/clean.sh, test.sh, build.sh, lint.sh as single source
  of truth for local and CI builds (always clean, always parallel)
- Fix -Wmissing-braces: replace {{0}} with memset for struct arrays
- Fix -Wrestrict: use intermediate buffer for same-struct snprintf
- Update dry-run.yml and release.yml to use the new scripts
2026-03-18 12:31:53 +01:00
Martin Vogel f646da2935 Pin GCC everywhere: local (gcc-14), CI Ubuntu (gcc), CI macOS (gcc-14) 2026-03-18 12:03:28 +01:00
Martin Vogel 66bb2a7638 CI: fail-fast: false so one flaky test doesn't cancel other OS tests 2026-03-18 00:54:32 +01:00
Martin Vogel 1957612469 Unify dry-run + release pipelines: lint(format+cppcheck) → test → build → smoke → release 2026-03-18 00:17:59 +01:00
Martin Vogel 20b6932f82 CI: install cppcheck with cfg files, add to PATH 2026-03-17 23:52:01 +01:00
Martin Vogel be10b73f55 CI: fix cppcheck tag (2.20.0), cache build, verify tool versions 2026-03-17 23:45:57 +01:00
Martin Vogel ac527878c3 CI: pin LLVM 20 + cppcheck 2.20 from source, add _DEFAULT_SOURCE for POSIX compat 2026-03-17 23:40:38 +01:00
Martin Vogel bfbe3d4502 CI: drop clang-tidy from CI (v18 vs v22 incompatible), keep clang-format + cppcheck 2026-03-17 23:34:44 +01:00
Martin Vogel dc7bc91e84 CI: pin clang-format-18/clang-tidy-18, suppress cppcheck cross-version false positives 2026-03-17 23:23:17 +01:00
Martin Vogel 5b18f7da95 Fix CI: use Language: Cpp in clang-format (compatible with all versions) 2026-03-17 22:41:03 +01:00
Martin Vogel 50e0ff6244 Dry-run: lint → test → build (standard+UI) → smoke test pipeline 2026-03-17 22:33:50 +01:00
Martin Vogel ce96ecc085 Add comprehensive smoke test: version + index + search + trace + folder integrity 2026-03-17 22:25:26 +01:00
Martin Vogel 18fa9979ee Replace vmem with mimalloc global allocator, add extraction-phase prescan, fix __init__.py QN collision
Memory management:
- Vendor mimalloc v2.1.9 as global allocator (MI_OVERRIDE=1 in prod)
- New mem.h/mem.c: RSS-based budget tracking via mi_process_info()
- Remove vmem.c/vmem.h (mmap-based budget tracking)
- Remove slab tier2 bump allocator (~300 LOC); >64B goes to mimalloc
- Slab tier1 pages from malloc (= mimalloc) instead of vmem
- Arena blocks from malloc instead of vmem
- Budget raised from 35% to 50% RAM (no more untracked C++ heap)

Extraction-phase prescan (eliminates disk re-reads):
- HTTP call sites: keyword check + URL extraction during extraction
- HTTP routes: decorator + source-based extraction during extraction
- Config file refs: regex scan during extraction
- httplinks: 41.8s → 13ms on Linux kernel (3,212x faster)
- configlink: 41.4s → 0.8s on Linux kernel (54x faster)
- Linux kernel fast-mode total: 2m38s → 1m18s

Bug fixes:
- __init__.py Module QN no longer collides with Folder QN
- index.ts same fix for JS/TS packages
- 13 regression tests for QN collision at FQN + extraction layers
- search_graph/search_code default limit raised from 10 to 500k
- Resolve all clang-tidy, cppcheck, and clang-format warnings

Repo cleanup:
- tree-sitter-form, tree-sitter-magma moved to tools/
- .gitignore: build/, node_modules/, graph-ui/dist/, TEST_PLAN.md
2026-03-17 22:05:57 +01:00