Commit Graph
145 Commits
Author SHA1 Message Date
Martin Vogel 5f7cd226cb Use system regex on MSYS2, TRE only for pure MinGW (CBM_USE_TRE) 2026-03-19 02:05:20 +01:00
Martin Vogel 07f82b9c54 Vendor TRE regex for Windows, fix cbm_mkdtemp buffer overflow
- Vendor TRE regex library (BSD-licensed, ~9K LOC) for Windows —
  eliminates the TODO stub that caused all regex tests to fail
- Fix cbm_mkdtemp stack-buffer-overflow: use static buffer for
  /tmp/ → %TEMP%\ path translation (Windows paths are longer)
- Wine test results: 1861 passed, 50 failed (Wine filesystem limits),
  75 skipped — remaining failures are Wine-specific, not Windows bugs
2026-03-19 01:47:37 +01:00
Martin Vogel d1632ec7e1 Fix cbm_mkdtemp for Windows, add Wine test execution to Docker
- cbm_mkdtemp: translate /tmp/ to %TEMP%\ on Windows
- Docker test-windows: cross-compile + run under Wine (without ASan,
  which doesn't work under Wine)
- Wine on PATH fix for Ubuntu noble (/usr/lib/wine/)
- Tool preflight check in Dockerfile
- 1887/1992 tests pass under Wine (52 fail: regex not available in
  pure MinGW — works on real MSYS2 which provides <regex.h>)
2026-03-19 00:59:54 +01:00
Martin Vogel 51839ca3bd Set 8MB stack on Windows (matches Linux/macOS default) 2026-03-19 00:16:41 +01:00
Martin Vogel 0f83f08653 Auto-detect WIN32_LIBS in Makefile, remove shell quoting entirely 2026-03-19 00:00:19 +01:00
Martin Vogel b5f14f115d Fix WIN32_LIBS quoting via env vars, add skip_lint to dry-run
- Use env: block instead of shell quoting for WIN32_LIBS in CI
- Forward WIN32_LIBS from environment in test.sh and build.sh
- Add skip_lint input to dry-run workflow for faster iteration
2026-03-18 23:35:35 +01:00
Martin Vogel 5c9f97b5b4 Switch to llvm-mingw for local Windows testing, fix CI quoting
- Replace mingw-w64 with llvm-mingw Docker image (mstorsjo/llvm-mingw)
  for local cross-compilation — includes Wine for future test execution
- Fix WIN32_LIBS quoting: use single quotes so -Wl,--allow-multiple-
  definition actually reaches the linker
- Install MSYS2 zlib into llvm-mingw sysroot for cross-compilation
2026-03-18 23:12:47 +01:00
Martin Vogel fa6aa581aa Fix Windows linker: allow-multiple-definition for MinGW CRT symbols 2026-03-18 22:32:18 +01:00
Martin Vogel 30505ec596 Switch Windows CI to CLANG64 for ASan/UBSan support
MinGW GCC does not ship ASan runtime on Windows. MSYS2 CLANG64
provides full sanitizer support via compiler-rt. All platforms
now run tests with ASan + UBSan — no exceptions.
2026-03-18 21:21:30 +01:00
Martin Vogel fdb83f83f7 Fix Windows test compilation: mkdtemp, mkdir, setenv, pipe compat
- cbm_mkdtemp: _mktemp + _mkdir on Windows
- cbm_setenv/cbm_unsetenv: _putenv_s on Windows
- cbm_pipe: _pipe on Windows
- Replace bare POSIX calls in 12 test files with compat wrappers
- All test files now include compat.h
2026-03-18 19:19:42 +01:00
Martin Vogel 85fc74fc31 Full Windows portability: proper implementations, not stubs
- handle_processes: GetProcessMemoryInfo + GetProcessTimes (Windows),
  getrusage (POSIX) — real process metrics on both platforms
- handle_process_kill: TerminateProcess via OpenProcess (Windows),
  kill(SIGTERM) (POSIX)
- index_thread_fn: CreateProcess + WaitForSingleObject (Windows),
  fork+exec+waitpid (POSIX)
- cbm_clock_gettime: QueryPerformanceCounter (Windows) — replaces
  all bare clock_gettime calls across 6 files
- cbm_nanosleep, cbm_strcasestr, cbm_mkdir: compat shims
- CBM_TLS: _Thread_local (C11 standard, works on all compilers)
- Replace bare strndup with cbm_strndup, guard POSIX headers
- Add mingw-w64 cross-compile to Docker test infrastructure
2026-03-18 18:47:43 +01:00
Martin Vogel 26a78142d6 Fix Windows/MinGW build: TLS, strndup, sys/wait.h
- CBM_TLS: use _Thread_local (C11 standard) instead of __declspec(thread)
  which only works on MSVC, not MinGW GCC
- Replace bare strndup() with cbm_strndup() in 6 files (strndup is POSIX,
  not available on Windows)
- Guard sys/wait.h with #ifndef _WIN32 in http_server.c
2026-03-18 17:42:44 +01:00
Martin Vogel c8bdf924e5 Fix production build errors, add build step to Docker test infra
- Fix GCC -O2 warnings: calloc for cf (maybe-uninitialized),
  suppress stringop-truncation/alloc-size-larger-than (false positives),
  guard louvain against negative node_count
- Default compiler: cc on macOS, gcc on Linux/Windows (fixes gcc-14
  not found on MSYS2)
- Docker test infra now runs BOTH test (ASan) and build (-O2 -Werror)
2026-03-18 17:15:46 +01:00
Martin Vogel 9b22c1180c Add Windows to test matrix, drop libsqlite3-dev dependency
- test-windows job runs scripts/test.sh under MSYS2 (same as build)
- Remove libsqlite3-dev from all CI steps (sqlite3 is vendored now)
- Both dry-run.yml and release.yml updated
2026-03-18 16:49:42 +01:00
Martin Vogel ae0d1754ed Add multi-arch Docker test: arm64 (native) + amd64 (QEMU)
run.sh supports: test, amd64, all (parallel), lint, shell
2026-03-18 16:33:12 +01:00
Martin Vogel a1d849aaaa Fix all memory leaks, vendor sqlite3, add Docker test infrastructure
Leak fixes (13 sources, 1.4MB → 0 bytes):
- Cypher: free col_names after rb_set_columns, free rejected inline
  prop nodes, store WITH aliases in qualified_name for auto-cleanup
- Store: add cbm_project_free_fields, fix find_node_ids_by_qns leak
- Graph buffer: free strdup'd hash keys during delete operations
- SQL scanner: free old start_tag before reassignment in deserialize

Platform independence:
- Vendor sqlite3 amalgamation (3.49.1) — eliminates system libsqlite3
  dependency, enables full ASan/LeakSanitizer instrumentation
- Add -I for vendored tree-sitter unicode headers — eliminates hidden
  dependency on system libicu-dev
- Add test-infrastructure/ with Docker Compose for local GCC+ASan
  testing that mirrors CI exactly
2026-03-18 16:28:27 +01:00
Martin Vogel bc66541765 Remove unused system_info fields: cache, free_ram, efficiency_cores
These 5 fields were detected on 3 platforms but never consumed by any
production code. The L2 cache sysfs fallback added in the previous
commit existed solely to fix a test for a value nobody reads.

Removes ~120 lines of platform-specific detection (Mach VM stats,
/proc/meminfo parsing, sysfs cache scan) and 3 dead tests.
2026-03-18 14:51:29 +01:00
Martin Vogel c9ec1d1eb4 Fix Ubuntu CI: stack-use-after-return, L2 cache detection, memory leaks
- Arena-allocate pair_field_types in cpp_stdlib_data.c (was stack array
  escaping scope via registry pointer — caught by GCC ASan)
- Add sysfs fallback for L2 cache on Linux ARM (sysconf returns 0)
- Expose cbm_node_free_fields() and fix 12+ leak sites in tests where
  find_node_by_id/qn results were never freed (caught by LeakSanitizer)
2026-03-18 14:23:46 +01:00
Martin Vogel 93c0b865c4 Detect GCC vs Clang for compiler-specific warning flags
GCC-only flags (-Wno-format-truncation, -Wno-unused-result) are now
conditionally added via __GNUC__ / __clang__ preprocessor detection.
Fixes Clang rejecting unknown GCC flags with -Werror on CI runners.
2026-03-18 13:27:27 +01:00
Martin Vogel 8f225a37dd Use cc (Apple Clang) on macOS, gcc on Linux — unified CI/local parity
- Add scripts/env.sh: detects true hardware arch (even under Rosetta),
  verifies compiler supports target arch, adds arch prefix on macOS
- macOS uses cc/c++ everywhere (tests + builds) — ASan works natively
- Linux uses gcc/g++ everywhere — full sanitizer support
- Remove broken CC ?= gcc-14 from Makefile (GNU Make built-in overrides it)
- Scripts accept --arch flag and CC=/CXX= overrides
2026-03-18 13:08:33 +01:00
Martin Vogel fd1640ec29 Add build scripts, fix GCC warnings, unify CI workflows
- Add scripts/clean.sh, test.sh, build.sh, lint.sh as single source
  of truth for local and CI builds (always clean, always parallel)
- Fix -Wmissing-braces: replace {{0}} with memset for struct arrays
- Fix -Wrestrict: use intermediate buffer for same-struct snprintf
- Update dry-run.yml and release.yml to use the new scripts
2026-03-18 12:31:53 +01:00
Martin Vogel f646da2935 Pin GCC everywhere: local (gcc-14), CI Ubuntu (gcc), CI macOS (gcc-14) 2026-03-18 12:03:28 +01:00
Martin Vogel 80299abf86 Fix clang-format: sort includes in log.c 2026-03-18 01:53:29 +01:00
Martin Vogel ef512c3508 Remove macOS-internal headers inserted by clang-tidy (breaks Linux/GCC) 2026-03-18 01:52:05 +01:00
Martin Vogel e57dba8fd9 Fix: use -Wno-unknown-warning-option for cross-compiler compat (Clang+GCC) 2026-03-18 01:38:42 +01:00
Martin Vogel a97bfc684e Fix: GCC warning flags conditional (Clang doesn't support them) 2026-03-18 01:30:02 +01:00
Martin Vogel c8294ce160 Fix GCC: suppress format-truncation, restrict, unused-result warnings 2026-03-18 01:22:47 +01:00
Martin Vogel 05a6516597 Fix httplink_linker_route_nodes: remove store dep, test pure route extraction 2026-03-18 01:12:27 +01:00
Martin Vogel 66bb2a7638 CI: fail-fast: false so one flaky test doesn't cancel other OS tests 2026-03-18 00:54:32 +01:00
Martin Vogel 4f3f2f4b22 Fix UBSan: guard cbm_type_is_unknown against misaligned dangling pointers 2026-03-18 00:42:12 +01:00
Martin Vogel 0aabc447f6 Add _DEFAULT_SOURCE to all CFLAGS (mongoose, grammars) for Linux POSIX compat 2026-03-18 00:28:13 +01:00
Martin Vogel 1957612469 Unify dry-run + release pipelines: lint(format+cppcheck) → test → build → smoke → release 2026-03-18 00:17:59 +01:00
Martin Vogel 6f4a564e05 CI: suppress insecureAPI.strcpy (no strlcat on Linux) 2026-03-18 00:00:11 +01:00
Martin Vogel 20b6932f82 CI: install cppcheck with cfg files, add to PATH 2026-03-17 23:52:01 +01:00
Martin Vogel be10b73f55 CI: fix cppcheck tag (2.20.0), cache build, verify tool versions 2026-03-17 23:45:57 +01:00
Martin Vogel ac527878c3 CI: pin LLVM 20 + cppcheck 2.20 from source, add _DEFAULT_SOURCE for POSIX compat 2026-03-17 23:40:38 +01:00
Martin Vogel bfbe3d4502 CI: drop clang-tidy from CI (v18 vs v22 incompatible), keep clang-format + cppcheck 2026-03-17 23:34:44 +01:00
Martin Vogel dc7bc91e84 CI: pin clang-format-18/clang-tidy-18, suppress cppcheck cross-version false positives 2026-03-17 23:23:17 +01:00
Martin Vogel e906af9568 Move version-specific cppcheck suppressions to --suppress flags for cross-OS compatibility 2026-03-17 23:08:04 +01:00
Martin Vogel 5b18f7da95 Fix CI: use Language: Cpp in clang-format (compatible with all versions) 2026-03-17 22:41:03 +01:00
Martin Vogel 50e0ff6244 Dry-run: lint → test → build (standard+UI) → smoke test pipeline 2026-03-17 22:33:50 +01:00
Martin Vogel ce96ecc085 Add comprehensive smoke test: version + index + search + trace + folder integrity 2026-03-17 22:25:26 +01:00
Martin Vogel 18fa9979ee Replace vmem with mimalloc global allocator, add extraction-phase prescan, fix __init__.py QN collision
Memory management:
- Vendor mimalloc v2.1.9 as global allocator (MI_OVERRIDE=1 in prod)
- New mem.h/mem.c: RSS-based budget tracking via mi_process_info()
- Remove vmem.c/vmem.h (mmap-based budget tracking)
- Remove slab tier2 bump allocator (~300 LOC); >64B goes to mimalloc
- Slab tier1 pages from malloc (= mimalloc) instead of vmem
- Arena blocks from malloc instead of vmem
- Budget raised from 35% to 50% RAM (no more untracked C++ heap)

Extraction-phase prescan (eliminates disk re-reads):
- HTTP call sites: keyword check + URL extraction during extraction
- HTTP routes: decorator + source-based extraction during extraction
- Config file refs: regex scan during extraction
- httplinks: 41.8s → 13ms on Linux kernel (3,212x faster)
- configlink: 41.4s → 0.8s on Linux kernel (54x faster)
- Linux kernel fast-mode total: 2m38s → 1m18s

Bug fixes:
- __init__.py Module QN no longer collides with Folder QN
- index.ts same fix for JS/TS packages
- 13 regression tests for QN collision at FQN + extraction layers
- search_graph/search_code default limit raised from 10 to 500k
- Resolve all clang-tidy, cppcheck, and clang-format warnings

Repo cleanup:
- tree-sitter-form, tree-sitter-magma moved to tools/
- .gitignore: build/, node_modules/, graph-ui/dist/, TEST_PLAN.md
2026-03-17 22:05:57 +01:00
Martin Vogel 0d216c7936 Resolve all clang-tidy, cppcheck, and clang-format warnings across 94 files
Enforce WarningsAsErrors: '*' with zero suppressions of entire check categories.
All non-FP warnings fixed at source; true false positives get per-instance NOLINT
with reasoning. Fixes include: braces around statements (1679), isolate declarations
(75), explicit casts for multilevel pointer conversions, missing includes, function
parameter documentation via NOLINT, and proper NOLINTBEGIN/NOLINTEND nesting.
2026-03-15 22:37:34 +01:00
Martin Vogel 6f0910278b Remove all Go code — project is now pure C
Delete Go MCP server, CGo bridges, Go pipeline, store, tools, watcher,
cypher engine, language specs, and all Go tests. Replace Go pre-commit
hook with C-only linting (clang-tidy, cppcheck, clang-format, tests).
Tree-sitter extraction engine (internal/cbm/*.c) retained for C pipeline.
2026-03-15 20:32:19 +01:00
Martin Vogel 94a9a92f83 Add C-native pipeline with blocking clang-tidy, cppcheck, clang-format linting
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.

Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
2026-03-15 19:26:12 +01:00
Martin Vogel f85a4e1c14 RAM-first pipeline with fused C LZ4 + Aho-Corasick multi-pattern matching
Full indexing now runs entirely in RAM:
- One filepath.Walk, one bulk file read (LZ4 HC compressed sourceStore)
- In-memory SQLite for all passes, single Backup API dump at end
- Source access via C LZ4 decompression (~3μs per file, 5.7 GB/s)

Aho-Corasick engine (internal/cbm/ac.c):
- Custom ACISM automaton with bitmask output (≤64 patterns)
- Fused LZ4 decompress + AC scan in C-side thread-local buffer
- Batch API: single CGo call scans all files (ScanLZ4Batch)
- Compact alphabet mode for configlinker (38-char table)

Pipeline optimizations:
- httplinks: AC pre-screens files for HTTP client + route keywords
- configlinker: AC batch scan replaces O(n×m) substring matching
- Infrascan/envscan: iterate sourceStore keys instead of walking disk
- All source reads via getSource() (LZ4 decompress from RAM)
- File hashes pre-computed during bulk load (zero re-reads)
2026-03-14 10:42:34 +01:00
Martin Vogel 522bd5b1fd Expand C/C++ LSP resolver: template substitution, smart pointer chains, 700+ tests
Major improvements to the C/C++ type-aware call resolver:

- Template argument substitution for qualified calls (std::get<T>, std::any_cast<T>,
  std::make_shared<T>) — strip template args from AST, substitute TYPE_PARAMs in
  return types, wrap NAMED→TEMPLATE for downstream smart pointer deduction
- Smart pointer deref uses suffix matching (handles module-prefixed QNs like
  "test.main.std.shared_ptr")
- Fix storage_class_specifier/type_qualifier skipping in c_parse_declaration_type
  (static/const/virtual methods now get correct return types)
- Scope-aware alias resolution in c_resolve_name_to_type (typedef/using)
- Template namespace fallback in c_parse_type_node (resolve types within
  current namespace during template class pre-pass)
- Pending template call resolution emits with correct caller QN
- Add ~330 new tests covering C++17 (structured bindings, if-init, CTAD, optional),
  C++20 (coroutines, concepts, ranges), heavy templates, real-world patterns
  (smart pointers, iterators, RAII, factories), STL completeness, and C patterns
2026-03-13 19:47:36 +01:00
Martin Vogel d201a5faa6 Close C/C++ LSP gaps: overload scoring, lambda/decltype inference, virtual dispatch
Type system improvements:
- Overload resolution by parameter type (score_overload_match with type compatibility)
- Lambda return type inference (trailing return type + shallow body walk)
- Inline namespace normalization (strip __1, __cxx11, __gnu_cxx)
- Implicit conversions in overload scoring (const char* → string, numeric promotions)
- Virtual dispatch: prefer derived override over base method
- CRTP detection: template param → derived class mapping
- Range-for iterator protocol: map element fix + begin()/operator*() fallback
- Default argument support via min_params range matching
- Free function auto deduction: identifier eval order fix (function before type)
- decltype() return type parsing in c_parse_return_type_text
- Trailing return type syncs both return_type and return_types array
2026-03-13 00:49:42 +01:00
Martin Vogel a8daedf759 Add C/C++ hybrid LSP resolver with type-aware call resolution
Implement a tree-sitter-based type evaluator for C/C++ that resolves
method calls, template instantiations, and cross-file references without
requiring a full compiler. Covers ~95% of C and ~93% of moderate C++.

Key features:
- Type-aware overload resolution scoring by parameter types
- Lambda return type inference (trailing return + body deduction)
- Inline namespace normalization (__1, __cxx11, __gnu_cxx)
- Implicit conversions (char*->string, numeric promotions)
- Virtual dispatch with override preference
- CRTP detection and self-type resolution
- Range-for iterator protocol (maps, custom containers)
- Smart pointer arrow dispatch (unique_ptr, shared_ptr)
- Dependent member calls in templates via pending call resolution
- Explicit template argument substitution (make_shared<T>)
- Nested class scope resolution
- C/C++ stdlib type stubs (containers, strings, I/O, smart ptrs)
- Cross-file LSP with compile_commands.json support
- C/C++ parameter extraction via declarator chain traversal

63+ test cases covering all resolution strategies.
2026-03-13 00:07:28 +01:00