mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-13 09:53:15 +08:00
test: widen ThreadSanitizer coverage and add native ARM64 Windows UBSan
Close the cross-platform sanitizer gaps that were leaving real concurrency and undefined-behavior bugs uncaught, and fix a data race the first widened run surfaced. - tsan: the data-race gate ran three suites (mem, slab_alloc, parallel) over no real threaded production code. It now covers every threaded surface that runs clean and stable under TSan: the parallel-extraction worker pool (parallel, worker_pool, pipeline), the filesystem watcher, the embedded HTTP server (httpd), diagnostics sampling, the MCP server and mutation guard, subprocess supervision, and the runnable daemon-coordination paths (daemon, daemon_application). daemon_runtime (deadlocks under TSan+fork), daemon_ipc and daemon_frontend (test-harness synchronization, not production) are excluded with the reasons recorded in the Makefile. - tsan: fixed a genuine data race the widened gate immediately found — cbm_lsp_max_walk_depth's lazy cache was read and written by parallel LSP-extraction workers without synchronization. A data race is undefined behavior even when every worker computes the same value, so the cache slot is now a relaxed atomic: a plain load on the hot path, and a first-touch double-compute simply stores the same value. - tsan(ci + local): the test-tsan job now runs on Linux amd64, Linux arm64, AND native ARM64 macOS (the threading code is shared, so a race is usually caught on all three, but scheduler differences let each surface one the others miss). The local ladder gained `run.sh tsan` and `tsan-amd64` plus the matching compose services. TSan's shadow memory aborts under modern high-entropy ASLR, so the containers run under `setarch -R` with an unconfined seccomp profile (the personality syscall is otherwise blocked) and the CI Linux legs lower vm.mmap_rnd_bits first; amd64 TSan cannot run under x86_64-on-ARM translation and is a real-hardware/CI gate only (documented in run.sh). - ubsan(win/arm64): native ARM64 Windows had no sanitizer at all — AddressSanitizer ships no aarch64-w64-windows-gnu runtime. UBSan in trap mode (-fsanitize-trap=undefined) needs no runtime library, so it instruments natively and turns undefined behavior into an illegal-instruction trap; -fstack-protector-strong adds stack-smash coverage the heap tools miss. The GitHub windows-11-arm leg switches from unsanitized to this, and vm/win.sh gains trap-ubsan-build / trap-ubsan-test for local iteration (reproduce under the emulated x86_64 UBSan to see which check fired). The whole codebase builds and runs clean under it. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel
+47
-13
@@ -115,31 +115,60 @@ jobs:
|
||||
env:
|
||||
CCACHE_DIR: ${{ github.workspace }}/.ccache
|
||||
|
||||
# ThreadSanitizer data-race gate. The threaded production surfaces (watcher,
|
||||
# subprocess, httpd, pipeline, mcp, worker pool, diagnostics, the runnable
|
||||
# daemon-coordination paths) plus allocator concurrency all run under TSan
|
||||
# here — see TEST_TSAN_SUITES in Makefile.cbm. Runs on both Linux
|
||||
# architectures AND native ARM64 macOS: the threading code is shared, so a
|
||||
# race is usually caught on all three, but scheduler differences mean each
|
||||
# platform can surface a race the others miss. Windows has no TSan runtime on
|
||||
# any toolchain (documented irreducible gap); this shared-code coverage is
|
||||
# its substitute.
|
||||
test-tsan:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- { os: ubuntu-latest, cc: clang, cxx: clang++ }
|
||||
- { os: ubuntu-24.04-arm, cc: clang, cxx: clang++ }
|
||||
- { os: macos-14, cc: cc, cxx: c++ }
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 120
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
|
||||
- name: Install deps (Ubuntu)
|
||||
if: runner.os == 'Linux'
|
||||
run: sudo apt-get update && sudo apt-get install -y clang zlib1g-dev ccache
|
||||
|
||||
- name: Install deps (macOS)
|
||||
if: runner.os == 'macOS'
|
||||
run: brew install ccache
|
||||
|
||||
- name: Compiler cache (content-verified)
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ${{ github.workspace }}/.ccache
|
||||
key: ccache-tsan-${{ github.ref }}-${{ github.sha }}
|
||||
key: ccache-tsan-${{ matrix.os }}-${{ github.ref }}-${{ github.sha }}
|
||||
restore-keys: |
|
||||
ccache-tsan-${{ github.ref }}-
|
||||
ccache-tsan-${{ matrix.os }}-${{ github.ref }}-
|
||||
|
||||
- name: ThreadSanitizer tests
|
||||
# This job calls make directly (no env.sh), so route the compilers
|
||||
# through ccache's Debian masquerade dir; update-ccache-symlinks in the
|
||||
# ccache postinst has already linked the clang installed above.
|
||||
# Calls make directly (no env.sh), so route the compilers through
|
||||
# ccache's masquerade dir: /usr/lib/ccache on Debian, libexec under the
|
||||
# Homebrew prefix on macOS. On Linux, reduce mmap ASLR entropy first:
|
||||
# modern (esp. aarch64) kernels randomize high enough that TSan's
|
||||
# shadow mapping aborts with "unexpected memory mapping" before any
|
||||
# test runs. macOS TSan needs no such adjustment.
|
||||
run: |
|
||||
export PATH=/usr/lib/ccache:$PATH
|
||||
if [ "$RUNNER_OS" = "macOS" ]; then
|
||||
export PATH="$(brew --prefix ccache)/libexec:$PATH"
|
||||
else
|
||||
export PATH=/usr/lib/ccache:$PATH
|
||||
sudo sysctl -w vm.mmap_rnd_bits=28 || true
|
||||
fi
|
||||
export CCACHE_COMPILERCHECK=content
|
||||
make -f Makefile.cbm test-tsan CC=clang CXX=clang++
|
||||
make -f Makefile.cbm test-tsan CC=${{ matrix.cc }} CXX=${{ matrix.cxx }}
|
||||
env:
|
||||
CCACHE_DIR: ${{ github.workspace }}/.ccache
|
||||
CCACHE_MAXSIZE: 1500M
|
||||
@@ -209,10 +238,15 @@ jobs:
|
||||
shell: msys2 {0}
|
||||
# AddressSanitizer is unavailable on native ARM64 Windows (LLVM ships no
|
||||
# libclang_rt.asan for aarch64-w64-windows-gnu) and cannot intercept the
|
||||
# system DLLs under x86-64 emulation either, so windows-11-arm runs the
|
||||
# native ARM64 build with SANITIZE= (no sanitizer) — still a real
|
||||
# functional gate. ASan/UBSan coverage comes from the other 9 legs,
|
||||
# including native-ARM Linux/macOS. x86-64 Windows keeps full sanitizers.
|
||||
# system DLLs under x86-64 emulation either. Instead of running the
|
||||
# ARM64 leg unsanitized, it runs UBSan in TRAP mode: -fsanitize-trap
|
||||
# needs no runtime library (the very thing aarch64-w64-windows-gnu
|
||||
# lacks), so it instruments natively and turns any undefined behavior
|
||||
# into an illegal-instruction trap; -fstack-protector-strong adds
|
||||
# stack-smash coverage the heap tools miss. To see WHICH check fired,
|
||||
# reproduce under the emulated x86_64 UBSan (win.sh ubsan-build), which
|
||||
# carries the full message. x86-64 Windows keeps full ASan+UBSan; the
|
||||
# other native-ARM legs (Linux/macOS) carry ASan+LSan+TSan.
|
||||
run: |
|
||||
# Native tests resolve temp via TEMP/TMP (cbm_tmpdir); shell tools use
|
||||
# TMPDIR. Route both through the protected root created above so no
|
||||
@@ -220,7 +254,7 @@ jobs:
|
||||
export TEMP="$(cygpath -m "$CBM_CI_TEMP_ROOT")"
|
||||
export TMP="$TEMP"
|
||||
export TMPDIR="$(cygpath -u "$CBM_CI_TEMP_ROOT")"
|
||||
scripts/test.sh CC=clang CXX=clang++ ${{ matrix.os == 'windows-11-arm' && 'SANITIZE=' || '' }}
|
||||
scripts/test.sh CC=clang CXX=clang++ ${{ matrix.os == 'windows-11-arm' && 'SANITIZE=-fsanitize=undefined -fsanitize-trap=undefined -fstack-protector-strong -fno-omit-frame-pointer' || '' }}
|
||||
env:
|
||||
CBM_SKIP_PERF: ${{ inputs.skip_perf && '1' || '' }}
|
||||
CCACHE_DIR: ${{ github.workspace }}/.ccache
|
||||
|
||||
Reference in New Issue
Block a user