mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-10-25 06:33:13 +08:00
Martin Vogel
af250e7d65
Release binaries carried their full symbol table. Production compiles without -g, but the linker keeps .symtab regardless, so every archive shipped ~536 KB of internal function names that nothing needs: a bigger download and a free map of the internals, by accident rather than by decision. Nothing symbolizes at runtime -- mem_profile.c is not in the production build and never calls backtrace_symbols -- so no diagnostics are lost. It also had a concrete cost. Microsoft's ML scored the unstripped linux-amd64 binary Trojan:Script/Wacatac.B!ml (1 engine of 62) and blocked release run 30398064336 at the VirusTotal gate. That verdict is a decision-boundary artifact, not a property of the code, and the evidence is unambiguous: * the dry-run build two days earlier (d587dea) is the same program 10 KB larger and scans CLEAN -- a 0.003% delta flips the verdict * the delta in that window is almost entirely DELETION of Windows-only files that Linux never compiled * the ui build of the same commit was never flagged * two independently flagged builds drew different sub-variants (Wacatac.B and Wacatac.C), which a real signature does not do * v0.9.0, re-analysed against the same engine build, is still clean -- so this is not model drift either Stripping removes the symbol-name feature surface those models score. It cleared BOTH flagged builds, and every other platform stays clean, so this fixes Linux without trading the problem sideways. Verified before merge: linux-amd64 (f440743) stripped 0 malicious / 62 engines linux-amd64 (0802689) stripped 0 malicious / 62 engines linux-arm64 stripped 0 malicious / 61 engines linux-amd64-portable stripped 0 malicious / 62 engines windows-amd64 stripped 0 malicious / 68 engines darwin-arm64 stripped + re-signed 0 malicious / 60 engines macOS needs care and gets it. The build workflow ad-hoc signs BEFORE this script runs, so stripping invalidates that signature and the kernel then refuses to exec the image; Mach-O is re-signed here. It also uses `strip -x` rather than --strip-all, because a full strip can leave an image dyld will not load. The packaged macOS binary verifies (`valid on disk`, `satisfies its Designated Requirement`) and runs, and the full macOS smoke passes against a stripped, re-signed binary. Build fingerprints are derived from the file itself during execution, with nothing embedded during compilation, so staged and target agree and activation is unaffected. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/ — the canonical leg entries
The doctrine (enforced, not advisory): a venue may provision a machine;
only a canonical leg script may exercise the product. Local CI, PR CI, the
dry run and the release all call the SAME files below — venues differ only in
host specs, architecture and inputs, never in what a leg does. Platform
differences (arm64 sanitizer flags, the Windows launcher, the linux portable
binary) live INSIDE these scripts, once, never per-venue.
tests/test_venue_parity_contract.sh (run as Step 0j of every test leg) turns
any violation — inline harness logic in a workflow, a bypassed wrapper, a
missing --help — into a red build.
Every entry answers --help (authoritative, agent-oriented: modes, env, exit
codes) and rejects unknown flags with exit 2 + Please consult --help.
The legs
| leg | entry | what a run gives you |
|---|---|---|
| test | test.sh |
DEFAULT = the venue leg: static contracts (Step 0a–0j) + CLEAN ASan+UBSan build + all suites via the parallel harness + prod-binary guards. --suites a,b = iteration mode (incremental rebuild, subset, seconds). --tsan = the ThreadSanitizer leg. CLANGARM64 gets CI's trap-UBSan flags automatically. |
| build | build.sh |
CLEAN production build (+ launcher on Windows). ccache via env.sh makes repeats fast; CCACHE_COMPILERCHECK=content guarantees a hit is byte-identical to a cold compile — never stale. --with-ui, --version, STATIC=1, BUILD_DIR=. |
| lint | lint.sh |
clang-tidy + cppcheck + clang-format (+ no-skips policy). --ci = the CI gate set (no clang-tidy). Drives the same make targets as make lint/lint-ci. |
| smoke (unix) | smoke-local.sh |
Stages a full release fixture, serves it on a kernel-assigned port, runs smoke-test.sh (ALL phases incl. download/install/update E2E) inside a disposable HOME/XDG/TMP sandbox. ui variant makes a missing embedded UI a FAILURE. CBM_SMOKE_ARTIFACT_DIR = smoke an extracted release artifact verbatim (release mode). |
| smoke (windows) | ../test-infrastructure/vm/vm-smoke.sh |
Same contract for the launcher+payload pair, plus the user-PATH registry guard (prepare/verify/cleanup). |
| smoke-invariants | smoke-invariants.sh |
The "shipped binary does not fail" battery (MCP handshake, all tools invocable, malformed-input resilience) — no fixture server, no install E2E. smoke.yml runs it on the WIDEST build-from-source matrix (incl. older-glibc legs the release artifacts cannot cover). |
| soak | soak-legs.sh |
The release-gating soak SEQUENCE: quick then query-leak (the #581 detector — never reindexes, so RSS growth = query-path leak), each guarded by a completion-summary check. --legs quick for the ASan single-leg variant. Duration is per leg. |
Internal harnesses — never called directly by a venue (the contract forbids
it): smoke-test.sh (phases; wrappers provide fixture server + sandbox),
soak-test.sh (one soak run; soak-legs.sh provides the sequence + guards),
run-tests-parallel.sh (reached through test.sh).
Conventions
- Exit codes: 0 = pass · 2 = usage error · 90 = guard (a run died without its completion summary — never counts as green) · anything else = the leg's real failure.
- Iteration is a flag, not a side-tool: the fast paths (
--suites,--legs) are modes of the SAME entry the gates run, so a dev loop can never drift from the venue behaviour. - Env sandboxing: the smoke wrappers neutralize every agent-config destination override; a smoke can never scribble on your real config.
Recommended workflows
- Iterating on a change:
scripts/test.sh --suites <suite>(seconds, incremental, same ASan+UBSan flags as the gate). List suites:build/c/test-runner --list-suites. Debugging a Windows-on-ARM trap: re-run withSANITIZE=for a plain build, or use the emulatedwin.sh ubsan-*pair for full diagnostics. - Before any push (the 3-OS ladder):
scripts/test.sh(macOS, full) →./test-infrastructure/run.sh full(Linux + TSan + smoke) →test-infrastructure/vm/win.sh test-par+guards+smoke-install(+soakwhen the change touches memory/daemon paths). - Concurrency-touching change: add
scripts/test.sh --tsanearly — the same leg CI gates on. - Release-shaped verification:
CBM_SMOKE_ARTIFACT_DIR=<extracted artifact> scripts/smoke-local.sh <binary> [ui]smokes exactly what would ship. - A leg is red in CI but green locally: first suspect environment shape,
not code — the preflights (
win.shautomatic;scripts/ci/preflight-docker.sh) andtest-infrastructure/README.md's residuals list cover the knowable differences.
See scripts/ci/README.md for the CI plumbing and
test-infrastructure/README.md for the venue map.