Release evidence from run 31744302624 shows the tolerated Microsoft `!ml`
verdict is close to a coin flip per byte image rather than a property of the
code. Across the eight targets the stripped and unstripped candidates of the
SAME linker output disagreed on four, and in both directions:
linux-amd64 stripped microsoft-ml unstripped clean
darwin-arm64 stripped clean unstripped microsoft-ml
linux-arm64 stripped clean unstripped microsoft-ml
If the classifier were keying on something intrinsic to our code the siblings
would agree; they do not. So each variant is close to an independent draw, and
5 of 16 candidates drew the flag.
Two draws is not always enough. On that run linux-amd64-portable came back
microsoft-ml on BOTH candidates, leaving no clean binary to ship for that
target. A third independent draw at a ~31% observed per-candidate hit rate takes
the both/all-flagged case from roughly 1-in-10 per target to roughly 1-in-30.
The third candidate is `--strip-debug` (Apple: `-S`): debug information removed,
symbol table kept. Behaviourally identical to the other two — same linker
output, only metadata differs — but a distinct byte image, which is all
VirusTotal needs to scan it as its own file. Verified on the real v0.10.4
candidates: linux-amd64 gives three distinct hashes (294,634,656 /
294,623,208 / 293,746,096 bytes) and darwin-arm64 likewise, with the ad-hoc
signature verifying after strip.
Selection is unchanged in spirit and now ordered: smallest artifact first
(stripped, debug-stripped, unstripped), take the first CLEAN one, and only if
every candidate drew the tolerated verdict ship the smallest flagged one. A
hard verdict on any candidate still blocks the release before selection.
Cost is 24 objects per release instead of 16.
Derivation enforces that all three hashes differ — identical candidates would be
one draw wearing three hats, and the selector would believe it had alternatives
it does not have. Public claims in README, SECURITY.md and docs/index.html
updated from "both stripped and unstripped" to the three candidates.
All three release contract tests pass, including the native derivation test
which exercises the real strip and codesign path on this host.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every release now carries .mcpb one-click-install bundles alongside the
archives, and the MCP Registry entry lists them with per-file sha256:
- package-release.sh (canonical) builds codebase-memory-mcp-<target>.mcpb
for darwin/windows and the STATIC linux builds — manifest.json + the same
staged (stripped, gated) binary + LICENSE + THIRD_PARTY_NOTICES.md. The
glibc-dynamic linux targets stay archive-only: a dynamic binary defeats
the one-click promise.
- _build.yml / release-draft: bundles flow through provenance attestation,
checksums.txt, cosign signing and the release asset list; checksums.txt
is also preserved as a same-run artifact for the registry job.
- verify: the canonical scan matrix grows to 14 containers; MCPB manifests
are validated (parse, binary server, entry_point member, command binds
the entry point). Bundle binaries dedupe to the archive scan objects, so
the VT gate gains only the three distinct manifest.json files.
- publish-mcp-registry: gen-mcpb-registry-entries.sh appends one mcpb
package entry per bundle (release-asset URL + fileSha256 from the
attested checksums) to server.json before mcp-publisher runs.
Idempotent; a checksums file without bundles is a hard failure.
- contracts: Step 0o pins the bundle shape at its producer on every leg,
Step 0p pins the registry entries against the live server.json, and the
extractor contract covers the 14-container matrix incl. broken-manifest
fail-closed cases. The linux test image gains zip for the packager.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Externalizing the integration templates (#1492/#1493) and the UI bundle
(#1501/#1503) was done to reduce the Microsoft `Wacatac.B!ml` surface. It did
not work: across dry runs the flagged artifact count stayed at ~3 and the
detections merely moved between artifacts.
Dissection of run 31286803592 shows there is no structural cause to fix. The
verdicts split across every axis at once — linux-amd64 (dynamic) flagged while
linux-amd64-portable (static) is clean, but linux-arm64 (dynamic) clean while
linux-arm64-portable (static) is flagged. The two macOS binaries have identical
segment structure and split clean/flagged. Siblings from one build landed in
different variant buckets (.B vs .C). Entropy is low everywhere
(code_vectors.bin 4.166, grammar tables 3.464 bits/byte, against 7.5-8.0 for
packed payloads), so the packed-payload hypothesis is excluded too.
So the complexity bought nothing, and installation goes back to being
self-contained: one binary that carries its own UI and agent integration
templates, with no adjacent data file that has to resolve before `install`
works. Only the UI-capable composition ships from now on, under the historical
unsuffixed archive name.
Removed: src/ui/asset_pack.{c,h}, asset_pack_stub.c, asset_manifest_stub.c,
scripts/pack-ui-assets.mjs, src/cli/integration_assets.{c,h},
assets/cbm-integrations.json, scripts/gen-integrations-hash.sh, the
--verify-runtime-assets probe (nothing adjacent left to verify), and the
composition gates A6/A7 whose property is now deliberately inverted.
Restored: scripts/embed-frontend.sh, src/ui/embedded_{assets.h,stub.c}, the
compiled-in hook/adapter template bodies, and the embed/EMBED_OBJS build path.
Kept from the reverted commits, re-applied by hand where a wholesale file
restore would have dropped them:
- cbm_module_path_utf8() in both self-path sites. GetModuleFileNameA renders
through the ANSI code page and mangles non-ASCII install paths.
- the /__cbm/ui-readiness HMAC proof, secure_random and cbm_hmac_sha256, so
`daemon start --open` still waits for a genuine CBM listener.
- X-Content-Type-Options: nosniff on served assets.
- the MinGW noexecstack gate, -lbcrypt, and the cppcheck/zip CI fixes.
Archives are now codebase-memory-mcp-<os>-<arch>[-portable] with exactly four
members (binary, LICENSE, installer, THIRD_PARTY_NOTICES.md). That restores the
names every static package manifest already points at — aur, chocolatey,
homebrew, scoop, winget and glama were all broken by the -ui- rename.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Closes the closable local-vs-remote venue gaps so that, outside arch
legs / YAML glue / release plumbing, every local red is a remote red
and vice versa on shared legs:
- scripts/package-release.sh: THE canonical archive step (names +
five-file Windows bundle layout defined once); _build.yml's eight
inline archive blocks become calls to it.
- scripts/ci/smoke-artifact.sh: the artifact-flow smoke lane — build,
package, extract, then the canonical wrapper in artifact mode; wired
as run.sh smoke-artifact (compose service), win.sh smoke-artifact,
and directly runnable on macOS. Archive-layout bugs now surface
locally instead of in a release dry run.
- glibc-floor leg (Dockerfile.glibc22 + compose + run.sh): portable
binary smokes on ubuntu-22.04/glibc 2.35; the dynamic binary must
refuse there (2.38+ floor by design).
- Defender-ON parity (user directive): scripts/ci/ensure-defender.ps1
enables + VERIFIES real-time protection, fail-closed; runs in every
Windows CI job (_test x2, _soak x3, _smoke, pr.yml) AND in the VM
preflight; _smoke.yml's scan engine-failure soft-skip becomes a red
gate. Expected cost: slower Windows jobs (AV scanning during
install/build/test I/O); the next dry run proves the runner side.
- Contracts: launcher-bundle five-file check retargeted onto
package-release.sh + per-archive canonical-call association;
venue-parity contract requires the new lanes, counts one
ensure-defender step per Windows job, adds --help probes for the new
entries. Extended contract fails on the pre-change tree (verified:
20 violations on HEAD).
- VM README: ephemerality/Defender posture documented — utmctl has no
snapshot verb, so per-run revert stays a manual qcow2 option; the
sweep preflight remains the standing mechanism.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two release-blockers surfaced once #882 made every smoke leg a required gate
(continue-on-error previously masked them):
- smoke-unix ran the *dynamic* linux binary on ubuntu-22.04 / 22.04-arm, where
it cannot start (it links glibc 2.38+) -> Phase 1 died. Older-glibc coverage
is the -portable (static) binary's job and is green via the smoke-linux-portable
broad legs, so drop the impossible dynamic broad legs (keep macOS).
- the ClamAV scan died on every linux leg: apt auto-starts the clamav-freshclam
daemon which holds the freshclam log lock, so the manual freshclam failed with
"Failed to lock the log file ... Resource temporarily unavailable". Stop the
daemon first.
Also add native ARM64 Windows as a first-class release target -- the one platform
we test (windows-11-arm) but never shipped:
- build-windows-arm64 job (CLANGARM64) producing codebase-memory-mcp[-ui]-windows-arm64.zip
- smoke-windows is now arch-aware: windows-11-arm smokes the NATIVE arm64 binary
instead of the x86_64 binary under emulation
- a native soak-quick-windows-arm64 leg (no sanitizer -- ASan is unavailable on
native ARM64 Windows)
- arm64 detection in install.ps1
release.yml already publishes windows-arm64 (asset filter + merge-multiple) and
build_update_url/detect_arch already resolve it; npm/pip/install.sh compute arm64
already. scoop/winget/chocolatey per-version manifests are updated at release time.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The ubuntu-24.04-arm test leg (native ARM64 Linux running the full suite under
AddressSanitizer + the FastAPI incremental index) exceeds the 60-minute job
timeout and was killed, failing ci-ok even though the suite was on track to pass.
ARM + ASan is legitimately 2-3x slower than x86-64.
Raise the substantial compute jobs (test, build, bug-repro board, soak's short
legs, security/codeql, cross-platform smoke, fast-repro) to a generous 240-minute
(4h) cap so a slow-but-correct runner can't false-timeout. Trivial aggregator
jobs (5/10/15m: setup-matrix, ci-ok, dco, lint, license-gate, quick smokes) and
the intentional soak caps (300/320m) are unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Release archives now carry THIRD_PARTY_NOTICES.md, generated by
scripts/gen-third-party-notices.sh from THIRD_PARTY.md, the grammar
manifest, and the per-component license texts; the Homebrew formula
and AUR PKGBUILD install it alongside the binary. The SBOM gains
per-component license metadata, corrected versions, and the previously
missing vendored libraries. The security workflow gains a
vendored-license scan with an explicit allow-list policy, and the
release workflow exposes a skip_perf input for releases that do not
touch pipeline logic.
The macos-15-intel leg had continue-on-error, so when that runner was unavailable the darwin-amd64 artifact silently never built and releases shipped with no Intel macOS binary. Remove continue-on-error so the leg is blocking - the binary ships or the build fails loudly. macos-15-intel is GitHub's supported Intel image through Aug 2027.
GitHub is sunsetting Intel-macOS runners, so the macos-15-intel build leg is
repeatedly cancelled before it can be allocated, turning the aggregate build
result non-success and skipping the smoke + soak jobs even though every other
build/test passed.
Mark only the macos-15-intel matrix leg continue-on-error, and relax the
smoke/soak gate to run unless builds were skipped or a leg genuinely failed
(tolerating a cancelled aggregate caused solely by the non-blocking Intel leg).
All other platforms remain required.
Replace direct `${{ ... }}` interpolation in `run:` steps of `.github/workflows/_build.yml` with `env:` blocks per GitHub's hardening guide. Every workflow input (`inputs.version`) and matrix value (`matrix.cc`, `matrix.cxx`, `matrix.goos`, `matrix.goarch`, `matrix.arch`) now flows into the shell through a named environment variable rather than being textually spliced into the script.
Pattern applied uniformly across the 6 flagged steps plus the archive steps that used `${{ matrix.* }}` in tar filenames. The values become opaque shell variables on expansion, so attacker input becomes a literal argv entry rather than interpreted shell code. The if/else form on `$VERSION` avoids word-splitting pitfalls of conditional arg construction.
semgrep `run-shell-injection` findings: 6 → 0. Behaviour preserved — same build.sh invocations, same artefact names, same conditional version handling.
Reference: https://securitylab.github.com/research/github-actions-untrusted-input/Closes#247.
Local test infra now covers 4 platforms:
- Linux arm64 (Ubuntu, ASan+LeakSan)
- Linux portable (Alpine musl, static binary)
- Windows (mingw cross-compile)
- macOS (native)
Fixes:
- Remove sys/unistd.h and sys/poll.h (glibc-only, musl lacks them)
- Add STATIC=1 support to Makefile (appends -static to LDFLAGS)
- CI portable build uses docker run alpine instead of container directive
(GitHub Actions JS actions break in Alpine containers on ARM64)
dependabot[bot]