Completes the collapse to a single shipped composition and replaces the
zero-tolerance VirusTotal gate with a narrow, disclosed policy.
Packaging and installers
- package-release.sh loses --variant; archives are codebase-memory-mcp-<os>-<arch>
with exactly four members. install.sh/install.ps1 lose --ui/--standard.
- The extractor drops CBMUIPK pack parsing and --archive-scope; its scan-set and
association manifests (which the gate depends on) are unchanged otherwise.
- npm/PyPI/Go wrappers: the runtime "set" is one file again. The Windows lock
and race fixes from #1495/#1496 are kept; only multi-file set membership goes.
This also fixes `pip install` on Windows, which rejected the fifth archive
member against a hardcoded four-name allowlist.
- The wrappers' post-download probe moves from --verify-runtime-assets (removed)
to --version, which proves the same thing: the binary executes.
VirusTotal gate
- Exactly ONE detection is tolerated, and only when the engine is Microsoft AND
the label ends in `!ml`. Two or more engines, any non-`!ml` label, any other
vendor, any suspicious verdict and every infrastructure error still block.
- A tolerated object prints TOLERATED:, never OK:, and its counts are recorded
in vt-results.tsv exactly as a blocked one would be.
- append-vt-notes.sh mirrors the policy. It previously hard-failed on any
malicious count, so loosening only the gate would have passed the scan and
then died at note publication. The notes now DISCLOSE a tolerated detection
and link to SECURITY.md rather than claiming "0 malicious" for everything.
Rationale for the tolerance is in the gate itself: the verdict is not a property
of our bytes. It inverts across architectures and link modes, moves between
sibling artifacts of one build, and lands in different variant buckets for the
same source. The same `!ml` family hits llama.cpp, GitHub's own `gh`, Microsoft's
own Go toolchain and Anthropic's Claude installer.
The zero-tolerance contract becomes test_vt_gate_policy_contract.sh, asserting
the full matrix: 1x Microsoft !ml passes and reports TOLERATED; a Microsoft
signature label, a non-Microsoft engine, two engines, a suspicious verdict and
every malformed-response case still block. Its tripwire is narrowed to the
reverted endpoint-verification mechanism rather than the words "false positive",
so it no longer fires on a deliberate in-gate policy branch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every unix build leg of release run 30499236230 died during packaging:
scripts/package-release.sh: line 190: scripts/ci/check-binary-composition.sh: Permission denied
The composition gate was committed 100644 while being invoked as a command. It
passed every local check because my WORKING COPY had the exec bit -- only the
committed mode was wrong, and nothing you can run locally reveals that.
scripts/ci/append-vt-notes.sh had the identical defect waiting in the verify
step, the last step of the release, so this would have failed a second time after
two hours of tests, build, smoke and soak.
Fixed on BOTH sides, because either alone suffices and the pair is mode-proof:
the two scripts are now 100755, and their call sites invoke them through `bash`,
which is what most of this repo already does and which cannot break if a mode bit
is ever lost to a patch application or a non-POSIX checkout.
tests/test_script_exec_bit_contract.sh pins the class: any tracked .sh whose
COMMITTED mode is non-executable must not appear as the first word of a command
in workflows, scripts, test-infrastructure or the Makefiles. Verified in both
directions -- it passes on this tree, and fails on the exact defect when the mode
and the call site are reverted. It also joins backslash continuations before
analysing, because its own first draft reported a false positive on
... && bash \
test-infrastructure/vm/vm-run-tests.sh --soak
and a contract that cries wolf teaches people to ignore contracts.
Product code is untouched: the test phase of the failed run was 27/27 green on
this exact tree, and a file mode cannot change a test outcome.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The pre-release ML false-positive tolerance (single-engine Microsoft "!ml"
verdicts downgradable with Defender endpoint evidence, #1340) is reverted by
owner decision: cbm does not ship binaries carrying a VirusTotal detection,
demonstrably false or not. A "trojan" badge on a release asset is a
reputation cost the project is not willing to price in, however good the
accompanying evidence.
The gate returns to its original form: any detection, by any engine, on any
artifact, on any version blocks the release. The endpoint verification tool
and the evidence side-channel are removed with it; the notes renderer keeps
its extracted-script form but only ever states a verified "0 detections".
False positives are resolved upstream instead: verify the bytes on a real
Defender endpoint, submit a Microsoft false-positive report for the exact
hashes, wait for the detection to clear, then RE-RUN the failed verify job --
which does not rebuild, so the cleared hashes are the shipped hashes.
tests/test_vt_gate_zero_tolerance_contract.sh pins the decision: clean
passes; 1 malicious and 1 suspicious each block across stable, -rc., -pre
and -alpha versions; plus a tripwire for the specific reverted evidence
mechanism returning. Loosening this gate again has to consciously delete
that contract.
The skip_tests dispatch input and the script-extracted notes step survive
the revert -- both are orthogonal to gating policy.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Release run 30464288732 was blocked by the VirusTotal gate: three linux-amd64
binaries flagged 1/62 by Microsoft's Wacatac.B!ml -- fully stripped binaries
(0 symbols, verified on the exact artifacts), the state that scanned clean in
the two previous cycles. Meanwhile a real Defender endpoint (engine
1.1.26060.3008, signatures 1.455.410.0 updated the same day, RTP on) scans
the identical bytes clean. Four cycles of evidence now say the same thing:
this verdict is an unstable ML decision boundary, not a property of the code,
and no build-side lever moves it durably -- stripping, downloader removal and
metadata changes each "worked" only until a later build flipped it back.
So stop treating the flag as buildable-away and verify it honestly instead:
check-virustotal.sh may downgrade BLOCKED to TOLERATED only when ALL hold:
- pre-release version (-rc./-pre/-alpha/-beta); stable releases never
- every failing file flagged by exactly ONE engine
- that engine is Microsoft and the verdict ends in "!ml" (never a
signature name)
- hash-pinned Defender ENDPOINT evidence is attached to the draft release
(defender-endpoint-verification.txt) proving Microsoft's shipping
product, signature-updated at scan time, reports the exact bytes clean
av-endpoint-verify.sh (new) produces that evidence: downloads the draft
assets, scans them on the local Windows VM endpoint, refuses to attest if
RTP is off or Defender itself detects, uploads the hash-pinned result.
The gate prints the exact command when evidence is missing; re-running the
failed verify job does not rebuild, so the bytes stay fixed.
append-vt-notes.sh (new, extracted from inline YAML per venue-parity) then
renders the release-notes table honestly: a tolerated file reads "1/62 ML
false positive, endpoint-verified clean", never "0 detections".
tests/test_vt_gate_tolerance_contract.sh pins all nine decision directions
against a stubbed VT API and release store -- clean pass, stable-never,
missing/stale/DETECTED evidence, signature-named verdict, non-Microsoft
engine, multi-engine -- so the tolerance provably fails closed.
Also: release.yml gains skip_tests for re-releases of an already test-green
tree (build/smoke/soak/verify always run; lint failures still gate via
!cancelled() && !failure()).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel