The detector's 100x bound blocked the v0.10.1 release twice on macos-15-intel
with 101.1x — and control runs on a quiet arm64 host measure the SAME curve
as 57-68x, deterministically. The resolve path has never been linear for this
fixture shape: per-function cost grows 0.5ms -> 1.7ms from 100 to 2000
functions, the audited short-name-lookup/no-negative-memo superlinearity now
tracked as #1527. A bound calibrated as "linear plus generous overhead" on a
curve that was never linear measures host constants, not complexity.
The bound moves to 200x — mid-way in log space between the known ~60-100x
curve and clear quadratic near 400x — so a real quadratic regression still
fails by 2x margin while known behavior stops producing release-blocking
false alarms on slower hosts. The measured table is pinned in the test
comment; when #1527 fixes the curve, the bound comes back down to ~40x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three regressions shipped in 0.10.0 share one failure shape: an empty result
with a success status, indistinguishable from "nothing found" for the LLM
clients that are cbm's primary consumers.
1) structuredContent {} on the whole tree-format surface (#1522 bug 1).
#1488 replaced the duplicated payload with an EMPTY structuredContent
object while every tool still declared a blanket permissive outputSchema.
Spec-honoring clients (Claude Code among them) treat structuredContent as
THE result when a schema is declared, so search_graph, trace_path,
query_graph, get_architecture, search_code, and detect_changes all rendered
as literally "{}" on their DEFAULT format, on every platform. The corrected
contract: no tool declares an outputSchema (tool output is
format-parameter-polymorphic — no static schema is truthful), JSON-object
payloads keep their parsed structuredContent, errors keep
structuredContent.error, and text-shaped payloads carry NO structuredContent
key at all — which also preserves #1375's no-duplication win.
2) Frontend queue overflow killed the session (found by the #1522 sweep).
Any 7+ requests pipelined in one stdin burst — an agent issuing parallel
tool calls does exactly this — overflowed the 8-frame frontend queue, which
failed the whole session: rc=1 with ZERO bytes of output, every buffered
response lost. A full queue is now backpressure: the stdin reader blocks
until the worker drains (bounded by the same stop/fail flags every teardown
path already sets); only a single frame larger than the entire 12 MiB byte
budget — which could never be admitted — remains a hard failure.
3) config get printed "" with exit 0 for every unset and every unknown key
(#1522 bug 2). list printed stored-or-DEFAULT while get printed
stored-or-EMPTY, and no subcommand validated key names, so a typo was
indistinguishable from a correctly-read setting. One config-key table now
drives help, list, get, set, and reset: get prints the stored value or the
key's real default (the same fallback the runtime readers use), and unknown
keys error with exit 1 on get, set, and reset alike.
Tests — each RED on the pre-fix tree and RED again on revert:
* test_mcp.c: text results carry no structuredContent key; tools/list
declares no outputSchema; the tool-table guard now binds all three
branches (absent / parsed-object / error) for every registered tool.
* test_daemon_frontend.c: the over-capacity contract flips from
"session fails" to "backpressure without loss" — the held first request
plus all 32 over-capacity frames are answered and the run closes cleanly.
* test_cli.c: the config command contract — defaults, round-trip, reset,
and unknown-key rejection on all three subcommands.
* smoke-test.sh: Phase 3z rewritten to the corrected structuredContent
contract, new Phase 3z1 (default-format replies usable in schema-honoring
clients, no outputSchema advertised), 3z2 (24 pipelined calls all
answered), 3z3 (config defaults + unknown-key rejection) — all asserted
against the SHIPPED artifact, where #1488's smoke phase previously pinned
the empty-object behavior as correct.
Verified end-to-end on the locally built production binary: all 15 tools
declare no schema; text tools return ABSENT structuredContent, object tools
populated, error envelopes intact; 7/24/64-deep pipelined bursts all answered
with rc=0; config get prints real defaults and exits 1 on unknown keys.
hook_augment (structuredContent.projects) and index_resilience
(structuredContent.status) consume object payloads and are unaffected.
Fixes#1522.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
test_py_lsp_bench.c:262 failed the v0.10.0 release run TWICE on genuine
attempts (ubuntu-24.04-arm 3/3), while the SAME sha passed that shard in the
morning's dry run. A wall-clock bound on a shared 4-vCPU runner is not a
property of the code, and this repo's testing doctrine says exactly that:
perf tests assert a ratio; a time bound may exist only as a liveness backstop.
This test's perf content already IS ratio-form (resolution ratio with a
sanitizer-aware floor). The 150/1500 ms budget was the anti-pattern half.
It becomes a 30 s liveness backstop so a catastrophic hang still fails the
suite rather than the 900 s shard wall clock, and absolute timings stay where
they belong: scripts/benchmark-*.sh, which record rather than gate.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both Linux soak-quick legs in the dry run failed with:
FAIL: RSS 17592186044293MB > 200MB ceiling
17592186044293 MB is 2^44-123 -- a NEGATIVE ~123MB wrapped through size_t. The
diagnostics snapshot reported mi_process_info()'s current_rss verbatim, and on
Linux mimalloc never sets that field: it keeps mimalloc's internal
committed-page counter, which this project deliberately tunes low
(purge_decommits=1, purge_delay=0) and which can transiently go negative under
purge. macOS and Windows are untouched -- mimalloc reports real RSS there --
which is exactly the observed split: both Linux legs red, both macOS legs green.
mem.c already learned this lesson in full (its comment block documents the same
trap for cbm_mem_over_budget): cbm_mem_rss() is /proc-statm-primary on Linux and
mimalloc-correct elsewhere, and cbm_mem_peak_rss() reconciles peak >= current.
The diagnostics writer now uses both instead of the raw mi fields, so the soak
leak gates measure actual process RSS on every platform.
Verified: diagnostics suite passes; compiles and formats clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first full dry run after #1508 failed in all three smoke-linux-portable
legs: _smoke.yml still expanded a variant matrix and extracted
codebase-memory-mcp-ui-<os>-<arch>.tar.gz — a name the build no longer
produces. PR CI never sees this job (pr.yml calls the smoke wrappers directly),
so the miss only surfaced in the dry-run/release path this workflow serves.
The matrix loses its variant dimension, all three legs extract the unsuffixed
archive, and the positional/SMOKE_VARIANT plumbing is replaced by
SMOKE_REQUIRE_UI=1: these legs smoke the SHIPPED artifact, so a binary serving
no embedded UI is a defect here, exactly like scripts/ci/smoke-artifact.sh.
Verified locally: venue-parity and smoke-fixture contracts pass; YAML parses.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Root cause of the Cyrillic guard failure, found via the finalize diagnostics:
seal_existing rc=-1. The primary store open normalizes the path through
cbm_path_for_file_api (UTF-8 -> wide -> UTF-8, the form SQLite's Windows VFS
needs), but four maintenance opens passed the raw UTF-8 path:
cbm_store_seal_existing_path_for_replace (the one this path hits)
cbm_store_prepare_path_for_replace
cbm_store_backup_path (both source and staging)
cbm_store_dump_to_file
So a generation was WRITTEN through the normalized path and then could not be
RE-OPENED through the raw one. On the second index into a cache the seal step
failed, finalize returned PERSIST_FAILED, and the user was told "Check repo_path
exists and contains source files" about a repo that had just indexed cleanly
(nodes=12 edges=21 in the same worker log).
On POSIX cbm_path_for_file_api is a no-op memcpy, which is why the full 3-OS
ladder and every non-Windows test passed -- the defect is Windows-only and only
manifests on a path the raw UTF-8 -> SQLite open mishandles. It surfaced on the
Cyrillic case first because the guard indexes the same repo TWICE (once for the
count, once on retry), making it the first case to re-open an existing
generation; the earlier ASCII/latin1 cases index into fresh caches.
All four now use cbm_path_for_file_api exactly as the primary open does. Also
upgrades dump_to_file from sqlite3_open to sqlite3_open_v2 with explicit flags.
The finalize.prepare_failed / rename_failed diagnostics from the previous commit
stay: a silent PERSIST_FAILED that blamed the repo cost a day of remote bisection
and must never be silent again.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Cyrillic guard failure came down to this: the worker indexes the repo
perfectly (gbuf.dump nodes=12 edges=21, DB written, sealed, sidecars staged) and
then finalize fails somewhere between stage_sidecars and prepare_live -- and
every failure edge on that path returns PERSIST_FAILED without a word. The user
then reads "Pipeline failed. Check repo_path exists and contains source files"
about a repo that indexed flawlessly.
A silent failure that misattributes itself cost a full day of remote-only
bisection: five instrumentation commits in the TEST were needed to even locate
the failing function. That is a product defect independent of the original bug.
Every edge now logs before returning -- destination-not-regular, seal rc,
sidecar-cleanup errno (both branches), and the final rename with stage and
destination paths. The next guard run names the exact edge and value instead of
requiring a sixth probe.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The CLI probe reproduces the empty pipeline (rc=1, same one-line hint), so the
remaining record is the worker's own log -- which the supervisor unlinks on a
clean exit. CBM_PROFILE=1 flips that exact branch to retention
(index.supervisor.profile_log) with no product change, and the probe now prints
every retained log tail from its isolated cache.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The retry with a fresh cache fails identically, so this is path-dependent, not
order-dependent — the daemon-lifetime theory is dead. The MCP result compresses
the pipeline's own diagnostics into one hint string; the documented CLI
entrypoint prints them. Third fresh cache, same repo, output verbatim.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The worker log shows the discriminating shape: the supervisor reaps the worker
CLEAN (exit 0, 2.3s) yet index_repository reports an empty pipeline -- and the
test daemon now serves the UI on fixed port 9749, which main's build does not
do here (no pack beside the test binary keeps its UI off, while the embedded
build auto-enables). Sequential per-case daemons therefore contend on one port,
and Cyrillic is simply the SECOND case in the run order.
An in-process retry against the same repo with a fresh cache separates the two
remaining explanations: a passing retry convicts the previous case's daemon
lifetime (order), a failing retry convicts the path itself.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The previous round narrowed Cyrillic-on-x64 to: correct project name derived,
server responsive, but the pipeline reports 'Check repo_path exists and
contains source files' and only _config.db + logs/ land in the cache. The
supervisor wrote worker logs, so the actual error is already on disk in the
failing job -- print their tails plus the repo listing instead of guessing
across another cycle.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Cyrillic fails deterministically on the x64 runner and passes on the arm64 VM
and on the main-based sibling PR at identical path depth, so neither script nor
path length explains it. The remaining unknown is what index_repository
actually said and whether anything landed in the isolated cache; print both.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Cyrillic case fails deterministically on the GitHub runner (two genuine
attempts, fresh temp roots each time) while latin1/CJK/Greek pass around it and
the SAME commit passes all cases on the real-Windows VM. The guard records an
'error' field for exactly this situation and then never prints it, so the CI log
shows nodes=None and nothing else. Surface it verbatim; the next run carries the
diagnosis instead of a guess.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Same restoration as the pypi wrapper, same reasoning: serialization of
concurrent publishers, killed-publisher reconciliation via locked readiness,
the crash-phase matrix, and multiply-linked-leaf rejection are properties of
the LOCK design, not of how many files publish.
Adaptations for one file: the mid-publication pause hooks the binary rename
itself (the runtime lock is held until publish returns, so the serialization
property is exercised exactly as before); the "partial sidecar publish" crash
phase is gone with the sidecars; backup-member expectations drop to the single
member, including cleanup-interrupted leaving 0 rather than 1; and the
hardlink-rejection guard now targets the binary leaf.
go vet + gofmt clean; full Go suite ok including the restored
TestConcurrentRuntimePublishersAreSerialized,
TestKilledRuntimePublisherIsReconciledByLockedReadiness (5 crash phases),
TestRuntimePublicationCrashHelper and
TestRuntimeReadinessRejectsMultiplyLinkedLeaves.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Deleting these with the multi-file runtime set threw away properties that do not
depend on how many files publish: two wrappers racing to publish the SAME cache
still must serialize, a killed publisher still must be reconciled by locked
readiness rather than trusted, a failed publication still must restore the prior
binary, and a complete foreign winner still must never be deleted.
Restored from main and adapted: the staging fixtures publish one binary instead
of binary+integrations+pack, and the assertions that read sidecar bytes are gone
with the sidecars. Each surviving test keeps its lock/race logic intact
(7/10/5/3 assertions respectively, all still reading published binary bytes).
Revert-checked against production, not just green: disabling
_runtime_backup_target_name makes test_publication_failure_restores... fail and
restoring it passes. pypi suite: 28 tests, OK.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The new contract hit the WSL-alias trap on its first Windows CI run: BASH is a
shell variable, NOT exported, so os.environ.get("BASH", "bash") silently fell
back — and native Windows resolved bare 'bash' to the WSL stub, which fails
every invocation with 'Windows Subsystem for Linux has no installed
distributions' before the extractor runs at all.
This is the exact pattern the venue-parity contract bans; it slipped past that
rule's regex because the literal sits inside a .get() fallback rather than a
bare argv element. Fixed the repo's documented way (df2088a7): the shell wrapper
passes "$BASH" in as argv and Python uses it verbatim.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Ran the format gate before adding --dir rather than after, so the ternary
landed unformatted and lint/lint went red. Reformatted with the pinned Homebrew
LLVM clang-format; the diff is confined to that block and cli.c still compiles.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Deleting this contract with the UI-pack architecture threw away coverage of
machinery that SURVIVED: the extractor still builds the scan-set and association
manifests that check-virustotal.sh reads. Without it the gate could report a
confident green over an incomplete set.
It immediately paid for itself. The extractor was BROKEN by my own edit:
NameError: name 'variant' is not defined (process_tar)
TypeError: add_association() missing 'variant'
Removing the variant from validate_namespace's return left the call sites still
threading it. python3 -m py_compile passes on that -- it is a runtime failure --
and the extractor runs ONLY in the release and dry-run VirusTotal jobs, so
nothing else in the tree or on the 3-OS ladder executes it. It would have
shipped and broken every release verification.
The contract is rewritten rather than adapted: the original was woven through
the pack format (16 archives, 88 associations, per-pack asset math), and a
half-adapted version that passes for the wrong reasons is worse than none. It
now asserts what still exists:
- the exact 8-archive matrix, 32 member associations, nothing dropped
- archive containers never enter the scan set as members
- deduplication is exact: 8 byte-identical LICENSE members collapse to ONE
scan object while 8 distinct binaries stay 8
- manifest metadata agrees with its own rows, and every staged object matches
its recorded SHA-256
- fail-closed on a surplus member, a missing member, and a short matrix
Registered as Step 0l so it runs in every venue, not just when someone
remembers.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows long-path guard failed with:
error: unknown uninstall option: --dir=C:\...\install_Ωμέγα\...\bin
Restoring cli.c from before the externalization dropped --dir from uninstall
while install kept it. That is worse than a test failure: anyone who installed
outside ~/.local/bin had no supported uninstall path at all — uninstall would
look in the default location, find nothing, and leave the real install in place.
Parsing mirrors cbm_cmd_install (--dir=PATH and --dir PATH, both rejecting an
empty value), and the resolved directory now feeds the target path with an
explicit truncation check instead of an unchecked snprintf into a fixed buffer.
Confirms the earlier cascade read: with the extended-path probes fixed,
daemon_stability, hook_augment, cli_non_ascii_arg and ui_drive_listing all went
green on their own. non_ascii_path was the only real failure, and it was hiding
two distinct product regressions behind one red guard.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
test_non_ascii_path.py caught a real product regression I introduced:
[FAIL] long-path existing target was overwritten despite --no
(target existence probe is not extended-path safe)
Restoring cli.c from before the externalization also reverted three probes from
cbm_path_info_utf8() back to plain stat(). On Windows stat() resolves through
the ANSI code page, so an extended-length or non-ASCII target reports "absent" —
and a non-force `install --no` then overwrites the exact bytes the user asked to
keep. Silent data loss, not a cosmetic difference.
That fix rode along in the externalization commit but is orthogonal to it: the
wide-path helpers it depends on (win_utf8.h, compat_fs.c) were kept. Only their
call sites in cli.c were lost.
Two of main's five cbm_path_info_utf8 sites lived in asset code that is
correctly gone; the two restored here are the install target probe and the
uninstall binary probe.
Not yet explained: test_daemon_stability.py also went red in the same run
("daemon did not accept the UI configuration"). Its code path is identical to
main's apart from the intended capability predicate, and the guard that failed
before it completes an install into a long path it should have refused, so this
may be downstream state. Pushing this fix to find out rather than guessing.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pr-smoke was green but test-windows-guards failed, and the shape of the failure
named the cause:
[PASS] non-ascii/latin1_accents nodes=12 edges=21 definitions=5
[FAIL] non-ascii/cyrillic nodes=None edges=None definitions=None
ASCII and latin1 pass, Cyrillic does not -- latin1 survives a Western ANSI code
page and Cyrillic does not. But the product code was never the problem: the
wide-path helpers (win_utf8.h, runtime.c extended paths) were KEPT. I had
reverted this test to a base that predates them, so it lost windows_extended_path()
and the explicit `executable=` argument to subprocess.run. With
lpApplicationName NULL, CreateProcess applies the legacy MAX_PATH limit while
deriving the module name from the command line, so on CI's long guard-bundle
path the process never started and every count came back None.
Same mismatched-base class as the package wrappers: production kept the fix, its
test was reverted past it.
Restored from main and adapted rather than dropped -- it still covers
extended-length publication AND non-ASCII indexing. Only the sidecar
assertions are gone (one self-contained file to publish now), and the probe
uses --version instead of the removed --verify-runtime-assets.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Making REQUIRE_UI unconditional broke all three pr-smoke legs:
FAIL 15a: SMOKE_REQUIRE_UI=1 but this binary serves no embedded UI assets
The PR lane runs scripts/build.sh WITHOUT --with-ui on purpose, so an npm
frontend build does not land on every product PR. Demanding embedded assets
there asserts a property that lane deliberately does not produce.
The guard still exists where it means something: scripts/ci/smoke-artifact.sh
builds --with-ui, packages the real archive and smokes the EXTRACTED result, so
it exports SMOKE_REQUIRE_UI=1 and a frontend-less binary fails there. That is
the lane whose whole job is release fidelity.
Deliberately NOT fixed by adding --with-ui to pr.yml: that would put `npm ci &&
npm run build` on three runners for every product PR to re-prove something the
dry-run and release lanes already gate. PR CI cost is not free and this branch
should not quietly raise it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The arm64 W^X hole regressed silently once and section flags cannot catch it:
.rodata is correctly marked A (not AX), but the kernel applies SEGMENT
permissions, so a merged R E PT_LOAD maps it executable anyway. A Makefile flag
alone would regress the same way the next time a linker default differs.
A1b sums the executable PT_LOAD segments and fails when they cover most of the
file. Measured on aarch64 with a 1.2 MB const table:
without -z separate-code FAIL executable segments cover 96% of the file
with -z separate-code PASS executable segments cover 0%
Nearly shipped this vacuous: the first version summed with awk's strtonum(),
which is a gawk extension. CI's awk is mawk, where it is undefined -- the total
would have been 0 and the gate would have passed on exactly the artifacts it
exists to catch. It now emits the hex MemSiz fields and converts in the shell,
and the numbers above were produced with the container's mawk, not gawk.
Mach-O and PE report n/a; the real macOS artifact still passes all 13 assertions.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Restoring files from two different pre-revert commits left the package wrappers
internally inconsistent, and one of them did not compile. Found by actually
building and running them rather than by grepping for removed symbols.
pkg/npm — bin.js was never restored, so main's version was destructuring
runtimeSetReady/acquireRuntimeLock out of an install.js restored from an older
base that does not export them. `npx codebase-memory-mcp` would have died at
require time on Windows. install.js, bin.js and the launcher test now all sit on
the pre-externalization base, where installWindowsBinaryAtomically is the right
primitive for publishing exactly one file. The runtime-set locking that goes
away with it existed only to publish binary+sidecars atomically.
pkg/go — main.go kept the Windows lock work while main_test.go had been reverted
past it, so the package did not build at all:
main.go:805: no new variables on left side of := (my edit dropped uiPack)
main_test.go: undefined: windowsPayloadName (reverted past its removal)
The test is back on main's base, keeping every lock-race test from #1495/#1496
(TestRuntimeSetLock*, TestExpiredLease*, TestStalledRuntimeLockCreator*,
TestRuntimeSetLockSerializesProcesses). Dropped 19 whose premise -- a multi-file
runtime set with content-addressed sidecars -- no longer exists, and reduced the
writeTestRuntimeSet/assertRuntimeTag helpers to the single file that now ships.
pkg/pypi — same base repair, keeping the lock and orphan-reconciliation tests.
The three archive-SAFETY tests (hardlink members, symlink metadata, unexpected
root member) were ADAPTED rather than dropped: they only named a removed file in
their fixtures, and they guard traversal properties that still matter.
test_release_archives_require_the_integrations_sidecar is inverted rather than
deleted -- asserting the sidecar is ABSENT is the stronger guard, since a
returning sidecar is exactly what broke `pip install` on Windows.
Verified: go vet + go build + go test ok; pypi 24 tests OK; npm 10/10;
Windows single-binary bundle contract still passes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Deleting the archive-extractor contract took an assertion with it that had
nothing to do with the UI-pack architecture that contract tested: release.yml
and dry-run.yml must stage downloaded archives into $RUNNER_TEMP, not into the
tracked checkout, so the extractor never scans a directory that also holds repo
files.
The fix itself survived the revert; only its guard died. Moved to the
venue-parity contract, which already reads both workflows and is registered in
scripts/test.sh.
Revert-checked: reverting release.yml to ARCHIVE_DIR="assets" makes this fail
with the exact message, and restoring it passes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Wiring this contract into scripts/test.sh Step 0l made it run on the Windows VM
for the first time, where it failed with 'cmp: command not found' -- MSYS2 does
not ship cmp, and diff is not guaranteed either. The script already requires
python3, so compare the bytes there instead.
The failure was real coverage working as intended: the contract had never run on
Windows before, because it was registered only in the test.sh version that this
branch replaces.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Restoring THIRD_PARTY.md from before the UI externalization also dropped a note
that commit had added for an UNRELATED reason: two loop counters in the vendored
objectscript_common.h are widened from uint8_t to int.
That deviation is real, still present in the tree, and recorded in the grammar
MANIFEST. Dropping its THIRD_PARTY entry would have left a vendored-source
modification documented in only one of the two places the vendoring policy
requires -- collateral damage from a wholesale file restore, not an intended
part of this revert.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Its PASS line still advertised the old policy, which is exactly the string a
future reader would trust when deciding what the gate enforces.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
ui_server_root_without_ready_assets_is_retryable asserted the pack architecture's
readiness semantics: the daemon warmed an external pack asynchronously, so a
request could legitimately arrive before the assets existed, and 503 +
Retry-After told the client to come back.
With the frontend linked into the image there is no warm-up window. Whether the
UI exists is decided at build time: a binary built without --with-ui has no
index.html and never will. Promising a retry for a condition that cannot change
would make every client poll forever, so this is a 404.
The replacement also asserts Retry-After is ABSENT, so the retryable behavior
cannot quietly come back with the header still attached.
macOS suites after this: cli 261 passed; ui/httpd/mcp/daemon* 469 passed with
this as the only failure, and httpd re-runs 62 passed / 0 failed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/embed-frontend.sh generates src/ui/embedded_assets.c into the SOURCE
tree (the linker needs it beside the other UI sources, and scripts/clean.sh
removes it). It was never gitignored, so a --with-ui build leaves a large
untracked generated file sitting in `git status` waiting to be committed by
accident. It predates this branch, but this change makes that build path the
default one, so close it here.
smoke-local.sh still documented a `ui` argument that no longer exists.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Caught by running the contracts rather than by inspection:
- package-release.sh built its archives from an ARCHIVE_MEMBERS array, which
only ever existed to append a UI pack conditionally. With a fixed four-file
inventory the array hides the member list from the very contract that is
supposed to pin it, so the members go back inline on the archive call.
- test_windows_bundle_contract.sh asserted the pypi wrapper exposes
_windows_binary_ready(). That function became _runtime_set_ready() when the
wrapper locks were reworked; the contract now asserts the current API, since
the property it guards is "resolves the single Windows binary", not a name.
- test_smoke_fixture_contract.sh required smoke-local.sh to emit both a
variant-suffixed archive and a standard alias. There is one archive name now,
so it asserts that instead -- and that ${SUFFIX} is gone entirely, so the
alias machinery cannot quietly return.
Verified on the built artifact: the binary runs, carries 9 shebangs, 1
node:child_process and 14 hook-augment references (the deliberately inverted
property), and contains zero references to cbm-integrations.json, CBMUIPK or
--verify-runtime-assets.
Contract status on macOS: build_dir_safety, makefile_ts_runtime_dependencies,
parallel_harness, security_fuzz_harness, security_strings_allowlist,
soak_daemon_recovery, spawn_no_window, ui_dev_proxy_security, venue_parity,
vm_worktree_manifest, windows_bundle, smoke_fixture, vt_gate_policy and
vt_release_notes all pass. hook_conflict_notice, parent_watchdog and
worker_error_response fail for the documented environmental reasons (TEST_SEAMS
build seam, and the developer's own installed daemon holding the version
cohort), not from this change.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
SECURITY.md gains an "Antivirus False Positives" section placed directly after
the existing Verification block so it reuses those commands rather than
restating them. It states the detection plainly, shows the measured evidence
(the verdict inverts across architectures and link modes; identical macOS
segment structure splits clean/flagged; entropy is 3.5-4.2 bits/byte, nowhere
near packed), lists the cross-project precedent, and — deliberately — records
that removing the embedded scripts and externalizing the assets did NOT move the
detection. A negative result is still evidence, and publishing it is the point.
It also documents the release policy, invites independent audit through the
provenance/cosign/checksum path we already ship, and adds an `av-analysis` issue
label route for anyone who finds something real. Tone is evidence-first: no
vendor blame, and an explicit note that signing helps Windows but that no
AV-honoured signing scheme exists for Linux ELF, so it is not a full answer.
README carries a short pointer near Quick Start, where someone who just hit a
Defender warning will actually look.
Unrelated to AV, found while dissecting the artifacts: add -Wl,-z,separate-code.
GNU ld enables it by default on x86-64 but not on aarch64, so arm64 emitted ONE
R E PT_LOAD covering the whole image and mapped ~259 MB of tree-sitter parse
tables executable at runtime. Section flags said A, not AX -- but the kernel
applies segment permissions, so the section flags were never the control. The
amd64 build of the same source mapped the same data R only.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Completes the collapse to a single shipped composition and replaces the
zero-tolerance VirusTotal gate with a narrow, disclosed policy.
Packaging and installers
- package-release.sh loses --variant; archives are codebase-memory-mcp-<os>-<arch>
with exactly four members. install.sh/install.ps1 lose --ui/--standard.
- The extractor drops CBMUIPK pack parsing and --archive-scope; its scan-set and
association manifests (which the gate depends on) are unchanged otherwise.
- npm/PyPI/Go wrappers: the runtime "set" is one file again. The Windows lock
and race fixes from #1495/#1496 are kept; only multi-file set membership goes.
This also fixes `pip install` on Windows, which rejected the fifth archive
member against a hardcoded four-name allowlist.
- The wrappers' post-download probe moves from --verify-runtime-assets (removed)
to --version, which proves the same thing: the binary executes.
VirusTotal gate
- Exactly ONE detection is tolerated, and only when the engine is Microsoft AND
the label ends in `!ml`. Two or more engines, any non-`!ml` label, any other
vendor, any suspicious verdict and every infrastructure error still block.
- A tolerated object prints TOLERATED:, never OK:, and its counts are recorded
in vt-results.tsv exactly as a blocked one would be.
- append-vt-notes.sh mirrors the policy. It previously hard-failed on any
malicious count, so loosening only the gate would have passed the scan and
then died at note publication. The notes now DISCLOSE a tolerated detection
and link to SECURITY.md rather than claiming "0 malicious" for everything.
Rationale for the tolerance is in the gate itself: the verdict is not a property
of our bytes. It inverts across architectures and link modes, moves between
sibling artifacts of one build, and lands in different variant buckets for the
same source. The same `!ml` family hits llama.cpp, GitHub's own `gh`, Microsoft's
own Go toolchain and Anthropic's Claude installer.
The zero-tolerance contract becomes test_vt_gate_policy_contract.sh, asserting
the full matrix: 1x Microsoft !ml passes and reports TOLERATED; a Microsoft
signature label, a non-Microsoft engine, two engines, a suspicious verdict and
every malformed-response case still block. Its tripwire is narrowed to the
reverted endpoint-verification mechanism rather than the words "false positive",
so it no longer fires on a deliberate in-gate policy branch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Externalizing the integration templates (#1492/#1493) and the UI bundle
(#1501/#1503) was done to reduce the Microsoft `Wacatac.B!ml` surface. It did
not work: across dry runs the flagged artifact count stayed at ~3 and the
detections merely moved between artifacts.
Dissection of run 31286803592 shows there is no structural cause to fix. The
verdicts split across every axis at once — linux-amd64 (dynamic) flagged while
linux-amd64-portable (static) is clean, but linux-arm64 (dynamic) clean while
linux-arm64-portable (static) is flagged. The two macOS binaries have identical
segment structure and split clean/flagged. Siblings from one build landed in
different variant buckets (.B vs .C). Entropy is low everywhere
(code_vectors.bin 4.166, grammar tables 3.464 bits/byte, against 7.5-8.0 for
packed payloads), so the packed-payload hypothesis is excluded too.
So the complexity bought nothing, and installation goes back to being
self-contained: one binary that carries its own UI and agent integration
templates, with no adjacent data file that has to resolve before `install`
works. Only the UI-capable composition ships from now on, under the historical
unsuffixed archive name.
Removed: src/ui/asset_pack.{c,h}, asset_pack_stub.c, asset_manifest_stub.c,
scripts/pack-ui-assets.mjs, src/cli/integration_assets.{c,h},
assets/cbm-integrations.json, scripts/gen-integrations-hash.sh, the
--verify-runtime-assets probe (nothing adjacent left to verify), and the
composition gates A6/A7 whose property is now deliberately inverted.
Restored: scripts/embed-frontend.sh, src/ui/embedded_{assets.h,stub.c}, the
compiled-in hook/adapter template bodies, and the embed/EMBED_OBJS build path.
Kept from the reverted commits, re-applied by hand where a wholesale file
restore would have dropped them:
- cbm_module_path_utf8() in both self-path sites. GetModuleFileNameA renders
through the ANSI code page and mangles non-ASCII install paths.
- the /__cbm/ui-readiness HMAC proof, secure_random and cbm_hmac_sha256, so
`daemon start --open` still waits for a genuine CBM listener.
- X-Content-Type-Options: nosniff on served assets.
- the MinGW noexecstack gate, -lbcrypt, and the cppcheck/zip CI fixes.
Archives are now codebase-memory-mcp-<os>-<arch>[-portable] with exactly four
members (binary, LICENSE, installer, THIRD_PARTY_NOTICES.md). That restores the
names every static package manifest already points at — aur, chocolatey,
homebrew, scoop, winget and glama were all broken by the -ui- rename.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel