- Windows updates run from install.ps1, not the running binary: a running
executable cannot replace its own image on Windows. Document the exact
command, that re-running the script IS the update, and the Unblock-File
caveat for Mark-of-the-Web. npm/pip users keep updating via their package
manager on every platform.
- Build from source: state that every platform produces ONE self-contained
binary with no companion executables, and document how to run the suite
(scripts/test.sh, the same entry the CI gates use) and how to package an
archive locally with the canonical script.
- Test badge 5604 -> 6768, the current count from build/c/test-runner.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Windows shipped a PAIR: a small permanent launcher (codebase-memory-mcp.exe)
plus the real product binary (codebase-memory-mcp.payload.exe). The launcher
existed for exactly one reason — a running .exe cannot replace its own image
on Windows, so an in-process self-update needs a second resident binary to do
the swap.
That stub is statically indistinguishable from a dropper: a small, unsigned,
zero-prevalence PE whose whole job is verify-and-execute another binary.
Defender's ML scored it Trojan:Win32/Wacatac.B!ml and blocked the v0.9.1-rc.1
release at the VirusTotal gate. It is not fixable in our code on x64 —
bcrypt-free, stripped, VERSIONINFO'd, minimal-resource and even
resource-FREE builds on CI's own MSYS2 CLANG64 toolchain were all flagged,
while the product binary scans clean on every platform.
So remove the stub and move self-update OUT of the process into install.ps1,
which runs while cbm is NOT running: Windows' image lock only blocks a
process from replacing ITSELF. now prints the exact PowerShell
command (with the Unblock-File hint for Mark-of-the-Web); install.ps1 is
idempotent, so re-running it IS the update — it stops the daemon, renames the
running binary aside (the one mutation Windows permits on a running image),
publishes the new one, and sweeps retired copies.
Windows now matches Linux and macOS: ONE binary per platform.
* packaging, install.ps1, npm and PyPI wrappers all carry a single binary
* the launcher/payload ABI contract and ~2500 lines of stub state machinery
are deleted
* every daemon start, CLI call and hook fire loses a process spawn, a named
pipe handshake and an stdio relay
* test_windows_bundle_contract.sh is rewritten as an INVERTED contract: it
now asserts no shipped surface can reintroduce a launcher/payload pair,
and that install.ps1 retires the running binary before publishing
Verified: VirusTotal 0/67 on the packaged binary and 0/58 on install.ps1 (no
certificate involved); macOS and Linux full suites green; Windows guards all
green including the new update-handoff contract; npm 10/10; PyPI 3/3.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The fresh-request guard test waited for 'job terminal AND >= N/2 subscribers
still attached'. Publish flips terminal and releases the blocked prior
requests in the same locked step, so that conjunction is a transient window
whose width is thread-wakeup scheduling: the busy-spin caught it by luck,
a 1ms-yield poll (#1307) sampled past it on fast x64 runners, and no budget
can pin it (release runs 30305464193 and 30309182389 failed it from both
directions).
The production guard never consults subscriber counts —
application_find_active_job_locked skips any terminal job — and the test's
downstream assertions (starts==2, destroys==2, stale/fresh response
separation) catch a terminal-job reuse in every interleaving. So wait only
for the stable end-state (active jobs == 0) and drop the racy helper.
47/47 locally.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
app_wait_for_terminal_job_with_subscribers was the only app_wait_for_*
helper polling without cbm_usleep(1000): a sleepless spin pins a core for
up to 10s and can starve the daemon threads it is polling on scarce-CPU
runners. Release run 30305464193 hit exactly this on windows-11-arm
(test_daemon_application.c:2671 ASSERT(terminal_with_prior_subscribers));
the suite passes 47/47 locally with the yield in place.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The blocked-git child's PID-marker wait was a fixed 5s. On the 4-vCPU
windows-11-arm release runners that budget was missed twice in a row
(run 30300256783, attempts 1+2: test_watcher.c:1038 ASSERT(marker_ready))
while every faster leg stayed green — spawn latency under CPU starvation,
not a watcher defect. Raise the four spawn-marker waits to a shared 20s
budget (WATCHER_TEST_SPAWN_MARKER_BUDGET_MS). The wait stays bounded: a
genuinely wedged spawn still fails, at 20s instead of 5s. Termination,
completion, and join budgets are untouched.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A version containing a prerelease suffix (v0.9.1-rc.1) now publishes as an
RC on every channel instead of masquerading as stable:
- GitHub release is created with prerelease: true (never marked Latest)
- npm publishes under the `next` dist-tag, so plain `npm install` keeps
serving the last stable; testers opt in with codebase-memory-mcp@next
- the MCP Registry sync is skipped (the registry has no channel concept)
- PyPI needs no change: PEP 440 already treats rc versions as pre-releases
that pip only installs with --pre
Stable releases (no dash) are byte-identical in behavior.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
v4's PASSING smoke legs already ran 12-14 minutes against the 15-min
timeout; v5 lost 13 green-path jobs to that wall (GitHub reports job
timeouts as 'cancelled', zero real failures). The unified smoke
legitimately runs more than the pre-unification one — artifact mode,
real download/update phases, a Phase 15 that actually executes.
Budget doctrine (see _soak.yml, violated twice there historically):
the timeout sits above the worst case with headroom.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both hosted-runner Windows soak legs failed at Phase 1/diagnostics
with an empty frontend stderr while the daemon's own log showed a
healthy diagnostics.start: the workflows pass the suffix-less binary
path, msys resolves it transparently (so the old 'append .exe only
when the plain name is not executable' branch never fired), and
soak-test keys BOTH native-Windows gates — the cygpath'd
CBM_CACHE_DIR and the coproc stdio branch — off the literal .exe
suffix. A suffix-less native binary therefore received a POSIX-form
cache path: daemon logs mis-rooted (diagnostics grep zero, arm64) and
the frontend's cache path handling died silently (x64 SIGPIPE at
first write).
Reproduced on the VM's CLANG64 environment with a suffix-less
invocation (same FAIL) and red->green proven: with the normalization
the identical invocation runs the full quick soak to PASSED.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The version preflight PASSED (both images printed their version) and
then the job died in the cleanup trap: Windows keeps the just-exited
image's file busy for ~100ms (section teardown / first-touch AV), so
'rm -rf' in the EXIT trap raced it — 'Device or resource busy', exit 1
on a job whose actual work was green (observed on windows-11-arm ui in
dry run 30232946513). Temp-dir cleanup on an ephemeral runner is
best-effort with one retry, never a gate.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both Windows runner soak legs died right after 'server running' while
the daemon's own log shows a healthy diagnostics.start with valid
paths — the wait polled 10s for a snapshot whose first WRITE lands
one 5s interval after start and can exceed 10s on a cold 4-vCPU
runner mid-initial-index (the VM's 18 cores never miss it; the x64
leg's SIGPIPE was the same failure path dying inside the pipeline).
Budget doctrine: the wait sits above the worst case.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Hosted Windows runners reset the readiness poll's full-archive GET
mid-body (WinError 10054 on the server, ~99 aborted transfers per
job) while the identical stack passes on the VM; every windows smoke
job died at 'fixture server did not serve' with a healthy server. A
HEAD proves the artifact routes without transferring the body per
poll; the download phases still own full-body transport and report
phase-precise if that is ever broken.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every ui-variant smoke job failed on every platform in dry run
30226483380 — and it turns out a ui-variant Phase 15 had never
legitimately passed anywhere: the old probe (a) never passed --ui=true,
but the HTTP UI is a persisted DEFAULT-OFF setting, so a fresh profile
(every runner, every smoke HOME) can never serve; (b) fed stdin an
empty file, but the UI does not pin the process — stdio EOF ends it
cleanly (rc=0) before any poll sees it serve; (c) gave a 10s window to
a server measured to bind ~6s after launch on a FAST host; (d) POSTed
an MCP initialize at /rpc, which speaks the UI's own narrow query
protocol, not MCP.
The rebuilt probe mirrors the drive-listing guard's proven invocation
(--ui=true --port=N equals-form, held-open stdin), waits 30s, and
verifies the API surface with the stateless GET /api/ui-config.
Validated red->green against the release-shaped artifact: the full
darwin ui artifact lane now passes 15a/15b legitimately.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The unified smoke-local.sh readiness-polls the fixture server with
curl; the Alpine image never had it, so every poll iteration failed
command-not-found and the portable smoke leg reported 'fixture server
did not serve' with a perfectly healthy server. Local-only image (CI
portable smokes run on ubuntu runners, curl preinstalled).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On Windows the fixture server's atomic port publish (os.replace) — or
a first-touch AV scan of the fresh file, with Defender ACTIVE on the
VM — briefly denies the read: is_file() is true, read_text() raises
PermissionError, and the poll loop crashed the whole contract with a
bare traceback (observed on the VM once the 0e/0h needle fixes let
the chain reach this self-test at all). Treat an unreadable-yet port
file as not-ready and keep polling inside the existing deadline.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
BSD mktemp substitutes only TRAILING X's, so the template
cbm-soak-leg-XXXXXX.log creates a near-literal file on macOS and the
second leg of the same job collides with the first leg's leftover
('File exists'). First runs mask it — which is why the quick leg
passed and query-leak failed locally, and why CI's macos soak jobs
(BSD mktemp, two legs per job) would fail the same way.
Drop the suffix; the path is only a log handle.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The manifest content (leg, slice, list hash) is fully determined at
slicing time, so write it up front: a red run's manifest is exactly as
load-bearing for the cross-shard union proof as a green one's.
With that, a missing manifest at upload time can only mean the job
died before the harness started (contract step / build failure) — that
failure is already the job's red, so the upload warns instead of
stacking a second error on top (the recurring 'No files were found
with the provided path: build/c/test-logs/shard-manifest.txt'
annotation). The shard-completeness job still gates the union.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Local soak legs write soak-results*/ into the worktree; a bulk add
swept one run's logs into the previous commit. Remove them from
tracking and ignore the output directories (CI uploads them as
artifacts; they are never repository content).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dry run 30225685743 failed at the ensure-defender step on every
Windows job: the hosted images policy-lock real-time protection OFF
(WinDefend starts, Set-MpPreference accepts, RTP stays off —
untamperable from the job).
Revert to the pre-gate runner behavior: no ensure-defender step in
runner jobs, the release Defender scan is on-demand and fail-soft
again. The local VM preflight KEEPS the fail-closed Defender-ON gate
(same canonical script) — AV-interaction coverage is a deliberate
local superset where the platform allows it.
The venue-parity contract now enforces the INVERSE for runners
(ensure-defender must not gate hosted jobs) and still requires the VM
preflight gate; VM README documents the verified posture delta.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The venue unification moved the VM soak sequence and its completion
guard into scripts/soak-legs.sh; the smoke-fixture contract still
pinned the old direct soak-test.sh call in vm-run-tests.sh, failing
Step 0h — and with it every test leg on every venue of dry run
30224804737 (the test wave's 20 reds share this one cause).
Masked until now: the contract chain runs sequentially and Step 0e
failed first on the pre-fix tree, so 0h was never reached; post-fix
verification used --suites mode, which skips the chain. All contract
steps 0a-0j now verified green standalone in bulk.
Invariant preserved, anchor moved: the VM wrapper must route --soak
through the canonical entry under its protected temp root, and the
canonical entry must completion-guard every leg on the soak summary
(grep for '=== soak-test: PASSED ===' lives in soak-legs.sh:101).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
LSan (Linux amd64 leg): 200 bytes — the runtime client (192) plus its
IPC connection (8) — leaked whenever an earlier stage of
daemon_runtime_disconnect_cancels_blocked_non_index_child... failed:
both close blocks were gated on the request thread having completed,
so a failed run skipped the close entirely.
Teardown now closes unconditionally: close_begin interrupts the
transport (forcing the blocked call to complete), the join becomes
safe, and close_finish frees the client on every path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dry run 30206811293: macos-15-intel release smoke failed B3 with a
visibly correct 'semantic: 50' table — BSD grep returns no-match for
the ENTIRE output when any line carries a NUL, control byte, or
invalid UTF-8 (verified: printf poisoned input -> grep -qE fails), so
one bad byte in one row silently unmatches every anchored check.
Close the class at the emission choke point: control bytes and
invalid UTF-8 force the quoted form; quoting escapes controls as
\u00XX and replaces invalid sequences with U+FFFD (RFC 3629
validation, NUL-safe). Tool output is valid UTF-8 by construction.
Unit test proven RED-on-revert (raw bytes emitted) and green with the
fix. The smoke's B3 failure path now also dumps od -c bytes so a
future red names the exact byte.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dry run 30206811293: windows-11-arm (both variants) and windows-2025
smoke failed in the update/uninstall E2E phases with 'private staged
copy failed secure open (error 32)' — ERROR_SHARING_VIOLATION. On the
newer hosted images a first-touch scanner transiently holds the
just-staged executable without FILE_SHARE_DELETE, so the launcher's
GENERIC_READ|DELETE retirement open collides on cold images.
Absorb the scan window with the same bounded backoff the staged
RENAME path already uses (<=~2.5s, exponential); every retry re-runs
the full secure-open validation, so no TOCTOU surface changes.
No deterministic local repro exists (the VM passes this phase with
Defender active; the collision is image-specific) — the next dry run
on the same three jobs is the arbiter.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
git-apply of the staged unification recorded the new canonical entries
as 100644; workflows execute them directly (run: scripts/ci/...), so a
fresh CI checkout would fail at ci-ok/shard-union/soak/package steps
with permission denied — the same class the parity worktree's own
preflight just tripped over locally.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Closes the closable local-vs-remote venue gaps so that, outside arch
legs / YAML glue / release plumbing, every local red is a remote red
and vice versa on shared legs:
- scripts/package-release.sh: THE canonical archive step (names +
five-file Windows bundle layout defined once); _build.yml's eight
inline archive blocks become calls to it.
- scripts/ci/smoke-artifact.sh: the artifact-flow smoke lane — build,
package, extract, then the canonical wrapper in artifact mode; wired
as run.sh smoke-artifact (compose service), win.sh smoke-artifact,
and directly runnable on macOS. Archive-layout bugs now surface
locally instead of in a release dry run.
- glibc-floor leg (Dockerfile.glibc22 + compose + run.sh): portable
binary smokes on ubuntu-22.04/glibc 2.35; the dynamic binary must
refuse there (2.38+ floor by design).
- Defender-ON parity (user directive): scripts/ci/ensure-defender.ps1
enables + VERIFIES real-time protection, fail-closed; runs in every
Windows CI job (_test x2, _soak x3, _smoke, pr.yml) AND in the VM
preflight; _smoke.yml's scan engine-failure soft-skip becomes a red
gate. Expected cost: slower Windows jobs (AV scanning during
install/build/test I/O); the next dry run proves the runner side.
- Contracts: launcher-bundle five-file check retargeted onto
package-release.sh + per-archive canonical-call association;
venue-parity contract requires the new lanes, counts one
ensure-defender step per Windows job, adds --help probes for the new
entries. Extended contract fails on the pre-change tree (verified:
20 violations on HEAD).
- VM README: ephemerality/Defender posture documented — utmctl has no
snapshot verb, so per-run revert stays a manual qcow2 option; the
sweep preflight remains the standing mechanism.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The venue unification moved release-smoke staging/execution out of
_smoke.yml into the canonical wrappers (vm-smoke.sh / smoke-local.sh),
so Step 0e's needle checks against inline YAML failed on every venue.
Re-anchor each pinned invariant where the behavior now lives:
- canonical-exe execution + profile-rooted fixtures + update fixture
identification: asserted in vm-smoke.sh / smoke-local.sh, plus the
workflow must call the wrapper on the extracted artifact
- version/install-audit preflight needles updated to the artifact-dir
text shape (still workflow provisioning)
- RUNNER_TEMP: absolute ban replaced by a provision/scan allowlist
walker + an explicit no-exe-under-RUNNER_TEMP backstop
No pinned security property is dropped; the retargeted contract fails
on the pre-unification architecture (verified against HEAD).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The staged venue-unification bundle: _smoke.yml onto the shared
wrappers with an extracted-artifact input (CBM_SMOKE_ARTIFACT_DIR),
soak-legs.sh as the one canonical soak entry (quick + query-leak legs
across all venues), protected per-user TEMP roots shared across
venues, clean-disk preflights before every VM/Docker run, win.sh and
run.sh routed through the canonical scripts, soak.yml retired, and
the venue-parity contract (whitelist walker: workflows may provision
or call canonical entries, nothing else).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This reverts commit f843db8, which removed the #581 fix on a false
attribution. The arm64 shards were crashing because of an unaligned SQLite
page-cache slab (fixed in #1274, arm64 shard 2/2 green since); removing this
callback never changed those crashes, which is what exonerated it. The Windows
soak then failed for the obvious reason: the leak was back.
Static MinGW links have no DllMain and register no TLS callback, so
mi_thread_done never runs and every thread leaks its heap -- 607 heaps after 300
requests holding 170 MiB against a ~300 KiB live set. With the callback the
10-minute Windows soak measured 1.1x against the original 53x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The slab was a static char array handed to SQLITE_CONFIG_PAGECACHE. A static
char array carries no alignment guarantee and SQLite requires 8-byte-aligned
page-cache memory: x86_64 tolerates the unaligned access, arm64 faults. Every
suite that opens a store died instantly on windows-11-arm with
STATUS_ILLEGAL_INSTRUCTION (pass=0, secs=0) while pure-logic suites in the same
wave passed, and both arm64 shards were green on the commit before the merge.
It was also never the fix it was added for: measurement showed the slab
installed correctly and #581's growth unchanged, because the cause was thread
heaps never being released. Removing it rather than aligning it -- an unverified
optimisation has no claim on a release gate.
Note PR CI does not build the windows-11-arm leg, so this class of defect can
only surface in the dry run.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The refusal notes changed the error text the Windows launcher guards match
against, and test-windows-guards went red on this PR after they landed. They were
investigation instrumentation and they succeeded -- they turned 'staging
transaction open failed (status -3, os 0)' into ERROR_DISK_FULL and named the
exact site -- but a diagnostic that shifts a matched error string does not belong
in a release-blocking change.
Worth redoing deliberately on its own, with the guards' expectations updated in
the same commit.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The hosted windows-11-arm shards crash at process load (STATUS_ILLEGAL_INSTRUCTION,
pass=0, secs=0) while main passes them, and removing the TLS detach callback did
not help -- so the remaining candidate is this layer, which touches every
allocation on Windows through the interposer. It declares a _Thread_local guard
read on every malloc, and thread-local storage in a static MinGW aarch64 link is
exactly the fragile case; nothing in it needs to ship.
The profiler did its job: it produced the attribution that identified #581's
mechanism, and that analysis is recorded. Sources stay in the tree, unbuilt, so
a future investigation can re-enable them deliberately.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reverts the #581 fix mechanism, not the diagnosis.
main's dry run passes both windows-11-arm shards and test-windows-guards; this
branch fails all three. The only change here that affects process LOAD is the
.CRT$XLB detach callback, and the arm64 failures are load-time crashes
(STATUS_ILLEGAL_INSTRUCTION, pass=0, secs=0, five suites in the same second) --
while the guards exercise many launcher and payload spawns. The same binary
loads fine on a UTM arm64 VM, so the placement is tolerated by some loaders and
not others, which is not something to ship on a supported platform.
#581's root cause stands and is documented: a static MinGW link has no DllMain
and registers no TLS callback, so mi_thread_done never runs and every thread
leaks its heap (607 heaps after 300 requests, 170 MiB against ~300 KiB live).
The delivery mechanism needs to be one the loader accepts everywhere -- an FLS
callback registered at runtime is the obvious candidate, with no linker section
placement involved.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The copy loop's failure was reported as 'os 0' because the cleanup and close
calls after the loop clobber the thread's last error. Captures it at the failure
point, and separates a failed read from a failed write -- copying a payload that
another process still holds is a different problem from a full disk.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The copy loop's five distinct failures -- short read/write, an empty source, a
failed fsync, a failed close, and a failed directory sync -- shared one branch
and one unnamed IO status. Exhaustive labelling identified this as the path the
Windows launcher guard's immediate reinstall takes; this names which of the five
actually fires.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fourteen paths returned CBM_ACTIVATION_TRANSACTION_IO without setting a refusal
note, so the caller could only ever print 'status -3, os 0' -- an I/O error with
no cause. Identifying which one fires cost a ten-minute Windows guard run per
hypothesis, and two targeted guesses missed.
Labels every site so a single run names the path. Temporary labels; once the
guard's path is known they become descriptive.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel