Adds mem_malloc_owned to every diagnostics snapshot: a real malloc/free pair
probed with mi_is_in_heap_region rather than an inference from build flags,
because the Windows static-CRT override is defined at compile time yet can
fail to take effect at link time -- which is exactly what happens here.
This single bit distinguishes 'nothing is live' from 'the allocator does not
own this build', which are opposite conclusions drawn from the same zero
live_bytes. On Windows it reads false, which means every allocator tuning in
cbm_mem_init -- immediate purge, purge-decommits, page reclaim, arena policy
-- is inert for ordinary allocations there, while POSIX gets all of it. That
is the parity gap behind #581, and reading this bit first would have replaced
a long elimination hunt with one measurement.
Also reverts the interim _heapmin trim explored while localising #581. It cut
the ten-minute Windows soak from 1168 MB to 232 MB but never reached the
13-to-15 MB flatness POSIX achieves, and it cost roughly four times the query
latency because it walks the whole heap on the request path. Trimming a heap
the allocator should have owned treats the symptom; the override itself is
the fix.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The allocator walk could say how much memory was live and in what size class,
but not who was holding it -- and when growth does not route through the
allocator at all, as on Windows, it could say nothing useful. Phase marks
close that gap using the one metric that does see such growth: OS committed
bytes. Each mark attributes the committed delta since the previous mark to the
previous label, so over many passes the label whose total climbs is where
memory stays.
The design is deliberately coarse and says so. Committed bytes are
process-wide, so a busy concurrent thread lands noise in whichever label is
open; signal comes from accumulation over many passes, never one delta. Marks
must bracket a path with no unlabelled gap, or growth hides in the gap -- so
the request path labels its tail 'idle' explicitly, which is also what makes
request-path retention distinguishable from background growth. Off unless
CBM_MEM_PHASES=1.
Pointed at #581 it localised the growth in two steps. At request granularity
dispatch_tool held +102 MB over 27 requests while scope begin/end and idle
were flat, which independently killed the theory that the growth was
time-based. Splitting the handler then put +88.4 MB of it in resolve_store
against +14.1 MB for the query body, with the matching release returning only
-35.9 MB. So each request resolves a store, commits ~3.4 MB, and gets ~1.4 MB
back: a ~2 MB net ratchet per request that never returns. That accounts for
the observed growth, for its scaling with corpus size, for why every tool
leaks identically, and for why only Windows shows it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Growth investigations were guessing, because the diagnostics snapshot only
carried process totals: RSS and committed bytes say memory grew, never
where. The map adds allocator-live bytes and a size-class histogram to each
snapshot, so a distinctive class identifies the allocation without
per-callsite instrumentation.
The design constraint is honesty about coverage. Walking the allocator
reaches the aggregate heap plus this thread's heap and its abandoned pages,
which is not the whole process, and in a build where malloc does not route
through the allocator at all it reaches nothing. A per-subsystem tally
would report zeros there and read as a clean bill of health. So every
snapshot carries three independent totals and an explicit residual --
os_committed, allocator-live, residual = the difference -- and the triple
localises growth instead of implying it:
live grows leak on a walked heap; buckets say which size
residual grows, flat another thread, retention, or non-allocator memory
both flat, RSS grows mappings or stacks, not the heap
That guard paid for itself immediately. Pointed at the open Windows growth
in #581 it reported live=0 with zero blocks and the residual carrying all
of it -- not 'the heap is innocent' but 'this walk saw nothing', which is
the correct reading for a process demonstrably holding tens of megabytes.
Together with the growth surviving a build with the allocator override
compiled out, that says the Windows allocations under investigation never
reach the allocator, so an allocator-based map cannot localise them and the
next instrument has to sit lower down.
Covered by a test that allocates a known volume in a known size class and
requires the map to attribute it there, so the map can never regress into
silently reporting zeros.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The macOS Intel leg failed the fixture contract on every run while every
other platform passed. Removing the readiness fsync did not change it; the
hardened diagnostics named the real cause on the first remote run:
waited=30.0s, exit_status=alive, port_file=absent, staged_temp_files=none,
empty startup log -- the process was healthy but had never reached
publish_port.
http.server.HTTPServer.server_bind() resolves socket.getfqdn(host). On a
host whose resolver does not answer for the bind address that call blocks
for the resolver timeout, so the constructor never returns and no port is
ever published. The fixture now binds through a subclass that keeps the
threading server but skips the FQDN resolution, which only feeds CGI-style
variables this fixture never serves.
Proven both directions locally by forcing socket.getfqdn to hang: the
subclass publishes its port immediately, the stock server never does. That
forcing hook is kept as a permanent contract guard, so the reverse-DNS
dependency cannot return without turning the gate red on every platform
rather than on one runner nobody can reproduce.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The native Windows soak could never start its server, so that gate had
never actually measured anything. The root cause was the harness, not the
daemon: an MSYS filesystem FIFO feeding a native Windows process delivers
empty stdin (reproduced 5/5), while a Bash anonymous coprocess carries the
full JSON-RPC initialize exchange to the same protected payload, and the
failing daemon log showed an orderly client disconnect rather than a
daemon-side eviction. The soak now uses a named coprocess on native
Windows only, duplicating its endpoints to stable fd3/fd4 and closing the
originals so closing fd3 still delivers EOF; POSIX keeps the FIFO path.
The coproc syntax sits inside eval because macOS system Bash 3.2 must
still parse this file even though only MSYS2 Bash 5 executes that branch.
With transport repaired the next exact failure surfaced: native Windows
Python cannot open an MSYS /c/... diagnostics path. Both consumers now
pipe the file through stdin, the pattern already established elsewhere in
the repo, and the soak recovery contract forbids reintroducing a direct
native-Python open.
Also on the release path:
- The smoke fixture server no longer fsyncs before atomically publishing
its port. This is ephemeral readiness signalling, not crash-durable
state, and the macOS Intel runner failed inside that durability sync.
When the contract does fail it now reports the observable state --
waited, exit status, port file, staged temp files, interpreter, startup
log -- because the previous verdict named nothing on the one runner we
cannot reproduce locally.
- The POSIX publication test hook no longer compiles into Windows builds,
fixing an exact -Werror unused-variable failure. The setter keeps a
parameter-consuming Windows stub because it is public API.
- The Wine leg assembles the real release layout (payload plus canonical
launcher) and version-checks both, running the launcher through cmd so
it has a Windows-visible parent. The unsupported Wine soak is removed:
Wine stays a fast compile/package/version check, and native Windows is
authoritative for daemon, locking, ACL and process-lifetime semantics.
- run.sh soak-windows routes to the native Windows VM, which validates the
payload, builds the protected per-user temp root, stamps ACLs, and
refuses success without a completion summary.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The parity harness drives the pipeline passes with a bare ctx and never
ran the production teardown, so the return-type table pass_calls builds
leaked — latent until the fixture gained typed Java methods, and visible
only under Linux LeakSanitizer. Both runners now free it exactly the way
pipeline.c does.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Resolves the two open Pinned-Dependencies code-scanning alerts (and one
sibling the scanner had not flagged yet):
- ubuntu:noble in Dockerfile and Dockerfile.lint is pinned to its
multi-arch manifest-list digest (verified against the registry and the
scanner's remediation digest, which match).
- mstorsjo/llvm-mingw drops the floating :latest and pins the current
digest the same way.
- The MSYS2 zlib sysroot package in Dockerfile.mingw is verified against
a pinned sha256 before extraction instead of piping the download
straight into tar.
Dockerfile.alpine was already pinned; both rebuilt images (test, lint)
verified building with the new digests.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The parallel per-file resolve hardcoded INHERITS for every base-class
relation while the sequential semantic pass made the Interface split, so
any corpus large enough for the parallel path had its explicit implements
demoted: elasticsearch carried 11,179 INHERITS edges pointing at Interface
nodes and 3 IMPLEMENTS. Both venues now share one decision keyed off the
target node's label — the graph truth the edge attaches to.
OVERRIDE edges existed only for Go's implicit interface satisfaction. A
serial full-graph tail run by both pipelines now matches class methods
against explicit bases by name and emits Method-to-Method OVERRIDE edges
for the explicit languages, with Go excluded to avoid double cover.
The venue parity fixture gains a Java interface/implements/extends trio —
IMPLEMENTS parity used to compare 0 == 0 and guarded nothing — plus an
exact-count test so the parity can never go vacuous again. At scale:
elasticsearch IMPLEMENTS 3 to 11,182 and OVERRIDE 0 to 49,695, TypeScript
0 to 4,496 and 594, dotnet/runtime 0 to 12,953 and 45,667, with INHERITS
plus IMPLEMENTS sums conserved exactly.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A directory with more immediate subdirectories than the fixed 512-frame
walk stack made the whole discovery walk hard-fail with files=0 and a
misleading 'check repo_path' hint — dotnet/runtime carries 855 sibling
dirs in one JIT regression folder and could not be indexed at all. The
stack now grows geometrically from the same initial capacity; the parent
frame is a popped copy, so growth cannot dangle it. Covered by a 600-dir
fanout reproduction.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The build system now stamps CBM_SANITIZED_BUILD for any non-empty SANITIZE,
so timing budgets key off build-system truth instead of compiler-specific
__SANITIZE_ADDRESS__ probes that miss trap-UBSan and TSan; the two bench
suites use it.
The httpd interrupt test proved 'the interrupt worked' with a wall-clock
bound, which is a lottery under sanitizers. A send-deadline test hook pins
the deadline out of reach so the join itself is the causality proof, and
the stop watchdog becomes a true ten-second hang detector.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The fuzz harness swallowed the target's exit status through '|| true', so
a SIGSEGV counted as a pass; the status now propagates (a planted crash
yields 139) and payloads derive from a logged, replayable seed, with a
missing python3 failing the gate instead of generating zero mutations.
The Windows guard runner classified unknown exit codes as skips; the
contract is now explicit (0 green, 1 red, 2 precondition-skip, anything
else a failure) and an all-skip run fails as verifying nothing.
The smoke suite gains crash-class detection on the phase-9b tolerance
paths and phase 11 kill handling (rc >= 128 or a missing jsonrpc banner
fails), a free-port pick plus readiness poll for the UI phase, and a
phase-15b failure that actually exits nonzero.
The soak reader resynchronizes on late JSON-RPC responses by draining to
the matching id, and the analysis fails when fewer than 60 percent of
snapshot attempts produced rows — a vacuous analysis previously passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows build-dir DACL stamp resets inherited ACEs before granting,
verifies itself out loud, and re-stamps at the serial-tail boundary — the
arm image ships explicit ACEs that survived a grant-only stamp and turned
the whole suite red. Stamp verification failures now fail the run instead
of warning. A suite that exits 0 without printing its summary line is
counted as a failure rather than silently passing.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two changes to the index coordinator, developed together and validated as
one state:
Determinism seams. A cancel racing a worker start was only observable on a
loaded runner; now the interleaving is forced by construction: a test hold
parks every job thread before its first pre-start cancel check, and a
background-initialize completion counter turns sleep-then-assert negatives
into positive-signal waits. The cancel-before-worker-start case is pinned
by a dedicated test; three auto-index admission tests now wait for the
admission tail instead of sleeping through it.
Queued admission. An explicit index_repository request that hits the
physical job limit used to surface a raw busy error. The request thread
already blocks for the whole index, so it now queues: a cancellable,
shutdown-aware wait-and-resubscribe loop replaces the error path, with a
wait counter as the parked-state observable. The programmatic coordinator
API keeps its non-blocking busy answer for callers with their own retry
policy (watcher, auto-index). The job-limit test asserts the new contract:
parked without a second worker start, admitted and completed once the
occupying job finishes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The private-file prepare treated slow CONTENTION like permanent
OBSTRUCTION: one 250 ms deadline for both. With many concurrent
appenders on a loaded, sanitized runner the last writer can
legitimately wait longer than that behind its peers' brief exclusive
windows, and the concurrent-append guard correctly failed a dropped
conflict event (dry run: one appender of the pack lost). Transient
share/lock violations now retry against a 2000 ms contention budget
while every other error class keeps the 250 ms fast-fail — a rejected
client is still waiting behind this on the hello path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On one dry run the two arm shards split: the shard whose serial tail
held the install-flow suites failed the source-directory policy minutes
after its pre-wave stamp, while its sibling — same runner image, same
stamp — passed with no install-flow suites to notice. Wave suites spawn
Cygwin-family tooling that can rewrite the build directory's DACL
behind the first stamp, so the stamp is now a function invoked both
pre-wave and at the tail boundary, where the deadline-sensitive suites
that depend on it actually run. Both invocations self-verify out loud.
Verified on the VM from a deliberately reset (inherited) build
directory through the real --par path: both stamps report clean, mini
shard green.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The dry run's windows-11-arm shard failed its install-flow tests with
the source-directory refusal the stamp exists to prevent — while the
identical commands, run from the identically dirty state on the ARM64
VM, stamp clean and pass. The difference is invisible because the stamp
was fully silenced; a silent load-bearing step cost a full CI round to
even learn whether it had run.
The stamp now reports each icacls failure with the user and directory,
re-inspects the DACL afterwards, and prints one line in either
direction: stamped clean, or the surviving cross-account grants.
Verified on the VM from a deliberately reset (inherited, Authenticated-
Users-writable) build directory through the real --par path: stamp
reports clean, suites green.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The suite's observation waits shared a 2000 ms budget that is a hang
detector, not a latency assertion — every waiter returns the moment its
condition holds, so green runs never pay it. Under ThreadSanitizer's
5-20x slowdown on a loaded 3-core macOS CI runner the cancel-delivery
wait lost the tail of that distribution once in seven otherwise-green
TSan rounds (the cancel itself had succeeded; only the fake worker's
counter observation timed out). The budget is now 10000 ms.
Verified: nine consecutive daemon_application runs under native macOS
TSan with the widened budget, all green, zero ThreadSanitizer
warnings.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The windows-11-arm leg's first execution (broad matrix, release dry
run) died before any test ran: the trap-UBSan SANITIZE value passed
unquoted through the workflow ternary, so -fsanitize-trap=undefined
arrived as its own word and make consumed the leading -f as its
makefile flag ('No rule to make target sanitize-trap=undefined').
- The workflow quotes the ternary value so test.sh receives one
argument.
- test.sh forwards make arguments through an ARRAY instead of a
re-split string, so a VAR=VAL whose value contains spaces survives
every boundary (empty-array expansion kept bash-3.2-safe for the
macOS runners under set -u).
Verified: argument-parse proof (argc=1 with the four-flag value) and a
full macOS suite run through the changed plumbing, 6776 passed,
0 failed — identical totals.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Phase 9b's fixture installs spawn ephemeral daemons whose mapped
generation backings pin files in the fixture HOME; the cleanup rm then
races the daemon's asynchronous drain and loses nondeterministically on
Windows (Device or resource busy on a generations backing — the same
class the final-phase daemon retirement already fixed for the smoke
cache, two rounds green then one red on identical code). The retirement
is now a shared helper with a cheap not-running probe, called before
every fixture cleanup that follows an install and by the final phase; a
daemon that will not retire stays a hard failure.
The 9b fixtures also route LOCALAPPDATA under the fixture HOME: their
installs staged launcher pairs into the real runner profile, outside
the fixture's isolation and the harness's cleanup reach.
Verified: full VM smoke green end to end (all 9b subphases, phase 17
through the shared helper).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The workflow pre-created build\c with a protected DACL so build
products would inherit it, but the diagnostic note from the failing CI
shard named the real actor: MSYS2's Cygwin layer writes POSIX-emulating
DACLs — including a CREATOR OWNER (S-1-3-0) mutation grant — onto
directories its tools touch, overwriting the pre-created shape during
the build. The activation transaction's source-directory policy
correctly refuses that grant, so the install-flow tests failed on CI
while the VM (whose harness stamps AFTER building) stayed green.
The stamp now runs inside run-tests-parallel.sh on Windows, after any
builder has had its say and immediately before the suites — one code
path for CI and the VM. Two idempotent steps, as before: protect the
directory, then /reset the children onto the clean inherited set.
Verified: full VM parallel run green (6669 passed, 0 failed) through
the new path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Local runs left most of the machine idle: the serial tail ran sixteen
suites one at a time on otherwise-idle cores, the three platform legs
were launched by hand (usually sequentially), and every container-leg
object write crossed the virtiofs bind mount.
- run-tests-parallel.sh tail scheduling in two phases: the FLEX suites
(timing-shaped but free of the shared per-account daemon runtime
namespace) run CBM_TAIL_JOBS-wide (default 2), then the EXCL group —
daemon-family plus the suites that drive daemon one-shots or
supervisor rendezvous — runs strictly sequentially on a machine
exactly as quiet as the old fully-serial tail gave it. The wave was
already fed longest-first by the shard dealing order, so the drain-out
no longer ends on a heavy straggler.
- ladder.sh: one maintained entry point for the full local push gate
with the legs overlapped — lint, the Linux container suite, and the
Windows VM suite in the background, the macOS suite in the
foreground, one verdict per leg, logs kept per leg. A missing
prerequisite fails its leg loudly instead of silently skipping.
- win.sh test-par now runs through vm-run-tests.sh (--par mode): the
full parallel harness under the CI-shaped protected temp root with
complete output. It previously ran under the MSYS-shared /tmp and
piped through `tail -25` — the same truncated-blindness class that
hid 40 Windows failures from the `test` command.
- docker-compose: build artifacts and the incremental fixture cache
move to named volumes on the container VM's native filesystem. Object
writes over the virtiofs bind mount are the container legs' largest
avoidable I/O cost, and the fixture cache now survives across
container runs.
- test_mem(win): the first valid full-parallel VM run proved working-set
trimming beats the re-touch mitigation (19 MB resident of a 256 MB
double-touch at 18 parallel suites). The RSS probe now VirtualLocks a
64 MB span — locked pages are exempt from trimming, making the
measurement pressure-immune — with bounded touch-and-sample retries
when the lock is unavailable. Red-to-green under the same 18-job load.
- cli: the portable install's staging error now appends the activation
refusal note (predicate, SID, object) like the managed path already
does — a bare "activation transaction I/O failed" on a CI-only
failure is undiagnosable without it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Flip shard_suites on for all three callers of the reusable test
workflow. This commit is deliberately the only topology change:
reverting it alone restores the previous single-job-per-leg CI shape
while keeping the (inert without it) sharding machinery and the
cache-warm pipeline.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
PR iteration was bounded by single-job suite runs (~40-50 min critical
path) and by every first-of-ref build starting cold. Same suites, same
sanitizers, same gates — the schedule, the cache reuse, and a new
runtime coverage proof change.
- run-tests-parallel.sh accepts CBM_TEST_SHARD="i/N" and runs a
deterministic slice of --list-suites; unset selects everything, so the
path is inert outside CI. Known-heavy suites are dealt to shards in
weight order first (naive modulo stacked the two slowest, store_arch
and daemon_runtime, onto one job), the parallel wave and the serial
tail are sliced separately so every shard keeps its own quiet tail,
and the per-shard union guard proves each job ran exactly its slice.
- shard-completeness job: per-shard guards cannot see a mis-plumbed
CBM_TEST_SHARD (two jobs running the same slice passes every local
check while a slice runs nowhere). Every leg now uploads a manifest
(leg, i/N, sha of the full suite list, its slice) and one aggregation
job re-proves per leg that the shards agreed on the list, indices are
exactly 1..N, and the union of slices IS the list. Runs on unsharded
topologies too, where each leg's single manifest must cover the list.
- workflow topology (shard_suites input, off by default = byte-identical
to the pre-shard matrix): ubuntu legs run 3 shards, Windows 2 (every
extra Windows shard re-pays ~5 min of MSYS2 setup), macOS stays at 1
(not the critical path; mac runner concurrency ceilings are the
tightest). The Windows test job also pre-creates build\c with a
protected DACL so build products inherit it — workspace drive roots
grant Authenticated Users Modify by inheritance, which the activation
transaction's source-directory policy correctly refuses.
- ccache: a final ref-less restore key lets a fresh PR start from the
newest cache GitHub's scoping permits, and a nightly build-only
cache-warm workflow keeps main-scoped caches at most a day stale —
without it the fallback had no warm source, since nothing built in
main's scope. The strictly-per-ref policy this replaces cost ~8-12
minutes on every first-of-ref build for no safety gain:
CCACHE_COMPILERCHECK=content makes a stale or foreign cache able to
miss but never to return wrong output. Sharded legs restore-all/
save-one (every shard builds the identical objects, so shard 1's
cache carries the full set) — save volume stays flat and warm caches
stop being evicted by per-shard duplicates.
- suite timeouts: daemon_runtime joins the slow tier. It measures ~610s
solo on arm64 under ASan (10-round loop, no hang), so the 900s default
under a loaded 4-job CI runner was a slowness kill masquerading as a
hang detector.
Validated locally: 3-shard union over the real suite list is complete
with no duplicates, one top-heavy suite per shard, and the summed
3-shard totals reproduce the unsharded run exactly (6776 passed,
0 failed, 4 skipped).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The incremental suite cloned fastapi from the network on every run and
indexed a DIFFERENT corpus per environment: CI used a sparse checkout
without docs/ and tests/ (the assertion thresholds are sized for it)
while local runs indexed the full tree — twice the files for the same
assertions, a per-run network dependency, and the suite's dominant cost
on the sanitized container leg.
The fixture is now cloned from the network at most once per machine
into a persistent cache (committed with an atomic rename so a torn
download can never pose as a valid cache; CBM_TEST_FIXTURE_CACHE
overrides the location), every run local-clones from it offline, and
docs/ plus tests/ are trimmed everywhere — the portable equivalent of
CI's historical sparse profile, so all environments now index the same
corpus the assertions were written against.
Same 163 tests, same assertions: 74s -> 31s native, with the container
leg's cut scaling with the halved indexing volume, and the "network
offline" flake source reduced to first-run-per-machine.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The 40 hidden Windows failures fixed in the previous commit reached CI
unseen because the local Windows leg never validly ran the C suite:
MSYS2's shared /tmp fails the secure-ancestry validation (refusals, not
signal) and the ssh driver piped everything through `tail -40`. This
closes those holes and hardens the gates that let an install failure
scroll past.
- vm-run-tests.sh (new, used by win.sh test / ubsan-test /
trap-ubsan-test): gives the suites CI's exact protected per-user temp
root (owner-stamped, protected current-SID DACL, mirroring the
workflow's pwsh step), shapes the runner's build directory like a real
user checkout (drive-root trees inherit an Authenticated-Users Modify
ACE that profile-rooted checkouts do not have, which the activation
source-directory policy correctly refuses; DACL re-rooted two-step —
directory protect then child /reset, since inheritance flags are
directory-only and a /T re-root leaves files with empty deny-all
DACLs), streams the FULL output, and refuses to report success unless
the runner printed its completion summary.
- smoke: Phase 8 no longer swallows the install exit code. On Windows it
now requires the managed install to SUCCEED and to publish an exact
two-link canonical launcher — a staging refusal used to scroll past as
tolerated noise while the downstream config assertions kept passing.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first full Windows x64 CI run of this branch (earlier contract-step
walls had always aborted the leg) surfaced 40 failures: the managed
launcher's fail-closed install/update/uninstall gates and the suite's
POSIX-modeled activation tests were both new in this branch and had
never been reconciled, and they hid three real production bugs.
Production fixes:
- ui: config publishes with POSIX rename semantics. MoveFileExW's legacy
replace cannot supersede a destination a reader holds open (the name
lingers until the last handle closes, even with FILE_SHARE_DELETE), so
a config save racing an open reader failed. The temp file is opened
with DELETE access and renamed in-hand (bare-path target, NUL-padded
rename record) before close, falling back to MoveFileExW on any
rename-by-handle failure so non-NTFS config homes keep the previous
behavior.
- subprocess(win): the child's redirected log handle now shares write
access. POSIX places no lock on the redirected log, so a second
producer may append while the child runs; the missing FILE_SHARE_WRITE
mandatory-locked every other writer out, which the index-supervisor
callback-isolation test legitimately exercises.
- cli(win): the portable uninstall path composes the managed binary name
with its .exe suffix; it could never find (or remove) the installed
binary before.
- activation: security-predicate refusals are now diagnosable. The
transaction's predicates refuse without a usable OS last-error, which
collapsed five distinct refusal classes into one blind "status -3,
os 0" at every caller. Each refusing predicate records what refused —
predicate, untrusted SID, and object path — first-wins so the deepest
cause survives, and the staging error line appends it. This is what
identified every Windows refusal in this batch, including the real
install-failure class on drive-root-rooted payload directories
(Authenticated Users holds inherited Modify there; profile-rooted
directories do not).
Windows test dispatch (release binaries unchanged):
- Test builds (CBM_CLI_ENABLE_TEST_API, the existing test-object macro)
compile the portable activation flow on Windows, reachable solely
through the installed activation test seam and announced loudly on
stderr; release binaries compile exclusively the managed transaction,
so the dispatch polarity fails closed. The suite's install/update/
uninstall calls route through a seam-if-absent wrapper, keeping the
shared activation and agent-config semantics unit-covered on Windows;
POSIX arms keep exercising the real activation machinery.
- Two new release-gate tests assert the fail-closed contract with the
seam OFF: a portable payload may not self-update/self-uninstall, and
install refuses to adopt or replace a foreign file at the canonical
launcher path even under --force.
- Test-bug fixes the first valid Windows runs exposed: the version-
cohort crash test reused one conflict record across the probe and the
turnover acquire (whose entry memset erased the probe's detail before
the assertions read it); the activation staged-path assertion now
checks containment rather than a separator (the \\?\ namespace
performs no forward-slash translation); the ui atomic test captures,
cleans up, then asserts so a failure cannot poison later tests; the
PATH-visibility guard is POSIX-only (Windows configures PATH through
the user registry, not a shell rc).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The account daemon legitimately outlives its last client for a moment
while it drains, holding its log and config DB open. POSIX rm does not
care, but on Windows an open file blocks deletion, so the caller's
cleanup of the smoke cache raced the daemon's asynchronous shutdown and
failed with "Device or resource busy" after every phase had passed —
first visible on windows-latest once the agent-config phases stopped
failing earlier, and reproduced identically on the local Windows VM.
The smoke now ends with an explicit phase: stop the daemon through its
own lifecycle command, wait (bounded) until status reports not-running,
and give Windows one beat for the final handle close. A daemon that
will not retire is now a smoke FAILURE in its own right instead of a
cleanup accident. Verified red-to-green on the VM and green on macOS.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Close the cross-platform sanitizer gaps that were leaving real
concurrency and undefined-behavior bugs uncaught, and fix a data race
the first widened run surfaced.
- tsan: the data-race gate ran three suites (mem, slab_alloc, parallel)
over no real threaded production code. It now covers every threaded
surface that runs clean and stable under TSan: the parallel-extraction
worker pool (parallel, worker_pool, pipeline), the filesystem watcher,
the embedded HTTP server (httpd), diagnostics sampling, the MCP server
and mutation guard, subprocess supervision, and the runnable
daemon-coordination paths (daemon, daemon_application). daemon_runtime
(deadlocks under TSan+fork), daemon_ipc and daemon_frontend
(test-harness synchronization, not production) are excluded with the
reasons recorded in the Makefile.
- tsan: fixed a genuine data race the widened gate immediately found —
cbm_lsp_max_walk_depth's lazy cache was read and written by parallel
LSP-extraction workers without synchronization. A data race is
undefined behavior even when every worker computes the same value, so
the cache slot is now a relaxed atomic: a plain load on the hot path,
and a first-touch double-compute simply stores the same value.
- tsan(ci + local): the test-tsan job now runs on Linux amd64, Linux
arm64, AND native ARM64 macOS (the threading code is shared, so a race
is usually caught on all three, but scheduler differences let each
surface one the others miss). The local ladder gained `run.sh tsan`
and `tsan-amd64` plus the matching compose services. TSan's shadow
memory aborts under modern high-entropy ASLR, so the containers run
under `setarch -R` with an unconfined seccomp profile (the personality
syscall is otherwise blocked) and the CI Linux legs lower
vm.mmap_rnd_bits first; amd64 TSan cannot run under x86_64-on-ARM
translation and is a real-hardware/CI gate only (documented in
run.sh).
- ubsan(win/arm64): native ARM64 Windows had no sanitizer at all —
AddressSanitizer ships no aarch64-w64-windows-gnu runtime. UBSan in
trap mode (-fsanitize-trap=undefined) needs no runtime library, so it
instruments natively and turns undefined behavior into an
illegal-instruction trap; -fstack-protector-strong adds stack-smash
coverage the heap tools miss. The GitHub windows-11-arm leg switches
from unsanitized to this, and vm/win.sh gains trap-ubsan-build /
trap-ubsan-test for local iteration (reproduce under the emulated
x86_64 UBSan to see which check fired). The whole codebase builds and
runs clean under it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fixes surfaced while bringing the shared-coordination-daemon branch
green across macOS, the Linux containers, and the real Windows-ARM64 VM,
plus a new stability guard that caught one of them.
- daemon(win): the CLI teardown transition latched a failed release as
permanent, but a Windows participant-state release must briefly
try-hold the shared startup/legacy gates and legitimately collides
with a concurrent one-shot's teardown; the release is retriable by
contract (it always retains the transition), so parallel one-shot
commands no longer report "CLI coordination cleanup failed" despite
succeeding. Found by the new stability guard's churn section.
- cli/main(win): install/update/uninstall and the Augment hook-script
removal derive the managed launcher's identity from its plain
drive-form path (cli_windows_plain_utf8), not the \\?\ extended-length
form, so agent-config ownership matches on Windows and uninstall no
longer leaves the MCP entry or owned hook scripts behind.
- security: the `daemon start --open` browser launch is now shell-free
(ShellExecuteW on Windows, cbm_exec_no_shell elsewhere) instead of
system(); the CORS origin check spells out its two literal loopback
URLs so the static URL audit sees a complete value. Both clear the
Layer-1 allow-list audit.
- coordination: version-cohort lock retries now sleep a per-process
jittered interval — fixed-period retries can phase-lock two
participants so one starves. The activation-quiesce test's
observation window now covers worst-case candidate staging (three
tamper-defense hashes over a ~1 GB sanitizer binary exceed the old
30 s budget on container I/O).
- smoke/soak: the agent-config smoke matches Windows config paths
through their escaped-backslash quoted form; the soak parser skips the
update-available banner before the JSON summary and gained a one-shot
CLI admission-churn phase inside the RSS/FD leak window. The build-dir
safety fixture creates a real symlink/junction (MSYS2 ln -s otherwise
deep-copies) so its traversal-refusal contract is genuinely exercised,
and the symlinked-agent-roots test demotes its planted links to an
unprivileged owner when run as root so the refusal is really tested.
- tests(win): a new daemon-stability guard exercises the parameter
surface, hook fail-open with its rate-limited notice, start-twice and
occupied-port handling, busy-stop refusal against a live MCP session,
kill -9 crash recovery, and sequential/parallel one-shot churn.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.
Daemon lifecycle:
- daemon start/stop/status subcommands. `daemon start` launches a
PERMANENT daemon (spawn shape is byte-exact argv; survives idle
periods and session ends) and reports an already-active daemon
instead of failing. Permanence is honored at every stop latch:
last-committed-client disconnect, host initial-client window,
coordinator release, and application final-session close — a
permanent daemon also keeps admitting new sessions after its last
one closes.
- daemon stop refuses while sessions are active and lists the blocking
peers (pid/role) that must finish first; an idle daemon drains
through the activation-shutdown machinery with the ACK ordered after
connection interrupts. A second stop is idempotent. The wire ops are
no-cohort first-frame requests with peer fingerprint authentication,
so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
keep their local supervised path). A cold CLI run that had to spawn a
temporary daemon prints a hint that `daemon start` removes the
per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
emits a visible, rate-limited notice (Claude-dialect systemMessage
plus stderr for other dialects) and always exits 0 — augmentation is
never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
older-build permanent daemon (strict semantic-version triples only;
dev builds never auto-drain) and the build-conflict message names
`cbm daemon stop` as the manual escape hatch.
Windows IPC/runtime (real-VM verified):
- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
path used to destroy its listening pipe instance on every 20 ms poll
timeout; a client attaching in the teardown window was severed or left
on an orphaned pipe object whose HELLO no server handle could ever
read, absorbing the connect until the client's own timeout expired.
The pending connect now survives poll timeouts and nothing is
destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
server handle can discard a just-sent response before the peer reads
it (POSIX stream sockets never lose buffered data on close). A bounded
cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
immediate on interrupted connections) now precedes close in
runtime_worker_finish and runtime_reject_inline, so hello-conflict,
capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
transport half-close, so close_begin now announces departure with an
explicit frame (ordered after APPLICATION_CANCEL, before the local
interrupt); the server releases the client's admission on receipt
instead of waiting for the handle to close. Admission-drop timing is
now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
request publication under the send lock, best-effort sends the active
token's APPLICATION_CANCEL frame, then interrupts local I/O; the
server cancels MCP/subprocess work promptly. Contract tests accept
both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
the requester's license to act on "snapshotted and draining", so every
connection interrupt is now initiated before the ACK is sent; a
session could previously get one more request serviced after the
requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
timer granularity) burned ~1.6 s against permanently obstructed paths,
stalling hello rejections past the client's timeout. The retry budget
is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
nonblocking send() on Windows is absorbed wholesale into AFD kernel
buffering regardless of SO_SNDBUF, so send deadlines and interrupts
could never engage against a slow peer (and the full payload was
pinned in nonpaged pool). Slicing restores a deterministic
backpressure point; a test hook pins SO_SNDBUF for the deadline and
interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
deadline, explicit connection states, refusal to free a server while
a listener-owned connection is active.
Windows long-path support:
- Central path-aware wide conversion (canonicalize via GetFullPathNameW
and prepend the extended-length prefix for absolute paths >=240) at
the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
store opens, and the daemon build-fingerprint/log paths. Deep managed
installs (a 64-hex generation directory routinely exceeds MAX_PATH)
now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
ancestry validators now operate in the extended-length namespace;
the launcher path is canonicalized (and prefixed when deep) once at
entry so every downstream exact-string comparison stays
form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
GetFileAttributesW) so non-ASCII install paths survive argv[0]
resolution.
Windows launcher install/uninstall transaction:
- FileRenameInfoEx names are NUL-terminated in an over-allocated
buffer. FileNameLength governs per the contract, but filter drivers
read FileName as NUL-terminated and appended adjacent heap bytes to
created names — a flaky, garbage-suffixed rename target. Both the CLI
and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
.cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
chars so the bare rename target stays under the FileRenameInfoEx
NT-conversion ceiling at guard depths; 64 bits still uniquely
identify the generation.
- When the running launcher's mapped generation backings pin .cbm
against rename, the backings are relocated to activation-<pid>-N
.retired tombstones beside the install (a mapped image may be renamed,
never deleted; the launcher's liveness-guarded sweep reclaims stale
tombstones). Every relocation is recorded, and a FAILED uninstall
reverses the moves after restoring .cbm — via MoveFileExW with
extended-length paths on both arguments, since the deep generation
target is beyond the handle-based rename's bare-path reach — so a
restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
out so the tree is shallow enough for the detached cleanup's rd, and
the cleanup's working directory strips the extended-length prefix
(CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
(CopyFileW destinations, CREATE_NEW tombstones, probe directories)
are explicitly owner-stamped so the exact-owner validators hold on
runner images; guard fixtures stamp hand-built trees the same way.
Diagnostics, tests and infra:
- diagnostics: discovery is now an always-delivered JSON control record
(new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
paths containing spaces; placement honors $TMPDIR with /tmp fallback
via a diagnostics-local helper; the soak parser reads the JSON record;
documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
log levels, TMPDIR-with-spaces, and native Windows output-contract
assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
parents with production-shaped ancestry (LocalAppData on Windows, via
th_secure_runtime_parent_new) — the runtime ancestry validation
correctly refuses temp roots whose ancestors grant mutation rights to
Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
drive the documented startup-owner publication flow before reading
generation-bound endpoint addresses. This turns the 26 Windows
failures previously visible in CI's full-test job green without
weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
contract including failed-uninstall restore and immediate reinstall
after uninstall; new daemon lifecycle and reworked hook-augment
guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
(it was set by CI but never read, so perf suites ran everywhere);
four throughput/bench suites are classified as perf, the heavy
store_arch suite moved to the slow-timeout tier, and two
wall-clock-sensitive assertions were rewritten as invariant checks
with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
contract, soak daemon-recovery contract, path-safety helper, the
Windows VM worktree-sync contract wired into scripts/test.sh, and
vm/win.sh guards building its clean embedded-UI product in an
isolated BUILD_DIR so it cannot clobber the incremental test build.
provision-windows.sh now installs Node.js for the guards UI build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows-VM bootstrap embedded a personal SSH public key and the
drivers connected with StrictHostKeyChecking=no. Key material and host
identity are now strictly operator-supplied and live outside the
repository (~/.claude/cbm-vm/, never committed):
- windows-bootstrap.ps1 takes the operator's public key as a mandatory
-SshPublicKeyPath argument (validated single Ed25519 line) and prints
the VM host-key fingerprint to pin. Generating your own key is
documented in the script and README:
ssh-keygen -t ed25519 -N "" -C <label> -f ~/.claude/cbm-vm/id_ed25519
- win.sh and provision-windows.sh verify the pinned
CBM_VM_HOST_KEY_SHA256 fingerprint before every connection
(StrictHostKeyChecking=yes against a keyscanned known_hosts, Ed25519
only, IdentitiesOnly), validate branch names before remote
interpolation, and fail remote pipelines loudly (set -e -o pipefail).
- The MSYS2 installer download is pinned to a versioned release URL with
SHA-256 verification instead of an unpinned latest.
- win.sh gains a sync subcommand mirroring the uncommitted worktree onto
the VM; its manifest logic is guarded by
tests/test_vm_worktree_manifest.sh, wired into scripts/test.sh.
guards now runs the maintained PowerShell driver against a clean
product build; smoke-install drives vm-smoke.sh.
- run.sh drops the stale duplicate windows-vm/mac-vm legs (the former
carried its own unpinned ssh path) in favor of the vm/ drivers.
The previously committed public key has been rotated off the VM and is
no longer authorized anywhere.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add vm-smoke.sh, which stages the release pair (launcher + payload) in a
profile-rooted temp dir and runs scripts/smoke-test.sh exactly as the CI
windows pr-smoke step does, so the real-Windows leg exercises the same
managed-install path. Extend win.sh with ubsan-build/ubsan-test (UBSan works
under x64-on-ARM emulation; ASan does not), a PageHeap toggle for OS-level
heap checking of the native runner, and a full-suite parallel run. Document
the honest sanitizer limits and the daily loop in the README.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/build.sh, test.sh, and clean.sh hardcoded build/c, and clean.sh
rm -rf'd it unconditionally. Running the container legs concurrently with a
native macOS build then had them delete and relink each other's objects
mid-build (ld "file cannot be opened"). Thread a BUILD_DIR through the three
scripts (default build/c) and point every compose service at its own dir
(build/linux-arm64, build/linux-amd64, build/win-cross, build/alpine); native
macOS keeps build/c exclusively. Also run the lint service in CI-parity mode
(--ci: cppcheck + clang-format only), matching CI, since clang-tidy is in
neither the lint image nor the CI job.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cli_windows_staging_root builds the per-user install staging root from
LOCALAPPDATA/USERPROFILE, but msys and CI harnesses export those in POSIX
drive form ("/c/Users/..."), which the CRT cannot resolve, and the target's
AppData\Local ancestors may not exist yet on a fake-profile fixture. Rewrite
a leading "/c/" to native "C:/" before use, create the whole ancestor chain
(a single mkdir left the root missing), and fall back to the temp root only
after verifying the directory exists. Also name the failing sub-step when
private staging cannot even create its directory, so the opaque "did not
remain runnable" message is no longer the only signal.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The backpressure/EOF fixtures waited only 10s for the forked daemon to
announce readiness, but the daemon's own startup includes the cohort-claim
path whose internal budget is HOST_DAEMON_CLAIM_TIMEOUT_MS (12s). Under the
4-core oversubscription of CI runners a slow-but-legitimate startup blew
past 10s and was SIGKILLed before it could announce, so daemon_frontend
backpressure failed almost every ubuntu run while passing on fast machines.
Raise the announce deadline to 30s, comfortably above the daemon's own
worst-case budget. Verified: the backpressure failures no longer reproduce
under a local CI-fidelity 4-core run.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Back-to-back requests on one frontend<->daemon connection failed roughly
half the time on loaded Windows hosts, surfacing as wandering MCP session
drops (smoke Phase 5/5e/5f "no tools/list response", "timed out waiting for
id=2"). Root cause, confirmed on real Windows: the application worker stored
its done flag AFTER writing the response, so a client that pipelines its
next request the instant it sees a response raced the previous worker's
final instructions and its join, and admission rejected the new request
with BUSY. On EOF that BUSY was then classified as an expected shutdown and
the response was silently dropped with a success exit; mid-session it exited
non-zero. A timing-sensitive heisenbug — any added stderr shifted the window
and it passed.
- runtime_application_worker publishes the done flag BEFORE sending the
response, so completion is one atomic transition from the client's view:
by the time any client can react, the slot already reads reusable. The
admission side joins the thread after observing the flag, which absorbs
the send tail. No timing constants.
- frontend_should_stop's input-closed leg now also requires no request
in flight, so a daemon-side failure of the final post-EOF item is loud,
never swallowed as an expected stop.
- The EOF drain is now progress-aware: it resets its window on each
completed item, bounding stuckness rather than total batch length, so a
slow cold daemon spawn no longer abandons queued tail responses.
- Latch the never-connected shutdown window on a monotonic admitted-total
counter instead of sampling the live client count, which can read zero
between two short-lived sessions and stop a daemon mid-conversation.
- Add structured lifecycle diagnostics (daemon.runtime_stopping reason=,
lifetime_end reason=, connection_end) so this class is observable in the
logs instead of being inferred.
Validated: real-Windows loop 12/12 (was ~50% loss), macOS daemon suites
143/143.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The activation transaction and the release-descriptor probe validate the
full ancestor chain of everything they stage: every path component must
have a trusted owner and grant mutation rights only to trusted identities.
That walk is deliberate TOCTOU defense and stays strict. The staging
directory, however, came from the process temp root, which routinely lives
in a tree that legitimately fails the walk through no fault of ours: under
msys2 TMP resolves inside the msys install tree (C:\msys64\tmp), and on
GitHub runners inside the runner work directory, both of which carry broad
inherited grants such as Authenticated Users modify on an upper component.
The staged pair then failed validation ("launcher descriptor candidate is
unsafe" / the smoke Phase 8 --version probe refusal) even though the staged
files themselves were private.
Add cli_windows_staging_root(): stage install and update transactions under
%LOCALAPPDATA%\codebase-memory-mcp, whose ancestor chain is user-owned end
to end, so the strict walk passes by construction instead of by environment
luck. Falls back to USERPROFILE\AppData\Local and finally the process temp
root when the environment is degraded.
Also make the staging observable where it previously failed blind:
cli_windows_stage_private_file reports which transaction step failed with
the OS error code, and cbm_mkdtemp warns once if the private security
descriptor cannot be applied and it falls back to default directory
security. A blind boolean here cost several blind CI round-trips.
win.sh: scp targets need native C:/ paths on Windows OpenSSH.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows VM becomes a first-class, reproducible local-CI leg driven
entirely over the local ssh link by maintained scripts:
- provision-windows.sh: idempotent host-side provisioner — msys2 base,
CLANGARM64 (native, fast) + CLANG64 (CI arch) toolchains, repo
clone/update, full build, smoke check. One command recovers a fresh VM
or a lost disk.
- win.sh: daily driver — status/update/build/test/guards/smoke-install/
sh/push-file/asan-build. smoke-install runs the managed-install E2E
with stderr visible, the class the CI smoke probe structurally hides.
- windows-bootstrap.ps1: reduced to the one-time in-VM door-opener
(OpenSSH + host key + GitHub-runner default-owner policy mirror), with
the ISO delivery trick documented; toolchain work moved to the
host-side provisioner. The previous winget path does not work over ssh
and clipboard/shared folders do not exist on Windows-ARM guests.
- README.md: full lifecycle — VM creation, clone-before-drivers rollback
point, SPICE guest tools, bootstrap, provisioning, daily loop, and the
honest limits (no ASan on aarch64-windows; ASan also faults under
x64-on-ARM emulation, so Windows sanitizer coverage stays on CI).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_mkdtemp on Windows created the directory with plain _mkdir, so it took
the token's default owner and the parent's inheritable DACL. Under an
Administrators-default-owner policy -- standard on Windows Server and
GitHub's elevated runners -- the temp directory was born owned by
BUILTIN\Administrators with foreign inherited grants. The activation
transaction validates its staging directory with a strict owner-is-current-
user check plus a trusted-only DACL scan (activation_directory_secure), so
install staging failed at the directory gate before any file was staged:
"private staging failed for the launcher" in the Phase 8 smoke, previously
masked as "did not remain runnable after private staging".
Create the directory with CreateDirectoryW and an explicit descriptor:
owner stamped to the token user and a protected, inheritable, user-only
DACL -- private by construction, the same policy the daemon applies to its
runtime directory. If the descriptor cannot be built the code falls back
to plain _mkdir so degraded environments (Wine) keep working; downstream
validation still gates security there.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Phase 8 staging diagnostic added two long lines that clang-format-20
(the CI formatter) wraps differently. Reformat them so lint passes; no
behavior change.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
activation_windows_security_init built the creation security descriptor with
a protected current-user DACL but never set an owner, so a file created with
these attributes inherited the token's default owner. Under an
Administrators-default-owner policy -- standard on Windows Server and on
GitHub's elevated Windows runners -- that default is BUILTIN\Administrators,
not the calling user. Every staged file is then validated with the strict
activation_windows_owner_is_current check, which rejected the
Administrators-owned file and made private staging fail with "launcher/payload
source pair did not remain runnable after private staging" (smoke Phase 8).
Stamp the owner explicitly to the token user in the creation descriptor, the
same fix the daemon side already applies. This keeps the strict owner check
intact -- the staged file genuinely becomes current-user-owned -- so it is not
a validation relaxation; setting the owner to our own token user needs no
extra privilege. Complements the launcher's trusted-owner acceptance, which
covers the installed binary that a non-admin user cannot re-stamp.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cli_windows_prepare_install_pair folded launcher staging, payload staging,
the payload fingerprint match, and the staged-launcher --version probe into
one bool, so the caller could only report "launcher/payload source pair did
not remain runnable after private staging" without saying which sub-step
failed. Windows cbm_exec_no_shell does not inherit the child's stderr, so the
launcher's own refusal reason never reaches the log either. Split the checks
so each failing sub-step prints a distinct error, turning the opaque Phase 8
smoke failure into a precise signal on the next run.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel