Git exports repository-local variables (GIT_DIR, GIT_WORK_TREE,
GIT_INDEX_FILE, GIT_COMMON_DIR, ...) into hooks, and some editors export
them into their terminals. They take precedence over `git -C <dir>`, and
every git child cbm spawned inherited them. A cbm started from such an
environment therefore resolved every project against the CALLER's
repository: git_context reported a plain directory as is_git=true with
the caller's worktree root, branch and HEAD, the history pass attributed
the caller's commits to it, and the watcher / detect_changes /
artifact git calls read (and `config merge.ours.driver` could write) the
wrong repository.
Fix: one production helper, src/foundation/git_env.{c,h}, holds the
15-variable list `git rev-parse --local-env-vars` prints and builds the
child environment with exactly those removed -- a POSIX envp array or a
Windows CREATE_UNICODE_ENVIRONMENT block (case-insensitive names). It is
applied per spawn; cbm's own environment is never modified (no unsetenv
in the parent, which would race other threads):
- cbm_popen_git(cmd): read-mode popen for git command lines. POSIX spawns
/bin/sh -c via posix_spawn with the scrubbed envp (stdin /dev/null,
pipe ends close-on-exec, reaped by cbm_pclose); Windows passes the block
to the existing handle-isolated CreateProcessW spawn. Fails closed.
- cbm_proc_opts_t.strip_git_repo_env: cbm_subprocess_spawn passes the
scrubbed envp to posix_spawnp / the fork+exec child, or the block to
CreateProcessW. Fails closed.
- Every git spawn uses them: git_context, pass_githistory, artifact
(4 sites), the watcher's git runner, and detect_changes' git commands.
search_code's grep scan keeps the inherited environment.
Tests (RED before, green after, RED on revert of the helper and of the
call sites):
- git_context::git_context_ignores_inherited_repo_env -- GIT_DIR and the
hook set pointing at a decoy repo: a plain dir is is_git=false, a
fixture repo reports its own HEAD and branch, the decoy is untouched.
- git_context::githistory_ignores_inherited_repo_env -- the history pass
on a plain dir sees 0 commits (was the decoy's 4).
- subprocess::subprocess_strip_git_repo_env_is_per_child and
popen_git_strips_repo_env_and_reports_exit_status -- all 15 variables
are gone in the child, unrelated variables pass through, the parent
environment is unchanged, and cbm_pclose keeps pclose() status
semantics.
Proof (macOS arm64, GIT_DIR/GIT_WORK_TREE/GIT_INDEX_FILE/GIT_PREFIX
exported at a decoy clone, `cli index_status verbose=true`): a plain dir
went from is_git=true, worktree_root=<decoy>, branch=<decoy's> (v0.11.0)
to is_git=false; a fixture repo went from the decoy's root/branch/HEAD
to its own root and branch.
Refs #2003
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The EAGAIN retry shipped in v0.10.3 was too small to matter. It waited a flat
3 x 10ms, which covers a momentary dip and not the real thing: a CI runner
building and testing in parallel stays process-starved for hundreds of
milliseconds at a stretch. `subprocess_run_spawn_failure` kept failing WITH the
retry in place — macos-15-intel on a release matrix, then macos-14 under
ThreadSanitizer, twice on the same SHA, which is what moved this out of the
flake bucket. A fixed short delay just samples the same congested instant
repeatedly; doubling walks out of it.
Now 10/20/40/80/160/320ms — about 0.6s of total patience. That is invisible
next to spawning a process that does real work, and a machine still refusing
after it is genuinely out of capacity, where failing fast beats hanging.
The previous attempt shipped a constant with no way to prove it worked, so this
adds the seam that was missing: CBM_ENABLE_TEST_SEAMS builds can force N
simulated refusals, and two tests pin the behaviour deterministically — fewer
refusals than the budget must still spawn, more must fail rather than retry
forever. Both are revert-checked. The seam compiles out entirely in production
(cppcheck correctly called the always-false branch dead code, so it is gone
rather than suppressed).
Two things the tests caught that review had not:
- The macOS primary path is posix_spawn, not fork; injecting only into the fork
fallback exercised nothing on macOS. Both paths now carry it.
- cbm_fork_with_retry ended with an unconditional fork() OUTSIDE the loop, so
the final attempt could never be reached by a test. Every attempt now goes
through the same branch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.
Daemon lifecycle:
- daemon start/stop/status subcommands. `daemon start` launches a
PERMANENT daemon (spawn shape is byte-exact argv; survives idle
periods and session ends) and reports an already-active daemon
instead of failing. Permanence is honored at every stop latch:
last-committed-client disconnect, host initial-client window,
coordinator release, and application final-session close — a
permanent daemon also keeps admitting new sessions after its last
one closes.
- daemon stop refuses while sessions are active and lists the blocking
peers (pid/role) that must finish first; an idle daemon drains
through the activation-shutdown machinery with the ACK ordered after
connection interrupts. A second stop is idempotent. The wire ops are
no-cohort first-frame requests with peer fingerprint authentication,
so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
keep their local supervised path). A cold CLI run that had to spawn a
temporary daemon prints a hint that `daemon start` removes the
per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
emits a visible, rate-limited notice (Claude-dialect systemMessage
plus stderr for other dialects) and always exits 0 — augmentation is
never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
older-build permanent daemon (strict semantic-version triples only;
dev builds never auto-drain) and the build-conflict message names
`cbm daemon stop` as the manual escape hatch.
Windows IPC/runtime (real-VM verified):
- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
path used to destroy its listening pipe instance on every 20 ms poll
timeout; a client attaching in the teardown window was severed or left
on an orphaned pipe object whose HELLO no server handle could ever
read, absorbing the connect until the client's own timeout expired.
The pending connect now survives poll timeouts and nothing is
destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
server handle can discard a just-sent response before the peer reads
it (POSIX stream sockets never lose buffered data on close). A bounded
cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
immediate on interrupted connections) now precedes close in
runtime_worker_finish and runtime_reject_inline, so hello-conflict,
capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
transport half-close, so close_begin now announces departure with an
explicit frame (ordered after APPLICATION_CANCEL, before the local
interrupt); the server releases the client's admission on receipt
instead of waiting for the handle to close. Admission-drop timing is
now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
request publication under the send lock, best-effort sends the active
token's APPLICATION_CANCEL frame, then interrupts local I/O; the
server cancels MCP/subprocess work promptly. Contract tests accept
both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
the requester's license to act on "snapshotted and draining", so every
connection interrupt is now initiated before the ACK is sent; a
session could previously get one more request serviced after the
requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
timer granularity) burned ~1.6 s against permanently obstructed paths,
stalling hello rejections past the client's timeout. The retry budget
is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
nonblocking send() on Windows is absorbed wholesale into AFD kernel
buffering regardless of SO_SNDBUF, so send deadlines and interrupts
could never engage against a slow peer (and the full payload was
pinned in nonpaged pool). Slicing restores a deterministic
backpressure point; a test hook pins SO_SNDBUF for the deadline and
interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
deadline, explicit connection states, refusal to free a server while
a listener-owned connection is active.
Windows long-path support:
- Central path-aware wide conversion (canonicalize via GetFullPathNameW
and prepend the extended-length prefix for absolute paths >=240) at
the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
store opens, and the daemon build-fingerprint/log paths. Deep managed
installs (a 64-hex generation directory routinely exceeds MAX_PATH)
now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
ancestry validators now operate in the extended-length namespace;
the launcher path is canonicalized (and prefixed when deep) once at
entry so every downstream exact-string comparison stays
form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
GetFileAttributesW) so non-ASCII install paths survive argv[0]
resolution.
Windows launcher install/uninstall transaction:
- FileRenameInfoEx names are NUL-terminated in an over-allocated
buffer. FileNameLength governs per the contract, but filter drivers
read FileName as NUL-terminated and appended adjacent heap bytes to
created names — a flaky, garbage-suffixed rename target. Both the CLI
and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
.cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
chars so the bare rename target stays under the FileRenameInfoEx
NT-conversion ceiling at guard depths; 64 bits still uniquely
identify the generation.
- When the running launcher's mapped generation backings pin .cbm
against rename, the backings are relocated to activation-<pid>-N
.retired tombstones beside the install (a mapped image may be renamed,
never deleted; the launcher's liveness-guarded sweep reclaims stale
tombstones). Every relocation is recorded, and a FAILED uninstall
reverses the moves after restoring .cbm — via MoveFileExW with
extended-length paths on both arguments, since the deep generation
target is beyond the handle-based rename's bare-path reach — so a
restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
out so the tree is shallow enough for the detached cleanup's rd, and
the cleanup's working directory strips the extended-length prefix
(CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
(CopyFileW destinations, CREATE_NEW tombstones, probe directories)
are explicitly owner-stamped so the exact-owner validators hold on
runner images; guard fixtures stamp hand-built trees the same way.
Diagnostics, tests and infra:
- diagnostics: discovery is now an always-delivered JSON control record
(new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
paths containing spaces; placement honors $TMPDIR with /tmp fallback
via a diagnostics-local helper; the soak parser reads the JSON record;
documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
log levels, TMPDIR-with-spaces, and native Windows output-contract
assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
parents with production-shaped ancestry (LocalAppData on Windows, via
th_secure_runtime_parent_new) — the runtime ancestry validation
correctly refuses temp roots whose ancestors grant mutation rights to
Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
drive the documented startup-owner publication flow before reading
generation-bound endpoint addresses. This turns the 26 Windows
failures previously visible in CI's full-test job green without
weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
contract including failed-uninstall restore and immediate reinstall
after uninstall; new daemon lifecycle and reworked hook-augment
guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
(it was set by CI but never read, so perf suites ran everywhere);
four throughput/bench suites are classified as perf, the heavy
store_arch suite moved to the slow-timeout tier, and two
wall-clock-sensitive assertions were rewritten as invariant checks
with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
contract, soak daemon-recovery contract, path-safety helper, the
Windows VM worktree-sync contract wired into scripts/test.sh, and
vm/win.sh guards building its clean embedded-UI product in an
isolated BUILD_DIR so it cannot clobber the incremental test build.
provision-windows.sh now installs Node.js for the guards UI build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up to #881. cbm_build_win_cmdline() returned false from inside its loop on
buffer overflow without NUL-terminating buf, although its header contract states the
buffer holds a string. No live bug — both call sites (cbm_run_win and the UI
http_server index spawn) ignore buf when the function returns false — but a future
caller trusting the documented contract could read an unterminated (possibly
uninitialized) buffer. Set buf[0]='\0' on the overflow path so it is always a valid
string, and correct the header comment to match.
Reproduce-first: the new win_cmdline_overflow_leaves_empty_string test feeds an argv
that overflows a small cap and asserts buf[0]=='\0'. It is RED on the pre-fix code
(buf[0] holds the first quoted byte '"' = 0x22) and GREEN with the fix.
Also tidies two review nits in the same files:
- fix the stale cbm_cmdline_put comment (it returns pos UNCHANGED on overflow;
the overflow is signalled via *ovf, not an advancing pos),
- bounds-guard the reference CommandLineToArgvW parser in the round-trip test.
Refs: #881
Signed-off-by: Flipper <jacobphilipp@ymail.com>
Fixes the Windows-only smoke-windows (standard + ui) failure in the release dry
run, where index_repository reported outcome=exit_nonzero and "Indexing worker
crashed on a file" on big_templated.hpp. It is not the #424 allocator class and
not big_templated.hpp — the file parses fine; the worker never reaches it.
This is the Windows residual of #838. That issue fixed the CLI parser to accept
the supervisor's raw-JSON-positional + --response-out flag layout, verified with a
PowerShell repro where the JSON argument arrives intact. But the real supervisor
spawns the worker via CreateProcessA, whose command-line re-parse strips the JSON's
quotes, so on Windows the now-fixed parser still receives corrupted JSON.
On Windows the index supervisor spawns the worker as
`<self> cli --index-worker index_repository <args_json> ...`, but cbm_run_win
built the CreateProcess command line by wrapping each argv element in bare quotes
with no escaping. Windows re-parses that single string back into argv and strips
the inner quotes, so the JSON argument {"repo_path":"..."} arrived at the child as
{repo_path:...} — invalid JSON. The worker exited non-zero at arg parse
("repo_path is required"); the supervisor classified exit_nonzero and blamed the
last-marked file (big_templated.hpp), producing the misleading crash message.
POSIX is unaffected: cbm_run_posix passes the argv array straight to execv with no
re-parse, which is why only smoke-windows was red while Linux/macOS indexed the
same fixture fine.
Fix: cbm_build_win_cmdline() applies the Microsoft C runtime quoting rules
(quote-wrap + escape embedded quotes and their preceding backslash runs) so the
child re-parses byte-identical argv. It is defined unconditionally (pure string
logic, no Windows headers) and unit-tested on every platform via a round-trip
against a reference CommandLineToArgvW parser.
Also in this change:
- ui/http_server.c: the UI index spawn had a byte-for-byte duplicate of the same
bug (its own "%s" cmdline). Route it through cbm_build_win_cmdline too.
- mcp/index_supervisor.c: log the worker's raw exit_code and KEEP its log on any
failure (delete only on a clean run). Previously the log was always deleted and
only outcome+signal were logged, so a non-zero worker left nothing to diagnose —
the CI blind spot that hid this bug behind a generic message.
A separate hardening of pipeline/artifact.c's single-quoted git shell-outs (a
different bug class — popen command-string quoting, not CreateProcess argv) is
split into a follow-up PR with its own guard test, per review.
Verified on a CLANG64 build: the supervised big_templated.hpp fixture now indexes
clean (reap outcome=clean exit_code=0); subprocess suite 11/11, affected suites
(git_context, index_resilience, ui, httpd, mcp, security, str_util) green.
Refs: #838, #423
Signed-off-by: Flipper <jacobphilipp@ymail.com>
New src/foundation/subprocess.{h,c}: spawn a child, tail its output, and
classify how it ended — clean / exit-nonzero / crash / hang / killed — from
POSIX WIFSIGNALED/WTERMSIG and the Windows NTSTATUS exception exit codes
(0xC0000005 access violation, 0xC00000FD stack overflow). Adds an EINTR-safe
reap loop, a quiet-timeout that kills and reports a hung child (no new output
within the window), and partial-line-safe log tailing.
Generalized from the ad-hoc index spawn in src/ui/http_server.c so the
crash/hang supervisor can reuse one primitive across platforms. The exit-code
to outcome mapping lives in a pure cbm_proc_classify() so the Windows
crash-code path is unit-tested on every platform, not just an untested
Windows branch.
tests/test_subprocess.c: 13 tests — the classifier on all platforms plus real
POSIX spawn/reap of clean, non-zero, SIGSEGV-crash and hang children. Green
under ASan/UBSan.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel