Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.
Daemon lifecycle:
- daemon start/stop/status subcommands. `daemon start` launches a
PERMANENT daemon (spawn shape is byte-exact argv; survives idle
periods and session ends) and reports an already-active daemon
instead of failing. Permanence is honored at every stop latch:
last-committed-client disconnect, host initial-client window,
coordinator release, and application final-session close — a
permanent daemon also keeps admitting new sessions after its last
one closes.
- daemon stop refuses while sessions are active and lists the blocking
peers (pid/role) that must finish first; an idle daemon drains
through the activation-shutdown machinery with the ACK ordered after
connection interrupts. A second stop is idempotent. The wire ops are
no-cohort first-frame requests with peer fingerprint authentication,
so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
keep their local supervised path). A cold CLI run that had to spawn a
temporary daemon prints a hint that `daemon start` removes the
per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
emits a visible, rate-limited notice (Claude-dialect systemMessage
plus stderr for other dialects) and always exits 0 — augmentation is
never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
older-build permanent daemon (strict semantic-version triples only;
dev builds never auto-drain) and the build-conflict message names
`cbm daemon stop` as the manual escape hatch.
Windows IPC/runtime (real-VM verified):
- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
path used to destroy its listening pipe instance on every 20 ms poll
timeout; a client attaching in the teardown window was severed or left
on an orphaned pipe object whose HELLO no server handle could ever
read, absorbing the connect until the client's own timeout expired.
The pending connect now survives poll timeouts and nothing is
destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
server handle can discard a just-sent response before the peer reads
it (POSIX stream sockets never lose buffered data on close). A bounded
cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
immediate on interrupted connections) now precedes close in
runtime_worker_finish and runtime_reject_inline, so hello-conflict,
capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
transport half-close, so close_begin now announces departure with an
explicit frame (ordered after APPLICATION_CANCEL, before the local
interrupt); the server releases the client's admission on receipt
instead of waiting for the handle to close. Admission-drop timing is
now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
request publication under the send lock, best-effort sends the active
token's APPLICATION_CANCEL frame, then interrupts local I/O; the
server cancels MCP/subprocess work promptly. Contract tests accept
both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
the requester's license to act on "snapshotted and draining", so every
connection interrupt is now initiated before the ACK is sent; a
session could previously get one more request serviced after the
requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
timer granularity) burned ~1.6 s against permanently obstructed paths,
stalling hello rejections past the client's timeout. The retry budget
is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
nonblocking send() on Windows is absorbed wholesale into AFD kernel
buffering regardless of SO_SNDBUF, so send deadlines and interrupts
could never engage against a slow peer (and the full payload was
pinned in nonpaged pool). Slicing restores a deterministic
backpressure point; a test hook pins SO_SNDBUF for the deadline and
interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
deadline, explicit connection states, refusal to free a server while
a listener-owned connection is active.
Windows long-path support:
- Central path-aware wide conversion (canonicalize via GetFullPathNameW
and prepend the extended-length prefix for absolute paths >=240) at
the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
store opens, and the daemon build-fingerprint/log paths. Deep managed
installs (a 64-hex generation directory routinely exceeds MAX_PATH)
now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
ancestry validators now operate in the extended-length namespace;
the launcher path is canonicalized (and prefixed when deep) once at
entry so every downstream exact-string comparison stays
form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
GetFileAttributesW) so non-ASCII install paths survive argv[0]
resolution.
Windows launcher install/uninstall transaction:
- FileRenameInfoEx names are NUL-terminated in an over-allocated
buffer. FileNameLength governs per the contract, but filter drivers
read FileName as NUL-terminated and appended adjacent heap bytes to
created names — a flaky, garbage-suffixed rename target. Both the CLI
and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
.cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
chars so the bare rename target stays under the FileRenameInfoEx
NT-conversion ceiling at guard depths; 64 bits still uniquely
identify the generation.
- When the running launcher's mapped generation backings pin .cbm
against rename, the backings are relocated to activation-<pid>-N
.retired tombstones beside the install (a mapped image may be renamed,
never deleted; the launcher's liveness-guarded sweep reclaims stale
tombstones). Every relocation is recorded, and a FAILED uninstall
reverses the moves after restoring .cbm — via MoveFileExW with
extended-length paths on both arguments, since the deep generation
target is beyond the handle-based rename's bare-path reach — so a
restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
out so the tree is shallow enough for the detached cleanup's rd, and
the cleanup's working directory strips the extended-length prefix
(CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
(CopyFileW destinations, CREATE_NEW tombstones, probe directories)
are explicitly owner-stamped so the exact-owner validators hold on
runner images; guard fixtures stamp hand-built trees the same way.
Diagnostics, tests and infra:
- diagnostics: discovery is now an always-delivered JSON control record
(new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
paths containing spaces; placement honors $TMPDIR with /tmp fallback
via a diagnostics-local helper; the soak parser reads the JSON record;
documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
log levels, TMPDIR-with-spaces, and native Windows output-contract
assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
parents with production-shaped ancestry (LocalAppData on Windows, via
th_secure_runtime_parent_new) — the runtime ancestry validation
correctly refuses temp roots whose ancestors grant mutation rights to
Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
drive the documented startup-owner publication flow before reading
generation-bound endpoint addresses. This turns the 26 Windows
failures previously visible in CI's full-test job green without
weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
contract including failed-uninstall restore and immediate reinstall
after uninstall; new daemon lifecycle and reworked hook-augment
guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
(it was set by CI but never read, so perf suites ran everywhere);
four throughput/bench suites are classified as perf, the heavy
store_arch suite moved to the slow-timeout tier, and two
wall-clock-sensitive assertions were rewritten as invariant checks
with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
contract, soak daemon-recovery contract, path-safety helper, the
Windows VM worktree-sync contract wired into scripts/test.sh, and
vm/win.sh guards building its clean embedded-UI product in an
isolated BUILD_DIR so it cannot clobber the incremental test build.
provision-windows.sh now installs Node.js for the guards UI build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A persistently dirty or untracked worktree re-triggered a full reindex
(and its DB/artifact rewrite) on every poll cycle - git_is_dirty()
answered 'is the tree dirty' where the watcher needs 'did the tree
CHANGE'. Idle repos with one uncommitted file produced sustained disk
write amplification (reported at 1 TB/day across several repos).
The watcher now tracks a dirty-state signature: FNV-1a over the
entries of git status --porcelain -uall -z plus each listed path's
(size, mtime_ns). Same signature -> no reindex; editing a dirty file,
adding/removing one, or reverting the tree to clean each produce a new
signature and trigger exactly one reindex. Submodule status is folded
in on POSIX, mirroring the old submodule dirty probe. 0 is reserved
for a clean tree; a dirty-at-baseline tree still reindexes once (the
watcher cannot know whether that state reached the DB, e.g. after a
server restart with a stale artifact).
Baselines (HEAD and signature) are now committed only after a
SUCCESSFUL reindex: check_changes stages observations in pending_*
fields, and busy-skips (index_fn > 0, e.g. pipeline lock held) or
failed runs leave the baselines untouched so the change is retried on
the next poll instead of being recorded as seen and silently lost -
the old code stored the new HEAD at check time, dropping any commit
that arrived while the pipeline was busy.
Closes#937
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DELETE /api/project removed the project's .db file but never told the
watcher; the zombie watch kept polling the repo and the next change
silently resurrected the deleted DB (#803). Wire the watcher into the
UI HTTP server (non-owned ref; created before the server, freed after
it) and unwatch after a successful unlink and on ENOENT (clears an
already-orphaned watch, still 404). Invalid name stays a plain 404
with no unwatch; a non-ENOENT unlink failure stays 500 and keeps the
watch. The cbm_file_exists pre-check is replaced by unlink-errno
handling, removing the TOCTOU window.
Distilled from #830, rebased over the defer_state_free refactor
(#817): the OOM hardening of the deferred-free path moves into
defer_state_free itself so BOTH unwatch and stale-root prune get it.
On pending-free realloc failure the helper no longer frees the state
immediately (a potential use-after-free against an in-flight poll_once
snapshot) — it logs watcher.unwatch.oom and returns failure, and the
callers keep the entry registered; the prune path retries on the next
poll cycle.
Carries six httpd regression tests from #830 driving the live UI
server with a real watcher wired in.
Closes#803.
Co-authored-by: Dustin Persek <dustin.persek@protonmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #738: the watcher now prunes a watched project whose root
directory has genuinely disappeared - deleting the cached DB (+wal/shm,
validated name, cache-dir-only paths) and removing the watch entry - so
vanished worktrees stop being watched forever (#286).
Hardened beyond the original PR to remove a data-loss hazard (the cached
DB can hold user-authored data such as the ADR, unrecoverable once
deleted):
- Only ENOENT/ENOTDIR stat failures count as missing. Any other failure
(EACCES, EIO, transient mounts, macOS TCC revocation) resets the
streak and logs watcher.root_stat_error with the errno; Windows
(mingw/UCRT) maps not-found to ENOENT so the check holds there.
- Pruning requires BOTH >=3 consecutive missing polls AND a sustained-
absence grace window since the streak's first miss (default 600s,
override via CBM_WATCHER_PRUNE_GRACE_S), tracked with monotonic
cbm_now_ms.
- Root reappearance resets streak + timestamp (watcher.root_restored).
- The pruned entry is released via the deferred-free list poll_once
drains (one freeing model shared with cbm_watcher_unwatch).
Limitation: only currently-watched projects are pruned; stale DBs left
by older sessions are out of scope.
Co-authored-by: pcristin <xxxokzxxx@protonmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The watcher shells out to git via cbm_popen, which runs through cmd.exe on Windows. The commands used POSIX-only constructs: single-quoted paths ('%s'), which cmd.exe does NOT strip — so git received a literal-quoted path, failed with 'cannot find the path', and the watcher fell back to strategy=none (no change detection); plus /dev/null and '| wc -l'. Use double-quoted paths (stripped by cmd.exe and POSIX sh alike), a platform null device (NUL / dev/null), count ls-files lines in C, and gate the submodule-foreach enhancement (whose inner shell command cmd.exe cannot pass) to POSIX. The watcher's git change-detection now works on Windows; this is what the (newly un-skipped) watcher tests exercise.
Follow-up to #207:
- cbm_watcher_new: NULL-check cbm_ht_create() and free w on OOM so
later watch/unwatch/touch don't dereference a NULL hash table.
- cbm_watcher_watch_count: drop const from the signature and remove
the const_cast around projects_lock. The function necessarily
touches synchronization primitives so a const receiver was
misleading; all current callers pass a non-const pointer.
- http_server: replace lazy log mutex init with 3-state UNINIT/INITING/INITED CAS pattern; cbm_ui_log_init() called from main before thread creation, also still invoked lazily from cbm_ui_log_append() so early single-threaded logs are not dropped
- watcher: protect projects hash table with cbm_mutex_t (watch/unwatch/touch/watch_count/free); poll_once snapshots project pointers under lock then iterates without it so git I/O and indexing don't block watch/unwatch
- watcher: handle state_new() OOM in watch path
- compat_thread: add cbm_thread_detach() for POSIX and Windows; both join() and detach() clear the handle on success for consistent lifecycle tracking
- http_server: detach index job threads to prevent handle leaks
The watcher's git_is_dirty() only ran git status on the parent repo,
missing uncommitted changes inside submodules. Add a second check via
git submodule foreach --recursive (portable, works on Apple Git which
lacks --recurse-submodules).
Fixes#71
Co-Authored-By: halindrome <halindrome@users.noreply.github.com>
- platform.c: move nlen into if(env) block (variableScope)
- userconfig.c: move home into else block (variableScope)
- pipeline.c: remove dead rc!=0 check after httplinks removal
- cli.c: rmdir_recursive propagates cbm_rmdir errors, remove redundant local
- clang-tidy: whitelist cosmetic-only checks, enable all safety checks
- Thread-safe getenv via environ iteration (concurrency-mt-unsafe fix)
- Multiple files: add constants.h includes for named constants
Enforce WarningsAsErrors: '*' with zero suppressions of entire check categories.
All non-FP warnings fixed at source; true false positives get per-instance NOLINT
with reasoning. Fixes include: braces around statements (1679), isolate declarations
(75), explicit casts for multilevel pointer conversions, missing includes, function
parameter documentation via NOLINT, and proper NOLINTBEGIN/NOLINTEND nesting.
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.
Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
Martin Vogel