A persistently dirty or untracked worktree re-triggered a full reindex
(and its DB/artifact rewrite) on every poll cycle - git_is_dirty()
answered 'is the tree dirty' where the watcher needs 'did the tree
CHANGE'. Idle repos with one uncommitted file produced sustained disk
write amplification (reported at 1 TB/day across several repos).
The watcher now tracks a dirty-state signature: FNV-1a over the
entries of git status --porcelain -uall -z plus each listed path's
(size, mtime_ns). Same signature -> no reindex; editing a dirty file,
adding/removing one, or reverting the tree to clean each produce a new
signature and trigger exactly one reindex. Submodule status is folded
in on POSIX, mirroring the old submodule dirty probe. 0 is reserved
for a clean tree; a dirty-at-baseline tree still reindexes once (the
watcher cannot know whether that state reached the DB, e.g. after a
server restart with a stale artifact).
Baselines (HEAD and signature) are now committed only after a
SUCCESSFUL reindex: check_changes stages observations in pending_*
fields, and busy-skips (index_fn > 0, e.g. pipeline lock held) or
failed runs leave the baselines untouched so the change is retried on
the next poll instead of being recorded as seen and silently lost -
the old code stored the new HEAD at check time, dropping any commit
that arrived while the pipeline was busy.
Closes#937
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DELETE /api/project removed the project's .db file but never told the
watcher; the zombie watch kept polling the repo and the next change
silently resurrected the deleted DB (#803). Wire the watcher into the
UI HTTP server (non-owned ref; created before the server, freed after
it) and unwatch after a successful unlink and on ENOENT (clears an
already-orphaned watch, still 404). Invalid name stays a plain 404
with no unwatch; a non-ENOENT unlink failure stays 500 and keeps the
watch. The cbm_file_exists pre-check is replaced by unlink-errno
handling, removing the TOCTOU window.
Distilled from #830, rebased over the defer_state_free refactor
(#817): the OOM hardening of the deferred-free path moves into
defer_state_free itself so BOTH unwatch and stale-root prune get it.
On pending-free realloc failure the helper no longer frees the state
immediately (a potential use-after-free against an in-flight poll_once
snapshot) — it logs watcher.unwatch.oom and returns failure, and the
callers keep the entry registered; the prune path retries on the next
poll cycle.
Carries six httpd regression tests from #830 driving the live UI
server with a real watcher wired in.
Closes#803.
Co-authored-by: Dustin Persek <dustin.persek@protonmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #738: the watcher now prunes a watched project whose root
directory has genuinely disappeared - deleting the cached DB (+wal/shm,
validated name, cache-dir-only paths) and removing the watch entry - so
vanished worktrees stop being watched forever (#286).
Hardened beyond the original PR to remove a data-loss hazard (the cached
DB can hold user-authored data such as the ADR, unrecoverable once
deleted):
- Only ENOENT/ENOTDIR stat failures count as missing. Any other failure
(EACCES, EIO, transient mounts, macOS TCC revocation) resets the
streak and logs watcher.root_stat_error with the errno; Windows
(mingw/UCRT) maps not-found to ENOENT so the check holds there.
- Pruning requires BOTH >=3 consecutive missing polls AND a sustained-
absence grace window since the streak's first miss (default 600s,
override via CBM_WATCHER_PRUNE_GRACE_S), tracked with monotonic
cbm_now_ms.
- Root reappearance resets streak + timestamp (watcher.root_restored).
- The pruned entry is released via the deferred-free list poll_once
drains (one freeing model shared with cbm_watcher_unwatch).
Limitation: only currently-watched projects are pruned; stale DBs left
by older sessions are out of scope.
Co-authored-by: pcristin <xxxokzxxx@protonmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The watcher shells out to git via cbm_popen, which runs through cmd.exe on Windows. The commands used POSIX-only constructs: single-quoted paths ('%s'), which cmd.exe does NOT strip — so git received a literal-quoted path, failed with 'cannot find the path', and the watcher fell back to strategy=none (no change detection); plus /dev/null and '| wc -l'. Use double-quoted paths (stripped by cmd.exe and POSIX sh alike), a platform null device (NUL / dev/null), count ls-files lines in C, and gate the submodule-foreach enhancement (whose inner shell command cmd.exe cannot pass) to POSIX. The watcher's git change-detection now works on Windows; this is what the (newly un-skipped) watcher tests exercise.
Follow-up to #207:
- cbm_watcher_new: NULL-check cbm_ht_create() and free w on OOM so
later watch/unwatch/touch don't dereference a NULL hash table.
- cbm_watcher_watch_count: drop const from the signature and remove
the const_cast around projects_lock. The function necessarily
touches synchronization primitives so a const receiver was
misleading; all current callers pass a non-const pointer.
- http_server: replace lazy log mutex init with 3-state UNINIT/INITING/INITED CAS pattern; cbm_ui_log_init() called from main before thread creation, also still invoked lazily from cbm_ui_log_append() so early single-threaded logs are not dropped
- watcher: protect projects hash table with cbm_mutex_t (watch/unwatch/touch/watch_count/free); poll_once snapshots project pointers under lock then iterates without it so git I/O and indexing don't block watch/unwatch
- watcher: handle state_new() OOM in watch path
- compat_thread: add cbm_thread_detach() for POSIX and Windows; both join() and detach() clear the handle on success for consistent lifecycle tracking
- http_server: detach index job threads to prevent handle leaks
The watcher's git_is_dirty() only ran git status on the parent repo,
missing uncommitted changes inside submodules. Add a second check via
git submodule foreach --recursive (portable, works on Apple Git which
lacks --recurse-submodules).
Fixes#71
Co-Authored-By: halindrome <halindrome@users.noreply.github.com>
- platform.c: move nlen into if(env) block (variableScope)
- userconfig.c: move home into else block (variableScope)
- pipeline.c: remove dead rc!=0 check after httplinks removal
- cli.c: rmdir_recursive propagates cbm_rmdir errors, remove redundant local
- clang-tidy: whitelist cosmetic-only checks, enable all safety checks
- Thread-safe getenv via environ iteration (concurrency-mt-unsafe fix)
- Multiple files: add constants.h includes for named constants
Enforce WarningsAsErrors: '*' with zero suppressions of entire check categories.
All non-FP warnings fixed at source; true false positives get per-instance NOLINT
with reasoning. Fixes include: braces around statements (1679), isolate declarations
(75), explicit casts for multilevel pointer conversions, missing includes, function
parameter documentation via NOLINT, and proper NOLINTBEGIN/NOLINTEND nesting.
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.
Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
Martin Vogel