The per-file tree-sitter parse was bounded by a 5s WALL-CLOCK budget. Under CI
contention a worker thread can be descheduled long enough for 5s of wall time
to elapse against near-zero CPU, so a small, perfectly parseable file was
abandoned with a false "parse timeout" and yielded ZERO defs. When that file
defined a symbol other files resolve against (e.g. celery.app.Celery.send_task
backing an ASYNC_CALLS edge, or a userpb gRPC client backing GRPC_CALLS), the
cross-file service edge then failed to classify -- surfacing as the flaky
convergence_probe / edge_types_probe service-edge assertions (count 0), a
verdict decided by wall-clock scheduling rather than the input (violates O9,
"no timeout decides a test").
Gate the parse on per-thread CPU time instead, with a generous wall ceiling as
a backstop:
- New portable helper cbm_thread_cpu_time_ns() (compat.c/.h): POSIX/macOS
clock_gettime(CLOCK_THREAD_CPUTIME_ID), Windows GetThreadTimes (kernel+user);
returns 0 on clock failure, degrading safely to the wall ceiling. It is NOT
cbm_clock_gettime, which is wall on every platform (QueryPerformanceCounter
on Windows).
- The progress callback's payload becomes {cpu_deadline_ns, wall_ceiling_ns};
it trips when this thread's CPU time passes the budget OR wall time passes
t0 + budget * 12 (~60s for the 5s budget). Descheduling burns wall, not CPU,
so a starved-but-parseable file survives; a CPU-spinning parse still trips
the CPU deadline; a genuinely stuck parse still trips the wall ceiling
(bounded, no hang).
RED-repro (deterministic, CBM_ENABLE_TEST_SEAMS only): a thread-local wall-clock
offset armed by CBM_TEST_WALL_STALL_ON emulates a descheduled worker (wall
reads budget + 1s ahead while CPU is untouched -- above the old 1x wall budget,
below the ceiling). async_calls_celery_wall_stall_seam arms it for the Celery
DEFINITION file and asserts ASYNC_CALLS >= 1. With the fix the parse completes
and the edge resolves; with the budget reverted to wall-only + a 1x ceiling the
def file is dropped and the edge count is 0.
Verified (macOS, sanitized): edge_types_probe + convergence_probe 107 passed /
0 failed with the fix; 58 passed / 1 failed on revert (only the seam test, at
the ASYNC_CALLS assertion); 107 restored. Regression smoke (extraction
parse_coverage pipeline lang_contract grammar_regression) 698 passed / 0
failed. lint-ci clean (cppcheck + clang-format Homebrew LLVM + NOLINT); the
Windows GetThreadTimes path is compile-checked by CI (not runnable on macOS).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A burst of one-shot clients (e.g. six parallel `cli list_projects`) shares one
ephemeral daemon generation and races to connect. The generation retired the
instant its last committed client disconnected, even when another one-shot's
connection was already accepted and still mid-HELLO -- stranding that racer on
a STOPPING daemon and forcing a full cold respawn (up to the 12s daemon-claim
handoff each). Under load the retire/respawn churn could exhaust a client's
30s bootstrap budget, surfacing as the Windows guard's "cold-storm client
failed (racing daemon spawn)" regression and, on POSIX, a startup that never
reached "serving".
When the last committed client of an ephemeral, non-permanent generation
disconnects, linger a bounded window if a peer connection is already accepted
but has not yet passed HELLO admission (in_use, connected, not admitted, not
disconnecting -- runtime_has_pending_hello_peer_locked). The accept loop's
reconcile retires the generation once that racer drains without committing or
the bounded window (RUNTIME_EPHEMERAL_LINGER_MS, seam-overridable) elapses; a
racer that does commit clears the linger. With no such racer -- the common
single-client case -- the generation retires immediately, unchanged.
The linger deliberately covers ONLY a pre-HELLO racer, never an already-
admitted provisional session (HELLO done, application session opening): the
contract that a provisional coordinator client cannot keep a retiring
generation alive is preserved (daemon_runtime_final_disconnect_rejects_blocked_
provisional_session). The daemon cannot count cross-process cohort peers
(advisory locks expose presence, not a holder count, and Windows LockFileEx has
no non-owning probe), so this uses the one peer signal it has locally: accepted
connections not yet admitted. The window is bounded, so a racer that never
commits cannot wedge the generation (rules out the unbounded idle hang).
Coordinator gains a transient `linger` hold (distinct from `permanent`) so its
own last-client self-transition to STOPPING is suppressed while the racer is
outstanding; clearing it while idle retires the coordinator at once.
Verified (macOS, sanitized build): daemon suites 224 passed / 0 failed / 1
skipped (the skip is the pre-existing Linux-only userns case). New tests pin
both directions -- daemon_runtime_ephemeral_lingers_while_peer_mid_hello
(a raw accepted-but-unHELLO'd peer keeps the generation RUNNING after the
committed client leaves) and daemon_runtime_ephemeral_linger_retires_within_bound
(the bounded backstop retires even while the peer is still accepted). RED on
revert: neutering the gate flips ASSERT(lingered) at test_daemon_runtime.c and
nothing else. clang-format (Homebrew LLVM) + cppcheck + NOLINT clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Windows install/daemon runtime directories created by an elevated process are
owned by the built-in Administrator ACCOUNT (RID 500 under the local machine
account-domain SID), even when that account is renamed or disabled. cbm's
directory-owner/ACL trust refused it, breaking install/activation on such
machines.
#1705: resolve THIS machine's built-in Administrator SID via LSA
(LsaOpenPolicy(POLICY_VIEW_LOCAL_INFORMATION) +
LsaQueryInformationPolicy(PolicyAccountDomainInformation) ->
CreateWellKnownSid(WinAccountAdministratorSid, accountDomainSid)), cached once
per process, and accept it as a trusted owner/grantee in the daemon
(win_sid_trusted) and the CLI activation transaction. Resolution is
deliberately NOT IsWellKnownSid(WinAccountAdministratorSid) and NOT a
trailing-RID-500 match: both accept a FOREIGN S-1-5-21-*-500 (a domain
administrator, or another machine's built-in Administrator), which would be a
cross-machine trust escalation. Any LSA or synthesis failure caches "none" and
grants no tolerance (fail closed). owner-not-current-user (the exact-owner
target rule) is unchanged.
#2023/#1686: an owner-trust refusal now names the offending owner SID (the path
was already carried), so a terse "status -3, os 0" becomes actionable.
Scope notes: on a domain controller -- which has no separate local SAM, so its
account domain IS the AD domain -- this trusts the domain Administrator, by
design (on a DC that account is the machine-equivalent top admin). An
un-sysprepped cloned Windows image shares the machine account-domain SID, so a
sibling clone's built-in Administrator would be non-foreign; this is inherent
to machine-SID identity.
Verified on a native Windows (arm64) VM: compiles under the sanitized build;
the daemon_ipc SID-trust test passes (local RID-500 trusted, a fabricated
foreign S-1-5-21-*-500 rejected, BUILTIN\Administrators trusted) at 35 passed /
0 failed with the fix, and 34 passed / 1 failed (ASSERT(local_ok)) with the
trust clause reverted -- the test binds precisely to the fix. #1856 (stock
ancestor ACEs): the Windows ancestor-ACL guards already pass; no defect
reproduced, no code change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Inside a 1:1 user namespace (podman --userns=keep-id, unshare -U
--map-current-user), the root-owned ancestors /, /tmp appear owned by the
kernel overflow uid; the daemon's ancestor-ownership walk refused every
overflow-owned ancestor, making the daemon unusable in that legitimate setup.
Tolerate an overflow-owned ancestor ONLY when /proc/self/uid_map is a
single-uid map whose sole inside id is our euid AND /proc/sys/kernel/overflowuid
is readable (never hardcoded; unreadable => no tolerance). No environment
escape hatch. Group-writable overflow ancestor refused (host group unknown);
world-writable overflow tolerated only with the sticky bit (the /tmp shape).
The created/final/snapshot gates stay strict == geteuid(), so a host-squatted
overflow-owned private directory is still refused; the tolerance only widens
which ancestors the walk traverses. macOS byte-identical (overflow=-1).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_remove_indexes unlinked each *.db and *.db.tmp but never the SQLite
sidecars (<db>-wal/-shm/-journal), so `cbm update`, `cbm uninstall`, and
`install --reset-indexes` (the four callers of the single deletion
chokepoint) left orphan -wal/-shm files behind. Call the existing
wide-path-safe cbm_remove_db_sidecars() helper on both the live and staged
db paths after the .db unlink; it is idempotent and ENOENT-tolerant, so an
orphan -wal that outlived its .db is still cleaned. Sidecars are not
indexes, so the removed-count is unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The #1828 fail-fast test asserted `elapsed_ms < 5000`, a wall-clock budget
that flaked on the ASan test-diag lane under CI contention while the fail-fast
mechanism itself engaged correctly: the whole daemon_bootstrap suite ran in
11 s, so the test never reached the 30 s slow-timeout path -- it only overran
the 5 s wall-clock budget. Per O9 a gate never asserts a transient timing
window, and a rerun is not a fix.
Drop the wall-clock measurement and rely on the deterministic mechanism the
test already observes: the client surfaces the recorded ENOSPC cause verbatim
("failed to start" + errno + path), the slow "active or starting" timeout
wording is absent, and exactly one daemon is spawned. That "failed to start"
message is produced only on the fast-fail break
(cbm_daemon_bootstrap_start_failure_format), never on the 30 s deadline path,
so these assertions bind the fast-fail contract without any timing.
Verified RED-on-revert: defeating the fast-fail detection makes the client
hang to the 30 s deadline and report "active or starting", tripping
ASSERT_FALSE(stale_wording). Full daemon_bootstrap suite green (28/28) under
the ASan/UBSan build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On Windows, install registers the install directory in the persistent
HKCU\Environment\Path, but uninstall never removed it — so every
install/uninstall cycle left its entry behind and the user PATH grew
without bound (a real test host reached ~8000 chars, emptying the
effective PATH and breaking tooling).
The reported non-ASCII "mojibake" is historical, not a current-code bug.
Both install paths already write the value through the UTF-16 registry
API: install.ps1 (shipped in each release archive) runs `cbm install`,
which calls RegSetValueExW(L"Path", ...) — git history shows no ANSI
(RegSetValueExA / SetEnvironmentVariableA) predecessor — and
setup-windows.ps1 writes via .NET [Environment]::SetEnvironmentVariable
(Unicode) with an ASCII-only install dir. The only path that handles a
non-ASCII directory is the wide-correct C path, so current code cannot
produce a garbled entry; the garbage the reporter saw was left by an
older binary. No code change is needed for the encoding itself.
The real defect is the missing uninstall-side removal. This adds
cli_remove_windows_user_path(), the counterpart to
cli_ensure_windows_user_path(): it opens the same key, walks the
';'-separated segments, and drops exactly the ones equal to the install
dir using the same case- and trailing-separator-insensitive comparison
the install `present` scan uses, so it removes precisely what install
would have de-duplicated. Every other segment is preserved
byte-for-byte, the value kind (REG_SZ / REG_EXPAND_SZ) is preserved, and
a registry error is a warning rather than a hard failure so a user is
never blocked from finishing an uninstall over a cosmetic PATH edit.
Removing the entry on uninstall also stops accumulation on reinstall: a
garbled historical entry defeats the install de-dup and gets re-added,
so taking our own entry back out keeps the value bounded.
Testing:
- tests/test_cli.c: a hermetic Windows unit test drives the append and
remove logic against a GUID-scoped scratch key via the existing
CBM_TEST_WINDOWS_USER_PATH_RUN_ID seam, never the developer's live
HKCU\Environment\Path. A non-ASCII install dir doubles as a mojibake
guard (a broken UTF-8->UTF-16 round-trip would fail the match and leak
a duplicate). It asserts append, idempotent reinstall, exact removal
from the tail and the middle, idempotent removal, and byte-for-byte
survival of unrelated entries.
- tests/test_smoke_fixture_contract.sh: a macOS-runnable source contract
that fails when the removal is absent and passes with it, proving the
wiring binds (verified RED-on-revert).
The Windows-only registry code is #ifdef _WIN32; its compile and the
behavioral unit test are verified by CI's Windows leg (the local Windows
VM is down). Pre-existing mojibaked entries from old binaries are a
separate one-time cleanup, out of scope here, and setup-windows.ps1's
own lack of an uninstall mode is noted as a possible follow-up.
Fixes#2117
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
install wrote only part of the OpenHands config. Register the server globally
under ~/.openhands/settings.json -> mcp_config {transport:stdio, command, enabled}
(owned-entry, unrelated content preserved), and add "codebase-memory-mcp" to
each ~/.openhands/agent-profiles/<p>.json mcp_server_refs array (null -> [it],
existing -> appended once). uninstall removes exactly what it added.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
auto_index_limit was bypassed for a non-git root because the admission guard
counted `git ls-files`, which is 0 outside a repo, so the whole tree was
walked into tens of GB of RAM. Bound the check by the discovery layer's
bounded count with the limit as the ceiling; a non-git root over the limit is
refused with the limit named, and git roots are unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Review of the previous head reproduced a check-then-use window in the
parent-chain walk: the link was judged by name (fstatat) and then opened
by name (openat), with nothing tying the two to the same entry. An
LD_PRELOAD shim that swaps the link for one pointing elsewhere the moment
the judgement returns made both earlier heads follow the swapped link,
including one owned by a foreign uid under root; the post-follow target
check only bounds where such a swap can land.
The link is now never opened by name after it has been judged. On Linux
an O_PATH|O_NOFOLLOW descriptor pins the inode, fstat on that descriptor
makes the ownership decision and readlinkat(fd, "") reads the target text
from the same inode; on other systems the link is stat'ed by name before
and after the readlinkat and both must be the same inode with the same
owner. The walk then opens the text from the parent directory, so a
relative text resolves against the link's own directory exactly as the
kernel resolves it, and the target check runs on what that opens. The
readlinkat form is used rather than /proc/self/fd so the walk does not
depend on /proc being mounted. What is still done by path is resolving
the text itself, which the target check bounds; the comment says so.
Test: relative, ../, absolute and chained link texts resolve from the
link's directory with the process CWD moved elsewhere. The race itself
is exercised out of tree with the harness from the review thread.
Refs #1722
Signed-off-by: TheStranjer <791672+TheStranjer@users.noreply.github.com>
Review of the previous head found the global rule unsound for one caller:
cbm_mkdir_p also creates <repo_path>/.codebase-memory inside the checkout
being indexed, and git creates symlinks owned by whoever cloned, so a
repository shipping `.codebase-memory -> ~/.ssh` would have been followed
by the artifact writer. "User-owned" says nothing about "user-intended"
once the path root is a checkout.
cbm_mkdir_p keeps the strict walk (root-owned links only). The new
cbm_mkdir_p_ex(path, mode, policy) with CBM_MKDIR_FOLLOW_OWNED also
follows a symlink owned by the invoking user, and only callers whose
paths are rooted in the user's own configuration opt in: the installer's
agent roots, skills, instruction files and hook directories, the
JSON-like config editor's parent creation, the cache root, <cache>/logs,
the UI config directory and the DB parent. The artifact directory inside
the checkout and the ~/.local/bin creations stay strict; the activation
transaction re-walks the latter with its own no-follow policy anyway.
What a follow lands on is now checked too: the opened target must be a
directory owned by root or the invoking user and not world-writable
unless sticky, so a link swapped between the fstatat and the openat
cannot steer the walk into a directory an attacker controls or one where
anyone can pre-plant entries. This applies to root-owned links as well;
macOS /private/tmp is root-owned and sticky and still passes. Refusal
restores the first openat's errno.
Tests: the plain walk refuses an own link that the opted-in walk
follows; targets that are world-writable without the sticky bit are
refused; as root, a foreign-owned target and a user-owned link are both
refused, each followed by restoring ownership and walking again; the
identical `.codebase-memory -> <user-owned dir>` link is refused by the
plain walk and followed once opted in; cbm_artifact_export refuses a
checked-in artifact-dir link and leaves its target empty; the installer
test gains the euid-0 leg with home, link and target owned by a non-root
uid. Under euid 0 the tests demote their links to a non-root owner, since
root-owned links are trusted by both walks.
Refs #1722
Signed-off-by: TheStranjer <791672+TheStranjer@users.noreply.github.com>
A daemon that died at listener publication (a full /tmp) took the
lifetime reservation, failed, and released it; the waiting client saw the
generation vanish, respawned it, and repeated that for its whole 30 s
deadline before reporting "CBM daemon is active or starting but could not
accept this client" -- the opposite of what happened, with no cause and
no path. MCP hosts then dropped the server silently (#1828).
A detached daemon has no stderr and no channel back to its launcher, so
the host now appends one start-failure record per failed start to
<cache>/logs/cbm-daemon.start-failures.log -- next to cbm-daemon.log,
through the same owner-only private-log open. Deliberately NOT the
runtime directory: that is exactly what a full /tmp cannot write, and a
new artifact there would entangle the stale-generation cleanup that
#1894 just hardened. The record carries the listener detail from the
previous commit (component, stage, errno, path) plus the runtime
directory and a wall-clock stamp, and is written while the lifetime
reservation is still held, so a client that watches the generation
vanish already finds the cause.
The client bootstrap loop reads that record (every 50 ms after its own
spawn, and unconditionally before any relaunch), accepts only records for
its endpoint not older than its spawn instant (2 s skew), and stops at
once:
error: CBM daemon failed to start: pending_publication failed with
ENOSPC (No space left on device) at /tmp/cbm-daemon-1000/cbm-<key>.sock.pending.tmp;
see ~/.cache/codebase-memory-mcp/logs/cbm-daemon.log
The read is an optional bootstrap op (unit contracts with fake ops are
untouched); the production op resolves the cache log directory. Every
other wait state -- a live or stopping generation, a mute holder, a
concurrent starter -- keeps its existing semantics and diagnostics.
Tests (daemon_bootstrap):
- daemon_bootstrap_fails_fast_when_daemon_dies_at_publication: the
production bootstrap (real cohort, probe, startup lock, handoff) with a
seam-only spawn override that forks a REAL daemon host whose record
writes fail with ENOSPC. RED before: 34.7 s and "active or starting";
GREEN after: FAILED in well under 5 s naming ENOSPC and the temp path,
one child spawned, daemon log carries errno=ENOSPC path=... .
- daemon_bootstrap_start_failure_record_round_trip: record/read/format
contract, a stale record and another endpoint's record are ignored.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The daemon handed every index worker aggregate/physical_job_limit (a
fixed quarter by default) regardless of how many jobs were running, so a
lone worker on a host like #1864 was capped at a quarter of the memory
the daemon itself was allowed — and with the budget now failing whole
(decision A) that fixed slice would have turned an idle host into a
guaranteed over_memory_budget. Decision 2 (#1997, #832): the slice is
decided at spawn time as aggregate / jobs active right then (the job
being spawned included), so a lone worker may use the whole aggregate
and concurrent jobs split it. The explicit CBM_MEM_BUDGET_MB override
keeps its precedence inside the worker (bf46b91f); the sub-byte-per-slot
guard is unchanged. The spawn line logs project, active_jobs,
aggregate_mb and slice_mb so a fail-whole verdict can be read against
the budget the worker really had.
Tests: sequenced admission proves first=aggregate, second=aggregate/2;
the racing 4-way admission test accepts any aggregate/1..4 slice (the
divisor is scheduling-dependent there); a scripted CLEAN worker
returning the over_memory_budget error response is passed through
verbatim on the first attempt — no recovery loop, no quarantine.
Refs #1997#832#1864
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
index_repository rendered every non-zero pipeline code as "Pipeline
failed. Check repo_path exists and contains source files", which sent
people debugging a path that was fine (#2020). Three outcomes now say
what actually happened:
- CBM_PIPELINE_ABORT_OVER_BUDGET (#1997, #832): status "error" with
reason "over_memory_budget", previous_index "preserved", the budget and
peak RSS in MB, and a hint naming CBM_MEM_BUDGET_MB / CBM_WORKERS /
excludes. The handler logs index.abort reason=over_memory_budget. The
CLI prints the same JSON and exits 1; a supervised worker still exits 0
with the response written, so the daemon passes it through verbatim
instead of entering skip-and-continue recovery.
- CBM_PIPELINE_ABORT_PRESERVE_DB: status "aborted_previous_preserved"
(retry; the previous index still serves).
- CBM_PIPELINE_PERSIST_FAILED: status "persist_failed" (disk/permissions).
The incremental test helper goes fail-closed on error responses so a
transient pre-publication abort surfaces once at the source instead of
cascading into dozens of downstream count failures.
Test: an in-process index over 64 files at a 1 MiB budget seam returns
the named error, leaves the project DB the same size on disk, and
index_status still reports the previous node count.
Refs #2020#1997#832
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A full /tmp made every daemon start die at pending publication, and the
only durable trace was `daemon.ipc.listen_failed stage=pending_publication`
-- no syscall, no errno, no path. The reporter needed hours (and a `df` on
the wrong mount) to find the cause (#1828).
Every listener failure now records the stage, the errno the failing step
reported, and the artifact path it was operating on, and the log line
carries all three:
daemon.ipc.listen_failed stage=pending_publication errno=ENOSPC
path=/tmp/cbm-daemon-1000/cbm-<key>.pending.tmp
The record publication helper notes which step refused (create, write,
link, unlink, sync, close, read-back) and on which artifact, so the
listener can name it. The bind failure keeps its errno but joins the same
shape (stage=socket_bind errno=... path=...) instead of a bare numeric
bind_errno. cbm_errno_name() maps the portable errno set to its symbol.
cbm_daemon_ipc_listen_failure_detail() exposes the same triple to the
daemon host for the fail-fast record that follows.
Test: daemon_ipc_listen_failure_names_errno_and_path forces the record
write to fail with ENOSPC through a narrow inert seam (the socket binds,
the file is created, only its data write is refused -- the reporter's
exact shape) and asserts the log line names the errno and the temp path,
and that the namespace is left clean.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The extraction memory gate was advisory: when a full back-pressure cycle
ended still over budget it latched "futile" and kept admitting files, so a
host that could not hold the graph ran on until the kernel killed the
worker (#1997, #832) and the supervisor then quarantined innocent files.
Now the worker that latches futility runs ONE confirmation cycle (collect
+ bounded naps). If resident memory drained, the gate re-arms as before.
If it is still over budget, the attempt fails whole: every worker stops
pulling, extract returns the new CBM_PIPELINE_ABORT_OVER_BUDGET, the
orchestrator discards the staging DB on that non-zero code like any other,
and the previously serving generation is left byte-identical. The verdict
is a plain return value so it travels as a clean-exit worker response,
never a kill. The abort flag is its own atomic, not the shared cancel
token, which belongs to client cancellation.
The abort/persist codes move to pipeline.h alongside the new one (callers
legitimately distinguish them); FORCE_FULL_REINDEX stays internal.
Tests: the futility test now asserts the new code (the confirmation adds
one cycle to its bound), and a new test proves fail-whole preserves the
live DB byte-for-byte, keeps its node count, and leaves no stage residue.
Refs #1997#832#1973#1864
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
After every publish the pipeline re-exports <root>/.codebase-memory/ (graph.db.zst + artifact.json) whenever an artifact already lives there: once a session indexed with persistence, or when the team committed it and every linked worktree checks it out. The watcher's dirty signature folded those files' (size, mtime) in, so each successful reindex rewrote them, the next poll saw a new dirty state, and the daemon re-triggered itself forever: index.supervisor.reap outcome=clean immediately followed by watcher.changed strategy=git, 100+ times in 15 minutes with index workers pinned (#1953). Nothing under .git moved and no source changed, which is why the reporters blamed their long hyphenated worktree branch names (the #1254 theory its own reporter retracted); the regression test builds that exact worktree and shows the branch is irrelevant.
Scope git status with the same :(exclude).codebase-memory pathspec the exporter's own clean-tree probe already uses for the same reason. Nothing under that directory is ever an index input (discovery skips it), so a change there can never require a reindex; the pathspec is CWD-relative, so a monorepo sub-package watch excludes its own artifact directory. Untracked and committed artifacts both looped before; a real edit and a real HEAD move are still detected.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Hermes `install` failed with `op=pre_llm_hook_install` whenever config.yaml
held a PLAIN scalar with a `>` or `|` anywhere after the value start —
`probe: kaomoji face >w< here` — while the identical string double-quoted,
or a mid-value `*`, installed fine. An agent-config failure aborts
activation, so the whole install exited 1 and user-PATH registration was
skipped although eleven other agents had configured cleanly. Bisected by
@ukind, key by key and persona by persona, down to two persona strings
(#1924, continuing #1631/#1656).
Root cause: yaml_sequence_line_has_unsupported — the document validator on
the mapping-sequence path that the hook upsert/remove uses — ran a second
scan after the shared yaml_range_has_unsupported_ex and returned
"unsupported" for `[`, `]`, `|` and `>` at ANY position of a plain scalar.
The mapping-body path (mcp_servers) never had that check, which is exactly
why the mcp op succeeded and only the hook op failed on the same file.
Fix: the positional rule the `*`/`&` fix (v0.10.4) and the quote fix
(#1631, v0.10.6) already use — an indicator counts only where a NODE begins
(range start, after a mapping colon, after a block-sequence dash).
Interior ones are text. `probe: >`, `probe: |`, `probe: >-` and
`probe: [a, b]` still begin a value and stay unsupported, byte-identically.
`[`/`]` get the same treatment on purpose: in block context they are
ordinary characters inside a plain scalar (`a: b[0]` parses as the string
`b[0]`), a value-start `[` still opens a flow sequence and stays refused
(the #1631 line: exact `[]` accepted, non-empty flow sequences unsupported),
and the mapping-body path already accepted them mid-value — leaving the
sequence scan stricter would keep the mcp-op-passes / hook-op-fails
asymmetry this issue is about. Known conservative residue, shared with the
`*` rule: a plain-scalar CONTINUATION line that itself starts with one of
these characters is still refused (nothing precedes it on its line).
Tests (tests/test_config_yaml_edit.c): three accept cases run both real
Hermes ops (mcp_servers + pre_llm_call) against a persona corpus with a
mid-value `>`, `|` and `[]` and require the original document to survive as
an untouched prefix; two pins keep value-start block scalars and a non-empty
flow sequence refused with the file byte-identical. RED on the unfixed
editor: 61 passed, 3 failed (`yaml_hermes_hook_upsert(&fixture) == -1`);
GREEN: 64 passed; RED again with the production change reverted.
config_yaml_edit 64 · yaml 73 · cli 306 — 443 passed, 0 failed (macOS,
ASan/UBSan test build).
Not in this change: the refusal still reaches the user as
`op=pre_llm_hook_install path=... (target: regular file, N bytes)` with no
reason. The editor collapses every one of its ~236 YAML_ERROR sites into
CBM_YAML_IDENTITY_EDIT_ERROR, so naming the refusing rule needs a detailed
variant with a typed failure out-parameter, the shape the Codex TOML hook
path already has (cbm_toml_codex_hook_failure_t +
record_agent_config_error_with_reason). Follow-up.
Fixes#1924.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
try_load_nested_gitignore() returned NULL as soon as a walk frame already
carried a local matcher, and that matcher was inherited by every child
frame. So only the SHALLOWEST nested .gitignore on a path was ever loaded;
any deeper one was silently dropped — an EMPTY storage/.gitignore was
enough to disable storage/dump/.gitignore's "*". On the standard Laravel
layout (storage/.gitignore above storage/<dir>/.gitignore) that meant
thousands of git-ignored cache/debugbar dumps were discovered and indexed,
which on the reporter's app grew to 25,638 discovered files against 6,682
git-visible ones and ended in an OOM kill (#1973, discovery half; also the
§2 root cause of #530).
Discovery now keeps a root->leaf chain of .gitignore links per directory
frame: every directory below the root loads its own .gitignore, a frame
without one shares its parent's link, and a path is judged by walking the
chain leaf->root with git's precedence — the deepest file with an opinion
wins (so a deeper negation re-includes what an ancestor ignores), and inside
one file the last matching pattern wins. The repository root's .gitignore
(merged with info/exclude) is the chain head, so the same precedence holds
between root and nested files. Cost is one match per .gitignore on the
path — O(depth) per entry, no rescans; the walk frame shrinks by the 4 KB
local-prefix buffer it no longer needs.
Tests pin the reporter's layout (empty ancestor + deeper "*"), a deeper
negation overriding an ancestor pattern, and three nesting levels with an
empty middle file. Measured on a Laravel app (koel) seeded with typical
compiled-view/cache artifacts: 2,885 discovered files with 450 git-ignored
-> 2,435 with 0; clean clones of linux, dotnet/runtime, koel and redis
discover byte-identical file sets before and after.
Fixes the discovery half of #1973.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/setup-windows.ps1 has 326 lines and its parse-coverage report read
"113-113,113-113,245-327" — the same line named twice, and an end line that
does not exist. Two separate faults, both in cbm_error_regions_push.
Past-EOF end line. A tree-sitter node that ends at column 0 stopped right
after the previous line's newline, so it holds no text on the row it points
at. Adding 1 to that row named a line past the end of the file whenever the
region ran to EOF. The node here is start=(244,2) end=(326,0). Clamp the end
to the row above when the end column is 0 and the node spans more than one
row.
Duplicate range. Line 113 carries two separate ERROR nodes, at columns 25-29
and 31-32, and each pushed its own range. A line range says nothing new the
second time. Drop a range that exactly repeats the one already open. The
drop runs BEFORE the cap check, so a repeat is never miscounted as a range
the cap threw away.
Only an EXACT repeat is dropped, never a range that merely overlaps. Each
range is judged separately afterwards by cbm_region_is_recovered, which asks
whether definitions starting inside that range cover it. Two ranges holding
the same numbers always get the same verdict, so dropping the repeat changes
nothing. Two different ranges do not. Merging 3-3 into 2-3 hands the wider
range's covering definition to an error that definition does not explain, and
a real parse failure then vanishes from the report.
That is not hypothetical. An earlier version of this commit merged on overlap
and broke perl_malformed_source_remains_partial_issue1838, the test added
with the Perl grammar refresh in 17b5a432. The malformed fixture produces two
ERROR nodes, at lines 2-3 and 3-3. Merged, the 2-3 range looks fully covered
by before_error and is removed, so parse_incomplete comes back false on a file
that plainly does not parse. That test now pins this boundary.
The real file reports "113-113,245-326".
Two tests, both proved RED first with the exact expected text:
coverage_repeated_error_line_reports_one_range_issue963 "2-2,2-2" != "2-2"
coverage_range_never_ends_past_the_last_line_issue963 "1-5" != "1-4"
Suites run on this change: parse_coverage 34, extraction 325, pipeline 264,
mcp 246, index_resilience 7 — all passing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
The markerless stale-rendezvous recovery test reset the activation runtime
parent to NULL instead of restoring g_cli_suite_runtime_parent, so every
later cli test ran the activation probe against the developer's real
/tmp/cbm-daemon-<uid>. Restore the suite value per the contract at
tests/test_cli.c:61-65 and the sibling at :1310.
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
phase1_scan_functions() in pass_semantic_edges.c sorts the collected
Function/Method node pointers so the func order is deterministic. When
the graph has no Function or Method node at all -- a struct-only or
config-only project -- node_ptrs is never allocated and the call becomes
qsort(NULL, 0, ...). The func_count <= 0 early-out lives in phase 1b,
which runs only after phase 1 has returned, so it never guards this call.
A NULL qsort base is undefined behaviour. glibc declares the parameter
nonnull, so UBSan on the Linux leg reports "null pointer passed as
argument 1, which is declared to never be null" at the call. The macOS
SDK carries no such attribute, so neither the default sanitizer build nor
the Homebrew LLVM diag lane can flag it there. The existing
pipeline_docstring_go_class test already reaches the call on Linux; the
harness never gated on recoverable UBSan reports, so it went unnoticed.
Guard the sort on func_count > 0. The new test
pipeline_semantic_edges_no_functions indexes a Go struct-only fixture
through the full pipeline and pins the fixture to what it claims
(Struct >= 1, Function = Method = 0). On the Linux leg with
UBSAN_OPTIONS=halt_on_error=1 the pipeline suite aborts at the qsort
call before the fix and completes with it.
Distilled from #1810 with co-author credit.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-authored-by: Yusuf S. Demirci <ysf-samet@outlook.com>
Martin Vogel