The macos-15-intel leg killed the cli suite at its 900s wall clock while
reporting 7537 passed, 0 failed. Two separate things were wrong.
The suite was misclassified. cli spends 497s of the 900s default on macos-14,
the FASTEST macOS runner, while macos-15-intel in the same matrix is 2.4-3.6x
slower on comparable suites (daemon_runtime 842s vs 349s, stack_overflow_b
277s vs 76s). 497s at that ratio cannot fit. daemon_runtime, at 842s on that
same runner, passes only because it was already in SLOW_SUITES. cli belongs
there too -- it is deterministic and simply large, which is the only thing that
list is allowed to mean.
The drain test was also waiting on a clock instead of on the system. Profiling
the suite per test (315 tests, 300s wall / 166s cpu) put
cli_install_into_host_namespace_still_drains_host_cohort at 42s wall for 19s of
cpu -- 23s idle, the largest single idle block, where the next worst was 11s.
The existing cli_install_force_quiesces_active_cohort_before_replacing_binary
drains a cohort too and idles 0.5s, which is what pointed at the difference.
The cost was the host_serving probe. Its generous budget exists for the
POSITIVE question -- is this daemon still up? -- where a slow reply on a loaded
runner must not be misread as drained; that fixed a real flake and is kept.
Asked in the negative it inverts: no reply is coming, so the full 15s is spent
establishing silence and ASSERT_FALSE is decided by the timeout expiring rather
than by the daemon's behaviour. The drain is already proven positively in that
test -- install returns 0 only after the activation completed, and the host
child exits CLI_SCOPE_HOST_DRAINED -- so the probe only has to confirm it.
Also shortens the drained child's teardown budget, which retried service_free /
lease_release / manager_free against 10s on a path where the activation had
already torn the service down. Behaviour at the deadline is unchanged; it is
reached sooner when nothing is wedged. Measured at only -3s on its own, kept
because it is correct and free.
Local, same machine, ASan+UBSan: the drain test 42.0s -> 25.8s (-38%), the
suite 300s -> 279s, 315 passed before and after -- no test removed, no
assertion weakened. PR #2188, which runs the same leg without these two tests,
passed macos-15-intel in 37m34s, so the suite was not already over budget on
main.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
sha256_transform held its 64-word message schedule as a 256-byte local.
That is large enough for ASan's use-after-return fake stack, so an
instrumented build heap-allocated it through __asan_stack_malloc on every
64-byte block -- millions of allocations to hash one large file. Under the
sanitized CI settings (detect_stack_use_after_return=1) fingerprinting a
freshly installed binary therefore took minutes, and the cli suite blew its
900s wall-clock budget and was killed as hung on the Linux arm64 leg. The
suite passes on macOS, where the fake stack is not engaged the same way,
which is why this only ever surfaced on one leg.
Move the schedule into cbm_sha256_ctx so it is allocated once per hash
instead of once per block. The computed values and the resulting digest are
identical; no sanitizer option is relaxed and use-after-return detection
stays enabled everywhere.
Verified in the Linux arm64 sanitized container under the exact CI
ASAN_OPTIONS (detect_stack_use_after_return=1:strict_string_checks=1:
detect_stack_use_after_scope=1): cli 316 passed in 356s, previously killed
at the 900s budget; daemon_version 10 passed, covering the build-fingerprint
digests that would break first on a wrong hash.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The per-file tree-sitter parse was bounded by a 5s WALL-CLOCK budget. Under CI
contention a worker thread can be descheduled long enough for 5s of wall time
to elapse against near-zero CPU, so a small, perfectly parseable file was
abandoned with a false "parse timeout" and yielded ZERO defs. When that file
defined a symbol other files resolve against (e.g. celery.app.Celery.send_task
backing an ASYNC_CALLS edge, or a userpb gRPC client backing GRPC_CALLS), the
cross-file service edge then failed to classify -- surfacing as the flaky
convergence_probe / edge_types_probe service-edge assertions (count 0), a
verdict decided by wall-clock scheduling rather than the input (violates O9,
"no timeout decides a test").
Gate the parse on per-thread CPU time instead, with a generous wall ceiling as
a backstop:
- New portable helper cbm_thread_cpu_time_ns() (compat.c/.h): POSIX/macOS
clock_gettime(CLOCK_THREAD_CPUTIME_ID), Windows GetThreadTimes (kernel+user);
returns 0 on clock failure, degrading safely to the wall ceiling. It is NOT
cbm_clock_gettime, which is wall on every platform (QueryPerformanceCounter
on Windows).
- The progress callback's payload becomes {cpu_deadline_ns, wall_ceiling_ns};
it trips when this thread's CPU time passes the budget OR wall time passes
t0 + budget * 12 (~60s for the 5s budget). Descheduling burns wall, not CPU,
so a starved-but-parseable file survives; a CPU-spinning parse still trips
the CPU deadline; a genuinely stuck parse still trips the wall ceiling
(bounded, no hang).
RED-repro (deterministic, CBM_ENABLE_TEST_SEAMS only): a thread-local wall-clock
offset armed by CBM_TEST_WALL_STALL_ON emulates a descheduled worker (wall
reads budget + 1s ahead while CPU is untouched -- above the old 1x wall budget,
below the ceiling). async_calls_celery_wall_stall_seam arms it for the Celery
DEFINITION file and asserts ASYNC_CALLS >= 1. With the fix the parse completes
and the edge resolves; with the budget reverted to wall-only + a 1x ceiling the
def file is dropped and the edge count is 0.
Verified (macOS, sanitized): edge_types_probe + convergence_probe 107 passed /
0 failed with the fix; 58 passed / 1 failed on revert (only the seam test, at
the ASYNC_CALLS assertion); 107 restored. Regression smoke (extraction
parse_coverage pipeline lang_contract grammar_regression) 698 passed / 0
failed. lint-ci clean (cppcheck + clang-format Homebrew LLVM + NOLINT); the
Windows GetThreadTimes path is compile-checked by CI (not runnable on macOS).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The autoindex over-limit skip warning logged `root=` with the raw
srv->session_root, which on Windows carries backslashes. The #713 guard test
autoindex_limit_guards_non_git_root_issue713 asserts the warning names the
skipped root via strstr(root, "/repo"), so it failed on the Windows CLANG64
(x86_64) leg where the path is `...\repo`. Normalize the logged root to
forward slashes with cbm_normalize_path_sep() (a no-op on POSIX), matching
cbm's forward-slash path convention used everywhere else in the graph and MCP
output; the diagnostic is now identical across platforms. Production-side fix,
not a test relaxation.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A burst of one-shot clients (e.g. six parallel `cli list_projects`) shares one
ephemeral daemon generation and races to connect. The generation retired the
instant its last committed client disconnected, even when another one-shot's
connection was already accepted and still mid-HELLO -- stranding that racer on
a STOPPING daemon and forcing a full cold respawn (up to the 12s daemon-claim
handoff each). Under load the retire/respawn churn could exhaust a client's
30s bootstrap budget, surfacing as the Windows guard's "cold-storm client
failed (racing daemon spawn)" regression and, on POSIX, a startup that never
reached "serving".
When the last committed client of an ephemeral, non-permanent generation
disconnects, linger a bounded window if a peer connection is already accepted
but has not yet passed HELLO admission (in_use, connected, not admitted, not
disconnecting -- runtime_has_pending_hello_peer_locked). The accept loop's
reconcile retires the generation once that racer drains without committing or
the bounded window (RUNTIME_EPHEMERAL_LINGER_MS, seam-overridable) elapses; a
racer that does commit clears the linger. With no such racer -- the common
single-client case -- the generation retires immediately, unchanged.
The linger deliberately covers ONLY a pre-HELLO racer, never an already-
admitted provisional session (HELLO done, application session opening): the
contract that a provisional coordinator client cannot keep a retiring
generation alive is preserved (daemon_runtime_final_disconnect_rejects_blocked_
provisional_session). The daemon cannot count cross-process cohort peers
(advisory locks expose presence, not a holder count, and Windows LockFileEx has
no non-owning probe), so this uses the one peer signal it has locally: accepted
connections not yet admitted. The window is bounded, so a racer that never
commits cannot wedge the generation (rules out the unbounded idle hang).
Coordinator gains a transient `linger` hold (distinct from `permanent`) so its
own last-client self-transition to STOPPING is suppressed while the racer is
outstanding; clearing it while idle retires the coordinator at once.
Verified (macOS, sanitized build): daemon suites 224 passed / 0 failed / 1
skipped (the skip is the pre-existing Linux-only userns case). New tests pin
both directions -- daemon_runtime_ephemeral_lingers_while_peer_mid_hello
(a raw accepted-but-unHELLO'd peer keeps the generation RUNNING after the
committed client leaves) and daemon_runtime_ephemeral_linger_retires_within_bound
(the bounded backstop retires even while the peer is still accepted). RED on
revert: neutering the gate flips ASSERT(lingered) at test_daemon_runtime.c and
nothing else. clang-format (Homebrew LLVM) + cppcheck + NOLINT clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Inside a 1:1 user namespace (podman --userns=keep-id, unshare -U
--map-current-user), the root-owned ancestors /, /tmp appear owned by the
kernel overflow uid; the daemon's ancestor-ownership walk refused every
overflow-owned ancestor, making the daemon unusable in that legitimate setup.
Tolerate an overflow-owned ancestor ONLY when /proc/self/uid_map is a
single-uid map whose sole inside id is our euid AND /proc/sys/kernel/overflowuid
is readable (never hardcoded; unreadable => no tolerance). No environment
escape hatch. Group-writable overflow ancestor refused (host group unknown);
world-writable overflow tolerated only with the sticky bit (the /tmp shape).
The created/final/snapshot gates stay strict == geteuid(), so a host-squatted
overflow-owned private directory is still refused; the tolerance only widens
which ancestors the walk traverses. macOS byte-identical (overflow=-1).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
apt.llvm.org rotates its per-major noble snapshot channels; the
llvm-toolchain-noble-22 channel transiently emptied, breaking every lane that
installs clang from it. #2124 already moved Dockerfile.msan (clang-22 ->
clang-21) for the same reason. The two GitHub-hosted lanes that install from
apt.llvm.org were still on 22 and would break identically on the next
rotation: _test.yml test-diag (ASan+UBSan) and _lint.yml lint-mem
(clang-analyzer, clang-tidy). Repin both to clang-21 / clang-tidy-21 /
llvm-toolchain-noble-21 (the channel #2124 verified apt.llvm.org serves on
amd64+arm64). The macOS LSan lane's Homebrew llvm@22 pin is intentional and
unrelated to apt churn (left unchanged); the clang-format-20 CI-lint lane is
untouched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The earlier leak-fix commit was formatted with Homebrew clang-format v22, which spaces the trailing comment on cbm_mem_collect() differently than CI's clang-format-20 (AlignTrailingComments). CI's lint / lint leg failed on the resulting drift. Reformat only that line with the exact CI binary.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The test-msan lane was red repo-wide: the Dockerfile.msan image build could
no longer install clang-22 from apt.llvm.org's noble-22 channel, failing with
"E: Unable to locate package clang-22 / libclang-rt-22-dev / llvm-22" after
the 5 apt retries, so the image never built and every PR's required test-msan
check could not go green (#2123).
apt.llvm.org keeps only the last few majors for noble and rotates newer ones
in and out; the -22 channel's package index was empty when CI fetched it (the
channel has since flapped back, confirming the transience). Rather than stay on
that volatile channel, repin to clang-21 - the highest well-established stable
noble channel apt.llvm.org currently serves (verified installable on both
amd64 and arm64):
- apt source noble-22 -> noble-21, clang-22/libclang-rt-22-dev/llvm-22 ->
clang-21/libclang-rt-21-dev/llvm-21, and the clang/clang++ symlinks
- MSan runtimes clone llvmorg-22.1.0 -> llvmorg-21.1.8 (matches the apt
clang 21.1.8, keeping the libc++/libc++abi/libunwind build at the
compiler's major, as the layer requires)
- MSAN_SYMBOLIZER_PATH /usr/lib/llvm-22 -> /usr/lib/llvm-21
Verified in an ubuntu:noble container that clang-21/libclang-rt-21-dev/llvm-21
install cleanly, the clang/clang++ symlinks resolve, libclang_rt.msan-*.a and
llvm-symbolizer are present, and the full image builds via Colima.
Infra only; the MSan lane's other files (docker-compose.yml, scripts/msan.sh)
use the version-agnostic clang/clang++ symlinks and need no change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_kind_in_set lazily builds a THREAD-LOCAL node-type bitset cache during
extraction. That cache must be freed before a worker thread exits, or the
calloc'd bitsets are orphaned when the thread's TLS is torn down and
LeakSanitizer reports them at process exit. Parallel workers already free it
(pass_parallel.c:993) and the main thread frees it at process teardown; the
IN-PROCESS sequential auto-index runs extraction on autoindex_thread — a
short-lived pthread joined by cbm_mcp_server_free — and was the one gap: it
built the cache but never freed it (360 bytes in 9 allocations leaked, all from
ks_build).
This is a PRE-EXISTING latent leak: the extraction path and autoindex_thread are
byte-identical to main. It is newly EXPOSED by #713's new
autoindex_limit_admits_non_git_root_under_limit test, the first LSan-observed
path that runs an in-process auto-index and joins the worker thread in-process —
so the fix ships with the test that exposes it.
Free the calling thread's bitset cache after cbm_pipeline_free(p), mirroring the
established per-worker-thread teardown. Zero behavior change to the CLI/main
path. macOS LSan lane, mcp suite: 360 B / 9 allocations -> 0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The four namespace-scope tests fork a real host daemon and, in the parent,
waited on it with two UNBOUNDED calls: read() for the child's one-byte
readiness signal in cli_scope_fixture_start, and waitpid(host, 0) in
cli_scope_fixture_finish. If the forked child stalls before it can signal —
a fork-time sanitizer allocator stall is the classic cause, the same hazard
the earlier posix_spawn work addressed — the parent blocks forever and the
WHOLE cli suite hangs to the CI wall-clock kill (the observed rc=124 at 900 s
on the ubuntu-24.04-arm gcc shard, which passed on every other #2115 shard).
Attribution: the production drain path is already bounded (DRAIN 15 s, CONTROL
2 s) and a foreign-namespace install takes the non-blocking scope probe and
skips draining entirely, so this is not a production deadlock — it is a
test-fixture determinism gap, amplified by ASan on a loaded arm runner.
Fix, test-only:
- cli_scope_wait_ready(): poll the ready pipe to a generous, bounded deadline
(90 s, comfortably past the child's own 45 s cohort-admission deadline plus
service start), so a wedged child yields a clean ASSERT_TRUE(ready) failure
instead of an unbounded read().
- cli_scope_reap_host(): reap with WNOHANG to a bound (60 s), then SIGKILL and
reap, so a wedged child is force-killed rather than hanging finish(); a
killed child leaves host_exit == -1, which fails the caller's assertion
cleanly. This also prevents a stuck child from holding cohort locks that
would poison later tests.
- cli_scope_host_serving(): raise the status probe deadline from 5 s to 15 s so
a slow-but-alive daemon on a loaded runner is not misread as "drained" (the
flaky ASSERT(host_serving) failure this fixture showed); it still fails
cleanly when the daemon is genuinely gone or reports stopping.
None of these decides a healthy test — the child signals in well under a
second on a healthy runner; the bounds only convert an otherwise unbounded
hang into a deterministic pass/fail.
The scoping contract still binds: with cli_activation_resolve_scope forced to
always return ACTIVE, the three "must-not-drain" tests go RED at
ASSERT(host_serving) and the host-namespace "must-drain" test stays GREEN.
cli suite: 313 passed locally (macOS, twice); 30-round pre-fix and 8-round
post-fix stress of the four tests were clean; a fault-injected never-ready
child fails all four cleanly in ~37 s with no suite hang.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
CI's clang-format-20 flagged continuation-line alignment in the new
OpenHands MCP install/uninstall helpers (cbm_build_openhands_mcp_entry,
cbm_upsert_openhands_settings_mcp, cbm_remove_openhands_settings_mcp,
openhands_update_profile_refs). Reformat only those regions with
clang-format-20 to satisfy the lint gate; whitespace-only, no behaviour
change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On Windows, install registers the install directory in the persistent
HKCU\Environment\Path, but uninstall never removed it — so every
install/uninstall cycle left its entry behind and the user PATH grew
without bound (a real test host reached ~8000 chars, emptying the
effective PATH and breaking tooling).
The reported non-ASCII "mojibake" is historical, not a current-code bug.
Both install paths already write the value through the UTF-16 registry
API: install.ps1 (shipped in each release archive) runs `cbm install`,
which calls RegSetValueExW(L"Path", ...) — git history shows no ANSI
(RegSetValueExA / SetEnvironmentVariableA) predecessor — and
setup-windows.ps1 writes via .NET [Environment]::SetEnvironmentVariable
(Unicode) with an ASCII-only install dir. The only path that handles a
non-ASCII directory is the wide-correct C path, so current code cannot
produce a garbled entry; the garbage the reporter saw was left by an
older binary. No code change is needed for the encoding itself.
The real defect is the missing uninstall-side removal. This adds
cli_remove_windows_user_path(), the counterpart to
cli_ensure_windows_user_path(): it opens the same key, walks the
';'-separated segments, and drops exactly the ones equal to the install
dir using the same case- and trailing-separator-insensitive comparison
the install `present` scan uses, so it removes precisely what install
would have de-duplicated. Every other segment is preserved
byte-for-byte, the value kind (REG_SZ / REG_EXPAND_SZ) is preserved, and
a registry error is a warning rather than a hard failure so a user is
never blocked from finishing an uninstall over a cosmetic PATH edit.
Removing the entry on uninstall also stops accumulation on reinstall: a
garbled historical entry defeats the install de-dup and gets re-added,
so taking our own entry back out keeps the value bounded.
Testing:
- tests/test_cli.c: a hermetic Windows unit test drives the append and
remove logic against a GUID-scoped scratch key via the existing
CBM_TEST_WINDOWS_USER_PATH_RUN_ID seam, never the developer's live
HKCU\Environment\Path. A non-ASCII install dir doubles as a mojibake
guard (a broken UTF-8->UTF-16 round-trip would fail the match and leak
a duplicate). It asserts append, idempotent reinstall, exact removal
from the tail and the middle, idempotent removal, and byte-for-byte
survival of unrelated entries.
- tests/test_smoke_fixture_contract.sh: a macOS-runnable source contract
that fails when the removal is absent and passes with it, proving the
wiring binds (verified RED-on-revert).
The Windows-only registry code is #ifdef _WIN32; its compile and the
behavioral unit test are verified by CI's Windows leg (the local Windows
VM is down). Pre-existing mojibaked entries from old binaries are a
separate one-time cleanup, out of scope here, and setup-windows.ps1's
own lack of an uninstall mode is noted as a possible follow-up.
Fixes#2117
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
install wrote only part of the OpenHands config. Register the server globally
under ~/.openhands/settings.json -> mcp_config {transport:stdio, command, enabled}
(owned-entry, unrelated content preserved), and add "codebase-memory-mcp" to
each ~/.openhands/agent-profiles/<p>.json mcp_server_refs array (null -> [it],
existing -> appended once). uninstall removes exactly what it added.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
auto_index_limit was bypassed for a non-git root because the admission guard
counted `git ls-files`, which is 0 outside a repo, so the whole tree was
walked into tens of GB of RAM. Bound the check by the discovery layer's
bounded count with the limit as the ceiling; a non-git root over the limit is
refused with the limit named, and git roots are unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The rendezvous directory the CLI's activation guard talks to is per OS
account (service.h), not per HOME/CBM_CACHE_DIR: every daemon owned by
one user meets at the same endpoint, and only the cache fingerprint
carried in the cohort identity separates one HOME/CBM_CACHE_DIR
namespace from another. `cli_activation_production_reserve` used
`cbm_version_cohort_reserve_for_mutation`, which treats any lifetime-
lock holder at that shared endpoint as the daemon this activation must
drain, with no cache-fingerprint awareness at all. An `install` run
against a second HOME (a sandbox, a second profile, `--skip-binary`
into an alternate CBM_CACHE_DIR) therefore reached the LIVE host
daemon at the shared endpoint and drained it — disconnecting every one
of the host's real MCP clients — even though nothing the sandboxed
install touched belonged to the host's cache namespace.
The fix adds `cli_activation_resolve_scope`, a non-blocking probe via
the already cache-fingerprint-aware `cbm_version_cohort_acquire`, run
before the blind drain path. It distinguishes three cases from the
resolved target of THIS activation:
- nothing to quiesce at all: a `--skip-binary` install that also
resets no index publishes nothing, so no session anywhere needs to
stop (`quiesce_required` on `cli_activation_guard_scoped`, plumbed
from `cbm_cmd_install`'s own binary/index-reset decision);
- the active cohort's cache fingerprint doesn't match this
activation's target cache: a foreign namespace, left untouched;
- same cache fingerprint (or no active cohort): this activation
really does replace that daemon, so the existing drain/quiesce
path runs unchanged, including the barrier wait when another
activation already holds maintenance.
Verified against a real forked daemon (a live runtime service with one
committed client) standing in for the host: an install into a foreign
HOME/CBM_CACHE_DIR (`--skip-binary` and a full binary install) leaves
that daemon serving with its client still attached, while an install
that targets the host's own namespace still drains it and the
activation audit still names the client it disconnected.
New/updated tests in tests/test_cli.c (cli suite):
- cli_install_skip_binary_into_foreign_home_never_drains_host_cohort
- cli_install_binary_into_foreign_home_never_drains_host_cohort
- cli_install_into_host_namespace_still_drains_host_cohort (regression)
- cli_install_skip_binary_unchanged_in_host_namespace_quiesces_nothing
- cli_activation_quiesce_does_not_wait_on_bootstrap_startup updated to
carry a real cache fingerprint for its participant, now that scope
is fingerprint-gated
src/mcp/index_supervisor.{c,h}: CBM_CLI_ENABLE_TEST_API-gated setter
so the fixture's forked daemon can carry the same build fingerprint
the guard captures from the supervisor.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Review of the previous head reproduced a check-then-use window in the
parent-chain walk: the link was judged by name (fstatat) and then opened
by name (openat), with nothing tying the two to the same entry. An
LD_PRELOAD shim that swaps the link for one pointing elsewhere the moment
the judgement returns made both earlier heads follow the swapped link,
including one owned by a foreign uid under root; the post-follow target
check only bounds where such a swap can land.
The link is now never opened by name after it has been judged. On Linux
an O_PATH|O_NOFOLLOW descriptor pins the inode, fstat on that descriptor
makes the ownership decision and readlinkat(fd, "") reads the target text
from the same inode; on other systems the link is stat'ed by name before
and after the readlinkat and both must be the same inode with the same
owner. The walk then opens the text from the parent directory, so a
relative text resolves against the link's own directory exactly as the
kernel resolves it, and the target check runs on what that opens. The
readlinkat form is used rather than /proc/self/fd so the walk does not
depend on /proc being mounted. What is still done by path is resolving
the text itself, which the target check bounds; the comment says so.
Test: relative, ../, absolute and chained link texts resolve from the
link's directory with the process CWD moved elsewhere. The race itself
is exercised out of tree with the harness from the review thread.
Refs #1722
Signed-off-by: TheStranjer <791672+TheStranjer@users.noreply.github.com>
Review of the previous head found the global rule unsound for one caller:
cbm_mkdir_p also creates <repo_path>/.codebase-memory inside the checkout
being indexed, and git creates symlinks owned by whoever cloned, so a
repository shipping `.codebase-memory -> ~/.ssh` would have been followed
by the artifact writer. "User-owned" says nothing about "user-intended"
once the path root is a checkout.
cbm_mkdir_p keeps the strict walk (root-owned links only). The new
cbm_mkdir_p_ex(path, mode, policy) with CBM_MKDIR_FOLLOW_OWNED also
follows a symlink owned by the invoking user, and only callers whose
paths are rooted in the user's own configuration opt in: the installer's
agent roots, skills, instruction files and hook directories, the
JSON-like config editor's parent creation, the cache root, <cache>/logs,
the UI config directory and the DB parent. The artifact directory inside
the checkout and the ~/.local/bin creations stay strict; the activation
transaction re-walks the latter with its own no-follow policy anyway.
What a follow lands on is now checked too: the opened target must be a
directory owned by root or the invoking user and not world-writable
unless sticky, so a link swapped between the fstatat and the openat
cannot steer the walk into a directory an attacker controls or one where
anyone can pre-plant entries. This applies to root-owned links as well;
macOS /private/tmp is root-owned and sticky and still passes. Refusal
restores the first openat's errno.
Tests: the plain walk refuses an own link that the opted-in walk
follows; targets that are world-writable without the sticky bit are
refused; as root, a foreign-owned target and a user-owned link are both
refused, each followed by restoring ownership and walking again; the
identical `.codebase-memory -> <user-owned dir>` link is refused by the
plain walk and followed once opted in; cbm_artifact_export refuses a
checked-in artifact-dir link and leaves its target empty; the installer
test gains the euid-0 leg with home, link and target owned by a non-root
uid. Under euid 0 the tests demote their links to a non-root owner, since
root-owned links are trusted by both walks.
Refs #1722
Signed-off-by: TheStranjer <791672+TheStranjer@users.noreply.github.com>
The daemon handed every index worker aggregate/physical_job_limit (a
fixed quarter by default) regardless of how many jobs were running, so a
lone worker on a host like #1864 was capped at a quarter of the memory
the daemon itself was allowed — and with the budget now failing whole
(decision A) that fixed slice would have turned an idle host into a
guaranteed over_memory_budget. Decision 2 (#1997, #832): the slice is
decided at spawn time as aggregate / jobs active right then (the job
being spawned included), so a lone worker may use the whole aggregate
and concurrent jobs split it. The explicit CBM_MEM_BUDGET_MB override
keeps its precedence inside the worker (bf46b91f); the sub-byte-per-slot
guard is unchanged. The spawn line logs project, active_jobs,
aggregate_mb and slice_mb so a fail-whole verdict can be read against
the budget the worker really had.
Tests: sequenced admission proves first=aggregate, second=aggregate/2;
the racing 4-way admission test accepts any aggregate/1..4 slice (the
divisor is scheduling-dependent there); a scripted CLEAN worker
returning the over_memory_budget error response is passed through
verbatim on the first attempt — no recovery loop, no quarantine.
Refs #1997#832#1864
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
index_repository rendered every non-zero pipeline code as "Pipeline
failed. Check repo_path exists and contains source files", which sent
people debugging a path that was fine (#2020). Three outcomes now say
what actually happened:
- CBM_PIPELINE_ABORT_OVER_BUDGET (#1997, #832): status "error" with
reason "over_memory_budget", previous_index "preserved", the budget and
peak RSS in MB, and a hint naming CBM_MEM_BUDGET_MB / CBM_WORKERS /
excludes. The handler logs index.abort reason=over_memory_budget. The
CLI prints the same JSON and exits 1; a supervised worker still exits 0
with the response written, so the daemon passes it through verbatim
instead of entering skip-and-continue recovery.
- CBM_PIPELINE_ABORT_PRESERVE_DB: status "aborted_previous_preserved"
(retry; the previous index still serves).
- CBM_PIPELINE_PERSIST_FAILED: status "persist_failed" (disk/permissions).
The incremental test helper goes fail-closed on error responses so a
transient pre-publication abort surfaces once at the source instead of
cascading into dozens of downstream count failures.
Test: an in-process index over 64 files at a 1 MiB budget seam returns
the named error, leaves the project DB the same size on disk, and
index_status still reports the previous node count.
Refs #2020#1997#832
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The extraction memory gate was advisory: when a full back-pressure cycle
ended still over budget it latched "futile" and kept admitting files, so a
host that could not hold the graph ran on until the kernel killed the
worker (#1997, #832) and the supervisor then quarantined innocent files.
Now the worker that latches futility runs ONE confirmation cycle (collect
+ bounded naps). If resident memory drained, the gate re-arms as before.
If it is still over budget, the attempt fails whole: every worker stops
pulling, extract returns the new CBM_PIPELINE_ABORT_OVER_BUDGET, the
orchestrator discards the staging DB on that non-zero code like any other,
and the previously serving generation is left byte-identical. The verdict
is a plain return value so it travels as a clean-exit worker response,
never a kill. The abort flag is its own atomic, not the shared cancel
token, which belongs to client cancellation.
The abort/persist codes move to pipeline.h alongside the new one (callers
legitimately distinguish them); FORCE_FULL_REINDEX stays internal.
Tests: the futility test now asserts the new code (the confirmation adds
one cycle to its bound), and a new test proves fail-whole preserves the
live DB byte-for-byte, keeps its node count, and leaves no stage residue.
Refs #1997#832#1973#1864
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
After every publish the pipeline re-exports <root>/.codebase-memory/ (graph.db.zst + artifact.json) whenever an artifact already lives there: once a session indexed with persistence, or when the team committed it and every linked worktree checks it out. The watcher's dirty signature folded those files' (size, mtime) in, so each successful reindex rewrote them, the next poll saw a new dirty state, and the daemon re-triggered itself forever: index.supervisor.reap outcome=clean immediately followed by watcher.changed strategy=git, 100+ times in 15 minutes with index workers pinned (#1953). Nothing under .git moved and no source changed, which is why the reporters blamed their long hyphenated worktree branch names (the #1254 theory its own reporter retracted); the regression test builds that exact worktree and shows the branch is irrelevant.
Scope git status with the same :(exclude).codebase-memory pathspec the exporter's own clean-tree probe already uses for the same reason. Nothing under that directory is ever an index input (discovery skips it), so a change there can never require a reindex; the pathspec is CWD-relative, so a monorepo sub-package watch excludes its own artifact directory. Untracked and committed artifacts both looped before; a real edit and a real HEAD move are still detected.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Hermes `install` failed with `op=pre_llm_hook_install` whenever config.yaml
held a PLAIN scalar with a `>` or `|` anywhere after the value start —
`probe: kaomoji face >w< here` — while the identical string double-quoted,
or a mid-value `*`, installed fine. An agent-config failure aborts
activation, so the whole install exited 1 and user-PATH registration was
skipped although eleven other agents had configured cleanly. Bisected by
@ukind, key by key and persona by persona, down to two persona strings
(#1924, continuing #1631/#1656).
Root cause: yaml_sequence_line_has_unsupported — the document validator on
the mapping-sequence path that the hook upsert/remove uses — ran a second
scan after the shared yaml_range_has_unsupported_ex and returned
"unsupported" for `[`, `]`, `|` and `>` at ANY position of a plain scalar.
The mapping-body path (mcp_servers) never had that check, which is exactly
why the mcp op succeeded and only the hook op failed on the same file.
Fix: the positional rule the `*`/`&` fix (v0.10.4) and the quote fix
(#1631, v0.10.6) already use — an indicator counts only where a NODE begins
(range start, after a mapping colon, after a block-sequence dash).
Interior ones are text. `probe: >`, `probe: |`, `probe: >-` and
`probe: [a, b]` still begin a value and stay unsupported, byte-identically.
`[`/`]` get the same treatment on purpose: in block context they are
ordinary characters inside a plain scalar (`a: b[0]` parses as the string
`b[0]`), a value-start `[` still opens a flow sequence and stays refused
(the #1631 line: exact `[]` accepted, non-empty flow sequences unsupported),
and the mapping-body path already accepted them mid-value — leaving the
sequence scan stricter would keep the mcp-op-passes / hook-op-fails
asymmetry this issue is about. Known conservative residue, shared with the
`*` rule: a plain-scalar CONTINUATION line that itself starts with one of
these characters is still refused (nothing precedes it on its line).
Tests (tests/test_config_yaml_edit.c): three accept cases run both real
Hermes ops (mcp_servers + pre_llm_call) against a persona corpus with a
mid-value `>`, `|` and `[]` and require the original document to survive as
an untouched prefix; two pins keep value-start block scalars and a non-empty
flow sequence refused with the file byte-identical. RED on the unfixed
editor: 61 passed, 3 failed (`yaml_hermes_hook_upsert(&fixture) == -1`);
GREEN: 64 passed; RED again with the production change reverted.
config_yaml_edit 64 · yaml 73 · cli 306 — 443 passed, 0 failed (macOS,
ASan/UBSan test build).
Not in this change: the refusal still reaches the user as
`op=pre_llm_hook_install path=... (target: regular file, N bytes)` with no
reason. The editor collapses every one of its ~236 YAML_ERROR sites into
CBM_YAML_IDENTITY_EDIT_ERROR, so naming the refusing rule needs a detailed
variant with a typed failure out-parameter, the shape the Codex TOML hook
path already has (cbm_toml_codex_hook_failure_t +
record_agent_config_error_with_reason). Follow-up.
Fixes#1924.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
try_load_nested_gitignore() returned NULL as soon as a walk frame already
carried a local matcher, and that matcher was inherited by every child
frame. So only the SHALLOWEST nested .gitignore on a path was ever loaded;
any deeper one was silently dropped — an EMPTY storage/.gitignore was
enough to disable storage/dump/.gitignore's "*". On the standard Laravel
layout (storage/.gitignore above storage/<dir>/.gitignore) that meant
thousands of git-ignored cache/debugbar dumps were discovered and indexed,
which on the reporter's app grew to 25,638 discovered files against 6,682
git-visible ones and ended in an OOM kill (#1973, discovery half; also the
§2 root cause of #530).
Discovery now keeps a root->leaf chain of .gitignore links per directory
frame: every directory below the root loads its own .gitignore, a frame
without one shares its parent's link, and a path is judged by walking the
chain leaf->root with git's precedence — the deepest file with an opinion
wins (so a deeper negation re-includes what an ancestor ignores), and inside
one file the last matching pattern wins. The repository root's .gitignore
(merged with info/exclude) is the chain head, so the same precedence holds
between root and nested files. Cost is one match per .gitignore on the
path — O(depth) per entry, no rescans; the walk frame shrinks by the 4 KB
local-prefix buffer it no longer needs.
Tests pin the reporter's layout (empty ancestor + deeper "*"), a deeper
negation overriding an ancestor pattern, and three nesting levels with an
empty middle file. Measured on a Laravel app (koel) seeded with typical
compiled-view/cache artifacts: 2,885 discovered files with 450 git-ignored
-> 2,435 with 0; clean clones of linux, dotnet/runtime, koel and redis
discover byte-identical file sets before and after.
Fixes the discovery half of #1973.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/setup-windows.ps1 has 326 lines and its parse-coverage report read
"113-113,113-113,245-327" — the same line named twice, and an end line that
does not exist. Two separate faults, both in cbm_error_regions_push.
Past-EOF end line. A tree-sitter node that ends at column 0 stopped right
after the previous line's newline, so it holds no text on the row it points
at. Adding 1 to that row named a line past the end of the file whenever the
region ran to EOF. The node here is start=(244,2) end=(326,0). Clamp the end
to the row above when the end column is 0 and the node spans more than one
row.
Duplicate range. Line 113 carries two separate ERROR nodes, at columns 25-29
and 31-32, and each pushed its own range. A line range says nothing new the
second time. Drop a range that exactly repeats the one already open. The
drop runs BEFORE the cap check, so a repeat is never miscounted as a range
the cap threw away.
Only an EXACT repeat is dropped, never a range that merely overlaps. Each
range is judged separately afterwards by cbm_region_is_recovered, which asks
whether definitions starting inside that range cover it. Two ranges holding
the same numbers always get the same verdict, so dropping the repeat changes
nothing. Two different ranges do not. Merging 3-3 into 2-3 hands the wider
range's covering definition to an error that definition does not explain, and
a real parse failure then vanishes from the report.
That is not hypothetical. An earlier version of this commit merged on overlap
and broke perl_malformed_source_remains_partial_issue1838, the test added
with the Perl grammar refresh in 17b5a432. The malformed fixture produces two
ERROR nodes, at lines 2-3 and 3-3. Merged, the 2-3 range looks fully covered
by before_error and is removed, so parse_incomplete comes back false on a file
that plainly does not parse. That test now pins this boundary.
The real file reports "113-113,245-326".
Two tests, both proved RED first with the exact expected text:
coverage_repeated_error_line_reports_one_range_issue963 "2-2,2-2" != "2-2"
coverage_range_never_ends_past_the_last_line_issue963 "1-5" != "1-4"
Suites run on this change: parse_coverage 34, extraction 325, pipeline 264,
mcp 246, index_resilience 7 — all passing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
The markerless stale-rendezvous recovery test reset the activation runtime
parent to NULL instead of restoring g_cli_suite_runtime_parent, so every
later cli test ran the activation probe against the developer's real
/tmp/cbm-daemon-<uid>. Restore the suite value per the contract at
tests/test_cli.c:61-65 and the sibling at :1310.
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Martin Vogel