Distilled from #738: the watcher now prunes a watched project whose root
directory has genuinely disappeared - deleting the cached DB (+wal/shm,
validated name, cache-dir-only paths) and removing the watch entry - so
vanished worktrees stop being watched forever (#286).
Hardened beyond the original PR to remove a data-loss hazard (the cached
DB can hold user-authored data such as the ADR, unrecoverable once
deleted):
- Only ENOENT/ENOTDIR stat failures count as missing. Any other failure
(EACCES, EIO, transient mounts, macOS TCC revocation) resets the
streak and logs watcher.root_stat_error with the errno; Windows
(mingw/UCRT) maps not-found to ENOENT so the check holds there.
- Pruning requires BOTH >=3 consecutive missing polls AND a sustained-
absence grace window since the streak's first miss (default 600s,
override via CBM_WATCHER_PRUNE_GRACE_S), tracked with monotonic
cbm_now_ms.
- Root reappearance resets streak + timestamp (watcher.root_restored).
- The pruned entry is released via the deferred-free list poll_once
drains (one freeing model shared with cbm_watcher_unwatch).
Limitation: only currently-watched projects are pruned; stale DBs left
by older sessions are out of scope.
Co-authored-by: pcristin <xxxokzxxx@protonmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Spring class-level RequestMapping prefixes were not folded into method mapping route_path values, so the graph could contain partial Route nodes like /orders instead of the actual endpoint /api/orders even while HANDLES edges existed.
Combine Java/Kotlin class mapping prefixes with method-level mappings during method definition extraction, and scan bounded annotation-argument subtrees so named or array-valued Spring annotations can provide the path literal.
Strengthen edge_types_probe to assert exact Spring Route node names for Java and Kotlin, not just HANDLES edge counts.
Signed-off-by: jonghyun.lee <jonghyun.lee@musinsa.com>
index_repository silently dropped files that failed to index: CBMFileResult.has_error
was set but never read, oversized files (>100 MB) were dropped with no signal, and
read/extract failures only bumped a logged-only counter — a file that couldn't be
indexed just vanished from the graph.
Collect and surface them:
- New retained {path, reason, phase} error list on struct cbm_pipeline (mirrors the
excluded_dirs pattern) + accessor + a back-pointer on cbm_pipeline_ctx_t so both
extraction paths can append. Wired in BOTH the sequential (pass_definitions) and
parallel (per-worker, merged lock-free) paths — small repos take the sequential
path, so wiring only one would leave the guard vacuous.
- Feeds: read-fail, extract-fail, the newly-CONSUMED has_error (parse timeout /
parse failed, with error_msg), and oversized. The cross_lsp phase is reserved for
the crash supervisor (Track C) and not fed here (the cross-LSP passes are
best-effort with no failure return; feeding the no-source case would be false
positives).
- MCP/CLI response gains top-level "skipped_count" (0 on clean) and, when >0, a
capped "skipped":{files[<=50],count,truncated} + "logfile". Status stays "indexed"
— a reported skip is the expected, handled outcome, not a degradation.
- Per-run logfile (full uncapped list) written ONLY when >=1 file is skipped:
$CBM_INDEX_LOG override else <cache_dir>/logs/<project>-<epoch>.log.
- Generous env-configurable caps (src/foundation/limits.c): CBM_MAX_FILE_BYTES,
default raised 100 MB -> 512 MiB; over-cap files are REPORTED (phase oversized) +
WARNed, never silently dropped.
Reproduce-first: tests/test_index_resilience.c (gating) — an oversized file (cap
lowered via env) must appear in skipped[] with the 2 good files still indexed and a
logfile written; a clean run has skipped_count 0 and no logfile. Genuine guard:
no-op'ing the recorder makes the oversized file silently vanish (RED). Full suite
5750/0, no ASan/UBSan.
Part of the resilient-indexing effort (Track B surfacing layer). Refs #668.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The per-language "resolve calls in AST node" walkers recurse once per nesting
level. Four of them — Python (py_resolve_calls_in), Go (resolve_calls_in_node),
PHP (php_resolve_calls_in_node), Kotlin (kt_resolve_calls_in_node) — had no depth
guard, so a deeply-nested or cyclic file could drive them into a native stack
overflow (SIGSEGV) that takes down the whole index run. C and Java already guard
this (C_LSP_MAX_WALK_DEPTH / JAVA_LSP_MAX_WALK_DEPTH).
Add the same wrapper/inner depth guard to all four, backed by a shared,
env-configurable cap: CBM_LSP_MAX_WALK_DEPTH (default 512) in scope.h. Past the
cap the wrapper skips the subtree — those calls stay unresolved (graceful
degradation, not a crash). The walk_depth-- runs after the inner body returns, so
early returns can't leak the counter.
Reproduce-first: lsp_{python,go,php,kotlin}_deep_nesting_no_crash (30000-deep
nested calls, run under a forked child + WIFSIGNALED). RED without the cap (the
child SIGSEGVs — verified by raising CBM_LSP_MAX_WALK_DEPTH to 100M), GREEN with
it. Full suite 5752/0, no ASan/leak.
Part of the resilient-indexing effort (Track B prevention layer). Refs #668.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
walk_defs used a fixed `walk_defs_frame_t stack[4096]` — a single ~160 KB frame on
the C stack. Two problems:
- On small thread stacks (notably the pre-2026-03 Windows 1 MB main thread) that
huge frame overflowed the stack during the definitions pass — the 0xC00000FD
crash a reporter hit indexing a large SQL file (#668). Reproduced locally by
running extraction under a constrained stack (~208 KB) + ASan.
- The `top < 4096` push guards SILENTLY DROPPED every top-level definition past
4096, so a file with more than 4096 top-level defs was only partially indexed.
Replace the fixed array with a growable heap stack (wd_stack_t + wd_push): a
256-frame initial buffer that doubles on demand, bounded by a generous,
env-configurable ceiling (CBM_WALK_DEFS_MAX, default 8M frames) that WARNs once
instead of dropping, and returns cleanly on realloc OOM (no NULL deref). A normal
file now uses ~10 KB of heap instead of 160 KB of stack, and files with thousands
of top-level defs are fully extracted. The frame stack is a pure index-addressed
LIFO (frames copied by value on pop), so heap-backing is semantically identical.
Reproduce-first: walk_defs_no_truncation_over_4096_issue668 — 5000 top-level defs
must all be extracted (RED on the old 4096 cap, GREEN after). Full suite 5748/0,
no ASan/UBSan reports on the new alloc.
Root cause was pinned by reproduction (constrained-stack + ASan), refuting earlier
grammar/scanner hypotheses — extraction and the vendored tree-sitter tree ops are
otherwise iterative. The separate error-recovery assertion mode
(symbol<token_count) was not locally reproducible and is addressed by the
forthcoming general crash-containment layer.
Refs #668
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A wildcard tsconfig path alias (e.g. "@lib/*" -> "./src/lib/*") matches
any import specifier sharing that prefix, including an unrelated scoped
npm package meant to resolve from node_modules. The direct match
already failed cleanly (no node exists at the substituted path), but
cbm_pipeline_resolve_import_node's Strategy 4 - a generic "strip a
trailing path segment and retry" fallback applied to every language,
not just its intended Rust crate::/self::/super:: case - re-resolves
the truncated import string and can land on the tsconfig's OTHER, bare
alias entry that maps straight to a directory. The resulting QN
collides with that directory's real Folder node (cbm_pipeline_fqn_module
and cbm_pipeline_fqn_folder both dot-join path segments with no type
tag), and Strategy 4 accepted whatever node it found there.
Strategy 3 (symbol-name fallback) already guards its matches with
import_targetable_label(), which excludes Folder/Project/etc. Strategy
1b (resolve_sibling_file, used only for markup/build-tool sibling
imports - SCSS/Just/BitBake/Meson/Pony, none of which are directory-
module languages) and Strategy 4 lacked the same guard; both now use
it too.
Strategy 1 (the primary, direct resolution path) intentionally keeps
no such filter: Go and Java are directory-module languages in this
pipeline (cbm_lang_module_is_dir), so a bare package import correctly
and deliberately resolves straight to a Folder node there - adding the
guard to Strategy 1 broke that (verified via the Go/Java integration
tests) before being scoped down to just the fallback strategies where
the collision actually originates.
Closes#767.
Signed-off-by: Alexandros Pappas <11921291+apappas1129@users.noreply.github.com>
tsconfig.json `paths` entries that climb out of their config's directory
(e.g. a tsconfig at apps/web/tsconfig.json mapping "@shared/*" to
"../../packages/shared/src/*", the standard monorepo layout) resolved to
a broken path: resolve_target_relative() only stripped a single leading
"./" and otherwise concatenated dir_prefix and target verbatim, leaving
literal ".." segments in the result. Those never match a real module's
FQN, since cbm_pipeline_fqn_module tokenizes on '/' without collapsing
them - so every cross-package import through such an alias looked like
dead code.
resolve_target_relative now walks the target's path segments and pops
a directory off dir_prefix for each "..", the same normalization
cbm_pipeline_resolve_relative_import already does for plain relative
imports.
Closes#730.
Signed-off-by: Alexandros Pappas <11921291+apappas1129@users.noreply.github.com>
Replace the manual join(input_path, --git-common-dir) + realpath/_fullpath with
`git rev-parse --path-format=absolute --git-common-dir` (git 2.31+), which returns
git's own absolute, canonical common-dir. Because a main repo and its linked
worktree both query the same git binary, they resolve to the IDENTICAL path, so
canonical_root is consistent across worktrees and platforms with no
platform-specific path canonicalization.
This is the truly platform-agnostic #659 fix:
- git resolves the relative ".." internally, so a project indexed from a
subdirectory gets the real repo root (the #659 bug).
- No realpath/_fullpath, so the msys-vs-native path-representation divergence that
broke the worktree==main-root invariant (test_pipeline.c
git_context_linked_worktree) on the Windows CI leg is eliminated — both sides
get git's single canonical representation.
Falls back to the previous relative-join derivation (Unix realpath) when git is
older than 2.31 and doesn't support --path-format (abs_common_dir empty). The
input_path-join fix and the Windows test-harness SKIP_PLATFORM guards from the
earlier commits on this branch remain for that fallback path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-Authored-By: anivaryam <anivaryam.dev@gmail.com>
search_code's scoped grep writes an "<root>/<file>" filelist and pipes it to
xargs. With a newline-separated filelist + plain `xargs`, a space anywhere in
the absolute path splits one file into several bogus arguments, so grep finds
nothing and scoped search returns 0 matches for any repo indexed under a path
containing a space (#687).
Fix: write NUL-separated records and consume them with `xargs -0`, so each path
stays a single argument regardless of spaces. On Windows the scoped path uses
PowerShell `Get-Content | Select-String -LiteralPath`, which reads line-by-line
and already handles spaces, so the filelist keeps newline separators there.
write_scoped_filelist writes each record piece-by-piece with fwrite/fputc (no
fixed-size stack buffer), so arbitrarily long absolute paths cannot overflow.
Reproduce-first: search_code_scoped_path_with_spaces_issue687 indexes a repo
rooted at "<tmp>/my project" and asserts total_grep_matches > 0 (RED on the old
newline + plain-xargs code -> 0 matches; GREEN after). Full suite green.
Distilled from #693 (thanks @lg320531124). Keeps that PR's correct NUL/xargs-0
idea and its Windows newline branch, but replaces its char buf[2048] + unbounded
memcpy (a stack buffer overflow reachable via long indexed paths, PATH_MAX 4096)
with the direct piece-by-piece write, and adds the reproduce-first guard it
lacked.
Closes#687
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-Authored-By: lg320531124 <lg320531124@users.noreply.github.com>
Follow-up to #683 (@Naam). The worktree info/exclude fix read the `.git` gitlink
file and its `commondir` with raw fopen(); on Windows raw fopen uses the ANSI
code page and fails to open non-ASCII (e.g. CJK) repo paths, so a worktree
indexed under such a path would silently not honor info/exclude /
core.excludesFile. Switch both reads to cbm_fopen() (the UTF-8 -> _wfopen
wrapper), matching the rest of discover.c (e.g. read_core_excludes_file at 248).
No behavior change on POSIX (raw fopen already opens UTF-8 byte paths there);
this is a Windows-only correctness/consistency fix, verified by the full test
suite staying green and cross-platform CI. discover.c now has no raw fopen.
Refs #682, #683
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On Windows every git_context test SKIP_PLATFORMs, leaving git_run/make_git_repo
as unused static functions -> -Wunused-function under -Werror failed the Windows
build. Guard the two helpers (only called from the non-Windows test bodies) so
the file compiles clean on Windows while the tests still run on Linux/macOS.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
detect_changes returned an empty impacted_symbols set for projects indexed
inside a git worktree or from a repo subdirectory (#659): canonical_root was
computed wrong. `git rev-parse --git-common-dir` emits a path relative to the
directory passed via -C (input_path), NOT to worktree_root, and the "/.git"
suffix was stripped textually without resolving ".." components. So e.g.
"../.git" joined against worktree_root, or "/ws/scripts/../.git" strip, left an
unresolved ".." and never matched the project's stored root path.
Fix (src/git/git_context.c): join the relative --git-common-dir against
input_path (the -C dir) instead of worktree_root, and realpath()/_fullpath()-
normalize the result BEFORE stripping "/.git" (the .git dir always exists for a
valid repo, so resolution succeeds). This is consistent with the existing
realpath'd root_path used for prefix matching (mcp.c).
Reproduce-first: tests/test_git_context.c. The subdirectory case is the genuine
guard — a repo indexed from a subdir yields a *relative* --git-common-dir, so the
unfixed code leaves an un-normalized "<root>/subdir/.." (verified RED; GREEN after
the realpath fix). The linked-worktree case is a supporting invariant, not the
#659 reproducer: git that emits an *absolute* worktree --git-common-dir (e.g.
2.48.x) doesn't manifest the bug, so it passes either way; it still enforces the
worktree->main-root invariant. All three git-based tests SKIP_PLATFORM on Windows
(the CI shell there cannot init a repo via system()).
Distilled from #672 (thanks @anivaryam); that PR's production fix is taken
verbatim. Its repo_root/subdir tests ran git unconditionally and failed the
Windows CI leg ("failed to init git repo"); here all three git-based tests are
Windows-skipped so ci-ok is green cross-platform. Supersedes #696, which only
realpath'd after the strip without changing the join base and so still yielded
the workspace parent for the subdirectory case.
Closes#659
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-Authored-By: anivaryam <anivaryam.dev@gmail.com>
The #546 fix (bfs_union_same_name) BFSs from every graph node sharing the
resolved name and merges the results, so a symbol split across a real
implementation and a body-less ambient .d.ts stub reports a complete caller
set. But pick_resolved_node flagged ambiguity only on an exact score tie, and
node_resolution_score adds the body span, so two genuinely-DIFFERENT functions
that share a name but differ in body length score differently, dodge the tie,
and get their caller sets unioned into one confidently-conflated answer -- the
mirror of the #546 under-report.
Complete the ambiguity rule: two or more *real* callable definitions sharing a
name (callable label + non-empty body span) are ambiguous and routed to the
existing disambiguation (snippet_suggestions) path. A body-less span==start node
is an ambient declaration / signature stub -- a fragment of one logical symbol
-- so a real implementation plus its .d.ts stub stays non-ambiguous and still
unions, preserving the #546 behavior.
Reproduce-first: tool_trace_call_path_distinct_defs_not_over_unioned (two
unrelated same-named functions with different spans -> merged caller set on the
old code = RED; ambiguous on the fix = GREEN) plus
tool_trace_call_path_dts_stub_unions_with_impl guarding that the stub+impl union
still works. Full suite green.
Distilled from #650 (thanks @mvanhorn); its trace_-prefixed reimplementation
collided with main's already-landed bfs_union_same_name and could not merge.
Refs #546, #650
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-Authored-By: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Django REST Framework ViewSets expose extra endpoints through @action-decorated
methods, e.g. @action(detail=True, methods=["post"], url_path="approve"). On
main these methods carried no route_path, so Phase 2a created no Route node and
no HANDLES edge for them -- the endpoint was invisible in the graph (#603).
Add try_drf_action_decorator (internal/cbm/extract_defs.c): when a method's
decorator identifier is "action", read the methods / url_path / detail kwargs
and set *out_path/*out_method. The existing decorator-route pipeline
(ensure_one_decorator_route, Phase 2a) then creates the Route + handler->Route
HANDLES edge in the established direction. Fallbacks: method name for url_path,
"GET" for methods; detail=True -> /{pk}/<seg>, else /<seg>.
Reproduce-first guard: handles_drf_action_python in the edge_types_probe suite
-- a DRF @action method must produce >=1 HANDLES edge (RED with 0 edges before
the extractor, GREEN after).
Known limitation: multi-method actions (methods=["get","post"]) capture only the
first method -> a single Route.
Distilled from #647 (thanks @muskiteer). Drops that PR's unrelated project-wide
HANDLES edge-direction reversal, which was unnecessary for the fix and broke the
handler->Route convention relied on by pass_cross_repo.c's find_route_handler.
Closes#603
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-Authored-By: muskiteer <agarwal.vansh3110@gmail.com>
cbm_project_name_from_path hex-encodes non-ASCII path bytes (#571), but had no
length cap -- a deep CJK / non-ASCII path triples in length past the
filesystem's 255-byte filename-component limit, so <cache>/<name>.db becomes
un-openable (ENAMETOOLONG).
Cap the derived name at FQN_MAX_NAME_LEN (200): names within the cap are
returned UNCHANGED (no drift for already-indexed projects), while a longer name
keeps its first (CAP-9) bytes plus an FNV-1a hash of the full name as a "-%08x"
suffix, so distinct long paths stay distinct. The hex encoding scheme is
unchanged.
Distilled from #624 by Matt Van Horn (the ENAMETOOLONG catch); the rest of that
PR was redundant with the #571 fix already on main and would have swapped the
encoding scheme, causing project-name drift.
Reproduce-first: project_name_length_capped_issue624 builds a deep CJK path
(~376-byte raw name) and asserts the result is <= 200, validator-safe, and that
two long paths differing only in the trailing segment map to different names; a
short CJK path stays byte-identical (no drift). RED (376 > 200) -> GREEN. Full
suite 5738/0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-Authored-By: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Martin Vogel