Follow-up to #787/#828, disclosed during #828's review: the same clobber
class also affects two call sites #828 deliberately left out of scope.
create_env_configures_for_file (pass_definitions.c) and resolve_file_throws
(pass_parallel.c) both resolved their edge source via a direct
cbm_gbuf_find_by_qn(enclosing_func_qn) lookup with no guard against the
lookup landing on the shared Folder/Project node for directory-module
languages (Java/Go) -- the exact #787 root cause, just unguarded at two
more sites. A class-level construct (no enclosing function) has its
enclosing_func_qn fall back to the MODULE QN, which for a directory-based
module collides with the package's Folder node, conflating every
same-package file's CONFIGURES/THROWS edges onto one arbitrary source.
- create_env_configures_for_file: added the same
cbm_pipeline_node_is_dir_container guard already used by the three
#828-fixed finders, falling back to the per-file File node on a
directory-container hit.
- resolve_file_throws: was missing the `rel` parameter its sibling
resolve_file_calls/usages/rw already carry, so it inlined a raw lookup
instead of calling the shared, already-guarded find_source_node. Added
`rel` and switched to find_source_node, matching its siblings exactly.
Regression guard: tests/repro/repro_issue842.c, reproduce-first per the
issue's request for "an #828-style multi-order red fixture":
- CONFIGURES reproduces on a small 5-file sequential fixture (Go, since
Java's System.getenv detection is separately broken -- extract_env_key_from_call
reads a "function" field Java's method_invocation node doesn't have; noted
in the test, out of scope here).
- THROWS only reproduces past MIN_FILES_FOR_PARALLEL=50 (the worker-merge
path) -- confirmed empirically that the small sequential fixture already
attributes correctly even on unfixed HEAD, mirroring #787's own two-faced
(sequential/parallel) bug. Padded with 60 filler classes, following
repro_issue787.c's build_parallel_fixture precedent.
RED on unfixed HEAD (both tests): 0 passed, 2 failed. GREEN with this fix:
2 passed, 0 failed. Full scripts/test.sh (5958 cases) passes clean with
this change; scripts/lint.sh's two pre-existing failures (pass_lsp_cross.h
parameter-name mismatch, extract_defs.c cppcheck AST parse) are unrelated
and present identically on unfixed main.
Fixes#842
Signed-off-by: Dhruvy0804 <dhruvgarg553@gmail.com>
The per-language chapters were LLM-drafted to establish format and
dimension mapping — their symbol targets are not yet ground-truth-derived
and must be regenerated at execution time per §3.1/§12. Make that status
explicit at the top of Appendix A so the drafted symbol names are not
read as validated.
Supersedes #420 (same diff; recreated for DCO author/sign-off).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Completes the #963 coverage taxonomy: alongside the best-effort FAILURE
signals (parse_partial, skipped), the report now lists what was
deliberately NOT indexed — a deterministic, by-design class, never mixed
into the failure surfaces.
- discover: file skips are classified by reason (gitignore / cbmignore /
skip-list / ignored-suffix / fast-pattern / size-cap) and collected via
the new cbm_discover_ex2 (per-file entries capped at 2000 with the
uncapped total reported — truncation is explicit, never silent; whole
excluded subtrees stay exhaustive as before). Unsupported-extension
files are deliberately NOT recorded: no grammar for an extension is
not an ignore decision, and listing every README/asset would drown
the signal
- pipeline: ignored files captured per run (full + incremental) and
persisted into the index_coverage table as not_indexed_file rows,
excluded dirs as not_indexed_dir; incremental runs rewrite the
by-design set fresh from discovery instead of carrying old rows
- store: the deleted-file prune exempts not_indexed_* (deliberately
unindexed paths never have hash rows); the missed shadow graph keeps
showing FAILURES only — the UI's report-an-edge-case callout can
never fire for gitignored paths, and a project whose only coverage
rows are by-design gets no miss graph at all
- mcp: index_repository response gains not_indexed_files {files, count,
truncated, note} (dirs stay under excluded); index_status's coverage
report gains a not_indexed section (dirs + files + note) and no longer
lumps by-design rows into skipped; tool descriptions updated — the
by-design class is presented as deliberate and deterministic (change
the ignore rules and re-index), with no best-effort hedging
- tests: e2e guard (gitignored file + dir reported with reasons on both
surfaces, failure surfaces stay clean, missed graph excludes them,
rows survive an incremental re-index, ignored constructs genuinely
absent from the graph); store unit test covers the prune exemption
and shadow-graph exclusion
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The UI security audit (security-ui.sh, layer A1) forbids hardcoded
external URLs in graph-ui source — the callout's GitHub issues link
tripped it on all pr-smoke legs. Follow the established pattern for
external targets (/api/repo-info deep-links): /api/ui-config now
carries upstream_issues_url and the callout consumes it, rendering the
issue button only when the backend provides an https URL. The protocol
check uses a regex literal on purpose: a bare protocol string in source
also aborts the audit's URL extraction.
Verified: security-ui.sh passes locally; /api/ui-config serves the URL;
the callout opens with the prefilled link in the browser; frontend
build + tests green.
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Clicking a node of the missed skeleton now opens a dedicated right-panel
callout instead of the standard node panel (code snippet and
callers/callees are meaningless for a not-fully-indexed file):
- explains the gap in plain words (best-effort detection; the file
content itself is ground truth)
- asks the user to have their agent summarize what fails to parse and
report it upstream so the edge case can be handled
- two working actions: a prefilled upstream GitHub issue (title
'Indexing gap: <file>'; body carries ONLY the file path and project
name, with an explicit add-snippets-only-if-shareable note) and a
copy-to-clipboard agent prompt (index_status -> summarize flagged
ranges -> file the issue) with visible copied feedback
Verified in the browser: skeleton click opens the callout with both
actions; frontend build + tests green.
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Per review: no dedicated coverage tool.
- index_status now carries the coverage report (parse_partial + skipped
lists from the separate index_coverage table, plus the best-effort
coverage_note); get_index_coverage is removed. Tool description
updated so agents learn the coverage semantics from index_status;
index_repository and the response note now point there
- the read hook resolves coverage through index_status (same envelope
fields, unchanged note wording); verified end-to-end against a real
index (flagged file -> note with exact ranges, clean file -> silence)
- get_code_snippet responses from a parse_partial file now carry a
correlated coverage_note ("line range(s) X could not be parsed ...
the source above is ground truth") — the result names its file, so
the warning is precisely anchored. Entirely-skipped files cannot
appear here (no nodes), so no uncorrelated noise is possible; clean
files carry no note (asserted both ways in the e2e test)
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Check 8d still locked the matcher to the pre-#963 'Grep|Glob' and failed
the pr-smoke leg on all platforms. The matcher now includes Read (the
augmenter injects the coverage note when a not-fully-indexed file is
read; structurally non-blocking, so the issue-#362 gate hazard cannot
recur). The guard now pins the exact new matcher and still rejects
Search/catch-all creep.
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Render the miss graph side by side with the code graph instead of
swapping layouts:
- /api/layout (code graph) now also attaches "missed_graph": {nodes,
edges, offset} — the shadow-project layout placed below the primary
cluster (same satellite pattern as linked_projects; -Y slot so
cross-repo satellites collide last). graph=missed stays the isolated
view for API users
- the UI paints the skeleton white and ghostly beside the galaxy,
auto-frames BOTH clusters on load, and navigates naturally: clicking
the skeleton flies the camera into it (file labels + detail panel),
clicking a code node flies back to the code side, and clicking empty
space while the skeleton has focus returns to the overview (the
galaxy can be entirely off-screen at that point)
- the sidebar toggle now shows/hides the skeleton (with a missed-file
count) rather than swapping the layout
Verified in the browser: overview composition, skeleton focus with
split.c labeled, and the empty-space return flight; ui/httpd suites and
frontend build + tests green.
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Four refinements to the coverage signal:
- extraction: subtract DEFINITE recovery before flagging. Tree-sitter
error recovery plus the ERROR-descending def walker often re-extract
constructs inside a failed region (verified: `def broken(:` comes
back as a def); a region whose every line is covered by definitions
that START inside it is not a miss, and a fully recovered file is not
flagged at all. Container defs (Module/Package) don't count as
evidence, and partially covered regions stay flagged — the
#ifdef-split case keeps its flag because the first branch's function
is genuinely lost
- naming: the query_graph option is graph="missed" (the graph shows
ONLY misses — "coverage" was misleading); shadow project renamed to
"<name>::missed"; tool descriptions updated so agents discover both
the option and its semantics
- hook: the CLI-installed PreToolUse augmenter now also matches Read
and injects a coverage note when the file being read is listed as not
fully indexed ("line ranges X-Y could not be parsed — the file
content you are reading is ground truth"). Safe against the old
issue-362 hazard: the augmenter is structurally non-blocking (always
exit 0, additionalContext only), mirroring the Gemini matcher that
already includes read_file; matcher upgrade bookkeeping updated
- ui: "Missed files" toggle in the graph sidebar renders the miss
graph as a second graph option — /api/layout gains graph=missed
(same db file, shadow-project scoping; base project name validated
as before)
Tests: recovery-subtraction cases (recovered def unflagged, garbage
region flagged, trailing recovered defs keep the flag), CLI matcher
tests updated, e2e resilience fixtures switched to an unrecovered
miss; verified end-to-end: hook emits the note for a flagged file and
stays silent otherwise, /api/layout?graph=missed serves the miss graph
and the code graph is unchanged, frontend build + tests green.
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Persist the parse-coverage signal outside the graph and make the misses
queryable both as a list and as a graph.
- store: new index_coverage table (project, rel_path, kind, detail) —
coverage is metadata ABOUT the graph and never mixes into the graph
tables; replace/get/free APIs with a deleted-file prune keyed off
file_hashes (the live-file set)
- store: materialize a derived miss graph under the shadow project
"<project>::coverage" (Project -> Folder chain -> File{kind,detail},
CONTAINS_FOLDER/CONTAINS_FILE edges), rebuilt from the table inside
the same transaction on every replace; invisible to list_projects
(which scans the cache directory, not the projects table)
- pipeline: coverage is written in the same persist step as every other
post-dump SQL write (hashes, ADR, FTS). Full runs replace the whole
set; incremental runs merge rows surviving from files not re-extracted
with this run's fresh entries, so a fixed file's flag clears and
deleted files prune automatically
- mcp: new get_index_coverage tool (per-project parse_partial + skipped
lists with the best-effort note); query_graph gains graph="coverage"
to run the same cypher against the miss graph; the File-node property
marker from the previous commit is dropped in favor of the separate
table — the code graph gains no coverage rows
- wording: constructs in flagged ranges MAY be missing — tree-sitter
error recovery still salvages some (verified: a function inside a
flagged #ifdef region and even a broken def were still extracted)
- tests: store round-trip/prune/shadow-graph unit test; e2e asserts the
response fields, the advertised coverage-graph query, code-graph
purity, and that the flag clears after the file is fixed (incremental
route)
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Files whose parse tree contains tree-sitter ERROR/MISSING regions were
silently indexed as if complete — constructs inside those regions are
absent from the graph with no signal anywhere (classic trigger: the
preprocessor-blind #ifdef-split-brace pattern in C).
- cbm_extract_file: set parse_incomplete and record the 1-based line
ranges of the top-most error regions (bounded at 64), serialized as
"start-end,start-end" into the result arena
- pipeline: record such files under the new phase "parse_partial"
(distinct from skips — the file IS indexed) in both the parallel and
sequential extraction paths; stamp the File node with
{parse_incomplete:true, error_ranges} on full and incremental runs
via a new cbm_gbuf_set_node_props (upsert survivor rules bypass)
- mcp: report parse_partial {files, count, truncated, note} separately
from skipped[] (skipped_count semantics unchanged); the per-run
logfile lists parse_partial rows; the index_repository tool
description documents the signal and the query_graph pattern to find
flagged files
- tests: new parse_coverage suite (RED-first, 7 cases incl. false-
positive guards and the region cap) plus an end-to-end resilience
guard asserting response fields, logfile, File-node marker, and the
advertised query_graph query
The signal is explicitly best-effort: a flag means constructs in the
listed ranges were dropped (prefer grep there); the absence of a flag
is NOT a completeness guarantee.
Refs #963
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_path_within_root used an #ifdef _WIN32/#else that opened the if(...) {
brace in each branch and shared a single close brace. That is legal C, but it
splits the function body's braces across preprocessor branches, so a
source-level parser running without the preprocessor (e.g. the tree-sitter
code indexer) sees unbalanced braces, emits an error node, and never creates a
Function node for it — the definition went missing from the graph while its
callers stayed linked to an unresolved name.
Extract a small per-OS resolve_canonical_path() helper so the function has one
unconditional control flow. Behavior is unchanged (the path-containment
security tests still pass); it now indexes correctly.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Defender step treated any non-zero MpCmdRun exit as 'flagged binary!'
and failed the job. But MpCmdRun -Scan returns 2 for a real detection and
0 for clean; any other non-zero means the scan engine itself could not run
(e.g. hr=0x800106ba: the Defender antimalware service is unavailable on the
runner), which is a transient runner-side flake, not a detection. This
false-failed the v0.9.0 release on its first attempt. Only exit 2 now hard-
blocks; an engine failure emits a warning and continues.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The npm and PyPI installers only ever fetched the standard (headless)
build, so the UI build (embedded graph visualization) was reachable only
through install.sh --ui or a manual archive download. Add a CBM_VARIANT=ui
opt-in to both installers that fetches the codebase-memory-mcp-ui-*
archive instead; the default is unchanged (standard headless build).
Document the per-channel opt-in in the README.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The #363 memory axis (CBM_MEM_BUDGET_MB) has shipped, so this reproduce-first
case is now a permanent GREEN guard, not an open bug. Its header claimed the
override "does not exist" and asserted the budget is honoured "regardless of
host RAM" — both stale: the resolver now clamps the override to effective
(cgroup/host) RAM. On a <4 GiB host the old 4096 MiB assertion would clamp and
falsely flip the bug board RED.
Rewrite the header to describe the shipped + clamped behaviour and drop the
override to 128 MiB, which is at/below any realistic host's RAM so it is
honoured exactly and stays stable on every runner. The clamp path itself is
unit-tested in tests/test_mem.c (resolve_budget_override_clamped_to_total).
Signed-off-by: Sam Li <yangsec888@gmail.com>
Fold the inline CBM_MEM_BUDGET_MB override logic in cbm_mem_init into a pure,
testable cbm_mem_resolve_budget() that returns a result struct
(budget/source/clamped/invalid), so the parse + clamp lives in exactly one
place and cbm_mem_init only surfaces the outcome as log lines.
Parsing now matches the strict src/foundation/limits.c convention (errno/ERANGE,
reject trailing garbage, positive-only): a fat-fingered value like "8GB" or a
20-digit typo becomes a warning + safe ram_fraction fallback instead of a
silently wrong budget. A valid-but-huge value clamps to detected total RAM
(logged mem.budget.clamped) rather than overflowing the MiB->bytes multiply and
wrapping to a near-zero budget (which would pin cbm_mem_over_budget() true and
stall indexing). mem.init now logs source= on both the override and fraction
paths, and the README gains the CBM_MEM_BUDGET_MB row it lacked.
Hardening of the existing #363 knob.
Signed-off-by: Sam Li <yangsec888@gmail.com>
Martin Vogel