Commit Graph
621 Commits
Author SHA1 Message Date
Martin Vogel 1a38d0a2e8 Merge pull request #1016 from DeusData/fix/stale-wal-sidecars
fix(store): remove destination WAL/SHM sidecars before installing a fresh DB
2026-07-11 05:36:24 +02:00
Martin Vogel d7e6082be7 Merge pull request #1019 from DeusData/fix/artifact-torn-export
fix(artifact): snapshot exports consistently and refuse corrupt imports
2026-07-11 04:51:02 +02:00
Martin Vogel 4ac5bb6d80 Merge pull request #1018 from DeusData/fix/surface-sqlite-corrupt
fix(store): surface SQLITE_CORRUPT from row scans instead of truncating
2026-07-11 04:36:52 +02:00
Martin Vogel 43d84ee6f2 fix(store): replace-on-rename for dump installs on Windows
The new stale-sidecar guard exposed a pre-existing Windows gap in the
same code path: rename() fails with EEXIST when the destination
exists, so cbm_store_dump_to_file could never install over a previous
DB generation on Windows. New cbm_rename_replace uses
MoveFileExW(MOVEFILE_REPLACE_EXISTING) with wide paths (raw
MoveFileExA would re-mangle non-ASCII cache paths); POSIX keeps plain
atomic rename.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 04:14:04 +02:00
Martin Vogel b79c6da931 fix(artifact): snapshot exports consistently and refuse corrupt imports
The FAST export path (the watcher/incremental auto-update) read the
raw main-file bytes of a live WAL-mode store: committed rows still in
the -wal were missing entirely and mid-checkpoint reads produced torn
snapshots (field artifact: four short indexes + btreeInitPage error 11
across ~12 nodes-table pages). Import then installed the torn file
byte-for-byte — the integrity check only sanity-checked the projects
table.

- Export: both quality levels now snapshot via VACUUM INTO (the
  consistent copy); only the index-stripping remains BEST-only, FAST
  keeps zstd-3. The fast path pays a full-copy VACUUM INTO now —
  correctness over speed; a torn team-shared artifact costs far more.
- Import: new cbm_store_check_integrity_deep (shallow check + PRAGMA
  quick_check) — used only on this rare path, hot-path callers keep
  the cheap shallow check.

Reproduce-first: artifact_fast_export_snapshots_live_wal_store
exports FAST while the writer connection is open — RED on the old
code with 0 of 60 nodes surviving the round-trip (all rows lived in
the WAL); GREEN now. store_deep_integrity_detects_page_corruption
pins the import guard both directions (healthy passes, zero-banded
pages refused).

Closes #895

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 03:28:32 +02:00
Martin Vogel b422a16889 fix(store): surface SQLITE_CORRUPT from row scans instead of truncating
Every row-scan loop discarded the terminal sqlite3_step code, so
SQLITE_CORRUPT mid-scan was indistinguishable from a clean end of
results: against a page-corrupted DB, counts answered from covering
indexes stayed correct while row fetches died at the first corrupt
table page — every query surface returned plausible truncated/empty
answers with no error (search total=379 with results=[], Cypher counts
diverging from schema counts, BM25 total=0, empty schema sections).

All 31 scan loops now capture the terminal code and fail loudly with
CBM_STORE_ERR + a store error message when a scan aborts, with
per-site cleanup (partial results freed or handed to the caller's
normal free path). Two deliberate soft spots, both enrichment-only and
documented inline: include_connected neighbor names (no store handle)
and the cluster adjacency scan (partial adjacency; primary surfaces
already fail loudly). Best-effort schema property-name discovery and
two capped helper scans keep their by-design early-exit semantics.

Reproduce-first: corrupt_page_scan_returns_error_not_truncation
zeroes a 40-page mid-file band per the issue's dd recipe and asserts
scans return CBM_STORE_ERR. RED on the old code with the exact
reported symptom (OK with 264 of 2000 rows silently truncated); GREEN
now. Fixture-health guards make a healthy-scan pass impossible to
mistake for a fix.

Closes #896

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 03:13:45 +02:00
Martin Vogel 394198e8e9 fix(store): remove destination WAL/SHM sidecars before installing a fresh DB
Three code paths install a brand-new DB file where a previous
generation lived without removing the destination's -wal/-shm:
the pipeline page writer (cbm_writer_open), artifact import (which
cleaned the TMP file's sidecars but not the destination's), and
cbm_store_dump_to_file (whose comment wrongly asserted stale sidecars
'become stale and get recreated'). SQLite validates a WAL against the
sidecar's own header/checksums — not the main file — so a crashed
session's leftover WAL is replayed ON TOP of the freshly installed
file at the next open, splicing old-generation pages into it.

New shared helper cbm_remove_db_sidecars (compat_fs, wide-path safe on
Windows) called at all three install points.

Reproduce-first: dump_install_ignores_stale_wal_sidecar follows the
issue's recipe (hot-copy a live WAL, clean close, restore the copy,
install generation 2, reopen). RED on the unguarded code — the replay
clobbered the fresh generation entirely (fresh row unqueryable);
GREEN with the cleanup. Non-vacuous: asserts the hot WAL is non-empty
before the copy.

Closes #897

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 02:40:54 +02:00
Martin Vogel a432cf729e fix(mcp): answer resources/prompts discovery with empty lists, not -32601
Clients like Cline probe resources/list, prompts/list and
resources/templates/list on connect regardless of declared server
capabilities; the -32601 Method-not-found errors surfaced as a failed
connection (status=error) even though tools worked fine. Empty lists
are the interoperable answer for a tools-only server.

Reproduce-first: mcp_discovery_methods_return_empty_lists is RED on
the unhandled dispatch (Method not found) and GREEN with the empty
lists.

Closes #958

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 02:25:27 +02:00
Martin Vogel 751d2293f9 Merge pull request #1013 from DeusData/feat/compact-tool-output
feat(mcp): TOON compact tool output — ~10x fewer response tokens across the query surface
2026-07-11 02:13:00 +02:00
Martin Vogel be9976dd31 fix(hook-augment): request format=json for the internal search_graph call
search_graph now defaults to TOON text, but ha_format_context parses
the inner payload as JSON ("results"), so the Grep/Glob/Read hook
augmenter silently produced no hookSpecificOutput — caught by the
test_hook_augment.py Windows guard going red. Both internal call sites
route through ha_build_args; request the JSON encoding there.

Verified end-to-end: indexed fixture + PreToolUse payload via stdin
emits the symbol context again.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-11 00:23:36 +02:00
Martin Vogel bd996c8b91 Merge pull request #1012 from DeusData/fix/cli-unknown-flag-reject
fix(cli): reject unknown tool flags with a nearest-flag suggestion
2026-07-10 22:35:23 +02:00
Martin Vogel 0fd175cf54 feat(mcp): TOON output for trace/query/search_code/architecture + context-bomb defaults
Completes the compact-output pass across the remaining query tools
(consumers are LLM agents; every response byte is context tokens):

- trace_path: callees[N]{qn,hop}/callers[N]{qn,hop} tables (risk/test/
  args columns per flags); the per-hop JSON key envelope was 84% of the
  payload. 3,589B -> 1,634B on a representative trace.
- query_graph: rows[N]{cols}: table (columns double as the header);
  format:"json" restores columns/rows arrays.
- search_code compact mode: results/raw/dirs tables; drops the
  field (duplicate of the qn tail). 2,488B -> 1,135B for 17 hits.
- search_code full mode: per-hit source capped at a 60-line window
  anchored on the first match, with source_start/source_truncated
  markers — uncapped whole-symbol dumps ran to 142KB per response; the
  complete symbol stays one get_code_snippet call away.
- get_architecture: default (no aspects) is now a summary (languages,
  packages, entry_points + always-on totals/label/type counts) with an
  aspects_hint — the old default rendered EVERYTHING including the full
  file_tree (94KB -> 4.5KB -> 2.8KB with TOON tables); all sections
  emit as TOON tables, aspects:[...] and ["all"] keep full access
  (52KB vs 94KB even for the explicit full dump).
- get_code_snippet: drops the property-blob enrichment (41% of the
  response; signature/docstring are literally in the source; fp/sp/bt
  never belonged there). Metrics remain reachable via search_graph
  fields=[...].
- list_projects: per-project 12-field git block (mostly null) replaced
  by the branch name; name/root/nodes/edges/size kept.
- search_graph description: in/out documented as TOTAL all-edge degree,
  NOT caller/callee counts (agent evals mis-read it as callers).
- format:"json" escape hatch on every converted tool.

Guards: tool_output_byte_budgets pins absolute byte ceilings on default
search/trace outputs (a property blob sneaking back into row emission
blows them immediately); snippet_enriched_properties inverted to pin
the no-spill contract. Suite 5,984 green; smoke updated to the TOON
contract end-to-end.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-10 21:18:56 +02:00
Martin Vogel 2950e4ec03 fix(cli): reject unknown tool flags with a nearest-flag suggestion
cbm_cli_build_args_json typed unknown flags as strings and shipped them
as JSON args the server ignores: 'trace_path --max-depth 1' succeeded
but traced at the default depth 3 — silently-wrong output that inflated
an agent evaluation by thousands of tokens of unwanted hop-2/3 rows.

Unknown flags for a known tool now fail loudly with a Levenshtein
nearest-flag suggestion:
  unknown flag --max-depth for this tool (did you mean --depth?) — run
  'cli trace_path --help' for the supported flags

Unknown-tool (no schema), raw-JSON, --args-file, stdin, and supervisor
worker argv paths are unaffected — none of them route through the flag
converter.

Reproduce-first: cli_build_args_json_unknown_flag_rejected is RED on
the permissive code (JSON produced) and GREEN with the rejection.

Closes #997

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-10 21:12:33 +02:00
Martin Vogel afcce2d42b fix(registry): don't swallow call suffixes on aliased-import direct hits
The #979 direct-hit early return in resolve_import_map fires before
suffix handling, so any imported_symbol.method() call whose base symbol
is itself an indexed node resolved to the base node instead of
base.method — the exact mis-resolution the neighboring comment warns
about. On django this degraded the graph by ~11K edges (CALLS
74,685->72,175, TESTS 38,032->29,236); auth.login => Signal.send became
an edge onto the user_logged_in Variable node.

Take the direct hit only for suffix-less callees — precisely the
aliased direct-symbol import case (#875) the early return was added
for. Verified on django: CALLS 74,708 (baseline + the 23 genuine #979
wins), TESTS 38,035; the #979 bare-alias test stays green.

Reproduce-first: resolve_import_map_alias_with_suffix_hits_method is
RED on the unguarded code (returns the bare symbol QN) and GREEN with
the guard.

Closes #1000

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-10 21:07:11 +02:00
Martin Vogel 4843a34065 feat(mcp): TOON compact output for search_graph — ~90% fewer response tokens
Tool responses land in an LLM agent's context window, where the legacy
per-node JSON objects were dominated by data no agent can use: the
similarity/semantic pipeline intermediates fp (~450B minhash hex), bt
(body-token bag) and sp (structural profile) plus 14 complexity metrics
were grafted onto every result by enrich_node_properties — ~64% of every
node emission, ~370 tokens per search hit.

search_graph now defaults to TOON (Token-Oriented Object Notation,
toonformat.dev): scalars as 'key: value' lines and results as a
'results[N]{qn,label,file,lines,in,out}:' header + one row per hit.
Published benchmarks show equal-or-better LLM retrieval accuracy at
40-60% fewer tokens than JSON; rows now also carry line ranges, which
the legacy output lacked.

- fields:[...] opts into per-node property columns (complexity,
  signature, docstring, ...); fp/sp/bt are blocklisted even there
- format:"json" restores the legacy verbose objects byte-identically;
  include_connected forces it (nested neighbor lists)
- BM25 and semantic modes emit the same table shape (rank/score column)
- semantic-only calls no longer run the unfiltered regex search that
  silently prepended up to 200 enriched nodes (102.8KB -> 4.2KB)
- default limit 200 -> 50 (BM25 100 -> 50): cheap first page, page via
  has_more/offset as before
- new emitter module src/mcp/compact_out.{c,h} (string builder + TOON
  scalar/table emission with spec quoting rules)

Measured on a real index (14-hit regex search): 20,793B -> 1,327B
(-94%); BM25 19,667B -> 5,081B; semantic 102,837B -> 4,181B. A typical
discovery session drops ~50.9K -> ~4.7K tokens combined with the
follow-up slices.

Guards: tool_search_graph_toon_never_leaks_internal_fields (sentinel
fp/sp/bt never emitted, even when requested), rewritten
tool_search_graph_includes_node_properties pins default-lean/fields-
opt-in/json-escape-hatch; suite 5983 passed, smoke green.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-10 08:23:19 +02:00
Martin Vogel 2dca0a39be Merge pull request #461 from halindrome/perl-lsp-semantic-resolution
feat(perl-lsp): Perl LSP-tier semantic resolution (Closes #459)
2026-07-09 23:37:34 +02:00
Shane McCarron 9c6b26e55d fix(perl-lsp): surface resolved method/static calls as graph CALLS edges
- Add method_call_expression to perl_call_types in lang_specs.c so the
  structural tier emits a method-call edge for the LSP bridge to refine
  (parity with PHP member_call_expression). callee_name is the bare method
  via the field-based extractor's `method` branch.
- Normalize the textual callee in cbm_pipeline_find_lsp_resolution to its
  last "::"-separated segment so qualified static Pkg::sub() calls match the
  resolved sub's dotted short-name (parity with PHP scoped_call_expression).
- Zero-edge guarantee preserved: untyped receivers still emit no edge.

Resolves DEVN-04 from plan 22-03. Verified on a Base/Derived/main fixture:
run_typed->{greet,describe}, run_static->helper, run_classcall->greet,
describe->greet (inherited), run_untyped->none.

Signed-off-by: Shane McCarron <shane.mccarron@corvexconnect.com>
2026-07-09 14:09:39 -05:00
Martin Vogel 23fd0e84e2 Merge pull request #982 from Dhruvy0804/fix/issue-842-clobber-class
fix(pipeline): guard env-access CONFIGURES and throws sources against…
2026-07-09 21:07:57 +02:00
Martin Vogel 4071877086 Merge pull request #980 from dpersek/fix/tools-list-pagination-971
Fix tools/list default pagination for non-paginating clients
2026-07-09 21:07:54 +02:00
Martin Vogel a6207c6e4c Merge pull request #979 from LA-10/fix/875-alias-import-resolution
fix(registry): resolve aliased direct-symbol imports in import_map (#875)
2026-07-09 21:07:50 +02:00
Martin Vogel 0906b532a2 Merge pull request #954 from yangsec888/feat/mem-budget-resolver-clamp
feat(mem): pure resolver + strict parse + clamp for CBM_MEM_BUDGET_MB
2026-07-09 20:13:00 +02:00
Martin Vogel 3772777307 Merge pull request #939 from aaiyer/agent/mojo-language-wiring
Add Mojo language wiring
2026-07-09 20:12:57 +02:00
Martin Vogel a70d39d1eb Merge pull request #956 from Cosm1cAC/fix/windows-search-code-utf8-paths
fix(windows): read search_code temp files as UTF-8
2026-07-09 19:30:18 +02:00
Martin Vogel 1c0a714621 Merge pull request #927 from apappas1129/fix/856-native-fetch-detection
fix(pipeline): classify native fetch() as HTTP_CALLS, not local shadows
2026-07-09 19:30:10 +02:00
Dhruv Garg 17f856cc97 Merge branch 'main' into fix/issue-842-clobber-class 2026-07-09 22:26:19 +05:30
Martin Vogel 3ea4bb024b Merge pull request #916 from alex-plosceac/fix/artifact-gitattributes-merge-ours
fix(artifact): put binary macro before merge=ours in generated .gitattributes
2026-07-09 18:54:00 +02:00
Martin Vogel fb215142db Merge pull request #906 from pcristin/fix/cypher-distinct-order-limit-pr
fix(cypher): apply distinct before return limits
2026-07-09 18:53:57 +02:00
Dhruvy0804 6b2cd08a12 fix(pipeline): guard env-access CONFIGURES and throws sources against directory-module collisions
Follow-up to #787/#828, disclosed during #828's review: the same clobber
class also affects two call sites #828 deliberately left out of scope.

create_env_configures_for_file (pass_definitions.c) and resolve_file_throws
(pass_parallel.c) both resolved their edge source via a direct
cbm_gbuf_find_by_qn(enclosing_func_qn) lookup with no guard against the
lookup landing on the shared Folder/Project node for directory-module
languages (Java/Go) -- the exact #787 root cause, just unguarded at two
more sites. A class-level construct (no enclosing function) has its
enclosing_func_qn fall back to the MODULE QN, which for a directory-based
module collides with the package's Folder node, conflating every
same-package file's CONFIGURES/THROWS edges onto one arbitrary source.

- create_env_configures_for_file: added the same
  cbm_pipeline_node_is_dir_container guard already used by the three
  #828-fixed finders, falling back to the per-file File node on a
  directory-container hit.
- resolve_file_throws: was missing the `rel` parameter its sibling
  resolve_file_calls/usages/rw already carry, so it inlined a raw lookup
  instead of calling the shared, already-guarded find_source_node. Added
  `rel` and switched to find_source_node, matching its siblings exactly.

Regression guard: tests/repro/repro_issue842.c, reproduce-first per the
issue's request for "an #828-style multi-order red fixture":

- CONFIGURES reproduces on a small 5-file sequential fixture (Go, since
  Java's System.getenv detection is separately broken -- extract_env_key_from_call
  reads a "function" field Java's method_invocation node doesn't have; noted
  in the test, out of scope here).
- THROWS only reproduces past MIN_FILES_FOR_PARALLEL=50 (the worker-merge
  path) -- confirmed empirically that the small sequential fixture already
  attributes correctly even on unfixed HEAD, mirroring #787's own two-faced
  (sequential/parallel) bug. Padded with 60 filler classes, following
  repro_issue787.c's build_parallel_fixture precedent.

RED on unfixed HEAD (both tests): 0 passed, 2 failed. GREEN with this fix:
2 passed, 0 failed. Full scripts/test.sh (5958 cases) passes clean with
this change; scripts/lint.sh's two pre-existing failures (pass_lsp_cross.h
parameter-name mismatch, extract_defs.c cppcheck AST parse) are unrelated
and present identically on unfixed main.

Fixes #842

Signed-off-by: Dhruvy0804 <dhruvgarg553@gmail.com>
2026-07-09 22:22:16 +05:30
Dustin Persek 8a4bef25ad Merge branch 'main' into fix/tools-list-pagination-971 2026-07-09 11:28:51 -04:00
Martin Vogel 374734445e Merge pull request #960 from DeusData/fix/path-within-root-indexable
mcp: make cbm_path_within_root parseable without the preprocessor
2026-07-09 17:16:14 +02:00
Dustin Persek dff70f5eb0 fix(mcp): avoid default tools/list truncation
Signed-off-by: Dustin Persek <dustin.persek@protonmail.com>
2026-07-09 11:09:53 -04:00
L 421b39f2ef Merge branch 'main' into fix/875-alias-import-resolution 2026-07-09 14:31:25 +03:00
LA-10 89e1c47fd1 fix(registry): resolve aliased direct-symbol imports in import_map
Signed-off-by: LA-10 <lashuha@uwaterloo.com>
2026-07-09 14:20:16 +03:00
Pcristin 6c7fe78e3b fix(cypher): apply distinct before return limits
Signed-off-by: Pcristin <xxxokzxxx@protonmail.com>
2026-07-09 09:40:44 +03:00
Martin Vogel 2ba1107921 feat(index): report deliberately-ignored files as a by-design coverage class
Completes the #963 coverage taxonomy: alongside the best-effort FAILURE
signals (parse_partial, skipped), the report now lists what was
deliberately NOT indexed — a deterministic, by-design class, never mixed
into the failure surfaces.

- discover: file skips are classified by reason (gitignore / cbmignore /
  skip-list / ignored-suffix / fast-pattern / size-cap) and collected via
  the new cbm_discover_ex2 (per-file entries capped at 2000 with the
  uncapped total reported — truncation is explicit, never silent; whole
  excluded subtrees stay exhaustive as before). Unsupported-extension
  files are deliberately NOT recorded: no grammar for an extension is
  not an ignore decision, and listing every README/asset would drown
  the signal
- pipeline: ignored files captured per run (full + incremental) and
  persisted into the index_coverage table as not_indexed_file rows,
  excluded dirs as not_indexed_dir; incremental runs rewrite the
  by-design set fresh from discovery instead of carrying old rows
- store: the deleted-file prune exempts not_indexed_* (deliberately
  unindexed paths never have hash rows); the missed shadow graph keeps
  showing FAILURES only — the UI's report-an-edge-case callout can
  never fire for gitignored paths, and a project whose only coverage
  rows are by-design gets no miss graph at all
- mcp: index_repository response gains not_indexed_files {files, count,
  truncated, note} (dirs stay under excluded); index_status's coverage
  report gains a not_indexed section (dirs + files + note) and no longer
  lumps by-design rows into skipped; tool descriptions updated — the
  by-design class is presented as deliberate and deterministic (change
  the ignore rules and re-index), with no best-effort hedging
- tests: e2e guard (gitignored file + dir reported with reasons on both
  surfaces, failure surfaces stay clean, missed graph excludes them,
  rows survive an incremental re-index, ignored constructs genuinely
  absent from the graph); store unit test covers the prune exemption
  and shadow-graph exclusion

Refs #963

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-09 07:51:45 +02:00
Martin Vogel 3eb294fa11 fix(ui): serve the upstream issues URL from the backend
The UI security audit (security-ui.sh, layer A1) forbids hardcoded
external URLs in graph-ui source — the callout's GitHub issues link
tripped it on all pr-smoke legs. Follow the established pattern for
external targets (/api/repo-info deep-links): /api/ui-config now
carries upstream_issues_url and the callout consumes it, rendering the
issue button only when the backend provides an https URL. The protocol
check uses a regex literal on purpose: a bare protocol string in source
also aborts the audit's URL extraction.

Verified: security-ui.sh passes locally; /api/ui-config serves the URL;
the callout opens with the prefilled link in the browser; frontend
build + tests green.

Refs #963

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-09 03:38:34 +02:00
Martin Vogel 545971cd88 feat(mcp): fold coverage into index_status; coverage note on get_code_snippet
Per review: no dedicated coverage tool.

- index_status now carries the coverage report (parse_partial + skipped
  lists from the separate index_coverage table, plus the best-effort
  coverage_note); get_index_coverage is removed. Tool description
  updated so agents learn the coverage semantics from index_status;
  index_repository and the response note now point there
- the read hook resolves coverage through index_status (same envelope
  fields, unchanged note wording); verified end-to-end against a real
  index (flagged file -> note with exact ranges, clean file -> silence)
- get_code_snippet responses from a parse_partial file now carry a
  correlated coverage_note ("line range(s) X could not be parsed ...
  the source above is ground truth") — the result names its file, so
  the warning is precisely anchored. Entirely-skipped files cannot
  appear here (no nodes), so no uncorrelated noise is possible; clean
  files carry no note (asserted both ways in the e2e test)

Refs #963

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-09 00:34:46 +02:00
Alexandros Pappas a902263439 Merge branch 'main' into fix/856-native-fetch-detection 2026-07-09 02:11:18 +08:00
Martin Vogel afc649f4e8 feat(ui): missed skeleton beside the code galaxy with click-to-focus
Render the miss graph side by side with the code graph instead of
swapping layouts:

- /api/layout (code graph) now also attaches "missed_graph": {nodes,
  edges, offset} — the shadow-project layout placed below the primary
  cluster (same satellite pattern as linked_projects; -Y slot so
  cross-repo satellites collide last). graph=missed stays the isolated
  view for API users
- the UI paints the skeleton white and ghostly beside the galaxy,
  auto-frames BOTH clusters on load, and navigates naturally: clicking
  the skeleton flies the camera into it (file labels + detail panel),
  clicking a code node flies back to the code side, and clicking empty
  space while the skeleton has focus returns to the overview (the
  galaxy can be entirely off-screen at that point)
- the sidebar toggle now shows/hides the skeleton (with a missed-file
  count) rather than swapping the layout

Verified in the browser: overview composition, skeleton focus with
split.c labeled, and the empty-space return flight; ui/httpd suites and
frontend build + tests green.

Refs #963

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-08 20:07:01 +02:00
Martin Vogel 664313e4ac feat(index): missed graph in UI + read-time coverage hook + recovery subtraction
Four refinements to the coverage signal:

- extraction: subtract DEFINITE recovery before flagging. Tree-sitter
  error recovery plus the ERROR-descending def walker often re-extract
  constructs inside a failed region (verified: `def broken(:` comes
  back as a def); a region whose every line is covered by definitions
  that START inside it is not a miss, and a fully recovered file is not
  flagged at all. Container defs (Module/Package) don't count as
  evidence, and partially covered regions stay flagged — the
  #ifdef-split case keeps its flag because the first branch's function
  is genuinely lost
- naming: the query_graph option is graph="missed" (the graph shows
  ONLY misses — "coverage" was misleading); shadow project renamed to
  "<name>::missed"; tool descriptions updated so agents discover both
  the option and its semantics
- hook: the CLI-installed PreToolUse augmenter now also matches Read
  and injects a coverage note when the file being read is listed as not
  fully indexed ("line ranges X-Y could not be parsed — the file
  content you are reading is ground truth"). Safe against the old
  issue-362 hazard: the augmenter is structurally non-blocking (always
  exit 0, additionalContext only), mirroring the Gemini matcher that
  already includes read_file; matcher upgrade bookkeeping updated
- ui: "Missed files" toggle in the graph sidebar renders the miss
  graph as a second graph option — /api/layout gains graph=missed
  (same db file, shadow-project scoping; base project name validated
  as before)

Tests: recovery-subtraction cases (recovered def unflagged, garbage
region flagged, trailing recovered defs keep the flag), CLI matcher
tests updated, e2e resilience fixtures switched to an unrecovered
miss; verified end-to-end: hook emits the note for a flagged file and
stays silent otherwise, /api/layout?graph=missed serves the miss graph
and the code graph is unchanged, frontend build + tests green.

Refs #963

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-08 19:13:25 +02:00
Martin Vogel f9d7f4aa61 feat(index): persist coverage in separate table + queryable miss graph
Persist the parse-coverage signal outside the graph and make the misses
queryable both as a list and as a graph.

- store: new index_coverage table (project, rel_path, kind, detail) —
  coverage is metadata ABOUT the graph and never mixes into the graph
  tables; replace/get/free APIs with a deleted-file prune keyed off
  file_hashes (the live-file set)
- store: materialize a derived miss graph under the shadow project
  "<project>::coverage" (Project -> Folder chain -> File{kind,detail},
  CONTAINS_FOLDER/CONTAINS_FILE edges), rebuilt from the table inside
  the same transaction on every replace; invisible to list_projects
  (which scans the cache directory, not the projects table)
- pipeline: coverage is written in the same persist step as every other
  post-dump SQL write (hashes, ADR, FTS). Full runs replace the whole
  set; incremental runs merge rows surviving from files not re-extracted
  with this run's fresh entries, so a fixed file's flag clears and
  deleted files prune automatically
- mcp: new get_index_coverage tool (per-project parse_partial + skipped
  lists with the best-effort note); query_graph gains graph="coverage"
  to run the same cypher against the miss graph; the File-node property
  marker from the previous commit is dropped in favor of the separate
  table — the code graph gains no coverage rows
- wording: constructs in flagged ranges MAY be missing — tree-sitter
  error recovery still salvages some (verified: a function inside a
  flagged #ifdef region and even a broken def were still extracted)
- tests: store round-trip/prune/shadow-graph unit test; e2e asserts the
  response fields, the advertised coverage-graph query, code-graph
  purity, and that the flag clears after the file is fixed (incremental
  route)

Refs #963

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-08 17:47:59 +02:00
Martin Vogel a4c17bd467 feat(index): best-effort parse-coverage signal for partially parsed files
Files whose parse tree contains tree-sitter ERROR/MISSING regions were
silently indexed as if complete — constructs inside those regions are
absent from the graph with no signal anywhere (classic trigger: the
preprocessor-blind #ifdef-split-brace pattern in C).

- cbm_extract_file: set parse_incomplete and record the 1-based line
  ranges of the top-most error regions (bounded at 64), serialized as
  "start-end,start-end" into the result arena
- pipeline: record such files under the new phase "parse_partial"
  (distinct from skips — the file IS indexed) in both the parallel and
  sequential extraction paths; stamp the File node with
  {parse_incomplete:true, error_ranges} on full and incremental runs
  via a new cbm_gbuf_set_node_props (upsert survivor rules bypass)
- mcp: report parse_partial {files, count, truncated, note} separately
  from skipped[] (skipped_count semantics unchanged); the per-run
  logfile lists parse_partial rows; the index_repository tool
  description documents the signal and the query_graph pattern to find
  flagged files
- tests: new parse_coverage suite (RED-first, 7 cases incl. false-
  positive guards and the region cap) plus an end-to-end resilience
  guard asserting response fields, logfile, File-node marker, and the
  advertised query_graph query

The signal is explicitly best-effort: a flag means constructs in the
listed ranges were dropped (prefer grep there); the absence of a flag
is NOT a completeness guarantee.

Refs #963

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-08 17:18:47 +02:00
Cosm1cAC 8e0c29ade0 style: apply clang-format to search_code command
Signed-off-by: Cosm1cAC <87748355+Cosm1cAC@users.noreply.github.com>
2026-07-08 15:08:08 +00:00
Cosm1cAC 9971fabf45 fix(windows): read search_code UTF-8 paths
Signed-off-by: Cosm1cAC <cosm1cac@github.com>
2026-07-08 14:43:50 +00:00
Martin Vogel be7534b385 mcp: make cbm_path_within_root parseable without the preprocessor
cbm_path_within_root used an #ifdef _WIN32/#else that opened the if(...) {
brace in each branch and shared a single close brace. That is legal C, but it
splits the function body's braces across preprocessor branches, so a
source-level parser running without the preprocessor (e.g. the tree-sitter
code indexer) sees unbalanced braces, emits an error node, and never creates a
Function node for it — the definition went missing from the graph while its
callers stayed linked to an unresolved name.

Extract a small per-OS resolve_canonical_path() helper so the function has one
unconditional control flow. Behavior is unchanged (the path-containment
security tests still pass); it now indexes correctly.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-08 15:48:31 +02:00
Sam Li 04f95ce7b8 feat(mem): pure budget resolver with strict parse + clamp for CBM_MEM_BUDGET_MB
Fold the inline CBM_MEM_BUDGET_MB override logic in cbm_mem_init into a pure,
testable cbm_mem_resolve_budget() that returns a result struct
(budget/source/clamped/invalid), so the parse + clamp lives in exactly one
place and cbm_mem_init only surfaces the outcome as log lines.

Parsing now matches the strict src/foundation/limits.c convention (errno/ERANGE,
reject trailing garbage, positive-only): a fat-fingered value like "8GB" or a
20-digit typo becomes a warning + safe ram_fraction fallback instead of a
silently wrong budget. A valid-but-huge value clamps to detected total RAM
(logged mem.budget.clamped) rather than overflowing the MiB->bytes multiply and
wrapping to a near-zero budget (which would pin cbm_mem_over_budget() true and
stall indexing). mem.init now logs source= on both the override and fraction
paths, and the README gains the CBM_MEM_BUDGET_MB row it lacked.

Hardening of the existing #363 knob.

Signed-off-by: Sam Li <yangsec888@gmail.com>
2026-07-08 06:03:54 -04:00
Anand Aiyer 69edbaa164 Merge branch 'main' into agent/mojo-language-wiring 2026-07-08 08:27:40 +05:30
Martin Vogel be410a6be9 cli: compute self-update checksum in-process instead of shelling out
cbm_cli_sha256_file shelled out to sha256sum/shasum, which is unportable:
the tool may be absent, and under cmd.exe the single-quoted path is not
dequoted so the hash of a downloaded release could never be computed on
Windows. Add a small in-process SHA-256 (FIPS 180-4, src/foundation/sha256.c)
and use it directly — no external tool, no per-OS branching, no popen. The
checksum test now covers three NIST vectors (empty, "abc", and a 56-byte
input that forces padding into a second block).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-08 00:05:52 +02:00
Martin Vogel 6e118fc80c Merge pull request #942 from DeusData/feat/ui-node-budget
graph-ui: node budget (up to 10M), scale rendering, and contrast controls
2026-07-07 23:44:46 +02:00
Martin Vogel 926e818122 feat(ui): honor the requested node budget up to a 10M ceiling
The layout endpoint treated the default render cap (2000) as a ceiling:
an explicit max_nodes above it was silently clamped back down, so the UI
could never show more than 2000 nodes regardless of the request. Make the
default a default — an explicit request is now honored up to a hard
ceiling of 10M nodes; CBM_UI_MAX_RENDER_NODES still adjusts the ceiling
for constrained deployments. The default budget rises to 5000, and the
layout refinement pass scales its iteration count down past 100k/500k
bodies to keep very large budgets responsive. Edges need no changes: the
layout already returns every edge between loaded nodes.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-07 23:09:05 +02:00