Commit Graph
1201 Commits
Author SHA1 Message Date
Pcristin 834e4f0231 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-08-30 00:26:47 +03:00
Martin Vogel 997d087b21 Merge pull request #1904 from DeusData/feat/adr-section-update
feat(mcp): add manage_adr mode='set_sections' and splice ADR sections instead of rebuilding
2026-08-29 20:34:46 +02:00
Martin Vogel 1189c875db Merge pull request #1908 from DeusData/feat/lsp-cross-inheritance
feat(lsp): resolve cross-file base classes in the Python/TS cross-LSP
2026-08-29 19:59:21 +02:00
Martin Vogel a27266309c Merge pull request #1888 from DeusData/feat/919-chialisp-distill
feat(lang): add Chialisp support with a first-party tree-sitter grammar
2026-08-29 19:59:17 +02:00
Martin Vogelandandis777 d90f98f5f4 fix(store): keep ADR line endings intact when splicing a section
Every ADR fixture in the tree used \n, so nothing exercised a CRLF-stored
document — and CI could not have found this, because the fixtures are LF
whichever platform runs them. A red would only have appeared once a real
user's CRLF ADR met this code.

The exposure is specific to splicing. The old rebuild normalised everything
on the way out, so line-ending confusion was invisible; a byte-span
replacement can miscount a separator run, and byte-identity is the property
the splice design exists to provide.

Two of the three paths were already correct, and now say so in tests:

- Replacing a section preserves CRLF exactly. The walk-back over a trailing
  break run steps across both '\r' and '\n', so it cannot cut mid-pair.
- Heading matching is ending-agnostic: "## PURPOSE\r\n" locates the same as
  "## PURPOSE\n", because the scanner trims '\r' from the name. Trimming and
  matching were different claims and only trimming was pinned.

Appending a new section was wrong. It counted trailing '\n' characters, so a
document ending "...Foo\r\n" was read as having one break but then had a
BARE '\n' appended, and the "## NAME" line it wrote used '\n' too — leaving a
mixed document that neither the author nor the tool asked for. A CRLF file
ending in a blank line also gained a spurious extra break.

Appends now count a "\r\n" pair as one break, and write the separator and
heading line using the document's own ending.

The contract, now that a document can be mixed:

- Bytes that already exist are never rewritten, so each line keeps whatever
  ending it had. This follows from splicing and is what the acceptance
  property already guaranteed.
- Text this code writes itself — the separator and the "## NAME" line —
  follows the document's majority ending, LF on a tie.
- A caller's body is inserted verbatim. Rewriting bytes a caller supplied
  would be the same silent modification this whole change set removes.

Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-29 17:44:43 +02:00
Martin Vogel 2910e2842f feat(lsp): resolve cross-file base classes for the Python/TS cross-LSP
`CBMDefinition.base_classes` carries the SOURCE SPELLING of each base
("Base", "django.db.Model"): extraction strips keywords and generic
arguments, but it cannot know where the name is declared. The Python and
TS cross-file registrars, however, consume `CBMLSPDef.embedded_types` as
fully-qualified names — py_lookup_attribute and ts_lookup_member feed each
entry straight into cbm_registry_lookup_type.

An unqualified spelling therefore matched nothing declared in ANOTHER
file. `class Child(Base)` in child.py never saw Base in base.py, so a
call to an inherited method through a typed receiver found no member and
fell through to the weak textual cascade, where the receiver-aware guard
(#592/#606) correctly kills it. The member-lookup walk over
embedded_types was already there in both languages; only the names it
walked were unusable across files.

cbm_pxc_collect_all_defs now resolves each base spelling to a project QN
using exactly the inputs pass_semantic uses to draw its INHERITS edge:
the project registry, the declaring module, and the file's import map.
Two properties follow. The LSP's inheritance view is the same relation
the graph records, so the two cannot diverge. And the binding is import-
or same-module-backed rather than a short-name guess, so the CALLS edge
it enables is a supported fact that the weak-member guard keeps.

Weak registry strategies (suffix_match / unique_name / field_type_hint /
fuzzy) are rejected for bases via an explicit drop-list: a base bound
because some project type happens to share its name is precisely the
fabricated relation #606 removed, and inheritance multiplies it — every
inherited member of the wrong base would become a callable target. An
unresolved base keeps its raw spelling and the behaviour that predates
this change, so stdlib and third-party bases are unaffected.

Scoped to the languages whose registrars read embedded_types as QNs
(Python, JS/TS/TSX). Go, JVM, C#, C++ and Rust already qualify their own
embedded types and keep the raw join.

Cost is O(defs) hash lookups plus ONE import map per file: no per-call-
site hierarchy walk, no registry scan, no per-file registry rebuild. The
complexity guard stays linear (nodes 1.93, edges 2.07, per-file defs
2.00 against a files x corpus coupling of ~4).

Tests:
- ts/S6 flips to its own documented condition: was `calls == 0` as a
  tripwire recording the gap, now asserts calls >= 1 AND an lsp_ts_*
  strategy. Measured: strategy=lsp_ts_method, confidence 0.95.
- python/S6b is new and covers what python/S6 structurally cannot: S6's
  `def run(c)` parameter is un-annotated, so no receiver type exists to
  inherit through and any binding there is a guess about a name (it
  resolves via unique_name at 0.75). S6b supplies the receiver three ways
  — annotated parameter, `self` in the subclass, constructor result — and
  asserts INHERITS >= 1 plus an lsp_* strategy. Revert-check: all three
  fall back to unique_name at 0.75 without this change, lsp_method at
  0.90 with it.
- Both rows assert the STRATEGY, not just the edge count: a short-name
  guess and a resolved inherited member produce the same count in a
  two-file fixture, and only the strategy tells them apart.
- Corrects python/S6's stale comment citing a base_classes extraction
  bug; extraction_inheritance is green for Python and TS, and
  base_classes now holds clean names.

macOS full suite 7690 passed / 0 failed / 8 skipped (140 suites);
make lint-ci clean.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-29 17:30:00 +02:00
Martin Vogel 5d96377aa5 Merge pull request #1903 from DeusData/feat/1324-python-weak-member
fix(pipeline): suppress weak Python member calls
2026-08-29 17:28:02 +02:00
Martin Vogelandandis777 d013555d28 feat(mcp): splice ADR sections instead of rebuilding the document
manage_adr mode='set_sections' rebuilt the ADR from cbm_adr_parse_sections()
and cbm_adr_render(). That model is lossy, so the rebuild silently rewrote
text nobody asked to change. Measured on the previous commit:

- Everything before the first recognised heading was dropped.
- A mis-cased '## Purpose' was not a heading, so when it came first its
  whole block was dropped with the preamble — an ordinary typo cost the
  author a section.
- Sections stored out of canonical order were reordered.

mode='update' never had this exposure because it writes the caller's bytes
verbatim; the section merge introduced it.

The merge now locates the target heading's byte span and replaces only that
span, appending a block when the heading is absent. Bytes outside the span
are never reconstructed, so a preamble, a code fence, an unrecognised
heading and the author's section order all survive byte-for-byte. That
property is the acceptance test: every document in
adr_splice_preserves_untouched_bytes is a case the rebuild corrupted.

Arbitrary headings now work, which is what an "add an entry" mode needs.
The canonical six become a convention rather than a privilege: they are
still what ADR_EMPTY_HINT suggests, but they get no ordering priority and
none is required. Under splicing, ordering is whatever the document says,
and any privileged reordering would reintroduce the data loss above.

Names match exactly, including case, so '## Purpose' and '## PURPOSE' are
different sections. Folding them would silently merge two blocks the
author chose to keep apart — the same class of loss this commit removes.

cbm_adr_validate_section_keys() no longer enforces the canonical set. It
now rejects only names that could not round-trip through a '## NAME' line
(empty, '#'-leading, newline-bearing, edge-whitespace, over-long), because
such a name scans back as a different heading or as none, and a second
identical write would append a duplicate instead of being a no-op.

A heading inside a fenced code block is a code sample, not a section. An
unterminated fence hides every heading after it, so a write would append a
duplicate rather than replace: that is refused explicitly, and mode='sections'
reports the same ambiguity instead of answering with a partial list.

manage_adr mode='sections' now uses cbm_adr_scan_headings(), the same
classifier the write path splices with. It used to list any '#'-prefixed
line, so a '## Foo' in prose or inside a fence was reported as a section no
write could target. Two components disagreeing about what a section is was
how a section write came to be able to destroy one.

cbm_adr_parse_sections() is unchanged, so documents parse exactly as before.
TEST(adr_parse_sections_non_canonical) still passes untouched.

Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-29 12:43:38 +02:00
Martin Vogelandandis777 7bef7f4d66 feat(mcp): add manage_adr mode='set_sections'
manage_adr could only replace: mode='update' overwrites the stored
document in full, so adding one entry costs a re-send of the whole ADR.
That is a data-integrity problem before it is a cost one — the caller has
to reproduce every byte it did not intend to change, so the unchanged
prefix survives only as well as the round-trip that carried it.

mode='set_sections' rewrites only the sections named in `section_updates`
and leaves the rest of the stored document untouched, so the stored copy
stays the authority for everything the caller did not name. Unlike a
whole-document append it is idempotent: a client that loses a response
and retries re-sets the same section to the same body and the document is
byte-identical, where an append would silently duplicate the chunk.

Only the six canonical section names are writable. That is a correctness
constraint rather than a style rule: adr_try_section_header() parses ONLY
canonical headers, so a non-canonical '## FOO' written here would be read
back as body text of the section above it, and a second identical write
would append a duplicate — destroying the idempotence the mode exists for.

Three things the new mode needed that were not there:

- cbm_store_adr_update_sections() now wraps its read-modify-write in
  BEGIN IMMEDIATE. Three writers replace this row wholesale — the indexing
  pipeline, the UI POST /api/adr handler, and mode='update' — so the
  unguarded get/merge/store lost whichever of them committed between the
  read and the UPSERT. mode='update' is a single atomic UPSERT and never
  had that window; a section merge introduces it.
- set_sections joins the write_request classification. A mode missing from
  it takes no per-project mutation lease, resolves the store query-only,
  and never reaches open_adr_store_for_write — its write would be
  attempted through a read-only handle while an index runs.
- The write path reads the legacy <root>/.codebase-memory/adr.md itself.
  The existing migration runs on the read path only because it must not
  block on the lease; without this a section write would merge onto an
  empty document and discard an ADR still present on disk.

CBM_ADR_MAX_LENGTH now applies to an MCP write path: it is enforced inside
cbm_store_adr_update_sections, which mode='update' does not go through. An
empty section body, an unknown section name and a missing section_updates
are all rejected before any store is opened, so a caller that meant to
write never receives a success-shaped read.

Section-level update was chosen over the whole-document append proposed in
PR #1243; the analysis that established the problem is from that PR.

Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-29 12:43:38 +02:00
Martin Vogel b6a2284320 fix(pipeline): suppress weak Python member calls (distilled from #1324)
Python attribute calls were not classified as member calls, so weak
short-name strategies (suffix_match / unique_name / field_type_hint)
could bind `obj.method()` to an arbitrary same-named project symbol —
`accelerator.print()` resolving to MockAccelerator.print.

Flag a Python attribute call is_method when its receiver is unresolved,
and extend the existing TS/JS weak-member guard to Python. Receivers that
ARE known stay exempt: self/cls, super(), and any identifier bound by one
of the file's imports (including the root of an attribute chain), so
Python's canonical `module.function()` cross-file call still resolves.

The guard helper is renamed cbm_tsjs_suppress_weak_method_match ->
cbm_suppress_weak_member_match. Its drop-list is unchanged, and the
per-language gate stays at the two call sites, which enumerate the same
language set in pass_calls.c and pass_parallel.c so the sequential and
parallel resolvers cannot diverge. ArkTS remains in that set.

python/S6/inherited_method in the resolution probe now asserts calls == 0,
recording the same trade TypeScript took when the receiver-aware guard
landed: the edge it used to produce came from a unique_name registry
fallback (measured on main: strategy=unique_name, cands=1, conf=0.7500)
where a weak short-name guess happened to be right in a 2-file fixture.
It flips back to >= 1 once py_lsp_cross resolves inheritance, matching the
flip-back note already carried on the TypeScript S6 case.

Fixes #1276
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-authored-by: Enferlain <15861396+Enferlain@users.noreply.github.com>
2026-08-29 11:44:03 +02:00
Martin Vogel 3b2c6cbe3b Merge pull request #1866 from DeusData/fix/issue-474-search-code-scan-deadline
fix(mcp): bound search_code scan deadlines
2026-08-29 11:32:18 +02:00
Martin Vogel 1130f577ee Merge pull request #1883 from DeusData/feat/879-importance-distill
feat(pipeline): index-time importance scoring (weighted-degree)
2026-08-29 11:32:15 +02:00
Martin Vogel 4ae5a7353c Merge pull request #1885 from DeusData/feat/issue-525-graph-compare
feat: compare indexed graph snapshots
2026-08-29 11:10:49 +02:00
Martin Vogel be7b57828e Merge pull request #1882 from DeusData/feat/518-519-bm25-prose
feat(search): index prose content for BM25 full-text search
2026-08-29 10:51:57 +02:00
Martin Vogel 31bdd4d682 fix(mcp): bound search code scans
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-29 01:10:28 +02:00
Martin Vogel 51c48e2093 Merge pull request #1858 from DeusData/fix/issue-1834-pi-stdin-transport
fix(pi): send tool arguments over stdin
2026-08-29 01:07:08 +02:00
Martin Vogel 56dc82ec11 Merge pull request #1850 from DeusData/fix/issue-56-rust-parallel-cargo-context-resume
fix: preserve Cargo context in parallel Rust resolution
2026-08-29 01:07:03 +02:00
Martin Vogel 202ea657bc Merge pull request #1848 from DeusData/fix/issue-403-sensitive-programs-root-resume
fix: block sensitive program roots before background work
2026-08-29 01:07:00 +02:00
Martin Vogel 574356343d Merge pull request #1845 from DeusData/fix/issue-1764-idle-frontend-cpu-resume
fix(daemon): avoid duplicate idle maintenance polling
2026-08-29 01:06:50 +02:00
Martin VogelandShauryaaSharma 01c7e549ef feat(search): index prose in BM25 via a nodes_fts body column
Closes #518 (Section nodes don't index body text) and #519 (META.yaml /
frontmatter description values not indexed). A question asked in words
could not reach the nodes that hold the words: Section carried only its
heading, a config Module only its path, and both labels were filtered out
of BM25 results outright.

nodes_fts gains a fifth column, `body`, backfilled from each node's
EXISTING properties.docstring. No new column on `nodes`, no new table, and
CBM_INDEX_FORMAT_VERSION deliberately does NOT move -- nobody is forced to
reindex. A legacy four-column table survives untouched under CREATE VIRTUAL
TABLE IF NOT EXISTS: the backfill probes for the column before naming it,
and bm25()'s surplus fifth weight is inert on a table that has no fifth
column (FTS5 reads a weight only for a column an instance landed in). Such
a database keeps opening and searching, just without prose.

Two node kinds gain the docstring they lacked:
  - Markdown section bodies (#518) -- the blocks following a heading, up to
    the next heading or subsection, collapsed to single spaces and capped at
    MAX_COMMENT_LEN (500 B) with a UTF-8 backoff that never splits a
    multi-byte sequence.
  - Top-level YAML/JSON description / summary / purpose (#519), promoted
    onto the file's Module node.

Section and Module are no longer excluded from BM25 results. Both the
ranked query and the mirrored count query were changed together; removing
the filter from only one desynchronises results from counts. Column weights
are bm25(nodes_fts, 1.0, 1.0, 1.0, 1.0, 0.3): the four identifier columns
keep parity with each other and prose sits well below them, so a prose-only
hit surfaces but never outranks an identifier match.

The FTS backfill is consolidated into one writer, cbm_store_fts_rebuild(),
used by the wholesale rebuild AND the delta merge. That second site matters:
it had its own hand-written four-column INSERT, which with a fifth column
would leave body NULL for every node arriving by delta -- silently, on the
warm path most users hit, while a full reindex looked perfect. There is now
exactly one place the column list is decided, and a test that fails if the
delta path stops writing prose.

The backfill guards json_extract with json_valid: malformed properties JSON
exists in pre-fix databases and would otherwise abort the whole rebuild.

Tests: markdown section bodies (own body, stops at the next heading, UTF-8
safe cap, empty body stays NULL), YAML/JSON description promotion (block
scalars, synonym fallback, top-level only, config-only), FTS body indexing
and its column isolation, malformed-JSON tolerance, legacy four-column
degrade, incremental watermark, BM25 finding Section and Module by prose,
results/total consistency, identifier-outranks-prose, and the delta-merge
regression.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

Co-authored-by: ShauryaaSharma <196896785+ShauryaaSharma@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-29 01:04:35 +02:00
Pcristin 28d9892f95 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-08-28 23:05:40 +03:00
Martin Vogelanddarkverbito 706bb2ce81 feat(lang): add Chialisp support with a first-party tree-sitter grammar
Chialisp (.clsp/.clib/.clinc) is the Chia smart-coin language. The only
public grammar, Quexington/tree-sitter-chialisp, cannot parse it: on the five
reference files from chia-blockchain@main it produced 3-34 ERROR nodes each.
Three defects account for all of them, and each desynchronised the rest of the
file rather than losing one node:

  * comments were `/;.*\r\n/` -- CRLF-only, while real files are LF;
  * `(include foo.clib)` was rejected because of the dot in the filename;
  * `(defconstant NAME <expr>)` accepted only a primitive after the name.

So the grammar is written here instead, in tools/tree-sitter-chialisp/, and
files as first-party: its vendored LICENSE is the project's own, byte-identical
to the repository root, carrying no third-party copyright.

It is deliberately GENERIC -- source_file / list / symbol / string / number /
hex / dot / comment, modelling the clvm_tools reader rather than the Chialisp
form vocabulary. `mod`, `defun`, `defconstant` and `include` are ordinary head
symbols; which lists are definitions is decided in extract_defs.c. A dialect
that adds a form (cl-21 -> cl-23 -> cl-26 each added several) therefore needs
no regenerated parser. Acceptance: all five reference files parse with zero
ERROR and zero MISSING nodes; the three defects above are pinned as tests.

Extraction, and the four decisions in it:

  * `Constant` is admitted to the cross-file name registry
    (cbm_label_is_registry_symbol). Chialisp's condition codes live as
    `(defconstant ...)` in a shared .clib and are referenced by name from every
    puzzle; labelled but unregistered they are seeded nowhere, and roughly half
    of a Chialisp graph would be present as nodes yet never the target of an
    edge. All three registry-seeding sites route through that one predicate, so
    they cannot diverge. No other language emits `Constant`, so the blast radius
    is exactly Chialisp. Pinned by a two-file graph test that goes red when the
    label is removed from the predicate.
  * `export` / `namespace` are NOT definition heads. `(export foo)` names a
    function already defined in the same file, so treating it as a def head
    mints a second node for one symbol. They stay in the not-a-call filter.
  * A parameter list is a `list` too, but it binds rather than invokes. Without
    that gate `(defun check_conditions (HEIGHTLOCK conditions) ...)` mints a
    phantom call to HEIGHTLOCK, and real Chialisp is mostly such definitions.
  * `(include *standard-cl-26*)` is a dialect directive, not a file dependency;
    recording it would invent a dependency on a file that does not exist.

Three helpers that the defs, calls and unified walks all need -- the quote-data
gate, comment-skipping named-child access, and the Chialisp def-head set -- live
in helpers.c rather than being copied per translation unit. They previously
existed as byte-identical duplicates that had already drifted: extract_defs.c
skipped comments when reading a def head while its mirror in extract_unified.c
did not, so a comment between a head and its name desynchronised defs from call
scope. Both now use the same accessor.

Ledger: 161 -> 162 grammars across the seven published surfaces, ABI-14 86 -> 87,
first-party 13 -> 14, CBM_LANG_COUNT 164 -> 165.

Distilled from PR #919 by darkverbito, whose extraction logic -- the CLVM
not-a-call vocabulary, the quoted-data gate, the filename-derived `mod` name and
the include/embed import handling -- is kept and is the substance of this change.
The grammar that PR vendored is not used.

Co-authored-by: darkverbito <8049985+darkverbito@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 21:30:06 +02:00
Martin Vogel 7ba26f5216 fix: sanitize graph comparison metadata
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 21:00:41 +02:00
Martin Vogel 079d8dcb89 fix: harden graph comparison identities
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 21:00:41 +02:00
Martin Vogel cf6e620431 feat(mcp): add bounded graph comparison tool
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 21:00:41 +02:00
Martin Vogel 396ac348b5 fix(pipeline): score importance over the staging store on the delta route
The closure-delta incremental route scored importance off a graph that does
not contain the project's edges, and persisted the result.

cbm_delta_preseed fills that route's gbuf with PROXY nodes -- id, label, name,
qualified name, file path, and nothing else. Only the re-extracted files carry
real edges; every inbound edge from an unchanged file is snapshotted and
re-linked later, inside cbm_delta_patch. Scoring in run_postpasses therefore
read an in-degree of zero for symbols with hundreds of real callers, and
because the repaired files' rows are purged and re-inserted, cbm_delta_patch
wrote those zeros to disk. Measured on a fixture whose `helper` has 25
callers: full index 5.000000, after one closure-delta re-index 0.000000.

closure_try_plan runs first, so this is the preferred warm path, not an edge
case. The comment at the call site claimed a rehydrated whole-project buffer;
that is true only of the legacy-partial route, and it is now stated correctly.

The fix rescores in SQL over the staging store, immediately after
cbm_delta_patch has inserted the new nodes and re-linked the inbound edges --
the first moment on this route where the complete graph is queryable. Loading
the full graph into RAM instead would have given back the point of the delta
route.

Rescoring is project-wide, not limited to the changed files. Distinct-files-
per-name and in-degree are both GLOBAL inputs: a body-only edit in one file
can change the score of a symbol defined in a file that did not change. A
scoped rescore would leave those stale.

TWO ROUTES, ONE RULE. The scoring rule and the JSON write-back are not
reimplemented in SQL. cbm_pipeline_importance_score() and
cbm_pipeline_importance_set_prop() are now single definitions that both routes
call -- the SQL side reaches them through two SQLite user-defined functions,
so the same C code runs either way and the rule cannot drift. SQL contributes
only the three aggregates, each an indexed GROUP BY. What remains free is the
input GATHERING, and tests bind that directly: a full index and a closure-delta
incremental index converging on the same tree must produce identical scores for
every symbol. Both new tests assert the route was actually taken, so neither
can pass by silently running some other path.

Cost, measured on a 666 MB Java corpus: cold full index unchanged at 83 s
(the recompute does not run on that path); warm closure-delta re-index 20 s
without the recompute, 26 s with it. The delta route keeps its advantage over
a full reindex (26 s vs 83 s).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 20:56:33 +02:00
Martin Vogelandpetercoxphoto e5f3aab02e feat(pipeline): index-time importance scoring (weighted degree)
Score every Function/Method/Class node during indexing and persist the
result as a numeric "importance" key inside the node's existing
properties_json:

  importance = sqrt(num_refs) * priv * generic * distinct * test_penalty

num_refs is the incoming CALLS + USAGE degree. The multipliers demote
private (leading-underscore) and generic names (defined in >= 5 distinct
files), promote distinctive identifiers (snake_case or camelCase, len >=
8), and demote test scaffolding -- symbols in a test file per the graph's
own cbm_is_test_path() classifier, or targets of an incoming TESTS edge.
Weighted degree only; a transitive (PageRank) refinement is deliberately
not built here until it can be shown to beat this on a fixed judgment
set.

No schema change and no CBM_INDEX_FORMAT_VERSION bump: the score lives in
the existing properties_json TEXT column, so no index is forced to
rebuild and indexes written by older builds simply lack the key.

Three failure modes this shape invites, all of them silent, are addressed
directly:

* Cost. The generic-name multiplier needs |{files a name is defined in}|.
  Computing that per NODE with a pairwise file-path scan is O(k^3) in a
  same-name group of size k -- ruinous on real corpora, where a single
  Java name group reaches k > 4000. Here the count is computed once per
  distinct NAME and memoized, and file paths are deduplicated through a
  hash set, so total distinct-file work is linear in the graph. Measured
  cold full index of a 666 MB Java corpus, same -O2 binary: 85 s with the
  pass disabled, 80 s with this implementation, 481 s with the per-node
  shape.

* Registration. PREDUMP_PASS_COUNT is now derived from the pass table
  with sizeof instead of being hand-written. A hand-written count that
  lags an appended entry silently skips the last-registered pass with
  every test still green.

* Incremental. The incremental post-pass sequence runs the pass too, over
  a graph rehydrated from the store whose nodes already carry the key.
  The write-back overwrites an existing key in place instead of appending
  a second one, which would otherwise corrupt properties_json into
  {"importance":1.0,...,"importance":2.0}.

tests/test_importance.c covers the formula and binds all three failure
modes; the complexity suite gains a GATED counter asserting the pass's
same-name-group work stays linear (~2x, not ~4x) under a corpus doubling.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-authored-by: petercoxphoto <20230727+petercoxphoto@users.noreply.github.com>
2026-08-28 20:04:21 +02:00
Martin Vogel 31b611a30e Merge pull request #1872 from DeusData/feat/1412-cfml-distill
feat(lang): CFML tag-dialect .cfc routing + embedded <cfscript> defs (distilled from #1412)
2026-08-28 16:28:06 +02:00
Pcristin 2a7f4427dc Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-08-28 16:39:53 +03:00
Martin Vogel d706c33ac2 Merge pull request #1337 from abendrothj/bash-tool-hook-augment
feat(hook-augment): augment Bash tool searches via graph
2026-08-28 14:40:32 +02:00
Martin Vogel 4b3ab38e28 Merge pull request #1438 from shashwatMishra96/fix/quarantine-exit-nonzero
fix: quarantine files on EXIT_NONZERO instead of aborting chunk
2026-08-28 14:26:28 +02:00
Martin Vogel f899770865 Merge pull request #1342 from cmeerw/fix/netbsd-daemon-support
fix(netbsd): add NetBSD support to daemon IPC and runtime
2026-08-28 13:43:07 +02:00
Martin VogelandAllan Oepping 229b4fe1f8 feat(lang): CFML tag-dialect .cfc routing + embedded <cfscript> defs
Distilled from #1412 by @allanoepping. Fixes a real recall hole: tag-dialect
.cfc components were routed to the cfscript grammar (extracting nothing),
and <cfscript> bodies inside tag-dialect files are opaque cf_script_content
to the cfml grammar — whole legacy components produced zero functions.

Fix 1 — his .cfc dialect sniffer, nearly verbatim: cbm_disambiguate_cfc()
reads a 16K head via cbm_fopen; <cfcomponent>/<cffunction> anywhere means
tag dialect; a leading <cfscript (after whitespace and <!--- ---> comments)
or no markup at all keeps the old CFSCRIPT routing, so script-dialect files
and read failures behave exactly as before. Wired in detect_file_language
beside the existing _m/_cls/_inc sniffers, with his 8 direct tests.

Fix 2 — his embedded-defs capability, re-based onto the shared machinery
that superseded his mechanism mid-flight (#1852, merged after his last
rebase): a cfml_embedded_imports spec row (cf_script_tag/cf_script_content
-> CFSCRIPT), CFML added to the embedded-imports dispatch, and the
extract_structure gate extended to CFML. Included ranges on the original
buffer give absolute coordinates for free, so his line-number assertions
hold verbatim, and calls inside blocks become extractable as a bonus. His
~90-line slice+shift implementation is deliberately NOT carried — it
mirrored the sanctioned shape of its day, which #1852 replaced.

One correctness guard the port needed: the Vue attribute resolver is now
gated to Vue — cf_script_tag carries no lang=/src= attributes, and running
the resolver would have silently rewritten CFML's embedded language from
CFSCRIPT to the resolver's JavaScript default.

His embedded extraction test (line remap + <cfsetting> ERROR-cascade case)
carries over verbatim and binds: reverting the spec/gate wiring fails it at
ASSERT(has_def(r, "Function", "greet")). Suites: extraction, language,
edge_imports, lang_contract, grammar_labels, grammar_regression — 637/0.

Co-authored-by: Allan Oepping <allan@intellipay.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 13:26:59 +02:00
Martin Vogel 2cc80f20bb Merge pull request #1204 from DavidMusk93/deleted-self-exec-fix
fix(mcp): survive atomic binary replacement
2026-08-28 12:46:43 +02:00
Martin Vogel b1d9dba578 Merge pull request #1870 from DeusData/chore/1298-surviving-subset
chore(headers): sync stale decl names, name genuine buffer sizes (distilled from #1298)
2026-08-28 12:39:13 +02:00
Martin VogelandJake Abendroth 09c6624166 perf(hook-augment): gate no-op Bash calls before identity hashing
Maintainer commit on top of the Bash-matcher work, closing the cost
question the 2026-08-03 review left open before default-on shipped.

Measured on a production binary (2026-08-28): every hook invocation paid
~1.1-1.2 s of user CPU BEFORE any gating — not in cbm_mcp_server_new as
first suspected, but in main()'s executable-identity hash for cohort
admission, which the hook-client path runs ahead of any input inspection.
With Bash in the installed matcher, nearly every agent command (`ls`,
`make`, `git`, ...) would have paid that for nothing: agents issue far
more Bash calls than Grep/Glob, and almost none are searches.

The fix: main() now reads stdin on the hook-client path and exits before
identity hashing when the event can never produce output — an un-forced
PreToolUse Bash event whose command yields no search pattern or no
queryable token. A no-op touches no shared state, so it needs no
identity. The gate is a pure exported function using the exact parser
pair ha_process uses, so gate and augmenter cannot disagree; anything
uncertain (lifecycle events, other tools, coverage adapters, oversized
stdin, forced events) pays full fare unchanged. Consumed stdin is handed
back through a prefetch so downstream readers see it exactly once.

Measured after: no-op Bash calls ~10 ms wall / 0.00 s user CPU (was
~1.2 s); real Bash searches and Grep/Glob unchanged at full fare.
cli 259/0, mcp 179/0.

Co-authored-by: Jake Abendroth <contact@jakea.net>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 12:37:53 +02:00
Pcristin fc8da75993 Merge branch 'main' into fix/issue-1682-dotted-ts-imports 2026-08-28 12:22:23 +03:00
Martin Vogelandaoright 7fe2f87ba7 chore(headers): sync stale decl names, name genuine buffer sizes
Distilled from #1298 by @aoright — the subset that survived review and
main's drift, per the 2026-08-17 ruling.

- pass_lsp_cross.h: the header still documented and declared
  all_defs/imp_keys/imp_vals while the implementation renamed them to
  defs/imp_names/imp_qns — doc-only drift, now synced; plus the braced
  guard in cbm_pxc_registry_for_lang.
- runtime.h: semantic_version[12] gains a named size
  (CBM_DAEMON_SEMVER_SIZE), the "defensible: a real size" case from the
  review.
- httpd.h: request-field buffers take the CBM_SZ_* names for their actual
  sizes; the constants.h include sits in the top include block rather than
  mid-file.

Deliberately NOT carried, per the same ruling: the store.c calloc hunk
(the claimed uninitialized read does not exist), the generic-constant
renames (semantic laundering), the watcher.c split (its target function
gained a semantic fix on 2026-08-20 and the refactor must be re-derived),
and the lsp_resolve.h extractions (their target loop was rebuilt on main).

Co-authored-by: aoright <102943475+aoright@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 10:29:24 +02:00
Martin Vogel 10a92a0ebd Merge remote-tracking branch 'origin/main' into pr1229-work
# Conflicts:
#	README.md
#	docs/index.html
#	docs/llms.txt
#	pkg/npm/README.md
#	src/cli/agent_profiles.c
#	src/cli/agent_profiles.h
#	tests/test_agent_profiles.c
2026-08-28 09:22:12 +02:00
Martin Vogelandsunmingqiang 5b96b06754 fix(mcp): resolve deleted self-image per platform ruling
Maintainer follow-up on top of the contributor's rebased commit,
implementing the Half-B strategy decision from the 2026-08-21 review:

- Linux: when the self-resolved path fails the executable check after an
  atomic rename-over, hand back the /proc/self/exe magic link. The link
  still executes the in-memory OLD build, which is the only spawn the
  worker's build-fingerprint gate accepts. Deliberately NOT the saved
  launch path: after a replacement that path holds a DIFFERENT build, so
  preferring it swaps an ENOENT for a fingerprint refusal.
- macOS: no magic link exists; fail closed. The supervisor logs
  index.supervisor.no_self_path and indexing resumes on daemon restart
  instead of spawning a missing binary. A rename-over that leaves a VALID
  binary still resolves (and the fingerprint gate refuses downstream, with
  its own coverage).
- The prefer-saved-launch-path branch from the original head is dropped
  per the same ruling.

Also fixes both Half-A review items:
- Aliasing in cbm_http_server_set_binary_path: `path` may alias
  g_binary_path on a repeated call. The overlapping snprintf cannot
  actually fire today — resolve's leading out[0]='\0' zeroes the shared
  buffer first — but that same reset means the call silently DISCARDS the
  stored path it was asked to re-resolve, and the in-place write is one
  reset-reorder away from real UB. Resolve into a local first: correct
  input, no aliasing trap.
- Coverage asymmetry: the probe and driver ran only on Linux while the
  production edit covered all of POSIX. Both now run on macOS too, and the
  Linux probe first proves the "(deleted)" state via readlink so its
  assertions cannot pass vacuously on an intact image.

Revert-check: with main's http_server.c restored, the driver fails at
WEXITSTATUS 44 (resolver returned a dead path) on macOS; with the guard it
passes 63/63.

Co-authored-by: sunmingqiang <sunmingqiang@bytedance.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 02:39:21 +02:00
sunmingqiang 11bbf0d847 fix(mcp): survive atomic binary replacement
Prefer the saved executable launch path when it still points to an
executable, and reject a deleted /proc/self/exe path before spawning
index workers.

Add a Linux integration regression that forks, atomically replaces the
running test binary on disk, and verifies that subprocess resolution
uses the saved launch path instead of the now-deleted self path.

Signed-off-by: David Musk <davidmusk93@users.noreply.github.com>
Signed-off-by: sunmingqiang <sunmingqiang@bytedance.com>
2026-08-28 02:15:19 +02:00
Martin Vogel 0f1e65d625 fix: preserve Cargo context in parallel Rust resolution
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-27 21:14:22 +02:00
Martin Vogel 7710f86e92 fix(daemon): avoid duplicate idle maintenance polling
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-27 20:32:19 +02:00
shashwatMishra96 3ffcbc9c3b Merge branch 'main' into fix/quarantine-exit-nonzero 2026-08-27 23:56:34 +05:30
Martin Vogel b0a3ca2861 Merge pull request #1105 from nguiaSoren/feat/335-watcher-enabled-config
feat(config): add watcher_enabled config key to disable the background watcher (#335)
2026-08-27 19:40:54 +02:00
Martin Vogel feb720809f Merge pull request #1104 from LA-10/fix/769-file-qn-keeps-extension
feat(store): index-format rebuild boundary for the File-QN change
2026-08-27 19:30:12 +02:00
shashwatMishra96 f599076fff Merge branch 'main' into fix/quarantine-exit-nonzero 2026-08-27 22:58:46 +05:30
Martin Vogel 095acc5cee Merge pull request #1854 from DeusData/feat/plsql-language
feat(lang): add Oracle PL/SQL support (distilled from #1033)
2026-08-27 19:17:50 +02:00
Martin Vogel eaf9b5278e Merge pull request #1855 from DeusData/feat/grok-build-client
feat(cli): add Grok Build as an install target
2026-08-27 19:08:54 +02:00
Martin Vogel ee32816e3c fix(pi): send tool arguments over stdin
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-27 18:51:36 +02:00