Commit Graph
3150 Commits
Author SHA1 Message Date
Martin Vogel 9b5bf70dc4 Merge branch 'main' into fix/test-daemon-bootstrap-sync-spawn 2026-09-24 04:51:00 +02:00
Martin Vogel 5958f546dc Merge pull request #2273 from DeusData/distill/1245-service-pattern-boundaries
fix(service-patterns): match library ids on identifier boundaries
2026-09-24 04:35:34 +02:00
Martin Vogel 5c0b241877 Merge pull request #2275 from DeusData/fix/win-endpoint-dir-create-race
fix(daemon): losing the runtime-directory creation race is not a failure on Windows
2026-09-24 04:35:31 +02:00
Martin Vogel 1160fa3591 Merge pull request #2274 from DeusData/distill/1245-c-pointer-return-types
fix(extract): a C-family function's return type keeps its pointer and qualifiers
2026-09-24 01:21:43 +02:00
Martin Vogel b0691e0d2f Merge pull request #1742 from bmcnaboe/codex/fix-daemon-runtime-sanitizer-timeout-only
fix(tests): scale daemon runtime sanitizer timeout
2026-09-24 01:21:38 +02:00
Martin Vogel 181c03dc18 Merge pull request #1741 from bmcnaboe/codex/fix-precommit-git-environment
fix(hooks): clear repository git environment for tests
2026-09-24 01:21:32 +02:00
Martin Vogel 95c91b8d8d Merge pull request #2140 from Lowpower/cursor/cli-help-format-json-cb65
feat(cli): point top-level cli --help at --format json
2026-09-23 23:53:21 +02:00
Martin Vogel e783f73d75 Merge pull request #2269 from DeusData/fix/ci-changes-large-diff-fallback
ci(pr): the changes job survives an un-renderable diff and no longer fails open on large file lists
2026-09-22 03:04:08 +02:00
Martin Vogel 8b57fe2a2b Merge pull request #2225 from tonydzi/docs/tool-count-17
docs: MCP tool count is 17; document get_file_outline, compare_graphs, check_index_coverage
2026-09-22 02:18:40 +02:00
Martin Vogel d1b2548ee8 Merge pull request #2270 from DeusData/docs/contributing-ai-contributions
docs(contributing): terms for AI-assisted and AI-authored contributions; who speaks for the project
2026-09-22 02:17:58 +02:00
Martin Vogel 9b7d63f579 Merge pull request #1931 from ilyabrykau-orca/fix/go-cgo-pseudo-import
fix(extract): drop cgo's `import "C"` pseudo-package
2026-09-22 02:00:11 +02:00
Lowpower 2f557f5398 style(cli): clang-format CBM_CLI_USAGE and help-text test
Signed-off-by: Lowpower <3133734+Lowpower@users.noreply.github.com>
2026-09-21 23:47:50 +00:00
Lowpower aaa72fbe60 feat(cli): point top-level cli --help at --format json
CLI_USAGE described outer --json but did not mention tool-level
--format json. Add a help-only pointer that tools which accept format
support --format tree|json, and distinguish payload JSON from the full
MCP envelope. No new flags, env vars, or output-default changes.

Fixes #2102

Signed-off-by: Lowpower <3133734+Lowpower@users.noreply.github.com>
2026-09-21 23:47:50 +00:00
Martin Vogel 28b4093c99 Merge pull request #1976 from CaptainMittens/fix/swift-nested-url-constructor
fix(extraction): reach a Swift URL built by a constructor
2026-09-22 01:46:34 +02:00
Martin Vogel dfae618b0e Merge branch 'main' into codex/fix-daemon-runtime-sanitizer-timeout-only 2026-09-22 01:28:18 +02:00
Martin Vogel c0bfd073ff Merge branch 'main' into codex/fix-precommit-git-environment 2026-09-22 01:28:15 +02:00
Martin Vogel a041794f8e fix(daemon): losing the runtime-directory creation race is not a failure on Windows
Several processes first-starting together against a runtime directory
that does not exist yet -- test-windows-guards' section_cold_storm, or a
host that launches more than one MCP server on its very first run -- all
observe the final path component absent. One CreateDirectoryW wins; the
others get ERROR_ALREADY_EXISTS. win_private_directory_tree_secure()
treated that as a failure and recorded no validation detail, so every
loser exited with a bare

    secure CLI coordination could not be created (endpoint)

The function called right after the walk, win_runtime_directory_secure(),
has always tolerated ERROR_ALREADY_EXISTS for the same directory, and the
POSIX walk tolerates EEXIST at the same point; only the Windows walk in
front of them did not.

Seen four times on 2026-09-21 on unrelated PRs (#1768 three attempts in
a row, #2140, #1999, #808). The racy lines date from July; the guard
began exercising them on 2026-09-03, when each guard section was given
its own empty CBM_RUNTIME_DIR, so the final component is now absent at
storm time on every run.

The walk now treats ERROR_ALREADY_EXISTS from its own CreateDirectoryW as
"the directory I wanted exists". Nothing is trusted because of that: an
ancestor still goes through win_directory_component_secure(), and the
final component through win_runtime_directory_secure(), which refuses a
non-directory or reparse point and enforces owner and DACL.

Every refusal on this path now names its component and its rule. The
walk reports the Windows error when it can neither create nor inspect a
component, and win_runtime_directory_secure() says whether the path
could not be created, cannot be inspected, exists but is not a directory,
or is a reparse point. One helper owns the wide-to-UTF-8 conversion for
these messages and the existing ancestor message now uses it too, so
src/daemon/ipc.c stays at its memory-core baseline.

Deterministic reproduction, no threads and no timing: a test seam fires
in the walk between "component observed absent" and CreateDirectoryW,
and the test plays the process that wins the creation.

  daemon_ipc_windows_private_directory_survives_lost_creation_race
  daemon_ipc_windows_private_directory_refuses_and_names_a_planted_file

Verification. macOS arm64 (ASan+UBSan): build clean, daemon_ipc 53 passed
(2 Windows-only skips), daemon_bootstrap 28 passed, lint-memory-core
unchanged (ipc.c stays at 146 raw sites). The two new tests are Windows-only
and have NOT been run locally: the Windows VM was down when this was written,
so the RED run on the seam-only tree and the GREEN run with this fix are
delegated to CI's windows-latest legs (test-windows shards, test-windows-guards)
by an explicit maintainer decision. If the guard suite's cold storm still
fails with this in place, the attribution above is wrong and this reverts.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-22 01:12:51 +02:00
Martin VogelandAndrew Hundt 343d7dd5e4 fix(extract): a C-family function's return type keeps its pointer and qualifiers
Distilled from #1245 by Andrew Hundt (c51e5cb0, the extract_defs.c
half; the mcp.c and token-reduction parts are not taken).

For C-family functions return_type was the text of the declaration's
`type` node alone. In these grammars the `*`, `&` and `&&` live on the
declarator and cv-qualifiers are sibling nodes, so

    const char *get_name(void)     published   char
    robj **lookupKeys(...)         published   robj
    Text &operator=(const Text &)  published   Text

and no C or C++ function in any index had a pointer return type at all.
The C/C++ resolver's return-type parser already strips a leading
const/volatile and handles trailing `*`, `&`, `&&` -- branches that were
unreachable for functions, so every `Foo *f()` was registered as
returning Foo by value.

c_declared_return_type() renders the declared type at both sites
(functions and methods) for C, C++, CUDA, GLSL, HLSL, ISPC, Slang and
Objective-C -- the list resolve_func_name_c_family uses:

    <leading cv-qualifiers, source order> <base type, verbatim>
    <one space><markers, outermost first>

so `const char *`, `char **`, `Text &`, `Text &&`, `Text *&`,
`volatile unsigned long *`; a pointer-level qualifier follows its star
(`char *const *`); east-const `char const *` normalises to
`const char *`. With no qualifier and no marker the old text is returned
unchanged. It measures, then writes into one exact-size arena
allocation; the declarator walk is a plain child chain, no recursion.
def.return_types is untouched.

Two deliberate differences from the upstream hunk. Upstream copied every
type_qualifier node, which in these grammars includes constexpr,
_Noreturn, mutable and __extension__ -- `constexpr int f()` would have
become `constexpr int`. Only const, volatile, restrict, __restrict,
__restrict__ and _Atomic are kept, and both cases are asserted unchanged.
And the walk stops at the first declarator that is neither pointer nor
reference, so `int (*f(void))(int)` stays `int` rather than becoming a
wrong `int *`.

RED, production reverted and the final tests kept:
  c_function_return_type_preserves_pointer_and_qualifier
      FAIL tests/test_extraction.c:1005: "char" != "const char *"
  cpp_method_return_type_preserves_pointer_and_qualifier
      FAIL tests/test_extraction.c:1072: "char" != "const char *"
  351 passed, 2 failed
c_function_return_type_plain_unchanged is the control: it passes before
and after.

GREEN: extraction 353, c_lsp 762, lang_contract 41, pipeline 281.

Effect on real code -- Redis, production binaries, old against new.
39,132 nodes in both. 1,442 Function nodes change, in return_type only:
void -> void * (270), char -> const char * (101), unsigned char ->
unsigned char * (91), char -> char * (81), robj -> robj * (57), ...
Functions with a `*` in return_type: 0 -> 1,440. With one worker the
complete edge set apart from SEMANTICALLY_RELATED is byte-identical old
against new across six runs (three each) -- CALLS, USAGE, WRITES, IMPORTS
and the rest do not move. SEMANTICALLY_RELATED goes 350 -> 325 because
that pass tokenises the return-type string into its type vector.

One thing measured rather than assumed. With several workers, which
`dict.h` Redis' src/dict.c imports (src/ or deps/hiredis/) depends on
worker merge order. That is main's defect, not this change's: unmodified
main picked src/dict.h in 2 of 10 runs and the hiredis header in 8; with
one worker both binaries always pick src/dict.h. This change picked
src/dict.h in 6 of 10. At ten runs a side that gap is not separable from
chance (Fisher exact, two-sided p = 0.17), and a shifted rate would be
unsurprising for a timing-dependent pick; it is recorded here rather
than explained. The include-target fix is a separate change (#2227).

Known limits, all pre-existing and out of scope: struct and class FIELDS
have the same defect (`const char *data;` publishes char); C prototypes
and declaration-only C++ methods are not extracted as definitions, so
they are not reached; where a macro is misparsed as the type
(`LUA_API const char *f()`), the base was already wrong and now reads
`const LUA_API *`; the resolver's parser does not yet understand a
pointer-level qualifier or a leading _Atomic/restrict and falls back to
a named-type lookup for those, as it effectively did for every pointer
return before.

Co-authored-by: Andrew Hundt <ATHundt@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-22 00:48:44 +02:00
Martin VogelandAndrew Hundt c314dea4c1 fix(service-patterns): match library ids on identifier boundaries
Distilled from #1245 by Andrew Hundt (d15071c9, the match_qn half).

match_qn was a raw strstr(qn, library_id). Short ids fired inside
unrelated words, and a service kind REPLACES the plain CALLS edge, so a
false match both invents a service edge and deletes a real call:

  "gin."    in plugin.           -> route registration; any "/"-prefixed
                                    first argument then minted a Route node
  "nconf"   in encryptionconfig  -> CONFIGURES for every call into the
                                    package
  "tonic"   in Monotonic         -> gRPC; unparseable, so NO edge at all
  "express" in expression        -> route registration
  "get_env" in get_envelope, get_environ -> CONFIGURES
  "dio" "surf" "curl" "resty" "phin" "hyper" "treq" "rocket" in studio,
  surface, curly, restyle, dolphin, hyperbolic, streq, rocketmq

The rule (qn_hit_on_boundary; every occurrence of an id is tried, not
only the first). A separator is any character that is not an ASCII
letter or digit.

  before: start of string | separator | the id itself starts with a
          separator ("@trpc/server") | the id starts with an uppercase
          letter -- a capital opens a new CamelCase word whatever
          precedes it (AsyncHttpClient, IHttpClientFactory, NSURLSession)
  after:  end of string | separator | the id itself ends in a separator
          ("gin.") | uppercase letter (GuzzleHttp, FeignClient,
          KafkaProducer) | digit (urllib2, Mint.HTTP2, amqp091-go)

Rejected: a lowercase- or digit-initial id glued to a preceding letter
or digit, and any id continued by a lowercase letter. Two deliberate
differences from the upstream hunk: upstream's before-rule needed a
GuzzleHttp table entry and would have dropped IHttpClientFactory and
NSURLSession, and without the digit after-boundary urllib2 and
amqp091-go stop matching. All three match on main today.

No recall is traded for this. No boundary rule can tell "grequests"
from "myrequests", so a library whose own name glues a prefix or suffix
onto another id gets an explicit entry with the kind (and broker) it had
on main -- 47 of them, found mechanically by running main's matcher and
the new one over 173 candidate names and listing every one that went
from a kind to NONE:

  HTTP      grequests txrequests redaxios gaxios libcurl curlpp curlcpp
            hyperlocal guzzlehttp
  ASYNC     aiokafka pykafka rskafka librdkafka rdkafkacpp aioamqp
            amqpstorm pamqp pyamqp amqprs amqpcpp pynats jnats aiomqtt
            amqtt hbmqtt umqtt mqttools emqtt rumqtt gomqtt libmosquitto
            mosquittopp pubsublite gocelery
  CONFIG    wgetenv getenvb qgetenv dotenvx phpdotenv
  ROUTE_REG apiflask honox fasthttprouter
  GRAPHQL   gqlparser gqlgenc aiogqlc

Left at NONE on purpose: names that are not clients of the id they
contain and were misclassified on main -- hypercorn, hyperlink,
hyperopt, openresty, gqlalchemy, celeryconfig, kafkacat, gnatsd,
pypubsub. One visible reclassification: hyperium/tonic was HTTP through
"hyper" inside "hyperium" and is now gRPC through "tonic".

RED, production reverted and the final tests kept:
  infrascan_service_pattern_match_rejects_ids_inside_words
      FAIL tests/test_infrascan.c:105:
      svc_case_mismatches(cases) == 28, expected 0 == 0
  4 passed, 1 failed
All 28 negatives mismatch on main. The 47 glued-library positives are
RED the other way round: against the boundary rule without the entries.
Each positive uses a name containing no second id, so it binds to its
own entry -- "aiokafka.AIOKafkaProducer" would have passed through
"KafkaProducer" and proven nothing.

GREEN: infrascan 5, pipeline 281, parallel 74, extraction 350,
edge_types_probe 59, route_canon 11, lang_contract 41, cross_repo 8,
registry 64, mcp 318 (4 Windows-only skips). 0 failed.

Effect on real code -- production binaries, main 92abefa3 against this
change. main against main differs by 0 edges and 0 nodes on all three
corpora, so every delta below is the change:

  django      CONFIGURES 76 -> 74; both calls return as CALLS
              (geom.get_envelope, WSGIRequestHandler.get_environ)
  typescript  CONFIGURES 12,026 -> 12,024; one bogus Route "/*type*/"
              gone with its CALLS and HANDLES; 3 calls return as CALLS
  kubernetes  CONFIGURES 7,962 -> 7,609: all 353 are "nconf" inside
              encryptionconfig / authorizationconfig /
              authenticationconfig, and all 353 return as CALLS.
              8 bogus Routes gone -- os.ReadFile of the serviceaccount
              namespace file, t.Run("/TwoWay"), "/tmp/test" -- with 26
              CALLS, 2 HANDLES and 21 TESTS edges that pointed at them.
              374 plain CALLS restored; 8 of them (validate.Monotonic and
              friends) had NO edge at all on main. GRPC_CALLS 60 -> 60.
              HTTP_CALLS 398 -> 407: nine calls main swallowed as route
              registrations now reach the existing arg_url heuristic like
              every other package.

88 of the 408 removed service-derived edges were read at the call site
-- all of django and typescript, 82 of 402 in kubernetes: 88 false
positives, 0 true positives lost. None of the three corpora uses a
glued-name library, so the 47 entries are covered by the unit test only.

Known limits: "_" is a separator, so optical_fiber.len still matches
"fiber." exactly as requests_get matches "requests" by design; a
lowercase id followed by a capital matches (kafkaProducer), which also
admits a dioXide-style name; a user wrapper glued without a separator
(mygetenv) is no longer classified; a glued-name library missing from
the list is NONE until someone adds it.

Co-authored-by: Andrew Hundt <ATHundt@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-22 00:48:43 +02:00
Martin Vogel da4bd25a01 fix(test): the ENOSPC bootstrap test no longer lets a deadline decide its verdict
daemon_bootstrap_fails_fast_when_daemon_dies_at_publication went red on
the MSan lane of three PRs that touch no daemon code at all (#2245,
#2158, #2140) with

    tests/test_daemon_bootstrap.c:1116: ASSERT(daemon_named_cause)

and green on a rerun of the identical commit.

Cause, attributed 2026-09-20. The test forks a real daemon host whose
record publication fails with ENOSPC, and asserts that the client names
that cause instead of waiting out its 30 s startup deadline. But before
it listens, the host SHA-256s its own executable image, and the test
runner is ~450 MB: ~3 s natively, 20-32 s under MSan (48 CI samples,
median 25.8 s). The spawn override returned as soon as it had forked, so
the client's 30 s clock raced the host's hash. On a slow run the
deadline fired first, the reaper SIGKILLed a host that had not failed
yet, and the log had no listen_failed line to find. Nothing was wrong
with the product: with a 28 s artificial delay the client does learn the
recorded ENOSPC cause.

The spawn override now reaps the forked host before it returns. The
host writes its start-failure record and only then releases its lifetime
reservation and exits, so "record on disk, reservation released" is a
stable state pinned by construction when the client takes its first look
-- and that first look is guaranteed: the client's post-spawn wait loop
runs at least once even past the deadline, and its first failure check
after a spawn is unthrottled. How long the host takes no longer matters.
No production code changes; the assertions are unchanged.

A regression to the pre-#1828 behaviour (a client that ignores the
record) still fails the unchanged assertions: its message no longer
names the failure, the errno and the path.

Proof, one build with temporary env-switched instrumentation that is
not part of this commit (a sleep in the forked host before it starts,
and a switch back to the old fire-and-forget spawn):

  spawn          forked host delayed   daemon_bootstrap suite
  synchronous    no                    28 passed
  old (async)    31 s                  27 passed, 1 failed:
                                       ASSERT(log_read)
  synchronous    31 s                  28 passed

The old spawn fails under the delay one assertion earlier than CI did
(log_read, not daemon_named_cause): the injected sleep sits before the
host opens its log, whereas in CI the host was killed mid-hash with the
log already open. Same mechanism -- the client's deadline beating the
host -- and the 2026-09-20 attribution reproduced the CI message itself
with a delay placed after the log is opened.

The final tree, without the instrumentation: daemon_bootstrap 28 passed,
three runs in a row.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-22 00:48:20 +02:00
Martin Vogel bf4c476d5e Merge pull request #2227 from DeusData/fix/graph-quality-2026-09-16
fix(graph): quality round after the v0.11.0 probe — deterministic resolution, findability, coverage, names
2026-09-22 00:46:52 +02:00
Martin Vogel 65d60e72f5 ci(pr): the changes classifier no longer fails open on a large file list
Found while proving the previous commit, in the same step, and the worse
of the two defects because it fails OPEN. The classification was

    if printf '%s\n' "$FILES" | grep -qE '^(src/|internal/|...)'; then

under the runner's `bash -eo pipefail`. grep -q exits on its first match;
once the list is larger than the pipe buffer printf is still writing,
dies of SIGPIPE, and pipefail turns the MATCH into a failed pipeline. The
else branch then writes product=false, and pr-smoke and memwaste are
skipped -- on exactly the PRs that change the most files. The previous
commit's message flagged this as a follow-up; it is fixed here instead.

The list now reaches grep through a here-string, so there is no pipe to
break. The regex is unchanged.

Proof, by executing the workflow's own run block under
`bash -eo pipefail` with a stub gh, ten runs each:

  849 KB list, first line src/a.c   before: product=false 10/10
                                    after:  product=true  10/10
  849 KB list, docs only            product=false before and after
  small docs-only list              product=false
  422 fallback on the real #2246    product=true
  fallback, nonexistent merge ref   rc 128, no output written

It takes roughly 1,500 changed files to cross the buffer, which is why
ordinary PRs never showed it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-21 20:44:19 +02:00
Martin Vogel a262a35724 ci(pr): classify changed files from the merge ref when the files API 422s
The `changes` job decides whether a PR touches product code, and ci-ok
requires it. It asks GitHub for the PR's file list. An earlier fix moved
it off the .diff endpoint, which rejects PRs over 20k lines, onto the
paginated pulls/files endpoint -- but that endpoint renders the diff
server-side as well, and for a PR that regenerates a vendored parser it
answers

    HTTP 422: Sorry, this diff is taking too long to generate.

Observed on #2246 (a 42 MB sql/parser.c): `changes` red, therefore ci-ok
red, for a reason that has nothing to do with the contribution. The
compare endpoint fails the same way, so there is no API route to the
file list for such a PR. Every grammar refresh would hit this.

When the API call fails, the job now takes the list from git instead.
The PR merge ref's first parent is the base, so a name-only diff across
that one merge commit is exactly the PR's changed files. The fetch is
depth 2 with --filter=blob:none into a bare repository under RUNNER_TEMP:
commits and trees only, no blobs, nothing checked out and nothing from
the PR executed. No new action, no new permission, and the API path and
the classification regex are untouched, so ordinary PRs behave as before.
A ::notice:: line records when the fallback was used. If the fetch fails
too, the step exits non-zero and writes no output -- the gate fails
closed, as it did before.

Proof, by executing the workflow's own run block under
`bash -eo pipefail` with a stub gh:

  API path, docs-only list        rc 0    product=false
  fallback, real #2246            rc 0    product=true, the same five
                                          files `git diff --numstat`
                                          reports for the PR; 0.85 s,
                                          188 KB fetched
  fallback, nonexistent merge ref rc 128  no output written

Contract tests that read pr.yml -- security_gate_fail_closed,
smoke_fixture_contract, windows_bundle_contract, venue_parity_contract --
pass.

Not addressed here, noted for a follow-up: the classification pipes the
list into `grep -q` under pipefail, which can report a match as a
failure once the list outgrows the pipe buffer.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-21 20:44:19 +02:00
Martin Vogel f428d58300 docs(contributing): state the terms for AI-assisted and AI-authored contributions
CONTRIBUTING.md said nothing about contributions written by or with an
AI agent, and the project now receives both kinds: PRs from accounts
that openly identify as an agent acting for a person (#2225 is one, and
a good one), and, in September, an account that posted approvals and
change requests styled as maintainer decisions across nineteen threads.
The first deserves a clear welcome; the second needs a rule that was
until now only enforced, never written down.

The new section says: disclosed AI authorship is welcome and never held
against a change; the DCO sign-off is a legal certification only a
person can make, so a human is accountable for every commit; the
evidence bar is the same for everyone -- a claim is something a reviewer
can check. Co-authored-by trailers naming a tool are fine, links to
private agent sessions are not. High-volume low-effort submissions are
closed whoever produced them.

A second subsection states who speaks for the project -- the maintainer
account only -- and that impersonating a maintainer, by a person or an
agent, gets the reviews dismissed, the comments hidden, a public
correction in the affected threads, and the account reported and
blocked. That is what was done in September; it is now written where a
contributor can read it first.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-21 20:44:17 +02:00
Martin Vogel df83d25ace fix(extract): drop the parse CPU reading nothing reads any more
Fallout of rebasing this branch onto main. This branch was written while
the LSP walk could be skipped for a file whose parse had used more than
its share of the budget, and that share was measured in the thread's CPU
time. Main has since removed the rule entirely -- no clock decides what
ends up in the graph -- so cpu_start_ns is gone from the function and the
only line still reading it was computing a value with no remaining
consumer.

The conflict resolution kept main's side in cbm.c, which is what leaves
this line stranded: it is dead by then, and the build says so.

    error: use of undeclared identifier 'cpu_start_ns'

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-21 20:29:54 +02:00
Martin Vogel 24afb562dc fix(graph): route the new scratch through the memory core
The lint chain stops at its first red target, so a clang-format failure hid the memory-core linter
until PR CI: the coverage line-offset table, the Cypher direction swap and the search label each
added a raw allocator site. The table now comes from cbm_alloc/cbm_free (extract class), the swap
frees through safe_str_free, and the BM25 row array moves to cbm_calloc/cbm_free so mcp.c's raw-site
count goes down (baseline 814 -> 813) rather than up.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-21 20:29:54 +02:00
Martin Vogel 1a3425536b fix(graph): quality round after the v0.11.0 probe — deterministic resolution, findability, coverage, names
Nine findings from the 2026-09-16 graph-quality probe over the eleven bench corpora, each with a
test that fails without its fix and a before/after measurement on the real corpus.

Deterministic resolution (O9). Two indexes of one kernel tree differed in 5,821 IMPORTS and 632
CALLS edges. Three order/load dependencies carried it: the `#include` target was the first by-name
hit whose path ends with the include path — a bucket in worker merge order (include/linux/device.h
vs tools/virtio/linux/device.h); the registry broke suffix_match score ties by bucket position; the
LSP budget-share rule measured wall time. Include targets now prefer the File node, the least nested
path, then the lexicographic path; registry ties prefer the least nested QN, then the smaller QN; the
budget share is measured in thread CPU time like the budget itself. Two kernel indexes are now
identical (8,529,723 nodes, 1,974,940 CALLS, zero churn), as are two elasticsearch indexes.

Findability. search_graph's BM25 path honours the `label` filter and ranks an exact-name definition
above test methods that repeat the token (15/15 probe queries).

Coverage. The parse-coverage gap tree subtracts definitions the walker recovered inside an error
region (multi-line only); kernel/sched/core.c reports 476 flagged lines instead of 7,674.

Precision. Config-language callers (Makefile, CMake, YAML) no longer bind unique_name into code
(kernel Makefile -> C CALLS 195 -> 0). JS-family token names ({}, 1, (x: T) => U) are not
definitions; computed, quoted and escaped member keys stay (TypeScript 204 junk -> 0, 1,782 real
names kept). Names are cut at their first line (elasticsearch YAML 8,306 -> 0); a comment inside an
argument list is not an argument, so no Route node is named by a Java block comment; the wildcard
route `/*` is still a route. HTML/Vue/Svelte/Astro embedded member calls stay unbound.

Recall. A Python call through a self- or cls-rooted attribute keeps a specific unique_name member
target (Django +751, elasticsearch +3,288 CALLS, no new callee names).

Robustness. Edge properties never carry a split UTF-8 sequence (the 120-byte expression cut and the
JSON escaper both trim a partial code point). CLI array flags accept a JSON-array literal. Cypher
seeds a single-hop pattern from its selective end (kernel `MATCH (a)-[:CALLS]->(b) WHERE b.name =
'_printk'`: execution limit -> 7.9 s).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-21 20:29:27 +02:00
Martin Vogel d615088a3c Merge pull request #2259 from DeusData/carry/2051-atomic-publish-errno
fix(graph-buffer): log why an atomic publish failed instead of a successful dump
2026-09-21 20:23:03 +02:00
Martin Vogel b82cf7b1a3 Merge pull request #2260 from knewstimek/fix/windows-aligned-mimalloc
fix(windows): keep aligned allocations on mimalloc
2026-09-21 20:22:09 +02:00
Martin Vogel c7488b031c Merge pull request #2005 from XIYBHK/fix/1989-export-macro-parsing
fix(parsing): recover export-macro-hidden type definitions (#1989)
2026-09-21 20:22:03 +02:00
Martin Vogel 9b10337907 Merge pull request #1809 from Sakil786/fix-pi-extension-typebox-schemas
Fix Pi extension to use TypeBox schemas instead of JSON
2026-09-21 20:21:50 +02:00
Martin Vogel a7187ad250 Merge pull request #2075 from halindrome/fix/watcher-index-failure-backoff
fix(watcher): back off re-forking a hard-failing index worker
2026-09-21 20:21:36 +02:00
Martin Vogel 6f2f836181 Merge branch 'main' into carry/2051-atomic-publish-errno 2026-09-21 07:40:26 +02:00
Martin Vogel 7d01d3ec03 Merge branch 'main' into fix/1989-export-macro-parsing 2026-09-21 07:40:07 +02:00
Martin Vogel e321b1fcea Merge branch 'main' into fix/watcher-index-failure-backoff 2026-09-21 07:40:04 +02:00
Martin Vogel 19d9e4ac4d Merge branch 'main' into fix-pi-extension-typebox-schemas 2026-09-21 07:40:01 +02:00
Martin Vogel 2dba3cf817 Merge branch 'main' into fix/windows-aligned-mimalloc 2026-09-21 07:39:55 +02:00
Martin Vogel def38f3ebc Merge pull request #2257 from DeusData/fix/mcp-index-args-cleanup-helper
fix: restore main to green after #1723 — memory-core linter and worker-scope Step 5f
2026-09-21 07:39:26 +02:00
Martin Vogel c1128db818 fix(test): the worker-scope script sends the policy a supervisor sends
main fails scripts/test.sh at Step 5f on every platform:

    === Step 5f: worker request-scope regression ===
    worker did not index the admitted repository
    {"content":[{"type":"text","text":"missing or incomplete trusted
    worker policy"}], ... "isError":true}

Second collision from the #1723 merge (92abefa3), and the same kind as
the first. #1723 made an index worker refuse any request that does not
carry `_cbm_index_policy`: a worker never resolves the resource policy
from config or environment, the supervisor does and sends it along.
tests/test_worker_session_scope.sh, added on 2026-09-06 by a08b9ecb,
plays the supervisor by hand -- it runs `cli --index-worker
index_repository '{"repo_path":...,"mode":"fast"}'` directly -- and
#1723's branch never contained that script, so its CI never ran the two
together. The merge was textually clean and the result was wrong. It
stayed hidden behind the lint failure the previous commit fixes, because
the test stage only runs once lint is green.

The worker is right and the script was incomplete: it now sends what the
supervisor sends, both limits off, which is the default
(`"index_max_files":"off","index_max_source_mb":"off"`, the same object
tests/test_index_policy.c uses). The fail-closed case carries it too, so
that a request without repo_path is still refused for its missing scope
("request workspace scope invalid") and not, by accident, for a missing
policy. No assertion changed.

RED, production binary built from this branch, script unmodified:
    worker did not index the admitted repository
    ... "missing or incomplete trusted worker policy" ...      rc=1
GREEN:
    ok: index worker is scoped to the admitted request, not the daemon
    environment                                                  rc=0

On the failed CI run of this PR, 12 of the 13 red test jobs carry exactly
that signature and nothing else. The 13th, test-windows-guards, is
test_daemon_stability.py section_cold_storm ("secure CLI coordination
could not be created (endpoint)", a racing cold start) -- a known
nondeterministic guard, unrelated to either commit here. Step 6, which
Step 5f's failure kept CI from reaching, passes locally:
security-strings allow-list 4 passed, destructive-ordering contract PASS.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-20 21:40:37 +02:00
News 25d08258e5 fix(windows): keep aligned allocations on mimalloc
Signed-off-by: News <knewstimek@users.noreply.github.com>
2026-09-21 01:37:51 +09:00
Martin Vogel af3e003ad8 fix(mcp): one release point for handle_index_repository's argument strings
main fails the memory-core linter:

    src/mcp/mcp.c: grew by 3 (814 -> 817)

PR #1723 (merged as 92abefa3) added a new early-return path to
handle_index_repository and, following the function's convention, freed
its three argument strings there with raw free(). Its CI lint was green
on 2026-09-09. On 2026-09-19 the memwaste merge cut this file's raw sites
and ratcheted the baseline down to 814, so by the time #1723 landed its
green was measured against a rule that no longer existed. The linter is
right and the merge-result check that let it through was incomplete: it
built and ran the suites and did not run the linter. It does now.

The fix is not cbm_free. The three strings come from
cbm_mcp_get_string_arg and resolved_repo_path_from_project_arg and are
raw heap memory, released raw by every other path in the function;
routing three of them through the core would free raw memory through the
accounting layer. The actual defect is that the function had 41 raw
free() calls of the same three arguments across a dozen early returns,
and #1723 added one more block in the same shape.

One helper, index_args_free(repo_path, mode_str, name_override), now
replaces twelve of those blocks. free(NULL) is a no-op, so the one path
where repo_path is still NULL passes NULL. On the cross-repo branch
mode_str used to be freed before handle_cross_repo_mode and the other
two after it; mode_str is neither passed to nor read by that call, so
releasing all three after it is not observable. Five sites on the
normal-completion path are left as they were: they free the strings at
different points as ownership is handed to the pipeline, and collapsing
them would change that order for no gain.

Raw sites in this file: 817 -> 785. The baseline follows the improvement,
814 -> 785, as the linter asks; it can only tighten.

Build: clean, -Werror, ASan/UBSan. mcp cli daemon_application:
692 passed, 4 skipped, 0 failed -- index_repository_honors_allowed_root,
which runs through a replaced early return, included.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-20 18:15:33 +02:00
juan Carlos Estrada MontoyaandClaude Opus 5 460ebefc87 fix(graph-buffer): log why an atomic publish failed instead of a successful dump
cbm_gbuf_dump_to_sqlite() emitted gbuf.dump regardless of the writer's
result, so a run that published nothing still logged node and edge counts
as if it had worked, and the failure reached the user only as the generic
"Pipeline failed. Check repo_path exists and contains source files."

The writer's temp -> final rename in publish_writer_output() returned a
bare ERR_WRITE_FAILED. #1628 taught cbm_rename_replace() to translate the
platform error into errno so callers could report why, and wired up the
stage -> final rename in pipeline.c. This is the other rename on that
path, and it is the one that runs first.

Preserve errno across the cleanup in every publish error branch. That
includes cbm_writer_open(), where the cleanup unlinks a file that was never
created: without the save it leaves ENOENT behind, so a directory that
denied the create is reported as a missing path — the #1620 case, stated
wrongly. Read the value in graph_buffer.c before the profiling macro can
clobber it, and report gbuf.dump_failed with the return code and that errno.
A sticky append failure clears errno rather than attach a reason it does not
have, and the field is omitted when there is none. No logging is added
inside internal/cbm/, which has none today.

Tests cover both halves: the dump reports a failure instead of a summary,
the writer names its truncation reason, and the publish rename's errno
survives a cleanup that itself fails.

Refs #1620, #2001.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012FxkVF773gDGNmBmsLnzN7
Signed-off-by: juan Carlos Estrada Montoya <junk151516@users.noreply.github.com>
2026-09-20 17:52:45 +02:00
XIYBHK fcf451c6a6 fix(parsing): recover export-macro-hidden type definitions (#1989)
Collect a bounded set of conventional export macro candidates and inject empty definitions into the existing C/C++ preprocessing second pass without overriding explicit caller definitions.

Conservatively reconcile remapped definitions to recover hidden classes, structs, enums, free functions, and inline methods while suppressing matching base-class phantom callables.

Add focused regression coverage for supported suffixes, ordinary all-caps negative controls, candidate limits, comments, strings, raw strings, overlong names, explicit define priority, and C/C++ extraction.

Rebase onto main 92abefa3 and route the temporary superseded-definition bitmap through the extraction memory class to comply with the current allocation contract. Preserve the error-region dropped counter initialization.

Signed-off-by: XIYBHK <xiybhk@163.com>
2026-09-20 23:36:54 +08:00
Martin Vogel 92abefa3f5 Merge pull request #1723 from liuchong/feat/index-resource-discovery
feat(index): add opt-in discovery resource limits
2026-09-20 17:17:47 +02:00
Martin Vogel 9b6c99e794 Merge pull request #2230 from yasmoradi/fix/dotnet-xml-extensions
fix(discover): map MSBuild, resx, XAML and app-manifest files to XML
2026-09-20 17:04:36 +02:00
Martin Vogel 5b4be65b75 Merge pull request #2203 from DavidHLP/fix/issue-2175
fix(cli): handle trusted Linux system aliases during install activation
2026-09-20 17:03:28 +02:00
Martin Vogel 36f770842a Merge pull request #2248 from DeusData/fix/1714-freshness-merge-loss
test(mcp): ask for JSON where the coverage test asserts a JSON field
2026-09-20 17:02:18 +02:00
Martin Vogel 96a2fc35df test(mcp): ask for JSON where the coverage test asserts a JSON field
#1787 landed correct production code and a test that cannot pass on today's
main. The test is the only thing wrong, and only in how it asks.

check_index_coverage answers in the compact table by default, where a field
name never appears:

    main.go main.go coverage_unavailable metadata_match read_source_and_reindex []

The test asserts on the JSON field spelling:

    strstr(response, "\"freshness\":\"metadata_match\"")

Both halves are right; they were simply written against different response
formats. The branch was cut on 2026-09-02, before that default changed, and
was never rebased -- its CI green is from a main that no longer exists. The
neighbouring coverage tests already pass "format":"json" for exactly this
reason; this one predates them.

So the request gains format=json and the assertions are untouched. Asserting
the field rather than a bare strstr for "metadata_match" is what keeps the
check exact: the value also appears in the table's recommended_action column
and would be matched by any other path's row.

Verified: build/c/test-runner mcp -> 318 passed, 0 failed, 4 skipped. Before
this change the same suite reported 317 passed, 1 failed on identical
production code.

What this is NOT: a production defect. coverage_path_freshness on main uses
cbm_path_info_utf8, exactly as #1787 intended, and the Windows second-vs-
nanosecond truncation it fixes is gone. I checked that by reading origin/main
rather than the local checkout, which was stale at 59a05eb1 and still carried
the old helper -- that misreading briefly convinced me the merge had dropped
the fix, and it had not.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-09-20 14:07:12 +02:00
Martin Vogel f347d59650 Merge pull request #1787 from umi008/fix/issue-1714-freshness-mtime
fix(mcp): report freshness with the indexer's own mtime source (#1714)
2026-09-20 13:30:50 +02:00
Martin Vogel d6082f2025 Merge pull request #2002 from metehanulusoy/fix/cypher-nonnumeric-limit-silently-dropped
fix(cypher): reject a non-numeric SKIP/LIMIT operand instead of dropping it
2026-09-20 13:27:40 +02:00