The doc<->code candidate scoring (#2574, doc_section_worker in
semantic_edges) scores every documentation section against the functions
that share a key term with it, key terms being those held by at most
funcs/20 functions. Its cost therefore grows with sections x functions.
Measured on 11 corpora (10-09 main vs 4d9592b5, semantic_edges): under a
second up to ~100k functions, +12.6 s on java (325k), +14.6 s on csharp
(451k), +174 s on the Linux kernel (784k functions, 28k sections), almost
all of the kernel's regression against v0.11.0.
A new boolean setting, doc_candidates (default true), and an optional
index_repository argument of the same name switch the scoring off: no
candidate rows are stored and nodes and edges are unchanged. The argument
wins over the setting. The index handler resolves the value once and
passes it to a daemon job or supervised worker as an argument, the same way
as the resource policy, so every route sees one decision; the session
auto-index reads the setting. With the switch off, a delta-route reindex
replaces the candidate rows it would otherwise keep, so turning it off
leaves no stale candidates. The default output is unchanged.
Tests (each fails on its assertion when the switch has no effect):
- pipeline_doc_candidates_off_stores_none_and_keeps_graph: no rows, same
node and edge counts as the default run, which has rows;
- pipeline_doc_candidates_off_clears_rows_on_delta: a closure-repair
reindex with the switch off removes the earlier rows;
- tool_index_repository_doc_candidates_switch: argument false -> none;
setting default -> rows; setting false -> none; argument true over
setting false -> rows.
Proof, release-built binaries on macOS arm64 (M5 Pro 18c/48 GB):
- django, main 4d9592b5 vs this change with defaults: nodes (63,441),
edges (307,584) and doc_link_candidates rows (11,304) hash-identical;
- django with doc_candidates false: nodes and edges hash-identical, no
candidate rows;
- Linux kernel with doc_candidates false: semantic_edges 17.9 s (191.9 s on
main, 17.6 s before #2574), whole index 210.9 s (v0.11.0: ~234 s), no
candidate rows.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Discovery reads the tracked-file set through the supervised subprocess
API instead of cbm_popen, so no shell parses the command:
- git is resolved to an absolute path from absolute PATH entries only
(cbm_test_impact_find_git), never the current directory;
- the repository path is a single argv entry;
- the child environment drops repository-local variables, so an
inherited GIT_DIR cannot redirect -C;
- core.fsmonitor is off for the call;
- stdout goes to a private mkstemp file that is read and removed.
discover.c needs no entry on the security allow-list.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The 2026-10-12 dry run (run 38186491924, windows-11-arm 2/2) killed
doc_mentions_go at the 900 s wall clock. The test framework prints a test's
name before running it, so the last line shows the hang was in
doc_mentions_go_local_rules, before the pipeline's first info line
(pipeline.discover). Nothing in the log says which step. The hang has not
come back: 0 of 32 runs on a local Windows arm64 VM, and the next CI
sample passed (run 38204190338, 80 s).
The steps before discovery have no deadline: the fixture's temp
directory, the fixture writes, three `git` children per index (git context
in cbm_pipeline_new and again in the run), staging, the coverage check and
the test config. This change makes each of them name itself at debug level,
and the Go suite's fixture index runs at debug level, so a recurrence ends
on the step it is stuck in:
- git_capture: spawn / read / close / done, with the git arguments;
- cbm_pipeline_run and its staged run: stage, staged, coverage,
git_context, test_config, discover;
- go_index_lookup: mkdtemp, fixture, index; it raises the level to debug
and restores the caller's level afterwards.
Debug level is otherwise unused on the indexing path, so the default
output does not change and only this suite's fixture indexes print the
markers.
Checked: doc_mentions_go passes 23 of 23 on macOS and on a Windows arm64
VM (win.sh test), each printing the same markers in order: 30 from the
helper, 60 pipeline setup steps and 120 git stages (10 indexes, 3 git
children each). clang-format is clean. CI's cppcheck 2.20 reports one style
finding in git_context.c (buf_size > 0 always true), which main has too, in
a function this change does not touch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Smoke's binary string audit blocked on every platform in the dry run on
70543ba8 (run 38204190338, all 12 smoke jobs): "Credential patterns
found in binary". #2068 rebuilt the graph-ui bundle. The two reviewed
Monaco 0.56.0 quick-input lines are still there but minified to different
bytes, so their SHA-256 no longer matched the pair recorded on 09-09.
Re-reviewed: the CI-built bundle has exactly two flagged lines (15,644 and
72,472 bytes). Every match is the input box's password-mode flag
(_password=!1, password=!!e.password, inputBox.password=this.password,
inputBox.password=!1), not a credential. All 12 smoke logs show the same
two lines. The new pair is computed as filter_reviewed_lines does
(printf '%s' line | sha256) on CI's binaries: GNU strings on the Linux
amd64 artifact and Apple strings on the darwin arm64 artifact give the same
two hashes, and both are exempted. The comment now says how to redo this
when the bundle changes again.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
git applies ignore rules to untracked paths only, but discovery applied
them to every path. A nested .gitignore line `manager`, meant for a
built binary, hid cluster-api's tracked config/manager/ kustomization,
and with it the Deployment its overlays point at.
When the indexed directory is in a git repository, discovery now reads
`git ls-files -z` once. A directory that git's rules ignore (.gitignore
files, info/exclude, the global excludes file) is still entered when it
holds tracked files. Inside it, untracked files and directories stay
ignored, as in git. A tracked file is kept although a rule matches it.
The built-in skip list, .cbmignore and the size cap apply as before. If
git cannot run, nothing changes.
On the 17 infra corpora git lists 34 tracked files that an ignore rule
matches. The fix indexes 26 of them, all from that list. The other 8 are
never indexed for other reasons: no language (.tfvars, .gitkeep) or a
built-in skip directory (vendor/). No file is lost and no untracked
file appears. cluster-api gains the manager kustomization (4 Resources,
its overlay IMPORTS), helm-charts a Chart, and otel-demo a Route.
Test: discover_gitignore_never_hides_tracked_files builds a real git
repository. It fails without the fix.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
After a parallel extract or resolve, the main buffer's counter was set to
the shared allocator's value. Merged worker nodes keep their shared ids,
and merged edges take fresh ids from the main counter after the highest
of them, so the main counter can end above the shared one. Setting it
down handed the merged edges' ids out a second time.
All four sync points (full route and closure repair, after extract and
after resolve) now go through cbm_pipeline_sync_next_id, which only
raises the counter. The delta preseed still pins it with
cbm_gbuf_set_next_id, as before.
On the 17 infra corpora the old sync left 94,239 duplicate edge ids in
the buffer (11 corpora; deckhouse 44,959), and the fix leaves none. The
persisted graph does not change, because the dump numbers edges itself.
The duplicates matter to anything that finds buffer edges by id:
cascade_delete_edges unindexes by id and then frees the edge, so a
duplicate can leave a freed edge in an index.
Tests: a parallel full index with the counters pinned by a test seam
fails with a duplicate edge id without the fix. Both full-route and both
closure-repair syncs are checked to run through the helper.
gbuf_raise_next_id_never_lowers covers the new buffer call.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
parse_match_pattern allocated the pattern's node and relationship arrays
and returned on a parse error without freeing them. Its callers count a
pattern only after it parsed, so cbm_query_free never saw it: every query
whose pattern failed partway leaked, including every named path
(`MATCH p = (...)`, not supported, fails at the first node). The element
whose parse failed leaked too (its variable, label or properties), and
parse_props freed only the array, not the entries it had already parsed.
The fuzz job found it on its first run, from a checked-in seed:
`MATCH p = (a:Class)-[:INHERITS*1..3]->(b:Class) RETURN a.name, length(p)`
(LeakSanitizer: 288 bytes in 2 allocations).
parse_match_pattern now frees and zeroes the pattern on every failure,
counting the failed element (parse_node and parse_rel zero it first, so
only what they allocated is freed). parse_props frees its parsed entries
on every failure through one helper.
New test cypher_parse_error_frees_partial_pattern drives five failure
points: a named path, a property list after one entry, a cut-off
relationship, a missing label after a relationship, and a second MATCH
clause's properties.
Checked with the fuzz binary (clang 18 ASan/LSan, arm64 container) on the
five queries: before this change the first four leak (288, 306, 339 and
348 bytes); after it all five are clean, and scripts/fuzz.sh all --runs
20000 --seed 1 passes for extract, cypher and config. The fifth goes
through the same parse_props failure as the second. macOS: the cypher
suite passes, 217 tests.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The cbm-fuzz rule links through $(CC), the C driver. On Linux, clang's C
driver links UBSan's C runtime but leaves out its C++ part
(ubsan_standalone_cxx), while the C++ objects (preprocessor.cpp and the
vendored simplecpp) carry -fsanitize=vptr checks. The link failed with
undefined references to __ubsan_vptr_type_cache and
__ubsan_handle_dynamic_type_cache_miss, so the fuzz job has never built in
CI since it was added (dry runs 37212102899 and 38186491924).
-fsanitize-link-c++-runtime on the link line pulls that runtime in.
Checked in an ubuntu:24.04 arm64 container with CI's packages (clang 18,
llvm, libclang-rt-dev):
- a minimal reproduction (a C++ virtual call under -fsanitize=undefined,
linked through clang) fails with exactly those two symbols without the
flag, and links and runs with it;
- `make fuzz` links cbm-fuzz (lsp_all.o was compiled with -g0 added: the
12.5 GB container ran out of memory on it with full debug info, which
CI's runner does not);
- macOS Homebrew clang accepts the flag under -Werror.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/memwaste.sh set no memory budget, so the daemon derived one from
the machine's RAM: 1792 MB on the 7 GB macos-14 runner, about 4 GB on the
16 GB Linux and Windows runners. The event layer itself holds 0.5-1 GB, and
on macOS the index stopped with over_memory_budget before it measured
anything, in both full-mode runs since the lane was added (dry runs
37212102899 and 38186491924, peaks 3449 and 3111 MB at the stop).
The script now passes CBM_MEM_BUDGET_MB=4096, what the larger runners
already derived, unless the caller sets one.
Checked on macOS arm64 with CBM_WORKERS=3, the macos-14 runner's core
count:
- budget 1792: over_memory_budget in parallel_extract (rss 3904 MB), the
CI failure;
- budget 4096: the index completes and the report is written;
- the edited script with no budget in the environment: 4096 is applied,
rc 0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
GCC 11's arm_neon.h declares the SHA2 intrinsics under
`#pragma GCC target ("+nothing+crypto")`. sha256_blocks_arm was marked
`target("+sha2")`, a narrower target, so GCC 11 refused to inline
vsha256hq_u32 and friends ("target specific option mismatch") and the
build stopped. GCC 13 accepts "+sha2", which is why PR CI (ubuntu-24.04-arm)
stayed green; the dry run's ubuntu-22.04-arm leg (GCC 11) failed on all
three shards (run 38186491924).
GCC now gets "+crypto", which implies "+sha2" on every GCC. The function
still uses only SHA2 instructions, and the runtime HWCAP check gates it.
clang keeps "sha2".
Checked in arm64 containers:
- sha256.c with CI's ASan/UBSan flags compiles on GCC 11, 12, 13 and 14;
GCC 11 with "+sha2" fails with the CI errors;
- a known-answer harness ("abc", "", a 1 MiB pattern) matches hashlib on
the arm-sha2 backend and the forced portable backend, GCC 11 and 14,
-O0 and -O2.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Normalize Windows cache paths when reporting runtime settings and resolve
Git executables before spawning history queries. Keep executable lookup
results per thread and cover the ownership contract deterministically.
Support Git 2.43 while denying history-query network transports, and close
the activity database on daemon startup failure so Windows can clean up.
Validation: 463 native macOS tests, 421 Windows tests, 44 MSan UI tests,
and focused canonical lint passed. Production-only revert checks confirm
the repaired failures.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Five suites (grammar_regression grammar_labels pipeline lang_contract
grammar_probe_e) were excluded from the MSan lane for "stack-overflow
under instrumentation". That was never deep recursion. MSan aborted while
REPORTING real uninitialized reads:
- every abort was on the main thread. MSan's "T<n>" is the OS thread id
and equalled the pid in all five x86-64 aborts (CI job 91868259558);
the old block read it as a worker-thread ordinal, so the worker-stack
knob, MSAN_ORIGINS and CBM_WORKERS=1 could never change anything;
- with the default slow unwinder (the image's MSan-instrumented
libunwind) a report exhausted the main thread's 256 MiB stack, and the
overflow handler then failed with "nested bug in the same thread";
- with fast_unwind_on_fatal=1 the same test prints the
use-of-uninitialized-value report instead (measured on one tree and
test: slow unwind aborts at the stack limit, fast unwind reports
wit/scanner.c:13).
The two reads are fixed in the previous commits of this series, so:
- MSAN_OPTIONS defaults to fast_unwind_on_fatal=1 (the build already
keeps frame pointers);
- MSAN_EXCLUDE is down to cli, whose install-path failure is a separate,
still-undiagnosed cause;
- the known-red block, the header (it still claimed CI runs without
exclusions) and the MSAN_ORIGINS and stack comments say what is true;
- the partial-lane warning and the exclude-name typo check lived in a
branch behind `elif true` that never ran. They now run in the
per-suite path, the check as a case match instead of a grep -q pipe
that pipefail can misreport;
- the comments in scripts/ci/msan-lane.sh and above the test-msan job
in _test.yml no longer say the five suites overflow.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The test filled 3675 commits' files and count but never their timestamp,
and cbm_compute_change_coupling reads each commit's timestamp to track a
pair's last co-change (pass_githistory.c:304). MemorySanitizer reports
the read; the stack array is now zero-initialized like every other
githistory test's commits. Production is unaffected: its commits come
from calloc and a zeroed parse state.
This was the read behind the pipeline suite's MSan "stack overflow"
(the x86-64 CI log of 2026-08-04 aborted in this test).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
tree_sitter_wit_external_scanner_create() is declared to return void *
but had an empty body. tree-sitter's runtime stores its result as the
scanner payload (parser.c ts_parser__external_scanner_create), so every
WIT parse used an indeterminate value, which C11 6.9.1p12 makes
undefined. The scanner never reads its payload, so ordinary builds were
unaffected, and the vendored build's -w hid -Wreturn-type.
MemorySanitizer reports it at wit/scanner.c:13 from grammar_labels,
grammar_probe_e, grammar_regression and lang_contract. Those four suites
were excluded from the MSan lane because the report itself crashed and
posed as a stack overflow (fixed in the msan.sh commit of this series).
Recorded in MANIFEST.md's local-patch table. Upstream
bytecodealliance/tree-sitter-wit still has the empty body at
059bfdd1af72, the last change to src/scanner.c.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reconcile the complete Atlas and Lens stream with current main while retaining both histories. Preserve architecture functionality, current process and watcher safeguards, test infrastructure, and dependency security updates. Add coverage for Git-environment isolation combined with child-local overrides.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows headers define far and near as empty macros, so
'const char *far = ...' became 'const char * = ...' on the CLANG64 leg.
The variable is far_src now.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Merge validated environment overrides into a child-owned environment on POSIX and Windows, preserving caller PATH lookup for executable names. Forward each ObjectScript export parse timeout to the extractor.
Cover inherited and replaced values, empty values, duplicate overrides, parent isolation, invalid names, executable lookup, and timeout forwarding.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Rename the reading workspace to CBM Lens Beta, distinguish Beta with an amber badge, and keep build diagnostics out of the header. Put architecture history beside the active context and record meaningful exploration steps.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up to #2590: three more waits that a timeout decided.
cli_activation_quiesce_does_not_wait_on_bootstrap_startup:
- Its participant child joined the version cohort with a 45 s deadline. It
now uses UINT64_MAX, which waits for locks and answers a conflicting
holder at once.
- The child watched for the installer's maintenance request for at most
120 s. It now watches until a definite event: the request, an UNSAFE/IO
answer, or the parent closing a new install-done pipe once its install
has returned. A failed install therefore ends the watch at once instead
of hanging, and a slow install is waited for instead of failing.
cli_install_into_host_namespace_still_drains_host_cohort:
- The "drained host stopped serving" probe was one status request with a
2 s budget, so a slow host that still served could pass as drained (a
false green). It now asks through the same mute-retry rule as the
positive probe: a drained host is unreachable or reports stopping at
once, and a slow host that still serves is asked again until it answers.
cli_scope_host_serving_within and CLI_SCOPE_HOST_DRAINED_PROBE_MS are
gone.
Checked by hand: with the install made to fail before it requests
maintenance (an unknown option), the test ends in 3 s through the
install-done pipe. Without the pipe and the old deadline it would hang;
with the deadline it took 120 s.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A Markdown section that embeds a diagram is about its candidate functions
more often than the Markdown curve says, by how much depends on the
diagram's kind. 553 candidates from such sections in 16 held-out
repositories, judged blind in three rounds:
0.20-0.30 0.30-0.40 0.40+ local extras
prose 0.24 0.28 0.45 0.275
structural 0.32 0.46 0.57 0.78 (9 items)
Structural is Mermaid sequence/class/ER/C4/architecture and PlantUML
C4/class/sequence/use-case/component; prose any other Mermaid or PlantUML
kind. A section takes the highest family of the Diagram nodes it defines;
its function candidates at or inside the doc's home and its local extras
carry the family's value, and the first band is now stored. Outside
functions, files, folders, other formats and sections without a diagram
keep the Markdown curve.
On kubernetes' apiserver folder the sequence-diagram section's
candidates move from 0.22/0.51 to 0.46/0.57, the flowchart sections gain
five first-band candidates at 0.24, other sections and the graph are
unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up to #2589.
No timeout decides a cli_scope test any more:
- The status probe and the parent's client connect repeat an attempt that
reaches a live but mute endpoint (a loaded runner answering slowly). A
reply or a definite refusal decides. Each attempt keeps the API's finite
5 s budget. This replaces the 15 s status budget and the single 5 s
connect.
- The host child is reaped without a deadline (it used to get SIGKILL after
60 s).
- The host child joins its version cohort with UINT64_MAX, which waits for
locks but answers a conflicting holder at once (it used to have a 45 s
deadline).
A wedged child is left to the harness's per-suite ceiling.
cli_scope_fixture_finish restored HOME, CBM_CACHE_DIR and SHELL even when
start had failed before saving them, which unset them for every later test
in the suite. It now restores only what start saved.
New tests:
- cli_scope_fixture_finish_keeps_env_after_an_early_failure fails on main
at ASSERT(home_kept);
- cli_scope_retries_a_mute_endpoint_until_it_answers checks the retry rule
on scripted attempt outcomes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The held-out audit of the eight new languages is scored. Each tier is a
language and a reference form, read on a sample of 150 links (census below
150) from repositories never used to build or tune the resolvers. Three
blind reading rounds; bar: Wilson 95% lower bound >= 0.90 and point >= 0.95.
tier population judged correct Wilson low
go doc_link 5980 150 150 0.975
java link 23934 150 150 0.975
java linkplain 1372 150 150 0.975
java see 7711 150 150 0.975
java throws 650 150 150 0.975
java value 79 79 79 0.954
kotlin kdoc_link 3888 150 150 0.975
kotlin sample 279 150 150 0.975
kotlin see 369 150 150 0.975
kotlin throws 73 73 73 0.950
rust intra_doc 14119 150 149 0.963
rust link_def 1491 150 150 0.975
perl pod_L 2255 150 150 0.975
ts/js link 1222 150 150 0.975
php see 5171 150 150 0.975
c doxygen_ref 463 150 150 0.975
c see 509 150 145 0.924
These 17 families now ship: a resolved reference is a MENTIONS edge.
TS/JS {@linkcode} (10 held-out links) and @see (18) were all judged
correct, but are too few to reach the bar; they stay held, as do the
diagram and OpenAPI link families (3 links on 21 held-out repositories).
The Go ship-gate test turned its family off and restored only the held
families afterwards; with Go shipping it left Go off for the tests that
follow it. It restores the production gates first now.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A Markdown Section's docstring is its body collapsed and cut at 500 bytes.
A diagram fence in that body put the format's syntax into it (`-->`,
`participant`, `classDiagram`, `subgraph`), which names no code, and pushed
the section's prose out of the cap. The doc -> code candidates are scored
on that text.
Each top-level diagram fence (mermaid, plantuml, puml, uml, dot, graphviz,
d2) is now replaced in the section's text by the labels its diagram parser
reads: the names of its elements, then its arrow labels, in source order.
The parse runs into a scratch result, so the file's own Diagram and
DiagramElement nodes are untouched. A section without such a fence keeps
its text byte for byte; other fences stay as written.
Measured before it was chosen, on held-out repositories (147 sections that
embed diagrams, 503 section/candidate pairs, three blind reading rounds):
fence as written 277 stored precision 0.45 42 sections with a correct candidate
labels (this) 310 stored precision 0.54 56 sections with a correct candidate
fence dropped 127 stored precision 0.54 29 sections with a correct candidate
Labels gave 20 sections a correct candidate that the fence as written did
not, and lost 6 (exact McNemar p = 0.009). Dropping the fence lost more
than it gained.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cli_install_skip_binary_unchanged_in_host_namespace_quiesces_nothing
failed once on 2026-10-09 at ASSERT(ready), with nothing else in the log.
cli_scope_fixture_start returned false silently from any of its setup
steps, and a 90 s deadline decided the wait for the host child's
readiness.
The fixture now records and prints the step that failed: the fixture
directory, the host directories, the runner's own image fingerprint, the
pipes, the fork, the host's start, or the client connect, with the OS error
where one applies. The host child prints which of its own steps failed
before it answers 'E'.
The readiness wait has no deadline any more, because no timeout may decide
a test. The child answers 'R' or 'E', or the pipe reaches EOF when it dies.
A child that wedges is left to the harness's per-suite ceiling.
New test: cli_scope_fixture_names_its_failing_setup_step.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On macOS the signed-candidate preparation folded every step into one
boolean, so each failure printed the same line: "signed update candidate
preparation failed". A full /tmp read exactly like a broken codesign. That
is what a cli test hit on 2026-10-09, while a multi-GB index was filling a
nearly full disk.
Each step now records why it failed, and the update prints, for example:
error: signed update candidate preparation failed while writing the
unsigned candidate: No space left on device
The staging functions in activation_transaction.c keep the errno of the
failing write (or sync, close, read) through their cleanup, so a caller can
name it. The non-macOS path adds the OS error to its existing "failed to
stage verified update" line (POSIX only; Windows reports through
GetLastError). A failed fingerprint of the staged file now gets its own
message instead of "...: activation transaction completed".
A test seam makes the next staging write fail with a chosen errno. The new
test cli_update_names_the_failing_staging_step_and_os_error fails without
the cli.c change and passes with it, and checks that the old executable
stays in place. Also checked on real input: with /tmp on a small volume
that fills between two updates, the update prints the line above.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The TS/JS, PHP, C and Perl doc-link layers keep a scope blob per file but
had no scope_delta hook, so any change of a file's declarations made the
incremental closure repair decline (doc_scope_changed) and the index
rebuild in full: removing a TypeScript function forced a full run.
cbm_doclinks_scope_delta_lines compares two blobs line by line in order,
as C# does: a dropped record that declares one name is repaired file by
file (the name is reported, so the files whose references named it are
re-resolved); anything new or changed, and every record others resolve
through, stays a full run. The records that declare one name: TS D
(a top-level declaration), PHP F and K (a function, a member), C D of a
function, variable, enum constant, field or macro and A (a Doxygen
anchor), Perl S (a sub).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Diagrams name the parts of a system and how they connect; an OpenAPI
document names its endpoints and their handlers. Both now enter the graph.
Diagrams:
- a Diagram node per diagram: a fenced block of a Markdown file (under its
Section), a directive of a reST file (`.. mermaid::`, `.. uml::`,
`.. graphviz::`, Sphinx `.. digraph::`), an Asciidoctor Diagram block of
an AsciiDoc file (`[plantuml]`, `[source,mermaid]`, ...), or a diagram
file of its own, named by its title, else by its file;
- a DiagramElement node per element, DEFINES from its Diagram, and a
DIAGRAM_ARROW edge per arrow (label, style, line);
- formats: Mermaid (flowchart, sequence, class, ER, state, C4,
architecture), PlantUML, Graphviz DOT, D2, draw.io (plain, compressed,
.drawio.svg) and Excalidraw (.excalidraw, .excalidraw.json,
.excalidraw.svg); .drawio.svg and .excalidraw.svg are no longer dropped
as images;
- an element's explicit link (Mermaid click/link, PlantUML [[url]], url of
and !include, draw.io and Excalidraw link, DOT URL/href, D2 link) is a
doc link read as a Markdown link, MENTIONS when it resolves.
OpenAPI:
- each operation of an OpenAPI or Swagger document (openapi.json and
swagger.json, now read with the JSON grammar but no node per key, and
any YAML or JSON file whose root says openapi/swagger) is a Route, named
by the same QN a route its handler registers in code gets
(cbm_route_canon_path, moved into the extraction library);
- its operationId is a doc link to the handler: a dotted one (Connexion,
with x-openapi-router-controller) as a qualified code name, a bare one
when exactly one function or method of the repository has that name.
The eight new link families are held until their held-out audit passes.
Read on real repositories, which shaped the rules: Excalidraw drawings put
text over shapes and draw arrows up to them without binding, so loose text
names the smallest box it sits in and a loose arrow end meets the smallest
named box it touches (the drawing's frame only at its border); arrows bound
to free text meet the text; D2 block strings and nested blocks; DOT line
breaks and sentence-long ids; PlantUML kinds of classes inside packages
and nested classes (`..+`, nested -> container, so they do not collide with
the container's association). Connexion's 24 specs: 160 operationIds, all
linked to their handlers.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel