- The galaxy stays mounted once visited: hidden with invisible (never
display:none, which zeroes a canvas), WebGL context and layout kept,
frameloop paused via the scene's active prop - switching tabs no
longer drops and refetches the whole 3D scene.
- Flows list merges journeys by entry point: one row per entry
(richest first) with an 'x N journeys from <entry> - show all'
expander, so ten main() walks stop drowning the list.
- Modules gains a map|bars toggle (persisted): the same children as a
sorted horizontal-bar list - region-colored, symbol counts - for
when proportion matters less than ranking.
- Filter chips strike through when disabled (excluded, not missing).
- One-time in-place hint on the region scene; dismissing persists,
leaving without dismissing does not.
Tests: groupFlowsByEntry ordering/merge (vitest); suite green with
the keep-mounted change.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The D4 orientation package, mechanics adapted from the two reference
UIs studied in the screened snapshots:
- Search scent propagates up the hierarchy: a 'find a symbol' input on
the region sidebar lights matching regions in the 3D scene, pins hit
counts on the region rows and the minimap, and reports the total
with an outside-regions remainder. Served by /api/scent - a
case-insensitive name match bucketed per region via the cached
file->region map (File/Folder nodes excluded so a file never
double-counts its own symbols; sub-2-char queries refused).
- 2D minimap panel (bottom right, collapsible): every region as a dot
on a fixed top-down projection, sized by sqrt(members), colored by
region, open region ringed, scent counts above the dots, click to
open, and a Home button that fits the current scene.
- Region hover card: members/files, cohesion as a word (tightly /
moderately / loosely connected) with the score, and the hub.
- Node tooltip on graphite tokens (last pre-D1 hardcoded surface) and
in/out call degree - layout nodes now carry out_calls, which the
degree pass already computed and discarded.
- Honest scope note in the region detail panel: '+N callers outside
this region' whenever the full-graph in-degree exceeds the loaded
window.
Tests: scent bucketing + LIKE-escape refusal (C), /api/scent route
wiring, minimap projection + cohesion words (vitest), region-list
queries disambiguated from minimap titles.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The two questions the research says developers ask most and tooling
answers worst: why is this code the way it is (rationale via change
history and ownership), and does A reach B (LaToza reachability,
asked ~9x/day). Plus the user-requested data-flow lens.
- /api/trace: shortest A-to-B path over CALLS or DATA_FLOWS (BFS,
depth-capped at 12), endpoints by id or qualified name; the flow
cache gains a DATA_FLOWS CSR beside the CALLS one.
- Symbol bundle: file_history (recent commits, commits/year, top
author share from a bounded git log at metrics time, cached and
honest - available:false when no history was parsed) and
data_in/data_out with the stored edge detail.
- Flows tab: trace bar with symbol pickers and a control/data
toggle; the result path is clickable hop by hop.
- Symbol tab: 'Recent changes to this file' and 'Data flows'
panels with honest empty states.
- Tests: trace reachability both modes + endpoint errors, symbol
bundle data-flow/history shape, /api/trace route wiring, and a
fetchTrace client test.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The metrics screens now answer questions instead of displaying data
(dashboard canon: Few, Grafana, CodeScene — the Field Guide carries the
citations):
Overview
- Four opinionated slots on top: Needs attention (knee-point count over
churn×complexity), Direction (deltas vs the previous index), Riskiest
area (named, with its why), and an untoned Trust strip (index stamp,
missed files, unmapped symbols).
- Surprising couplings become finding sentences with evidence, an
'inspect' next click, and an 'intended' dismissal whose key buckets by
order of magnitude — a dismissed finding resurfaces only when it grows
10x (regression re-alert, CodeScene's goals idea in one line of code).
- Hubs, entry points and boundaries demote to a collapsed Reference
disclosure; inventory facts move to an untoned footer.
Dashboard
- Hero: the churn×complexity list with the knee-point head highlighted, a
cost sentence ('N files — X% of the codebase, Y% of all commits this
year'), raw commits shown per row, mute-as-intended dispositions, and
rows opening the file's folder in Modules.
- Quality signals (resolution, dead, tested, documented) keep tones and
trends; the complexity histogram now states its takeaway and names the
tail; averages and equal-weight inventory cards are gone — facts live
in the footer.
- Metrics payload gains raw commits per risky entry, churn totals and the
distinct-file count to make the cost sentence computable.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Visual language, rebuilt on the researched rules (Primer/Carbon/Material/
Radix/Linear; details in the Atlas Field Guide):
- Graphite surfaces with fixed step jobs (#0D0F12 canvas → #293038
selected) replace the saturated navy; elevation is a lighter surface
step, not a shadow; hover/selection are surface washes.
- One accent (sky #4FA8E0) reserved for interactivity; semantic
good/warn/crit separate; the type floor is 12px (labels), body 13-14px,
system font stacks (sans UI, mono identifiers only — zero vendored
bytes), tabular numerals, radius 6px.
- Region colors move to the Okabe-Ito color-blind-safe palette, assigned
server-side to the ten largest regions with a neutral tail — the code
graph is the only colorful thing on screen.
Trust repairs, because every wrong-looking item costs belief in every
right one:
- builtins filtered out of hub lists (a 'len' hub reads as a bug);
- a product-code lens hides test regions by default everywhere regions
appear, one honest toggle away ('18 test regions hidden — show');
- flow labels carry file context when the bare name is ambiguous
('datastructures.__init__ → …' instead of '__init__ → …').
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The lazy-schema fix was ineffective: the UI never sent include_details,
so list_projects returned names only, every card lacked totals, and the
>1M-node skip compared against undefined — each Projects visit still ran
a ~30 s kernel schema scan, and revisits stacked uncancelled sweeps that
kept the store busy (the felt slowness).
Now: list_projects(include_details:true) carries totals and indexed_at;
the enrichment loop is generation-guarded (unmount or refresh abandons
it), schemas are cached for the session, auto-fetch is capped at 20
projects of ≤200k nodes, and larger stores keep a card with totals and
no chips. After a Projects visit the daemon log shows only sub-second
requests.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Opening the /rpc allowlist revealed a pre-existing cost: the Projects tab
awaited get_graph_schema for EVERY project before rendering anything, and
a schema scan walks every node and edge — ~30 s on a 13 GB kernel index.
(Before #1663's fix those calls failed instantly, which is why the page
ever felt fast.)
list_projects already reports per-project node/edge totals, so the page
now renders from those immediately; per-label chips arrive lazily, one
schema at a time, patching each card as it lands — and projects above a
million nodes skip the schema scan entirely, keeping a card with totals
and no chips rather than burning half a minute of server time per load.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The numbers view of a project, with the code behind every number one
click away and no composite health score — the cards are the score:
- /api/metrics (cached per project+indexed_at): complexity and function-
length histograms and top lists straight from pass_complexity's stored
cyclomatic/cognitive/lines numbers, edge certainty (CALLS vs
CALL_REFERENCE vs USAGE), dead-code candidates (same exclusions as the
galaxy classifier), TESTS linkage, SIMILAR_TO duplication, missed
files, documented-exports coverage.
- Per-file churn from one bounded git log pass (1 year, 10k commits, the
pass_githistory bounds) computed at metrics time — ZERO cost added to
indexing — plus churn × complexity, the classic where-the-bugs-live
list.
- A per-index history sidecar (JSONL in the cache dir, one snapshot per
indexed_at) feeds trend sparklines across reindexes.
- Dashboard tab: stat cards with warn tones, histograms, top lists
linking to symbol pages; churn honestly marked unavailable when the
project has no readable git history.
The wired-routes test covers /api/metrics; the ui suite pins totals,
certainty, histogram completeness and history presence on a fixture.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The UI section now names the two interfaces (MCP for the agent, Atlas
for the human) and documents the screens: Overview, Modules, region-LOD
Galaxy, Flows, Changes, Symbol pages, search and the prompt composer,
plus deep-linking.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The human side of the Atlas program:
- Overview: the first read of a project — header numbers, region cards
with provenance and cohesion, hubs by fan-in, entry points, surprising
couplings with enumerated reasons, suggested questions with a why, and
the cross-package boundary table. Assembled client-side from the
read-only /rpc surface (get_architecture format:json) + the regions
payload; no new server code.
- Modules: a squarified treemap (in-house, no new dependency) of any
folder via /api/tree — sized by symbols, colored by dominant region,
hatched where the indexer missed files, breadcrumbed drill-down, file
panel listing symbols via search_graph.
- Symbol: one page per node via /api/symbol — docstring, flags, region,
editor/GitHub links, code on demand, callers/callees with true totals
and per-type confidence badges (resolved / reference / unproven),
tests, co-change partners, near-clones.
- Flows: the /api/flows journeys grouped across/within regions, each a
clickable indented call tree with truncation stated, mermaid export.
- Changes: detect_changes on the working tree grouped by risk, blast
radius citable into the prompt in one click.
- ⌘K: symbols (search_graph) + code text (search_code) in one box.
- Prompt composer: a basket of cited symbols/regions/flows/questions that
renders a prompt naming exact qualified names, file:line ranges and the
MCP calls that reproduce the context — the bridge back to the agent.
Routing carries every screen in the URL (tab, project, path, sym, flow,
node, region) so all of it deep-links. Fixed en route: the /api/flow
route pattern '/api/flow?*' never matched (req->path has no query
string) — now exact, pinned by a wired-routes test that distinguishes a
matched route's 400 from the static handler's 404.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On Django the flow list was all setUpTestData/test_* entries: test
functions are natural call roots (no callers, many callees) so they
out-scored real entries. A test file is a journey into the code, not of
it — candidates in test paths score zero unless the indexer explicitly
flagged them as entry points.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three read-only services the Atlas screens consume, as pure JSON producers
over an open store (handlers stay thin; tests hit the functions directly):
- GET /api/tree?project&path — aggregated children of one folder: files
and symbol counts per child, dominant region (majority vote), missed-
coverage counts from the #963 shadow project, child cap reported rather
than silent. Feeds the Modules treemap at any project size without
shipping nodes.
- GET /api/symbol?project&id|qn — the symbol page in one response: node
with docstring and entry/test/export flags, region membership, callers
and callees with TRUE totals, per-edge-type counts and pagination
(CALLS before CALL_REFERENCE before USAGE), inbound TESTS, co-change
partners of the file (FILE_CHANGES_WITH), and near-clones (SIMILAR_TO).
- GET /api/flows?project, /api/flow?project&id — named entry→terminal call
journeys: entries scored from stored flags and call shape, bounded DFS
over CALLS (depth 6, 4 children per step, 40 steps), deduplicated by
entry/terminal pair, ranked sink-first, truncation counted; cached per
(project, indexed_at) like the region level. The busiest callee leads
each step so the trunk of the journey reads first.
The region cache gains an O(1) file→region lookup (hash-backed) shared by
tree, symbol and flows. Fixed en route: yyjson borrows object keys added
via add_*(), so the per-type count keys are now copied into the document
instead of dangling once their sqlite statement is finalized.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The graph tab becomes CBM Atlas and gains the region level as its default
for large projects:
- Projects above 5,000 nodes open on the REGION SCENE: one body per Leiden
call community (folder groups as fallback), weighted arcs between them,
a region list on the left and a region panel (provenance, cohesion, top
callables) on the right. Double-click or 'Open region' loads that
region's full-detail layout (scope=region:N) with a '< Regions'
breadcrumb; 'Load full galaxy' keeps the classic whole-graph view one
click away. Small projects skip the region hop entirely.
- Deep links (#564): ?node= and ?region= join ?tab=&project= — a selected
node or opened region survives reload and back/forward, and can be
shared.
- #1197: clicking a symbol row in the sidebar (tree or search) selects the
node itself — scene highlight AND detail panel; selecting a directory
clears a stale panel instead of leaving the previous node in it.
- #1198: connection lists page with 'Show N more' instead of a silent cap
of 25.
- Rename: the UI announces itself as CBM Atlas (header + title).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The galaxy loaded 'the first max_nodes nodes' of a project, which above
~10k nodes meant freezing the tab (#498 family) or truncating the graph.
Give it a real coarsest level instead: one body per REGION.
Regions are sets of files. Files are assigned by majority vote of their
callables' Leiden communities over the CALLS graph (seeded, deterministic);
files no kept community explains fall back to folder groups (first two
path components), and a final 'misc' bucket catches the rest. Every node
inherits its file's region, all edge types aggregate into region-to-region
weights, and each region carries its name's provenance ('call community:
202 files, 38% under tests/db_functions'), cohesion, member/file counts
and its top callables by degree.
GET /api/layout?project=X&level=regions -> region bodies + weighted arcs
GET /api/layout?project=X&scope=region:N -> full detail, that region only
Positions come from a golden-angle sphere seed refined by the shared
anchored optimizer (extracted to layout_internal.h alongside the new
cbm_layout_from_nodes core, so the region scope reuses the exact layout
path). Results are cached per (project, indexed_at): on a 48k-node index
the region scene computes in ~1.6s cold, <1ms cached; a 4.8k-node region
scope lays out in ~0.5s.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The graph UI could call only list_projects and get_code_snippet through
/rpc; every other tool answered 403 'UI RPC method is not allowed'. That
starved the browser of schema, search, tracing, architecture and impact
data the server already computes, and broke the Projects tab, which needs
get_graph_schema (#1663).
Allow the read-only tools (get_graph_schema, search_graph, search_code,
trace_path/trace_call_path, get_architecture, query_graph, detect_changes,
check_index_coverage, index_status) and manage_adr in its read modes only
(default get, sections). A duplicated or non-string mode refuses, so a
write cannot ride in behind a read. Mutating tools keep their 403; the
loopback-only boundary is unchanged.
Fixes#1663.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer asked for in-tree coverage of {content, details} wrapping and
how tool errors are surfaced, plus a cheap abort-signal line. String-shape
only; no live Pi process.
Signed-off-by: Alex Musichen <alex.musichen@gmail.com>
cli_install_plan_receipt_no_mutation_issue388 asserted inline, so any failure
returned before free(json) and test_rmdir_r(tmpdir). A red run therefore leaked
the receipt and left a stray /tmp/cli-plan-* directory behind -- the failure
report was accompanied by exactly the noise that makes the next debugging
session harder.
Record what went wrong, release everything, then fail. The message now names
the specific missing marker rather than reporting that some marker was absent,
which is the difference between reading a failure and bisecting one.
The env-isolation half of the original PR is not carried over: main's
tf_setup_cache_sentinel already unsets CODEX_HOME along with every other client
home override, so a per-test save/unset/restore would be redundant now.
Distilled from #1149 by Anand Aiyer, which predates that sentinel and could not
land as written once its companion PR closed unmerged.
Co-authored-by: Anand Aiyer <anand@aiyer.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/test.sh labels its pre-build contracts as a running 0a..0u sequence.
The new blob-generator contract was slotted in as "0f2" to sit next to the
other build-tooling contract, which breaks the one convention the list has.
Append it as 0v instead; these steps all run before the build, so position
carries no meaning beyond the label.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
vendored/nomic/code_vectors_blob.S is a GENERATED file that was hand-edited.
1f674c80 added the ELF .note.GNU-stack section to it after finding that every
Linux binary we had ever shipped requested an executable stack: an object
carrying no such note tells GNU ld nothing about its stack requirement, and ld
then assumes the worst for the entire link. That fix went into the artifact
only. scripts/extract_nomic_vectors.py still emitted the Mach-O branch alone,
so the next regeneration would have overwritten the tracked .S, dropped both
the ELF and COFF branches, and put GNU_STACK RWE back into every release --
silently, because nothing compared the generator against its own output.
write_blob_s() now emits the tracked wrapper verbatim, and the new contract
test fails if the two ever drift again. The template is compared rather than
executed so the test does not need the torch/transformers import the extraction
script pulls in. The write is pinned to UTF-8 because the WHY comment carries
an em dash and the default encoding is locale-dependent.
check-binary-composition.sh already fails a release whose binary has an
executable stack; this closes the same hole at edit time, where the diff is
still small enough to read.
Reported and originally fixed by Anand Aiyer in #1151, which also carried the
COFF branch. That PR sat unreviewed for a month while the .S half was
rediscovered and landed independently without credit.
Co-authored-by: Anand Aiyer <anand@aiyer.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Java enum methods already resolve on main: find_class_member_body
descends into enum_body_declarations (Java-gated) while find_class_body
keeps returning enum_body, which is what extract_enum_members needs to
reach the constants — they are siblings of enum_body_declarations, not
children.
That distinction was unguarded. java_enum_dedup_preserves_calls_issue1234
asserted the methods and the absence of duplicate Function defs, but
never asserted the constants, so collapsing the two lookups would have
passed the suite while silently dropping every enum constant from the
graph.
PR #984 proposed exactly that collapse — redirecting the shared
find_class_body — which is what surfaced the gap. Verified binding:
pointing extract_enum_members at find_class_member_body reddens the new
Variable assertions and nothing else.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-authored-by: sahil-mangla <manglasahil2017@gmail.com>
A busy startup transition always resolves, by one of two routes: the live
peer holding it finishes its command, or the holder is already dead and
the operating system has not finished reclaiming the lock. Waiting was
capped at 10s regardless, so a clock decided a user-visible outcome and
commands were refused with "CLI startup coordination remained busy" while
nothing was actually wrong.
The second route is Windows-specific and is what makes this reachable in
practice. On POSIX the kernel drops flock the instant an owner dies. Windows
byte-range locks do not: Microsoft documents that after a process terminates
holding one, "the time it takes for the operating system to unlock these
locks depends upon available system resources", and that until then "access
to these files may be denied". A loaded CI runner is exactly where those
resources are scarce, and tests/windows/test_daemon_stability.py creates the
condition deliberately by hard-killing daemons with `taskkill /F`, including
a crash-recovery section -- so the next client meets a lock whose owner no
longer exists and is refused for a reason that no longer applies.
The cap becomes a backstop against a peer that never finishes rather than a
budget for healthy contention, and the message it prints now names both
explanations instead of just "busy", which sent reporters looking for a CBM
session that had already exited.
Clean exits already release via main_local_transition_close, so no new
release path is needed; this only affects what happens after an abrupt
termination or under genuine concurrency.
VERIFICATION LIMIT, stated plainly: this cannot be reproduced or verified on
macOS -- the mechanism requires Windows lock-reclaim semantics, and a local
12-client storm passes even under the old cap. 458 daemon/ipc/runtime/cli/
watcher tests pass and lint is clean, but whether this removes the CI flake
can only be established on Windows CI.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three related defects in how the watcher decides what counts as a change.
1. `git rev-parse --git-dir` walks UP the tree, so an ordinary folder that
merely sits under an unrelated repository answered yes. The watcher then
inherited that ancestor's dirty state -- permanently non-empty and about
entirely different files -- and reindexed on every poll forever. A
directory is now git-managed only if it carries its own .git (directory
or gitlink file) or the ancestor actually tracks something inside it, so
a real monorepo sub-package still qualifies while a scratch or ignored
folder does not.
2. `git status` reports the whole repository regardless of -C, so every
package in a monorepo reindexed whenever any sibling was edited. The
status call is now scoped with a `-- .` pathspec.
3. Porcelain paths are REPOSITORY-relative but were stat'ed against
root_path. For a project watched at the repository root the two coincide
and the bug is invisible; for a subdirectory project every stat missed,
silently degrading the dirty signature to text-only and losing the
size/mtime component that makes an edit to an already-dirty file
detectable. Paths now resolve through `rev-parse --show-cdup`.
--show-cdup rather than --show-toplevel because MSYS/Cygwin git returns a
translated absolute path from the latter that will not join onto the
native path we hold; a relative hop composes on every platform.
Distilled from #1001 by bethzyy, whose diagnosis identified all three.
Main independently fixed that PR's two headline problems (the unbounded
non-git file count, and per-poll churn via a dirty-state signature) while
it waited, and the watcher was rewritten onto a supervised argv-spawn
harness in between, so this is a reimplementation against the new code
rather than a rebase of theirs.
Both new tests are verified binding: reverting the classification guard
reddens the nested-directory test, and removing the pathspec reddens the
monorepo one, each without disturbing the other.
Addresses the remaining parts of #713, #841 and #937.
Co-authored-by: bethzyy <bethzyy@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pkg/npm/README.md claimed 159 languages and 14 MCP tools while the root
README says 158 languages and 15 MCP tools. The tool count is verifiably
wrong: the TOOLS[] table in src/mcp/mcp.c has 15 entries. The language
count is aligned to the root README, which states 158 consistently in
five places.
Each stale figure appeared twice; both occurrences of each are fixed, so
the package README no longer contradicts itself or the root README.
Distilled from two PRs that each caught one half: #967 by ketpatil77
(language count) and #1268 by nhtkylc91 (tool count). Neither fixed all
four occurrences, and both touched the same file, so they are combined
here with credit to both.
Note for whoever refreshes these next: 159 grammar directories exist
under internal/cbm/vendored/grammars/, which does not reconcile trivially
with either figure (some languages share a grammar; some enums have no
vendored grammar). This change deliberately does not re-derive the
number, only stops the two READMEs disagreeing.
Co-authored-by: ketpatil77 <ket.patil77@gmail.com>
Co-authored-by: nhtkylc91 <nhtkylc@hotmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A Swift protocol requirement — a bodyless `func generate() -> String`
inside a protocol — was absent from the graph entirely. Swift codebases
are heavily protocol-driven, so the requirement is very often the
declaration a reader is looking for, and "who declares generate()" had no
answer.
Three edits deliver it: protocol_body joins the class-body types so a
protocol's members are walked at all; protocol_function_declaration joins
swift_func_types, because extract_class_methods gates on that set and
would otherwise walk the requirement and discard it; and both name
resolvers accept the node, which has the same simple_identifier shape as
function_declaration.
Distilled from #613 by xbsjason. That PR also carried an enum/struct half
whose two assertions fail on CI: tree-sitter-swift has no
struct_declaration or enum_declaration node type (it models both as
class_declaration), so the corresponding swift_class_types entries are
inert and a bare enum is still labeled Class. That is a real pre-existing
modelling gap, left untouched here and now documented at the dead entries
rather than silently deleted.
Also dropped from the original: a find_first_descendant_by_kind rescue in
extract_class_methods. Removing it changes no test outcome once the
function-type entry is present, so it was carrying nothing.
Advances #43.
Co-authored-by: xbsjason <xbsjason@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`nix develop .#cross` on macOS exposes the OTHER darwin architecture's
zlib, so `scripts/build.sh --arch <target>` can link an x86_64 binary on
Apple Silicon and vice versa. The native clang already cross-emits object
code via -arch; zlib is the product's single link-time dependency, and the
host-arch copy cannot satisfy a cross link, which is the whole gap.
Darwin-gated and opt-in: the attribute set is empty on Linux, the default
shell is untouched, flake.lock is unchanged and no new flake input is
introduced. The shell deliberately does not use inputsFrom, since that
would put the host-arch zlib back on the link path.
Distilled from #724 by Kris Williams. That version also wired libgit2 via
pkg-config, which is now dead weight: libgit2 was removed project-wide for
GPL-licensing reasons and appears nowhere in the tree, so both it and
pkg-config are dropped here — leaving zlib as the only library the cross
shell needs to place.
Validated by parsing the flake with nix-instantiate in a nixos/nix
container (no Nix toolchain on the maintainer host), which also confirms
the shellHook's ''${...} escaping resolves to literal shell expansions.
Evaluating or entering the shell needs a macOS host with Nix; no CI leg
exercises it, as noted on the original PR.
Addresses #705.
Co-authored-by: Kris Williams <115474+kriswill@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A dbt model is an ordinary .sql file whose dependencies are written
{{ ref('other_model') }} or {{ source('group','table') }}, never as
literal table names. The SQL grammar cannot read those -- FROM {{ ref(
'x') }} is a parse error to it -- so the dependency structure of an
entire dbt project was invisible to the graph.
extract_dbt.c runs as a sub-extractor inside the normal indexing
pipeline. Per qualifying file it emits one Model definition (named by
the file stem, dbt's own model identity) plus one usage per ref()/
source() call; pass_usages resolves those into model -> relation
lineage edges like any other reference.
Model is a relation label alongside Table and View, which is what makes
the rest work without new plumbing: registry seeding, the central
relation veto, the incremental surface hash, search ranking and the
architecture queries all pick it up from cbm_label_is_relation. Sharing
one label class also means a model's source('raw','customers') resolves
onto a Table declared in a plain DDL migration in the same repository,
so dbt lineage and SQL lineage form one graph rather than two, while
the veto keeps model names out of every non-lineage consumer.
The pass is self-gating: SQL files only, and only those carrying a real
ref()/source() call. The dbt builtins are themselves the evidence, which
is cheaper than a dbt_project.yml lookup and more precise -- templated
SQL that is not dbt (an Airflow {{ ds }} parameter) produces no Model
node and no usages even inside a dbt repository.
{% macro %} definitions are deliberately excluded: the vendored
tree-sitter-jinja2 grammar has no node types for {% %} statements, so
they can only be recovered by a hand-written scanner that cannot see
comments or find {% endmacro %} for a correct span. Filed as follow-up
rather than approximated.
Tests cover the lineage, the last-string-argument semantics of both
builtins, the gate (against a plain-SQL control extraction), plain DDL
staying untouched, and an end-to-end pipeline case asserting model ->
model across files, model -> Table onto plain DDL, and cross-language
isolation. Disabling the pass reddens three of them; removing Model from
the relation set breaks lineage outright.
Implements the lineage half of #575.
Co-authored-by: alexisperinger-ux <alexis.peringer@iss-stoxx.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
CREATE TABLE / CREATE VIEW / CREATE MATERIALIZED VIEW now produce Table
and View nodes (previously generic Variable), CREATE PROCEDURE produces
a Function, and schema-qualified DDL names (schema.table) are named by
the table identifier instead of the schema. A view's FROM/JOIN relations
are emitted as usages and resolve into view -> table USAGE lineage
edges.
Relations join the cross-file name registry so lineage can resolve, with
two structural safeguards:
- Registry membership is defined once by cbm_label_is_registry_symbol
(helpers.c); the full, parallel and incremental seed sites all call
it, ending the KEEP-IN-SYNC copies the old label lists required.
- The default cbm_registry_resolve vetoes relation-labeled results:
common table names (users, orders, config) collide with code
identifiers in every language, and no CALLS/USAGE/READS/WRITES/THROWS/
handler/decorator consumer may bind them. The SQL lineage path opts in
through the new cbm_registry_resolve_lineage.
Table/View also join the registry-only per-file LSP surface labels so a
table rename invalidates dependent SQL files on incremental (no stale
lineage edges), rank with the type tier in BM25 search, and appear in
the architecture boundary/package/cluster queries via the pinned
CBM_SQL_RELATION_LABELS fragment.
Tests: extraction trio (labels, lineage usages, schema-qualified names),
grammar golden + probe updates, relation-label contract pin, and two
pipeline tests: cross-language isolation (binding: fails without the
veto) and incremental table-rename stale-lineage.
Closes#574.
Co-authored-by: alexisperinger-ux <alexis.peringer@iss-stoxx.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The contract required the bare '!cancelled() && !failure()' idiom on
build/smoke/soak — the exact form the v0.10.7 incident proved fail-open
(failure() does not cover a CANCELLED needed job). It now requires each
gate's explicit accepted results, the sanctioned skip_tests clause on
build, and the preflight version guard wired before lint.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three fixes from the v0.10.7 release incident (2026-08-18):
1. build (and smoke/soak) required 'not failed' instead of explicit success.
failure() does not cover a needed job that TIMED OUT (conclusion
'cancelled'), so lint hitting its 15-min timeout cascaded test into
'skipped' and the pipeline published with the whole test matrix and
asan-soak silently skipped. build now requires lint success plus either
test success or the sanctioned skip_tests input; smoke/soak require build
success explicitly.
2. The tag is inputs.version verbatim: dispatching a bare '0.10.7' published
a release the installers can never resolve (they fetch
releases/download/v<version>/...), and under immutable releases the
mis-named tag cannot be retagged or its name reused. A preflight job now
refuses any non-v-prefixed version before anything runs.
3. Lint's 15-min timeout was one slow-runner day away from cancelling a
normally-5-min job; raised to 30 so only a genuine hang can hit it.
Release-path only (workflow_dispatch); adds one ~5s preflight job; no PR-CI
gating, cost, or trigger changes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel