An aborted incremental run preserves the previous index by design, but
the response said generic 'error — check repo_path exists', sending
people to debug a path that was fine. The abort and persist-failure
codes move to the public pipeline header (the contract always said
callers may distinguish them) and index_repository now answers with
what actually happened: aborted_previous_preserved (retry; the old
index still serves) vs persist_failed (disk/permissions) vs the
generic failure.
The incremental test helper goes fail-closed on error responses,
surfacing the real cause once at the source — a transient abort used
to slip past the non-NULL assert and cascade into dozens of mysterious
count failures downstream (the 99-test Windows pile-up).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
/api/who answers the most-sought information type in the developer-
questions literature for one file, from the churn scan the dashboard
already pays for: people with recorded history here, each with their
evidence — commits in this file, repo-wide breadth, and last-touched
where the retained newest commits prove it (absent otherwise, stated
honestly). Deliberate constraints hold: no cross-person performance
framing, no bare percentages, capped list with the distinct-author
denominator.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both predispatch-cancel tests probed the follow-up request with
ingest_traces traces:[] and expected the stub's 'accepted'; empty
traces are a tool error under the real contract. The outer-scope test
now probes with a valid ingest (unresolved names store nothing); the
raw-index test — where no project exists by design — probes with the
project-free list_projects. Clean means what it always meant: the next
request runs unpoisoned by the cancel.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Trace hops and flow steps that actually fired render '▶ observed ×N'
with the run label and date; a fully-observed trace says so in one
line, and the footer states the honesty rule: unmarked hops are
possible, not dead — runs only cover what they exercised. The wiki
gains the observed entry (first-class, with its caps), and F5's gap
narrows to what is still missing (automatic capture).
Live-smoke fix on the C side: resolve_store hands handlers a
query-only connection, so ingest writes silently no-op'd against real
project stores — ingest_traces now takes the dedicated read-write
connection (open_store_for_write, né open_adr_store_for_write), whose
open also creates the observed tables in pre-existing databases.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The stub counted the traces array, answered 'accepted', and discarded
everything. Now: traces are call paths of qualified names (adjacent =
caller→callee; bare {caller, callee} pairs still accepted), stored in
observed_calls/observed_paths — SEPARATE from the graph tables and
keyed by qualified name, because node ids are reassigned on every
reindex and id-keyed observation would silently rot. Counts accumulate
per run label; whole runs age out together past the 200k-pair cap
(oldest first, deterministic under the one-second timestamp resolution
via rowid tiebreaks — eviction is never a timing lottery). The response
is honest about resolution: only pairs whose both endpoints are indexed
get stored, and up to ten unresolved names come back so producers can
fix their naming instead of silently losing data.
Atlas joins observation at read time: trace and flow-detail hops that
actually ran gain observed {count, label, last_seen}; absence adds
nothing — an unobserved hop is merely possible, never 'dead'.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The wildcard swallowed the longer path, answering symbol-history
requests with the symbol bundle's 'missing id or qn' error. Caught
live; route order now states the constraint.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The most-deferred developer question in the literature gets its evidence
surface: /api/symbol-history runs a bounded git log -L over the symbol's
line range (follows renames, 12 commits, on demand — one subprocess,
never automatic), and file history now carries the forge base URL
(git@/ssh remotes normalized to https). The symbol page grows a 'Why is
this here?' card: trace button, commit list with hashes and #123 refs
linked to the forge, honest empty/capped states, and a footnote that
titles are not fetched. The existing recent-commits list linkifies
through the same component. F7's gap narrows accordingly.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
/api/impact answers the strongest-demand developer question in the
information-needs literature: if I change this, what breaks and what
must I retest. A reverse CALLS adjacency joins the flow cache (built in
the same edge pass); a distance-labelled BFS from any symbol reports
reachers per hop, a region rollup, and — separately — the test
functions that reach the symbol, nearest first: the tests to run.
Honest caps (depth 10, 50k visits, both flagged) and a stated basis:
static CALLS only, so every count is a floor.
The symbol page grows an 'If you change this' card: headline sentence,
cumulative hop tiers, region chips, run-these-first test list with an
explicit warning when no test reaches the symbol, and the floor
footnote. Question family F3 flips to answered (7/8/3); the
blast-radius wiki entry now names its caps.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A fixed 30 s startup budget races whatever else the host is doing;
under a parallel test wave a cold daemon loses through no fault of
its own and every client gives up with 'could not start within
30000 ms'. The env override (clamped to 1 s..10 min, malformed
values fall back to the default) lets loaded hosts widen the budget
without changing the default behavior anywhere else. Applied to the
MCP/CLI client bootstrap configs, the upgrade drain request, and
the client cohort-acquire deadline.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- Semantic zoom on the region scene: zooming INTO a region blob
opens it, guarded by the anti-annoyance set from the research
round - user wheel input within 600 ms only (programmatic fly-tos
never trigger), approach direction only, 1.5 s grace so backing
out doesn't bounce back in. The HUD hint says so.
- Flow steps carry the resolver confidence of their parent edge;
hops under 75% show a small warm approx marker with the number -
a misresolved hop now looks different from a certain one.
- Dashboard lists declare their caps honestly: 'top N of M' where
the payload names the population (callables, churned files),
plain 'top N' where it doesn't - never an invented M.
- FilterPanel All|None buttons became one tri-state master checkbox
(checked / unchecked / mixed dash, aria-checked, keyboard-native).
vitest 71/71, tsc clean, C suites 96/0, lint green.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- /api/why entries carry candidates from the CALLS edge; the tree
shows a '1 of N targets' diamond chip on dynamic dispatch -
genuine resolver uncertainty made visible.
- Decision-table pivot on the trigger tree (>=3 guarded callers):
rows = callers, columns = distinct conditions, cells = when /
unless / none. The experimentally-winning format for
condition-combination lookup (Huysmans 2011).
- /api/flow&guards=1: flow-detail steps gain per-step guard chips,
computed from each step's parent call site - journeys read as
'then X, when ok && *cursor' instead of bare arrows.
- Number formatting unified to en-US (mixed browser locales
rendered 1.053 next to 1,053).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The condition→action abstraction (Pennington 1987: the mental model
with the highest error rates, and the one code text serves worst),
served on demand. Design per the research synthesis: thin caller-only
spine (thin slicing), closed question menu, guard text ON the edge
in words (information scent), glyphs always paired with text
(Reacher's 22% missed-icon failures), complexity-gated (Scanlan),
declared elision, both directions (Green), honest footer.
- /api/why?id|qn&dir=up|down: callers (or callees) with the
SYNTACTIC guard chain around each call site — enclosing
if/else/switch-case/loop/ternary/catch conditions, outermost
first, else-arm negation detected — extracted by parsing the few
caller files live with the embedded tree-sitter grammars from the
call-site line the CALLS edges already carry. Zero indexing cost;
works on every existing index. Per-request parse cache; 2 MB file
cap; unparseable sources reported as guards_unavailable, never
guessed.
- /api/trace&guards=1: A-to-B paths gain per-hop guard chips -
'reachable in 5 hops' becomes 'reachable when ...' (the exact gap
Reacher's users hit: they hand-derived conditionals).
- Symbol page: 'When does this run?' tree - guard chips (amber
unless / blue when), loop marker, unguarded honestly labeled,
per-caller upward expansion, direction toggle, single-line answer
when every call site is unguarded, footer stating guards are
syntactic and dispatch/events are invisible.
- Trace results in Flows render the same chips per hop.
Tests: nested-if chain outermost-first + else-arm negation + loop
flag + downward direction + callsite helper on a real temp-dir C
file (C); /api/why route wiring; formatGuard/chain + the
complexity gate (vitest). Trap fixed en route: the extension table
keys WITH the dot ('.c'), not 'c'.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
GET /api/handout renders a self-contained, print-friendly HTML
document that explains the project to a newcomer: the region map
(cohesion as words, hubs), boundary spanners with a why-you-care
line, the main entry-to-effect journeys, and the churn x complexity
table. Linked from the Overview header ('Handout').
Honesty properties, tested: every graph-derived string is
HTML-escaped; no host paths leak into the artifact; caps are
declared inline ('20 of 33 regions shown', '10 of 60 journeys');
a project with no readable git history says so instead of skipping
silently; every fact is deterministic - the header states that no
model wrote any of it.
Exposing the same document as an MCP resource is recorded as a
follow-up (touches the MCP protocol surface; wants its own review).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
/api/blast buckets a file list by region (names included, 200-file
cap declared as truncated, every file accounted for: bucketed +
unmapped == files). The Changes tab feeds it the impacted symbols'
files (falling back to raw changed files) and renders the review
signal git cannot compute: 'your change reaches N files across K
regions' with per-region chips - disambiguated with the same
hub-suffix rule the galaxy uses, so two 'tests' communities don't
render as twins - and a seam warning at K>=3 suggesting a split or
citing the region names in the prompt.
Tests: blast bucketing + accounting + empty-input refusal (C),
/api/blast route wiring.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The D6 trust sweep: every capped or scoped surface says what it hid
and what it means.
- Symbol connections: when the page cuts the list, the hidden
remainder is grouped BY FILE ('src/auth/session.c x 12') next to
the Show-more button - a bare '+N more' hides the answer to 'who
calls this' on exactly the nodes where it matters.
- Connection rows carry the neighbor's region; a row whose edge
crosses an architectural seam gets a quiet accent dot.
- The Data-flows panel no longer renders on projects with no
DATA_FLOWS edges at all: a section that can only ever say 'none'
is noise, not honesty.
- /api/bridges: boundary spanners ranked by DISTINCT foreign regions
a node calls into (cross-call count as tiebreak), test files
excluded and inbound direction ignored - orchestrators like main
and cbm_daemon_host_run surface; utility sinks and test harnesses
(which the naive both-directions metric ranked first) do not.
Overview shows the top 5 with a why-you-care line.
- Staleness banner was built and then REMOVED: no server signal
means 'files changed since indexing' today, and detect_changes
(a git-base diff) would have shown the wrong number. Recorded as
a follow-up instead of shipped with wrong semantics.
Tests: bridges need >=2 foreign regions (a dense gamma community
mirrors the beta fixture pattern), overflow tail accounts for
exactly what the page cut, per-row regions + data-flow presence
flag, /api/bridges route wiring.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The D4 orientation package, mechanics adapted from the two reference
UIs studied in the screened snapshots:
- Search scent propagates up the hierarchy: a 'find a symbol' input on
the region sidebar lights matching regions in the 3D scene, pins hit
counts on the region rows and the minimap, and reports the total
with an outside-regions remainder. Served by /api/scent - a
case-insensitive name match bucketed per region via the cached
file->region map (File/Folder nodes excluded so a file never
double-counts its own symbols; sub-2-char queries refused).
- 2D minimap panel (bottom right, collapsible): every region as a dot
on a fixed top-down projection, sized by sqrt(members), colored by
region, open region ringed, scent counts above the dots, click to
open, and a Home button that fits the current scene.
- Region hover card: members/files, cohesion as a word (tightly /
moderately / loosely connected) with the score, and the hub.
- Node tooltip on graphite tokens (last pre-D1 hardcoded surface) and
in/out call degree - layout nodes now carry out_calls, which the
degree pass already computed and discarded.
- Honest scope note in the region detail panel: '+N callers outside
this region' whenever the full-graph in-degree exceeds the loaded
window.
Tests: scent bucketing + LIKE-escape refusal (C), /api/scent route
wiring, minimap projection + cohesion words (vitest), region-list
queries disambiguated from minimap titles.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The two questions the research says developers ask most and tooling
answers worst: why is this code the way it is (rationale via change
history and ownership), and does A reach B (LaToza reachability,
asked ~9x/day). Plus the user-requested data-flow lens.
- /api/trace: shortest A-to-B path over CALLS or DATA_FLOWS (BFS,
depth-capped at 12), endpoints by id or qualified name; the flow
cache gains a DATA_FLOWS CSR beside the CALLS one.
- Symbol bundle: file_history (recent commits, commits/year, top
author share from a bounded git log at metrics time, cached and
honest - available:false when no history was parsed) and
data_in/data_out with the stored edge detail.
- Flows tab: trace bar with symbol pickers and a control/data
toggle; the result path is clickable hop by hop.
- Symbol tab: 'Recent changes to this file' and 'Data flows'
panels with honest empty states.
- Tests: trace reachability both modes + endpoint errors, symbol
bundle data-flow/history shape, /api/trace route wiring, and a
fetchTrace client test.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The metrics screens now answer questions instead of displaying data
(dashboard canon: Few, Grafana, CodeScene — the Field Guide carries the
citations):
Overview
- Four opinionated slots on top: Needs attention (knee-point count over
churn×complexity), Direction (deltas vs the previous index), Riskiest
area (named, with its why), and an untoned Trust strip (index stamp,
missed files, unmapped symbols).
- Surprising couplings become finding sentences with evidence, an
'inspect' next click, and an 'intended' dismissal whose key buckets by
order of magnitude — a dismissed finding resurfaces only when it grows
10x (regression re-alert, CodeScene's goals idea in one line of code).
- Hubs, entry points and boundaries demote to a collapsed Reference
disclosure; inventory facts move to an untoned footer.
Dashboard
- Hero: the churn×complexity list with the knee-point head highlighted, a
cost sentence ('N files — X% of the codebase, Y% of all commits this
year'), raw commits shown per row, mute-as-intended dispositions, and
rows opening the file's folder in Modules.
- Quality signals (resolution, dead, tested, documented) keep tones and
trends; the complexity histogram now states its takeaway and names the
tail; averages and equal-weight inventory cards are gone — facts live
in the footer.
- Metrics payload gains raw commits per risky entry, churn totals and the
distinct-file count to make the cost sentence computable.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Visual language, rebuilt on the researched rules (Primer/Carbon/Material/
Radix/Linear; details in the Atlas Field Guide):
- Graphite surfaces with fixed step jobs (#0D0F12 canvas → #293038
selected) replace the saturated navy; elevation is a lighter surface
step, not a shadow; hover/selection are surface washes.
- One accent (sky #4FA8E0) reserved for interactivity; semantic
good/warn/crit separate; the type floor is 12px (labels), body 13-14px,
system font stacks (sans UI, mono identifiers only — zero vendored
bytes), tabular numerals, radius 6px.
- Region colors move to the Okabe-Ito color-blind-safe palette, assigned
server-side to the ten largest regions with a neutral tail — the code
graph is the only colorful thing on screen.
Trust repairs, because every wrong-looking item costs belief in every
right one:
- builtins filtered out of hub lists (a 'len' hub reads as a bug);
- a product-code lens hides test regions by default everywhere regions
appear, one honest toggle away ('18 test regions hidden — show');
- flow labels carry file context when the bare name is ambiguous
('datastructures.__init__ → …' instead of '__init__ → …').
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The numbers view of a project, with the code behind every number one
click away and no composite health score — the cards are the score:
- /api/metrics (cached per project+indexed_at): complexity and function-
length histograms and top lists straight from pass_complexity's stored
cyclomatic/cognitive/lines numbers, edge certainty (CALLS vs
CALL_REFERENCE vs USAGE), dead-code candidates (same exclusions as the
galaxy classifier), TESTS linkage, SIMILAR_TO duplication, missed
files, documented-exports coverage.
- Per-file churn from one bounded git log pass (1 year, 10k commits, the
pass_githistory bounds) computed at metrics time — ZERO cost added to
indexing — plus churn × complexity, the classic where-the-bugs-live
list.
- A per-index history sidecar (JSONL in the cache dir, one snapshot per
indexed_at) feeds trend sparklines across reindexes.
- Dashboard tab: stat cards with warn tones, histograms, top lists
linking to symbol pages; churn honestly marked unavailable when the
project has no readable git history.
The wired-routes test covers /api/metrics; the ui suite pins totals,
certainty, histogram completeness and history presence on a fixture.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The human side of the Atlas program:
- Overview: the first read of a project — header numbers, region cards
with provenance and cohesion, hubs by fan-in, entry points, surprising
couplings with enumerated reasons, suggested questions with a why, and
the cross-package boundary table. Assembled client-side from the
read-only /rpc surface (get_architecture format:json) + the regions
payload; no new server code.
- Modules: a squarified treemap (in-house, no new dependency) of any
folder via /api/tree — sized by symbols, colored by dominant region,
hatched where the indexer missed files, breadcrumbed drill-down, file
panel listing symbols via search_graph.
- Symbol: one page per node via /api/symbol — docstring, flags, region,
editor/GitHub links, code on demand, callers/callees with true totals
and per-type confidence badges (resolved / reference / unproven),
tests, co-change partners, near-clones.
- Flows: the /api/flows journeys grouped across/within regions, each a
clickable indented call tree with truncation stated, mermaid export.
- Changes: detect_changes on the working tree grouped by risk, blast
radius citable into the prompt in one click.
- ⌘K: symbols (search_graph) + code text (search_code) in one box.
- Prompt composer: a basket of cited symbols/regions/flows/questions that
renders a prompt naming exact qualified names, file:line ranges and the
MCP calls that reproduce the context — the bridge back to the agent.
Routing carries every screen in the URL (tab, project, path, sym, flow,
node, region) so all of it deep-links. Fixed en route: the /api/flow
route pattern '/api/flow?*' never matched (req->path has no query
string) — now exact, pinned by a wired-routes test that distinguishes a
matched route's 400 from the static handler's 404.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On Django the flow list was all setUpTestData/test_* entries: test
functions are natural call roots (no callers, many callees) so they
out-scored real entries. A test file is a journey into the code, not of
it — candidates in test paths score zero unless the indexer explicitly
flagged them as entry points.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three read-only services the Atlas screens consume, as pure JSON producers
over an open store (handlers stay thin; tests hit the functions directly):
- GET /api/tree?project&path — aggregated children of one folder: files
and symbol counts per child, dominant region (majority vote), missed-
coverage counts from the #963 shadow project, child cap reported rather
than silent. Feeds the Modules treemap at any project size without
shipping nodes.
- GET /api/symbol?project&id|qn — the symbol page in one response: node
with docstring and entry/test/export flags, region membership, callers
and callees with TRUE totals, per-edge-type counts and pagination
(CALLS before CALL_REFERENCE before USAGE), inbound TESTS, co-change
partners of the file (FILE_CHANGES_WITH), and near-clones (SIMILAR_TO).
- GET /api/flows?project, /api/flow?project&id — named entry→terminal call
journeys: entries scored from stored flags and call shape, bounded DFS
over CALLS (depth 6, 4 children per step, 40 steps), deduplicated by
entry/terminal pair, ranked sink-first, truncation counted; cached per
(project, indexed_at) like the region level. The busiest callee leads
each step so the trunk of the journey reads first.
The region cache gains an O(1) file→region lookup (hash-backed) shared by
tree, symbol and flows. Fixed en route: yyjson borrows object keys added
via add_*(), so the per-type count keys are now copied into the document
instead of dangling once their sqlite statement is finalized.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The galaxy loaded 'the first max_nodes nodes' of a project, which above
~10k nodes meant freezing the tab (#498 family) or truncating the graph.
Give it a real coarsest level instead: one body per REGION.
Regions are sets of files. Files are assigned by majority vote of their
callables' Leiden communities over the CALLS graph (seeded, deterministic);
files no kept community explains fall back to folder groups (first two
path components), and a final 'misc' bucket catches the rest. Every node
inherits its file's region, all edge types aggregate into region-to-region
weights, and each region carries its name's provenance ('call community:
202 files, 38% under tests/db_functions'), cohesion, member/file counts
and its top callables by degree.
GET /api/layout?project=X&level=regions -> region bodies + weighted arcs
GET /api/layout?project=X&scope=region:N -> full detail, that region only
Positions come from a golden-angle sphere seed refined by the shared
anchored optimizer (extracted to layout_internal.h alongside the new
cbm_layout_from_nodes core, so the region scope reuses the exact layout
path). Results are cached per (project, indexed_at): on a 48k-node index
the region scene computes in ~1.6s cold, <1ms cached; a 4.8k-node region
scope lays out in ~0.5s.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The graph UI could call only list_projects and get_code_snippet through
/rpc; every other tool answered 403 'UI RPC method is not allowed'. That
starved the browser of schema, search, tracing, architecture and impact
data the server already computes, and broke the Projects tab, which needs
get_graph_schema (#1663).
Allow the read-only tools (get_graph_schema, search_graph, search_code,
trace_path/trace_call_path, get_architecture, query_graph, detect_changes,
check_index_coverage, index_status) and manage_adr in its read modes only
(default get, sections). A duplicated or non-string mode refuses, so a
write cannot ride in behind a read. Mutating tools keep their 403; the
loopback-only boundary is unchanged.
Fixes#1663.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A busy startup transition always resolves, by one of two routes: the live
peer holding it finishes its command, or the holder is already dead and
the operating system has not finished reclaiming the lock. Waiting was
capped at 10s regardless, so a clock decided a user-visible outcome and
commands were refused with "CLI startup coordination remained busy" while
nothing was actually wrong.
The second route is Windows-specific and is what makes this reachable in
practice. On POSIX the kernel drops flock the instant an owner dies. Windows
byte-range locks do not: Microsoft documents that after a process terminates
holding one, "the time it takes for the operating system to unlock these
locks depends upon available system resources", and that until then "access
to these files may be denied". A loaded CI runner is exactly where those
resources are scarce, and tests/windows/test_daemon_stability.py creates the
condition deliberately by hard-killing daemons with `taskkill /F`, including
a crash-recovery section -- so the next client meets a lock whose owner no
longer exists and is refused for a reason that no longer applies.
The cap becomes a backstop against a peer that never finishes rather than a
budget for healthy contention, and the message it prints now names both
explanations instead of just "busy", which sent reporters looking for a CBM
session that had already exited.
Clean exits already release via main_local_transition_close, so no new
release path is needed; this only affects what happens after an abrupt
termination or under genuine concurrency.
VERIFICATION LIMIT, stated plainly: this cannot be reproduced or verified on
macOS -- the mechanism requires Windows lock-reclaim semantics, and a local
12-client storm passes even under the old cap. 458 daemon/ipc/runtime/cli/
watcher tests pass and lint is clean, but whether this removes the CI flake
can only be established on Windows CI.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three related defects in how the watcher decides what counts as a change.
1. `git rev-parse --git-dir` walks UP the tree, so an ordinary folder that
merely sits under an unrelated repository answered yes. The watcher then
inherited that ancestor's dirty state -- permanently non-empty and about
entirely different files -- and reindexed on every poll forever. A
directory is now git-managed only if it carries its own .git (directory
or gitlink file) or the ancestor actually tracks something inside it, so
a real monorepo sub-package still qualifies while a scratch or ignored
folder does not.
2. `git status` reports the whole repository regardless of -C, so every
package in a monorepo reindexed whenever any sibling was edited. The
status call is now scoped with a `-- .` pathspec.
3. Porcelain paths are REPOSITORY-relative but were stat'ed against
root_path. For a project watched at the repository root the two coincide
and the bug is invisible; for a subdirectory project every stat missed,
silently degrading the dirty signature to text-only and losing the
size/mtime component that makes an edit to an already-dirty file
detectable. Paths now resolve through `rev-parse --show-cdup`.
--show-cdup rather than --show-toplevel because MSYS/Cygwin git returns a
translated absolute path from the latter that will not join onto the
native path we hold; a relative hop composes on every platform.
Distilled from #1001 by bethzyy, whose diagnosis identified all three.
Main independently fixed that PR's two headline problems (the unbounded
non-git file count, and per-poll churn via a dirty-state signature) while
it waited, and the watcher was rewritten onto a supervised argv-spawn
harness in between, so this is a reimplementation against the new code
rather than a rebase of theirs.
Both new tests are verified binding: reverting the classification guard
reddens the nested-directory test, and removing the pathspec reddens the
monorepo one, each without disturbing the other.
Addresses the remaining parts of #713, #841 and #937.
Co-authored-by: bethzyy <bethzyy@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A dbt model is an ordinary .sql file whose dependencies are written
{{ ref('other_model') }} or {{ source('group','table') }}, never as
literal table names. The SQL grammar cannot read those -- FROM {{ ref(
'x') }} is a parse error to it -- so the dependency structure of an
entire dbt project was invisible to the graph.
extract_dbt.c runs as a sub-extractor inside the normal indexing
pipeline. Per qualifying file it emits one Model definition (named by
the file stem, dbt's own model identity) plus one usage per ref()/
source() call; pass_usages resolves those into model -> relation
lineage edges like any other reference.
Model is a relation label alongside Table and View, which is what makes
the rest work without new plumbing: registry seeding, the central
relation veto, the incremental surface hash, search ranking and the
architecture queries all pick it up from cbm_label_is_relation. Sharing
one label class also means a model's source('raw','customers') resolves
onto a Table declared in a plain DDL migration in the same repository,
so dbt lineage and SQL lineage form one graph rather than two, while
the veto keeps model names out of every non-lineage consumer.
The pass is self-gating: SQL files only, and only those carrying a real
ref()/source() call. The dbt builtins are themselves the evidence, which
is cheaper than a dbt_project.yml lookup and more precise -- templated
SQL that is not dbt (an Airflow {{ ds }} parameter) produces no Model
node and no usages even inside a dbt repository.
{% macro %} definitions are deliberately excluded: the vendored
tree-sitter-jinja2 grammar has no node types for {% %} statements, so
they can only be recovered by a hand-written scanner that cannot see
comments or find {% endmacro %} for a correct span. Filed as follow-up
rather than approximated.
Tests cover the lineage, the last-string-argument semantics of both
builtins, the gate (against a plain-SQL control extraction), plain DDL
staying untouched, and an end-to-end pipeline case asserting model ->
model across files, model -> Table onto plain DDL, and cross-language
isolation. Disabling the pass reddens three of them; removing Model from
the relation set breaks lineage outright.
Implements the lineage half of #575.
Co-authored-by: alexisperinger-ux <alexis.peringer@iss-stoxx.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
CREATE TABLE / CREATE VIEW / CREATE MATERIALIZED VIEW now produce Table
and View nodes (previously generic Variable), CREATE PROCEDURE produces
a Function, and schema-qualified DDL names (schema.table) are named by
the table identifier instead of the schema. A view's FROM/JOIN relations
are emitted as usages and resolve into view -> table USAGE lineage
edges.
Relations join the cross-file name registry so lineage can resolve, with
two structural safeguards:
- Registry membership is defined once by cbm_label_is_registry_symbol
(helpers.c); the full, parallel and incremental seed sites all call
it, ending the KEEP-IN-SYNC copies the old label lists required.
- The default cbm_registry_resolve vetoes relation-labeled results:
common table names (users, orders, config) collide with code
identifiers in every language, and no CALLS/USAGE/READS/WRITES/THROWS/
handler/decorator consumer may bind them. The SQL lineage path opts in
through the new cbm_registry_resolve_lineage.
Table/View also join the registry-only per-file LSP surface labels so a
table rename invalidates dependent SQL files on incremental (no stale
lineage edges), rank with the type tier in BM25 search, and appear in
the architecture boundary/package/cluster queries via the pinned
CBM_SQL_RELATION_LABELS fragment.
Tests: extraction trio (labels, lineage usages, schema-qualified names),
grammar golden + probe updates, relation-label contract pin, and two
pipeline tests: cross-language isolation (binding: fails without the
veto) and incremental table-rename stale-lineage.
Closes#574.
Co-authored-by: alexisperinger-ux <alexis.peringer@iss-stoxx.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Second mechanism of #1196, exposed by the reporter's v0.10.6 retest: every
relationship-expansion site capped its output buffer at bind_cap*10, so edges
past the cap were silently dropped BEFORE WHERE and aggregation ever ran.
count() then reported the scanned prefix as if it were a fact — 9,360 of
13,691 DEFINES field-measured at --max-rows 1000 — and a LABEL on the source
did not protect you (the label workaround only ever fixed source
enumeration, which #1323 already made exact). max_rows is an output-row
limit per the public header; projection already enforces it.
All five capped sites now share one growable append (geometric growth,
size_t sizing): the per-hop expansion, its fixed/variable-length helpers and
process_edges, the bound-terminal driver, and the cross-join outer buffer.
Only allocation failure stops materialisation; match_count stays truthful
either way, so the #627 OPTIONAL contracts hold (a saturated buffer can no
longer exist, and the fallback rows share the same append). The #601
deadline still bounds pathological time, and hop caps keep bounding depth —
this removes only the silent row-dropping.
Regression test: 2 labeled sources x 30 edges with max_rows=2 — the old cap
returned count=20; ground truth 60 now holds, and the list form returns
exactly max_rows rows. Proven RED before and RED again on revert; cypher
181/181 and mcp suites green, including every OPTIONAL/#627 semantics test.
Fixes#1196 (together with #1323, which fixed the unlabeled source scan).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Trivial insertion-point collision in tests/test_mcp.c: both this branch and
main (#1704's UTF-8 pin) add a Windows search_code test at the same spot.
Both tests kept.
PowerShell 5.1 encodes stdout for a native-process pipe in the console
OEM codepage, so raw search content containing characters the inherited
CP cannot carry (Cyrillic under CP437/850, ...) reached
collect_grep_matches as '?' — and whether it degraded depended entirely
on which console the server happened to inherit. That surfaced as the
intermittent test_mcp raw-Русский mojibake on the windows CI leg and
means real Windows users in a default console get '?' for all
non-ASCII search_code raw content.
Every generated command now pins [Console]::OutputEncoding to UTF-8, so
the pipe is codepage-independent by construction. The read side needs
no pin: Select-String decodes BOM-less UTF-8 via .NET StreamReader
defaults. A Windows-side builder test asserts all five command variants
carry the prelude.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1181 gave list_projects pagination parameters, retiring the last
empty-properties schema — and the two #1359 regression tests leaned on
list_projects as their live zero-argument example, so main went red the
moment the merge train composed (the PR was green on its July base, which
predated these tests).
The gate's schema→decision core is split behind a CBM_CLI_ENABLE_TEST_API
seam, so the zero-argument branch stays pinned directly (empty properties,
absent properties, populated properties) regardless of what the registry
ships; list_projects now asserts its NEW truth (piped args accepted, TTY
still refused); and the schema↔gate parity sweep keeps running over every
tool without the impossible >=1 zero-argument floor. Production behavior
is unchanged — this is the tests catching up with an intended schema
change, plus a seam so they never again depend on a shipped example.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Pi 0.84.2 calls execute(toolCallId, params, signal, onUpdate, ctx).
The generated (args, ctx) shape bound the call id as the MCP arguments.
Forward params and signal, pin the @earendil-works/pi-coding-agent
contract in the generated header, and lock the 5-arg form in tests.
Signed-off-by: Alex Musichen <alex.musichen@gmail.com>
Martin Vogel