The numbers view of a project, with the code behind every number one
click away and no composite health score — the cards are the score:
- /api/metrics (cached per project+indexed_at): complexity and function-
length histograms and top lists straight from pass_complexity's stored
cyclomatic/cognitive/lines numbers, edge certainty (CALLS vs
CALL_REFERENCE vs USAGE), dead-code candidates (same exclusions as the
galaxy classifier), TESTS linkage, SIMILAR_TO duplication, missed
files, documented-exports coverage.
- Per-file churn from one bounded git log pass (1 year, 10k commits, the
pass_githistory bounds) computed at metrics time — ZERO cost added to
indexing — plus churn × complexity, the classic where-the-bugs-live
list.
- A per-index history sidecar (JSONL in the cache dir, one snapshot per
indexed_at) feeds trend sparklines across reindexes.
- Dashboard tab: stat cards with warn tones, histograms, top lists
linking to symbol pages; churn honestly marked unavailable when the
project has no readable git history.
The wired-routes test covers /api/metrics; the ui suite pins totals,
certainty, histogram completeness and history presence on a fixture.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The human side of the Atlas program:
- Overview: the first read of a project — header numbers, region cards
with provenance and cohesion, hubs by fan-in, entry points, surprising
couplings with enumerated reasons, suggested questions with a why, and
the cross-package boundary table. Assembled client-side from the
read-only /rpc surface (get_architecture format:json) + the regions
payload; no new server code.
- Modules: a squarified treemap (in-house, no new dependency) of any
folder via /api/tree — sized by symbols, colored by dominant region,
hatched where the indexer missed files, breadcrumbed drill-down, file
panel listing symbols via search_graph.
- Symbol: one page per node via /api/symbol — docstring, flags, region,
editor/GitHub links, code on demand, callers/callees with true totals
and per-type confidence badges (resolved / reference / unproven),
tests, co-change partners, near-clones.
- Flows: the /api/flows journeys grouped across/within regions, each a
clickable indented call tree with truncation stated, mermaid export.
- Changes: detect_changes on the working tree grouped by risk, blast
radius citable into the prompt in one click.
- ⌘K: symbols (search_graph) + code text (search_code) in one box.
- Prompt composer: a basket of cited symbols/regions/flows/questions that
renders a prompt naming exact qualified names, file:line ranges and the
MCP calls that reproduce the context — the bridge back to the agent.
Routing carries every screen in the URL (tab, project, path, sym, flow,
node, region) so all of it deep-links. Fixed en route: the /api/flow
route pattern '/api/flow?*' never matched (req->path has no query
string) — now exact, pinned by a wired-routes test that distinguishes a
matched route's 400 from the static handler's 404.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On Django the flow list was all setUpTestData/test_* entries: test
functions are natural call roots (no callers, many callees) so they
out-scored real entries. A test file is a journey into the code, not of
it — candidates in test paths score zero unless the indexer explicitly
flagged them as entry points.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three read-only services the Atlas screens consume, as pure JSON producers
over an open store (handlers stay thin; tests hit the functions directly):
- GET /api/tree?project&path — aggregated children of one folder: files
and symbol counts per child, dominant region (majority vote), missed-
coverage counts from the #963 shadow project, child cap reported rather
than silent. Feeds the Modules treemap at any project size without
shipping nodes.
- GET /api/symbol?project&id|qn — the symbol page in one response: node
with docstring and entry/test/export flags, region membership, callers
and callees with TRUE totals, per-edge-type counts and pagination
(CALLS before CALL_REFERENCE before USAGE), inbound TESTS, co-change
partners of the file (FILE_CHANGES_WITH), and near-clones (SIMILAR_TO).
- GET /api/flows?project, /api/flow?project&id — named entry→terminal call
journeys: entries scored from stored flags and call shape, bounded DFS
over CALLS (depth 6, 4 children per step, 40 steps), deduplicated by
entry/terminal pair, ranked sink-first, truncation counted; cached per
(project, indexed_at) like the region level. The busiest callee leads
each step so the trunk of the journey reads first.
The region cache gains an O(1) file→region lookup (hash-backed) shared by
tree, symbol and flows. Fixed en route: yyjson borrows object keys added
via add_*(), so the per-type count keys are now copied into the document
instead of dangling once their sqlite statement is finalized.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The galaxy loaded 'the first max_nodes nodes' of a project, which above
~10k nodes meant freezing the tab (#498 family) or truncating the graph.
Give it a real coarsest level instead: one body per REGION.
Regions are sets of files. Files are assigned by majority vote of their
callables' Leiden communities over the CALLS graph (seeded, deterministic);
files no kept community explains fall back to folder groups (first two
path components), and a final 'misc' bucket catches the rest. Every node
inherits its file's region, all edge types aggregate into region-to-region
weights, and each region carries its name's provenance ('call community:
202 files, 38% under tests/db_functions'), cohesion, member/file counts
and its top callables by degree.
GET /api/layout?project=X&level=regions -> region bodies + weighted arcs
GET /api/layout?project=X&scope=region:N -> full detail, that region only
Positions come from a golden-angle sphere seed refined by the shared
anchored optimizer (extracted to layout_internal.h alongside the new
cbm_layout_from_nodes core, so the region scope reuses the exact layout
path). Results are cached per (project, indexed_at): on a 48k-node index
the region scene computes in ~1.6s cold, <1ms cached; a 4.8k-node region
scope lays out in ~0.5s.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The graph UI could call only list_projects and get_code_snippet through
/rpc; every other tool answered 403 'UI RPC method is not allowed'. That
starved the browser of schema, search, tracing, architecture and impact
data the server already computes, and broke the Projects tab, which needs
get_graph_schema (#1663).
Allow the read-only tools (get_graph_schema, search_graph, search_code,
trace_path/trace_call_path, get_architecture, query_graph, detect_changes,
check_index_coverage, index_status) and manage_adr in its read modes only
(default get, sections). A duplicated or non-string mode refuses, so a
write cannot ride in behind a read. Mutating tools keep their 403; the
loopback-only boundary is unchanged.
Fixes#1663.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A busy startup transition always resolves, by one of two routes: the live
peer holding it finishes its command, or the holder is already dead and
the operating system has not finished reclaiming the lock. Waiting was
capped at 10s regardless, so a clock decided a user-visible outcome and
commands were refused with "CLI startup coordination remained busy" while
nothing was actually wrong.
The second route is Windows-specific and is what makes this reachable in
practice. On POSIX the kernel drops flock the instant an owner dies. Windows
byte-range locks do not: Microsoft documents that after a process terminates
holding one, "the time it takes for the operating system to unlock these
locks depends upon available system resources", and that until then "access
to these files may be denied". A loaded CI runner is exactly where those
resources are scarce, and tests/windows/test_daemon_stability.py creates the
condition deliberately by hard-killing daemons with `taskkill /F`, including
a crash-recovery section -- so the next client meets a lock whose owner no
longer exists and is refused for a reason that no longer applies.
The cap becomes a backstop against a peer that never finishes rather than a
budget for healthy contention, and the message it prints now names both
explanations instead of just "busy", which sent reporters looking for a CBM
session that had already exited.
Clean exits already release via main_local_transition_close, so no new
release path is needed; this only affects what happens after an abrupt
termination or under genuine concurrency.
VERIFICATION LIMIT, stated plainly: this cannot be reproduced or verified on
macOS -- the mechanism requires Windows lock-reclaim semantics, and a local
12-client storm passes even under the old cap. 458 daemon/ipc/runtime/cli/
watcher tests pass and lint is clean, but whether this removes the CI flake
can only be established on Windows CI.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three related defects in how the watcher decides what counts as a change.
1. `git rev-parse --git-dir` walks UP the tree, so an ordinary folder that
merely sits under an unrelated repository answered yes. The watcher then
inherited that ancestor's dirty state -- permanently non-empty and about
entirely different files -- and reindexed on every poll forever. A
directory is now git-managed only if it carries its own .git (directory
or gitlink file) or the ancestor actually tracks something inside it, so
a real monorepo sub-package still qualifies while a scratch or ignored
folder does not.
2. `git status` reports the whole repository regardless of -C, so every
package in a monorepo reindexed whenever any sibling was edited. The
status call is now scoped with a `-- .` pathspec.
3. Porcelain paths are REPOSITORY-relative but were stat'ed against
root_path. For a project watched at the repository root the two coincide
and the bug is invisible; for a subdirectory project every stat missed,
silently degrading the dirty signature to text-only and losing the
size/mtime component that makes an edit to an already-dirty file
detectable. Paths now resolve through `rev-parse --show-cdup`.
--show-cdup rather than --show-toplevel because MSYS/Cygwin git returns a
translated absolute path from the latter that will not join onto the
native path we hold; a relative hop composes on every platform.
Distilled from #1001 by bethzyy, whose diagnosis identified all three.
Main independently fixed that PR's two headline problems (the unbounded
non-git file count, and per-poll churn via a dirty-state signature) while
it waited, and the watcher was rewritten onto a supervised argv-spawn
harness in between, so this is a reimplementation against the new code
rather than a rebase of theirs.
Both new tests are verified binding: reverting the classification guard
reddens the nested-directory test, and removing the pathspec reddens the
monorepo one, each without disturbing the other.
Addresses the remaining parts of #713, #841 and #937.
Co-authored-by: bethzyy <bethzyy@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A dbt model is an ordinary .sql file whose dependencies are written
{{ ref('other_model') }} or {{ source('group','table') }}, never as
literal table names. The SQL grammar cannot read those -- FROM {{ ref(
'x') }} is a parse error to it -- so the dependency structure of an
entire dbt project was invisible to the graph.
extract_dbt.c runs as a sub-extractor inside the normal indexing
pipeline. Per qualifying file it emits one Model definition (named by
the file stem, dbt's own model identity) plus one usage per ref()/
source() call; pass_usages resolves those into model -> relation
lineage edges like any other reference.
Model is a relation label alongside Table and View, which is what makes
the rest work without new plumbing: registry seeding, the central
relation veto, the incremental surface hash, search ranking and the
architecture queries all pick it up from cbm_label_is_relation. Sharing
one label class also means a model's source('raw','customers') resolves
onto a Table declared in a plain DDL migration in the same repository,
so dbt lineage and SQL lineage form one graph rather than two, while
the veto keeps model names out of every non-lineage consumer.
The pass is self-gating: SQL files only, and only those carrying a real
ref()/source() call. The dbt builtins are themselves the evidence, which
is cheaper than a dbt_project.yml lookup and more precise -- templated
SQL that is not dbt (an Airflow {{ ds }} parameter) produces no Model
node and no usages even inside a dbt repository.
{% macro %} definitions are deliberately excluded: the vendored
tree-sitter-jinja2 grammar has no node types for {% %} statements, so
they can only be recovered by a hand-written scanner that cannot see
comments or find {% endmacro %} for a correct span. Filed as follow-up
rather than approximated.
Tests cover the lineage, the last-string-argument semantics of both
builtins, the gate (against a plain-SQL control extraction), plain DDL
staying untouched, and an end-to-end pipeline case asserting model ->
model across files, model -> Table onto plain DDL, and cross-language
isolation. Disabling the pass reddens three of them; removing Model from
the relation set breaks lineage outright.
Implements the lineage half of #575.
Co-authored-by: alexisperinger-ux <alexis.peringer@iss-stoxx.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
CREATE TABLE / CREATE VIEW / CREATE MATERIALIZED VIEW now produce Table
and View nodes (previously generic Variable), CREATE PROCEDURE produces
a Function, and schema-qualified DDL names (schema.table) are named by
the table identifier instead of the schema. A view's FROM/JOIN relations
are emitted as usages and resolve into view -> table USAGE lineage
edges.
Relations join the cross-file name registry so lineage can resolve, with
two structural safeguards:
- Registry membership is defined once by cbm_label_is_registry_symbol
(helpers.c); the full, parallel and incremental seed sites all call
it, ending the KEEP-IN-SYNC copies the old label lists required.
- The default cbm_registry_resolve vetoes relation-labeled results:
common table names (users, orders, config) collide with code
identifiers in every language, and no CALLS/USAGE/READS/WRITES/THROWS/
handler/decorator consumer may bind them. The SQL lineage path opts in
through the new cbm_registry_resolve_lineage.
Table/View also join the registry-only per-file LSP surface labels so a
table rename invalidates dependent SQL files on incremental (no stale
lineage edges), rank with the type tier in BM25 search, and appear in
the architecture boundary/package/cluster queries via the pinned
CBM_SQL_RELATION_LABELS fragment.
Tests: extraction trio (labels, lineage usages, schema-qualified names),
grammar golden + probe updates, relation-label contract pin, and two
pipeline tests: cross-language isolation (binding: fails without the
veto) and incremental table-rename stale-lineage.
Closes#574.
Co-authored-by: alexisperinger-ux <alexis.peringer@iss-stoxx.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Second mechanism of #1196, exposed by the reporter's v0.10.6 retest: every
relationship-expansion site capped its output buffer at bind_cap*10, so edges
past the cap were silently dropped BEFORE WHERE and aggregation ever ran.
count() then reported the scanned prefix as if it were a fact — 9,360 of
13,691 DEFINES field-measured at --max-rows 1000 — and a LABEL on the source
did not protect you (the label workaround only ever fixed source
enumeration, which #1323 already made exact). max_rows is an output-row
limit per the public header; projection already enforces it.
All five capped sites now share one growable append (geometric growth,
size_t sizing): the per-hop expansion, its fixed/variable-length helpers and
process_edges, the bound-terminal driver, and the cross-join outer buffer.
Only allocation failure stops materialisation; match_count stays truthful
either way, so the #627 OPTIONAL contracts hold (a saturated buffer can no
longer exist, and the fallback rows share the same append). The #601
deadline still bounds pathological time, and hop caps keep bounding depth —
this removes only the silent row-dropping.
Regression test: 2 labeled sources x 30 edges with max_rows=2 — the old cap
returned count=20; ground truth 60 now holds, and the list form returns
exactly max_rows rows. Proven RED before and RED again on revert; cypher
181/181 and mcp suites green, including every OPTIONAL/#627 semantics test.
Fixes#1196 (together with #1323, which fixed the unlabeled source scan).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Trivial insertion-point collision in tests/test_mcp.c: both this branch and
main (#1704's UTF-8 pin) add a Windows search_code test at the same spot.
Both tests kept.
PowerShell 5.1 encodes stdout for a native-process pipe in the console
OEM codepage, so raw search content containing characters the inherited
CP cannot carry (Cyrillic under CP437/850, ...) reached
collect_grep_matches as '?' — and whether it degraded depended entirely
on which console the server happened to inherit. That surfaced as the
intermittent test_mcp raw-Русский mojibake on the windows CI leg and
means real Windows users in a default console get '?' for all
non-ASCII search_code raw content.
Every generated command now pins [Console]::OutputEncoding to UTF-8, so
the pipe is codepage-independent by construction. The read side needs
no pin: Select-String decodes BOM-less UTF-8 via .NET StreamReader
defaults. A Windows-side builder test asserts all five command variants
carry the prelude.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1181 gave list_projects pagination parameters, retiring the last
empty-properties schema — and the two #1359 regression tests leaned on
list_projects as their live zero-argument example, so main went red the
moment the merge train composed (the PR was green on its July base, which
predated these tests).
The gate's schema→decision core is split behind a CBM_CLI_ENABLE_TEST_API
seam, so the zero-argument branch stays pinned directly (empty properties,
absent properties, populated properties) regardless of what the registry
ships; list_projects now asserts its NEW truth (piped args accepted, TTY
still refused); and the schema↔gate parity sweep keeps running over every
tool without the impossible >=1 zero-argument floor. Production behavior
is unchanged — this is the tests catching up with an intended schema
change, plus a seam so they never again depend on a shipped example.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Pi 0.84.2 calls execute(toolCallId, params, signal, onUpdate, ctx).
The generated (args, ctx) shape bound the call id as the MCP arguments.
Forward params and signal, pin the @earendil-works/pi-coding-agent
contract in the generated header, and lock the 5-arg form in tests.
Signed-off-by: Alex Musichen <alex.musichen@gmail.com>
Keep caller-requested discovery scope while rebuilding changed weaker-mode requests at the stronger stored coverage. Export persistent artifacts only after the replacement database generation is published.
Signed-off-by: astandrik <astandrik@yandex-team.ru>
Re-stamp the runtime DACL only when it is actually wrong, and repair cache
children left unusable by the pre-v0.10.3 DACL regime.
The unconditional per-start re-stamp rewrote an already-correct security
descriptor and propagated it to children (#1601 counted eleven no-op
"Security change" USN records against one _config.db in a day), and every
rewrite is a window in which a concurrent atomic publish can be refused
DELETE on the destination (#1620). The old regime's PROTECTED,
non-inheritable ACE also left every child born unusable — the 0-byte
worker-log class behind #1416's diagnosis — so the secured directory now
walks its regular children and repairs any with an empty DACL or a foreign
owner.
The fast path is gated on the ADOPTION-level predicate, not the general
secure() check: lock-directory adoption (private_win_owner_only_dacl)
demands the exact protected owner-only single-ACE descriptor the stamp
writes, while secure() also admits SYSTEM/Administrators ACEs. A fresh
directory with an inherited DACL passed secure(), skipped the stamp, and
stranded every subsequent lock adoption — 59/77 daemon-suite failures on
the real Windows VM. With the ported predicate (SE_DACL_PROTECTED,
single non-inherited owner ACE, FILE_ALL_ACCESS/GENERIC_ALL) the same VM
runs 77/77 and the full suite 7346/0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Codex sanitizes stdio MCP subprocess environments to the names listed in
env_vars. Since #1645 CBM_RUNTIME_DIR relocates the daemon rendezvous, so a
Codex subprocess that does not receive it looks for the daemon in the DEFAULT
location and never finds it — the same silent client/daemon split
CBM_CACHE_DIR caused in #1562. Both names decide WHICH daemon a process talks
to and are now forwarded unconditionally (forward-if-present semantics);
behavioural knobs (log level, workers, budgets) deliberately stay
unforwarded — that broader list remains #1664's open enhancement question.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
gotspatel's live opencode.json stores our entry with backslashes
(`C:\...\codebase-memory-mcp.exe`) while the installer compares its own path
with forward slashes — the same file on disk, refused over the separator
spelling, so op=mcp_install failed on a correctly-installed machine (and on
Windows the dead-path probe rightly reported the binary PRESENT, which turned
the mismatch into a hard refusal).
Ownership comparison now treats `\` and `/` as equal everywhere and folds
case on Windows only, where the filesystem is case-insensitive; POSIX
byte-exactness otherwise holds. The annotated entry that names this binary is
recognised as already satisfied and preserved byte-for-byte.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
PowerShell 5.1's `Set-Content -Encoding UTF8` writes a BOM, so real
Windows-authored Hermes configs start with EF BB BF — and both edit ops
failed content-independently (the reporter's 26-byte reproduction is their
23-byte file plus exactly this BOM; reproduced RED on macOS with the same
bytes, so the platform was never the variable).
The document read now validates past a leading BOM and yaml_doc_init treats
it as a prologue: the first line's structure starts after it, the key lookup
still sees our own section when the BOM immediately precedes it (guarded by a
dedicated no-duplicate-section test), and every edit splices interior ranges,
so the BOM survives writes byte-for-byte. Non-document inputs — keys, entry
blocks, identity scalars — keep the strict no-BOM rule.
Also makes the moved-entry cli test fixture platform-correct: the Windows
dead-path probe can only prove a fixed-drive path absent, so the Windows
branch uses one; a POSIX-shaped path is refused there by design.
Fixes#1656.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two install failures with the same root theme — the entry we write is a
compatibility contract with the agent's parser, and both sides of that
contract needed repair:
goose (#1675): ExtensionConfig::Stdio declares `name` as a required serde
field with no default, and goose's loader silently drops entries that fail to
deserialize — install reported success and the extension was invisible. The
goose block now carries `name: codebase-memory-mcp`; the non-goose YAML
schema stays name-free. A CBM_CLI_ENABLE_TEST_API seam asserts the exact
block bytes per schema.
annotated MCP entries (#1630, the deferred field-merge): dbd20eaa recognised
an entry the client annotated ("enabled": true beside our command/type) but
could only leave it untouched, because replacing the whole entry would drop
the client's keys. config_json_like gains
cbm_json_like_replace_field_raw_if_unchanged — splice ONE member's value,
preserving every other byte (comments, ordering, client keys) — and the
upsert flow uses it on the two AUTHORIZED repair channels only:
- a relocating update (the entry names the previous managed binary), and
- the existing Windows dead-path probe, which previously fell back to a
wholesale rewrite and lost the annotations.
POSIX keeps its doctrine unchanged: a config-supplied path is never trusted,
so a moved-looking entry without that authority is preserved byte-for-byte
and install fails loudly (cli_editor_mcp_preserves_unrecorded_posix_absolute_
entries_without_probe holds). All repair/refusal paths are covered by tests
proven RED on the unfixed flow.
Fixes#1675.
Fixes#1630.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Four legal-YAML constructs from the reporters' real Hermes configs made
`install` fail permanently (any one of them aborted mcp_install and/or
pre_llm_hook_install):
- exact empty flow collections as values (`plugins: []`, `tool_choice: {}`)
— now validated key-only in both the mapping-body and sequence document
scans, mirroring the #1673 empty-mapping exception; non-empty flow
collections stay rejected.
- block sequences at the same indent as their mapping key (column-0 `- item`)
— item lines directly after a value-less key are structure, not malformed
keys, in the root walker, the key matcher, and the sequence mapping-range
walker.
- double-quoted scalars continued across lines with a trailing `\` — the doc
loader now precomputes per-line continuation flags; continuation lines are
value bytes every structural walker skips, and a document ending inside an
open continuation stays an error.
- mid-word quote characters in plain scalars (`LET'S`) — quotes are scalar
indicators only at a node start (range start, after `:`, after `-`),
exactly like the #1639 anchor/alias rule; real quoted values keep their
protection.
Byte-identity alone also froze users on canonicals older releases wrote:
galaxy's entry had `command:` unquoted, and the goose block gained `name:`
(#1675), so the existing entry was declared FOREIGN forever. An entry under
our key now repairs when it parses as a known prior shape (single command
line, or the pre-name goose block) with a codebase-memory-mcp[.exe] command
basename; anything else stays FOREIGN and the file untouched.
End-to-end: both reporters' full configs (iandol 15.6 KB, galaxy 15.2 KB) now
install with zero agent_config errors, every original line byte-preserved,
and the goose upgrade path rewrites the old block in place. Each construct
carries a distilled regression test proven RED on the unfixed editor.
Fixes#1631.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel