The Python exemption for a singular test/ directory was meant for shipped
packages like django/test/, but it also applied to a repository's own
test/ suite. Helpers such as test/helpers.py or SQLAlchemy's
test/orm/_fixtures.py became production components, their indexed is_test
flag was ignored, and a route handler there would count as an entry point.
Only a nested test/ is now exempt; a top-level test/ stays a test
directory for every language, together with its indexed flag. The segment
and suffix checks use small helpers instead of hand-counted lengths.
Django and the cbm control repository are unchanged.
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
The 4096 overview entries were picked by one global rank, so groups whose
components are all small got none. On Django the 785-component @no-source
group, js_tests and the .github test group listed zero entries, and the UI
can neither open such a group nor offer its components in the picker.
Displayed components are listed first, then each group's own top-ranked
components up to 16, and the global rank fills the rest of the 4096. With
at most 64 groups the shares always fit.
The new overview field is renamed to omitted_component_entries: it counts
overview entries left out, which the root limits.omitted_components
(components not displayed) does not.
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
On Django every handler that the route rule added came from a test-client
call such as self.client.get("/get_view/"), which the extractor indexes as
a route registration: the Route's only CALLS edges come from tests/. The
four false entry points took 11 of the 12 default path slots, and main
dropped from 4 paths to 1.
A handler now counts only when its Route is registered from non-test code
or has no registration call at all (decorator routes). Indexed
is_entry_point nodes come before handlers, so paths serve main first. The
label checks move to load time.
Django: entry points main (4 paths) and delete_selected (7 paths) instead
of five with main at 1 path.
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
The projection took entry points only from the indexed is_entry_point
flag, which the extractor sets for main and JS/TS exports. Python web
code never calls main, so Django's Behavior list stayed empty even once
the projection rendered.
Mark a Function or Method as an entry point when it has a HANDLES edge
to a Route node, the same route evidence atlas_flows.c already scores.
Handlers in test code stay out: they are fixtures, and the projection
keeps test and production roles apart everywhere else. The index and the
extractor are unchanged.
For Django 5.2.7 this adds four production handlers to main (five entry
points, 12 paths instead of 4). The other 130 indexed handlers are test
methods and stay excluded.
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
ap_test_path missed Django's app convention tests.py and treated every
file below a singular test/ directory as test code, so django/test/,
Django's public testing API, became test components.
Python discovers tests by file name, so recognize tests.py and
conftest.py next to the existing test_*.py and *_test.py rules, and stop
a singular test/ directory from being the only signal for a .py file
without a test name. The indexed is_test flag of such a file comes from
the same directory rule at extraction time, so it is set aside there as
well; otherwise the class (unflagged) and its methods (flagged) would get
different roles and split into separate components. tests/, __tests__/
and test/ for other languages keep their meaning.
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
get_architecture(system_structure) returned status "limited" for Django
5.2.7 (52,402 nodes, 274,549 edges, below both graph budgets). The
overview listed one entry with a full representative symbol for each of
its 12,700 components. That section alone grew the mutable document to
16.7 MB, the capped components and dependency witnesses took it to
29.4 MB, and serializing the result then crossed the 32 MB response
budget, so the whole projection fell back to the empty limited render.
List at most 4096 overview entries, chosen by the same rank that picks
the displayed components, so every displayed component keeps its entry.
Groups still carry every component id and all counts, and the omission
is explicit: overview.complete is false and overview.limits reports
omitted_components. Graphs with up to 4096 components are unchanged
apart from that new zero count.
Django now renders "ready" with 12,700 components, 256 displayed, 2048
dependencies and 4096 overview entries in a 4.1 MB result (peak 23.2 MB
of the budget, below the 8 MiB job result cap).
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
Separate agent configuration from chat visibility, keep chat compact, and
resume selection explanations after manual questions. Restore project
conversations locally for 24 hours with bounded storage and clear history.
Move daemon configuration into System and report persisted index metadata
and live watcher registration alongside its configuration control.
Validated with 303 focused frontend tests, TypeScript/Vite production build,
160 native watcher/HTTP tests, regression checks, and local browser checks.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add focused galaxy tracing, edge filters, organic cluster layouts and cached
graph exploration with compact architecture and impact summaries.
Make the browser-local agent explain the current selection proactively,
and add a Config panel with persisted daemon overrides, shared browser
preferences and a reference for external configuration inputs.
Validation: 182 focused frontend tests and TypeScript passed. Reused the
recorded production build and 485 passing native checks on unchanged
source. The pre-existing capped-layout loading-note test failure remains
documented separately; the full local CI ladder was waived.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add source-backed system structure and behavior analysis, container service maps, and graph navigation with directional edge styling. Refine repository metadata, graph focus resets, and local browser chat with a resizable Explorer graph.
Move index coverage into its own workspace and diagnostic history into System; simplify Explorer actions and header controls.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add spatial overview, dependency, entry-point and route views with source drilldown, directed relationship evidence, bounded graph expansion, and a plan view. Keep existing architecture details available and expose coverage and display limits.
Validation: frontend build, 2503 unit tests, 203 acceptance tests, and browser workflows on indexed dependency-cruiser, LikeC4 and CBM passed. Broader Linux and Windows native checks still have recorded failures.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add source-backed repository navigation, selection context and a shared change
analysis workspace for symbols, working-tree changes and Git comparisons.
Persist agent events and project logs through the existing daemon with SQLite
schema 4, bounded polling, retention and restart-safe deduplication.
Repair compact RPC decoding, path-search races, source pagination and identity,
large map responses, live error propagation and explicit local diagnosis flows.
Include the real-repository browser runners and focused regression coverage.
Validation: 2473 frontend tests, 203 acceptance checks, production build,
style/promise gates, 5 reference-boundary checks and 7 local hook tests pass.
The complete repository-map browser flow also passes after daemon restart.
Global C lint and cross-platform release validation remain open and are
recorded in the accompanying handoff report.
Signed-off-by: Bernhard Jackiewicz <bj@techport.io>
Retain the existing frontend diagnostics and unreadable-file reporting
alongside the selected-code inspector, graph navigation, and local chat.
Keep both process telemetry and log-tail models, and preserve the HTTP
logging tests alongside the browser model download policy checks.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add Explore, Galaxy, Architecture, Agents, and System workspaces with a
persistent, resizable local chat dock. Attach exact source selections and
graph context, render Markdown, and offer pinned opt-in browser models.
Keep the galaxy visible below chat and focused on the active file or code
selection. Restore hierarchy root selection, resolve projection identities
by symbol, and retain loaded search matches after an empty index reply.
Expose process measurement units, embed browser runtime MJS/WASM assets,
and constrain binary-audit exceptions to reviewed dependency strings.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The browser console was the only place Atlas wrote to, and it dies with
the tab: a reader who saw "get_code_snippet returned no source" had
nothing to hand over, and an agent looking into it had nothing to read.
The page now keeps a second copy of everything it logs, on the server,
next to the daemon's own records.
Server (src/ui/http_server.c):
- POST /api/ui-log takes {page, session, entries[]} and appends one JSON
line per entry to <cache_dir>/logs/ui.log (received, page, session,
seq, ts, level, source, message, detail, stack, url, line, col).
Bounds: 64 KiB per post, 100 entries per post, 4096 characters per
text field cut on a UTF-8 boundary and marked, the level normalised to
a known word, every string escaped again on the way out. The file
rotates once to ui.log.1 at CBM_UI_LOG_ROTATE_BYTES (default 5 MiB).
Warnings and errors are mirrored into the ring GET /api/logs serves,
so both sides of one failure sit in one list; the chatter stays in
the file. The reply names the file and says whether it could be
written.
- GET /api/ui-log?lines=N tails the file (default 200, at most 1000,
from the last 256 KiB) and names the file and its rotated
predecessor; `partial` says when the lines do not start at the top.
- cbm_ui_log_file_path() is the one place the path is built.
Frontend (graph-ui):
- app/ui-log.ts: a bounded buffer (200 entries, oldest dropped first
and the loss named at the head of the next batch), batched posts
(25 entries or 1.5 s), backoff on a refused post, never writes to the
console itself. app/ui-log-transport.ts posts with fetch, and with a
beacon when the page is going away.
- app/ui-log-install.ts wraps the five console methods (they still
print; the copy is taken after), listens to window error and
unhandledrejection, and flushes on pagehide. Installed first in
main.tsx, so nothing can fail before the copy exists.
- provider/error-observer.ts is the seam for failures the panels catch
and show as a sentence: the /rpc transport and the /api client
announce every failed call there before throwing, with a level that
tells a tool the server does not offer (info) from a tool that said
no (warn) from a broken or unreachable server (error). The reader
announces a file the index does not know.
- The projects panel shows the tail under "This server" as sentences,
names the file to attach to a report, and reloads it with the rest.
- README (both) and INVENTAR.md name the file and the routes.
Tests: three httpd cases (JSON lines and tail, refused bodies and
dropped entries, rotation); vitest for the buffer, the wiring in jsdom,
the transport, the seam through both clients, the API method, the model
and the panel. Verified live against a real project: the reader's
failure appears in the file and in the panel within two seconds.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Re-stacks the CodeAtlasWeb branch on the updated base (feat/atlas-r1 at
a44732b6, which merges main 7b0f553c). Ten conflicts, all in the seam
between the previous frontend and its replacement:
- README.md: main's surface count (45) with this branch's Atlas line.
- graph-ui/package-lock.json, graph-ui/src/App.tsx: CodeAtlasWeb's own
files stay; the previous frontend's edits to the same paths do not
apply.
- Seven previous-frontend files main had modified (GraphTab,
NodeDetailPanel and its test, StatsTab test, useProjects, rationale
and its test) stay deleted, and two tests main added for that
frontend (App.test.tsx, useProjects.test.tsx) are not brought in;
their subjects no longer exist in this tree.
Nothing under graph-ui changes relative to this branch; the C side
takes main's lean-output work and the branch's merge resolution.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the Atlas branch up to current main (7b0f553c). Eight files
conflicted; how each was settled:
- src/mcp/mcp.c: the real ingest_traces description stays, in main's tool
row shape (main dropped the title column); index_repository keeps the
truthful abort branches (aborted_previous_preserved, persist_failed)
and gains main's format_migration flag in the success branch.
- src/ui/http_server.c and tests/test_httpd.c: the branch's twelve-tool
read allowlist for /rpc (#1663) over main's three, with main's %zu
and initialised counter.
- README.md: main's surface count (45), the branch's Atlas line.
- graph-ui (the previous frontend, replaced by the stacked #2068 but
kept building and green here): the branch's node-selection wiring
(onSelectNode) is kept consistently, so main's duplicate
handleSelectPath and its 3-argument call go; main's version chip is
added to the branch's header; useProjects keeps the branch's
render-first flow on top of main's paginated JSON helpers
(fetchAllProjects, fetchFullSchema), which the lean output contract
requires, and a project whose size the server did not send now gets
its schema too. main's GraphTab.selection.test.tsx, written against
main's GraphTab, is dropped in favour of the branch's own #1197
coverage in GraphTab.regions.test.tsx ("reports selected nodes to
the route for deep links"); main's useProjects pagination test waits
for the lazy schema enrichment instead of asserting before it runs.
Verified: mcp, httpd and ui suites 414 passed; the old frontend builds
and its 149 tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer glue for the imported frontend; no product behaviour of the
UI itself changes.
- Makefile.cbm: `npm ci` runs with PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1.
The UI's dev dependencies include playwright for its local browser
proofs, and a plain install would download browsers that no build or
gate launches.
- graph-ui/vite.config.ts: the dev server listens on 5173 with the two
literal loopback origins, which is what
tests/test_ui_dev_proxy_security.sh pins.
- graph-ui/index.html: lang="en"; the catalog is English.
- CSP: the directive string moves to http_server.h as CBM_UI_CSP_VALUE
so a test can pin it, and connect-src now admits the two loopback
services the UI may talk to when the reader starts them, the
local-model sidecar on 127.0.0.1:4141 and the agent-event bridge on
127.0.0.1:4142. Under connect-src 'self' both fetches were silently
blocked in the embedded build. Every other host stays forbidden; a
new httpd test asserts that the only hosts in the policy are those
two loopback ports.
- scripts/security-ui.sh: two false positives that bit on the imported
source. The analytics pattern matched the English word "plausible"
(now plausible.io), and `\s` inside an ERE bracket is a literal
backslash and 's', which cut every localhost URL at its first 's' and
then failed the allowlist (now [:space:]).
Verified: httpd suite 64 passed; dev-proxy contract and UI security
audit pass; graph-ui builds and its 2024 vitest tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Land the four residual hardening points from #938 on the current
semantic-search design (semantic_limit/offset, the certified top-K
fetch loop):
- extract_semantic_keywords rejects a semantic_query array that holds a
non-string element instead of silently running the string subset; the
caller gets the same isError as for a bare-string semantic_query.
- run_semantic_query_core fails closed when the vector scan itself
fails ("semantic search failed", isError) instead of rendering a store
error as zero matches plus the broaden-your-keywords hint.
cbm_store_vector_search now tells the two states apart: a store with
no node_vectors table (lean index) is CBM_STORE_NOT_FOUND and keeps
the empty-page + moderate/full-index hint that main relied on by
swallowing the "no such table" prepare failure; a failing scan is
CBM_STORE_ERR.
- vs_min_cosine_score hoists the keyword norms (once per search) and
the node norm (once per node) out of the per-(node, keyword) loop.
The integer sums are exact, so every score stays bit-identical.
- vs_append_result checks its strdup results, frees the partial row on
failure and counts a row only once every copy succeeded, so the
caller's free path stays valid.
Two reproduce-first tests in the mcp suite fail on main and pass with
the change: non-string elements produce the type error, and a forced
cbm_cosine_i8 failure produces the fail-closed error for semantic-only
and combined calls while a missing node_vectors table still yields the
lean-index hint.
Refs #915
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-authored-by: Dustin Persek <dustin.persek@protonmail.com>
With the library default back at INFO, the branch's log and progress-sink
tests were still handing the process a WARN level on their way out —
main's `/* restore */` lines had been rewritten to WARN — so every suite
that ran after them in the same process lost its INFO records (the CI
shard that runs `log` before `index_format` is exactly that order).
test_log.c keeps main's tests and restores verbatim; the two branch
tests (the CBM_LOG_LEVEL opt-in and the startup policy) save the level on
entry and hand it back. The progress-sink tests pin the quiet level they
exercise explicitly and restore it, instead of assuming the process
entered at WARN.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The unreadable-operand scenario used mode-0 permission bits, which
Windows cannot express and root ignores (the Colima legs run as root), so
the test skipped on both — and bare SKIP() is forbidden. The scoped file
list deliberately drops every missing or non-regular indexed path before
grep sees it, so a refusal can only come from the scan itself. The scan
seam now delivers one plausible hit and then grep's error status, on every
platform, and the test asserts that neither that hit nor zero matches is
ever reported as a complete page.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- test_daemon_lifecycle.py: daemon coordination is account-wide and
CBM_CACHE_DIR does not move the rendezvous (main's CBM_RUNTIME_DIR
relocation), so a private CBM_RUNTIME_DIR per cache is what makes the
--verbose call cold and lets it observe the spawn hint.
- test_hook_augment.py: the CLI answers in the lean tree form by default;
the guard parses the JSON projection, so it asks for it.
- test_mcp.c: the fail-closed search test skipped on Windows past the
teardown that names its POSIX-only response.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fix-forward after rebasing onto main (545 commits, 26 conflicted-file
resolutions). Where main had shipped a behaviour in the meantime, main
wins; the PR keeps its output contract on top of it:
- search_code: the scan runs under main's supervised, deadline- and
output-bounded runner and feeds the branch's streaming classifier from
the runner's output file; the POSIX wrappers map grep's no-match to 0,
so every non-zero exit fails closed (no exit-1 special case).
- trace_path: strategy/confidence precede args in header and rows (#1542),
in both the tree table and the json legs.
- cypher: relationship expansion is never capped (a cap falsified
aggregates) and unlabeled MATCH scans every candidate; `truncated` is
raised only by a real ceiling — a clamped hop range is probed one hop
beyond the cap and reports truncated only when a candidate exists there.
The plain BFS and the trail BFS both probe one row past max_results so
saturation is observable instead of silent.
- search_code full mode and the final json use main's U+FFFD sanitizer
instead of '?' replacement.
- search_graph json keeps an empty `groups` array for semantic-only calls;
list_projects json echoes offset/limit next to next_offset and accepts
main's include_details spelling for the stats projection.
- discovery budget: main added get_file_outline, compare_graphs, manage_adr
set_sections and the debug/include_details parameters; their descriptions
are trimmed to the lean style and the ceiling moves from 15 to 18 KiB.
Tests that asserted a JSON default now request format:"json"; the
fail-closed search test uses an unreadable regular file (a missing or
non-regular indexed path is deliberately not a scan operand on main); the
non-regular fixture gives its real sources File nodes, as every indexed
project has; list_projects total counts listable projects (the 0-byte
ghost is never a row, so it is not a page either).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Bernhard Jackiewicz