Add Explore, Galaxy, Architecture, Agents, and System workspaces with a
persistent, resizable local chat dock. Attach exact source selections and
graph context, render Markdown, and offer pinned opt-in browser models.
Keep the galaxy visible below chat and focused on the active file or code
selection. Restore hierarchy root selection, resolve projection identities
by symbol, and retain loaded search matches after an empty index reply.
Expose process measurement units, embed browser runtime MJS/WASM assets,
and constrain binary-audit exceptions to reviewed dependency strings.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Re-stacks the CodeAtlasWeb branch on the updated base (feat/atlas-r1 at
a44732b6, which merges main 7b0f553c). Ten conflicts, all in the seam
between the previous frontend and its replacement:
- README.md: main's surface count (45) with this branch's Atlas line.
- graph-ui/package-lock.json, graph-ui/src/App.tsx: CodeAtlasWeb's own
files stay; the previous frontend's edits to the same paths do not
apply.
- Seven previous-frontend files main had modified (GraphTab,
NodeDetailPanel and its test, StatsTab test, useProjects, rationale
and its test) stay deleted, and two tests main added for that
frontend (App.test.tsx, useProjects.test.tsx) are not brought in;
their subjects no longer exist in this tree.
Nothing under graph-ui changes relative to this branch; the C side
takes main's lean-output work and the branch's merge resolution.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three findings scripts/security-ui.sh raised on this branch, none new to
the merge:
- The analytics pattern matched the English word "plausible" in the
wiki text, and `\s` inside an ERE bracket is a literal backslash and
's', which cut every localhost URL at its first 's' and then failed
the allowlist. Same two regex fixes as on the stacked branch
(plausible.io; [:space:]).
- graph-ui/src/lib/rationale.ts carried a literal GitHub URL in a doc
comment, and its test built forge links from a literal GitHub base.
The audit forbids external URLs anywhere in the UI source, tests
included; the comment now describes the shape, and the test composes
the same links from a loopback base (the composition is
host-agnostic).
Audit passes; rationale tests 5 passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the Atlas branch up to current main (7b0f553c). Eight files
conflicted; how each was settled:
- src/mcp/mcp.c: the real ingest_traces description stays, in main's tool
row shape (main dropped the title column); index_repository keeps the
truthful abort branches (aborted_previous_preserved, persist_failed)
and gains main's format_migration flag in the success branch.
- src/ui/http_server.c and tests/test_httpd.c: the branch's twelve-tool
read allowlist for /rpc (#1663) over main's three, with main's %zu
and initialised counter.
- README.md: main's surface count (45), the branch's Atlas line.
- graph-ui (the previous frontend, replaced by the stacked #2068 but
kept building and green here): the branch's node-selection wiring
(onSelectNode) is kept consistently, so main's duplicate
handleSelectPath and its 3-argument call go; main's version chip is
added to the branch's header; useProjects keeps the branch's
render-first flow on top of main's paginated JSON helpers
(fetchAllProjects, fetchFullSchema), which the lean output contract
requires, and a project whose size the server did not send now gets
its schema too. main's GraphTab.selection.test.tsx, written against
main's GraphTab, is dropped in favour of the branch's own #1197
coverage in GraphTab.regions.test.tsx ("reports selected nodes to
the route for deep links"); main's useProjects pagination test waits
for the lazy schema enrichment instead of asserting before it runs.
Verified: mcp, httpd and ui suites 414 passed; the old frontend builds
and its 149 tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
tests/test_venue_parity_contract.sh holds every venue to one harness: a
workflow step may provision, move artifacts, or call a canonical leg
entry, and the product-exercising commands live inside scripts/. The
test-ui job ran npm inline and tripped it on the local ladder.
scripts/ci/test-ui.sh is now the whole leg (install without a browser
download, vitest, the style and promise scans, the portable frozen
acceptance checks, the production build); the job calls it, and running
the same script locally is the same leg. Contract green again
(22 workflows marker-checked, 10 walked).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer glue for the imported frontend; no product behaviour of the
UI itself changes.
- Makefile.cbm: `npm ci` runs with PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1.
The UI's dev dependencies include playwright for its local browser
proofs, and a plain install would download browsers that no build or
gate launches.
- graph-ui/vite.config.ts: the dev server listens on 5173 with the two
literal loopback origins, which is what
tests/test_ui_dev_proxy_security.sh pins.
- graph-ui/index.html: lang="en"; the catalog is English.
- CSP: the directive string moves to http_server.h as CBM_UI_CSP_VALUE
so a test can pin it, and connect-src now admits the two loopback
services the UI may talk to when the reader starts them, the
local-model sidecar on 127.0.0.1:4141 and the agent-event bridge on
127.0.0.1:4142. Under connect-src 'self' both fetches were silently
blocked in the embedded build. Every other host stays forbidden; a
new httpd test asserts that the only hosts in the policy are those
two loopback ports.
- scripts/security-ui.sh: two false positives that bit on the imported
source. The analytics pattern matched the English word "plausible"
(now plausible.io), and `\s` inside an ERE bracket is a literal
backslash and 's', which cut every localhost URL at its first 's' and
then failed the allowlist (now [:space:]).
Verified: httpd suite 64 passed; dev-proxy contract and UI security
audit pass; graph-ui builds and its 2024 vitest tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fix-forward after rebasing onto main (545 commits, 26 conflicted-file
resolutions). Where main had shipped a behaviour in the meantime, main
wins; the PR keeps its output contract on top of it:
- search_code: the scan runs under main's supervised, deadline- and
output-bounded runner and feeds the branch's streaming classifier from
the runner's output file; the POSIX wrappers map grep's no-match to 0,
so every non-zero exit fails closed (no exit-1 special case).
- trace_path: strategy/confidence precede args in header and rows (#1542),
in both the tree table and the json legs.
- cypher: relationship expansion is never capped (a cap falsified
aggregates) and unlabeled MATCH scans every candidate; `truncated` is
raised only by a real ceiling — a clamped hop range is probed one hop
beyond the cap and reports truncated only when a candidate exists there.
The plain BFS and the trail BFS both probe one row past max_results so
saturation is observable instead of silent.
- search_code full mode and the final json use main's U+FFFD sanitizer
instead of '?' replacement.
- search_graph json keeps an empty `groups` array for semantic-only calls;
list_projects json echoes offset/limit next to next_offset and accepts
main's include_details spelling for the stats projection.
- discovery budget: main added get_file_outline, compare_graphs, manage_adr
set_sections and the debug/include_details parameters; their descriptions
are trimmed to the lean style and the ceiling moves from 15 to 18 KiB.
Tests that asserted a JSON default now request format:"json"; the
fail-closed search test uses an unreadable regular file (a missing or
non-regular indexed path is deliberately not a scan operand on main); the
non-regular fixture gives its real sources File nodes, as every indexed
project has; list_projects total counts listable projects (the 0-byte
ghost is never a row, so it is not a page either).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The rebase onto main brought the Bash-enabled PreToolUse matcher
(Grep|Glob|Bash), which the exec-form tests still counted as Grep|Glob.
The Windows lifecycle test seeded the .cmd shell spelling as the converged
identity; the shell-free registration converges every owned shell spelling
onto one exec-form entry per matcher instead. The test now asserts that,
plus their removal on uninstall, and hands cbm_install_agent_configs the
'/'-spelled binary path cbm_cmd_install registers (cbm_mkdtemp returns a
native '\' temp root on Windows, which uninstall never derives).
Smoke 8aq proved Claude's gate registration by grepping the shim name out
of settings.json; the exec form carries no shim name, so it now checks the
binary-plus-'hook-augment' PreToolUse hook. 9d's shim-name marker would
otherwise pass vacuously after uninstall; the 'hook-augment' argument is an
owned marker there too.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Step 0o (MCPB bundle contract) and Step 0q (release candidate derivation)
hand the composition gate a stub compiled straight from a .c file with the
compiler's defaults. That stub exists to exercise packaging, format detection
and the needle scans; it is not linked with -z now and never ships. A1d
measured eager binding on it anyway, so on a toolchain that binds lazily by
default (test-diag: clang-22) Step 0o died deterministically with
A1d-bind-now stage/codebase-memory-mcp: GOT section(s) .got.plt fall
outside PT_GNU_RELRO [15824,16384) — they stay WRITABLE after relocation
The property A1d protects is a property of the release link (-z now in
Makefile.cbm), so the assertion is scoped to it: a caller that feeds the gate
a packaging fixture declares so with CBM_COMPOSITION_FIXTURE=1 and A1d is
reported n/a — printed on every run as a named exemption, never a silent
skip. Every other assertion still runs on the fixture unchanged, and release
derivation never sets the variable.
Verified in the Linux arm64 container (Ubuntu 24.04, gcc 13.3): a stub
linked -z lazy reproduces the CI failure text on the gate without the
variable and reports n/a with it; both contracts pass under a CC wrapper
that links lazily (the shape that was red), and unchanged on macOS.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
When a suite leader has already exited, taskkill /T cannot walk its tree,
so run-test-wave.py proves cleanup by asking Windows whether anything is
still parented to the dead PID. That probe was timed with --kill-grace,
which the parallel gate passes as 1s. A cold powershell.exe + CIM start
routinely costs more than that on a runner, so TimeoutExpired -- a fact
about interpreter start-up, not about the process tree -- was read as
"assume the worst" and failed an already-clean shard with exit 2.
Give the probe its own stable-state budget (15s) plus one retry, keeping
the fail-closed semantics: an exhausted or failed probe still counts as
"cannot prove absence". The refusal now names which of the two happened,
an unfinished probe or a counted set of live descendants, instead of
reporting both as a leaked tree.
The contract test pins this structurally -- no sleeps, no timing
thresholds: the probe must be bounded by the module constant rather than
by the kill-grace argument, must retry a timed-out probe, must never read
an unusable probe as absence, and must report the two failures
distinctly. Its Windows leader-exit case now waits on the scheduler's
refusal rather than on a deadline shorter than the new budget.
Refs #1741
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The 8al-node probe imports the generated Pi extension in bare Node with an
empty node_modules. That was a sound stand-in for the Pi host only while
the extension imported nothing but node: builtins; the first package
import (`typebox`, Pi's schema library for tool parameters — a real Pi
dependency, verified against pi-coding-agent 0.84.4 on #1809) can never
resolve there, so the probe fails on a correct extension and would pass a
wrong one just as readily.
Give the probe what the host gives the extension: a Proxy-backed
`typebox` stub in the probe's node_modules (any Type.X(...) returns a
descriptor, so the probe never tracks which helpers the emitter uses — it
only exercises the tool lifecycle), plus an explicit allowlist: every
import specifier in the generated file must be a node: builtin or a
package on PI_HOST_PACKAGES, otherwise the probe fails naming the
specifier. That turns the guarantee the empty node_modules gave by
accident (no surprise dependency) into an asserted contract.
Verified locally by running only the 8al-node block against extensions
generated by two binaries:
main@5802ccdd extension, old probe: OK; new probe: OK
#1809 extension, old probe: ERR_MODULE_NOT_FOUND typebox (the CI red)
#1809 extension, new probe: OK
#1809 extension + an injected 'lodash' import, new probe:
FAIL 8al-node: ... packages the Pi host does not provide: lodash
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The vendoring table claims our copy of a grammar matches the pinned
upstream commit. A local patch makes that claim false unless the patch
is written down, so a future re-vendor would drop the fix without
anyone noticing.
The row covers both changed lines, because both are the same defect in
the same file: OP_SYMBOL_SUPPRESSOR at line 131 and eat_operators at
line 514.
The row also names the two upstream commits that already carry these
changes, and the table intro gains one clause for the case. Upstream
fixed eat_operators in fb63a7004f07 on 2026-04-06 (their issue #558)
and the OP_SYMBOL_SUPPRESSOR entry in 6ab8d1d74ebd on 2026-08-10. Our
pin 8abb3e8b3325 is from 2026-03-20 and predates both, so these are
backports rather than local inventions.
That distinction changes what a re-vendor should do. The table's
instruction is "must re-apply these", which is right for the other four
rows -- upstream never took those. For this row it would mean
hand-applying a change the newer file already has. The row now says to
delete it instead, and the intro allows for that case.
MANIFEST.md is itself listed in scripts/vendored-checksums.txt, so its
own recorded checksum moves with the edit.
Refs #1892
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
scanner.c line 131 shifts a literal 1UL by 32 to build the entry that
suppresses the bang of a try!. unsigned long is 64 bits on Linux and
macOS but 32 bits on Windows, so there the shift count equals the width
of the type, which is undefined.
1ULL is 64 bits on every target this project builds for.
Nothing reports it today. The expression is a compile-time constant, so
no sanitizer sees it run, and Makefile.cbm:719 builds vendored grammars
with -w, which switches off -Wshift-count-overflow.
Upstream tree-sitter-swift made the same change in 6ab8d1d74ebd, after
the commit this grammar is pinned to.
Refs #1892
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
The Swift scanner keeps a 64-bit mask of the symbols that suppress a
match -- the rule that stops `try!` emitting its `!` as a token of its
own. It tests one bit per candidate:
uint64_t suppressing_symbols = OP_SYMBOL_SUPPRESSOR[full_match];
for (uint64_t suppressor = 0; suppressor < TOKEN_COUNT; suppressor++) {
if (!(suppressing_symbols & 1 << suppressor)) {
The mask is uint64_t but the literal `1` is an int, so the shift is an
int shift. TOKEN_COUNT is larger than 32, so once suppressor reaches 31
the shift runs past the width of the type. That is undefined behavior,
and every bit above 31 is tested against a value the standard does not
define.
Nothing caught it because nothing in the tree reached the suppressor
path. Any Swift force-unwrap does: `cached!` is enough.
UBSan reports it as:
scanner.c:514:47: runtime error: left shift of 1 by 31 places
cannot be represented in type 'int'
`1ULL` makes the literal as wide as the mask it is tested against.
The new test in tests/test_extraction.c cannot go red on its own. The
normal test build prints the UBSan message and carries on, which is why
this survived. The Windows CLANGARM64 leg runs UBSan in trap mode, and
there the same shift is an illegal-instruction crash -- so the test
exists to make sure that leg keeps parsing a force-unwrap at all.
scripts/vendored-checksums.txt records the new hash for the one changed
file, as scripts/security-vendored.sh --update writes it. Layer 8 of the
security gate compares vendored content against that manifest, so the
edit and its recorded hash belong in the same commit.
Found while adding Swift URL extraction in #1892 / #1976, and split out
of that PR so the vendored change can be reviewed on its own.
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
Refresh the complete generated Perl grammar bundle from upstream v1.2.1 at c3e17b31179bf8f658c9f37c7a3ea6a202212d5a, raising it from ABI 14 to supported ABI 15. Preserve the existing license and record the source and per-file provenance.
Add deterministic coverage for a valid format followed by a named sub, while retaining a malformed-source control.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Adds -Wl,-z,relro and -Wl,-z,now to ELF_HARDENING_FLAGS, and asserts the
resulting property on the produced artifact rather than on the compiler's
willingness to accept a flag.
The shipped Linux binaries measurably lacked the property. Built the way
.github/workflows/_build.yml builds them (STATIC=1, gcc 13.3 / GNU ld 2.42,
Ubuntu 24.04 aarch64), PT_GNU_RELRO ends at 0x11DA0000 in both cases:
without -z now: .got 0x11d9ec20+0x13c8, .got.plt 0x11d9ffe8+0x40
-> .got.plt ends at 0x11DA0028, 40 bytes PAST the window.
Those GOT slots stayed writable for the whole process
lifetime, in every Linux binary shipped to date.
with -z now: .got.plt folds into .got, the whole GOT sits inside the
window and is re-mapped read-only after startup.
Both flags are free at runtime for a static binary -- everything resolves at
link time, so there is no lazy binding left to pay for. -z relro is already
this toolchain's default and changes nothing here; it is named anyway so the
property stops depending on one distro's spec file, since the musl/portable
and glibc-floor images are different toolchains and a default is not a
guarantee.
A1c/A1d in scripts/ci/check-binary-composition.sh assert the OUTCOME on the
binary via readelf, for the same reason A1 exists beside the .note.GNU-stack
annotation: a flag the linker accepts is not evidence the artifact gained
anything. PR #1138 demonstrated that directly -- its -pie was silently
discarded under -static, and its -D_FORTIFY_SOURCE=2 silently DOWNGRADED
Ubuntu's default of 3, both while every compile probe passed.
The readers use awk rather than a short-circuiting reader on purpose: a
reader that exits at the first match leaves the upstream taking EPIPE, and
under `set -o pipefail` the satisfied case is reported as the failing one.
Deliberately NOT taken from #1138: -D_FORTIFY_SOURCE=2 (a downgrade on the
release platform), -fPIE/-pie (inert under STATIC=1), -fstack-protector-strong
(real but unmeasured cost, and #1138 injected it into SQLite, tree-sitter and
mimalloc -- the indexing hot loop), and the $(shell) probe machinery (~10
compiler spawns per Makefile parse, making the binary a function of the
builder's toolchain).
Co-authored-by: PR9000 <119280965+PR9000@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Seven packaging surfaces declared 0.8.1 while v0.10.8 was the shipped
release — nine releases of drift in metadata published to package
registries, and the third version-metadata drift this month.
pkg/go was not cosmetic: it hardcodes its version into the release URL
it downloads from, so `go install` fetched v0.8.1 binaries.
Chocolatey had to move as a unit. The nuspec is version-only but
chocolateyInstall.ps1 pins the asset sha256, so bumping the nuspec alone
would ship a package declaring 0.10.8 while installing 0.8.1. The new
checksum is the v0.10.8 windows-amd64.zip digest, verified by downloading
the published asset and hashing it, not merely copied from checksums.txt.
The gate (tests/test_version_metadata_contract.sh, scripts/test.sh Step
0x) checks against the newest stable v-prefixed tag, because nothing in
the tree declares the version: CBM_VERSION is injected at build time and
the release is dispatched with a version input. That absence is exactly
why these files drifted. Mutual consistency across the surfaces needs no
git and always runs, so a shallow clone degrades the gate loudly instead
of passing silently.
Homebrew (0.10.3), Scoop and AUR (0.8.1) are left pinned: each embeds
per-release asset checksums, so bumping them is a distribution chore
rather than a metadata edit. They are still gated — internally
consistent, and never naming a release that does not exist — and each
carries a recorded reason for being frozen.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The line read `7x ABI-13, 86x ABI-14, 65x ABI-15`, which sums to 158 against
161 vendored grammars. Counting `#define LANGUAGE_VERSION` in each parser.c
gives 9 / 79 / 73 = 161, so the line was already wrong before Chialisp; bumping
the ABI-14 bucket by one, as the Chialisp change did, would have carried a
four-off error forward under a fresh timestamp.
Recount with Chialisp included: 9 / 80 / 73 = 162, which matches both the
grammar-directory count and the language-count contract. The regeneration
command is now recorded next to the numbers so the next person does not have to
rediscover it:
grep -h '#define LANGUAGE_VERSION' internal/cbm/vendored/grammars/*/parser.c \
| sort | uniq -c
Vendored-integrity gate and the language-count contract both re-run green.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Chialisp (.clsp/.clib/.clinc) is the Chia smart-coin language. The only
public grammar, Quexington/tree-sitter-chialisp, cannot parse it: on the five
reference files from chia-blockchain@main it produced 3-34 ERROR nodes each.
Three defects account for all of them, and each desynchronised the rest of the
file rather than losing one node:
* comments were `/;.*\r\n/` -- CRLF-only, while real files are LF;
* `(include foo.clib)` was rejected because of the dot in the filename;
* `(defconstant NAME <expr>)` accepted only a primitive after the name.
So the grammar is written here instead, in tools/tree-sitter-chialisp/, and
files as first-party: its vendored LICENSE is the project's own, byte-identical
to the repository root, carrying no third-party copyright.
It is deliberately GENERIC -- source_file / list / symbol / string / number /
hex / dot / comment, modelling the clvm_tools reader rather than the Chialisp
form vocabulary. `mod`, `defun`, `defconstant` and `include` are ordinary head
symbols; which lists are definitions is decided in extract_defs.c. A dialect
that adds a form (cl-21 -> cl-23 -> cl-26 each added several) therefore needs
no regenerated parser. Acceptance: all five reference files parse with zero
ERROR and zero MISSING nodes; the three defects above are pinned as tests.
Extraction, and the four decisions in it:
* `Constant` is admitted to the cross-file name registry
(cbm_label_is_registry_symbol). Chialisp's condition codes live as
`(defconstant ...)` in a shared .clib and are referenced by name from every
puzzle; labelled but unregistered they are seeded nowhere, and roughly half
of a Chialisp graph would be present as nodes yet never the target of an
edge. All three registry-seeding sites route through that one predicate, so
they cannot diverge. No other language emits `Constant`, so the blast radius
is exactly Chialisp. Pinned by a two-file graph test that goes red when the
label is removed from the predicate.
* `export` / `namespace` are NOT definition heads. `(export foo)` names a
function already defined in the same file, so treating it as a def head
mints a second node for one symbol. They stay in the not-a-call filter.
* A parameter list is a `list` too, but it binds rather than invokes. Without
that gate `(defun check_conditions (HEIGHTLOCK conditions) ...)` mints a
phantom call to HEIGHTLOCK, and real Chialisp is mostly such definitions.
* `(include *standard-cl-26*)` is a dialect directive, not a file dependency;
recording it would invent a dependency on a file that does not exist.
Three helpers that the defs, calls and unified walks all need -- the quote-data
gate, comment-skipping named-child access, and the Chialisp def-head set -- live
in helpers.c rather than being copied per translation unit. They previously
existed as byte-identical duplicates that had already drifted: extract_defs.c
skipped comments when reading a def head while its mirror in extract_unified.c
did not, so a comment between a head and its name desynchronised defs from call
scope. Both now use the same accessor.
Ledger: 161 -> 162 grammars across the seven published surfaces, ABI-14 86 -> 87,
first-party 13 -> 14, CBM_LANG_COUNT 164 -> 165.
Distilled from PR #919 by darkverbito, whose extraction logic -- the CLVM
not-a-call vocabulary, the quoted-data gate, the filename-derived `mod` name and
the include/embed import handling -- is kept and is the substance of this change.
The grammar that PR vendored is not used.
Co-authored-by: darkverbito <8049985+darkverbito@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The watcher-poll family gets its production fix and its doctrine
rebuild in one pass:
- the dirty-state signature statted files through stat(), whose
st_mtime is SECOND-granularity on Windows — two same-second edits
hashed identically and a reindex went missing (the Windows leg
caught it as test_watcher:2045). The signature now reads
cbm_path_info_utf8 (FILETIME 100ns ticks, wide-path correct) and
retries once when a just-written file is transiently unreadable
under AV real-time scanning
- a test seam exposes the committed/pending signatures, and the suite
gains self-diagnosing count asserts — the family blocked two
releases because a bare count mismatch could not say whether
detection missed or the reindex path misbehaved; now it names which
- reach-assertions get a bounded wait-for-state helper (safe: an
unchanged dirty tree never increments, so extra polls cannot
overshoot); must-stay-quiet asserts keep their fixed polls
py_lsp_scale's quadratic detector samples min-of-N so scheduler noise
on a loaded host cannot inflate the calibrated ratio (noise only adds
time; the minimum approximates uncontended cost — the assert failed
the 18-job mac leg while passing isolated on the same tree).
The full-pipeline probe suites join the slow scheduler tier with their
reasons stated: a fresh checkout's first run pays a deterministic
AV-scan-cold tax that killed node_creation_probe at exactly 900s.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Seven surfaces disagreed with reality and with each other: README.md,
docs/index.html, docs/llms.txt and pkg/npm/README.md said 158, while
server.json, the chocolatey nuspec and scripts/package-release.sh said 159.
161 grammars are vendored.
21 individual claims corrected. The count is now gated by
tests/test_language_count_contract.sh, so the next language to land cannot
ship a stale figure.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The count was hand-maintained on seven separate surfaces and had drifted apart
from reality AND from itself: README, docs/index.html, docs/llms.txt and the npm
README said 158, while server.json, the chocolatey nuspec and package-release.sh
said 159.
Derived from the vendored grammar directories rather than CBM_LANG_COUNT,
because the registry contains entries with no parser at all: CBM_LANG_NIM
(grammar removed 2026-06-12) and CBM_LANG_OBJECTSCRIPT_EXPORT have no grammar
directory, no tree_sitter function and no MANIFEST row, yet files still route to
them by extension. Publishing CBM_LANG_COUNT would claim languages we do not
parse. A vendored grammar directory demonstrably exists and is checksum-gated.
Accepted trade-off, stated in the test: this UNDERCOUNTS languages that share
one grammar (TypeScript/TSX, objectscript_udl/_routine). We publish the number
we can prove rather than the larger number we cannot.
The contract binds four things so it cannot go quietly vacuous:
- the grammar directory count is sanity-bounded, so a layout change fails loudly
instead of making every check below it trivially true
- MANIFEST.md's declared figure must already agree with the directory count
- every registered surface must state exactly that number, and a registered
surface that states NO number fails (the wording changed, or it should be
deregistered)
- any UNREGISTERED file stating a language count fails, so an eighth surface
cannot reintroduce the drift this exists to prevent
Files that legitimately say "<N> languages" about something else — benchmark
corpus subsets, probe scopes, the version-pinned winget manifests — are listed
with what their number actually counts, so no exemption is unexplained.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Vendor AndreasMaierDe/tree-sitter-plsql @ 28aebef209be (MIT, ABI 14, no
external scanner) and wire CBM_LANG_PLSQL end to end:
- extraction onto existing labels only: package spec/body -> Class,
procedure/function (create/definition/declaration) -> Function,
file -> Module; callee extraction for ref_call incl. the
package-qualified pkg.name form; identifier usage vocabulary with the
upstream 'parameter' kind (a ref_call ARGUMENT wrapper) carved out of
the whole-binding rule
- discover: .pks/.pkb/.pck/.pls/.plb/.plsql/.fnc/.trg/.bdy/.tps/.tpb
(.sql stays SQL and .prc stays FORM, both asserted negatively)
- ledgers: CALL_WITHOUT_REFERENCE_VOCAB(PLSQL), call-node manifest
ref_call row (historical total 219 -> 220), call-argument matrix B row
- tests: extraction triple (package body + package-qualified call,
standalone function, honest negative pin of the upstream
CREATE TYPE ... AS OBJECT ERROR-node limitation), grammar-regression
case, label golden Class:1,Function:1,Module:1, MUST_EXTRACT_DEFS,
extension mapping, bug-repro board battery row
- MANIFEST.md (community-niche verdict, local include-quote patch row),
THIRD_PARTY.md section, refreshed vendored checksums
PL/SQL support was contributed by Oguz (@ouzsrcm) in #1033; re-vendored
from upstream per vendoring policy with his language wiring distilled on
top.
Co-authored-by: Oğuz <oguzhan.saricam@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Grok Build (xAI's grok CLI/TUI) already reached codebase-memory-mcp through
its Claude/Cursor compat layer, but a Grok-only user got nothing. Detect
$GROK_HOME (default ~/.grok) or a grok CLI and install:
- an owned [mcp_servers.codebase-memory-mcp] managed block in config.toml;
a same-name foreign table is left byte-identical because a second header
would make Grok reject the whole file, while the known owned shape written
by `grok mcp add` is adopted. No env forwarding: grok 1.0.5 passes the
full parent environment to stdio servers (verified), unlike Codex (#1562).
- an owned rules file under $GROK_HOME/rules/ (always scanned, all projects)
- the shared skill under $GROK_HOME/skills/
- Scout/Verify/Auditor agents with named-server mcpInheritance and the tier
allowlist spelled out as the server__tool ids Grok's search_tool/use_tool
dispatcher accepts (inheritance filters per server, never per tool)
- hooks withheld: Grok's passive events discard stdout and PreToolUse honors
only deny/rewrite decisions, so there is no context-output contract.
Verified against grok 1.0.5: inspect discovers the server, agents, skill and
rules; mcp doctor handshakes and discovers 15 tools. --clients=grok token,
README/npm/llms/landing-page matrices (43 -> 44 surfaces), and smoke coverage.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Layer 8 (scripts/security-vendored.sh) failed because the seven new arkts
files were untracked in the checksum baseline. Regenerated with --update
after reviewing the diff: the only additions are the arkts grammar's own
files, plus MANIFEST.md's hash changing because the preceding commit edited
it. Nothing else was re-blessed.
Verified independently rather than taken on trust:
arkts/_common_scanner.h sha256 == typescript/_common_scanner.h sha256
(da66ef2bd14a...), i.e. the external scanner body is the already-reviewed
typescript one, and arkts/scanner.c is a 21-line symbol-rename trampoline
that allocates nothing and delegates scan straight through.
All three security-static layers now pass locally: static allow-list, UI
audit, and vendored integrity -- the last also reporting no subprocess,
network, or dangerous calls anywhere in vendored grammar code.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The vendored LICENSE carried tree-sitter-typescript's MIT text plus an
appended derivative notice. That is not what MIT asks of a derivative -- it
asks that upstream's notice travel with the code -- and the appended text
meant the provenance audit could not byte-verify the file against upstream,
so arkts landed as NO-MANIFEST-ENTRY and failed license-gate.
The vendored LICENSE is now byte-identical to upstream's, and arkts is
registered in the audit's FORKS map, which is defined for exactly this case:
a self-maintained fork whose vendored LICENSE must match its upstream. That
is a routing entry, not an exemption -- check_upstream still byte-verifies
against the real upstream repository, so the fact is checked rather than
asserted. Audit now reports arkts as IDENTICAL.
Our copyright for the ArkTS additions is not lost, it moves to where our
SOURCE is: the grammar.js header (which now states all three copyrights
explicitly) and THIRD_PARTY.md, which ships inside the release archives.
The edit to scripts/audit-license-provenance.py is a maintainer-only
enforcement-surface change, made deliberately and stated here rather than
slipped in; a contribution may never supply its own audit entry.
No generated parser or source changes -- comments, licence text, manifest
prose and one routing entry only.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up to the PR #1105 review: the unit test proves the config predicate but
would still pass even if the daemon-side gate were removed. Add a process-level
regression that drives the real binary and proves the gate actually prevents the
watcher from being built, started or registered when disabled.
- tests/test_watcher_disabled.sh (wired into scripts/test.sh as Step 5e) drives
the binary against an isolated CBM_CACHE_DIR + tiny git fixture. With
watcher_enabled=false it asserts: watcher.disabled reason=config emitted,
watcher.start absent, no watcher.watch registration, auto_index still runs, and
manual index_repository is served. A watcher_enabled=true positive control
asserts watcher.start + watcher.watch appear, so the test fails if the gate is
removed (verified by neutering the gate in host_state_prepare and observing the
failure).
No assertion is decided by a timeout. The watcher lives in the daemon, which
logs to $CBM_CACHE_DIR/logs/cbm-daemon.log in a fixed order: watcher.disabled
(host_state_prepare) -> daemon.start -> the thread's own watcher.start -> the
join -> daemon.stop. Each run therefore retires its daemon and waits for that
CLOSED lifecycle before asserting, so absence is a fact rather than a race.
Every wait is a bounded poll on an asserted state following the soak harness's
wait_for_daemon_stop doctrine, and an exhausted budget fails loudly with the
reason instead of passing quietly; session stdin is held open through a FIFO
until the last JSON-RPC response lands, not for an interval.
Registration is asynchronous and unreachable until the project DB exists
(application_refresh_watch_locked requires application_regular_db_exists), so a
closed lifecycle alone would not carry the negative — the session could simply
have ended first. The disabled+auto_index run waits for that DB, i.e.
registration's own precondition; the enabled control waits for watcher.watch
itself, so its absence in the disabled runs is evidence rather than noise.
Honors CBM_TEST_BINARY like tests/test_worker_watchdog.sh, so the BUILD_DIR=
sanitizer and container legs test the binary they built.
- docs: CONFIGURATION.md + README.md document that watcher_enabled is read once
when the daemon starts — the daemon outlives individual MCP sessions, so
reconnecting a client is not enough; `daemon stop` retires it. This corrects
the earlier claim that auto_watch is also startup-read: auto_watch is consulted
per registration, and the two keys are now contrasted on read time as well as
on scope (session registration vs whether the subsystem starts at all).
No production code change in this commit: watcher.start and watcher.watch already
exist as production logs.
Signed-off-by: soren <nguiasoren@gmail.com>
Align the exact Claude PreToolUse smoke contract and documentation with the Bash-enabled search matcher.
Signed-off-by: Jake Abendroth <contact@jakea.net>
scripts/test.sh labels its pre-build contracts as a running 0a..0u sequence.
The new blob-generator contract was slotted in as "0f2" to sit next to the
other build-tooling contract, which breaks the one convention the list has.
Append it as 0v instead; these steps all run before the build, so position
carries no meaning beyond the label.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
vendored/nomic/code_vectors_blob.S is a GENERATED file that was hand-edited.
1f674c80 added the ELF .note.GNU-stack section to it after finding that every
Linux binary we had ever shipped requested an executable stack: an object
carrying no such note tells GNU ld nothing about its stack requirement, and ld
then assumes the worst for the entire link. That fix went into the artifact
only. scripts/extract_nomic_vectors.py still emitted the Mach-O branch alone,
so the next regeneration would have overwritten the tracked .S, dropped both
the ELF and COFF branches, and put GNU_STACK RWE back into every release --
silently, because nothing compared the generator against its own output.
write_blob_s() now emits the tracked wrapper verbatim, and the new contract
test fails if the two ever drift again. The template is compared rather than
executed so the test does not need the torch/transformers import the extraction
script pulls in. The write is pinned to UTF-8 because the WHY comment carries
an em dash and the default encoding is locale-dependent.
check-binary-composition.sh already fails a release whose binary has an
executable stack; this closes the same hole at edit time, where the diff is
still small enough to read.
Reported and originally fixed by Anand Aiyer in #1151, which also carried the
COFF branch. That PR sat unreviewed for a month while the .S half was
rediscovered and landed independently without credit.
Co-authored-by: Anand Aiyer <anand@aiyer.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The *.sh eol=lf rule landed via #1314, but the git hooks
(scripts/git-hooks/commit-msg, scripts/hooks/pre-commit) are extensionless
and were still at the mercy of core.autocrlf. Add their explicit entries,
plus the contract test from #1272 wired as scripts/test.sh Step 0t: every
shell entrypoint (*.sh + both hook directories) must carry an eol=lf
attribute, with a matched-zero-files guard so a broken glob can never pass
vacuously.
Distilled from #1272: the diagnosis (CRLF checkouts breaking shebangs under
WSL/MSYS), the hook-file coverage, and the guard design are @xumian520's;
verified RED without the *.sh rule (105 uncovered entrypoints) and green
with it (107 files).
Co-Authored-By: xumian520 <126989134+xumian520@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.10.6 release run failed deterministically at verify:
BLOCKED: expected scan object is missing:
objects/scan-3099e91c...--codebase-memory-mcp.exe
exclude-rescanned-selected-objects.sh (added after v0.10.5, first exercised
by this release) deliberately deletes the selected executables from the
surface-scan directory — their bytes were already scanned as candidates and
re-submitting identical bytes re-rolls a probabilistic classifier — and
writes binaries/virustotal-withheld.tsv. But check-virustotal.sh still
received the pre-withhold scan-set listing all sixteen objects and failed
closed on the first missing file. The rework's two halves never talked.
The gate now accepts an optional VT_WITHHELD manifest (strict parse: v1
marker, the stated reason required, sha256-keyed rows): an expected-set row
whose hash the manifest vouches for is exempt from the on-disk and
action-output contracts, while everything else keeps the strict path.
Fail-closed properties preserved and extended:
- no VT_WITHHELD -> byte-for-byte previous behavior (candidate
stage and dry-run call sites are unaffected;
verified against the original failure)
- withheld object present -> blocked (inconsistent staging)
- hash outside the set -> blocked (spurious withhold)
- everything withheld -> blocked (scan would cover nothing)
- mismatched object name -> blocked
vt-results.tsv keeps its exact shape (scanned objects only) — the release
notes table already uses the candidate results, and the withheld manifest is
now preserved with the rest of the evidence artifacts. release.yml passes
VT_WITHHELD only in the verify stage, right after the withhold step.
Verified offline with a fixture reproducing the release failure verbatim
plus the four negative cases above; the positive case passes staging and
association validation and proceeds to VT polling.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The verify pass submitted every extracted object, selected executables included,
on the stated grounds that "VirusTotal is content-addressed, so identical bytes
return the analysis it already holds instead of re-running 70+ engines".
That is measurably false. On v0.10.5 all EIGHT re-submissions produced a NEW
analysis - same VirusTotal file-id, timestamp 47 minutes later:
candidate: file-id=2c00f485... ts=1786795957 (12:12:37Z)
verify : file-id=2c00f485... ts=1786798758 (12:59:18Z)
Re-analysing identical bytes re-rolls a probabilistic classifier, and Microsoft's
ML engine answered differently within that hour, in BOTH directions:
82750cd1 (linux-amd64) microsoft-ml -> clean
6d3c5be6 (darwin-arm64) clean -> microsoft-ml
The published notes are generated from the candidate scan, so v0.10.5 shipped a
table calling linux-amd64 flagged when VirusTotal had it clean, and darwin-arm64
clean when VirusTotal was reporting Trojan:Script/Wacatac.B!ml. Every hash in
that table links to the page that contradicted it. Corrected in place after
publication; this removes the cause.
The second scan proved nothing the first did not. Identity is settled by hash
before this step runs: verify-release-selection.py reconciles every published
container to the selected bytes, and checksums.txt binds the same digests
publicly. A re-scan adds no assurance - only another roll.
What still gets scanned is exactly what the candidate pass never saw: install.sh,
install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the
unpacked UI assets. install.sh and install.ps1 are the highest-consequence
non-executable bytes we publish - users pipe them straight into a shell - and
that coverage is untouched. Measured on the v0.10.5 object set: 16 objects in,
8 withheld, 8 still scanned.
The withheld set is recorded as evidence (cbm-virustotal-withheld-v1) naming each
sha256 and pointing at virustotal-candidate-results.tsv, so the published
evidence still accounts for every shipped object.
Fails closed three ways, each with an actionable message: no object matches a
selected sha (the containers do not carry the recorded bytes), everything is
withheld (the surface scan would be a no-op), or the selection names no shas at
all. The zero-match grep is wrapped rather than left to pipefail, because a
guard that aborts silently is not a guard - found by testing the guards rather
than assuming them.
Also drops 8 VirusTotal submissions per release.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
audit-license-provenance.py ran `curl https://www.apache.org/licenses/LICENSE-2.0.txt`
every time the gate executed and byte-compared the result against
vendored/nomic/LICENSE, with capture_output and no error check. Any fetch
failure therefore produced an empty string, compared unequal, and reported:
vendored/nomic: DIFFERS [apache.org canonical LICENSE-2.0.txt]
PROVENANCE AUDIT FAILED: 1 unexplained verdict(s)
which is indistinguishable from a real licence discrepancy.
A gating verdict must be a pure function of the tree, not of whether a web
server answered. This one was neither reproducible nor attributable: it
reddened `security / license-gate` on PR #1337 - a branch touching cli.h,
hook_augment.c and test_cli.c, and no licence at all - and left that
contributor blocked for over two weeks on a signal that had nothing to do with
their change.
Verified before pinning: our vendored copy is byte-identical to the upstream
canonical text, 11358 bytes on both sides, diff clean. The Apache-2.0 text is
immutable and versioned, so a digest is the honest way to express "this is that
text". A mismatch now means our vendored copy changed, which is exactly - and
only - what this audit exists to detect.
The audit passes locally with no network access on the nomic entry.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel