The two layer-5 interface probes tested for a substring by piping the
captured output into `grep -q`. Under `set -o pipefail` (line 32) that
turns a SATISFIED contract into a probe failure whenever the payload is
larger than the pipe buffer: `grep -q` exits at the first match, and the
writer takes EPIPE on the remainder, so the pipeline reports the writer's
non-zero status even though the match succeeded.
Observed on macos-15-intel:
test_venue_parity_contract.sh: line 404: printf: write error: Broken pipe
INTERFACE CONTRACT: scripts/ci/smoke-artifact.sh --help must exit 0 and
print a Usage: block (rc=0)
VENUE PARITY CONTRACT VIOLATED — interface probes failed (layer 5)
rc=0 and the Usage: block was present — the match is exactly what made
grep exit early. `smoke-artifact.sh --help` is 698 bytes against a
512-byte PIPE_BUF, and prints `Usage:` on its first line, so the writer
is killed at the earliest possible moment. Whether the remaining write
lands before grep exits is a scheduling race, which is why this only
ever showed up on one runner.
Replace both pipelines with in-shell `case` matching: no subprocess, no
pipe, no race, and a strictly literal match (the second probe's "--help."
was previously a regex whose `.` matched any character).
Verified: the contract passes (18 --help entries, 8 strict-flag entries),
and the matcher still returns 0 for output with no Usage: block, so the
probe can still fail.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #1412 by @allanoepping. Fixes a real recall hole: tag-dialect
.cfc components were routed to the cfscript grammar (extracting nothing),
and <cfscript> bodies inside tag-dialect files are opaque cf_script_content
to the cfml grammar — whole legacy components produced zero functions.
Fix 1 — his .cfc dialect sniffer, nearly verbatim: cbm_disambiguate_cfc()
reads a 16K head via cbm_fopen; <cfcomponent>/<cffunction> anywhere means
tag dialect; a leading <cfscript (after whitespace and <!--- ---> comments)
or no markup at all keeps the old CFSCRIPT routing, so script-dialect files
and read failures behave exactly as before. Wired in detect_file_language
beside the existing _m/_cls/_inc sniffers, with his 8 direct tests.
Fix 2 — his embedded-defs capability, re-based onto the shared machinery
that superseded his mechanism mid-flight (#1852, merged after his last
rebase): a cfml_embedded_imports spec row (cf_script_tag/cf_script_content
-> CFSCRIPT), CFML added to the embedded-imports dispatch, and the
extract_structure gate extended to CFML. Included ranges on the original
buffer give absolute coordinates for free, so his line-number assertions
hold verbatim, and calls inside blocks become extractable as a bonus. His
~90-line slice+shift implementation is deliberately NOT carried — it
mirrored the sanctioned shape of its day, which #1852 replaced.
One correctness guard the port needed: the Vue attribute resolver is now
gated to Vue — cf_script_tag carries no lang=/src= attributes, and running
the resolver would have silently rewritten CFML's embedded language from
CFSCRIPT to the resolver's JavaScript default.
His embedded extraction test (line remap + <cfsetting> ERROR-cascade case)
carries over verbatim and binds: reverting the spec/gate wiring fails it at
ASSERT(has_def(r, "Function", "greet")). Suites: extraction, language,
edge_imports, lang_contract, grammar_labels, grammar_regression — 637/0.
Co-authored-by: Allan Oepping <allan@intellipay.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer commit on top of the Bash-matcher work, closing the cost
question the 2026-08-03 review left open before default-on shipped.
Measured on a production binary (2026-08-28): every hook invocation paid
~1.1-1.2 s of user CPU BEFORE any gating — not in cbm_mcp_server_new as
first suspected, but in main()'s executable-identity hash for cohort
admission, which the hook-client path runs ahead of any input inspection.
With Bash in the installed matcher, nearly every agent command (`ls`,
`make`, `git`, ...) would have paid that for nothing: agents issue far
more Bash calls than Grep/Glob, and almost none are searches.
The fix: main() now reads stdin on the hook-client path and exits before
identity hashing when the event can never produce output — an un-forced
PreToolUse Bash event whose command yields no search pattern or no
queryable token. A no-op touches no shared state, so it needs no
identity. The gate is a pure exported function using the exact parser
pair ha_process uses, so gate and augmenter cannot disagree; anything
uncertain (lifecycle events, other tools, coverage adapters, oversized
stdin, forced events) pays full fare unchanged. Consumed stdin is handed
back through a prefetch so downstream readers see it exactly once.
Measured after: no-op Bash calls ~10 ms wall / 0.00 s user CPU (was
~1.2 s); real Bash searches and Grep/Glob unchanged at full fare.
cli 259/0, mcp 179/0.
Co-authored-by: Jake Abendroth <contact@jakea.net>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #1298 by @aoright — the subset that survived review and
main's drift, per the 2026-08-17 ruling.
- pass_lsp_cross.h: the header still documented and declared
all_defs/imp_keys/imp_vals while the implementation renamed them to
defs/imp_names/imp_qns — doc-only drift, now synced; plus the braced
guard in cbm_pxc_registry_for_lang.
- runtime.h: semantic_version[12] gains a named size
(CBM_DAEMON_SEMVER_SIZE), the "defensible: a real size" case from the
review.
- httpd.h: request-field buffers take the CBM_SZ_* names for their actual
sizes; the constants.h include sits in the top include block rather than
mid-file.
Deliberately NOT carried, per the same ruling: the store.c calloc hunk
(the claimed uninitialized read does not exist), the generic-constant
renames (semantic laundering), the watcher.c split (its target function
gained a semantic fix on 2026-08-20 and the refactor must be re-derived),
and the lsp_resolve.h extractions (their target loop was rebuilt on main).
Co-authored-by: aoright <102943475+aoright@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #1255 by @MasterFede5, rebased onto the single-binary
installer and completed with the two review conditions.
PowerShell 5.1 wraps redirected native stderr into ErrorRecords, so under
the script's global ErrorActionPreference=Stop a HEALTHY binary that
prints a single warning to stderr while exiting 0 aborts both
`--version 2>&1` probes with "downloaded binary failed to run". Both
probes now relax EAP to Continue for the probe call only, with an
exception-safe restore; failure detection stays on $LASTEXITCODE.
The review-required pre-seed ($global:LASTEXITCODE = 1) closes the window
the relaxation would otherwise open: a binary that fails to START under
Continue may leave $LASTEXITCODE untouched, and the installed-binary probe
runs after earlier successful native calls, so a stale 0 would have turned
a corrupt install into "Installed: ...". Seeding 1 makes start-failure
read as failure on both probes.
The bundle contract pins the guard's two load-bearing parts (the scoped
relaxation with restore, and the pre-seed), verified failing when either
is stripped. The change adds zero non-ASCII bytes; the file remains pure
ASCII per the PS 5.1 ANSI-decode rule.
Co-authored-by: MasterFede5 <friedrickms@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer follow-up on top of the contributor's rebased commit,
implementing the Half-B strategy decision from the 2026-08-21 review:
- Linux: when the self-resolved path fails the executable check after an
atomic rename-over, hand back the /proc/self/exe magic link. The link
still executes the in-memory OLD build, which is the only spawn the
worker's build-fingerprint gate accepts. Deliberately NOT the saved
launch path: after a replacement that path holds a DIFFERENT build, so
preferring it swaps an ENOENT for a fingerprint refusal.
- macOS: no magic link exists; fail closed. The supervisor logs
index.supervisor.no_self_path and indexing resumes on daemon restart
instead of spawning a missing binary. A rename-over that leaves a VALID
binary still resolves (and the fingerprint gate refuses downstream, with
its own coverage).
- The prefer-saved-launch-path branch from the original head is dropped
per the same ruling.
Also fixes both Half-A review items:
- Aliasing in cbm_http_server_set_binary_path: `path` may alias
g_binary_path on a repeated call. The overlapping snprintf cannot
actually fire today — resolve's leading out[0]='\0' zeroes the shared
buffer first — but that same reset means the call silently DISCARDS the
stored path it was asked to re-resolve, and the in-place write is one
reset-reorder away from real UB. Resolve into a local first: correct
input, no aliasing trap.
- Coverage asymmetry: the probe and driver ran only on Linux while the
production edit covered all of POSIX. Both now run on macOS too, and the
Linux probe first proves the "(deleted)" state via readlink so its
assertions cannot pass vacuously on an intact image.
Revert-check: with main's http_server.c restored, the driver fails at
WEXITSTATUS 44 (resolver returned a dead path) on macOS; with the guard it
passes 63/63.
Co-authored-by: sunmingqiang <sunmingqiang@bytedance.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Prefer the saved executable launch path when it still points to an
executable, and reject a deleted /proc/self/exe path before spawning
index workers.
Add a Linux integration regression that forks, atomically replaces the
running test binary on disk, and verifies that subprocess resolution
uses the saved launch path instead of the now-deleted self path.
Signed-off-by: David Musk <davidmusk93@users.noreply.github.com>
Signed-off-by: sunmingqiang <sunmingqiang@bytedance.com>
The language-count contract shipped this morning in #1859 had a hole: it
matched only the prose form "<N> languages", so README's shields badge
("languages-158-orange") never matched and the gate reported success while
the badge sat three behind the prose in the same file.
The cause is worth recording. The surfaces were enumerated with the same
regex the gate then used, so the verification could only ever confirm what
the enumeration had already seen. A gate and its own completeness check must
not share a pattern.
Both claim forms are now matched, in the per-surface check and in the
unregistered-file sweep, so a new file stating a count in either form still
fails. README's badge is corrected to 161.
Also corrects a stale range comment left by the PL/SQL merge: matrix B's
header said "CBMLanguage 80..163" when PL/SQL had made it 164. That is the
comment variant of the additive-counter trap — the numeric constants were
audited across that rebase, the prose describing them was not. The two
sibling range descriptors (GO..ARKTS = 68 rows, RACKET..PLSQL = 47 rows)
were checked and are correct.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Seven surfaces disagreed with reality and with each other: README.md,
docs/index.html, docs/llms.txt and pkg/npm/README.md said 158, while
server.json, the chocolatey nuspec and scripts/package-release.sh said 159.
161 grammars are vendored.
21 individual claims corrected. The count is now gated by
tests/test_language_count_contract.sh, so the next language to land cannot
ship a stale figure.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The count was hand-maintained on seven separate surfaces and had drifted apart
from reality AND from itself: README, docs/index.html, docs/llms.txt and the npm
README said 158, while server.json, the chocolatey nuspec and package-release.sh
said 159.
Derived from the vendored grammar directories rather than CBM_LANG_COUNT,
because the registry contains entries with no parser at all: CBM_LANG_NIM
(grammar removed 2026-06-12) and CBM_LANG_OBJECTSCRIPT_EXPORT have no grammar
directory, no tree_sitter function and no MANIFEST row, yet files still route to
them by extension. Publishing CBM_LANG_COUNT would claim languages we do not
parse. A vendored grammar directory demonstrably exists and is checksum-gated.
Accepted trade-off, stated in the test: this UNDERCOUNTS languages that share
one grammar (TypeScript/TSX, objectscript_udl/_routine). We publish the number
we can prove rather than the larger number we cannot.
The contract binds four things so it cannot go quietly vacuous:
- the grammar directory count is sanity-bounded, so a layout change fails loudly
instead of making every check below it trivially true
- MANIFEST.md's declared figure must already agree with the directory count
- every registered surface must state exactly that number, and a registered
surface that states NO number fails (the wording changed, or it should be
deregistered)
- any UNREGISTERED file stating a language count fails, so an eighth surface
cannot reintroduce the drift this exists to prevent
Files that legitimately say "<N> languages" about something else — benchmark
corpus subsets, probe scopes, the version-pinned winget manifests — are listed
with what their number actually counts, so no exemption is unexplained.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer edit on top of the rebase. The pin read 0.9.0, which was already
stale when the PR was opened and is two releases behind now. Kept as its own
commit so the contributor's change stays exactly what he wrote.
Recorded rather than fixed here: this pin is hand-maintained and will rot
again on the next release — the same failure mode as the published language
count. Deriving it from one source belongs with that work, not in a Nix PR.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fixes#1088
The flake only built the standard `cbm` target, so `nix run`/`nix shell`
produced a binary without the embedded UI — `--ui=true` refused to start
the HTTP server. Add two flake outputs:
- `graph-ui`: the React frontend built via buildNpmPackage (offline, from
the pinned package-lock.json; npmDepsHash tracks the lockfile)
- `codebase-memory-mcp-ui`: the server with those assets embedded, built
by dropping the prebuilt dist/ into place and running the embed + link
path inside the sandbox (no network access)
Also fold in adjacent Nix cleanups: bump default.version 0.6.0 -> 0.9.0
to match the release, refresh flake.lock (nixpkgs 2026-04 -> 2026-07),
ignore ./result build symlinks, and document both packages in the README.
Signed-off-by: Pim Snel <post@pimsnel.com>
Assert positively that the fixture's tree contains ERROR/MISSING nodes
(parse_incomplete), so a future grammar upgrade that fixes AS OBJECT
turns this RED and prompts claiming the Class def instead of pinning its
absence vacuously.
Co-authored-by: Oğuz <oguzhan.saricam@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Vendor AndreasMaierDe/tree-sitter-plsql @ 28aebef209be (MIT, ABI 14, no
external scanner) and wire CBM_LANG_PLSQL end to end:
- extraction onto existing labels only: package spec/body -> Class,
procedure/function (create/definition/declaration) -> Function,
file -> Module; callee extraction for ref_call incl. the
package-qualified pkg.name form; identifier usage vocabulary with the
upstream 'parameter' kind (a ref_call ARGUMENT wrapper) carved out of
the whole-binding rule
- discover: .pks/.pkb/.pck/.pls/.plb/.plsql/.fnc/.trg/.bdy/.tps/.tpb
(.sql stays SQL and .prc stays FORM, both asserted negatively)
- ledgers: CALL_WITHOUT_REFERENCE_VOCAB(PLSQL), call-node manifest
ref_call row (historical total 219 -> 220), call-argument matrix B row
- tests: extraction triple (package body + package-qualified call,
standalone function, honest negative pin of the upstream
CREATE TYPE ... AS OBJECT ERROR-node limitation), grammar-regression
case, label golden Class:1,Function:1,Module:1, MUST_EXTRACT_DEFS,
extension mapping, bug-repro board battery row
- MANIFEST.md (community-niche verdict, local include-quote patch row),
THIRD_PARTY.md section, refreshed vendored checksums
PL/SQL support was contributed by Oguz (@ouzsrcm) in #1033; re-vendored
from upstream per vendoring policy with his language wiring distilled on
top.
Co-authored-by: Oğuz <oguzhan.saricam@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Maintainer fixups pushed to the PR branch, each with a test that fails
without it.
- The TargetConfigNames/TargetConfigName path emitted nothing. Route nodes
are keyed __route__ensemble__<prod>.<item> (as emit_async_call and
emit_handles both do), but the settings loop looked the source item up by
its bare <prod>.<item> qn. cbm_gbuf_find_by_qn is an exact hashtable
lookup (graph_buffer.c:769), so item_node was always NULL and every
declarative routing edge was dropped by `continue`. An incomplete rename
from the EnsembleItem version. Proof: the new settings test reports
async_calls == 0 against the previous code.
- Heap-buffer-overflow on malformed XML. With an <Item that has no closing
</Item>, item_end became the buffer's NUL terminator and `p = item_end + 7`
then pointed 7 bytes past the allocation, which the loop dereferenced
before strstr scanned unbounded heap. Now the scan stops at an
unterminated item. Proof: ASan reports heap-buffer-overflow at
pass_ensemble_routing.c:544 against the previous code.
- extract_xml_attr scanned forward from the tag with no bound, so a tag
omitting an attribute silently inherited it from a LATER tag, mispairing
ClassName/Enabled across items. The search is now bounded at the tag's
closing '>'. Proof: an item with no ClassName produced a second, bogus
Route node against the previous code.
- read_file ignored the fread result, leaving an uninitialised tail on a
short read for the string scans to walk. Terminate at the byte count
actually read.
Three tests added: a production whose ONLY routing information is a
TargetConfigNames setting (so any ASYNC_CALLS edge must come from that
path), an unterminated <Item> in an otherwise well-formed file, and an item
missing ClassName ahead of a well-formed one.
Full local suite: 7576 passed, 0 failed, 8 skipped (139 suites).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Grok Build (xAI's grok CLI/TUI) already reached codebase-memory-mcp through
its Claude/Cursor compat layer, but a Grok-only user got nothing. Detect
$GROK_HOME (default ~/.grok) or a grok CLI and install:
- an owned [mcp_servers.codebase-memory-mcp] managed block in config.toml;
a same-name foreign table is left byte-identical because a second header
would make Grok reject the whole file, while the known owned shape written
by `grok mcp add` is adopted. No env forwarding: grok 1.0.5 passes the
full parent environment to stdio servers (verified), unlike Codex (#1562).
- an owned rules file under $GROK_HOME/rules/ (always scanned, all projects)
- the shared skill under $GROK_HOME/skills/
- Scout/Verify/Auditor agents with named-server mcpInheritance and the tier
allowlist spelled out as the server__tool ids Grok's search_tool/use_tool
dispatcher accepts (inheritance filters per server, never per tool)
- hooks withheld: Grok's passive events discard stdout and PreToolUse honors
only deny/rewrite decisions, so there is no context-output contract.
Verified against grok 1.0.5: inspect discovers the server, agents, skill and
rules; mcp doctor handshakes and discovers 15 tools. --clients=grok token,
README/npm/llms/landing-page matrices (43 -> 44 surfaces), and smoke coverage.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Vue single-file components yielded a Module node and IMPORTS edges but no
Function definitions or CALLS edges from the `<script>` block, so a component's
call graph was empty (#1410).
Adds a control test pinning current behaviour for Svelte, HTML and Astro, which
still extract imports but no functions or calls. That is deliberate scoping,
not an oversight: it documents the remaining gap rather than hiding it, and it
proves this fix does not leak into sibling languages. Svelte is tracked by
#1807 and will be a follow-up.
Re-opened from #1843, which carried no Signed-off-by and so failed DCO. The
tree is byte-identical to that commit; only the sign-off was added.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Layer 8 (scripts/security-vendored.sh) failed because the seven new arkts
files were untracked in the checksum baseline. Regenerated with --update
after reviewing the diff: the only additions are the arkts grammar's own
files, plus MANIFEST.md's hash changing because the preceding commit edited
it. Nothing else was re-blessed.
Verified independently rather than taken on trust:
arkts/_common_scanner.h sha256 == typescript/_common_scanner.h sha256
(da66ef2bd14a...), i.e. the external scanner body is the already-reviewed
typescript one, and arkts/scanner.c is a 21-line symbol-rename trampoline
that allocates nothing and delegates scan straight through.
All three security-static layers now pass locally: static allow-list, UI
audit, and vendored integrity -- the last also reporting no subprocess,
network, or dangerous calls anywhere in vendored grammar code.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The vendored LICENSE carried tree-sitter-typescript's MIT text plus an
appended derivative notice. That is not what MIT asks of a derivative -- it
asks that upstream's notice travel with the code -- and the appended text
meant the provenance audit could not byte-verify the file against upstream,
so arkts landed as NO-MANIFEST-ENTRY and failed license-gate.
The vendored LICENSE is now byte-identical to upstream's, and arkts is
registered in the audit's FORKS map, which is defined for exactly this case:
a self-maintained fork whose vendored LICENSE must match its upstream. That
is a routing entry, not an exemption -- check_upstream still byte-verifies
against the real upstream repository, so the fact is checked rather than
asserted. Audit now reports arkts as IDENTICAL.
Our copyright for the ArkTS additions is not lost, it moves to where our
SOURCE is: the grammar.js header (which now states all three copyrights
explicitly) and THIRD_PARTY.md, which ships inside the release archives.
The edit to scripts/audit-license-provenance.py is a maintainer-only
enforcement-surface change, made deliberately and stated here rather than
slipped in; a contribution may never supply its own audit entry.
No generated parser or source changes -- comments, licence text, manifest
prose and one routing entry only.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The fixture returned "/etc/myapp/conf.d", which is_filesystem_path() rejects
before is_rest_path() is consulted, so the value was never classified as a URL
and never recorded as a builder -- with or without the language gate. The test
passed identically with the gate removed, so it certified nothing.
"/srv/" is not one of the ten filesystem prefixes, so the value is URL-shaped
and only the JS/TS/TSX gate prevents it being recorded for C.
Verified by revert-check: with the gate removed the test now FAILS at
tests/test_extraction.c:3455; with the gate restored it passes.
Follow-up to #1010. The gate implementation is unchanged and is the author's.
Co-authored-by: Charles Queiroz <fcqueiroz@liquibase.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Routing .ets through the plain TypeScript grammar produced no definition
node for any @Component struct (0/19 across the OpenHarmony sample corpus),
parsed struct members as statement labels and build() as a call expression,
and registered ArkUI builtins (Column, Row, ListItem, ...) as user function
definitions with 28% fabricated cross-file CALLS edges. This adds a
dedicated ArkTS grammar that keeps everything TypeScript already handles
and models the ArkTS syntax TS cannot parse.
Grammar (tools/tree-sitter-arkts/, generated parser vendored to
internal/cbm/vendored/grammars/arkts/, ABI 15 via tree-sitter-cli 0.25.10):
first-party derivative of tree-sitter/tree-sitter-typescript v0.23.2
@ 75b3874edb2d (the commit our vendored typescript/tsx come from) on the
tree-sitter-javascript v0.23.1 base @ 3a837b6f3658 (the version upstream's
package-lock pins). LICENSE carries the upstream MIT text verbatim plus a
derivative notice; the external scanner is a symbol-rename trampoline over
a byte-identical copy of the already-vendored typescript scanner header.
ArkTS additions: decorated struct component declarations with class-shaped
bodies; UI-DSL trailing-closure component calls with attribute chains after
the closing brace; decorators on function declarations (@Builder,
@Extend(T), @Styles); @Extend/@Styles leading-dot attribute-chain bodies;
anonymous stateStyles style blocks; import lazy named imports.
Wiring: CBM_LANG_ARKTS enum + lang spec (struct_declaration as a class
kind labeled Struct, public_field_definition fields), .ets extension,
LANG_NAMES/userconfig entries, grammar shim, and TS-family joins across
the extractors and the name-resolution guards (parse_es_imports, JS
keywords/test-file/func-kind/scope tables, TS heritage/param/receiver/
namespace/type-ref/channel/usage paths, js_ts_family and the weak-method-
match suppression). Field definitions now also carry their decorators, so
@State/@Prop/@Link members are findable with their roles.
Tests: grammar corpus (35 cases), per-language regression + label golden +
import net + extension cases, seven ArkTS extraction tests including the
phantom-builtin regression (ArkUI builtins must never be definitions), and
capability-ledger / call-node-manifest / call-argument-matrix rows. On the
14-file OpenHarmony sample corpus the TS-grammar baseline had 12/14 files
with parse errors (231 ERROR nodes); the arkts grammar parses 14/14 clean,
and an end-to-end index yields all 19 components as Struct nodes, 68
methods, 119 fields, and zero phantom builtin definitions.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel