Externalizing the integration templates (#1492/#1493) and the UI bundle
(#1501/#1503) was done to reduce the Microsoft `Wacatac.B!ml` surface. It did
not work: across dry runs the flagged artifact count stayed at ~3 and the
detections merely moved between artifacts.
Dissection of run 31286803592 shows there is no structural cause to fix. The
verdicts split across every axis at once — linux-amd64 (dynamic) flagged while
linux-amd64-portable (static) is clean, but linux-arm64 (dynamic) clean while
linux-arm64-portable (static) is flagged. The two macOS binaries have identical
segment structure and split clean/flagged. Siblings from one build landed in
different variant buckets (.B vs .C). Entropy is low everywhere
(code_vectors.bin 4.166, grammar tables 3.464 bits/byte, against 7.5-8.0 for
packed payloads), so the packed-payload hypothesis is excluded too.
So the complexity bought nothing, and installation goes back to being
self-contained: one binary that carries its own UI and agent integration
templates, with no adjacent data file that has to resolve before `install`
works. Only the UI-capable composition ships from now on, under the historical
unsuffixed archive name.
Removed: src/ui/asset_pack.{c,h}, asset_pack_stub.c, asset_manifest_stub.c,
scripts/pack-ui-assets.mjs, src/cli/integration_assets.{c,h},
assets/cbm-integrations.json, scripts/gen-integrations-hash.sh, the
--verify-runtime-assets probe (nothing adjacent left to verify), and the
composition gates A6/A7 whose property is now deliberately inverted.
Restored: scripts/embed-frontend.sh, src/ui/embedded_{assets.h,stub.c}, the
compiled-in hook/adapter template bodies, and the embed/EMBED_OBJS build path.
Kept from the reverted commits, re-applied by hand where a wholesale file
restore would have dropped them:
- cbm_module_path_utf8() in both self-path sites. GetModuleFileNameA renders
through the ANSI code page and mangles non-ASCII install paths.
- the /__cbm/ui-readiness HMAC proof, secure_random and cbm_hmac_sha256, so
`daemon start --open` still waits for a genuine CBM listener.
- X-Content-Type-Options: nosniff on served assets.
- the MinGW noexecstack gate, -lbcrypt, and the cppcheck/zip CI fixes.
Archives are now codebase-memory-mcp-<os>-<arch>[-portable] with exactly four
members (binary, LICENSE, installer, THIRD_PARTY_NOTICES.md). That restores the
names every static package manifest already points at — aur, chocolatey,
homebrew, scoop, winget and glama were all broken by the -ui- rename.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The product binary embedded nine complete shebang'd shell scripts, their
PowerShell/.cmd twins, and two node:child_process client-adapter modules as C
string literals, and wrote them into agent config dirs at 0755 on install. A
block of script-shaped bytes inside a large unsigned executable is exactly the
surface Microsoft Defender's ML scored Trojan:Script/Wacatac.B!ml — the
detection that has recurred across the last several release attempts, on the
Linux ELF and darwin-arm64 standard binaries while every UI and Windows variant
stayed clean.
This moves every integration template out of the binary into a single shipped
data file, assets/cbm-integrations.json (compact: one asset, because each
shipped file is itself scanned). The binary now carries only the templates'
identity — one embedded SHA-256, generated at build time from the JSON so it
can never drift — plus the code to load, verify, and render them.
Why a hash and not just a loose file: content compiled into the binary was
tamper-evident with the binary. Moving it to disk must not turn it into
unauthenticated code the installer blindly executes. install verifies the asset
against the embedded SHA-256 and fails closed on mismatch ("integration assets
missing or modified - reinstall from the release archive") before writing
anything. Verified end to end: a one-byte edit to the asset makes install
refuse.
Ownership, for safe upgrade/removal: the verified copy is published to
~/.cbm/assets/<version>/ — a sibling of the DB cache, never inside it, so
clearing the cache cannot strip a user's hooks. A deployed integration file is
ours to remove only if it matches a template rendered from that stored copy;
foreign files and user-edited files at the reserved paths are preserved. The
historical template bodies travel in the JSON's released[] arrays so files
written by older versions are still recognised and cleaned up on uninstall.
Removed from src/: cmm_gate/session/subagent script-prefix and suffix
constants, cbm_build_released_gate_script, the shell/PowerShell/.cmd bodies in
cli.c, and the JS/TS generators in client_adapter.c. Confirmed on the built
standard binary: 0 occurrences of '#!/usr/bin/env bash', '#!/bin/sh',
'node:child_process', and 'ExecutionPolicy' (was non-zero for each).
test_no_embedded_scripts_contract.sh makes the removal a build property, not a
reachability assumption: no production C source may contain a shebang or
node:child_process literal, and the built binary is scanned directly. It strips
comments first (a contract a comment can satisfy is a false guard), and is
revert-checked — reintroducing one shebang literal makes it fail naming the
file:line. check-binary-composition.sh (A6) enforces the same needles on every
release artifact post-strip.
This does not, by itself, prove the detection is cured: the clean UI binary is a
strict superset of the flagged standard one, so no single passage is a
deterministic trigger. It removes the most coherent malware-shaped composite
from every shipped binary, aligns with how gh/starship/zoxide externalise
integration material, and makes the templates maintainable as data rather than
escaped C literals. Causal confirmation still needs a VirusTotal ablation.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
`cbm update` printed the Windows update command with an execution-policy
override baked in as a string literal. That put one of the most heavily
weighted malicious-script patterns there is inside every Windows artifact we
distribute — to save the user a single documented step.
The command now prints without it. Functionality is unchanged: the user still
gets a runnable command, and the failure it was pre-empting (PowerShell refusing
a file that arrived from the internet) is answered by Unblock-File, which the
message now names. The restricted-policy case is documented in the README's
Windows install section, which is the right home for that incantation — a
shipped binary is not.
Scope note, so this is not oversold: the 2026-08-07 dry-run detections are NOT
attributable to this string. It is `#ifdef _WIN32`, and every Windows artifact
scanned clean; the three flagged artifacts were Linux and macOS. This is margin
hygiene, not a fix for that regression — see the analysis in the PR.
Verification limit worth stating: the built-artifact check for this string is
vacuous on a non-Windows host, since the literal was Windows-only to begin with.
What is verified here is the source (no occurrence outside a comment) plus lint
and the cli suite; a real artifact check needs the Windows build leg in CI.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every non-JSON tool result shipped its payload twice: content[0].text, and an
identical copy as structuredContent {"text": <entire payload>}. Measured on a
20k-node query_graph, the reply was 2.05x the payload it carried. Half of every
large answer was redundant bytes — half the 10 MiB transport budget, and double
the tokens billed to every LLM caller on every call.
Measured, same query, same fixture:
before 8,529,990 bytes content.text 4,154,932 structuredContent.text SAME
after 4,265,047 bytes content.text 4,154,932 structuredContent {}
Exactly 50.0% smaller with content.text byte-identical: the payload is fully
delivered, only the second copy is gone. The 10 MiB ceiling now also admits
roughly twice the rows before #1375's limit applies.
Nothing is lost. structuredContent exists to carry STRUCTURE, and a string
re-wrapped in a one-key object has none — a client reading
structuredContent.text learned exactly what content[0].text already told it. The
empty object still satisfies the declared outputSchema, which is
{"type":"object","additionalProperties":true} and never required a text field.
Two cases are deliberately NOT changed:
* JSON payloads. structuredContent stays the PARSED object. That is the
spec's structured+serialized pattern rather than waste, and it is what our
own hook-augment consumes (structuredContent.projects from list_projects).
* Errors. structuredContent.error is kept: bounded, small, and the only
machine-readable form of a failure a client gets.
Guarded at both levels, because the defect was invisible per-tool — each result
looked reasonable alone, and only measuring the wire showed half of it was
redundant:
* tests/test_mcp.c enumerates the TOOL TABLE itself, so a new tool is covered
the moment it is registered, with no test edit. A guard pinned to
query_graph would not have caught search_graph, and would not catch whatever
is added next. It fails if no tool produced a non-JSON payload, so it cannot
report a green it never earned — which it did on the first attempt, catching
that {} args make every tool error out and assert nothing.
* scripts/smoke-test.sh Phase 3z asserts the same property on the SHIPPED
binary. This is a wire-format contract: what a real client receives from the
real artifact is not something a from-source test can prove.
Both revert-checked: restoring the duplication reddens the unit guard at the
strcmp, the plain-text guard, the search_graph expectation, and the smoke phase
(which names search_graph and get_architecture).
tests/test_mcp.c expectations that pinned the old duplicating shape are updated
rather than relaxed — the format changed on purpose.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A single query_graph reply larger than one 10 MiB frame killed the MCP server:
exit code 1, no JSON-RPC error, nothing on stderr, no crash report. Under a host
that does not respawn the server, every tool from it was gone for the rest of
the session — the reporter lost their tooling mid-investigation to one dead-code
query and finished by driving the binary over stdio by hand.
Root cause is a conflation, not the limit itself. application_mcp_request
checked only response_length > UINT32_MAX and passed anything smaller on, so an
oversized reply reached the transport, which rejected the frame. The frontend
worker cannot tell that rejection from a dead socket, and for a dead socket it
deliberately _Exit()s the process — closing the kernel IPC handle is the only
portable way to cancel daemon session ownership from a thread blocked in stdio.
Correct for a broken transport; a fatal over-reaction to a large answer.
The substitution therefore has to happen in application_mcp_request, which is
the last layer still holding the request id. Replies that fit are returned
byte-identical; a reply that cannot be framed becomes a JSON-RPC -32603 naming
both numbers and what to do ("narrow the projection, add LIMIT, or paginate"),
because the failure it replaces was a silent exit and an opaque "internal error"
would be no improvement.
Reproduced first, on a locally generated 20k-node fixture rather than the
reporter's repo (external code stays unverified until reviewed):
before LIMIT 10000 -> ok, 8,529,990 bytes
LIMIT 20000 -> SERVER DIED exit=1 stderr=''
after LIMIT 20000 -> JSON-RPC -32603, and a follow-up query on the SAME
session succeeds — which is the property that actually matters
The decision and the substitution live in one function on purpose. The first
version of the test built the error object directly and PASSED with the size
check removed: it proved the error was well-formed while the server still died.
Testing application_framable_response instead binds the branch — verified by
removing the guard, which reddens it at ASSERT(replaced != big).
Not addressed here, deliberately: every reply still carries its payload twice
(content[0].text and structuredContent.text are verbatim duplicates, ~2.05x
measured), which is why the ceiling is reached at ~4 MB of real content. That is
a wire-format question affecting every tool and every caller's token bill, so it
belongs in its own change with a decision behind it, not smuggled into a crash
fix.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1448 set CREATE_NO_WINDOW on the subprocess spawn and on cbm_popen_isolated,
but cbm_exec_no_shell (compat_fs.c) still passed a bare 0 for dwCreationFlags.
That is the helper behind git, codesign and open, so the most frequently hit
path kept flashing a console — and under a stdio MCP session with auto_watch
those windows steal focus while the user is typing. Reported by @noctrex on
#1448 after verifying the merged fix.
Also adds the contract nobody had. The flag has to be set per call site, the
four sites do not resemble each other, and reviewing "does this PR add the
flag" says nothing about the sites a PR does not touch — which is exactly how
the third one survived a fix applied twice:
daemon/bootstrap.c had it from the start (DETACHED_PROCESS | ... | flag)
subprocess.c added by #1448 (flags variable)
compat_fs.c :313 added by #1448 (inline literal)
compat_fs.c :689 missed by #1448 (bare 0)
tests/test_spawn_no_window_contract.sh asserts the whole-tree property instead,
so a fifth spawn site cannot be added without it.
The contract strips C comments before matching, and that is load-bearing rather
than tidiness: the first version of it PASSED with the fix reverted, because the
explanatory comment above the fixed call contains the flag name and satisfied
the match. A contract that can be satisfied by prose about the contract is a
false guard. Verified in both directions afterwards — green with the fix, and
red naming compat_fs.c:696 with the fix reverted and that comment still in
place.
Co-Authored-By: noctrex <noctrex@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_resolve_cache_dir() returns a pointer to a static thread-local buffer. The
cache prune called it for every directory the walk visited, so it rewrote a buffer
other code was entitled to still be holding — an aliasing bug, not a cosmetic
inefficiency.
The Windows guards job caught it as two REDs: a daemon cold-storm regression and
a following "index did not run" setup failure. Both are consistent with the cache
path being rewritten underneath a live reader during indexing.
Resolve once at the discovery entry point into a thread-local copy and compare
against that, so the prune costs one resolver call per walk rather than one per
directory, and holds no pointer into shared storage.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The comment above MIMALLOC_OVERRIDE_DEFINE claimed Unix "relies on
static-link-order override". That was never true. mimalloc emits strong
malloc/free definitions only when MI_MALLOC_OVERRIDE is set — the define its
source gates alloc-override.c on — and the build set it for MinGW only.
MI_OVERRIDE, which the build does set everywhere, is this project's own
prod/test marker that mimalloc never reads. With the override body compiled
out there were no strong symbols for link order to prefer, so ordinary malloc
went to libc.
Measured, not inferred. The shipped v0.9.1-rc.1 artifacts report 0/6
allocator-owned size classes on linux-arm64 glibc AND musl-static, and the
same A/B on ubuntu-arm64 here:
main warn mem.allocator.not_owned owned_classes=0/6
with this fix info mem.allocator.owned classes=all
Every purge/reclaim option cbm_mem_init sets was therefore inert on Linux,
applying only to the bound sqlite/tree-sitter populations — the same class of
defect as #581, where committed memory ratcheted for months because nothing
asserted the wiring on a real artifact.
macOS stays off deliberately and permanently: enabling the override there
compiles alloc-override.c's forwarding definitions, and under the two-level
namespace this binary's free becomes mi_free while system libraries keep
allocating from the system allocator, so the first pointer crossing that
boundary aborts with "mi_free: invalid pointer". ELF's flat namespace has no
such split, which is why Linux can have this and macOS cannot.
Three parts:
* Makefile.cbm switches MI_MALLOC_OVERRIDE and a new CBM_MEM_GLOBAL_OVERRIDE
together in one place, for MinGW and Linux. The latter tells our own sources
what to expect and goes into CFLAGS_PROD only, because MIMALLOC_CFLAGS_TEST
builds mimalloc with -DMI_OVERRIDE=0 and no override define: a test binary
has no global override BY CONSTRUCTION and must never be told to expect one.
Deriving that expectation from the platform instead would make every Linux
test run warn about a correctly configured build.
* mem.c splits the startup audit by what the build actually asked for. Where
no override was requested, ordinary malloc reaching libc is the design, so
it reports the measured ownership at INFO and names what IS bound. A warning
that fires on every run of a correct build is not a tripwire, it is noise —
it trains readers to ignore the one line that catches #581. The genuine
warning still fires wherever an override was requested and did not take.
* smoke-test.sh Phase 1b pins the SHIPPED artifact's wiring per platform and
fails in BOTH directions: Windows/Linux must own all classes, macOS must
not. This cannot live in a unit test — a from-source test build never has
the override — which is exactly why the defect survived so long.
Verified on ubuntu-arm64: prod owns all classes; the test build reports
bound-populations-only at INFO (0/6) and the mem suite stays green; smoke
Phase 1b passes, and fails with the expected message when the same tree is
built with the override forced off. On macOS: prod reports
bound_populations_only at INFO, smoke Phase 1b passes, mem suite 51/51.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A repository may ship .cbmpathwhitelist listing outside roots it would like
indexed alongside it. The file REQUESTS; it never GRANTS. Both attackers in this
project's threat model can write a file inside a repository — a malicious indexed
repo, and an agent with write access to the project — so a repo-local file cannot
be authoritative without handing them the boundary.
Approval is recorded user-level and keyed to a SHA-256 of the manifest bytes, so
editing the file lapses approval and it must be granted again. Without the hash a
repo approved once could widen itself forever on a later pull. This is direnv's
model, and it is the only shape that keeps a checked-in file's ergonomics without
making the file a permission slip. The hash covers the raw bytes rather than the
parsed entries: a reordering or comment change is still a change the approver has
not seen.
`allow-root --approve-manifest <project>` is the human action that grants.
Approval refuses outright if any requested entry would not stand on its own as an
indexing root, so approving cannot become a route around the breadth policy, and
entries are re-classified at use time as well so a stored approval cannot outrank
a credential list that has grown since.
Entries containing control characters are rejected — the same shape as the
newline splitting that let a crafted indexed path inject an extra entry into the
scoped file list.
Deliberately NOT wired into the root boundary. A project's manifest authorizes
outside roots FOR THAT PROJECT, so the question is "may project P pull in tree T",
not "may T be indexed standalone", and the boundary check only ever sees one path
with no project context to ask that with. A first draft did wire it there by
passing the candidate as its own project root, which read a manifest that by
definition was not the one requesting it and so authorized nothing — caught by
exercising it end to end. The consuming half belongs where project context exists,
in discovery walking a project's approved extra roots;
cbm_workspace_manifest_allows is the query it will use.
Five tests, including the one the design rests on: approval lapses when the
manifest content changes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
fork() duplicates the parent's address-space bookkeeping. Under an
ASan-instrumented parent that includes an enormous shadow mapping, and past a
footprint threshold the child is jetsam-killed BEFORE exec replaces the image.
The reaped corpse then looks like the launched tool dying, so a `git init` in a
fixture "fails" only once enough suites have run ahead of it — a spawn defect
that reads as an unrelated assertion. tests/test_daemon_runtime.c already
switched to posix_spawn for this exact reason; this moves the shared
subprocess layer over as well.
posix_spawn never copies the parent address space, so the parent's footprint
stops being a variable. Every guarantee of the fork path is carried over:
- own process group SETPGROUP + setpgroup(0) (kill-tree contract)
- default signal handling SETSIGDEF + SETSIGMASK
- std{in,out,err} wiring adddup2
- all other fds closed CLOEXEC_DEFAULT, Apple's equivalent of the
child's close-everything loop (the three dup2'd
fds survive, since dup2 clears close-on-exec)
- PATH lookup posix_spawnp keeps execvp semantics
One deliberate difference is reconciled rather than adopted: posix_spawn
reports an unusable binary to the PARENT, where fork+exec instead yields a
child that exits 127. cbm_subprocess_run's contract treats spawn_failed as
"the spawn mechanism failed", not "the tool was missing", so exec-class errors
fall back to fork+exec and macOS keeps classifying a bogus binary exactly as
Linux does. That path forks a child that exits immediately, so it does not
reintroduce the hazard.
The existing suite binds each guarantee, verified by breaking them:
dropping CLOEXEC_DEFAULT leaks the sentinel descriptor and reddens
subprocess_posix_child_closes_unrelated_descriptors; dropping SETPGROUP makes
the kill-tree turn on the runner's own group and reddens the suite wholesale.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A forced client/daemon build mismatch raises a cohort conflict locally but is
ADMITTED on every CI leg, which made the #1388 conflict regression untestable
there. Reading the code explains how that is possible: identity comparison
happens ONLY in the BUSY branch, i.e. when another process already holds the
cohort lifetime lock. When the EX acquire succeeds - no live holder - the
participant claims the cohort and NO comparison runs at all, so a mismatched
build joins.
That is correct behaviour when nothing is running. What is missing is the
ability to tell the two apart after the fact: "was the lifetime lock held?" is
exactly the question that separates the local run from the CI run, and no log
line answered it.
This is deliberately observability only - no admission behaviour changes. The
next CI run that exercises the path will say which branch it took, and that
decides whether the CI daemon simply is not holding the lease (an environment
fact) or the lease is being released early (a real bug).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The dry run claimed EVERY hook group as installable because gate_ok /
session_ok / subagent_ok were simply set to `true` when dry_run was on. When
the on-disk hook script is not ours - user-modified, or written by a manual
install pointing at another binary - the real install refuses to rewrite it,
so the preview promised what the run could not deliver. That is the second
half of #1387: the reporter had no way to see the loss coming, and said so.
- New read-only predicate cbm_text_owned_document_status() mirrors
text_migrate_owned_document's decision WITHOUT writing: absent / already
current / an exact released document => a write would proceed; anything else
=> it would be refused.
- The dry run uses it per script, so the three 'hooks:' lines now reflect what
will actually happen, and each refused script is NAMED with the reason.
Silence was the original defect: an unmentioned group reads as 'nothing to
do', not 'this will be skipped'.
Test: cli_dry_run_predicts_refused_hook_script_issue1387 runs the real
dry-run path over an unowned gate script, capturing stdout, and asserts the
warning names the script. RED before, GREEN after, RED again on revert.
cli+agent_profiles+config_text_edit: 292 passed; lint-ci clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The grant store was readable but nothing could write to it, so only
CBM_ALLOWED_ROOT could declare a root. `allow-root <path>` records one,
`--list` shows what is recorded, and `--approve-sensitive` is required for a
home or credential directory. Roots that are refused outright — volume and
drive roots, paths too broad to index as a unit — cannot be granted at all.
Enrollment is a command a person types and deliberately nothing else. That is
the property the whole store exists for: neither an indexed repository nor a
tool caller can widen its own boundary. A confirmation delivered through the
MCP surface would be answered by the same agent that may have been influenced,
so it would not be a human decision.
The path is canonicalized before recording, because the policy is defined over
resolved paths and a grant stored as a symlink would not match the resolved
path the indexer later presents.
Classified stateless in the daemon bootstrap: it writes one line of user-level
config and reads nothing from a project, so enrolling a root must not depend on
daemon state. Without that classification the argument fell through to MCP
server mode and exited silently on EOF.
Verified end to end: /etc refused as too broad, $HOME refused pending the flag,
a project directory recorded, a granted root indexes, and a non-granted root is
refused naming the exact command that would allow it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two loose ends from the bounds work.
The decompression side already took size_t so a >2 GiB capacity could not wrap
through int, but compression still took int for both the source length and the
destination capacity, and the artifact export cast a size_t database size down
to reach it. A database past 2 GiB would have handed the encoder a negative
length. Both lengths and the bound helper are size_t now, and the function
returns int64_t like its decompressing counterpart.
The discovery walk now prunes the cache directory by absolute path. A custom
CBM_CACHE_DIR may sit inside a repository — tests do it routinely — and walking
into it pulls every other project's graph database into this project's file
list. This is the narrow form of a concern that was briefly implemented as
refusing any root that contained the cache; refusing a whole root was too blunt,
and not walking the cache is what the concern actually asks for.
The test fails without the prune: a .go file planted under the cache is
otherwise discovered, and "cache" is not in the built-in skip list, so the
assertion is not vacuous.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
"Users" was matched the way credential directory names are — against every
path component. On Windows every user's files live under C:\Users\<name>, so
that refused every ordinary project path, including a CI runner's own
workspace. The Windows smoke job on the pull request caught it; the unit
tests could not, because they only ever run compiled on macOS here.
System trees (Windows, ProgramData, Program Files) now match only as the
first component below the drive, which still refuses anything inside them at
any depth. "Users" is handled separately as the tree root only, mirroring
POSIX where "/Users" is refused but "/Users/dev/projects" is not.
Tests cover the regression directly: C:/Users/dev/projects/app and
C:/Users/runneradmin/work/repo are allowed, C:/Users is not, C:/Windows/System32
is refused at depth, and C:/dev/Windows-app — merely named after a system
tree — is allowed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Routes the MCP index_repository handler and the graph UI's POST /api/index
through a single decision function. The UI route previously checked only that
root_path was a directory, so an operator's configured boundary held on one
entry point and not the other; it now canonicalizes first and applies the same
policy, answering 403 with the reason.
The decision is two-tier, because a bare default-deny would refuse every
first run and a bare opt-in leaves the default open:
- Breadth is always enforced, with nothing configured. Filesystem, drive and
share roots, top-level system trees, the home directory itself and
credential directories are refused as indexing roots out of the box.
- Containment in a declared root applies once CBM_ALLOWED_ROOT is set or a
grant exists, and is evaluated first so a path outside a configured root is
reported as exactly that.
Three things the tests caught, each a real defect rather than a test fix:
- On macOS /etc, /tmp and /var are firmlinked under /private, so
canonicalizing "/etc" yields "/private/etc" and counted two deep — sailing
past a minimum of two, missing the very path being guarded. Depth now
discounts a leading "private" component.
- An earlier draft refused any root containing the cache directory. That was
over-claimed: the indexer only parses recognised source files and a graph
database is binary SQLite it would never extract. Refusing a whole root is
also the wrong remedy where the concern does hold — not walking the cache
is. Removed, with the reasoning recorded at the site.
- Rewording the refusal to "outside every allowed root" broke an assertion
matching "outside the allowed root", and that test's early return skipped
its CBM_ALLOWED_ROOT cleanup, leaking the variable into every later test in
the suite. The original wording is kept and guidance appended instead.
Worth remembering: these contracts match strings, not properties.
Docs updated in the same change, since both env-var tables said "unset
imposes no restriction" and that is no longer true: CONFIGURATION.md and
README.md describe the two tiers, and CONFIGURATION.md lists the always-
refused roots along with the two limits that matter — this constrains scope
rather than sensitivity, and the credential list is a denylist that raises the
cost of a mistake rather than closing the class. SECURITY.md's supported-
versions table was still on 0.8.x and now reads 0.9.x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Adds cbm_workspace_classify_root, the breadth half of the workspace
boundary. It answers "is this path obviously too broad or too sensitive to
index as one root", not "is this caller allowed to index it" — that second
question needs the user-level grant store, since it is the only input a
caller cannot write. The header says so, because a reader who mistakes one
for the other would draw the wrong conclusion about what is enforced.
Three rules, each doing what the others cannot:
- Depth below the volume. Two components on POSIX refuses every top-level
tree in one rule with no list to maintain. Windows and UNC count
drive/share-relative and require one, because there the first component is
already user space ("D:/repos") — the system trees that sit at the same
depth ("C:/Windows") are covered by name instead, which is tractable
because that set is small and stable.
- The home directory itself, which is two deep on macOS and Linux and so is
invisible to depth.
- Credential directory names, matched against every component so both
"…/.ssh" and "…/.ssh/keys" are refused. This list is deliberately additive
so extending it needs no design authority.
A path that is or contains the cache directory is refused outright: indexing
such a tree would pull every other project's graph database into this
project's index.
Classification order is load-bearing and commented as such. The home
directory normally contains the cache directory, so testing the cache first
reported every home as "holds the cache" and made it non-overridable, which
contradicts the intent that a person may override it. Depth precedes the
cache test for the same reason: "/Users" satisfies both and "too broad" is
the reason that helps the reader.
Takes home and cache as parameters rather than reading the environment, so
the policy is a pure function and the tests need no filesystem. Twelve table
tests; three of them failed on the first implementation and drove the
ordering fix above.
Not yet wired to any caller — deliberately, so the policy can be reviewed on
its own.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A pass over places where a bound was either miscalculated or missing.
UI response builders (src/ui/http_server.c) mixed the clamping helper
http_appendf with raw cursor indexing. Route the per-line separators and
quotes through the helper too, so the cursor cannot leave the buffer once
output saturates it, and terminate explicitly afterwards — the helper pins
the cursor and then writes nothing, which would otherwise leave the "%s"
reply with no terminator in range. The log buffer was budgeted at
LOG_LINE_MAX + 10 per line while JSON escaping can double every byte, so
size it for the escaped worst case. Job status entries now escape their path
and error fields, which were interpolated raw.
Cypher (src/cypher/cypher.c): the WHERE grammar descends once per nested '('
and once per NOT, so parse depth followed the query text rather than
anything bounded. Track depth on parser_t and refuse past 256 levels with a
parse error. Bounding parse depth also bounds the resulting tree, so the
recursive evaluator inherits the limit.
Gitignore (src/discover/gitignore.c): '**' retries the remainder at every
position and consecutive groups multiply, so match cost is exponential in
the number of groups. Thread a step budget through the matcher and give up
past 20000 steps, reporting no-match so a pathological pattern fails to
ignore rather than ignoring the wrong files. glob_match keeps its name so
recursion_whitelist.h still describes the functions that recurse; the new
non-recursive wrapper that seeds the budget is glob_match_bounded.
Call extraction (internal/cbm/extract_calls.c): extract_fp_callee recursed
once per applied argument, so stack use followed the parse-tree depth of the
indexed file. Rewritten as a left-spine loop. Reassigning node/nk before
continuing leaves the fall-through cases where the recursive form left them,
so behaviour is unchanged and the extraction suite is untouched.
CALLS props (src/pipeline/pass_calls.c): `cap - pos - PAIR_LEN` wraps once
pos reaches cap - PAIR_LEN and stops bounding the copy below. Guard
additively, matching the closing write in the same function.
Shell-bound paths (src/foundation/str_util.*): the three git shell-out sites
each wrap a repo path in cmd.exe-compatible double quotes, where %VAR%,
!VAR! and ^ remain active. Two carried private copies of the check that
rejects those three on Windows and the third used the bare validator.
Promote the stricter form to cbm_validate_shell_path_arg and route all three
through it, so the copies cannot drift apart again.
Tests cover the buffer bounds, the parse-depth refusal, matcher termination
and the validator. The bounds tests run in a forked child so a violation
surfaces as a signal instead of silently.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The end-to-end shell regression could not run on CI (recorded in the
previous commit), which left the fix without a gating test. Move the
notice DECISION out of main.c - which is not linked into the test
runner, and is why no C test could reach it - into src/cli/
hook_augment.c, which is. main.c now calls the shared API for both the
absent-daemon and build-conflict cases instead of carrying its own
copies of the strings, so the test binds the real production path
rather than a parallel one.
cli_hook_conflict_emits_stdout_notice_issue1388 asserts what the bug
actually broke: a build conflict yields a stdout systemMessage naming a
different build and pointing at 'daemon stop'; the absent-daemon notice
stays distinct and keeps pointing at 'daemon start' (which cannot heal
a conflict); and non-Claude dialects receive no bare stdout JSON, whose
channel it would corrupt.
RED-on-revert verified: restoring the pre-fix behaviour (conflict emits
nothing on stdout) fails this test alone, for the right reason
('conflict is NULL'). cli suite 258 passed, the local end-to-end shell
test still green against a seam-bearing binary, lint-ci clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The over-cap ORDER BY rejection path (and the two sibling error paths
in parse_return_or_with) freed the items array but not the strings
inside it - ASan flagged 7 leaked bytes on the diag and Linux legs the
moment the new over-cap test exercised the path. Route all three error
paths through free_return_clause(), which already releases item
strings, CASE expressions, args and the ORDER BY key list.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Windows users hit 'active CBM sessions and operations could not be
stopped safely; no activation was committed' from install, uninstall
and doctor with ZERO CBM processes running - they rebooted, killed
phantom handles, and deleted runtime folders chasing sessions that did
not exist. The real failure was the activation transaction's Windows
ACL safety check refusing a directory carrying cross-account mutation
grants (commonly the stock Authenticated Users:(M) inheritance from a
drive root - reproduced on the Windows VM with a plain mkdir under C:\
and with an icacls-granted parent), and the refusal detail was recorded
internally but never shown.
cli_activation_diagnostic now prefixes the recorded refusal note - the
predicate, SID and path - plus one remediation line (remove the flagged
grant or use an owner-private directory). The sessions wording remains
for genuine stop/reservation failures, which record no note.
Adds a CBM_ENABLE_TEST_SEAMS setter for the refusal note so the
attribution is testable portably; the regression test asserts the note
reaches the diagnostic (and that the no-note path keeps the sessions
text). RED before the fix, GREEN after, RED again on revert. Verified
end-to-end on the Windows VM: uninstall/doctor against an ACL-tainted
tree now name the ACL check instead of sessions.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A hook client that could not join because the active daemon runs a
DIFFERENT build reported the conflict on stderr only - and stdout is
the only hook channel Claude Code surfaces, so in-session the result
was eternal silent skips. Worse, the connect-failure path classified a
conflicted daemon as ABSENT and suggested 'daemon start', which cannot
heal a build conflict. This is the rc.1-over-0.9.0 experience in
#1388/#1335: install the new build, the old warm daemon keeps running,
and every hook goes quiet.
Both failure sites now emit a throttled stdout systemMessage naming the
real state (active daemon runs a different build) with the actionable
step ('daemon stop', next command starts a matching daemon). The
version-cohort conflict path keeps its unconditional stderr detail; the
connect-conflict path now prints the formatted conflict to stderr too.
Test infrastructure: a CBM_ENABLE_TEST_SEAMS-only env override
(CBM_TEST_HOOK_CLIENT_BUILD) lets one binary present a foreign build
fingerprint as a hook client, so the conflict is reproducible without a
second build. tests/test_hook_conflict_notice.sh (scripts/test.sh step
5d, on the step-5 TEST_SEAMS binary) asserts fail-open exit 0 + the
stderr conflict + the stdout systemMessage with restart guidance. RED
before the fix (empty stdout), GREEN after, RED again on revert.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
An image-verification rejection logged daemon.client_image_rejected and
finished the worker WITHOUT sending a hello response - the only
admission-failure path that never answered the peer. The client
reported status "pending" indefinitely, indistinguishable from a slow
cold start, with the reason visible only in the daemon log.
The rejection now sends a CONNECT_REJECTED hello response carrying the
reason (fingerprint_mismatch / image_unverifiable) and an actionable
message, matching the version-conflict and capacity paths. The
version-conflict path already responds to unverified peers, so this
discloses nothing new to a same-uid local peer; admission stays
rejected either way.
Adds a CBM_ENABLE_TEST_SEAMS-only seam forcing peer-image verification
to fail, since the in-process harness's peer pid is OS-authenticated
socket credentials and always verifies against the service's own
active image.
Test: daemon_runtime_image_rejection_reaches_client_issue1383 - the
client receives CONNECT_REJECTED with the reason in the message. RED
without the response block (client times out), GREEN with it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A project name failing cbm_validate_project_name made project_db_path()
return an empty path, which cbm_store_open_path_query passed to SQLite -
and SQLite opens "" as an anonymous temp database. The healthy temp db
then failed the integrity check (no projects table) and
quarantine_corrupt_store rendered ".corrupt.<hex>" from the empty
prefix: a RELATIVE path, dropped as a 4 KB file into whatever directory
the daemon was started from, on every such query. The caller only ever
saw a clean 'project not found', so nothing pointed at the litter.
Two guards, per the reporter's analysis: resolve_store_internal skips
the direct open on an empty path and falls through to the existing
fallback scan (which can still resolve legacy dbs whose internal name
predates validation), and quarantine_corrupt_store refuses an empty
path outright (belt-and-braces - nothing at such a path is worth
quarantining).
Regression test: tools/call search_graph with project "bad name" from
a temp cwd asserts the clean not-found error AND that no .corrupt.*
file appears in the cwd. RED before (litter created), GREEN after,
RED again on revert.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Cypher request model stored a single order_by expression; on
ORDER BY key1 DESC, key2 ASC LIMIT n the parser consumed only key1,
left ', key2 ... LIMIT n' unparsed, and the query silently returned the
entire result set (6326 rows / 117 KB instead of 5 on the reporter's
graph - a token flood straight into agent context).
The return clause now models up to CBM_CYPHER_ORDER_KEYS_MAX (8) sort
keys with per-key direction (Cypher semantics). Both sort sites -
rb_apply_order_by for RETURN and sort_bindings for the WITH pipeline -
compare key-by-key with later keys breaking ties. More keys than the
modeled maximum is a loud parse error, never a silently dropped
remainder. The no-ORDER-BY projection fast path is preserved via
order_key_count == 0.
Tests: parse-level (2 keys, per-key direction, LIMIT consumed;
9-key over-cap rejected), exec-level RETURN (limit kept, tiebreak
ordering, per-key direction), and exec-level WITH pipeline (limit kept).
All RED before the fix (row_count 4 instead of 2), GREEN after.
Recorded, pre-existing and unchanged: an ORDER BY key that is not part
of the projection is silently skipped (single-key main behaves the
same); sorting by unprojected properties needs hidden-column projection
and is a separate issue.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Installing 0.9.1-rc.1 over an existing setup deleted the user's working
Claude Code hook entries: when the on-disk hook script is not byte-owned
(a manual install embedding another binary path, or a user-modified
reminder), cbm_write_owned_hook_script_with_legacy correctly refuses the
rewrite (TEXT_UNOWNED) - but the failure branch then called
cbm_remove_claude_hooks / cbm_remove_session_hooks /
cbm_remove_claude_subagent_hooks, destroying the registrations for
scripts that still exist and work. The reporter lost their PreToolUse
Grep|Glob entry and all four SessionStart entries.
A refused script write now records the error and leaves settings.json
untouched; entry removal belongs to uninstall only (which keeps its own
removal path, cli.c:9622).
Regression test builds the reporter's state - unowned gate script with a
foreign BIN path, user-modified session reminder, 0.9.0-form settings
entries plus a foreign Bash hook - runs the full install, and asserts
every pre-existing entry and both script bodies survive byte-identically
while the failure is still reported. RED before this fix (entries
deleted), GREEN after, RED again on revert.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two changes that move the delta path toward O(change) without touching
the parallel-resolve contract.
Proxy narrowing. The delta executor pre-loaded every project node so
resolution could find cross-file targets by qualified name. Nodes that
resolution can never look up are pure load cost, and on the kernel they
dominate: six of its 8.5M nodes are Macro. The set is narrowed by
EXCLUSION rather than an inclusion list, deliberately -- an earlier
inclusion list was disproven by counterexamples it did not anticipate
(synthetic Decorator nodes, then Macro), and excluding a short list of
labels that are never lookup targets fails safe where guessing the full
inclusion set did not.
Patch-time identity mapping. A resolver that upserts a symbol the
narrowed set did not pre-load now produces a stand-in node; the patch
maps it back onto its existing row by qualified name instead of raising
the UNIQUE violation the previous patch would have. Nodes from CHANGED
files cannot collide here -- the purge removed them -- so repaired files
still receive fresh rows. The map is a sorted array searched by
bisection, not a CBMHashTable: that table stores key POINTERS without
copying them, which a stack-formatted integer key cannot satisfy.
NOT attempted here, and recorded instead: making the proxy load itself
lazy. cbm_parallel_resolve documents main_gbuf as READ-ONLY during its
worker phase and its workers do call cbm_gbuf_find_by_qn on it, so a
find-time materializer would mutate a buffer under concurrent readers.
A safe version needs the materialization hoisted ahead of the worker
phase; that is a separate change with its own verification.
Kernel one-file warm: 50.8s -> 42.7s wall, peak RSS 13.2 -> 10.8GB,
proxies 8.5M -> 2.5M (preseed 18.3s -> 9.7s), and the run reports
remapped=0 -- no resolver needed a symbol the narrowing dropped. 535
pipeline/incremental/store/cross-repo/integration tests green, including
the full convergence matrix.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
closure_probe_surfaces runs cbm_parallel_extract to compute the changed
files' fresh surfaces, and parallel extraction builds the process-global
package map as a side effect. Both real extraction paths release it at an
explicit ownership boundary; the probe borrowed the machinery without
inheriting that contract, leaking one map per probed run.
Found by the macOS leak lane added earlier in this branch -- the lane
catching a defect introduced after it, which is the point of having it.
Verified: the incremental suite is clean under LSan with the fix, and the
same allocation site (cbm_pkgmap_build via merge_pkg_entries) no longer
appears.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two more measured blocks out of the delta path, closing the
optimization arc:
Parallel rehydration. The base-def decode (2.37M defs from ~89k
surface rows at kernel scale) fans out across workers with per-worker
arenas that live exactly as long as the resolve borrows them; assembly
stays in row order, so the registration input is byte-identical to the
serial loop's. Measured 315ms for the block; the arc also disproved an
earlier attribution -- the planner including its full surface load is
608ms, no projection needed.
Known-healthy finalize. prepare_existing_generation_for_replace runs
PRAGMA quick_check over the ENTIRE outgoing generation to choose
replace-vs-quarantine -- 35.5s of full-database page scan at kernel
scale. The delta route cloned that same file and ran complete
transactions against the clone minutes earlier; a corrupt live
database cannot reach the delta finalize because every earlier step
fails it into the dump path, whose finalize keeps the check and the
quarantine semantics unchanged (as its corruption tests continue to
prove). Sidecars are still removed on the fast path -- a replaced
database must never inherit the old generation's WAL. Delta publish
total: 35.7s -> 224ms.
finalize/publish timing brackets stay as durable telemetry.
Kernel one-file warm across today's arc: 306s (binary routing) -> 223s
(closure via dump) -> 121.7s (delta) -> 85.3s (query plans + shadow
gate) -> 50.8s wall / 34.3s worker; peak RSS 33.6 -> 13.2GB. Remaining
named blocks: proxy preseed 18.3s and repair/mdi 11.1s, both
serial-bound gbuf/index builds -- recorded follow-ups, not mysteries.
437 pipeline/incremental tests green throughout.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel