Resolves the index_status tool description: main's watch state (#2167) and this branch's doc_links both stay.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A reference written in a doc comment is extracted with the definition it
documents (internal/cbm/doclink.c; the C# scanner doclink_cs.c) and
resolved per file in the pipeline (src/pipeline/doc_links.c,
doc_links_cs.c, doc_links_msbuild.c). A reference that names a definition
of the repository with certainty becomes a MENTIONS edge from the
documented definition (the File node for a file-level comment) with the
properties via, syntax, tier, line and count. Any other reference becomes
a row of the new doc_link_unresolved table with its reason: missing,
ambiguous, external, test_only_target, graph_gap, unparseable or
below_bar_tier. Nothing is guessed. index_status reports the layer in a
doc_links block.
C# binds a cref by the compiler's lookup rule: the first scope with the
name decides, a cref never binds an inherited member, System is outside
the repository. Assemblies come from project files, and MSBuild global
usings (<Using>, ImplicitUsings, Directory.Build.props/.targets, imports,
conditions) are evaluated from the project files' content as data the
indexer has already read; the layer opens no file. Two declarations of a
name in different assemblies make the reference ambiguous.
Each file's scope is stored line-free with its LSP surface ("dl"), so an
incremental run re-resolves the files a change can reach and falls back
to a full run for a change that can re-route references anywhere (a
project file, a global using, a deleted file).
The scanner and the resolver treat the repository as untrusted input:
bounded nesting, no input-driven recursion, work that grows linearly
(held by test seams, never by a clock), no silent limits, and a damaged
or foreign stored scope fails closed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Since #1948 a non-git root can be polled by a file-tree scan when
watch_non_git is on. index_status still reported such a root as
strategy "none", which the enum documents as "not a git project: never
polled", so a root that is actively polled looked unwatched.
publish_status now maps a tree-polled root to the new
CBM_WATCHER_STRATEGY_TREE, rendered as "tree" (the value the
watcher.baseline log already uses). "none" keeps its meaning: a
registered non-git root that is never polled. CONFIGURATION.md lists
the four values.
Test: daemon_application_index_status_reports_tree_strategy_issue2167
fails without the production change and passes with it; suites
daemon_application, watcher and mcp: 507 passed, 4 platform skips.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The background watcher only watches the project an open MCP session is
rooted in, and drops the watch when the last session for that project
closes. A repository indexed with `cli index_repository` (or from a
session rooted elsewhere) is never watched, but README promised
"Auto-sync keeps it fresh after that" for every index_repository target
and the MCP instructions said "Indexes auto-refresh". Nothing in the
status output said which projects were actually watched, so a stale
CLI-indexed repo looked like a watcher bug.
What gets watched is unchanged. Instead:
- index_status gains a `watch` object: `watched`, plus `strategy`,
`poll_interval_ms` and `last_scan_at` when watched, or a `reason`
(not_session_project, cli_session, auto_watch_off, watcher_disabled,
not_registered, no_watcher) and a re-index hint when not. It does not
touch the #1561 `freshness` key.
- The watcher publishes per-project strategy, cadence and last completed
scan time through atomics and exposes them via
cbm_watcher_project_info(); the daemon answers index_status through a
session-scoped provider.
- README, docs/CONFIGURATION.md, docs/index.html, the index_status tool
description and the MCP instructions now state the real scope.
`watcher_enabled` already shows in `config list` since v0.11.0 (680bc72e).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A cache root whose resolved path runs through a directory owned by another
non-root account (typical on shared/HPC filesystems where ~/.cache is a
symlink into a shared tree) is refused by the private-directory walk. The
refusal itself is correct: the owner of an ancestor can rename or replace
everything below it after the walk, so it could redirect the cache. But
posix_directory_transition_secure() set no validation detail, so the caller
printed "ancestry component validation failed (errno 17)" -- a stale EEXIST
from mkdirat() that named neither the directory nor the reason.
- The refusal now names the directory, its owner uid and account name, the
caller's uid, and the remedy (a location with a self/root-owned ancestry,
e.g. via CBM_CACHE_DIR). The path is shortened from the left so the
remedy always fits the 384-byte detail buffer.
- On Linux the account name is read from /etc/passwd, not getpwuid(). The
release binary is statically linked against glibc, and getpwuid() there
loads the host's NSS modules at runtime, which can crash on a glibc
version mismatch (a real risk on sssd/LDAP cluster hosts). Accounts that
exist only in a directory service are shown by uid.
- New advice-only check: on Linux, a cache root on NFS, CIFS, SMB2, Lustre
or GPFS (statfs f_type) logs one daemon.cache_root_network_fs warning per
process. SQLite WAL is unreliable on network filesystems. Nothing is
refused.
- docs/CONFIGURATION.md gains a "Shared / HPC filesystems" section.
The ownership policy is unchanged, and there is still no override (same
stance as #1830).
Test seams: one seam reports a directory (matched by dev/ino) as
foreign-owned, and another supplies the statfs magic. The refusal, the
message and the once-per-process warning are therefore testable without
root and on every platform.
Refs #1717
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
README.md:132, README.md:633, README.md:761, and docs/CONFIGURATION.md:203
said one-shot `cli` commands "never start or connect to the coordination
daemon" / read "their own environment without starting the daemon". That
was already contradicted a few lines above it (README.md:126 lists
one-shot CLI commands among the processes that "share a crash-safe OS
admission barrier" with the daemon) and by the code:
main_local_cli_daemon_execute() (src/main.c) bootstraps a client
connection to the shared per-user coordination daemon via
main_client_bootstrap_with_upgrade(), spawning one (bootstrap.daemon_spawned)
when none is running, before dispatching the tool call.
Verified live: `cli --verbose list_projects` against an isolated
HOME/CBM_CACHE_DIR/CBM_RUNTIME_DIR printed "hint: this command started a
temporary CBM daemon", and its cbm-daemon.log showed
daemon.start -> daemon.runtime_stopping (reason=last_committed_client_disconnected)
-> daemon.lifetime_end, i.e. the daemon it spawned exited again once the
CLI command's own connection closed. bootstrap_production_spawn() passes
the calling process's `environ` straight to posix_spawn(), so a CLI
invocation that starts the daemon also seeds that daemon's captured
daemon-owned environment (CBM_DIAGNOSTICS, CBM_LOG_LEVEL, etc.) - it does
not read "its own environment" independently as the docs claimed.
Corrected all four passages to state the real behavior: CLI commands
connect to the shared daemon (starting one if none is running) for the
same admission barrier and per-project locks, hold a `cli_session` that
is never registered with the background watcher, and - if their own
connection is what started the daemon - that daemon exits again once the
command's connection closes and no other session is attached. No opt-out
env var or flag for an in-process/no-daemon CLI mode exists (grepped the
whole tree), so none is documented.
This is a pure documentation fix; no production behavior changed. Kept
clear of PR #2348 (fix/issue-2167), which separately rewrites the
watcher-scope paragraph at README.md:152/237/672 and
docs/CONFIGURATION.md:88 for issue #2167's auto-sync-scope correction -
none of the lines here overlap with that diff.
Refs #2183, #2167
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A project indexed from a directory that is not a git repository was never
refreshed after its first index: init_baseline classified it is_git=false
and poll_project/check_changes returned early for it forever, so the graph
silently went stale. The comment in init_baseline even claimed such folders
were "polled as a plain directory", which was never true.
Add an opt-in config key, watch_non_git (default false, read once at daemon
startup like watcher_enabled). When on, a non-git root is polled on the
existing adaptive cadence (cbm_watcher_poll_interval_ms) by a tree
signature: the indexer's own cbm_discover walk (same skip lists, .gitignore
stack and .cbmignore), sorted by path and folded over (relative path, size,
mtime). A changed signature takes the existing reindex path with the same
commit-after-success discipline as the #937 dirty signature. The baseline
leaves the signature "unknown", so the first poll reindexes once
(at-least-once, covering edits made while the daemon was down).
Runaway guard: only files discovery would index enter the signature, so
cbm's own .codebase-memory output (built-in skip) and the cache directory
(pruned by absolute path) can never make a finished reindex look like a
new change; a scratch folder under an unrelated dirty repository is polled
on its own files and never inherits the ancestor's dirty state.
cbm_file_info_t gains mtime_ns, filled from the stat discovery already
performs, so the signature needs no second stat per file and stays
UTF-8-correct on Windows (discovery's wide stat).
Docs: README and docs/CONFIGURATION.md now say non-git roots are not
watched by default and document watch_non_git. The staleness signal in
index_status is left to #1561's freshness object. Refs #1948.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
storage/framework/views/ holds Laravel's compiled Blade templates:
generated PHP with no source value. It was excluded only through
Laravel's stock .gitignore in that folder; when that file is missing,
every compiled view was indexed and reported as parse_partial by the
php grammar, burying real coverage gaps under generated noise.
Every built-in skip rule matched a directory BASENAME, and "views" is
far too common to skip everywhere. Add ALWAYS_SKIP_DIR_SUFFIXES, a
built-in rule list matched against the relative path suffix on a
'/'-component boundary, so storage/framework/views (at any depth, e.g.
backend/storage/framework/views) is skipped while resources/views,
app/views, app/framework/views, mystorage/framework/views and
storage/framework/views_old stay indexed. The rule is applied in
should_skip_directory next to the basename skip list, so it shares its
semantics: the subtree is reported as an excluded dir (#411) in the
index result and as not_indexed_dir "excluded subtree" in
check_index_coverage, and a .cbmignore negation
(!storage/framework/views/) can un-skip it like any non-safety-core
built-in skip (#500).
Test: discover::discover_skips_laravel_compiled_views_issue1735 (RED
before, green after, RED on revert) and
discover::discover_cbmignore_negates_laravel_compiled_views_issue1735.
Docs: docs/cbmignore.md precedence layer 1.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On WSL2 the Windows drives under /mnt/<drive> are DrvFs (9p) mounts. With
the default automount options they report 0777 for every directory and
ignore chmod, so a CBM_CACHE_DIR there is refused by the private-directory
walk: the ancestor rule (not world-writable) fails first, and the leaf
0700 rule would fail right after. The refusal is correct -- the 0777 is
real -- but it gave WSL users no way forward.
The security gate is unchanged. When the refusing directory is on a 9p
filesystem under a WSL kernel (fstatfs magic 0x01021997 plus
/proc/sys/kernel/osrelease), the ancestor refusal and the leaf
"mode survived chmod" refusal now name the remedy: set
[automount] options = "metadata,umask=22,fmask=11" in /etc/wsl.conf and
run `wsl --shutdown`, or keep the cache on the Linux filesystem. The
ancestor message uses shorter rule text on DrvFs so the remedy fits
the 384-byte detail buffer. Detection compiles only on Linux; other
platforms and non-WSL 9p mounts keep the generic message.
A CBM_ENABLE_TEST_SEAMS switch forces the DrvFs verdict so the daemon_ipc
suite checks the remedy text (and its absence off DrvFs) deterministically.
docs/CONFIGURATION.md documents the WSL requirement.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The card headline already reads "17 MCP tools" but the body named
eight tools and then said "and 7 more" — 8 + 7 = 15, the old count.
TOOL_ANNOTATIONS[] in src/mcp/mcp.c has 17 entries, so the tail is 9.
Assisted-by: Claude Code / claude-opus-5
Machine: MacBook-Anton
Account: a@gmail
Operator: robot:pr-reply-daily
Signed-off-by: Anton Dziatkovskii <194927794+tonydzi@users.noreply.github.com>
The TOOLS[] registry in src/mcp/mcp.c and the v0.11.0 --help output
list 17 tools, but README, the npm README, CONTRIBUTING and the Pages
site still say 15 (or 14), and the README tool table has no row for
get_file_outline, compare_graphs or check_index_coverage.
Refs #1297
Assisted-by: Claude Code / claude-opus-5
Machine: MacBook-Anton
Account: a@gmail
Operator: robot:git-s3-docs-fix-lane
Signed-off-by: Anton Dziatkovskii <194927794+tonydzi@users.noreply.github.com>
Indexing accepts whatever a repository contains. A tree carrying a vendored
monorepo, a generated dump, or a runaway build directory is discovered in
full, and the first sign of trouble is a host under memory pressure with
nothing that attributes it to indexing.
Add two opt-in limits evaluated during discovery against accepted source
files only: index_max_files and index_max_source_mb. Both default to off, so
nothing changes until an operator sets one. Crossing a limit fails the whole
attempt with a structured resource_limit_exceeded result naming the resource,
the observed value and the limit; no partial graph is published, and an
existing serving index keeps answering.
Limits are read from the CLI-managed _config.db and are not MCP request
arguments. A supervised parent replaces any caller-supplied policy before
spawning its worker, and the worker rejects a missing or incomplete contract,
so the CLI, the daemon and the supervised worker all enforce the same
decision.
Both keys reach an operator through the existing config get/set/list/reset
with no new subcommand. `set` suppresses its own generic message for them
because the policy writer names the precise reason -- so that writer speaks
on every failure it can return, including a validated value whose write then
fails on a database that cannot be written. Exiting non-zero in silence is
not an acceptable answer from a CLI.
The two shell regressions that hand-roll the supervisor's worker argv carry
that contract as well. Without it the worker exits before either guard can
observe anything, and the guard would go quietly vacuous.
Signed-off-by: 刘冲 <mail@liuchong.dev>
Chialisp (.clsp/.clib/.clinc) is the Chia smart-coin language. The only
public grammar, Quexington/tree-sitter-chialisp, cannot parse it: on the five
reference files from chia-blockchain@main it produced 3-34 ERROR nodes each.
Three defects account for all of them, and each desynchronised the rest of the
file rather than losing one node:
* comments were `/;.*\r\n/` -- CRLF-only, while real files are LF;
* `(include foo.clib)` was rejected because of the dot in the filename;
* `(defconstant NAME <expr>)` accepted only a primitive after the name.
So the grammar is written here instead, in tools/tree-sitter-chialisp/, and
files as first-party: its vendored LICENSE is the project's own, byte-identical
to the repository root, carrying no third-party copyright.
It is deliberately GENERIC -- source_file / list / symbol / string / number /
hex / dot / comment, modelling the clvm_tools reader rather than the Chialisp
form vocabulary. `mod`, `defun`, `defconstant` and `include` are ordinary head
symbols; which lists are definitions is decided in extract_defs.c. A dialect
that adds a form (cl-21 -> cl-23 -> cl-26 each added several) therefore needs
no regenerated parser. Acceptance: all five reference files parse with zero
ERROR and zero MISSING nodes; the three defects above are pinned as tests.
Extraction, and the four decisions in it:
* `Constant` is admitted to the cross-file name registry
(cbm_label_is_registry_symbol). Chialisp's condition codes live as
`(defconstant ...)` in a shared .clib and are referenced by name from every
puzzle; labelled but unregistered they are seeded nowhere, and roughly half
of a Chialisp graph would be present as nodes yet never the target of an
edge. All three registry-seeding sites route through that one predicate, so
they cannot diverge. No other language emits `Constant`, so the blast radius
is exactly Chialisp. Pinned by a two-file graph test that goes red when the
label is removed from the predicate.
* `export` / `namespace` are NOT definition heads. `(export foo)` names a
function already defined in the same file, so treating it as a def head
mints a second node for one symbol. They stay in the not-a-call filter.
* A parameter list is a `list` too, but it binds rather than invokes. Without
that gate `(defun check_conditions (HEIGHTLOCK conditions) ...)` mints a
phantom call to HEIGHTLOCK, and real Chialisp is mostly such definitions.
* `(include *standard-cl-26*)` is a dialect directive, not a file dependency;
recording it would invent a dependency on a file that does not exist.
Three helpers that the defs, calls and unified walks all need -- the quote-data
gate, comment-skipping named-child access, and the Chialisp def-head set -- live
in helpers.c rather than being copied per translation unit. They previously
existed as byte-identical duplicates that had already drifted: extract_defs.c
skipped comments when reading a def head while its mirror in extract_unified.c
did not, so a comment between a head and its name desynchronised defs from call
scope. Both now use the same accessor.
Ledger: 161 -> 162 grammars across the seven published surfaces, ABI-14 86 -> 87,
first-party 13 -> 14, CBM_LANG_COUNT 164 -> 165.
Distilled from PR #919 by darkverbito, whose extraction logic -- the CLVM
not-a-call vocabulary, the quoted-data gate, the filename-derived `mod` name and
the include/embed import handling -- is kept and is the substance of this change.
The grammar that PR vendored is not used.
Co-authored-by: darkverbito <8049985+darkverbito@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Seven surfaces disagreed with reality and with each other: README.md,
docs/index.html, docs/llms.txt and pkg/npm/README.md said 158, while
server.json, the chocolatey nuspec and scripts/package-release.sh said 159.
161 grammars are vendored.
21 individual claims corrected. The count is now gated by
tests/test_language_count_contract.sh, so the next language to land cannot
ship a stale figure.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Grok Build (xAI's grok CLI/TUI) already reached codebase-memory-mcp through
its Claude/Cursor compat layer, but a Grok-only user got nothing. Detect
$GROK_HOME (default ~/.grok) or a grok CLI and install:
- an owned [mcp_servers.codebase-memory-mcp] managed block in config.toml;
a same-name foreign table is left byte-identical because a second header
would make Grok reject the whole file, while the known owned shape written
by `grok mcp add` is adopted. No env forwarding: grok 1.0.5 passes the
full parent environment to stdio servers (verified), unlike Codex (#1562).
- an owned rules file under $GROK_HOME/rules/ (always scanned, all projects)
- the shared skill under $GROK_HOME/skills/
- Scout/Verify/Auditor agents with named-server mcpInheritance and the tier
allowlist spelled out as the server__tool ids Grok's search_tool/use_tool
dispatcher accepts (inheritance filters per server, never per tool)
- hooks withheld: Grok's passive events discard stdout and PreToolUse honors
only deny/rewrite decisions, so there is no context-output contract.
Verified against grok 1.0.5: inspect discovers the server, agents, skill and
rules; mcp doctor handshakes and discovers 15 tools. --clients=grok token,
README/npm/llms/landing-page matrices (43 -> 44 surfaces), and smoke coverage.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Follow-up to the PR #1105 review: the unit test proves the config predicate but
would still pass even if the daemon-side gate were removed. Add a process-level
regression that drives the real binary and proves the gate actually prevents the
watcher from being built, started or registered when disabled.
- tests/test_watcher_disabled.sh (wired into scripts/test.sh as Step 5e) drives
the binary against an isolated CBM_CACHE_DIR + tiny git fixture. With
watcher_enabled=false it asserts: watcher.disabled reason=config emitted,
watcher.start absent, no watcher.watch registration, auto_index still runs, and
manual index_repository is served. A watcher_enabled=true positive control
asserts watcher.start + watcher.watch appear, so the test fails if the gate is
removed (verified by neutering the gate in host_state_prepare and observing the
failure).
No assertion is decided by a timeout. The watcher lives in the daemon, which
logs to $CBM_CACHE_DIR/logs/cbm-daemon.log in a fixed order: watcher.disabled
(host_state_prepare) -> daemon.start -> the thread's own watcher.start -> the
join -> daemon.stop. Each run therefore retires its daemon and waits for that
CLOSED lifecycle before asserting, so absence is a fact rather than a race.
Every wait is a bounded poll on an asserted state following the soak harness's
wait_for_daemon_stop doctrine, and an exhausted budget fails loudly with the
reason instead of passing quietly; session stdin is held open through a FIFO
until the last JSON-RPC response lands, not for an interval.
Registration is asynchronous and unreachable until the project DB exists
(application_refresh_watch_locked requires application_regular_db_exists), so a
closed lifecycle alone would not carry the negative — the session could simply
have ended first. The disabled+auto_index run waits for that DB, i.e.
registration's own precondition; the enabled control waits for watcher.watch
itself, so its absence in the disabled runs is evidence rather than noise.
Honors CBM_TEST_BINARY like tests/test_worker_watchdog.sh, so the BUILD_DIR=
sanitizer and container legs test the binary they built.
- docs: CONFIGURATION.md + README.md document that watcher_enabled is read once
when the daemon starts — the daemon outlives individual MCP sessions, so
reconnecting a client is not enough; `daemon stop` retires it. This corrects
the earlier claim that auto_watch is also startup-read: auto_watch is consulted
per registration, and the two keys are now contrasted on read time as well as
on scope (session registration vs whether the subsystem starts at all).
No production code change in this commit: watcher.start and watcher.watch already
exist as production logs.
Signed-off-by: soren <nguiasoren@gmail.com>
Add a persistent `watcher_enabled` config key (default true). When false, the
daemon host skips building the background watcher entirely: the poll thread
never starts and no projects are registered. Manual index_repository is
unaffected, and the config wiring stays unconditional so auto_index /
auto_watch keep working even when the watcher is off.
This is distinct from auto_watch, which only gates per-session registration
while the watcher IS running. watcher_enabled is the master switch that stops
the subsystem from starting at all — the request in #335 (doc-heavy repos where
every save churns the graph baseline).
The key is read once, when the daemon builds its host state. Changing it takes
effect when the daemon next starts; docs/CONFIGURATION.md says so explicitly.
A disabled watcher is not a daemon-startup failure — the daemon still comes up
and still owns IPC, HTTP/UI, project locks, and manual indexing; only the
watcher and the in-memory store backing it go unbuilt.
- cli.h/cli.c: CBM_CONFIG_WATCHER_ENABLED key + cbm_config_watcher_enabled()
predicate (NULL-safe, default true), registered as a CONFIG_KEYS row next to
auto_watch. The table is no longer only a print list: since #1522/#1558 it
also backs key validation, so `config set/get/reset watcher_enabled` would be
rejected as an unknown key without that row, and the default recorded there
(true) is the one the table comment asks to keep in sync with the runtime
reader's fallback.
- daemon/host.c: gate watcher construction in host_state_prepare() and the
thread create in host_background_start(); log watcher.disabled when skipped.
Teardown needed no change — stop NULL-checks, join is gated on
watcher_started, and cbm_watcher_free is NULL-safe.
- test_cli.c: cli_config_watcher_enabled_default_and_persist pins the config
layer — default on, disable/re-enable spellings, persistence across reopen,
and a NULL-config default (a config-store failure must never silently
disable the watcher).
- docs: document watcher_enabled, and add the previously-undocumented auto_watch
row to CONFIGURATION.md so the watcher knobs are described together; README
example.
Closes#335.
Signed-off-by: soren <nguiasoren@gmail.com>
Add omp (Oh My Pi) as a STABLE, detected client surface with full
four-layer integration: MCP server registration, durable instructions,
skill, and three direct-MCP tiered subagent profiles (Scout/Verify/
Auditor). This matches Claude Code's integration depth — omp subagents
have direct MCP access (confirmed via ablation testing), unlike
parent-handoff-only clients (Pochi, Cursor, Rovo, Augment).
Schema (agent_clients.h):
New CBM_AGENT_CLIENT_OMP enum value, positioned before COUNT.
Profile (agent_clients.c):
- stability: CBM_AGENT_STABLE (auto-detected, auto-installed)
- capabilities: MCP | INSTRUCTIONS | SKILL | AGENT
No HOOK — omp's context event is fire-once advisory only; the
recommended upstream path is skill + agents + MCP, not TS hooks.
- detection_command: "omp" (specific enough per existing clients)
- marker: ~/.omp/agent (omp-managed config root)
- resolve_path: ~/.omp/agent/mcp.json
- json_client: true (uses standard mcpServers JSON edit path)
- agent_json_canonical: added to type:"stdio" branch alongside
GitLab Duo and Visual Studio (omp's schema expects this field)
Dialect (agent_profiles.h/.c):
New CBM_GRAPH_DIALECT_OMP, direct-capable (not in handoff-only list).
- tool prefix: "mcp__codebase_memory_mcp_" (single underscore,
hyphens→underscores — verified at runtime via omp device inventory)
- render: YAML frontmatter with tools (read/grep/glob + tier-specific
MCP tool allowlist), read-summarize: false (exact source verification
for evidence tiers — omp's read returns structural summaries by
default), autoloadSkills: [codebase-memory] (subagents start blank)
Install/uninstall (cli.c):
- install_omp_durable_context: writes AGENTS.md + skill +
3 tiered agent profiles to ~/.omp/agent/
- uninstall_omp_durable_context: removes managed block + skill +
tiered profiles
- legacy_omp_verify_agent_content: migration source for prior installs
- Wired into install_agent_client_registry and
uninstall_agent_client_registry loops
Design decisions (documented for reviewers):
1. No HOOK capability: omp's tool_call event can only block/pass (no
augmentation channel like Claude's additionalContext). A context-
event nudge was tested but is advisory-only and excluded from the
upstream path to keep the PR minimal.
2. read-summarize: false: omp's read tool returns structural summaries
by default; Tier 2/3 require exact snippets for material claims.
3. autoloadSkills: omp subagents start with no conversation history;
without autoload the skill decision matrix is never loaded.
4. Direct-capable (not handoff): ablation testing confirmed omp
subagents can call mcp__* tools directly. write→xd:// dispatch
does NOT work in subagents ("xd:// is not mounted in this session").
Test coverage:
- test_agent_clients.c: omp in stable_ids (19 entries), capabilities
array, json_schemas (type:stdio assertion), foreign_entries
- test_agent_profiles.c: omp in direct_dialects array, direct-capable
assertion, dedicated test verifying direct profile has
mcp__codebase_memory_mcp_ prefix + read-summarize + autoloadSkills,
and handoff profile excludes MCP tools
- test_cli.c: required_agents updated to 44, all 43→44 / 37→38 string
contracts updated across README/npm/index.html/llms.txt/main.c
Build: scripts/build.sh passes (exit 0).
Tests: scripts/test.sh passes (exit 0, all suites green).
Lint: scripts/lint.sh has one pre-existing error in pass_lsp_cross.h
(parameter name mismatch imp_keys/imp_vals vs imp_names/imp_qns),
unrelated to this change.
Signed-off-by: DavidHLP <lysf15520112973@163.com>
The daemon/CLI rendezvous directory is created under %LOCALAPPDATA% (Windows) or
/tmp -- /private/tmp on macOS -- and every ancestor of it must pass the
private-directory walk. That ancestry is not always acceptable, and when it is
not, EVERY invocation fails, `config list` included, so the settings surface
cannot be reached either:
codebase-memory-mcp: secure daemon endpoint could not be created
#1623 narrowed the Windows side of this by admitting AppContainer package and
capability SIDs on ancestors, and named the remainder explicitly: a live local
group, Authenticated Users inherited from a secondary volume root, and orphaned
unresolvable SIDs still refuse, and "those need CBM_RUNTIME_DIR or a separate
change". #1621 is the POSIX shape of the same dead end -- /private/tmp/cbm-daemon-<uid>
refused with no way to move it.
There was no way to move it in a shipped build. The only relocation hook,
CBM_TEST_DAEMON_RUNTIME_PARENT, is compiled out unless CBM_ENABLE_TEST_SEAMS is
defined, so a test build started while the shipped build did not; CBM_CACHE_DIR
is no help either, because it moves the cache and never the rendezvous.
CBM_RUNTIME_DIR names the parent directory the rendezvous is created under. It
does NOT relax the check: the directory it names goes through exactly the same
validation as the default -- ancestors owned by you or root, not world-writable,
no allow-ACL; the rendezvous directory itself still forced to owner-only -- and a
value that fails is refused rather than silently replaced by the default. The
operator only chooses an ancestry that passes. cbm_safe_getenv never truncates,
so no half of an over-long value can become a runtime parent.
The override is resolved in cbm_daemon_bootstrap_endpoint_new(), the one function
every product endpoint goes through: the daemon, the MCP client, the local CLI,
the index worker, and the install/update/uninstall activation path in cli.c. No
call site can silently keep the default, and the detached daemon inherits the
value with the rest of its environment. An explicit parent still wins, so the
compile-time test seam and the lifecycle guards' isolated namespace behave
exactly as before.
Approach and variable name from #1576 by Leonardo trindade miranda, resolved one
layer lower so the activation path is covered too.
Refs #1574
Refs #1621
Co-Authored-By: Leonardo trindade miranda <tmonestudio@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Release evidence from run 31744302624 shows the tolerated Microsoft `!ml`
verdict is close to a coin flip per byte image rather than a property of the
code. Across the eight targets the stripped and unstripped candidates of the
SAME linker output disagreed on four, and in both directions:
linux-amd64 stripped microsoft-ml unstripped clean
darwin-arm64 stripped clean unstripped microsoft-ml
linux-arm64 stripped clean unstripped microsoft-ml
If the classifier were keying on something intrinsic to our code the siblings
would agree; they do not. So each variant is close to an independent draw, and
5 of 16 candidates drew the flag.
Two draws is not always enough. On that run linux-amd64-portable came back
microsoft-ml on BOTH candidates, leaving no clean binary to ship for that
target. A third independent draw at a ~31% observed per-candidate hit rate takes
the both/all-flagged case from roughly 1-in-10 per target to roughly 1-in-30.
The third candidate is `--strip-debug` (Apple: `-S`): debug information removed,
symbol table kept. Behaviourally identical to the other two — same linker
output, only metadata differs — but a distinct byte image, which is all
VirusTotal needs to scan it as its own file. Verified on the real v0.10.4
candidates: linux-amd64 gives three distinct hashes (294,634,656 /
294,623,208 / 293,746,096 bytes) and darwin-arm64 likewise, with the ad-hoc
signature verifying after strip.
Selection is unchanged in spirit and now ordered: smallest artifact first
(stripped, debug-stripped, unstripped), take the first CLEAN one, and only if
every candidate drew the tolerated verdict ship the smallest flagged one. A
hard verdict on any candidate still blocks the release before selection.
Cost is 24 objects per release instead of 16.
Derivation enforces that all three hashes differ — identical candidates would be
one draw wearing three hats, and the selector would believe it had alternatives
it does not have. Public claims in README, SECURITY.md and docs/index.html
updated from "both stripped and unstripped" to the three candidates.
All three release contract tests pass, including the native derivation test
which exercises the real strip and codesign path on this host.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The build+VT dry-run (run 31744302624) blocked all eight targets on:
BLOCKED: objects/3eb486a4... completed with only 48/75 decisive engines (< 50)
That object is the windows-arm64 STRIPPED candidate with ZERO detections. It
was classified "hard" — the same bucket as a real malware verdict — purely
because 48 engines answered instead of 50. Its own sibling scanned clean with
66.
How many engines return a decisive result is VirusTotal fleet availability on
the day, not a property of our binary. These artifacts are ~300 MB and many
engines skip or time out at that size, so the count sits near the floor and
moves run to run: on this run one object landed at 48 and the other fifteen
spread 59-68. A floor there makes shipping a lottery decided by someone else's
infrastructure, which is exactly what our CI-determinism rule forbids, and it is
not something we can influence by fixing anything on our side.
Classification now depends only on what engines FOUND:
hard - two or more engines, any non-Microsoft engine, any label that is
not `!ml`, or anything suspicious. Still blocks the release.
microsoft-ml- exactly one Microsoft `!ml`. Tolerated and disclosed.
clean - nothing flagged.
The decisive-engine count is still recorded per candidate in the results
manifest and printed as a NOTE when it is below the reference value, so the
evidence is unchanged; only its power to block is gone. The selector keeps
rejecting an incoherent response (total < completed).
Public claims updated to match in README.md, SECURITY.md and docs/index.html —
all three promised "at least 50 decisive engines" as a publication requirement,
which would no longer have been true.
Contract test updated: a clean verdict now stands regardless of engine count,
and the below-reference NOTE must still be reported. All three release contract
tests pass (vt-gate-policy, release-gate-chain, candidate-derivation).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Routes the MCP index_repository handler and the graph UI's POST /api/index
through a single decision function. The UI route previously checked only that
root_path was a directory, so an operator's configured boundary held on one
entry point and not the other; it now canonicalizes first and applies the same
policy, answering 403 with the reason.
The decision is two-tier, because a bare default-deny would refuse every
first run and a bare opt-in leaves the default open:
- Breadth is always enforced, with nothing configured. Filesystem, drive and
share roots, top-level system trees, the home directory itself and
credential directories are refused as indexing roots out of the box.
- Containment in a declared root applies once CBM_ALLOWED_ROOT is set or a
grant exists, and is evaluated first so a path outside a configured root is
reported as exactly that.
Three things the tests caught, each a real defect rather than a test fix:
- On macOS /etc, /tmp and /var are firmlinked under /private, so
canonicalizing "/etc" yields "/private/etc" and counted two deep — sailing
past a minimum of two, missing the very path being guarded. Depth now
discounts a leading "private" component.
- An earlier draft refused any root containing the cache directory. That was
over-claimed: the indexer only parses recognised source files and a graph
database is binary SQLite it would never extract. Refusing a whole root is
also the wrong remedy where the concern does hold — not walking the cache
is. Removed, with the reasoning recorded at the site.
- Rewording the refusal to "outside every allowed root" broke an assertion
matching "outside the allowed root", and that test's early return skipped
its CBM_ALLOWED_ROOT cleanup, leaking the variable into every later test in
the suite. The original wording is kept and guidance appended instead.
Worth remembering: these contracts match strings, not properties.
Docs updated in the same change, since both env-var tables said "unset
imposes no restriction" and that is no longer true: CONFIGURATION.md and
README.md describe the two tiers, and CONFIGURATION.md lists the always-
refused roots along with the two limits that matter — this constrains scope
rather than sensitivity, and the credential list is a denylist that raises the
cost of a mistake rather than closing the class. SECURITY.md's supported-
versions table was still on 0.8.x and now reads 0.9.x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.
Daemon lifecycle:
- daemon start/stop/status subcommands. `daemon start` launches a
PERMANENT daemon (spawn shape is byte-exact argv; survives idle
periods and session ends) and reports an already-active daemon
instead of failing. Permanence is honored at every stop latch:
last-committed-client disconnect, host initial-client window,
coordinator release, and application final-session close — a
permanent daemon also keeps admitting new sessions after its last
one closes.
- daemon stop refuses while sessions are active and lists the blocking
peers (pid/role) that must finish first; an idle daemon drains
through the activation-shutdown machinery with the ACK ordered after
connection interrupts. A second stop is idempotent. The wire ops are
no-cohort first-frame requests with peer fingerprint authentication,
so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
keep their local supervised path). A cold CLI run that had to spawn a
temporary daemon prints a hint that `daemon start` removes the
per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
emits a visible, rate-limited notice (Claude-dialect systemMessage
plus stderr for other dialects) and always exits 0 — augmentation is
never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
older-build permanent daemon (strict semantic-version triples only;
dev builds never auto-drain) and the build-conflict message names
`cbm daemon stop` as the manual escape hatch.
Windows IPC/runtime (real-VM verified):
- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
path used to destroy its listening pipe instance on every 20 ms poll
timeout; a client attaching in the teardown window was severed or left
on an orphaned pipe object whose HELLO no server handle could ever
read, absorbing the connect until the client's own timeout expired.
The pending connect now survives poll timeouts and nothing is
destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
server handle can discard a just-sent response before the peer reads
it (POSIX stream sockets never lose buffered data on close). A bounded
cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
immediate on interrupted connections) now precedes close in
runtime_worker_finish and runtime_reject_inline, so hello-conflict,
capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
transport half-close, so close_begin now announces departure with an
explicit frame (ordered after APPLICATION_CANCEL, before the local
interrupt); the server releases the client's admission on receipt
instead of waiting for the handle to close. Admission-drop timing is
now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
request publication under the send lock, best-effort sends the active
token's APPLICATION_CANCEL frame, then interrupts local I/O; the
server cancels MCP/subprocess work promptly. Contract tests accept
both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
the requester's license to act on "snapshotted and draining", so every
connection interrupt is now initiated before the ACK is sent; a
session could previously get one more request serviced after the
requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
timer granularity) burned ~1.6 s against permanently obstructed paths,
stalling hello rejections past the client's timeout. The retry budget
is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
nonblocking send() on Windows is absorbed wholesale into AFD kernel
buffering regardless of SO_SNDBUF, so send deadlines and interrupts
could never engage against a slow peer (and the full payload was
pinned in nonpaged pool). Slicing restores a deterministic
backpressure point; a test hook pins SO_SNDBUF for the deadline and
interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
deadline, explicit connection states, refusal to free a server while
a listener-owned connection is active.
Windows long-path support:
- Central path-aware wide conversion (canonicalize via GetFullPathNameW
and prepend the extended-length prefix for absolute paths >=240) at
the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
store opens, and the daemon build-fingerprint/log paths. Deep managed
installs (a 64-hex generation directory routinely exceeds MAX_PATH)
now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
ancestry validators now operate in the extended-length namespace;
the launcher path is canonicalized (and prefixed when deep) once at
entry so every downstream exact-string comparison stays
form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
GetFileAttributesW) so non-ASCII install paths survive argv[0]
resolution.
Windows launcher install/uninstall transaction:
- FileRenameInfoEx names are NUL-terminated in an over-allocated
buffer. FileNameLength governs per the contract, but filter drivers
read FileName as NUL-terminated and appended adjacent heap bytes to
created names — a flaky, garbage-suffixed rename target. Both the CLI
and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
.cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
chars so the bare rename target stays under the FileRenameInfoEx
NT-conversion ceiling at guard depths; 64 bits still uniquely
identify the generation.
- When the running launcher's mapped generation backings pin .cbm
against rename, the backings are relocated to activation-<pid>-N
.retired tombstones beside the install (a mapped image may be renamed,
never deleted; the launcher's liveness-guarded sweep reclaims stale
tombstones). Every relocation is recorded, and a FAILED uninstall
reverses the moves after restoring .cbm — via MoveFileExW with
extended-length paths on both arguments, since the deep generation
target is beyond the handle-based rename's bare-path reach — so a
restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
out so the tree is shallow enough for the detached cleanup's rd, and
the cleanup's working directory strips the extended-length prefix
(CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
(CopyFileW destinations, CREATE_NEW tombstones, probe directories)
are explicitly owner-stamped so the exact-owner validators hold on
runner images; guard fixtures stamp hand-built trees the same way.
Diagnostics, tests and infra:
- diagnostics: discovery is now an always-delivered JSON control record
(new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
paths containing spaces; placement honors $TMPDIR with /tmp fallback
via a diagnostics-local helper; the soak parser reads the JSON record;
documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
log levels, TMPDIR-with-spaces, and native Windows output-contract
assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
parents with production-shaped ancestry (LocalAppData on Windows, via
th_secure_runtime_parent_new) — the runtime ancestry validation
correctly refuses temp roots whose ancestors grant mutation rights to
Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
drive the documented startup-owner publication flow before reading
generation-bound endpoint addresses. This turns the 26 Windows
failures previously visible in CI's full-test job green without
weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
contract including failed-uninstall restore and immediate reinstall
after uninstall; new daemon lifecycle and reworked hook-augment
guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
(it was set by CI but never read, so perf suites ran everywhere);
four throughput/bench suites are classified as perf, the heavy
store_arch suite moved to the slow-timeout tier, and two
wall-clock-sensitive assertions were rewritten as invariant checks
with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
contract, soak daemon-recovery contract, path-safety helper, the
Windows VM worktree-sync contract wired into scripts/test.sh, and
vm/win.sh guards building its clean embedded-UI product in an
isolated BUILD_DIR so it cannot clobber the incremental test build.
provision-windows.sh now installs Node.js for the guards UI build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The per-language chapters were LLM-drafted to establish format and
dimension mapping — their symbol targets are not yet ground-truth-derived
and must be regenerated at execution time per §3.1/§12. Make that status
explicit at the top of Appendix A so the drafted symbol names are not
read as validated.
Supersedes #420 (same diff; recreated for DCO author/sign-off).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Add CBM_ALLOWED_ROOT to the environment-variable tables in README.md and
docs/CONFIGURATION.md: when set, index_repository refuses a repo_path that
resolves outside it; unset imposes no restriction.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Documents the current behavior as implemented in src/discover/discover.c
and src/discover/gitignore.c: where .cbmignore is read from, the glob
features the parser supports, the layered precedence against built-in
skip lists / .gitignore hierarchy / git global excludes, and what
negation can and cannot override today. Planned negation unification
(un-skipping built-in dirs, non-negatable safety core, shared predicate
for auxiliary walkers) is listed in an explicitly not-yet-implemented
subsection. Linked from the README "Ignoring Files" section.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel