mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-10-30 07:23:47 +08:00
bb650ca45ae95fd8d8e91e74be2f9b684b6727fe
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Martin Vogel
|
859294561a |
feat(index): the documentation pass follows the index mode by default
Doc-section -> code candidates (doc_link_candidates, #2574/#2610) are the documentation-in-the-graph feature of this release. They now run by default for full indexes and not for moderate or fast ones; an explicit `doc_candidates` argument, or a `doc_candidates` setting the user set, overrides the default either way in every mode. - pipeline: the switch is unset/on/off; unset means on only when the effective mode is full. - MCP: forwards only an explicit choice (argument, then a user-set config value) to daemon jobs and workers; daemon auto-index and watcher/UI jobs now carry a user-set value too (their workers have no config, so `doc_candidates false` was ignored there). - fast + doc_candidates on runs the semantic pass's corpus phases and the doc phase only: candidates, and still no SIMILAR_TO or SEMANTICALLY_RELATED edges. - config: `doc_candidates` joins the config key table (default `auto`); `config set doc_candidates ...` was refused as an unknown key before. README, docs/CONFIGURATION.md and the MCP schema describe the default. Proof (macOS arm64, django fb11376): full 11,304 rows · moderate 0 · fast 0 · full+off 0 · moderate+on 1 · fast+on 1 (moderate/fast discovery skips docs/) -- nodes and edges are identical between on and off in every mode; fast+on adds 0 semantic edges. - New tests (RED on the old code): pipeline pipeline_doc_candidates_default_follows_mode, pipeline_doc_candidates_explicit_wins_over_mode, pipeline_doc_candidates_fast_on_legacy_incremental; mcp tool_index_repository_doc_candidates_follow_mode, tool_index_repository_doc_candidates_forwarded_only_when_explicit; daemon_application daemon_application_programmatic_index_forwards_set_doc_candidates. - mcp 374 passed (4 Windows-only skips), daemon_application 73 passed, pipeline/store_arch/cli/runtime_settings/index_policy/pipeline_frozen green on the release tree. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
d3b8d258be |
feat(index): doc_candidates setting and index_repository parameter
The doc<->code candidate scoring (#2574, doc_section_worker in semantic_edges) scores every documentation section against the functions that share a key term with it, key terms being those held by at most funcs/20 functions. Its cost therefore grows with sections x functions. Measured on 11 corpora (10-09 main vs |
||
|
Martin Vogel
|
72edfce600 |
Merge main into fix/issue-2167 for the release batch
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
6bda6e26de |
fix(watcher): report a tree-polled non-git root as strategy "tree"
Since #1948 a non-git root can be polled by a file-tree scan when watch_non_git is on. index_status still reported such a root as strategy "none", which the enum documents as "not a git project: never polled", so a root that is actively polled looked unwatched. publish_status now maps a tree-polled root to the new CBM_WATCHER_STRATEGY_TREE, rendered as "tree" (the value the watcher.baseline log already uses). "none" keeps its meaning: a registered non-git root that is never polled. CONFIGURATION.md lists the four values. Test: daemon_application_index_status_reports_tree_strategy_issue2167 fails without the production change and passes with it; suites daemon_application, watcher and mcp: 507 passed, 4 platform skips. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
3794085492 |
fix(watcher): show watch state in index_status, correct auto-sync docs (#2167)
The background watcher only watches the project an open MCP session is
rooted in, and drops the watch when the last session for that project
closes. A repository indexed with `cli index_repository` (or from a
session rooted elsewhere) is never watched, but README promised
"Auto-sync keeps it fresh after that" for every index_repository target
and the MCP instructions said "Indexes auto-refresh". Nothing in the
status output said which projects were actually watched, so a stale
CLI-indexed repo looked like a watcher bug.
What gets watched is unchanged. Instead:
- index_status gains a `watch` object: `watched`, plus `strategy`,
`poll_interval_ms` and `last_scan_at` when watched, or a `reason`
(not_session_project, cli_session, auto_watch_off, watcher_disabled,
not_registered, no_watcher) and a re-index hint when not. It does not
touch the #1561 `freshness` key.
- The watcher publishes per-project strategy, cadence and last completed
scan time through atomics and exposes them via
cbm_watcher_project_info(); the daemon answers index_status through a
session-scoped provider.
- README, docs/CONFIGURATION.md, docs/index.html, the index_status tool
description and the MCP instructions now state the real scope.
`watcher_enabled` already shows in `config list` since v0.11.0 (
|
||
|
Martin Vogel
|
5cd9891e57 | Merge branch 'main' into fix/issue-1717 | ||
|
Martin Vogel
|
db1dcf4600 |
Merge pull request #2333 from DeusData/fix/issue-1948
feat(watcher): opt-in polling of non-git project roots (#1948) |
||
|
Martin Vogel
|
6eb33534f6 |
fix(daemon): explain foreign-owned cache ancestors and warn on network filesystems (#1717)
A cache root whose resolved path runs through a directory owned by another non-root account (typical on shared/HPC filesystems where ~/.cache is a symlink into a shared tree) is refused by the private-directory walk. The refusal itself is correct: the owner of an ancestor can rename or replace everything below it after the walk, so it could redirect the cache. But posix_directory_transition_secure() set no validation detail, so the caller printed "ancestry component validation failed (errno 17)" -- a stale EEXIST from mkdirat() that named neither the directory nor the reason. - The refusal now names the directory, its owner uid and account name, the caller's uid, and the remedy (a location with a self/root-owned ancestry, e.g. via CBM_CACHE_DIR). The path is shortened from the left so the remedy always fits the 384-byte detail buffer. - On Linux the account name is read from /etc/passwd, not getpwuid(). The release binary is statically linked against glibc, and getpwuid() there loads the host's NSS modules at runtime, which can crash on a glibc version mismatch (a real risk on sssd/LDAP cluster hosts). Accounts that exist only in a directory service are shown by uid. - New advice-only check: on Linux, a cache root on NFS, CIFS, SMB2, Lustre or GPFS (statfs f_type) logs one daemon.cache_root_network_fs warning per process. SQLite WAL is unreliable on network filesystems. Nothing is refused. - docs/CONFIGURATION.md gains a "Shared / HPC filesystems" section. The ownership policy is unchanged, and there is still no override (same stance as #1830). Test seams: one seam reports a directory (matched by dev/ino) as foreign-owned, and another supplies the statfs magic. The refusal, the message and the once-per-process warning are therefore testable without root and on every platform. Refs #1717 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
c86be38954 |
docs: correct CLI-mode claim that one-shot commands never touch the daemon (#2183, #2167)
README.md:132, README.md:633, README.md:761, and docs/CONFIGURATION.md:203 said one-shot `cli` commands "never start or connect to the coordination daemon" / read "their own environment without starting the daemon". That was already contradicted a few lines above it (README.md:126 lists one-shot CLI commands among the processes that "share a crash-safe OS admission barrier" with the daemon) and by the code: main_local_cli_daemon_execute() (src/main.c) bootstraps a client connection to the shared per-user coordination daemon via main_client_bootstrap_with_upgrade(), spawning one (bootstrap.daemon_spawned) when none is running, before dispatching the tool call. Verified live: `cli --verbose list_projects` against an isolated HOME/CBM_CACHE_DIR/CBM_RUNTIME_DIR printed "hint: this command started a temporary CBM daemon", and its cbm-daemon.log showed daemon.start -> daemon.runtime_stopping (reason=last_committed_client_disconnected) -> daemon.lifetime_end, i.e. the daemon it spawned exited again once the CLI command's own connection closed. bootstrap_production_spawn() passes the calling process's `environ` straight to posix_spawn(), so a CLI invocation that starts the daemon also seeds that daemon's captured daemon-owned environment (CBM_DIAGNOSTICS, CBM_LOG_LEVEL, etc.) - it does not read "its own environment" independently as the docs claimed. Corrected all four passages to state the real behavior: CLI commands connect to the shared daemon (starting one if none is running) for the same admission barrier and per-project locks, hold a `cli_session` that is never registered with the background watcher, and - if their own connection is what started the daemon - that daemon exits again once the command's connection closes and no other session is attached. No opt-out env var or flag for an in-process/no-daemon CLI mode exists (grepped the whole tree), so none is documented. This is a pure documentation fix; no production behavior changed. Kept clear of PR #2348 (fix/issue-2167), which separately rewrites the watcher-scope paragraph at README.md:152/237/672 and docs/CONFIGURATION.md:88 for issue #2167's auto-sync-scope correction - none of the lines here overlap with that diff. Refs #2183, #2167 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
c06ceb1638 |
feat(watcher): opt-in polling of non-git project roots (#1948)
A project indexed from a directory that is not a git repository was never refreshed after its first index: init_baseline classified it is_git=false and poll_project/check_changes returned early for it forever, so the graph silently went stale. The comment in init_baseline even claimed such folders were "polled as a plain directory", which was never true. Add an opt-in config key, watch_non_git (default false, read once at daemon startup like watcher_enabled). When on, a non-git root is polled on the existing adaptive cadence (cbm_watcher_poll_interval_ms) by a tree signature: the indexer's own cbm_discover walk (same skip lists, .gitignore stack and .cbmignore), sorted by path and folded over (relative path, size, mtime). A changed signature takes the existing reindex path with the same commit-after-success discipline as the #937 dirty signature. The baseline leaves the signature "unknown", so the first poll reindexes once (at-least-once, covering edits made while the daemon was down). Runaway guard: only files discovery would index enter the signature, so cbm's own .codebase-memory output (built-in skip) and the cache directory (pruned by absolute path) can never make a finished reindex look like a new change; a scratch folder under an unrelated dirty repository is polled on its own files and never inherits the ancestor's dirty state. cbm_file_info_t gains mtime_ns, filled from the stat discovery already performs, so the signature needs no second stat per file and stays UTF-8-correct on Windows (discovery's wide stat). Docs: README and docs/CONFIGURATION.md now say non-git roots are not watched by default and document watch_non_git. The staleness signal in index_status is left to #1561's freshness object. Refs #1948. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
19f50de3d2 |
fix(daemon): name the WSL DrvFs remedy when a /mnt/<drive> cache is refused (#1687)
On WSL2 the Windows drives under /mnt/<drive> are DrvFs (9p) mounts. With the default automount options they report 0777 for every directory and ignore chmod, so a CBM_CACHE_DIR there is refused by the private-directory walk: the ancestor rule (not world-writable) fails first, and the leaf 0700 rule would fail right after. The refusal is correct -- the 0777 is real -- but it gave WSL users no way forward. The security gate is unchanged. When the refusing directory is on a 9p filesystem under a WSL kernel (fstatfs magic 0x01021997 plus /proc/sys/kernel/osrelease), the ancestor refusal and the leaf "mode survived chmod" refusal now name the remedy: set [automount] options = "metadata,umask=22,fmask=11" in /etc/wsl.conf and run `wsl --shutdown`, or keep the cache on the Linux filesystem. The ancestor message uses shorter rule text on DrvFs so the remedy fits the 384-byte detail buffer. Detection compiles only on Linux; other platforms and non-WSL 9p mounts keep the generic message. A CBM_ENABLE_TEST_SEAMS switch forces the DrvFs verdict so the daemon_ipc suite checks the remedy text (and its absence off DrvFs) deterministically. docs/CONFIGURATION.md documents the WSL requirement. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
刘冲
|
21ae02154a |
feat(index): add opt-in discovery resource limits
Indexing accepts whatever a repository contains. A tree carrying a vendored monorepo, a generated dump, or a runaway build directory is discovered in full, and the first sign of trouble is a host under memory pressure with nothing that attributes it to indexing. Add two opt-in limits evaluated during discovery against accepted source files only: index_max_files and index_max_source_mb. Both default to off, so nothing changes until an operator sets one. Crossing a limit fails the whole attempt with a structured resource_limit_exceeded result naming the resource, the observed value and the limit; no partial graph is published, and an existing serving index keeps answering. Limits are read from the CLI-managed _config.db and are not MCP request arguments. A supervised parent replaces any caller-supplied policy before spawning its worker, and the worker rejects a missing or incomplete contract, so the CLI, the daemon and the supervised worker all enforce the same decision. Both keys reach an operator through the existing config get/set/list/reset with no new subcommand. `set` suppresses its own generic message for them because the policy writer names the precise reason -- so that writer speaks on every failure it can return, including a validated value whose write then fails on a database that cannot be written. Exiting non-zero in silence is not an acceptable answer from a CLI. The two shell regressions that hand-roll the supervisor's worker argv carry that contract as well. Without it the worker exits before either guard can observe anything, and the guard would go quietly vacuous. Signed-off-by: 刘冲 <mail@liuchong.dev> |
||
|
Martin Vogel
|
5f58e233dc |
fix(output): preserve compact output safety signals
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
soren
|
d61811d27e |
test(watcher): process-level regression for watcher_enabled + startup-read docs (#335)
Follow-up to the PR #1105 review: the unit test proves the config predicate but would still pass even if the daemon-side gate were removed. Add a process-level regression that drives the real binary and proves the gate actually prevents the watcher from being built, started or registered when disabled. - tests/test_watcher_disabled.sh (wired into scripts/test.sh as Step 5e) drives the binary against an isolated CBM_CACHE_DIR + tiny git fixture. With watcher_enabled=false it asserts: watcher.disabled reason=config emitted, watcher.start absent, no watcher.watch registration, auto_index still runs, and manual index_repository is served. A watcher_enabled=true positive control asserts watcher.start + watcher.watch appear, so the test fails if the gate is removed (verified by neutering the gate in host_state_prepare and observing the failure). No assertion is decided by a timeout. The watcher lives in the daemon, which logs to $CBM_CACHE_DIR/logs/cbm-daemon.log in a fixed order: watcher.disabled (host_state_prepare) -> daemon.start -> the thread's own watcher.start -> the join -> daemon.stop. Each run therefore retires its daemon and waits for that CLOSED lifecycle before asserting, so absence is a fact rather than a race. Every wait is a bounded poll on an asserted state following the soak harness's wait_for_daemon_stop doctrine, and an exhausted budget fails loudly with the reason instead of passing quietly; session stdin is held open through a FIFO until the last JSON-RPC response lands, not for an interval. Registration is asynchronous and unreachable until the project DB exists (application_refresh_watch_locked requires application_regular_db_exists), so a closed lifecycle alone would not carry the negative — the session could simply have ended first. The disabled+auto_index run waits for that DB, i.e. registration's own precondition; the enabled control waits for watcher.watch itself, so its absence in the disabled runs is evidence rather than noise. Honors CBM_TEST_BINARY like tests/test_worker_watchdog.sh, so the BUILD_DIR= sanitizer and container legs test the binary they built. - docs: CONFIGURATION.md + README.md document that watcher_enabled is read once when the daemon starts — the daemon outlives individual MCP sessions, so reconnecting a client is not enough; `daemon stop` retires it. This corrects the earlier claim that auto_watch is also startup-read: auto_watch is consulted per registration, and the two keys are now contrasted on read time as well as on scope (session registration vs whether the subsystem starts at all). No production code change in this commit: watcher.start and watcher.watch already exist as production logs. Signed-off-by: soren <nguiasoren@gmail.com> |
||
|
soren
|
680bc72e75 |
feat(config): add watcher_enabled master switch for the background watcher (#335)
Add a persistent `watcher_enabled` config key (default true). When false, the daemon host skips building the background watcher entirely: the poll thread never starts and no projects are registered. Manual index_repository is unaffected, and the config wiring stays unconditional so auto_index / auto_watch keep working even when the watcher is off. This is distinct from auto_watch, which only gates per-session registration while the watcher IS running. watcher_enabled is the master switch that stops the subsystem from starting at all — the request in #335 (doc-heavy repos where every save churns the graph baseline). The key is read once, when the daemon builds its host state. Changing it takes effect when the daemon next starts; docs/CONFIGURATION.md says so explicitly. A disabled watcher is not a daemon-startup failure — the daemon still comes up and still owns IPC, HTTP/UI, project locks, and manual indexing; only the watcher and the in-memory store backing it go unbuilt. - cli.h/cli.c: CBM_CONFIG_WATCHER_ENABLED key + cbm_config_watcher_enabled() predicate (NULL-safe, default true), registered as a CONFIG_KEYS row next to auto_watch. The table is no longer only a print list: since #1522/#1558 it also backs key validation, so `config set/get/reset watcher_enabled` would be rejected as an unknown key without that row, and the default recorded there (true) is the one the table comment asks to keep in sync with the runtime reader's fallback. - daemon/host.c: gate watcher construction in host_state_prepare() and the thread create in host_background_start(); log watcher.disabled when skipped. Teardown needed no change — stop NULL-checks, join is gated on watcher_started, and cbm_watcher_free is NULL-safe. - test_cli.c: cli_config_watcher_enabled_default_and_persist pins the config layer — default on, disable/re-enable spellings, persistence across reopen, and a NULL-config default (a config-store failure must never silently disable the watcher). - docs: document watcher_enabled, and add the previously-undocumented auto_watch row to CONFIGURATION.md so the watcher knobs are described together; README example. Closes #335. Signed-off-by: soren <nguiasoren@gmail.com> |
||
|
Martin VogelandLeonardo trindade miranda
|
7f3e30e1e1 |
fix(daemon): let a shipped build relocate the rendezvous via CBM_RUNTIME_DIR
The daemon/CLI rendezvous directory is created under %LOCALAPPDATA% (Windows) or
/tmp -- /private/tmp on macOS -- and every ancestor of it must pass the
private-directory walk. That ancestry is not always acceptable, and when it is
not, EVERY invocation fails, `config list` included, so the settings surface
cannot be reached either:
codebase-memory-mcp: secure daemon endpoint could not be created
#1623 narrowed the Windows side of this by admitting AppContainer package and
capability SIDs on ancestors, and named the remainder explicitly: a live local
group, Authenticated Users inherited from a secondary volume root, and orphaned
unresolvable SIDs still refuse, and "those need CBM_RUNTIME_DIR or a separate
change". #1621 is the POSIX shape of the same dead end -- /private/tmp/cbm-daemon-<uid>
refused with no way to move it.
There was no way to move it in a shipped build. The only relocation hook,
CBM_TEST_DAEMON_RUNTIME_PARENT, is compiled out unless CBM_ENABLE_TEST_SEAMS is
defined, so a test build started while the shipped build did not; CBM_CACHE_DIR
is no help either, because it moves the cache and never the rendezvous.
CBM_RUNTIME_DIR names the parent directory the rendezvous is created under. It
does NOT relax the check: the directory it names goes through exactly the same
validation as the default -- ancestors owned by you or root, not world-writable,
no allow-ACL; the rendezvous directory itself still forced to owner-only -- and a
value that fails is refused rather than silently replaced by the default. The
operator only chooses an ancestry that passes. cbm_safe_getenv never truncates,
so no half of an over-long value can become a runtime parent.
The override is resolved in cbm_daemon_bootstrap_endpoint_new(), the one function
every product endpoint goes through: the daemon, the MCP client, the local CLI,
the index worker, and the install/update/uninstall activation path in cli.c. No
call site can silently keep the default, and the detached daemon inherits the
value with the rest of its environment. An explicit parent still wins, so the
compile-time test seam and the lifecycle guards' isolated namespace behave
exactly as before.
Approach and variable name from #1576 by Leonardo trindade miranda, resolved one
layer lower so the activation path is covered too.
Refs #1574
Refs #1621
Co-Authored-By: Leonardo trindade miranda <tmonestudio@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
8018561cfe |
fix(release): externalize runtime assets and harden VT verification
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
7fa5b07718 |
feat: enforce one workspace boundary on every indexing entry point
Routes the MCP index_repository handler and the graph UI's POST /api/index through a single decision function. The UI route previously checked only that root_path was a directory, so an operator's configured boundary held on one entry point and not the other; it now canonicalizes first and applies the same policy, answering 403 with the reason. The decision is two-tier, because a bare default-deny would refuse every first run and a bare opt-in leaves the default open: - Breadth is always enforced, with nothing configured. Filesystem, drive and share roots, top-level system trees, the home directory itself and credential directories are refused as indexing roots out of the box. - Containment in a declared root applies once CBM_ALLOWED_ROOT is set or a grant exists, and is evaluated first so a path outside a configured root is reported as exactly that. Three things the tests caught, each a real defect rather than a test fix: - On macOS /etc, /tmp and /var are firmlinked under /private, so canonicalizing "/etc" yields "/private/etc" and counted two deep — sailing past a minimum of two, missing the very path being guarded. Depth now discounts a leading "private" component. - An earlier draft refused any root containing the cache directory. That was over-claimed: the indexer only parses recognised source files and a graph database is binary SQLite it would never extract. Refusing a whole root is also the wrong remedy where the concern does hold — not walking the cache is. Removed, with the reasoning recorded at the site. - Rewording the refusal to "outside every allowed root" broke an assertion matching "outside the allowed root", and that test's early return skipped its CBM_ALLOWED_ROOT cleanup, leaking the variable into every later test in the suite. The original wording is kept and guidance appended instead. Worth remembering: these contracts match strings, not properties. Docs updated in the same change, since both env-var tables said "unset imposes no restriction" and that is no longer true: CONFIGURATION.md and README.md describe the two tiers, and CONFIGURATION.md lists the always- refused roots along with the two limits that matter — this constrains scope rather than sensitivity, and the credential list is a denylist that raises the cost of a mistake rather than closing the class. SECURITY.md's supported- versions table was still on 0.8.x and now reads 0.9.x. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
0baddf7887 |
daemon: permanent lifecycle, daemon-backed CLI/hooks, real-Windows hardening, long-path launcher transactions
Daemon lifecycle and Windows correctness, verified on a real Windows 11 ARM64 VM through the maintained test-infrastructure/vm drivers, plus the macOS and Linux arm64 suites and the container lint gate. Daemon lifecycle: - daemon start/stop/status subcommands. `daemon start` launches a PERMANENT daemon (spawn shape is byte-exact argv; survives idle periods and session ends) and reports an already-active daemon instead of failing. Permanence is honored at every stop latch: last-committed-client disconnect, host initial-client window, coordinator release, and application final-session close — a permanent daemon also keeps admitting new sessions after its last one closes. - daemon stop refuses while sessions are active and lists the blocking peers (pid/role) that must finish first; an idle daemon drains through the activation-shutdown machinery with the ACK ordered after connection interrupts. A second stop is idempotent. The wire ops are no-cohort first-frame requests with peer fingerprint authentication, so stop/status never conflict with an exact-build admission gate. - One-shot CLI commands now execute through the daemon (index workers keep their local supervised path). A cold CLI run that had to spawn a temporary daemon prints a hint that `daemon start` removes the per-command startup tax; a warm daemon is recycled silently. - Hooks are connect-only fail-open: with no daemon present the hook emits a visible, rate-limited notice (Claude-dialect systemMessage plus stderr for other dialects) and always exits 0 — augmentation is never allowed to block the caller's tool use. - Version skew: a newer-build client automatically drains an older-build permanent daemon (strict semantic-version triples only; dev builds never auto-drain) and the build-conflict message names `cbm daemon stop` as the manual escape hatch. Windows IPC/runtime (real-VM verified): - ipc(win): persistent pending overlapped ConnectNamedPipe. The accept path used to destroy its listening pipe instance on every 20 ms poll timeout; a client attaching in the teardown window was severed or left on an orphaned pipe object whose HELLO no server handle could ever read, absorbing the connect until the client's own timeout expired. The pending connect now survives poll timeouts and nothing is destroyed while a client could be attaching. - ipc(win): drain-before-close for final responses. Closing a named-pipe server handle can discard a just-sent response before the peer reads it (POSIX stream sockets never lose buffered data on close). A bounded cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX, immediate on interrupted connections) now precedes close in runtime_worker_finish and runtime_reject_inline, so hello-conflict, capacity and disconnect acknowledgements reliably reach the peer. - runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no transport half-close, so close_begin now announces departure with an explicit frame (ordered after APPLICATION_CANCEL, before the local interrupt); the server releases the client's admission on receipt instead of waiting for the handle to close. Admission-drop timing is now identical to POSIX shutdown() semantics on every platform. - runtime(win): client close cancellation. close_begin serializes with request publication under the send lock, best-effort sends the active token's APPLICATION_CANCEL frame, then interrupts local I/O; the server cancels MCP/subprocess work promptly. Contract tests accept both correct outcomes (interrupted transport or decoded CANCELLED). - runtime: activation acknowledgement ordering. The activation ACK is the requester's license to act on "snapshotted and draining", so every connection interrupt is now initiated before the ACK is sent; a session could previously get one more request serviced after the requester observed the ACK. - service(win): deadline-bounded private-file prepare. The conflict-log prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms timer granularity) burned ~1.6 s against permanently obstructed paths, stalling hello rejections past the client's timeout. The retry budget is now a 250 ms deadline; transient share collisions still retry. - subprocess(win): cmd.exe /C payload encoder quotes metacharacters correctly (root cause of the git-on-Windows failure cluster). - watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native Windows stop/unwatch cancellation test with exact-image verification. - httpd: send_all writes in bounded 64 KiB slices. A single giant nonblocking send() on Windows is absorbed wholesale into AFD kernel buffering regardless of SO_SNDBUF, so send deadlines and interrupts could never engage against a slow peer (and the full payload was pinned in nonpaged pool). Slicing restores a deterministic backpressure point; a test hook pins SO_SNDBUF for the deadline and interrupt tests. - ui/http: shutdown lifecycle — interrupt checks, response-wide send deadline, explicit connection states, refusal to free a server while a listener-owned connection is active. Windows long-path support: - Central path-aware wide conversion (canonicalize via GetFullPathNameW and prepend the extended-length prefix for absolute paths >=240) at the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite store opens, and the daemon build-fingerprint/log paths. Deep managed installs (a 64-hex generation directory routinely exceeds MAX_PATH) now index, stage and activate correctly. - activation transaction: its own file APIs and the component-walking ancestry validators now operate in the extended-length namespace; the launcher path is canonicalized (and prefixed when deep) once at entry so every downstream exact-string comparison stays form-consistent. - Executable self-resolution uses the wide APIs (GetModuleFileNameW, GetFileAttributesW) so non-ASCII install paths survive argv[0] resolution. Windows launcher install/uninstall transaction: - FileRenameInfoEx names are NUL-terminated in an over-allocated buffer. FileNameLength governs per the contract, but filter drivers read FileName as NUL-terminated and appended adjacent heap bytes to created names — a flaky, garbage-suffixed rename target. Both the CLI and the launcher rename helpers are fixed. - Uninstall retires state via rename-aside (.cbm -> .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex chars so the bare rename target stays under the FileRenameInfoEx NT-conversion ceiling at guard depths; 64 bits still uniquely identify the generation. - When the running launcher's mapped generation backings pin .cbm against rename, the backings are relocated to activation-<pid>-N .retired tombstones beside the install (a mapped image may be renamed, never deleted; the launcher's liveness-guarded sweep reclaims stale tombstones). Every relocation is recorded, and a FAILED uninstall reverses the moves after restoring .cbm — via MoveFileExW with extended-length paths on both arguments, since the deep generation target is beyond the handle-based rename's bare-path reach — so a restored install keeps its generation backings and stays runnable. - After a committed uninstall the retired tree's backings are relocated out so the tree is shallow enough for the detached cleanup's rd, and the cleanup's working directory strips the extended-length prefix (CreateProcessW lpCurrentDirectory silently ignores prefixed paths). - Files created under Administrators-default-owner directories (CopyFileW destinations, CREATE_NEW tombstones, probe directories) are explicitly owner-stamped so the exact-owner validators hold on runner images; guard fixtures stamp hand-built trees the same way. Diagnostics, tests and infra: - diagnostics: discovery is now an always-delivered JSON control record (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and paths containing spaces; placement honors $TMPDIR with /tmp fallback via a diagnostics-local helper; the soak parser reads the JSON record; documented in docs/CONFIGURATION.md. Red-first coverage for suppressed log levels, TMPDIR-with-spaces, and native Windows output-contract assertions. - tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint parents with production-shaped ancestry (LocalAppData on Windows, via th_secure_runtime_parent_new) — the runtime ancestry validation correctly refuses temp roots whose ancestors grant mutation rights to Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and drive the documented startup-owner publication flow before reading generation-bound endpoint addresses. This turns the 26 Windows failures previously visible in CI's full-test job green without weakening any validation. - tests(win): the launcher guard covers the full permanent-launcher contract including failed-uninstall restore and immediate reinstall after uninstall; new daemon lifecycle and reworked hook-augment guards run the start/recycle/stop flow end to end. - tests: CBM_SKIP_PERF is now actually consumed by the test runner (it was set by CI but never read, so perf suites ran everywhere); four throughput/bench suites are classified as perf, the heavy store_arch suite moved to the slow-timeout tier, and two wall-clock-sensitive assertions were rewritten as invariant checks with coarse hang-detector backstops. - build/test infra: build-dir safety contract, UI dev-proxy security contract, soak daemon-recovery contract, path-safety helper, the Windows VM worktree-sync contract wired into scripts/test.sh, and vm/win.sh guards building its clean embedded-UI product in an isolated BUILD_DIR so it cannot clobber the incremental test build. provision-windows.sh now installs Node.js for the guards UI build. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
83c137d2a5 |
feat: complete shared daemon lifecycle
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
cc1608210d |
docs: document the CBM_ALLOWED_ROOT indexing boundary
Add CBM_ALLOWED_ROOT to the environment-variable tables in README.md and docs/CONFIGURATION.md: when set, index_repository refuses a repo_path that resolves outside it; unset imposes no restriction. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Andrew Barnes
|
e7182cd3fe |
docs(config): add configuration reference for existing config files
Signed-off-by: Andrew Barnes <bortstheboat@gmail.com> |