The private-file prepare treated slow CONTENTION like permanent
OBSTRUCTION: one 250 ms deadline for both. With many concurrent
appenders on a loaded, sanitized runner the last writer can
legitimately wait longer than that behind its peers' brief exclusive
windows, and the concurrent-append guard correctly failed a dropped
conflict event (dry run: one appender of the pack lost). Transient
share/lock violations now retry against a 2000 ms contention budget
while every other error class keeps the 250 ms fast-fail — a rejected
client is still waiting behind this on the hello path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Local runs left most of the machine idle: the serial tail ran sixteen
suites one at a time on otherwise-idle cores, the three platform legs
were launched by hand (usually sequentially), and every container-leg
object write crossed the virtiofs bind mount.
- run-tests-parallel.sh tail scheduling in two phases: the FLEX suites
(timing-shaped but free of the shared per-account daemon runtime
namespace) run CBM_TAIL_JOBS-wide (default 2), then the EXCL group —
daemon-family plus the suites that drive daemon one-shots or
supervisor rendezvous — runs strictly sequentially on a machine
exactly as quiet as the old fully-serial tail gave it. The wave was
already fed longest-first by the shard dealing order, so the drain-out
no longer ends on a heavy straggler.
- ladder.sh: one maintained entry point for the full local push gate
with the legs overlapped — lint, the Linux container suite, and the
Windows VM suite in the background, the macOS suite in the
foreground, one verdict per leg, logs kept per leg. A missing
prerequisite fails its leg loudly instead of silently skipping.
- win.sh test-par now runs through vm-run-tests.sh (--par mode): the
full parallel harness under the CI-shaped protected temp root with
complete output. It previously ran under the MSYS-shared /tmp and
piped through `tail -25` — the same truncated-blindness class that
hid 40 Windows failures from the `test` command.
- docker-compose: build artifacts and the incremental fixture cache
move to named volumes on the container VM's native filesystem. Object
writes over the virtiofs bind mount are the container legs' largest
avoidable I/O cost, and the fixture cache now survives across
container runs.
- test_mem(win): the first valid full-parallel VM run proved working-set
trimming beats the re-touch mitigation (19 MB resident of a 256 MB
double-touch at 18 parallel suites). The RSS probe now VirtualLocks a
64 MB span — locked pages are exempt from trimming, making the
measurement pressure-immune — with bounded touch-and-sample retries
when the lock is unavailable. Red-to-green under the same 18-job load.
- cli: the portable install's staging error now appends the activation
refusal note (predicate, SID, object) like the managed path already
does — a bare "activation transaction I/O failed" on a CI-only
failure is undiagnosable without it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first full Windows x64 CI run of this branch (earlier contract-step
walls had always aborted the leg) surfaced 40 failures: the managed
launcher's fail-closed install/update/uninstall gates and the suite's
POSIX-modeled activation tests were both new in this branch and had
never been reconciled, and they hid three real production bugs.
Production fixes:
- ui: config publishes with POSIX rename semantics. MoveFileExW's legacy
replace cannot supersede a destination a reader holds open (the name
lingers until the last handle closes, even with FILE_SHARE_DELETE), so
a config save racing an open reader failed. The temp file is opened
with DELETE access and renamed in-hand (bare-path target, NUL-padded
rename record) before close, falling back to MoveFileExW on any
rename-by-handle failure so non-NTFS config homes keep the previous
behavior.
- subprocess(win): the child's redirected log handle now shares write
access. POSIX places no lock on the redirected log, so a second
producer may append while the child runs; the missing FILE_SHARE_WRITE
mandatory-locked every other writer out, which the index-supervisor
callback-isolation test legitimately exercises.
- cli(win): the portable uninstall path composes the managed binary name
with its .exe suffix; it could never find (or remove) the installed
binary before.
- activation: security-predicate refusals are now diagnosable. The
transaction's predicates refuse without a usable OS last-error, which
collapsed five distinct refusal classes into one blind "status -3,
os 0" at every caller. Each refusing predicate records what refused —
predicate, untrusted SID, and object path — first-wins so the deepest
cause survives, and the staging error line appends it. This is what
identified every Windows refusal in this batch, including the real
install-failure class on drive-root-rooted payload directories
(Authenticated Users holds inherited Modify there; profile-rooted
directories do not).
Windows test dispatch (release binaries unchanged):
- Test builds (CBM_CLI_ENABLE_TEST_API, the existing test-object macro)
compile the portable activation flow on Windows, reachable solely
through the installed activation test seam and announced loudly on
stderr; release binaries compile exclusively the managed transaction,
so the dispatch polarity fails closed. The suite's install/update/
uninstall calls route through a seam-if-absent wrapper, keeping the
shared activation and agent-config semantics unit-covered on Windows;
POSIX arms keep exercising the real activation machinery.
- Two new release-gate tests assert the fail-closed contract with the
seam OFF: a portable payload may not self-update/self-uninstall, and
install refuses to adopt or replace a foreign file at the canonical
launcher path even under --force.
- Test-bug fixes the first valid Windows runs exposed: the version-
cohort crash test reused one conflict record across the probe and the
turnover acquire (whose entry memset erased the probe's detail before
the assertions read it); the activation staged-path assertion now
checks containment rather than a separator (the \\?\ namespace
performs no forward-slash translation); the ui atomic test captures,
cleans up, then asserts so a failure cannot poison later tests; the
PATH-visibility guard is POSIX-only (Windows configures PATH through
the user registry, not a shell rc).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fixes surfaced while bringing the shared-coordination-daemon branch
green across macOS, the Linux containers, and the real Windows-ARM64 VM,
plus a new stability guard that caught one of them.
- daemon(win): the CLI teardown transition latched a failed release as
permanent, but a Windows participant-state release must briefly
try-hold the shared startup/legacy gates and legitimately collides
with a concurrent one-shot's teardown; the release is retriable by
contract (it always retains the transition), so parallel one-shot
commands no longer report "CLI coordination cleanup failed" despite
succeeding. Found by the new stability guard's churn section.
- cli/main(win): install/update/uninstall and the Augment hook-script
removal derive the managed launcher's identity from its plain
drive-form path (cli_windows_plain_utf8), not the \\?\ extended-length
form, so agent-config ownership matches on Windows and uninstall no
longer leaves the MCP entry or owned hook scripts behind.
- security: the `daemon start --open` browser launch is now shell-free
(ShellExecuteW on Windows, cbm_exec_no_shell elsewhere) instead of
system(); the CORS origin check spells out its two literal loopback
URLs so the static URL audit sees a complete value. Both clear the
Layer-1 allow-list audit.
- coordination: version-cohort lock retries now sleep a per-process
jittered interval — fixed-period retries can phase-lock two
participants so one starves. The activation-quiesce test's
observation window now covers worst-case candidate staging (three
tamper-defense hashes over a ~1 GB sanitizer binary exceed the old
30 s budget on container I/O).
- smoke/soak: the agent-config smoke matches Windows config paths
through their escaped-backslash quoted form; the soak parser skips the
update-available banner before the JSON summary and gained a one-shot
CLI admission-churn phase inside the RSS/FD leak window. The build-dir
safety fixture creates a real symlink/junction (MSYS2 ln -s otherwise
deep-copies) so its traversal-refusal contract is genuinely exercised,
and the symlinked-agent-roots test demotes its planted links to an
unprivileged owner when run as root so the refusal is really tested.
- tests(win): a new daemon-stability guard exercises the parameter
surface, hook fail-open with its rate-limited notice, start-twice and
occupied-port handling, busy-stop refusal against a live MCP session,
kill -9 crash recovery, and sequential/parallel one-shot churn.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.
Daemon lifecycle:
- daemon start/stop/status subcommands. `daemon start` launches a
PERMANENT daemon (spawn shape is byte-exact argv; survives idle
periods and session ends) and reports an already-active daemon
instead of failing. Permanence is honored at every stop latch:
last-committed-client disconnect, host initial-client window,
coordinator release, and application final-session close — a
permanent daemon also keeps admitting new sessions after its last
one closes.
- daemon stop refuses while sessions are active and lists the blocking
peers (pid/role) that must finish first; an idle daemon drains
through the activation-shutdown machinery with the ACK ordered after
connection interrupts. A second stop is idempotent. The wire ops are
no-cohort first-frame requests with peer fingerprint authentication,
so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
keep their local supervised path). A cold CLI run that had to spawn a
temporary daemon prints a hint that `daemon start` removes the
per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
emits a visible, rate-limited notice (Claude-dialect systemMessage
plus stderr for other dialects) and always exits 0 — augmentation is
never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
older-build permanent daemon (strict semantic-version triples only;
dev builds never auto-drain) and the build-conflict message names
`cbm daemon stop` as the manual escape hatch.
Windows IPC/runtime (real-VM verified):
- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
path used to destroy its listening pipe instance on every 20 ms poll
timeout; a client attaching in the teardown window was severed or left
on an orphaned pipe object whose HELLO no server handle could ever
read, absorbing the connect until the client's own timeout expired.
The pending connect now survives poll timeouts and nothing is
destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
server handle can discard a just-sent response before the peer reads
it (POSIX stream sockets never lose buffered data on close). A bounded
cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
immediate on interrupted connections) now precedes close in
runtime_worker_finish and runtime_reject_inline, so hello-conflict,
capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
transport half-close, so close_begin now announces departure with an
explicit frame (ordered after APPLICATION_CANCEL, before the local
interrupt); the server releases the client's admission on receipt
instead of waiting for the handle to close. Admission-drop timing is
now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
request publication under the send lock, best-effort sends the active
token's APPLICATION_CANCEL frame, then interrupts local I/O; the
server cancels MCP/subprocess work promptly. Contract tests accept
both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
the requester's license to act on "snapshotted and draining", so every
connection interrupt is now initiated before the ACK is sent; a
session could previously get one more request serviced after the
requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
timer granularity) burned ~1.6 s against permanently obstructed paths,
stalling hello rejections past the client's timeout. The retry budget
is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
nonblocking send() on Windows is absorbed wholesale into AFD kernel
buffering regardless of SO_SNDBUF, so send deadlines and interrupts
could never engage against a slow peer (and the full payload was
pinned in nonpaged pool). Slicing restores a deterministic
backpressure point; a test hook pins SO_SNDBUF for the deadline and
interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
deadline, explicit connection states, refusal to free a server while
a listener-owned connection is active.
Windows long-path support:
- Central path-aware wide conversion (canonicalize via GetFullPathNameW
and prepend the extended-length prefix for absolute paths >=240) at
the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
store opens, and the daemon build-fingerprint/log paths. Deep managed
installs (a 64-hex generation directory routinely exceeds MAX_PATH)
now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
ancestry validators now operate in the extended-length namespace;
the launcher path is canonicalized (and prefixed when deep) once at
entry so every downstream exact-string comparison stays
form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
GetFileAttributesW) so non-ASCII install paths survive argv[0]
resolution.
Windows launcher install/uninstall transaction:
- FileRenameInfoEx names are NUL-terminated in an over-allocated
buffer. FileNameLength governs per the contract, but filter drivers
read FileName as NUL-terminated and appended adjacent heap bytes to
created names — a flaky, garbage-suffixed rename target. Both the CLI
and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
.cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
chars so the bare rename target stays under the FileRenameInfoEx
NT-conversion ceiling at guard depths; 64 bits still uniquely
identify the generation.
- When the running launcher's mapped generation backings pin .cbm
against rename, the backings are relocated to activation-<pid>-N
.retired tombstones beside the install (a mapped image may be renamed,
never deleted; the launcher's liveness-guarded sweep reclaims stale
tombstones). Every relocation is recorded, and a FAILED uninstall
reverses the moves after restoring .cbm — via MoveFileExW with
extended-length paths on both arguments, since the deep generation
target is beyond the handle-based rename's bare-path reach — so a
restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
out so the tree is shallow enough for the detached cleanup's rd, and
the cleanup's working directory strips the extended-length prefix
(CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
(CopyFileW destinations, CREATE_NEW tombstones, probe directories)
are explicitly owner-stamped so the exact-owner validators hold on
runner images; guard fixtures stamp hand-built trees the same way.
Diagnostics, tests and infra:
- diagnostics: discovery is now an always-delivered JSON control record
(new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
paths containing spaces; placement honors $TMPDIR with /tmp fallback
via a diagnostics-local helper; the soak parser reads the JSON record;
documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
log levels, TMPDIR-with-spaces, and native Windows output-contract
assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
parents with production-shaped ancestry (LocalAppData on Windows, via
th_secure_runtime_parent_new) — the runtime ancestry validation
correctly refuses temp roots whose ancestors grant mutation rights to
Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
drive the documented startup-owner publication flow before reading
generation-bound endpoint addresses. This turns the 26 Windows
failures previously visible in CI's full-test job green without
weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
contract including failed-uninstall restore and immediate reinstall
after uninstall; new daemon lifecycle and reworked hook-augment
guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
(it was set by CI but never read, so perf suites ran everywhere);
four throughput/bench suites are classified as perf, the heavy
store_arch suite moved to the slow-timeout tier, and two
wall-clock-sensitive assertions were rewritten as invariant checks
with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
contract, soak daemon-recovery contract, path-safety helper, the
Windows VM worktree-sync contract wired into scripts/test.sh, and
vm/win.sh guards building its clean embedded-UI product in an
isolated BUILD_DIR so it cannot clobber the incremental test build.
provision-windows.sh now installs Node.js for the guards UI build.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cli_windows_staging_root builds the per-user install staging root from
LOCALAPPDATA/USERPROFILE, but msys and CI harnesses export those in POSIX
drive form ("/c/Users/..."), which the CRT cannot resolve, and the target's
AppData\Local ancestors may not exist yet on a fake-profile fixture. Rewrite
a leading "/c/" to native "C:/" before use, create the whole ancestor chain
(a single mkdir left the root missing), and fall back to the temp root only
after verifying the directory exists. Also name the failing sub-step when
private staging cannot even create its directory, so the opaque "did not
remain runnable" message is no longer the only signal.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Back-to-back requests on one frontend<->daemon connection failed roughly
half the time on loaded Windows hosts, surfacing as wandering MCP session
drops (smoke Phase 5/5e/5f "no tools/list response", "timed out waiting for
id=2"). Root cause, confirmed on real Windows: the application worker stored
its done flag AFTER writing the response, so a client that pipelines its
next request the instant it sees a response raced the previous worker's
final instructions and its join, and admission rejected the new request
with BUSY. On EOF that BUSY was then classified as an expected shutdown and
the response was silently dropped with a success exit; mid-session it exited
non-zero. A timing-sensitive heisenbug — any added stderr shifted the window
and it passed.
- runtime_application_worker publishes the done flag BEFORE sending the
response, so completion is one atomic transition from the client's view:
by the time any client can react, the slot already reads reusable. The
admission side joins the thread after observing the flag, which absorbs
the send tail. No timing constants.
- frontend_should_stop's input-closed leg now also requires no request
in flight, so a daemon-side failure of the final post-EOF item is loud,
never swallowed as an expected stop.
- The EOF drain is now progress-aware: it resets its window on each
completed item, bounding stuckness rather than total batch length, so a
slow cold daemon spawn no longer abandons queued tail responses.
- Latch the never-connected shutdown window on a monotonic admitted-total
counter instead of sampling the live client count, which can read zero
between two short-lived sessions and stop a daemon mid-conversation.
- Add structured lifecycle diagnostics (daemon.runtime_stopping reason=,
lifetime_end reason=, connection_end) so this class is observable in the
logs instead of being inferred.
Validated: real-Windows loop 12/12 (was ~50% loss), macOS daemon suites
143/143.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The activation transaction and the release-descriptor probe validate the
full ancestor chain of everything they stage: every path component must
have a trusted owner and grant mutation rights only to trusted identities.
That walk is deliberate TOCTOU defense and stays strict. The staging
directory, however, came from the process temp root, which routinely lives
in a tree that legitimately fails the walk through no fault of ours: under
msys2 TMP resolves inside the msys install tree (C:\msys64\tmp), and on
GitHub runners inside the runner work directory, both of which carry broad
inherited grants such as Authenticated Users modify on an upper component.
The staged pair then failed validation ("launcher descriptor candidate is
unsafe" / the smoke Phase 8 --version probe refusal) even though the staged
files themselves were private.
Add cli_windows_staging_root(): stage install and update transactions under
%LOCALAPPDATA%\codebase-memory-mcp, whose ancestor chain is user-owned end
to end, so the strict walk passes by construction instead of by environment
luck. Falls back to USERPROFILE\AppData\Local and finally the process temp
root when the environment is degraded.
Also make the staging observable where it previously failed blind:
cli_windows_stage_private_file reports which transaction step failed with
the OS error code, and cbm_mkdtemp warns once if the private security
descriptor cannot be applied and it falls back to default directory
security. A blind boolean here cost several blind CI round-trips.
win.sh: scp targets need native C:/ paths on Windows OpenSSH.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_mkdtemp on Windows created the directory with plain _mkdir, so it took
the token's default owner and the parent's inheritable DACL. Under an
Administrators-default-owner policy -- standard on Windows Server and
GitHub's elevated runners -- the temp directory was born owned by
BUILTIN\Administrators with foreign inherited grants. The activation
transaction validates its staging directory with a strict owner-is-current-
user check plus a trusted-only DACL scan (activation_directory_secure), so
install staging failed at the directory gate before any file was staged:
"private staging failed for the launcher" in the Phase 8 smoke, previously
masked as "did not remain runnable after private staging".
Create the directory with CreateDirectoryW and an explicit descriptor:
owner stamped to the token user and a protected, inheritable, user-only
DACL -- private by construction, the same policy the daemon applies to its
runtime directory. If the descriptor cannot be built the code falls back
to plain _mkdir so degraded environments (Wine) keep working; downstream
validation still gates security there.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Phase 8 staging diagnostic added two long lines that clang-format-20
(the CI formatter) wraps differently. Reformat them so lint passes; no
behavior change.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
activation_windows_security_init built the creation security descriptor with
a protected current-user DACL but never set an owner, so a file created with
these attributes inherited the token's default owner. Under an
Administrators-default-owner policy -- standard on Windows Server and on
GitHub's elevated Windows runners -- that default is BUILTIN\Administrators,
not the calling user. Every staged file is then validated with the strict
activation_windows_owner_is_current check, which rejected the
Administrators-owned file and made private staging fail with "launcher/payload
source pair did not remain runnable after private staging" (smoke Phase 8).
Stamp the owner explicitly to the token user in the creation descriptor, the
same fix the daemon side already applies. This keeps the strict owner check
intact -- the staged file genuinely becomes current-user-owned -- so it is not
a validation relaxation; setting the owner to our own token user needs no
extra privilege. Complements the launcher's trusted-owner acceptance, which
covers the installed binary that a non-admin user cannot re-stamp.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cli_windows_prepare_install_pair folded launcher staging, payload staging,
the payload fingerprint match, and the staged-launcher --version probe into
one bool, so the caller could only report "launcher/payload source pair did
not remain runnable after private staging" without saying which sub-step
failed. Windows cbm_exec_no_shell does not inherit the child's stderr, so the
launcher's own refusal reason never reaches the log either. Split the checks
so each failing sub-step prints a distinct error, turning the opaque Phase 8
smoke failure into a precise signal on the next run.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The launcher validated its own binary, staged payload, and their
directories with a strict "owner must be the exact token user" check
(launcher_security_is_private -> launcher_security_is_safe with
require_current_owner=true). On an elevated or managed install -- Program
Files, an admin-run installer, and GitHub's Windows runners -- the default
owner of newly created objects is BUILTIN\Administrators, not the calling
user, so the launcher rejected its own legitimately privileged-owned files
with "launcher ownership or access policy is unsafe [owner is not the
current user: S-1-5-32-544]" and refused to start.
Widen the owner check to the trusted-owner set that launcher_sid_is_trusted
already defines (self, SYSTEM, Administrators, TrustedInstaller) -- the same
set the daemon side applies for the identical default-owner reason. DACL
strictness is unchanged: every allow ACE must still grant only a trusted
identity, so tamper resistance is preserved; only the owner field is widened
to the privileged identities that manage the install. A less-privileged
attacker cannot stamp any of those owners without already holding that
privilege, so no real attack surface is added.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows managed-install path (cli_windows_managed_install_activate)
printed its plan as "Would install managed launcher -> ..." but never
the words "dry-run", unlike the POSIX path's "(dry-run — no files were
modified)" - so `install --dry-run` did not clearly indicate dry-run
mode (smoke Phase 6a, confirmed by echoing the captured output).
Append the same dry-run indicator to the Windows plan line.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The guards cache-dir rejection, once its SID was printed, was the
OWNER RIGHTS well-known SID S-1-3-4: an ACE that modulates the rights
granted to whoever OWNS the object. Both the daemon
(win_bounded_sid_trusted) and the launcher (launcher_bounded_ace_sid_
is_trusted) validate the owner separately as the exact current user, so
an OWNER RIGHTS ACE can only ever grant to us - it is safe. Default
Windows profile/temp ACLs (and GitHub runner profiles) carry it, and
rejecting it locked the daemon and launcher out of legitimate
current-user directories (three green guards went red: non-ascii path,
non-ascii cache dump, windows launcher). Accept WinCreatorOwnerRightsSid
in both bounded-ACE checks. Real-user Windows fix, not just CI.
mingw-cross-compiled clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The guards cache-dir rejection classified the offending ACE as an
untrusted "other" identity - a specific account, not a well-known
group - which a class label alone can't pin down. Print the SID in
string form (ConvertSidToStringSidW) so the exact account behind the
harness/profile ACL leak is named. mingw-verified.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A dry-run mutates nothing (every install mutation is guarded by
!dry_run), but on Windows the activation plan's PATH/agent-config probe
can return non-OK, and the caller then returned early WITHOUT the
dry-run summary - so `install --dry-run` neither indicated dry-run mode
nor exited success (smoke Phase 6a). For a dry-run, emit the
"(dry-run — no files were modified)" indicator and return CLI_OK on
that path (the plan simply noted something), and log the underlying
status to stderr for triage. POSIX is unaffected (its dry-run
activation returns OK, so the branch is never taken).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Replaces the earlier read-before-impersonate approach, which fixed the
connection but changed accept's contract (accept blocked on the first
frame read, breaking the daemon_ipc/daemon_runtime unit tests that
expect accept to return immediately). Validate the client via
GetNamedPipeClientProcessId + its process token - the exact mirror of
the existing client-side server check (GetNamedPipeServerProcessId) -
which needs neither impersonation nor a prior read, so accept keeps its
immediate-return contract while the ERROR_CANNOT_IMPERSONATE problem
disappears entirely. The prime buffer, first-frame timeout, and the now
-unused win_token_is_current_user are removed. mingw-cross-compiled
clean; POSIX daemon_ipc/daemon_runtime green locally.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every spawned worker/daemon re-hashes the entire executable image for
the exact-build fingerprint; for the ASan test-runner (hundreds of MB)
that is tens of seconds per spawn on constrained CI runners and was the
sole cause of the whole daemon-family readiness-timeout flake class
(index_supervisor, daemon_frontend, cli, and much of the Windows
daemon surface). Add a test-only seam under CBM_CLI_ENABLE_TEST_API:
when CBM_TEST_BUILD_FINGERPRINT holds a valid stub the hash is skipped.
test_main.c sets it once; every forked child and re-exec'd worker
inherits it, so exact-build match AND mismatch behaviour is preserved
(a mismatch test still passes a different fingerprint via argv). Never
compiled into production. Local mac run: index_supervisor 6/6, the four
daemon suites 111/0, exact-build test green. The generous frontend
catastrophic-alarm values are kept as backstops.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The cache-private diagnostic pinned the guards failure to a private-dir
DACL carrying a mutation ACE for an untrusted identity, but not WHICH -
needed to tell a harness/profile ACL leak (an inherited Users /
Authenticated Users / Everyone group ACE on the CI runner's admin
profile) from a genuinely hostile grant. Classify the offending ACE's
SID (BUILTIN\Users / Authenticated Users / Everyone / INTERACTIVE /
other) in the message. Windows-only; mingw-verified.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
With the Windows client-connection cluster fixed, the smoke's rapid
session churn exposed the next layer: each phase starts a fresh
first-session daemon, and a just-superseded generation still holds the
cohort daemon-claim EX lock for the brief window between going
client-terminal (which lets the next generation start) and releasing
its cohort locks. claim_acquire was a single non-blocking attempt, so
the new daemon failed immediately with daemon.start_failed
component=claim and the client saw "daemon could not start within
30000 ms".
The host now retries the claim on BUSY for a bounded 12 s handoff
window (well under the client's 30 s budget); a persistent BUSY still
fails as a real conflict. Cross-platform path, POSIX-syntax verified.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The prior fix waited on PeekNamedPipe availability, but that is
insufficient: ImpersonateNamedPipeClient requires a COMPLETED read on
the pipe, not merely buffered data - so impersonation still failed with
ERROR_CANNOT_IMPERSONATE (confirmed on CI: first_frame=0, impersonate
error=1368 x116, on a freshly-compiled prod payload).
The accept path now performs a real blocking read of the client's first
frame header into a per-connection prime buffer (the client sends its
HELLO immediately after connecting, so this never deadlocks and is
bounded by a finite deadline), which satisfies the impersonation
precondition. connection_read_full drains the prime buffer before
touching the pipe again, so the runtime's frame parser still sees an
intact HELLO. Identity policy is unchanged - it just runs once the
client is impersonatable. Cross-compiled clean with the mingw Windows
toolchain.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows crash-holder test reads a peer process's lifetime record
via a shared lock and gets an INVALID identity (conflict.status 0, not
the expected VERSION_CONFLICT), while the unix flock path passes. Split
the read-vs-decode outcome and log it: payload_status distinguishes a
cross-process payload that could not be read at all from one read but
malformed, and bytes gives its length. That names the next test-windows
round's cause - a Windows lock-payload cross-process visibility question
the unix path does not hit - instead of a bare INVALID.
Diagnostic only on the already-failing path; behaviour unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Root cause of the Windows client-rejection loop, named by the accept
classifier: ImpersonateNamedPipeClient fails with ERROR_CANNOT_IMPERSONATE
(1368) - "unable to impersonate using a named pipe until data has been
read from that pipe" - because the daemon authenticated the connecting
client the instant ConnectNamedPipe returned, before the client had put
any bytes on the wire. Every client was refused and the accept loop
spun (116 rejections in one smoke window).
The client always sends its HELLO request immediately after connecting
and only then waits for the response (cbm_daemon_runtime_client_connect:
ipc_connect -> send_frame -> receive), so the server can safely wait for
the client's first bytes before impersonating. win_pipe_wait_for_client_data
polls PeekNamedPipe - which inspects the buffer WITHOUT consuming it, so
the HELLO frame stays intact for the runtime's own read - and gates only
on data PRESENCE, the exact precondition the impersonation API requires.
Bounded by a finite deadline so a connect-but-never-send peer is refused,
not hung. The identity check itself is unchanged; it now simply runs
after the client is impersonatable.
Windows-only path; the unix matrix (green at 6742/0 on this tree) is
unaffected.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The instrumented smoke run pinned it: the daemon rejects the connecting
client's identity 242 times in one window (daemon.accept.client_rejected
stage=client_identity), and the client's error 233 (ERROR_PIPE_NOT_
CONNECTED) is only the downstream symptom of the server disconnecting
it. The rejection lives in win_pipe_client_is_current_user but its
single false covered three very different causes. Split them:
impersonate failure (ERROR_CANNOT_IMPERSONATE = client wrote no frame
yet / accept fired on an empty pipe), thread-token open failure, and a
genuine SID mismatch (now naming the impersonated client's SID class -
anonymous/system/administrators/other). The 242-in-a-loop count points
at an accept-spin on client-less connections; the next run's step label
confirms which.
Windows-only diagnostic path; unix gate is unaffected and green.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The classified smoke run pinned the Windows client loop on
GetNamedPipeServerProcessId failing against a successfully opened pipe
(step=server_pid_query, 12 hits in one 30 s window). Two data points
were still missing: the Win32 error of that exact call (and whether the
function pointer resolved at all), and the server-side view - the
daemon's accept path disconnects unverifiable clients silently, which a
client experiences as exactly this failure. Both now log.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The cache-private identity failure printed a bare status name while the
actual rule that refused - POSIX owner/mode/extended-ACL steps, Windows
owner class, mutation-capable DACL entries, ancestry components, the
owner/DACL repair - stayed mute, costing a blind CI round-trip per
guess. Both platform gates now record the object and rule at the exact
failing check (cbm_daemon_ipc_validation_detail), main.c appends it to
the identity message, and the Windows client's server-identity check
logs its two previously-silent early exits (server PID query, process
open - a dead server PID that Windows reaped or reused).
Validation policy is unchanged everywhere; refusals only stop being
anonymous.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The new pipe_rejected diagnostic proved the Windows client reaches the
daemon's pipe and then refuses the SERVER identity for its whole 30 s
window, while earlier sessions in the same run connect fine. The check
compares the server-process token user, so the interesting failures are
environmental: a dead server PID already reused by Windows (typically
for a SYSTEM service) is indistinguishable from a hostile pipe without
classification. The rejection now names the failing step (pid query,
process open, token open/query) or the server SID class
(system/admins/other) plus the server PID.
Also reflows the two owner-stamping call sites the lint gate flagged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Stamping the owner at creation covers objects born through the daemon's
own security descriptors, but the cache root is created by plain
mkdir_p before the gate runs, and harness- or user-created directories
arrive the same way. On an Administrators-default-owner token those
directories are born owned by BUILTIN\Administrators and the exact-user
gate refused them permanently - including a real elevated admin's own
cache directory.
win_runtime_directory_secure (the final component of both the runtime
dir and the cache-root validation) now treats a TRUSTED owner - the
same SYSTEM/Administrators/TrustedInstaller set the launcher already
accepts for directories - as repairable: the owner is re-stamped to the
exact token user inside the same set_security_info call that already
re-protects the DACL. Any other owner is refused exactly as before, the
post-repair validation still demands the exact user, and the handle
falls back to the old access mask when WRITE_OWNER is unavailable so no
previously-passing configuration can newly fail.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows SECURITY_ATTRIBUTES used for every private-namespace
creation (runtime dirs, rendezvous/startup/participant/lifetime/cohort
lock files, log files, pipes) carried a DACL but no owner, so created
objects took the token DEFAULT owner. Admin-group tokens can default
that to BUILTIN\Administrators - standard policy on Windows Server,
including CI runners - while every strict validation gate
(win_file_owner_secure(require_current_user=true),
private_win_owner_only_dacl) demands the exact token-user SID. Result:
the daemon rejected objects it had just created itself; the client
refused the daemon's rendezvous record, retried silently for its full
30 s window, and each clientless generation clean-stopped, looping
until the frontend gave up. The legacy kernel mutex was the one object
that already force-set its owner post-create - it is the template this
change generalizes.
Both descriptor builders now set the owner explicitly at creation, so
objects are born satisfying the same strict gates on every default-
owner policy. The gates themselves are unchanged.
The Windows client connect path also stops failing mute: terminal
rendezvous refusals, deadline expiry (with last status), and pipe
server-identity rejections now log their reason once, matching the
launcher-side refusal diagnostics.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The launcher answered every ownership/ACL problem with one generic
"launcher ownership or access policy is unsafe", which made the CI
guard failures (and any field report) undiagnosable: the same message
covers an untrusted owner SID, a mutation-capable ACE, a reparse point,
a hard-link clone, and a plain open failure across the whole ancestry
walk. Record the failing check at its site - offending path, owner or
ACE SID in SDDL string form, rights mask, Win32 error - and print it
once alongside the refusal. No policy changes; refusals stay refusals.
The interactive MCP smoke now also dumps daemon-conflicts.ndjson next
to the daemon log tail on failure, so an admission conflict names
itself instead of surfacing as a bare 30 s client timeout.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The PR CI caught two consumers the format migration missed — both parsed
tool output by its old shape and failed silently or with a wrong count:
- hook-augment built its PreToolUse additionalContext by reading the
legacy 'results' object-array from search_graph format:json; after the
json-tree reshape that key no longer exists, so the hook emitted nothing
(caught end-to-end by tests/windows/test_hook_augment.py). The parser
now walks groups -> qn_prefix/file + column-ordered rows. A new local
test feeds the parser the LIVE server envelope, so any future drift
between response shape and hook parser fails in the normal suite on
every platform — not only in the Windows CI guard. Reproduced RED
(ctx NULL, the exact CI failure) before the fix, GREEN after.
- smoke-test.sh carried six old-shape parsers (callers[N], rows[N]{,
clusters[N], semantic[N]{, and the assert_toon_table helper); the tree
headers never matched, so trace verification read 0 callers. All six
updated to the tree contract; the full smoke script passes end-to-end
locally.
cli suite 222 passed; lint clean; scripted smoke ALL PASSED.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fresh review pass over the whole branch; three classes of fix:
Cursor safety — the 'legacy' generation gap:
A pre-migration DB (indexed by an older binary, opened read-only) has no
store_meta, so its generation reads 'legacy' — and 'legacy' == 'legacy'
across rebuilds, which silently defeats stale-cursor detection for exactly
those DBs: a stale (hop,id) watermark would resume on wrong node ids.
Cursors are now neither minted (truncated + raise-limit hint instead) nor
accepted (cursor_unsupported teaching error) under a legacy generation.
The standard flow was verified safe empirically: every index run seeds
store_meta via cbm_store_upsert_project in the flush path, and a rebuilt
DB file gets a fresh random db_uid, so aliasing is impossible there. The
guard is unreachable through the public API (upsert always seeds) — it
protects only genuinely old DBs, hence no synthetic test.
format:'json' now emits the tree model on EVERY tool (the last five):
- BM25 query mode: {total, cols, rows[[qn,label,file,lines,rank]]} —
rank order preserved, ranked output is never regrouped.
- semantic: {semantic: {cols, rows[[qn,label,file,score]]}}.
- search_code compact/full: {cols, rows} with nested match-line arrays;
full appends a per-row source object (same guarded, windowed reader);
raw_matches likewise {cols, rows}. files mode stays a plain list.
- get_architecture: all twelve sections emit {cols, rows} via one shared
section helper; cluster row cells carry their nested lists as arrays.
- include_connected became a real column in BOTH encodings
(';'-joined cell in the text tree, nested name array in json) — the
include-connected-forces-JSON coupling is gone, and the last legacy
per-node-object emitters (emit_search_results, enrich_node_properties)
are deleted.
Teaching-error parity: detect_changes now rejects an unknown
with the valid values (it silently fell back to inbound; trace_path already
errored — the caller would have misread the result's semantics).
Full suite 6,373 passed / 0 failed; lint clean. Live-verified on the
self-index: bm25/search_code/architecture json shapes, the connected
column, and cursor pagination.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fresh-eyes review of the phase-2/3 work; five defects, each against our own
stated principles (no silent caps, exact counts, same model in both
encodings):
- trace callees_total/callers_total counted test-file rows even when
include_tests=false hides them from the table — the scalar disagreed with
the enumerable set (a field-eval agent read callers_total=175 against a
handful of visible rows and stopped trusting the tool). Totals now apply
the same filter as the emitters; description documents the transitive
(within-depth) semantics. Guard test: filtered=1 vs include_tests=2.
- cycles clamped the SCC list at 100 silently, losing the true count.
cycles_total is now always exact and cycles_omitted is emitted when the
display clamp engages (both encodings).
- detect_changes accepted a 'limit' param (referenced by its own omitted-
hint) that the schema never declared — undiscoverable. Declared.
- detect_changes format:json lacked the impacted_modules rollup the tree
encoding carries — model parity restored via a shared rollup helper.
- tree search extra-field cells were appended raw: a signature with spaces
shifted every following column. Cells now route through the shared
emitters (space-triggered quoting, '-' placeholder); the properties test
pins the quoted form.
Full suite 6,373 passed / 0 failed; lint clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Phase 3 of the graph-analysis overhaul. Two new capabilities on a shared
multi-source traversal substrate, plus a context-bomb guard.
detect_changes — rebuilt into real blast-radius analysis:
The old handler parsed and echoed depth but never traversed: impacted was
just the symbols DEFINED in changed files. It now resolves the git diff to
those symbols, then runs ONE multi-source BFS to the transitive impact set.
- default direction inbound = the blast radius (transitive callers that may
need review); outbound/both available.
- seeds are excluded from the result, and a changed file reached from
another changed file is not counted as extra impact (that is not
downstream fallout).
- merge_base SHA pins the exact diff base for reproducibility.
- impacted_modules: a 2-segment (src/mcp, tests/...) rollup — a quotient
view of where the radius lands, always complete (overflow bucketed, never
silently dropped). Per-symbol rows cap at limit (default 200, nearest
hops first) with impacted_total exact and impacted_omitted surfaced.
On a 7-file core diff: 1469 impacted symbols, 64KB unclipped -> 10.7KB with
the rollup + cap. format:json returns the same model as structured JSON.
cbm_store_bfs_multi (store): one recursive CTE anchored on ALL seeds via a
temp table — never seed_count separate walks over overlapping hubs.
Seed-exclusion NOT IN, MIN(hop) across the set, canonical (hop,id) order,
a memory-safety ceiling that reports *truncated instead of silently
capping. Reproduce-first: seed-exclusion + min-hop, and truncation.
get_architecture aspects:[cycles] — SCC condensation:
Iterative Tarjan (heap stacks, no recursion overflow) over the CALLS graph
reports circular dependencies (strongly-connected components of size > 1).
cbm_store_fetch_call_edges feeds it with a 400k-edge scan budget that
reports partial coverage rather than truncating silently. Opt-in ONLY —
never via a default or 'all' call, since it scans the whole call graph.
On this repo it surfaces 26 real cycles (e.g. the C-LSP type-evaluation
recursion) in ~5KB. Reproduce-first: a 3-node cycle is found, an acyclic
chain is not, and a default call does not run the scan.
get_code_snippet: cap the returned line span at 500 (a structural Module/File
node spans its whole file — an unclipped read returned ~400KB in one call).
source_clipped flags it; start_line/end_line still allow a targeted re-read.
Full suite 6,372 passed / 0 failed; lint clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- cbm_toon_* renamed to cbm_tree_* (the emitters have emitted tree syntax
since the format flip; the name now matches).
- format enums reduced to [tree, json], default tree; dead tree/toon flags
removed; tool RESPONSE descriptions rewritten for the grouped-tree
contract (group prefix printed once, full qn = prefix + '.' + name).
- format:'json' is now the json-stringified tree: the SAME grouped model
serialized as JSON — {total, cols, groups:[{qn_prefix, file,
rows:[[column-ordered cells]]}], has_more} for search, per-leg
{cols, groups} for trace (risk/args columns follow the flags),
query_graph's columns+rows shape already complied. Per-row key envelopes
and the legacy property blob are gone (blob-never-spills pinned by test);
requested add columns in both encodings.
- include_connected keeps the per-node-object shape (nested neighbors) —
the one remaining legacy consumer, conversion tracked.
- legacy verbose emitters deleted (bfs_to_json_array).
Full suite 6,367 passed / 0 failed; lint clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Round-2 overhaul of the tool-output surface, built eval-first (byte harness +
no-context field-eval agents, always vs the v0.9.0 release binary):
Format — tree syntax everywhere:
- compact_out emits tree tables: 'key: N (cols: a b c)' headers, indented
space-delimited rows, '-' placeholders, space-aware quoting. Every table
in every tool converts mechanically.
- Grouped prefix-factored output is the DEFAULT for search_graph and
trace_path: rows grouped under their shared (qn-prefix, file) printed
once, row qn = group prefix + '.' + name (stated in-band). Measured:
192-hit search 2,929B (was 4,811 TOON / 238,466 v0.9.0), 82-row trace
2,256B (was 4,234 / 8,094). Field eval: 5/5 answers correct, QN
reconstruction verified, no comprehension loss.
- detail:'ids' tier: bare-qn enumeration for wide sweeps (-44%).
Pagination — exactly-once, stateless:
- store_meta generation (per-DB random uid + mutation counter bumped in
cbm_store_upsert_project — every index run's choke point): cursors can
never silently validate against a rebuilt DB.
- trace cursors: opaque token (leg, generation, params-hash, (hop,id)
watermark); pages drain callees then callers with exact
callees_total/callers_total on every page; teaching errors for
params-mismatch and staleness. Guard: 12 callees paged 5+5+2, every row
on exactly one page.
- deterministic ordering: BFS ORDER BY hop,id; canonical (hop,id) sort +
min-hop dedup across same-name seeds (hop fed risk_labels wrong before);
search (name,id)/(rank,id) tie-breaks make offset pages stable.
Lean defaults + teaching errors:
- get_graph_schema no longer advertises blocked internal fields (fp/sp/bt).
- index_repository coverage lists: counts + 5 examples + one-line pointer
notes (full record was always in the logfile); 6.0KB -> 1.5KB here,
53KB observed on a large repo. index_status gates the git context block
behind verbose:true.
- trace limit param (<=5000) + truncated:true — never a silent truncation;
invalid direction now errors with the valid values; requesting core
columns via fields hints instead of emitting empty cells; cypher
resolves f.file/f.qn/f.path aliases.
Trace-correctness fixes (reproduce-first, RED verified):
- bfs_collect_edges 4KB id-string overflow: past ~1000 visited nodes the
edge SQL failed to prepare and EVERY trace edge silently vanished; now a
per-connection temp-table join (star guard: 1200/1200 edges).
- bfs_union_same_name: min-hop across seeds + edge dedup.
Guards: gating tool-output regression suite (context-explosion detector:
blocked-field leaks, repeated-key envelopes, oversized notes + calibrated
byte ceilings + semantic floors). Full suite 6,367 passed / 0 failed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel