trending/codebase-memory-mcp
1
0
Fork 0
mirror of https://github.com/DeusData/codebase-memory-mcp.git synced 2026-11-04 08:13:37 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,679 Commits 624 Branches 56 Tags
7fedaba331d8fbd241582f3ceeccd70af0ade149
Commit Graph
8 Commits
Author SHA1 Message Date
OhOkThisIsFineandClaude Fable 5 fc1b1ee74e fix(daemon): contain zombie generations from abandoned requests, name mute endpoint holders 
Fixes the 2026-08-29 daemon zombie class: a client disconnected while its
application request was in flight, the request never observed cancellation,
and runtime_worker_finish joined the request thread with no deadline. The
disconnect path wedged forever, the generation stayed formally RUNNING, and
a dead process held the endpoint pipes and UI port for nine hours while
every new client timed out with no diagnostic.

Four changes:

- runtime: the disconnect-path reap of an in-flight application request now
  waits at most RUNTIME_ABANDONED_REQUEST_JOIN_TIMEOUT_MS (30 s), then logs
  daemon.application_request_unresponsive (peer pid, request token) and
  fail-stops the process. The kernel releases every native claim and the
  next client starts a fresh generation. Test seams expose the ceiling and
  replace the terminal stop with a recordable hook.
- diagnostics: a transport connect that reaches a live process but gets no
  valid answer now resolves the holder's kernel pid
  (GetNamedPipeServerProcessId / SO_PEERCRED). `daemon start` classifies a
  mute holder as RESERVED (no doomed competitor spawns) and names the pid
  in its timeout message; `daemon status` prints "not responding (endpoint
  held by pid N)" with recovery guidance instead of "not running".
- frontend: the maintenance monitor's quiet-state poll drops from 10 ms to
  250 ms. Each probe revalidates the whole ancestor directory chain on
  Windows; at 100 Hz that burned ~20% of a core for the life of every MCP
  session (7762 CPU-seconds on one 9.5 h session). Detection latency spends
  0.25 s of the 15 s activation drain budget.
- version_cohort: the healthy fresh-claim record is renamed from
  "claimed_unheld" (misread as a stale-claim anomaly during the incident
  diagnosis) to "claimed_fresh" with prior_holder=none.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: OhOkThisIsFine <102485413+OhOkThisIsFine@users.noreply.github.com>
 2026-08-29 14:44:28 -07:00
Martin Vogelandaoright 7fe2f87ba7 chore(headers): sync stale decl names, name genuine buffer sizes 
Distilled from #1298 by @aoright — the subset that survived review and
main's drift, per the 2026-08-17 ruling.

- pass_lsp_cross.h: the header still documented and declared
  all_defs/imp_keys/imp_vals while the implementation renamed them to
  defs/imp_names/imp_qns — doc-only drift, now synced; plus the braced
  guard in cbm_pxc_registry_for_lang.
- runtime.h: semantic_version[12] gains a named size
  (CBM_DAEMON_SEMVER_SIZE), the "defensible: a real size" case from the
  review.
- httpd.h: request-field buffers take the CBM_SZ_* names for their actual
  sizes; the constants.h include sits in the top include block rather than
  mid-file.

Deliberately NOT carried, per the same ruling: the store.c calloc hunk
(the claimed uninitialized read does not exist), the generic-constant
renames (semantic laundering), the watcher.c split (its target function
gained a semantic fix on 2026-08-20 and the refactor must be re-derived),
and the lsp_resolve.h extractions (their target loop was rebuilt on main).

Co-authored-by: aoright <102943475+aoright@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-08-28 10:29:24 +02:00
Martin Vogel 8eabe191d2 fix(install,daemon): unbreak npx clients, group-writable homes, and legacy updaters 
Five field reports in the 24 hours after v0.10.0 all pointed at the same thing:
gates that were right in principle refused real, ordinary setups, and then
failed to say why. Per the consolidated strictness decision, each gate keeps the
protection that matters and drops the part that was refusing legitimate users —
and every refusal now names what it refused and how to proceed.

**Daemon image gate: npx and every ephemeral install path (#1539, #1383).**
The admission check treated "the peer's image hashes differently" and "the
peer's image cannot be examined at all" as one failure. The second is what
`npx codebase-memory-mcp` always produces (ephemeral cache path,
unfingerprintable), so every npx-invoked client was rejected — and, because the
client never reported it, agents saw a transport that closed mid-handshake with
zero bytes on stdout. Reported by @wassolles with the admission path already
read and the fix space mapped.

An unverifiable image is now admitted: the rendezvous HELLO immediately above it
has already proven semantic version, build fingerprint, and protocol/store/
feature ABI, and the image check was trading that real proof for an unavailable
one. It logs daemon.client_image_unverifiable_admitted so the weaker check is
never invisible. A fingerprint MISMATCH — the tamper case the gate exists for —
still rejects hard. Separate test seams keep the two modes testable apart.

**Client bootstrap failures are no longer silent (#1539).**
An MCP client that cannot reach the daemon now emits a JSON-RPC error on stdout
naming the reason, plus the same text on stderr. Previously the reason sat in
bootstrap_result.message and the process exited having written nothing at all.

**POSIX activation: group-writable ancestors (#1535, discussion #1526).**
activation_directory_secure required no group or other write bit on the install
directory AND every ancestor. WSL2 ships ~ and ~/.local at 0775, as do several
distro skeletons and any site using a shared primary group, so install.sh failed
for a large fraction of Linux users — reporting a policy refusal as "activation
transaction I/O failed", which sent reporters after disk errors and filesystem
types. Root-caused by @AmirF194 in a clean ubuntu container; @shochdoerfer and
@iandol confirmed independently.

World-writable ancestors are still refused (any local user could swap a path
component mid-transaction). Group-writable ancestors are now warned about and
admitted. The LEAF directory stays strictly owner-private — that is where the
binary is published, and group write there would let another account replace the
executable between validation and exec. Refusals now name the directory, its
mode, and which rule refused.

**The obsolete ui/standard chooser (#1538, from discussion #1526).**
v0.10.0 consolidated to one archive per platform with the UI always embedded,
but `update` still offered a variant choice: "ui" could only 404, and "standard"
quietly WAS the UI build. Reported by @iandol upgrading 0.9.0 -> 0.10.0. The
chooser, its --standard/--ui flags, and the ui- URL plumbing are removed, along
with the CBM_VARIANT=ui remnant in the npm installer.

Already-released 0.9.x binaries cannot be fixed retroactively, so the release
workflow now publishes byte-identical ui-*-named alias assets — their updaters
work again with no user action. The aliases are uploaded AFTER the VirusTotal
gate: they are the same bytes as archives it already cleared, and uploading them
earlier would duplicate every object in the scan set and the provenance manifest.

**macOS install noise and attribution (#1537).**
install.sh silenced the "No such xattr: com.apple.quarantine" line, which is
what happens when a curl-downloaded archive carries no quarantine attribute —
harmless, and it became the title of a bug report about an unrelated failure.
The session-stop refusal now points at `daemon status` to list the client
processes actually holding the daemon, instead of asserting sessions exist and
leaving the reader to guess. Reported by @listepo.

**Riders.** hatchling is pinned in pkg/pypi (an unpinned backend resolved fresh
inside `python -m build` is what emitted Metadata-Version 2.5 and broke the
v0.10.1 publish); SECURITY.md's supported-versions table moves to 0.10.x.

Tests: separate seams for unverifiable vs mismatched peer images with a test per
outcome; activation refusal must name directory + mode + rule; a group-writable
ancestor must stage successfully. The update tests drop the flag that no longer
exists. Verified against each reporter's environment shape.

**Open security alerts (all three, OSSF Scorecard).**
- HIGH, binary artifact: an 8.8 MB compiled Go ELF wrapper had been committed at
  pkg/go/codebase-memory-mcp by accident. Removed, and both it and its .exe
  sibling are gitignored so `go build` in that directory cannot repeat it.
- HIGH, GHSA-2v37-7h3g-55p8: nanoid < 3.3.17 loops forever when a custom
  generator is called with size 0. It reaches us transitively (postcss -> vite),
  so it is pinned through the existing graph-ui overrides block rather than
  promoted to a direct dependency; the lockfile resolves 3.3.18.
- MEDIUM, unpinned pip command: the publish step installed build/twine by
  version only, leaving the whole transitive graph resolved at run time.
  pkg/pypi/requirements-publish.txt now hash-pins the complete toolchain (316
  hashes), generated on a linux/amd64 python:3.12 image so the wheels match what
  ubuntu-latest resolves, and the step runs pip with --require-hashes. Verified
  by installing from it in that same image.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-08-11 13:37:32 +02:00
Martin Vogel 2dd296bab4 fix(daemon): answer the client when its binary image is rejected (#1383) 
An image-verification rejection logged daemon.client_image_rejected and
finished the worker WITHOUT sending a hello response - the only
admission-failure path that never answered the peer. The client
reported status "pending" indefinitely, indistinguishable from a slow
cold start, with the reason visible only in the daemon log.

The rejection now sends a CONNECT_REJECTED hello response carrying the
reason (fingerprint_mismatch / image_unverifiable) and an actionable
message, matching the version-conflict and capacity paths. The
version-conflict path already responds to unverified peers, so this
discloses nothing new to a same-uid local peer; admission stays
rejected either way.

Adds a CBM_ENABLE_TEST_SEAMS-only seam forcing peer-image verification
to fail, since the in-process harness's peer pid is OS-authenticated
socket credentials and always verifies against the service's own
active image.

Test: daemon_runtime_image_rejection_reaches_client_issue1383 - the
client receives CONNECT_REJECTED with the reason in the message. RED
without the response block (client times out), GREEN with it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-08-04 19:15:11 +02:00
Martin Vogel 0baddf7887 daemon: permanent lifecycle, daemon-backed CLI/hooks, real-Windows hardening, long-path launcher transactions 
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.

Daemon lifecycle:

- daemon start/stop/status subcommands. `daemon start` launches a
  PERMANENT daemon (spawn shape is byte-exact argv; survives idle
  periods and session ends) and reports an already-active daemon
  instead of failing. Permanence is honored at every stop latch:
  last-committed-client disconnect, host initial-client window,
  coordinator release, and application final-session close — a
  permanent daemon also keeps admitting new sessions after its last
  one closes.
- daemon stop refuses while sessions are active and lists the blocking
  peers (pid/role) that must finish first; an idle daemon drains
  through the activation-shutdown machinery with the ACK ordered after
  connection interrupts. A second stop is idempotent. The wire ops are
  no-cohort first-frame requests with peer fingerprint authentication,
  so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
  keep their local supervised path). A cold CLI run that had to spawn a
  temporary daemon prints a hint that `daemon start` removes the
  per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
  emits a visible, rate-limited notice (Claude-dialect systemMessage
  plus stderr for other dialects) and always exits 0 — augmentation is
  never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
  older-build permanent daemon (strict semantic-version triples only;
  dev builds never auto-drain) and the build-conflict message names
  `cbm daemon stop` as the manual escape hatch.

Windows IPC/runtime (real-VM verified):

- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
  path used to destroy its listening pipe instance on every 20 ms poll
  timeout; a client attaching in the teardown window was severed or left
  on an orphaned pipe object whose HELLO no server handle could ever
  read, absorbing the connect until the client's own timeout expired.
  The pending connect now survives poll timeouts and nothing is
  destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
  server handle can discard a just-sent response before the peer reads
  it (POSIX stream sockets never lose buffered data on close). A bounded
  cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
  immediate on interrupted connections) now precedes close in
  runtime_worker_finish and runtime_reject_inline, so hello-conflict,
  capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
  transport half-close, so close_begin now announces departure with an
  explicit frame (ordered after APPLICATION_CANCEL, before the local
  interrupt); the server releases the client's admission on receipt
  instead of waiting for the handle to close. Admission-drop timing is
  now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
  request publication under the send lock, best-effort sends the active
  token's APPLICATION_CANCEL frame, then interrupts local I/O; the
  server cancels MCP/subprocess work promptly. Contract tests accept
  both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
  the requester's license to act on "snapshotted and draining", so every
  connection interrupt is now initiated before the ACK is sent; a
  session could previously get one more request serviced after the
  requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
  prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
  timer granularity) burned ~1.6 s against permanently obstructed paths,
  stalling hello rejections past the client's timeout. The retry budget
  is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
  correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
  Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
  nonblocking send() on Windows is absorbed wholesale into AFD kernel
  buffering regardless of SO_SNDBUF, so send deadlines and interrupts
  could never engage against a slow peer (and the full payload was
  pinned in nonpaged pool). Slicing restores a deterministic
  backpressure point; a test hook pins SO_SNDBUF for the deadline and
  interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
  deadline, explicit connection states, refusal to free a server while
  a listener-owned connection is active.

Windows long-path support:

- Central path-aware wide conversion (canonicalize via GetFullPathNameW
  and prepend the extended-length prefix for absolute paths >=240) at
  the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
  store opens, and the daemon build-fingerprint/log paths. Deep managed
  installs (a 64-hex generation directory routinely exceeds MAX_PATH)
  now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
  ancestry validators now operate in the extended-length namespace;
  the launcher path is canonicalized (and prefixed when deep) once at
  entry so every downstream exact-string comparison stays
  form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
  GetFileAttributesW) so non-ASCII install paths survive argv[0]
  resolution.

Windows launcher install/uninstall transaction:

- FileRenameInfoEx names are NUL-terminated in an over-allocated
  buffer. FileNameLength governs per the contract, but filter drivers
  read FileName as NUL-terminated and appended adjacent heap bytes to
  created names — a flaky, garbage-suffixed rename target. Both the CLI
  and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
  .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
  chars so the bare rename target stays under the FileRenameInfoEx
  NT-conversion ceiling at guard depths; 64 bits still uniquely
  identify the generation.
- When the running launcher's mapped generation backings pin .cbm
  against rename, the backings are relocated to activation-<pid>-N
  .retired tombstones beside the install (a mapped image may be renamed,
  never deleted; the launcher's liveness-guarded sweep reclaims stale
  tombstones). Every relocation is recorded, and a FAILED uninstall
  reverses the moves after restoring .cbm — via MoveFileExW with
  extended-length paths on both arguments, since the deep generation
  target is beyond the handle-based rename's bare-path reach — so a
  restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
  out so the tree is shallow enough for the detached cleanup's rd, and
  the cleanup's working directory strips the extended-length prefix
  (CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
  (CopyFileW destinations, CREATE_NEW tombstones, probe directories)
  are explicitly owner-stamped so the exact-owner validators hold on
  runner images; guard fixtures stamp hand-built trees the same way.

Diagnostics, tests and infra:

- diagnostics: discovery is now an always-delivered JSON control record
  (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
  paths containing spaces; placement honors $TMPDIR with /tmp fallback
  via a diagnostics-local helper; the soak parser reads the JSON record;
  documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
  log levels, TMPDIR-with-spaces, and native Windows output-contract
  assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
  parents with production-shaped ancestry (LocalAppData on Windows, via
  th_secure_runtime_parent_new) — the runtime ancestry validation
  correctly refuses temp roots whose ancestors grant mutation rights to
  Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
  drive the documented startup-owner publication flow before reading
  generation-bound endpoint addresses. This turns the 26 Windows
  failures previously visible in CI's full-test job green without
  weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
  contract including failed-uninstall restore and immediate reinstall
  after uninstall; new daemon lifecycle and reworked hook-augment
  guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
  (it was set by CI but never read, so perf suites ran everywhere);
  four throughput/bench suites are classified as perf, the heavy
  store_arch suite moved to the slow-timeout tier, and two
  wall-clock-sensitive assertions were rewritten as invariant checks
  with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
  contract, soak daemon-recovery contract, path-safety helper, the
  Windows VM worktree-sync contract wired into scripts/test.sh, and
  vm/win.sh guards building its clean embedded-UI product in an
  isolated BUILD_DIR so it cannot clobber the incremental test build.
  provision-windows.sh now installs Node.js for the guards UI build.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-21 15:32:15 +02:00
Martin Vogel 5e23839f87 fix(daemon): make request completion transactional and EOF drain-safe 
Back-to-back requests on one frontend<->daemon connection failed roughly
half the time on loaded Windows hosts, surfacing as wandering MCP session
drops (smoke Phase 5/5e/5f "no tools/list response", "timed out waiting for
id=2"). Root cause, confirmed on real Windows: the application worker stored
its done flag AFTER writing the response, so a client that pipelines its
next request the instant it sees a response raced the previous worker's
final instructions and its join, and admission rejected the new request
with BUSY. On EOF that BUSY was then classified as an expected shutdown and
the response was silently dropped with a success exit; mid-session it exited
non-zero. A timing-sensitive heisenbug — any added stderr shifted the window
and it passed.

- runtime_application_worker publishes the done flag BEFORE sending the
  response, so completion is one atomic transition from the client's view:
  by the time any client can react, the slot already reads reusable. The
  admission side joins the thread after observing the flag, which absorbs
  the send tail. No timing constants.
- frontend_should_stop's input-closed leg now also requires no request
  in flight, so a daemon-side failure of the final post-EOF item is loud,
  never swallowed as an expected stop.
- The EOF drain is now progress-aware: it resets its window on each
  completed item, bounding stuckness rather than total batch length, so a
  slow cold daemon spawn no longer abandons queued tail responses.
- Latch the never-connected shutdown window on a monotonic admitted-total
  counter instead of sampling the live client count, which can read zero
  between two short-lived sessions and stop a daemon mid-conversation.
- Add structured lifecycle diagnostics (daemon.runtime_stopping reason=,
  lifetime_end reason=, connection_end) so this class is observable in the
  logs instead of being inferred.

Validated: real-Windows loop 12/12 (was ~50% loss), macOS daemon suites
143/143.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-19 16:55:25 +02:00
Martin Vogel 83c137d2a5 feat: complete shared daemon lifecycle 
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-18 01:26:08 +02:00
Martin Vogel 0e00ef5702 feat: coordinate concurrent CBM sessions 
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-16 19:20:46 +02:00