`posix_spawnp` and `fork` both fail with EAGAIN when the process table or a
per-user limit is momentarily full. Both paths treated that as a hard failure,
so a burst of process creation turned into CBM_PROC_SPAWN_FAILED — a git probe,
an LSP server, or an index worker refusing to run for a reason the user can
neither see nor act on, on a machine that is merely busy.
This is the production half of a flake we have been re-running for two days:
`subprocess_run_clean` failed spawning `/bin/sh -c "exit 0"` on macOS runners in
three separate CI runs, always under a loaded release matrix, always green on a
quiet rerun. Chasing it as a test race would have been chasing the symptom — the
test is right, and a user on a loaded laptop hits the same edge.
EAGAIN/ENOMEM now retry three times over ~30ms; everything else keeps failing
immediately, and an exhausted system still fails after the backoff rather than
spinning. exec-class errors (ENOENT/EACCES/ENOEXEC/...) are untouched, so the
documented "missing binary exits 127, spawn_failed means the mechanism failed"
contract still holds. The fork fallback gets the same treatment: it fails for
the same reason under the same pressure, and the fallback must not be less
robust than the primary path.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both Linux soak-quick legs in the dry run failed with:
FAIL: RSS 17592186044293MB > 200MB ceiling
17592186044293 MB is 2^44-123 -- a NEGATIVE ~123MB wrapped through size_t. The
diagnostics snapshot reported mi_process_info()'s current_rss verbatim, and on
Linux mimalloc never sets that field: it keeps mimalloc's internal
committed-page counter, which this project deliberately tunes low
(purge_decommits=1, purge_delay=0) and which can transiently go negative under
purge. macOS and Windows are untouched -- mimalloc reports real RSS there --
which is exactly the observed split: both Linux legs red, both macOS legs green.
mem.c already learned this lesson in full (its comment block documents the same
trap for cbm_mem_over_budget): cbm_mem_rss() is /proc-statm-primary on Linux and
mimalloc-correct elsewhere, and cbm_mem_peak_rss() reconciles peak >= current.
The diagnostics writer now uses both instead of the raw mi fields, so the soak
leak gates measure actual process RSS on every platform.
Verified: diagnostics suite passes; compiles and formats clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1448 set CREATE_NO_WINDOW on the subprocess spawn and on cbm_popen_isolated,
but cbm_exec_no_shell (compat_fs.c) still passed a bare 0 for dwCreationFlags.
That is the helper behind git, codesign and open, so the most frequently hit
path kept flashing a console — and under a stdio MCP session with auto_watch
those windows steal focus while the user is typing. Reported by @noctrex on
#1448 after verifying the merged fix.
Also adds the contract nobody had. The flag has to be set per call site, the
four sites do not resemble each other, and reviewing "does this PR add the
flag" says nothing about the sites a PR does not touch — which is exactly how
the third one survived a fix applied twice:
daemon/bootstrap.c had it from the start (DETACHED_PROCESS | ... | flag)
subprocess.c added by #1448 (flags variable)
compat_fs.c :313 added by #1448 (inline literal)
compat_fs.c :689 missed by #1448 (bare 0)
tests/test_spawn_no_window_contract.sh asserts the whole-tree property instead,
so a fifth spawn site cannot be added without it.
The contract strips C comments before matching, and that is load-bearing rather
than tidiness: the first version of it PASSED with the fix reverted, because the
explanatory comment above the fixed call contains the flag name and satisfied
the match. A contract that can be satisfied by prose about the contract is a
false guard. Verified in both directions afterwards — green with the fix, and
red naming compat_fs.c:696 with the fix reverted and that comment still in
place.
Co-Authored-By: noctrex <noctrex@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The comment above MIMALLOC_OVERRIDE_DEFINE claimed Unix "relies on
static-link-order override". That was never true. mimalloc emits strong
malloc/free definitions only when MI_MALLOC_OVERRIDE is set — the define its
source gates alloc-override.c on — and the build set it for MinGW only.
MI_OVERRIDE, which the build does set everywhere, is this project's own
prod/test marker that mimalloc never reads. With the override body compiled
out there were no strong symbols for link order to prefer, so ordinary malloc
went to libc.
Measured, not inferred. The shipped v0.9.1-rc.1 artifacts report 0/6
allocator-owned size classes on linux-arm64 glibc AND musl-static, and the
same A/B on ubuntu-arm64 here:
main warn mem.allocator.not_owned owned_classes=0/6
with this fix info mem.allocator.owned classes=all
Every purge/reclaim option cbm_mem_init sets was therefore inert on Linux,
applying only to the bound sqlite/tree-sitter populations — the same class of
defect as #581, where committed memory ratcheted for months because nothing
asserted the wiring on a real artifact.
macOS stays off deliberately and permanently: enabling the override there
compiles alloc-override.c's forwarding definitions, and under the two-level
namespace this binary's free becomes mi_free while system libraries keep
allocating from the system allocator, so the first pointer crossing that
boundary aborts with "mi_free: invalid pointer". ELF's flat namespace has no
such split, which is why Linux can have this and macOS cannot.
Three parts:
* Makefile.cbm switches MI_MALLOC_OVERRIDE and a new CBM_MEM_GLOBAL_OVERRIDE
together in one place, for MinGW and Linux. The latter tells our own sources
what to expect and goes into CFLAGS_PROD only, because MIMALLOC_CFLAGS_TEST
builds mimalloc with -DMI_OVERRIDE=0 and no override define: a test binary
has no global override BY CONSTRUCTION and must never be told to expect one.
Deriving that expectation from the platform instead would make every Linux
test run warn about a correctly configured build.
* mem.c splits the startup audit by what the build actually asked for. Where
no override was requested, ordinary malloc reaching libc is the design, so
it reports the measured ownership at INFO and names what IS bound. A warning
that fires on every run of a correct build is not a tripwire, it is noise —
it trains readers to ignore the one line that catches #581. The genuine
warning still fires wherever an override was requested and did not take.
* smoke-test.sh Phase 1b pins the SHIPPED artifact's wiring per platform and
fails in BOTH directions: Windows/Linux must own all classes, macOS must
not. This cannot live in a unit test — a from-source test build never has
the override — which is exactly why the defect survived so long.
Verified on ubuntu-arm64: prod owns all classes; the test build reports
bound-populations-only at INFO (0/6) and the mem suite stays green; smoke
Phase 1b passes, and fails with the expected message when the same tree is
built with the override forced off. On macOS: prod reports
bound_populations_only at INFO, smoke Phase 1b passes, mem suite 51/51.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A repository may ship .cbmpathwhitelist listing outside roots it would like
indexed alongside it. The file REQUESTS; it never GRANTS. Both attackers in this
project's threat model can write a file inside a repository — a malicious indexed
repo, and an agent with write access to the project — so a repo-local file cannot
be authoritative without handing them the boundary.
Approval is recorded user-level and keyed to a SHA-256 of the manifest bytes, so
editing the file lapses approval and it must be granted again. Without the hash a
repo approved once could widen itself forever on a later pull. This is direnv's
model, and it is the only shape that keeps a checked-in file's ergonomics without
making the file a permission slip. The hash covers the raw bytes rather than the
parsed entries: a reordering or comment change is still a change the approver has
not seen.
`allow-root --approve-manifest <project>` is the human action that grants.
Approval refuses outright if any requested entry would not stand on its own as an
indexing root, so approving cannot become a route around the breadth policy, and
entries are re-classified at use time as well so a stored approval cannot outrank
a credential list that has grown since.
Entries containing control characters are rejected — the same shape as the
newline splitting that let a crafted indexed path inject an extra entry into the
scoped file list.
Deliberately NOT wired into the root boundary. A project's manifest authorizes
outside roots FOR THAT PROJECT, so the question is "may project P pull in tree T",
not "may T be indexed standalone", and the boundary check only ever sees one path
with no project context to ask that with. A first draft did wire it there by
passing the candidate as its own project root, which read a manifest that by
definition was not the one requesting it and so authorized nothing — caught by
exercising it end to end. The consuming half belongs where project context exists,
in discovery walking a project's approved extra roots;
cbm_workspace_manifest_allows is the query it will use.
Five tests, including the one the design rests on: approval lapses when the
manifest content changes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
fork() duplicates the parent's address-space bookkeeping. Under an
ASan-instrumented parent that includes an enormous shadow mapping, and past a
footprint threshold the child is jetsam-killed BEFORE exec replaces the image.
The reaped corpse then looks like the launched tool dying, so a `git init` in a
fixture "fails" only once enough suites have run ahead of it — a spawn defect
that reads as an unrelated assertion. tests/test_daemon_runtime.c already
switched to posix_spawn for this exact reason; this moves the shared
subprocess layer over as well.
posix_spawn never copies the parent address space, so the parent's footprint
stops being a variable. Every guarantee of the fork path is carried over:
- own process group SETPGROUP + setpgroup(0) (kill-tree contract)
- default signal handling SETSIGDEF + SETSIGMASK
- std{in,out,err} wiring adddup2
- all other fds closed CLOEXEC_DEFAULT, Apple's equivalent of the
child's close-everything loop (the three dup2'd
fds survive, since dup2 clears close-on-exec)
- PATH lookup posix_spawnp keeps execvp semantics
One deliberate difference is reconciled rather than adopted: posix_spawn
reports an unusable binary to the PARENT, where fork+exec instead yields a
child that exits 127. cbm_subprocess_run's contract treats spawn_failed as
"the spawn mechanism failed", not "the tool was missing", so exec-class errors
fall back to fork+exec and macOS keeps classifying a bogus binary exactly as
Linux does. That path forks a child that exits immediately, so it does not
reintroduce the hazard.
The existing suite binds each guarantee, verified by breaking them:
dropping CLOEXEC_DEFAULT leaks the sentinel descriptor and reddens
subprocess_posix_child_closes_unrelated_descriptors; dropping SETPGROUP makes
the kill-tree turn on the runner's own group and reddens the suite wholesale.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
"Users" was matched the way credential directory names are — against every
path component. On Windows every user's files live under C:\Users\<name>, so
that refused every ordinary project path, including a CI runner's own
workspace. The Windows smoke job on the pull request caught it; the unit
tests could not, because they only ever run compiled on macOS here.
System trees (Windows, ProgramData, Program Files) now match only as the
first component below the drive, which still refuses anything inside them at
any depth. "Users" is handled separately as the tree root only, mirroring
POSIX where "/Users" is refused but "/Users/dev/projects" is not.
Tests cover the regression directly: C:/Users/dev/projects/app and
C:/Users/runneradmin/work/repo are allowed, C:/Users is not, C:/Windows/System32
is refused at depth, and C:/dev/Windows-app — merely named after a system
tree — is allowed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Routes the MCP index_repository handler and the graph UI's POST /api/index
through a single decision function. The UI route previously checked only that
root_path was a directory, so an operator's configured boundary held on one
entry point and not the other; it now canonicalizes first and applies the same
policy, answering 403 with the reason.
The decision is two-tier, because a bare default-deny would refuse every
first run and a bare opt-in leaves the default open:
- Breadth is always enforced, with nothing configured. Filesystem, drive and
share roots, top-level system trees, the home directory itself and
credential directories are refused as indexing roots out of the box.
- Containment in a declared root applies once CBM_ALLOWED_ROOT is set or a
grant exists, and is evaluated first so a path outside a configured root is
reported as exactly that.
Three things the tests caught, each a real defect rather than a test fix:
- On macOS /etc, /tmp and /var are firmlinked under /private, so
canonicalizing "/etc" yields "/private/etc" and counted two deep — sailing
past a minimum of two, missing the very path being guarded. Depth now
discounts a leading "private" component.
- An earlier draft refused any root containing the cache directory. That was
over-claimed: the indexer only parses recognised source files and a graph
database is binary SQLite it would never extract. Refusing a whole root is
also the wrong remedy where the concern does hold — not walking the cache
is. Removed, with the reasoning recorded at the site.
- Rewording the refusal to "outside every allowed root" broke an assertion
matching "outside the allowed root", and that test's early return skipped
its CBM_ALLOWED_ROOT cleanup, leaking the variable into every later test in
the suite. The original wording is kept and guidance appended instead.
Worth remembering: these contracts match strings, not properties.
Docs updated in the same change, since both env-var tables said "unset
imposes no restriction" and that is no longer true: CONFIGURATION.md and
README.md describe the two tiers, and CONFIGURATION.md lists the always-
refused roots along with the two limits that matter — this constrains scope
rather than sensitivity, and the credential list is a denylist that raises the
cost of a mistake rather than closing the class. SECURITY.md's supported-
versions table was still on 0.8.x and now reads 0.9.x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Adds cbm_workspace_classify_root, the breadth half of the workspace
boundary. It answers "is this path obviously too broad or too sensitive to
index as one root", not "is this caller allowed to index it" — that second
question needs the user-level grant store, since it is the only input a
caller cannot write. The header says so, because a reader who mistakes one
for the other would draw the wrong conclusion about what is enforced.
Three rules, each doing what the others cannot:
- Depth below the volume. Two components on POSIX refuses every top-level
tree in one rule with no list to maintain. Windows and UNC count
drive/share-relative and require one, because there the first component is
already user space ("D:/repos") — the system trees that sit at the same
depth ("C:/Windows") are covered by name instead, which is tractable
because that set is small and stable.
- The home directory itself, which is two deep on macOS and Linux and so is
invisible to depth.
- Credential directory names, matched against every component so both
"…/.ssh" and "…/.ssh/keys" are refused. This list is deliberately additive
so extending it needs no design authority.
A path that is or contains the cache directory is refused outright: indexing
such a tree would pull every other project's graph database into this
project's index.
Classification order is load-bearing and commented as such. The home
directory normally contains the cache directory, so testing the cache first
reported every home as "holds the cache" and made it non-overridable, which
contradicts the intent that a person may override it. Depth precedes the
cache test for the same reason: "/Users" satisfies both and "too broad" is
the reason that helps the reader.
Takes home and cache as parameters rather than reading the environment, so
the policy is a pure function and the tests need no filesystem. Twelve table
tests; three of them failed on the first implementation and drove the
ordering fix above.
Not yet wired to any caller — deliberately, so the policy can be reviewed on
its own.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A pass over places where a bound was either miscalculated or missing.
UI response builders (src/ui/http_server.c) mixed the clamping helper
http_appendf with raw cursor indexing. Route the per-line separators and
quotes through the helper too, so the cursor cannot leave the buffer once
output saturates it, and terminate explicitly afterwards — the helper pins
the cursor and then writes nothing, which would otherwise leave the "%s"
reply with no terminator in range. The log buffer was budgeted at
LOG_LINE_MAX + 10 per line while JSON escaping can double every byte, so
size it for the escaped worst case. Job status entries now escape their path
and error fields, which were interpolated raw.
Cypher (src/cypher/cypher.c): the WHERE grammar descends once per nested '('
and once per NOT, so parse depth followed the query text rather than
anything bounded. Track depth on parser_t and refuse past 256 levels with a
parse error. Bounding parse depth also bounds the resulting tree, so the
recursive evaluator inherits the limit.
Gitignore (src/discover/gitignore.c): '**' retries the remainder at every
position and consecutive groups multiply, so match cost is exponential in
the number of groups. Thread a step budget through the matcher and give up
past 20000 steps, reporting no-match so a pathological pattern fails to
ignore rather than ignoring the wrong files. glob_match keeps its name so
recursion_whitelist.h still describes the functions that recurse; the new
non-recursive wrapper that seeds the budget is glob_match_bounded.
Call extraction (internal/cbm/extract_calls.c): extract_fp_callee recursed
once per applied argument, so stack use followed the parse-tree depth of the
indexed file. Rewritten as a left-spine loop. Reassigning node/nk before
continuing leaves the fall-through cases where the recursive form left them,
so behaviour is unchanged and the extraction suite is untouched.
CALLS props (src/pipeline/pass_calls.c): `cap - pos - PAIR_LEN` wraps once
pos reaches cap - PAIR_LEN and stops bounding the copy below. Guard
additively, matching the closing write in the same function.
Shell-bound paths (src/foundation/str_util.*): the three git shell-out sites
each wrap a repo path in cmd.exe-compatible double quotes, where %VAR%,
!VAR! and ^ remain active. Two carried private copies of the check that
rejects those three on Windows and the third used the bare validator.
Promote the stricter form to cbm_validate_shell_path_arg and route all three
through it, so the copies cannot drift apart again.
Tests cover the buffer bounds, the parse-depth refusal, matcher termination
and the validator. The bounds tests run in a forked child so a violation
surfaces as a signal instead of silently.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two foundations for the delta-merge incremental subsystem (a dedicated
copy->patch->rename executor for the closure route; the general dump
pipeline is untouched):
cbm_clone_or_copy_file (foundation/compat_fs): stage a database by
copy-on-write clone where the filesystem has one -- clonefile(2) on
APFS, FICLONE on Linux reflink filesystems -- with a streamed copy as
the portable fallback. Verified byte-identical and write-independent.
For a multi-GB generation this is the difference between milliseconds
and seconds of staging cost.
cbm_pipeline_finalize_staged_generation: the final leg of publication
(sidecar removal, previous-generation quarantine, atomic rename with
rollback on every failure) extracted, behavior-preserving, from
cbm_pipeline_publish_generation so a patched staging copy can publish
through the exact same crash-safety tail as a dump-built one. The
FTS-rebuild and integrity-check policy deliberately stays OUTSIDE the
shared tail: the dump path rebuilds wholesale, while the delta path
will write row-level FTS inserts (safe against stale entries because
node ids are AUTOINCREMENT and never reused, so dead rowids drop out
of the join).
All 437 pipeline/incremental/publication tests green, unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Apple clang 15 (Xcode 15.4, the macOS CI image) rejects
static _Atomic cbm_log_sink_fn g_log_sink = NULL;
with "initializer element is not a compile-time constant": NULL expands
to ((void*)0), and the implicit void*-to-function-pointer conversion is
not a constant expression there. Casting to the function-pointer type
makes it an address constant, which is what a static initializer needs.
The local ladder could not have caught this. The macOS host here runs
Apple clang 21, which accepts the uncast form; the rejecting compiler
exists only on the CI image. Recording that plainly because it is a real
gap in what local verification can promise for macOS, not a slip in how
this batch was checked.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The level, format, sink and sink-mode globals are written by whichever
thread configures logging and read by every thread that logs: daemon
connection workers, pipeline workers, the watcher. They were plain
globals.
For the sink this is not a benign stale-value race. emit_line() read the
global function pointer, tested it, then called it -- so a concurrent
cbm_log_set_sink_ex() could turn a checked pointer into a call through a
NULL or partially-written one. The load is now done once into a local,
which closes the test-then-call window as well as the tear.
Ordering is relaxed: each value is an independent scalar with nothing to
publish alongside it, and the log path has to stay cheap enough that no
caller is tempted to route around it. cbm_log_set_sink_ex stores the mode
before the sink, so a reader that sees the new sink cannot then read the
mode belonging to the previous one.
Found by ThreadSanitizer once the daemon_runtime suite was added to the
TSan set. That suite had been excluded from the lane, which is precisely
what kept this hidden.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two follow-ups to the MSan lane, both from running it for real.
1. compat_thread.c gains a sanitized-build-only stack FLOOR
(CBM_THREAD_STACK_MB). Thread stacks here are sized in code, so a
sanitizer lane cannot raise them with ulimit -- RLIMIT_STACK at 8/64/256
MiB provably had no effect. The first version overrode only the DEFAULT
size, which silently did nothing for worker_pool/runtime/main because they
all pass an explicit size; it is now a floor applied to every thread.
Shipping builds are untouched (the whole hook is behind
CBM_SANITIZED_BUILD).
2. Two grammar-corpus suites are EXCLUDED from the lane, with the full
rationale, evidence, and everything tried recorded at the exclusion site
per O10 -- including that the floor above does NOT fix them, which narrows
the next person's search to a thread creator outside cbm_thread_create.
The exclusion list now names SUITES (an earlier version named a TEST and
therefore excluded nothing) and fails loudly on an entry that matches no
suite, so that silent-no-op cannot recur.
Also restores the exec bit on scripts/msan.sh, which the image ENTRYPOINT
needs.
worker_pool + parallel + pipeline + mcp on macOS: 526 passed, 2 skipped.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
make lint-ci was red on this branch before any of the rebase work.
cppcheck reported all four halves of two ULARGE_INTEGER values in
cbm_path_info_utf8 as assigned-but-never-read. The code is correct -- it is a
union, and .QuadPart reads exactly what .LowPart/.HighPart wrote -- but cppcheck
does not model that aliasing, so it cannot see the read.
Composing the two 64-bit values arithmetically says the same thing without the
union, so the checker needs no exception. That is the repository's stated
preference: refactor first, adjust the rule second, suppress only as a rare
justified exception -- and a suppression here would have to be re-justified by
every future reader.
Also applies clang-format to the lines this rebase touched in pipeline.c and
extract_usages.c, plus one pre-existing violation in pipeline_incremental.c.
Formatting only; the file set is limited to what LINT_SRCS/LINT_HDRS actually
covers, so no unrelated whole-file reflow rides along.
make -f Makefile.cbm lint-ci: passes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm.h documents cbm_label_is_type_like() as the single source of truth for
type-like labels, "so adding a new type-like label (e.g. Struct for
Rust/Go/Swift/D structs) updates them all at once instead of scattering
|| strcmp(label,"Struct")==0 across the tree".
A SQL string literal cannot call it. Four queries in store.c and the BM25
ranking in mcp.c carried their own hardcoded label lists, so they silently
opted out of that contract and stopped matching the moment Struct, Interface,
Enum, Type and Trait began being emitted:
store.c:5279 arch_boundaries ('Function','Method','Class')
store.c:5422 arch_packages_from_qn ('Function','Method','Class')
store.c:6689 arch_clusters ('Function','Method','Class')
store.c:7742 cbm_store_vector_search ('Function','Method','Class')
mcp.c:2759 BM25 ranking ('Class','Interface','Type','Enum')
Effect on main today: get_architecture (boundaries, packages, clusters) and
vector search drop every Struct, Interface, Enum, Type and Trait node in the
project, and search_code under-ranks structs -- for Rust, Go, Swift and D,
where struct is the primary type declaration. The BM25 list had drifted
differently again, omitting Struct and Trait but not Interface.
The fix is not a wider literal. CBM_SQL_TYPE_LIKE_LABELS and
CBM_SQL_CALLABLE_OR_TYPE_LABELS live next to each other in constants.h and are
pinned to cbm_label_is_type_like() by a test that checks both directions:
every label the C predicate accepts appears in the SQL fragment, and nothing it
rejects is smuggled in. Adding a type-like label without updating the SQL now
fails that test instead of quietly shrinking query results.
No new labels are introduced. Every label added here is one the extractors
already emit.
Reported by @win4r in #568, whose audit found four of the five sites. Their
patch widened the literals in place and added Actor, a label main does not
emit; this routes through the canonical set instead and also covers Interface,
Type and Trait, which that patch still omitted.
Co-Authored-By: win4r <win4r@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Microsoft's ML flagged the rc.1 release binaries. The decisive evidence is that
the SAME sha256 went from 0/62 clean to Microsoft-detected in about an hour with
no byte change, so the verdict lives partly in scanner-side state and no code
change can promise a clean result. What code CAN do is stop shipping things that
have no business in a release artifact, which is worth doing on its own merits
and incidentally widens the classifier margin. Every claim below is verified
against a built binary by the new gate, not by reading source.
Executable stack (the worst of the findings). vendored/nomic/code_vectors_blob.S
is the only assembly in the build and carried no .note.GNU-stack. An unannotated
object makes ld assume the worst for the whole link, so EVERY Linux release we
have ever shipped had GNU_STACK RWE. Adds the note (cause) plus ELF-only
-Wl,-z,noexecstack (outcome); the gate fails the release if it returns.
Test seams are now opt-in, never opt-out. TEST_SEAMS=1 defines
CBM_ENABLE_TEST_SEAMS; without it the crash-orphan probe -- which forks a child
that ignores SIGTERM and loops forever, then writes its pid to a caller-supplied
path -- and the lease-ownership marker compile to trivial stubs, so call sites are
untouched and the binary holds no fork, no signal handler and no env-var string.
Opt-IN is the point: forgetting the flag yields a clean binary rather than a leaky
one. scripts/test.sh requests it in the leg that consumes it, and
tests/test_worker_watchdog.sh now asserts the capability up front instead of
dying later with an opaque "Killed: 9".
The daemon's background version check is gone. It spawned curl against
api.github.com/repos/.../releases/latest on the first eligible session of every
run to say "a newer version exists" -- a release URL and an outbound request in
every shipped binary, for something the install scripts already report. The
INJECTABLE SEAM survives: update_ops is still honoured, the fakes in
tests/test_daemon_application.c still cover notice/ownership/cancellation/replay,
and with no provider application_update_subscribe_locked returns early so no
generation ever starts. "No network request by default" is now structural.
Dead capability out of release builds. The tar.gz/zip extraction block
(gzip_decompress through cbm_extract_binary_from_zip, plus its cli.h
declarations) moves under CBM_CLI_ENABLE_TEST_API -- verified self-contained, zero
uses of any helper outside it, only callers the excluded updater and
tests/test_cli.c. Downloading an archive, decompressing it, picking an executable
out of it and marking it executable is the canonical dropper composite; it is now
absent rather than merely unreachable. SQLite is built with
-DSQLITE_OMIT_LOAD_EXTENSION (no caller of load_extension anywhere in src/ or
internal/), removing that API surface and part of the dlopen/dlsym surface.
Temp files and environment scanning (S2/S3). Predictable paths in mcp.c,
artifact.c and diagnostics.c are created privately and exclusively and written
through the returned descriptor; pass_envscan.c no longer descends symlinked
directories out of the project root, and its fixed 512-byte path buffers no
longer truncate into pointer arithmetic that could land outside the buffer.
Build-time entropy. mimalloc's version banner baked __DATE__/__TIME__ into every
binary, so two builds of identical source seconds apart could never share a hash
and no release could inherit a false-positive determination made about its
predecessor. Local patch removes it (marked to survive refreshes), -Wdate-time
makes any future use a build error, and -Wl,--no-insert-timestamp stops the PE
header carrying the link clock.
scripts/ci/check-binary-composition.sh is the proof that each removal stays
removed, wired into package-release.sh after strip so the local artifact-flow
smoke enforces exactly what CI does. It asserts absences plus a CANARY string, so
handing it a compressed, stubbed or empty file fails instead of passing
vacuously, and a missing tool is a hard error -- a skipped assertion must never
look like a satisfied one.
Two build-system traps found by that gate, both of which had silently defeated a
fix: the product binary is compiled in one shot from sources, so a flag flip did
not rebuild it (now tracked by a .build-config stamp that also removes the
binary, making it independent of mtime granularity); and prod_sqlite3.o /
prod_mimalloc.o depended on a single named source, so SQLITE_OMIT_LOAD_EXTENSION
and the mimalloc patch BOTH compiled to nothing on the first incremental build.
Source review would have called them done.
Deliberately NOT changed. Three seams stay in release artifacts because
scripts/smoke-test.sh runs against the real artifact and needs them:
CBM_TEST_CRASH_ON and CBM_TEST_HANG_ON inject the faults that prove supervisor
recovery, and CBM_TEST_WINDOWS_USER_PATH_RUN_ID is what keeps the PATH smoke from
writing the tester's actual PATH. The gate treats those as an allowlist, so a
NOVEL seam still fails. The true no-UI standard build is deferred rather than
rushed: src/ui/* is in PROD_SRCS and four files outside src/ui reference UI
symbols, including the daemon that serves the UI, so that assertion reports
instead of failing until the split lands -- a gate everyone knows is red teaches
people to ignore gates.
No grammar is removed. ObjectScript accounts for essentially all binary growth
since the last provably-clean release (+21.4MB rodata, +1.1MB text from two
four-line shims), which made it the obvious ablation candidate, but a dry run
performed twelve real Defender endpoint scans across standard/UI and amd64/arm64
with ObjectScript, the daemon and the expanded hooks all present and every scan
was clean. Nothing there is a deterministic trigger, so cutting a
community-contributed language would spend a real feature on unproven margin.
Lean is not a candidate either: at 99.6MB of source it is by far the largest
grammar, but it shipped in v0.9.0 which scanned 20/20 clean, so removing it would
produce a novel unscanned profile instead of restoring a known-good one.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This reverts commit f843db8, which removed the #581 fix on a false
attribution. The arm64 shards were crashing because of an unaligned SQLite
page-cache slab (fixed in #1274, arm64 shard 2/2 green since); removing this
callback never changed those crashes, which is what exonerated it. The Windows
soak then failed for the obvious reason: the leak was back.
Static MinGW links have no DllMain and register no TLS callback, so
mi_thread_done never runs and every thread leaks its heap -- 607 heaps after 300
requests holding 170 MiB against a ~300 KiB live set. With the callback the
10-minute Windows soak measured 1.1x against the original 53x.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The hosted windows-11-arm shards crash at process load (STATUS_ILLEGAL_INSTRUCTION,
pass=0, secs=0) while main passes them, and removing the TLS detach callback did
not help -- so the remaining candidate is this layer, which touches every
allocation on Windows through the interposer. It declares a _Thread_local guard
read on every malloc, and thread-local storage in a static MinGW aarch64 link is
exactly the fragile case; nothing in it needs to ship.
The profiler did its job: it produced the attribution that identified #581's
mechanism, and that analysis is recorded. Sources stay in the tree, unbuilt, so
a future investigation can re-enable them deliberately.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reverts the #581 fix mechanism, not the diagnosis.
main's dry run passes both windows-11-arm shards and test-windows-guards; this
branch fails all three. The only change here that affects process LOAD is the
.CRT$XLB detach callback, and the arm64 failures are load-time crashes
(STATUS_ILLEGAL_INSTRUCTION, pass=0, secs=0, five suites in the same second) --
while the guards exercise many launcher and payload spawns. The same binary
loads fine on a UTM arm64 VM, so the placement is tolerated by some loaders and
not others, which is not something to ship on a supported platform.
#581's root cause stands and is documented: a static MinGW link has no DllMain
and registers no TLS callback, so mi_thread_done never runs and every thread
leaks its heap (607 heaps after 300 requests, 170 MiB against ~300 KiB live).
The delivery mechanism needs to be one the loader accepts everywhere -- an FLS
callback registered at runtime is the obvious candidate, with no linker section
placement involved.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reverts 3cd1d33. It was built on the theory that per-connection thread-heap
churn drove #581, and the measurements later falsified that: growth scaled with
requests rather than threads, and the real cause was thread heaps never being
released at all, fixed separately by the TLS detach callback.
It also broke a Windows regression guard -- test_daemon_stability
section_cold_storm, 'a fixed Windows bug is broken again' -- which is what
concurrent clients racing to spawn a daemon do to reworked reaping and shutdown
ordering. A concurrency rewrite that fixes nothing it claimed and reopens a
closed bug does not belong in a release-blocking PR.
The allocator thread-heap fix is untouched.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reading the allocator's per-thread bookkeeping races a thread exiting
concurrently: even mi_theap_get_default() reads state that _mi_thread_done
rewrites via _mi_theap_default_set, and TSan paired those two directly with the
read coming from the periodic diagnostics thread.
Splits the collector. cbm_mem_map_collect_os() fills OS totals only and is what
the periodic snapshot uses; the walking variant stays for tests and explicitly
requested diagnostics, where no other thread is exiting. What the OS-only form
cannot see is carried by the residual, as mem.h promises.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Calling mi_heap_main() is itself the race: it reads the allocator's main-heap
pointer while an exiting thread rewrites it via _mi_thread_done ->
_mi_theap_default_set. TSan paired those two directly on macOS, with the read
coming from the diagnostics thread. Removing the aggregate walk was not enough
because the abandoned-page walk still passed mi_heap_main() as its argument.
The map now touches only this thread's own theap. Everything it can no longer
see is carried by the residual, which is what mem.h promises.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Walking mi_heap_main() reads the process-wide aggregate heap while other
threads allocate into it, which is a data race by construction; TSan reported
it on macOS as init.c:452 in mi_heap_main. A diagnostic must not introduce a
race into the code it measures.
Now walks only what this thread may safely read: its own theap, plus abandoned
pages, which have no owning thread left to race with. The lost coverage is
carried by the residual, which is what it is for -- an unmeasured map must
never read as an empty one.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three CI failures, one cause each:
- Vendored integrity flagged a MISMATCH on the Windows mimalloc primitives.
The decommit counters added there were diagnostic and have served their
purpose -- 16k calls, zero failures, which retired the theory that purge was
silently failing. Restored to pristine, and the census fields they fed are
gone with them.
- GCC rejected an anonymous enum declared inside a struct ('declaration does
not declare anything'); clang accepted it, so it only broke the Ubuntu and
Windows smoke builds. Hoisted to file scope.
- clang-format across the files touched this session.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A static MinGW link has no DllMain and registers no TLS callback, so nothing
ran at thread exit and every thread left its mimalloc thread-heap behind for
the life of the process. After 300 MCP requests the allocator reported 607
thread-heaps, peak equal to total equal to current -- none ever released --
holding 170.5 MiB against a live set of roughly 300 KiB, growing linearly and
without bound (#581).
POSIX is unaffected because a pthread TSD destructor already calls this, which
is precisely why POSIX stayed flat under the identical workload. The heaps were
invisible to mi_heap_visit, which only walks the main and calling heaps, so the
growth appeared to belong to no allocator at all and survived every allocator
option that was tried against it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Growth is invariant to eleven allocator configurations, so the memory is not
the allocator's. The size profile of what accumulates distinguishes a swarm of
small commits from thread-stack-sized reservations.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The ARM64 fallback captured a return address inside the capture helper, so
every allocation collapsed onto the handful of hook addresses and the site
column named the profiler instead of the caller. Each hook now supplies its own
return address. The census also reports allocation volume, which separates
'the profiler sees nothing' from 'it sees traffic that is all freed'.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The hooks run inside malloc/free, so reading the env gate — which allocates —
re-entered them and recursed until the stack died. The process then produced no
stdout, no stderr and no log, which reads as the server refusing to start rather
than as a profiler defect. A thread-local guard makes the profiler invisible to
itself; the non-recursive mutex would also have deadlocked here.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Fixed corpus, fixed request count, one process, no daemon handshake and no
reindex or crash-recovery phases, so two runs differ only where the code does.
Requests go through a request/response driver rather than a batched pipe:
closing stdin makes the server treat it as a client disconnect and shut down
before answering, which yields a census of a process that never worked.
Linux gains a measurement-only --wrap shim so the profiler can observe
allocations the way the Windows interposer already does; macOS has no --wrap
and stays census-only, which the driver reports rather than passing off as
zero.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The census could say which POOL held memory but never which allocation put it
there, so each hypothesis needed a code change plus an eight-minute soak to
test and the investigation could only ask one yes/no question at a time.
Records allocations at or above a threshold (default 32 KiB, since arena
fragmentation is driven by large transient runs) against their captured stack,
into fixed tables whose exhaustion is counted and reported rather than silently
dropped. Emitted alongside every census sample, so pool totals and attribution
always describe the same instant, plus an explicit unattributed remainder so
incomplete coverage announces itself.
scripts/memlab-report.py ranks sites by retained-byte growth using a linear fit
and can diff two platforms' profiles against each other.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Each connection created its own OS thread, so the allocator minted a fresh
thread-heap every time. mimalloc then took new slices at its arena commit
frontier for each heap and never walked back to the purged ones behind it, so
Windows commit charge and RSS ratcheted upward for the life of the daemon
(#581). POSIX never showed it because madvise drops purged pages from RSS
regardless of where the frontier sits.
The worker slots were already pooled and bounded by max_clients; only the
threads were not. Each slot now starts its thread once and parks it on a
condition variable between connections, which keeps the set of allocator
thread-heaps constant. Slots are still released by the reaper as soon as a
connection finishes; threads are joined once, at shutdown.
Adds cbm_cond_t (pthread_cond_t / CONDITION_VARIABLE) since the pool needs a
wait primitive the codebase did not have.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel