Files
codebase-memory-mcp/src/foundation/diagnostics.c
T
Martin Vogel 1f674c805e harden: remove capability that should never have shipped
Microsoft's ML flagged the rc.1 release binaries. The decisive evidence is that
the SAME sha256 went from 0/62 clean to Microsoft-detected in about an hour with
no byte change, so the verdict lives partly in scanner-side state and no code
change can promise a clean result. What code CAN do is stop shipping things that
have no business in a release artifact, which is worth doing on its own merits
and incidentally widens the classifier margin. Every claim below is verified
against a built binary by the new gate, not by reading source.

Executable stack (the worst of the findings). vendored/nomic/code_vectors_blob.S
is the only assembly in the build and carried no .note.GNU-stack. An unannotated
object makes ld assume the worst for the whole link, so EVERY Linux release we
have ever shipped had GNU_STACK RWE. Adds the note (cause) plus ELF-only
-Wl,-z,noexecstack (outcome); the gate fails the release if it returns.

Test seams are now opt-in, never opt-out. TEST_SEAMS=1 defines
CBM_ENABLE_TEST_SEAMS; without it the crash-orphan probe -- which forks a child
that ignores SIGTERM and loops forever, then writes its pid to a caller-supplied
path -- and the lease-ownership marker compile to trivial stubs, so call sites are
untouched and the binary holds no fork, no signal handler and no env-var string.
Opt-IN is the point: forgetting the flag yields a clean binary rather than a leaky
one. scripts/test.sh requests it in the leg that consumes it, and
tests/test_worker_watchdog.sh now asserts the capability up front instead of
dying later with an opaque "Killed: 9".

The daemon's background version check is gone. It spawned curl against
api.github.com/repos/.../releases/latest on the first eligible session of every
run to say "a newer version exists" -- a release URL and an outbound request in
every shipped binary, for something the install scripts already report. The
INJECTABLE SEAM survives: update_ops is still honoured, the fakes in
tests/test_daemon_application.c still cover notice/ownership/cancellation/replay,
and with no provider application_update_subscribe_locked returns early so no
generation ever starts. "No network request by default" is now structural.

Dead capability out of release builds. The tar.gz/zip extraction block
(gzip_decompress through cbm_extract_binary_from_zip, plus its cli.h
declarations) moves under CBM_CLI_ENABLE_TEST_API -- verified self-contained, zero
uses of any helper outside it, only callers the excluded updater and
tests/test_cli.c. Downloading an archive, decompressing it, picking an executable
out of it and marking it executable is the canonical dropper composite; it is now
absent rather than merely unreachable. SQLite is built with
-DSQLITE_OMIT_LOAD_EXTENSION (no caller of load_extension anywhere in src/ or
internal/), removing that API surface and part of the dlopen/dlsym surface.

Temp files and environment scanning (S2/S3). Predictable paths in mcp.c,
artifact.c and diagnostics.c are created privately and exclusively and written
through the returned descriptor; pass_envscan.c no longer descends symlinked
directories out of the project root, and its fixed 512-byte path buffers no
longer truncate into pointer arithmetic that could land outside the buffer.

Build-time entropy. mimalloc's version banner baked __DATE__/__TIME__ into every
binary, so two builds of identical source seconds apart could never share a hash
and no release could inherit a false-positive determination made about its
predecessor. Local patch removes it (marked to survive refreshes), -Wdate-time
makes any future use a build error, and -Wl,--no-insert-timestamp stops the PE
header carrying the link clock.

scripts/ci/check-binary-composition.sh is the proof that each removal stays
removed, wired into package-release.sh after strip so the local artifact-flow
smoke enforces exactly what CI does. It asserts absences plus a CANARY string, so
handing it a compressed, stubbed or empty file fails instead of passing
vacuously, and a missing tool is a hard error -- a skipped assertion must never
look like a satisfied one.

Two build-system traps found by that gate, both of which had silently defeated a
fix: the product binary is compiled in one shot from sources, so a flag flip did
not rebuild it (now tracked by a .build-config stamp that also removes the
binary, making it independent of mtime granularity); and prod_sqlite3.o /
prod_mimalloc.o depended on a single named source, so SQLITE_OMIT_LOAD_EXTENSION
and the mimalloc patch BOTH compiled to nothing on the first incremental build.
Source review would have called them done.

Deliberately NOT changed. Three seams stay in release artifacts because
scripts/smoke-test.sh runs against the real artifact and needs them:
CBM_TEST_CRASH_ON and CBM_TEST_HANG_ON inject the faults that prove supervisor
recovery, and CBM_TEST_WINDOWS_USER_PATH_RUN_ID is what keeps the PATH smoke from
writing the tester's actual PATH. The gate treats those as an allowlist, so a
NOVEL seam still fails. The true no-UI standard build is deferred rather than
rushed: src/ui/* is in PROD_SRCS and four files outside src/ui reference UI
symbols, including the daemon that serves the UI, so that assertion reports
instead of failing until the split lands -- a gate everyone knows is red teaches
people to ignore gates.

No grammar is removed. ObjectScript accounts for essentially all binary growth
since the last provably-clean release (+21.4MB rodata, +1.1MB text from two
four-line shims), which made it the obvious ablation candidate, but a dry run
performed twelve real Defender endpoint scans across standard/UI and amd64/arm64
with ObjectScript, the daemon and the expanded hooks all present and every scan
was clean. Nothing there is a deterministic trigger, so cutting a
community-contributed language would spend a real feature on unproven margin.
Lean is not a candidate either: at 99.6MB of source it is by far the largest
grammar, but it shipped in v0.9.0 which scanned 20/20 clean, so removing it would
produce a novel unscanned profile instead of restoring a known-good one.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-29 23:50:53 +02:00

738 lines
28 KiB
C

/*
* diagnostics.c — Periodic diagnostics file writer.
*
* Every enabled process writes below a fresh owner-private directory. Files
* are created exclusively without following links, and POSIX I/O remains
* anchored to the validated directory descriptor. Diagnostics are strictly
* best-effort: shutdown is interruptible and bounded, so a stuck filesystem
* can never retain the daemon.
*/
#include "foundation/diagnostics.h"
#include "foundation/compat.h"
#include "foundation/compat_fs.h"
#include "foundation/compat_thread.h"
#include "foundation/constants.h"
#include "foundation/log.h"
#include "foundation/macos_acl.h"
#include "foundation/mem.h"
#include "foundation/platform.h"
#include "foundation/private_file_lock.h"
#include "foundation/private_file_lock_internal.h"
#include <mimalloc.h>
#include <stdatomic.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#ifdef _WIN32
#include "foundation/win_utf8.h"
#include <fcntl.h> /* _O_* for the exclusive stats-file create */
#include <io.h> /* _wopen / _close */
#include <process.h>
#include <sys/stat.h> /* _S_IREAD / _S_IWRITE */
#include <windows.h>
#define getpid _getpid
#else
#include <dirent.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#endif
enum {
DIAG_INTERVAL_MS = 5000,
DIAG_WAIT_SLICE_MS = 10,
DIAG_STOP_TIMEOUT_MS = 500,
};
#define DIAG_NDJSON_CAP_BYTES (8u * 1024u * 1024u)
#define DIAG_SNAPSHOT_NAME "snapshot.json"
#define DIAG_SNAPSHOT_TMP_NAME "snapshot.json.tmp"
#define DIAG_TRAJECTORY_NAME "trajectory.ndjson"
#define DIAG_TRAJECTORY_ROTATED_NAME "trajectory.ndjson.1"
/* ── Globals ────────────────────────────────────────────────────────────── */
cbm_query_stats_t g_query_stats = {0};
static atomic_bool g_diag_stop = false;
static atomic_bool g_diag_done = false;
static cbm_thread_t g_diag_thread;
static bool g_diag_started = false;
static bool g_diag_abandoned = false;
static time_t g_start_time = 0;
static cbm_private_lock_directory_t *g_diag_directory = NULL;
#ifndef _WIN32
static int g_diag_directory_fd = -1;
#endif
static char g_diag_directory_path[CBM_PATH_MAX] = "";
static char g_diag_path[CBM_PATH_MAX] = "";
static char g_diag_ndjson_path[CBM_PATH_MAX] = "";
static size_t g_diag_ndjson_size = 0;
#ifdef CBM_DIAGNOSTICS_ENABLE_TEST_API
static atomic_bool g_diag_test_hold_writer = false;
static atomic_bool g_diag_test_writer_reached = false;
#endif
/* ── Query stats ────────────────────────────────────────────────────────── */
void cbm_diag_record_query(long long duration_us, bool is_error) {
atomic_fetch_add(&g_query_stats.count, 1);
atomic_fetch_add(&g_query_stats.time_us, duration_us);
if (is_error) {
atomic_fetch_add(&g_query_stats.errors, 1);
}
long long old_max = atomic_load(&g_query_stats.max_us);
while (duration_us > old_max) {
if (atomic_compare_exchange_weak(&g_query_stats.max_us, &old_max, duration_us)) {
break;
}
}
}
/* ── FD count (platform-specific) ──────────────────────────────────────── */
static int count_open_fds(void) {
#ifdef __linux__
struct dirent **entries = NULL;
int n = scandir("/proc/self/fd", &entries, NULL, NULL);
if (n < 0) {
return CBM_NOT_FOUND;
}
for (int i = 0; i < n; i++) {
free(entries[i]);
}
free(entries);
return n - PAIR_LEN;
#elif defined(__APPLE__)
struct dirent **entries = NULL;
int n = scandir("/dev/fd", &entries, NULL, NULL);
if (n < 0) {
return CBM_NOT_FOUND;
}
for (int i = 0; i < n; i++) {
free(entries[i]);
}
free(entries);
return n - PAIR_LEN;
#else
return CBM_NOT_FOUND;
#endif
}
/* ── Private output directory ────────────────────────────────────────────── */
/* Capture the allocator's OWN accounting alongside the snapshot. This is the
* only measurement that separates "the allocator holds committed-but-free
* pages" (fragmentation / slice granularity) from "these blocks are still
* live" (a real leak) — opposite conclusions that process-level RSS and
* committed bytes cannot tell apart. Off unless CBM_MEM_STATS=1. */
static void diag_stats_write(const char *text, void *arg) {
FILE *sink = arg;
if (sink && text) {
(void)fputs(text, sink);
}
}
/* Create the stats file at its fixed, discoverable path without ever writing
* through something another process planted there. Exclusive creation IS the
* guarantee, so the predictable name stays safe: the unlink drops a stale file
* from an earlier run with this pid (and, if a local attacker pre-created a
* symlink, removes the link itself — never its target), and the O_EXCL create
* that follows fails closed if anything reappears at the path in between. The
* write therefore lands on a file this process just created, or not at all.
* O_NOFOLLOW is belt-and-braces for the same window on POSIX. Mode 0600: the
* snapshot describes this process's heap layout, so it is owner-only. */
static FILE *diag_open_private_stats_file(const char *path) {
(void)cbm_unlink(path);
#ifdef _WIN32
/* _wopen mirrors cbm_mkstemp's Windows contract — the ANSI CRT interprets
* the UTF-8 bytes of a non-ASCII %TEMP% in the local codepage and fails. */
wchar_t *wide = cbm_path_to_wide(path);
if (!wide) {
return NULL;
}
int descriptor = _wopen(wide, _O_WRONLY | _O_CREAT | _O_EXCL | _O_BINARY | _O_NOINHERIT,
_S_IREAD | _S_IWRITE);
free(wide);
if (descriptor < 0) {
return NULL;
}
FILE *sink = _fdopen(descriptor, "wb");
if (!sink) {
(void)_close(descriptor);
}
return sink;
#else
int flags = O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC;
#ifdef O_NOFOLLOW
flags |= O_NOFOLLOW;
#endif
int descriptor = open(path, flags, 0600);
if (descriptor < 0) {
return NULL;
}
FILE *sink = fdopen(descriptor, "wb");
if (!sink) {
(void)close(descriptor);
}
return sink;
#endif
}
static void diag_write_allocator_stats(void) {
char flag[CBM_SZ_16];
if (cbm_safe_getenv("CBM_MEM_STATS", flag, sizeof(flag), NULL) == NULL || flag[0] != '1') {
return;
}
char path[CBM_PATH_MAX];
/* Deliberately NOT inside the diagnostics directory: that tree is
* owner-private with anchored (openat-based) writes on POSIX, which a plain
* path-based open does not satisfy. This file is a developer diagnostic, so
* a predictable temp path keeps it working identically on every platform —
* diag_open_private_stats_file is what makes that path safe to write. */
int written =
snprintf(path, sizeof(path), "%s/cbm-allocator-stats-%d.txt", cbm_tmpdir(), (int)getpid());
if (written <= 0 || (size_t)written >= sizeof(path)) {
return;
}
FILE *sink = diag_open_private_stats_file(path);
if (!sink) {
return;
}
mi_stats_print_out(diag_stats_write, sink);
/* Arena slice map: the legend distinguishes free-committed (retained, "_")
* from free-reserved (already returned to the OS, ".") and free-purgeable
* ("~"). That is what localises retained memory to a specific arena state
* instead of inferring it from totals. mi_debug_show_arenas writes through
* mimalloc's own output hook, so redirect that into this file. */
(void)fputs("\n--- arenas ---\n", sink);
mi_register_output(diag_stats_write, sink);
mi_debug_show_arenas();
mi_register_output(NULL, NULL);
(void)fputs("\n--- options ---\n", sink);
mi_options_print_out(diag_stats_write, sink);
(void)fclose(sink);
}
static bool diag_set_output_paths(void) {
int snapshot = snprintf(g_diag_path, sizeof(g_diag_path), "%s/%s", g_diag_directory_path,
DIAG_SNAPSHOT_NAME);
int trajectory = snprintf(g_diag_ndjson_path, sizeof(g_diag_ndjson_path), "%s/%s",
g_diag_directory_path, DIAG_TRAJECTORY_NAME);
return snapshot > 0 && (size_t)snapshot < sizeof(g_diag_path) && trajectory > 0 &&
(size_t)trajectory < sizeof(g_diag_ndjson_path);
}
static void diag_directory_close(bool remove_directory) {
#ifndef _WIN32
if (g_diag_directory_fd >= 0) {
(void)close(g_diag_directory_fd);
g_diag_directory_fd = -1;
}
#endif
cbm_private_lock_directory_close(g_diag_directory);
g_diag_directory = NULL;
if (remove_directory && g_diag_directory_path[0] != '\0') {
(void)cbm_rmdir(g_diag_directory_path);
}
}
/* Diagnostics placement honors the documented contract ($TMPDIR, /tmp
* fallback) without touching cbm_tmpdir(), whose many other callers keep
* their established behavior. Windows already resolves the real user temp. */
static const char *diag_tmp_base(char *buffer, size_t size) {
#ifdef _WIN32
(void)buffer;
(void)size;
return cbm_tmpdir();
#else
cbm_safe_getenv("TMPDIR", buffer, size, NULL);
if (buffer[0] != '\0') {
return buffer;
}
return "/tmp";
#endif
}
static bool diag_directory_prepare(void) {
char tmp_base[CBM_PATH_MAX];
char directory_template[CBM_PATH_MAX];
int written =
snprintf(directory_template, sizeof(directory_template), "%s/cbm-diagnostics-%d-XXXXXX",
diag_tmp_base(tmp_base, sizeof(tmp_base)), (int)getpid());
if (written <= 0 || (size_t)written >= sizeof(directory_template) ||
!cbm_mkdtemp(directory_template)) {
return false;
}
written =
snprintf(g_diag_directory_path, sizeof(g_diag_directory_path), "%s", directory_template);
if (written <= 0 || (size_t)written >= sizeof(g_diag_directory_path)) {
(void)cbm_rmdir(directory_template);
return false;
}
#ifdef _WIN32
wchar_t *wide_directory = cbm_path_to_wide(g_diag_directory_path);
HANDLE handle =
wide_directory
? CreateFileW(wide_directory, FILE_READ_ATTRIBUTES | READ_CONTROL,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL,
OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OPEN_REPARSE_POINT,
NULL)
: INVALID_HANDLE_VALUE;
free(wide_directory);
cbm_private_file_lock_status_t status = CBM_PRIVATE_FILE_LOCK_IO;
if (handle != INVALID_HANDLE_VALUE) {
status = cbm_private_lock_directory_adopt_windows(handle, g_diag_directory_path,
&g_diag_directory);
if (status != CBM_PRIVATE_FILE_LOCK_OK) {
(void)CloseHandle(handle);
}
}
#else
int flags = O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW;
int validation_fd = open(g_diag_directory_path, flags);
int output_fd = open(g_diag_directory_path, flags);
struct stat validation_state;
struct stat output_state;
bool same_directory =
validation_fd >= 0 && output_fd >= 0 && fstat(validation_fd, &validation_state) == 0 &&
fstat(output_fd, &output_state) == 0 && validation_state.st_dev == output_state.st_dev &&
validation_state.st_ino == output_state.st_ino;
cbm_private_file_lock_status_t status = CBM_PRIVATE_FILE_LOCK_IO;
if (same_directory) {
status = cbm_private_lock_directory_adopt_posix(validation_fd, g_diag_directory_path,
&g_diag_directory);
if (status == CBM_PRIVATE_FILE_LOCK_OK) {
validation_fd = -1;
g_diag_directory_fd = output_fd;
output_fd = -1;
}
}
if (validation_fd >= 0) {
(void)close(validation_fd);
}
if (output_fd >= 0) {
(void)close(output_fd);
}
#endif
if (status != CBM_PRIVATE_FILE_LOCK_OK || !diag_set_output_paths()) {
diag_directory_close(true);
return false;
}
return true;
}
#ifdef _WIN32
static bool diag_full_path(const char *base_name, char *path, size_t path_size) {
int written = snprintf(path, path_size, "%s/%s", g_diag_directory_path, base_name);
return written > 0 && (size_t)written < path_size;
}
static bool diag_win_handle_valid(HANDLE handle) {
BY_HANDLE_FILE_INFORMATION information;
return handle != INVALID_HANDLE_VALUE && GetFileType(handle) == FILE_TYPE_DISK &&
GetFileInformationByHandle(handle, &information) != 0 &&
(information.dwFileAttributes &
(FILE_ATTRIBUTE_DIRECTORY | FILE_ATTRIBUTE_REPARSE_POINT)) == 0 &&
information.nNumberOfLinks == 1 &&
SetHandleInformation(handle, HANDLE_FLAG_INHERIT, 0) != 0;
}
static bool diag_write_file(const char *base_name, const char *data, size_t length, bool append) {
char path[CBM_PATH_MAX];
if (!diag_full_path(base_name, path, sizeof(path))) {
return false;
}
wchar_t *wide_path = cbm_path_to_wide(path);
DWORD access = append ? FILE_APPEND_DATA : GENERIC_WRITE;
HANDLE handle = wide_path
? CreateFileW(wide_path, access, FILE_SHARE_READ, NULL, CREATE_NEW,
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OPEN_REPARSE_POINT, NULL)
: INVALID_HANDLE_VALUE;
DWORD creation_error = handle == INVALID_HANDLE_VALUE ? GetLastError() : ERROR_SUCCESS;
if (handle == INVALID_HANDLE_VALUE && wide_path &&
(creation_error == ERROR_FILE_EXISTS || creation_error == ERROR_ALREADY_EXISTS)) {
handle = CreateFileW(wide_path, access, FILE_SHARE_READ, NULL,
append ? OPEN_EXISTING : TRUNCATE_EXISTING,
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OPEN_REPARSE_POINT, NULL);
}
free(wide_path);
if (!diag_win_handle_valid(handle)) {
if (handle != INVALID_HANDLE_VALUE) {
(void)CloseHandle(handle);
}
return false;
}
bool ok = true;
size_t offset = 0;
while (offset < length) {
DWORD chunk = length - offset > MAXDWORD ? MAXDWORD : (DWORD)(length - offset);
DWORD bytes_written = 0;
if (!WriteFile(handle, data + offset, chunk, &bytes_written, NULL) || bytes_written == 0) {
ok = false;
break;
}
offset += bytes_written;
}
return CloseHandle(handle) != 0 && ok;
}
static bool diag_native_rename(const char *source_name, const char *destination_name) {
char source[CBM_PATH_MAX];
char destination[CBM_PATH_MAX];
return diag_full_path(source_name, source, sizeof(source)) &&
diag_full_path(destination_name, destination, sizeof(destination)) &&
cbm_rename_replace(source, destination) == 0;
}
static void diag_native_unlink(const char *base_name) {
char path[CBM_PATH_MAX];
if (diag_full_path(base_name, path, sizeof(path))) {
(void)cbm_unlink(path);
}
}
#else
static bool diag_posix_file_valid(int descriptor) {
struct stat state;
return descriptor >= 0 && fstat(descriptor, &state) == 0 && S_ISREG(state.st_mode) &&
state.st_uid == geteuid() && state.st_nlink == 1 && (state.st_mode & 07777) == 0600 &&
cbm_macos_extended_acl_fd_is_empty(descriptor);
}
static bool diag_write_file(const char *base_name, const char *data, size_t length, bool append) {
int flags = O_WRONLY | O_CLOEXEC | O_NOFOLLOW | O_NONBLOCK;
flags |= append ? O_APPEND : O_TRUNC;
int descriptor = openat(g_diag_directory_fd, base_name, flags | O_CREAT | O_EXCL, 0600);
bool created = descriptor >= 0;
if (descriptor < 0 && errno == EEXIST) {
descriptor = openat(g_diag_directory_fd, base_name, flags);
}
if (created && fchmod(descriptor, 0600) != 0) {
(void)close(descriptor);
return false;
}
if (!diag_posix_file_valid(descriptor)) {
if (descriptor >= 0) {
(void)close(descriptor);
}
return false;
}
bool ok = true;
size_t offset = 0;
while (offset < length) {
ssize_t bytes_written = write(descriptor, data + offset, length - offset);
if (bytes_written > 0) {
offset += (size_t)bytes_written;
} else if (bytes_written < 0 && errno == EINTR) {
continue;
} else {
ok = false;
break;
}
}
return close(descriptor) == 0 && ok;
}
static bool diag_native_rename(const char *source_name, const char *destination_name) {
return renameat(g_diag_directory_fd, source_name, g_diag_directory_fd, destination_name) == 0;
}
static void diag_native_unlink(const char *base_name) {
(void)unlinkat(g_diag_directory_fd, base_name, 0);
}
#endif
/* ── Writer ─────────────────────────────────────────────────────────────────────── */
static void diag_sleep_ms(unsigned int milliseconds) {
struct timespec delay = {
.tv_sec = (time_t)(milliseconds / 1000U),
.tv_nsec = (long)((milliseconds % 1000U) * 1000000U),
};
(void)cbm_nanosleep(&delay, NULL);
}
static void diag_wait_for_interval(void) {
uint64_t deadline = cbm_now_ms() + DIAG_INTERVAL_MS;
while (!atomic_load_explicit(&g_diag_stop, memory_order_acquire)) {
uint64_t now = cbm_now_ms();
if (now >= deadline) {
return;
}
uint64_t remaining = deadline - now;
unsigned int slice =
remaining < DIAG_WAIT_SLICE_MS ? (unsigned int)remaining : DIAG_WAIT_SLICE_MS;
diag_sleep_ms(slice > 0 ? slice : 1);
}
}
#ifdef CBM_DIAGNOSTICS_ENABLE_TEST_API
static void diag_test_pause_writer(void) {
if (!atomic_load_explicit(&g_diag_test_hold_writer, memory_order_acquire)) {
return;
}
atomic_store_explicit(&g_diag_test_writer_reached, true, memory_order_release);
while (atomic_load_explicit(&g_diag_test_hold_writer, memory_order_acquire)) {
diag_sleep_ms(DIAG_WAIT_SLICE_MS);
}
}
#endif
static void append_trajectory(long uptime, size_t rss, size_t peak_rss, size_t commit,
size_t peak_commit, size_t page_faults, int fds, int qcount) {
if (g_diag_ndjson_size > DIAG_NDJSON_CAP_BYTES) {
if (!diag_native_rename(DIAG_TRAJECTORY_NAME, DIAG_TRAJECTORY_ROTATED_NAME)) {
return;
}
g_diag_ndjson_size = 0;
}
char line[CBM_SZ_512];
int length = snprintf(line, sizeof(line),
"{\"uptime_s\":%ld,\"rss\":%zu,\"peak_rss\":%zu,\"committed\":%zu,"
"\"peak_committed\":%zu,\"page_faults\":%zu,\"fd\":%d,\"queries\":%d}\n",
uptime, rss, peak_rss, commit, peak_commit, page_faults, fds, qcount);
if (length <= 0 || (size_t)length >= sizeof(line) ||
!diag_write_file(DIAG_TRAJECTORY_NAME, line, (size_t)length, true)) {
return;
}
g_diag_ndjson_size += (size_t)length;
}
static void write_diagnostics(void) {
#ifdef CBM_DIAGNOSTICS_ENABLE_TEST_API
diag_test_pause_writer();
#endif
if (atomic_load_explicit(&g_diag_stop, memory_order_acquire)) {
return;
}
size_t elapsed_ms = 0;
size_t user_ms = 0;
size_t sys_ms = 0;
size_t current_rss = 0;
size_t peak_rss = 0;
size_t current_commit = 0;
size_t peak_commit = 0;
size_t page_faults = 0;
mi_process_info(&elapsed_ms, &user_ms, &sys_ms, &current_rss, &peak_rss, &current_commit,
&peak_commit, &page_faults);
if (current_rss == 0) {
current_rss = cbm_mem_rss();
}
int fds = count_open_fds();
time_t now = time(NULL);
long uptime = (long)(now - g_start_time);
int qcount = atomic_load(&g_query_stats.count);
int qerrors = atomic_load(&g_query_stats.errors);
long long qtime = atomic_load(&g_query_stats.time_us);
long long qmax = atomic_load(&g_query_stats.max_us);
long long qavg = qcount > 0 ? qtime / qcount : 0;
/* Memory map: live allocator bytes on this thread's heap, a size-class
* histogram, and the residual the walk does NOT account for. The residual
* is what keeps this honest — see mem.h. */
diag_write_allocator_stats();
cbm_mem_map_t map;
(void)cbm_mem_map_collect_os(&map);
size_t residual =
map.os_committed_bytes > map.live_bytes ? map.os_committed_bytes - map.live_bytes : 0;
char buckets[CBM_SZ_512];
int bucket_length = 0;
for (int i = 0;
i < CBM_MEM_MAP_BUCKETS && bucket_length >= 0 && (size_t)bucket_length < sizeof(buckets);
i++) {
int written =
snprintf(buckets + bucket_length, sizeof(buckets) - (size_t)bucket_length,
"%s{\"limit\": %zu, \"bytes\": %zu, \"blocks\": %zu}", i == 0 ? "" : ", ",
cbm_mem_map_bucket_limit(i), map.bucket_bytes[i], map.bucket_blocks[i]);
if (written < 0) {
break;
}
bucket_length += written;
}
/* Phase attribution: which bracketed code path the committed bytes stayed
* in. Empty unless CBM_MEM_PHASES=1. */
char phases[CBM_SZ_1K];
if (cbm_mem_phase_report_json(phases, sizeof(phases)) <= 0) {
phases[0] = '\0';
}
char snapshot[CBM_SZ_4K];
int length =
snprintf(snapshot, sizeof(snapshot),
"{\n"
" \"uptime_s\": %ld,\n"
" \"rss_bytes\": %zu,\n"
" \"peak_rss_bytes\": %zu,\n"
" \"heap_committed_bytes\": %zu,\n"
" \"peak_committed_bytes\": %zu,\n"
" \"page_faults\": %zu,\n"
" \"fd_count\": %d,\n"
" \"query_count\": %d,\n"
" \"query_errors\": %d,\n"
" \"query_total_us\": %lld,\n"
" \"query_avg_us\": %lld,\n"
" \"query_max_us\": %lld,\n"
" \"mem_malloc_owned\": %s,\n"
" \"mem_live_bytes\": %zu,\n"
" \"mem_live_blocks\": %zu,\n"
" \"mem_area_committed_bytes\": %zu,\n"
" \"mem_residual_bytes\": %zu,\n"
" \"mem_buckets\": [%s],\n"
" \"mem_phases\": [%s],\n"
" \"pid\": %d\n"
"}\n",
uptime, current_rss, peak_rss, current_commit, peak_commit, page_faults, fds,
qcount, qerrors, qtime, qavg, qmax,
map.malloc_is_allocator_owned ? "true" : "false", map.live_bytes, map.live_blocks,
map.area_committed_bytes, residual, buckets, phases, (int)getpid());
if (length <= 0 || (size_t)length >= sizeof(snapshot) ||
!diag_write_file(DIAG_SNAPSHOT_TMP_NAME, snapshot, (size_t)length, false) ||
!diag_native_rename(DIAG_SNAPSHOT_TMP_NAME, DIAG_SNAPSHOT_NAME)) {
diag_native_unlink(DIAG_SNAPSHOT_TMP_NAME);
return;
}
if (!atomic_load_explicit(&g_diag_stop, memory_order_acquire)) {
append_trajectory(uptime, current_rss, peak_rss, current_commit, peak_commit, page_faults,
fds, qcount);
}
}
static void *diag_thread_fn(void *arg) {
(void)arg;
while (!atomic_load_explicit(&g_diag_stop, memory_order_acquire)) {
write_diagnostics();
if (!atomic_load_explicit(&g_diag_stop, memory_order_acquire)) {
diag_wait_for_interval();
}
}
atomic_store_explicit(&g_diag_done, true, memory_order_release);
return NULL;
}
static void diag_cleanup_live_files(void) {
diag_native_unlink(DIAG_SNAPSHOT_NAME);
diag_native_unlink(DIAG_SNAPSHOT_TMP_NAME);
}
/* ── Public API ──────────────────────────────────────────────────────── */
bool cbm_diag_start(void) {
char env_buf[CBM_SZ_32] = "";
cbm_safe_getenv("CBM_DIAGNOSTICS", env_buf, sizeof(env_buf), NULL);
if (env_buf[0] == '\0' || (strcmp(env_buf, "1") != 0 && strcmp(env_buf, "true") != 0) ||
g_diag_started || g_diag_abandoned) {
return false;
}
g_start_time = time(NULL);
g_diag_ndjson_size = 0;
atomic_store_explicit(&g_diag_stop, false, memory_order_release);
atomic_store_explicit(&g_diag_done, false, memory_order_release);
#ifdef CBM_DIAGNOSTICS_ENABLE_TEST_API
atomic_store_explicit(&g_diag_test_writer_reached, false, memory_order_release);
#endif
if (!diag_directory_prepare()) {
return false;
}
if (cbm_thread_create(&g_diag_thread, 0, diag_thread_fn, NULL) != 0) {
diag_directory_close(true);
return false;
}
g_diag_started = true;
char interval[CBM_SZ_32];
(void)snprintf(interval, sizeof(interval), "%d", DIAG_INTERVAL_MS / 1000);
/* Discovery must survive CBM_LOG_LEVEL suppression and paths containing
* spaces: a control record is the only announced copy of these paths. */
cbm_log_control("diagnostics.start", "snapshot", g_diag_path, "trajectory", g_diag_ndjson_path,
"interval_s", interval);
return true;
}
void cbm_diag_stop(void) {
if (!g_diag_started) {
return;
}
atomic_store_explicit(&g_diag_stop, true, memory_order_release);
uint64_t deadline = cbm_now_ms() + DIAG_STOP_TIMEOUT_MS;
while (!atomic_load_explicit(&g_diag_done, memory_order_acquire) && cbm_now_ms() < deadline) {
diag_sleep_ms(DIAG_WAIT_SLICE_MS);
}
bool completed = atomic_load_explicit(&g_diag_done, memory_order_acquire);
if (completed) {
/* The completion publication is the final access to diagnostics
* state. Detaching avoids turning an already-complete best-effort
* writer into an unbounded native join during daemon teardown. */
(void)cbm_thread_detach(&g_diag_thread);
diag_cleanup_live_files();
diag_directory_close(false);
} else {
/* The thread may still hold a native file or directory descriptor.
* Detach it and intentionally retain all static state until process
* exit; closing or reusing those handles would create an fd/handle
* reuse race if the stalled operation later resumes. */
(void)cbm_thread_detach(&g_diag_thread);
g_diag_abandoned = true;
}
g_diag_started = false;
}
#ifdef CBM_DIAGNOSTICS_ENABLE_TEST_API
static bool diag_test_copy(char *destination, size_t destination_size, const char *source) {
if (!destination || destination_size == 0) {
return false;
}
int written = snprintf(destination, destination_size, "%s", source);
return written >= 0 && (size_t)written < destination_size;
}
bool cbm_diag_test_copy_paths(char *directory, size_t directory_size, char *snapshot,
size_t snapshot_size, char *trajectory, size_t trajectory_size) {
return diag_test_copy(directory, directory_size, g_diag_directory_path) &&
diag_test_copy(snapshot, snapshot_size, g_diag_path) &&
diag_test_copy(trajectory, trajectory_size, g_diag_ndjson_path);
}
void cbm_diag_test_hold_writer(bool hold) {
atomic_store_explicit(&g_diag_test_hold_writer, hold, memory_order_release);
}
bool cbm_diag_test_writer_reached(void) {
return atomic_load_explicit(&g_diag_test_writer_reached, memory_order_acquire);
}
bool cbm_diag_test_abandoned(void) {
return g_diag_abandoned;
}
bool cbm_diag_test_reset_abandoned(void) {
if (!g_diag_abandoned || g_diag_started ||
!atomic_load_explicit(&g_diag_done, memory_order_acquire)) {
return false;
}
diag_cleanup_live_files();
diag_directory_close(false);
g_diag_abandoned = false;
return true;
}
#endif