Files
codebase-memory-mcp/internal/cbm/cbm.h
T
Joshua RichterandClaude Opus 5 a953509275 fix(coverage): stop the report hiding what it dropped, and name whole-file failures (#963)
Two silent failures in the parse-coverage report, both made visible by the
Phase 2 range refinement that came before this.

Two caps sat in series and both returned early without saying anything:

    CBM_MAX_ERROR_REGIONS = 64   internal/cbm/cbm.c
    COVERAGE_RANGE_MAX   = 128   src/mcp/mcp.c

Raising only the first would have moved the clip from 64 to 128, so both move
to 256. This was live behaviour, not a theoretical limit: after Phase 2 split
one whole-file range into many small ones, src/cli/cli.c and tests/test_cli.c
both reported exactly 64 ranges — the cap binding, dead-on, twice. Every
coverage figure measured before this change was a floor. With the cap at 256
the true numbers are cli.c 13.9% (not 9.8%) and test_cli.c 3.1%, and the
longest list in the repo is 85 ranges.

A raised cap is still a cap, so the report now says when it clipped:

- cbm_error_regions_t gained a `dropped` counter, and cbm_collect_error_regions
  walks to the end instead of stopping at the cap, so the count is exact rather
  than a lower bound. That costs little — the walk never descends into an ERROR
  subtree.
- cbm_error_ranges_str appends ",+<N>" when N ranges were thrown away.
- coverage_add_ranges reads that marker and sets "truncated": true, and also
  sets it when its own limit stops the loop. Before this the marker was
  invisible: the parser stopped at the '+' with no error and no leftover, so a
  clipped list arrived looking complete.
- objectscript_export_append_error_ranges strips markers off both operands
  before joining two Studio Export parts and adds one back at the end. A marker
  left mid-string would make every reader stop there and silently lose every
  range after it.

"Look at lines 1 to 13047" of a 13046-line file tells a reader nothing. Those
files now carry their own kind rather than being described as partially
covered.

New `parse_unusable` field in CBMFileResult, set when one range covers 80% or
more of the file. Its customers are non-C languages: the Phase 2 refinement
that narrows a whole-file range using the preprocessed parse only runs for C,
C++ and CUDA, so a Python, Java, Ruby or TypeScript file whose root node is
ERROR still reports 1-N. Verified against real files in all four.

The kind is `parse_unusable`, not `parse_failed`. index_coverage.kind already
means one of two things — indexed-but-partial, or a skip phase saying the file
was never indexed at all — and `parse_failed` reads as the second when it is
the first. The store.c schema comment, which is the only written record of this
vocabulary, now describes all three classes and says why.

Two places would have mislabelled the new kind as "skipped", which is exactly
that confusion: coverage_status fell through to its catch-all pass, and
add_coverage_report fell into its else branch. A reader who finds a file under
"skipped" believes it is absent from the graph, when it was indexed. Both now
have explicit branches. index_status gained parse_unusable_count so a CI gate
can read it without parsing anything else, get_code_snippet says "read the
source directly" instead of naming useless ranges, and the three tool
descriptions that listed two coverage kinds now list three.

Seven added. The cap test moved from 64 to 256; a new test asserts the marker
carries a real drop count and that nothing follows it; an inverse test asserts
an under-cap file carries no marker at all. For the new kind: a Python file
whose root is ERROR is unusable, a file with a local parse failure stays
partial, a clean file is neither, and — the one that matters most — the
that last one ever flips, the Phase 2 refinement has stopped working.

Full suite: 7732 passed, 28 failed, 7 skipped. The 28 are pre-existing
agent-client install/uninstall failures in the cli suite, identical in count
and identity at clean HEAD.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
2026-09-05 16:01:05 -04:00

818 lines
33 KiB
C

#ifndef CBM_H
#define CBM_H
#include <stdint.h>
#include <stdbool.h>
#include "arena.h"
#include "tree_sitter/api.h"
// Language enum mirrors lang.Language in Go.
// Order must match lang_specs.c tables.
typedef enum {
CBM_LANG_GO = 0,
CBM_LANG_PYTHON,
CBM_LANG_JAVASCRIPT,
CBM_LANG_TYPESCRIPT,
CBM_LANG_TSX,
CBM_LANG_RUST,
CBM_LANG_JAVA,
CBM_LANG_CPP,
CBM_LANG_CSHARP,
CBM_LANG_PHP,
CBM_LANG_LUA,
CBM_LANG_SCALA,
CBM_LANG_KOTLIN,
CBM_LANG_RUBY,
CBM_LANG_C,
CBM_LANG_BASH,
CBM_LANG_ZIG,
CBM_LANG_ELIXIR,
CBM_LANG_HASKELL,
CBM_LANG_OCAML,
CBM_LANG_OBJC,
CBM_LANG_SWIFT,
CBM_LANG_DART,
CBM_LANG_PERL,
CBM_LANG_GROOVY,
CBM_LANG_ERLANG,
CBM_LANG_R,
CBM_LANG_HTML,
CBM_LANG_CSS,
CBM_LANG_SCSS,
CBM_LANG_YAML,
CBM_LANG_TOML,
CBM_LANG_HCL,
CBM_LANG_SQL,
CBM_LANG_DOCKERFILE,
// New languages (v0.5 expansion)
CBM_LANG_CLOJURE,
CBM_LANG_FSHARP,
CBM_LANG_JULIA,
CBM_LANG_VIMSCRIPT,
CBM_LANG_NIX,
CBM_LANG_COMMONLISP,
CBM_LANG_ELM,
CBM_LANG_FORTRAN,
CBM_LANG_CUDA,
CBM_LANG_COBOL,
CBM_LANG_VERILOG,
CBM_LANG_EMACSLISP,
CBM_LANG_JSON,
CBM_LANG_XML,
CBM_LANG_MARKDOWN,
CBM_LANG_MAKEFILE,
CBM_LANG_CMAKE,
CBM_LANG_PROTOBUF,
CBM_LANG_GRAPHQL,
CBM_LANG_VUE,
CBM_LANG_SVELTE,
CBM_LANG_MESON,
CBM_LANG_GLSL,
CBM_LANG_INI,
// Scientific/math languages
CBM_LANG_MATLAB,
CBM_LANG_LEAN,
CBM_LANG_FORM,
CBM_LANG_MAGMA,
CBM_LANG_WOLFRAM,
CBM_LANG_SOLIDITY,
CBM_LANG_TYPST,
CBM_LANG_GDSCRIPT,
CBM_LANG_GLEAM,
CBM_LANG_POWERSHELL,
CBM_LANG_PASCAL,
CBM_LANG_DLANG,
CBM_LANG_NIM,
CBM_LANG_SCHEME,
CBM_LANG_FENNEL,
CBM_LANG_FISH,
CBM_LANG_AWK,
CBM_LANG_ZSH,
CBM_LANG_TCL,
CBM_LANG_ADA,
CBM_LANG_AGDA,
CBM_LANG_RACKET,
CBM_LANG_ODIN,
CBM_LANG_RESCRIPT,
CBM_LANG_PURESCRIPT,
CBM_LANG_NICKEL,
CBM_LANG_CRYSTAL,
CBM_LANG_TEAL,
CBM_LANG_HARE,
CBM_LANG_PONY,
CBM_LANG_LUAU,
CBM_LANG_JANET,
CBM_LANG_SWAY,
CBM_LANG_NASM,
CBM_LANG_ASSEMBLY,
CBM_LANG_ASTRO,
CBM_LANG_BLADE,
CBM_LANG_JUST,
CBM_LANG_GOTEMPLATE,
CBM_LANG_TEMPL,
CBM_LANG_LIQUID,
CBM_LANG_JINJA2,
CBM_LANG_PRISMA,
CBM_LANG_HYPRLANG,
CBM_LANG_DOTENV,
CBM_LANG_DIFF,
CBM_LANG_WGSL,
CBM_LANG_KDL,
CBM_LANG_JSON5,
CBM_LANG_JSONNET,
CBM_LANG_RON,
CBM_LANG_THRIFT,
CBM_LANG_CAPNP,
CBM_LANG_PROPERTIES,
CBM_LANG_SSHCONFIG,
CBM_LANG_BIBTEX,
CBM_LANG_STARLARK,
CBM_LANG_BICEP,
CBM_LANG_CSV,
CBM_LANG_REQUIREMENTS,
CBM_LANG_HLSL,
CBM_LANG_VHDL,
CBM_LANG_SYSTEMVERILOG,
CBM_LANG_DEVICETREE,
CBM_LANG_LINKERSCRIPT,
CBM_LANG_GN,
CBM_LANG_KCONFIG,
CBM_LANG_BITBAKE,
CBM_LANG_SMALI,
CBM_LANG_TABLEGEN,
CBM_LANG_ISPC,
CBM_LANG_CAIRO,
CBM_LANG_MOVE,
CBM_LANG_SQUIRREL,
CBM_LANG_FUNC,
CBM_LANG_REGEX,
CBM_LANG_JSDOC,
CBM_LANG_RST,
CBM_LANG_BEANCOUNT,
CBM_LANG_MERMAID,
CBM_LANG_PUPPET,
CBM_LANG_PO,
CBM_LANG_GITATTRIBUTES,
CBM_LANG_GITIGNORE,
CBM_LANG_SLANG,
CBM_LANG_LLVM_IR,
CBM_LANG_SMITHY,
CBM_LANG_WIT,
CBM_LANG_TLAPLUS,
CBM_LANG_PKL,
CBM_LANG_GOMOD,
CBM_LANG_APEX,
CBM_LANG_SOQL,
CBM_LANG_SOSL,
CBM_LANG_KUSTOMIZE, // kustomization.yaml — Kubernetes overlay tool
CBM_LANG_K8S, // Generic Kubernetes manifest (apiVersion: detected)
CBM_LANG_PINE, // Pine Script (TradingView indicator / strategy language)
CBM_LANG_QML, // Qt QML (Qt Modeling Language — declarative UI + embedded JS)
CBM_LANG_CFSCRIPT, // CFML script dialect (.cfc components — Lucee/ColdFusion)
CBM_LANG_CFML, // CFML tag dialect (.cfm templates — Lucee/ColdFusion)
CBM_LANG_MOJO, // Mojo
CBM_LANG_OBJECTSCRIPT_UDL, // InterSystems ObjectScript UDL (.cls class files)
CBM_LANG_OBJECTSCRIPT_ROUTINE, // InterSystems ObjectScript routine (.mac/.int/.rtn/.inc)
CBM_LANG_OBJECTSCRIPT_EXPORT, // InterSystems Studio Export XML (<Export generator="Cache">)
CBM_LANG_ARKTS, // ArkTS (HarmonyOS/OpenHarmony .ets — TypeScript superset + ArkUI)
CBM_LANG_PLSQL, // Oracle PL/SQL
CBM_LANG_CHIALISP, // Chialisp (.clsp/.clib/.clinc — Chia smart-coin s-expression language)
CBM_LANG_COUNT
} CBMLanguage;
// --- Extraction result structs ---
typedef struct {
const char *name; // short name
const char *qualified_name; // project.path.name
const char *label; // "Function", "Method", "Class", "Variable", "Module"
const char *file_path; // relative path
uint32_t start_line;
uint32_t end_line;
const char *signature; // parameter text (NULL if none)
const char *return_type; // return type text (NULL if none)
const char *receiver; // Go method receiver (NULL if none)
const char *docstring; // leading doc comment (NULL if none)
const char *parent_class; // enclosing class QN for methods (NULL if none)
const char **decorators; // NULL-terminated array (NULL if none)
const char **base_classes; // NULL-terminated array (NULL if none)
const char **param_names; // NULL-terminated array (NULL if none)
const char **param_types; // NULL-terminated array (NULL if none)
const char **signature_param_types; // ordered internal signature types; "?" means unknown
int signature_param_count; // number of entries in signature_param_types
const char **return_types; // NULL-terminated array (NULL if none)
const char *route_path; // HTTP route path from decorator (e.g., "/api/users") or NULL
const char *route_method; // HTTP method from decorator (e.g., "POST") or NULL
int complexity; // cyclomatic complexity
int cognitive; // cognitive complexity (nesting-weighted)
int loop_count; // number of loop constructs in the body
int loop_depth; // max nested-loop depth (bottleneck proxy)
bool is_recursive; // body contains a direct self-call (seed for "recursive")
int param_count; // number of parameters (large = complexity smell)
int max_access_depth; // deepest chained member/subscript access (a.b.c.d)
int linear_scan_in_loop; // count of linear-scan calls (find/contains/indexOf) inside loops
int alloc_in_loop; // count of allocation/append calls inside loops
bool recursion_in_loop; // a self-call occurs inside a loop body
bool unguarded_recursion; // recursive with no self-call guarded by a conditional
int lines; // body line count
uint32_t *fingerprint; // MinHash fingerprint (arena-allocated, K values) or NULL
int fingerprint_k; // number of hash values (CBM_MINHASH_K or 0)
bool is_exported;
bool is_abstract;
bool is_test;
bool is_entry_point;
const char *structural_profile; // AST structural profile (arena-allocated) or NULL
const char *body_tokens; // space-separated raw identifier tokens from body (arena) or NULL
/* Rust only: raw trait path from the exact `impl Trait for Type` block
* that declared this method. Kept at the tail so zero-initialised
* callers in every other language remain ABI/source compatible. */
const char *impl_trait;
} CBMDefinition;
/* Argument captured from a call expression */
typedef struct {
const char *expr; // raw expression text ("payload.info", "MY_URL", "'hello'")
const char *value; // resolved string value or NULL (constant propagation)
const char *keyword; // keyword name if keyword arg ("url", "topic_id"), NULL if positional
int index; // positional index (0-based)
} CBMCallArg;
#define CBM_MAX_CALL_ARGS 8
/* Byte offsets are meaningful only within the source buffer that produced
* them. C/C++/CUDA run both raw and preprocessed extraction passes, and those
* buffers can contain unrelated occurrences at the same numeric span. */
typedef enum {
CBM_SOURCE_ORIGIN_RAW = 0,
CBM_SOURCE_ORIGIN_PREPROCESSED,
} CBMSourceOrigin;
typedef struct {
const char *callee_name; // raw callee text ("pkg.Func", "foo")
const char *enclosing_func_qn; // QN of enclosing function (or module QN)
const char *first_string_arg; // first string literal argument (URL, topic, key) or NULL
const char *second_arg_name; // second argument identifier (handler ref) or NULL
CBMCallArg args[CBM_MAX_CALL_ARGS]; // first N arguments with expressions
int arg_count; // number of captured arguments
int loop_depth; // enclosing loop nesting at the call site
int branch_depth; // enclosing branch nesting at the call site
int start_line; // 1-based source line of the call (for def range-match)
uint32_t site_start_byte; // exact AST occurrence span; end > start when present
uint32_t site_end_byte; // exclusive byte offset in the source file
CBMSourceOrigin source_origin; // raw source or C-family preprocessed buffer
bool is_method; // method/member call with an UNRESOLVED receiver. Perl:
// arrow/method call ($obj->m). TS/JS/TSX: member call
// x.foo() whose receiver is not this/super. Python:
// x.foo() where x is not self/cls/super() and is not
// rooted in an imported name. Read by the weak-member
// guard and by the pxc synthetic-carrier dedup key in
// pass_lsp_cross.c. Default false.
bool requires_lsp_resolution; // synthetic semantic candidate (for example an implicit
// C++ operator). Never fall back to textual resolution.
bool callee_is_locally_bound; // bare call foo() whose callee identifier is bound as a
// parameter of an enclosing function, so it cannot be the
// module-level foo. Python only today. Read by the
// weak-local-binding guard. Default false.
} CBMCall;
typedef struct {
const char *local_name; // local alias or name
const char *module_path; // resolved module path / QN
} CBMImport;
typedef enum {
CBM_USAGE_VALUE = 0,
CBM_USAGE_CALL_REFERENCE,
} CBMUsageKind;
typedef struct {
const char *ref_name; // referenced identifier
const char *enclosing_func_qn; // QN of enclosing function (or module QN)
CBMUsageKind kind; // ordinary USAGE or explicit callable reference
bool may_be_call_reference; // syntactic candidate; exact LSP proof may upgrade its edge
bool semantic_reference_blocked; // lexical evidence blocks only unproven textual fallback
bool semantic_reference_local_shadow; // blocker belongs to a non-module lexical scope
uint32_t lexical_scope_id; // extraction-local scope instance; never graph identity
uint32_t site_start_byte; // exact reference-token span; end > start when present
uint32_t site_end_byte; // exclusive byte offset in the source file
CBMSourceOrigin source_origin; // raw source or C-family preprocessed buffer
bool is_member_access; // token is the member half of a selector/attribute
// (Go x.f — field_identifier). The extractor strips
// the receiver, so this is the only surviving record
// of selector shape (#1962). Default false.
} CBMUsage;
typedef struct {
const char *exception_name; // exception class/type name
const char *enclosing_func_qn; // QN of enclosing function
} CBMThrow;
typedef struct {
const char *var_name; // variable name
const char *enclosing_func_qn; // QN of enclosing function
bool is_write; // true = write, false = read
bool is_member_access; // var_name is the field half of a selector/member LHS
// (`t.err = x` → "err"); the receiver is stripped here,
// so this is the only record of selector shape (#1962)
} CBMReadWrite;
typedef struct {
const char *type_name; // referenced type/class name
const char *enclosing_func_qn; // QN of enclosing function
} CBMTypeRef;
typedef struct {
const char *env_key; // environment variable key
const char *enclosing_func_qn; // QN of enclosing function
} CBMEnvAccess;
typedef struct {
const char *var_name; // variable being assigned
const char *type_name; // class/type name of RHS constructor
const char *enclosing_func_qn; // QN of enclosing function
} CBMTypeAssign;
// String reference: URL, config key, or async target found in source.
// Extracted from string literals during AST walk.
typedef enum {
CBM_STRREF_URL = 0, // REST path or full URL
CBM_STRREF_CONFIG = 1, // config file path or env var key
} CBMStringRefKind;
typedef struct {
const char *value; // the string literal content
const char *enclosing_func_qn; // QN of enclosing function
const char *key_path; // dotted key path from YAML/JSON nesting (NULL if flat)
CBMStringRefKind kind; // URL, CONFIG
} CBMStringRef;
/* Infrastructure binding: topic/queue → endpoint URL.
* Extracted from YAML/HCL/JSON subscription/scheduler configs.
* Used by pass_route_nodes to connect async Route nodes to handler services. */
typedef struct {
const char *source_name; // topic, queue, or schedule name
const char *target_url; // push_endpoint, uri, or http_target URL
const char *broker; // "pubsub", "cloud_tasks", "cloud_scheduler", "sqs", "kafka"
} CBMInfraBinding;
/* Pub/sub channel participation. One record per emit() or on()/addListener()
* call detected in source — the receiver (e.g. Socket.IO client, EventEmitter
* instance) is intentionally NOT identified; matching is by channel_name
* across files, which captures the common pattern of one logical bus per
* service. Transport disambiguates Socket.IO vs EventEmitter vs future
* detectors (Kafka, Cloud Pub/Sub, etc.). */
typedef enum {
CBM_CHANNEL_EMIT = 0,
CBM_CHANNEL_LISTEN = 1,
} CBMChannelDirection;
typedef struct {
const char *channel_name; // literal channel name (e.g. "user.created")
const char *transport; // "socketio", "event_emitter", ...
const char *enclosing_func_qn; // QN of the function containing the emit/on call
CBMChannelDirection direction;
} CBMChannel;
// Rust: impl Trait for Struct
typedef struct {
const char *trait_name; // trait name (raw text)
const char *struct_name; // struct/type name (raw text)
/* Exact extracted QN of the implementing type. Unlike struct_name this
* does not need a later leaf-name guess, and the relation exists even for
* an empty `impl Trait for Type {}` block. */
const char *struct_qn;
} CBMImplTrait;
typedef enum {
CBM_RESOLVED_INVOCATION = 0,
CBM_RESOLVED_CALL_REFERENCE,
} CBMResolvedKind;
// LSP-resolved invocation/reference: high-confidence type-aware resolution.
typedef struct {
const char *caller_qn; // enclosing function QN
const char *callee_qn; // resolved target QN (fully qualified)
const char *strategy; // "lsp_type_dispatch", "lsp_direct", etc.
float confidence; // 0.90-0.95
const char *reason; // diagnostic label for unresolved calls (NULL if resolved)
CBMResolvedKind kind; // invocation (CALLS) or explicit callable reference
uint32_t site_start_byte; // exact source occurrence; end > start when present
uint32_t site_end_byte; // exclusive byte offset in the source file
CBMSourceOrigin source_origin; // raw source or C-family preprocessed buffer
} CBMResolvedCall;
typedef struct {
CBMResolvedCall *items;
int count;
int cap;
} CBMResolvedCallArray;
// Growable arrays used during extraction.
typedef struct {
CBMDefinition *items;
int count;
int cap;
} CBMDefArray;
typedef struct {
CBMCall *items;
int count;
int cap;
} CBMCallArray;
typedef struct {
CBMImport *items;
int count;
int cap;
} CBMImportArray;
typedef struct {
CBMUsage *items;
int count;
int cap;
} CBMUsageArray;
typedef struct {
CBMThrow *items;
int count;
int cap;
} CBMThrowArray;
typedef struct {
CBMReadWrite *items;
int count;
int cap;
} CBMRWArray;
typedef struct {
CBMTypeRef *items;
int count;
int cap;
} CBMTypeRefArray;
typedef struct {
CBMEnvAccess *items;
int count;
int cap;
} CBMEnvAccessArray;
typedef struct {
CBMTypeAssign *items;
int count;
int cap;
} CBMTypeAssignArray;
typedef struct {
CBMStringRef *items;
int count;
int cap;
} CBMStringRefArray;
typedef struct {
CBMInfraBinding *items;
int count;
int cap;
} CBMInfraBindingArray;
typedef struct {
CBMImplTrait *items;
int count;
int cap;
} CBMImplTraitArray;
typedef struct {
CBMChannel *items;
int count;
int cap;
} CBMChannelArray;
// Full extraction result for one file.
typedef struct CBMFileResult {
CBMArena arena; // owns local memory; composites may also retain child arenas below
CBMDefArray defs;
CBMCallArray calls;
CBMImportArray imports;
CBMUsageArray usages;
CBMThrowArray throws;
CBMRWArray rw;
CBMTypeRefArray type_refs;
CBMEnvAccessArray env_accesses;
CBMTypeAssignArray type_assigns;
CBMImplTraitArray impl_traits; // Rust: impl Trait for Struct pairs
CBMResolvedCallArray resolved_calls; // LSP-resolved invocations/references (high confidence)
CBMStringRefArray string_refs; // URL/config string literals from AST
CBMInfraBindingArray infra_bindings; // topic→URL pairs from IaC configs
CBMChannelArray channels; // Socket.IO / EventEmitter pub/sub participation
const char *module_qn; // module qualified name
const char *namespace_name; // declared namespace/package (Java/Kotlin/C#/PHP), NULL if none
const char **exports; // NULL-terminated (NULL if none)
const char **constants; // NULL-terminated (NULL if none)
const char **global_vars; // NULL-terminated (NULL if none)
const char **macros; // NULL-terminated, C/C++ only (NULL if none)
bool has_error;
const char *error_msg;
/* Best-effort parse-coverage signal (experimental). parse_incomplete is true
* when the parse tree contains tree-sitter ERROR/MISSING nodes — constructs
* in those regions are silently absent from the graph. error_ranges is a
* compact "start-end,start-end" list of 1-based line ranges (arena-owned) or
* NULL. This only marks what we can DETECT: the absence of a flag is NOT a
* completeness guarantee. Callers should treat a flagged file as "prefer
* grep here", never treat an unflagged file as provably complete. */
bool parse_incomplete;
/* True when the ranges cover so much of the file that they are no longer
* useful advice — one range over 80% of the line count. The file WAS
* indexed, but pointing a reader at almost every line tells them nothing,
* so the report says "read the source" instead of listing the range.
*
* Its main customers are non-C languages. The refinement that narrows a
* whole-file range using the preprocessed parse only runs for C, C++ and
* CUDA, so a Python, Java or Ruby file whose root node is ERROR still
* reports 1-N.
*
* Note the naming: this field and the phase string it produces are both
* `parse_unusable`. The older `parse_incomplete` field emits the phase
* `parse_partial` instead. That mismatch is historical, not deliberate —
* do not copy it. */
bool parse_unusable;
const char *error_ranges;
int error_region_count;
bool is_test_file;
int imports_count;
TSTree *cached_tree; // retained parse tree (caller frees via cbm_free_tree)
CBMLanguage cached_lang; // language of cached tree (for parser selection)
// Retained source bytes — copied into `arena` by the parallel
// extract pass so the fused cross-file LSP step in resolve_worker
// can run without re-reading the file from disk. NULL when the
// file exceeded the per-file (100 MB) or total (2 GB) retention
// cap; in that case the cross-file LSP step is skipped for this
// file (defs/calls already extracted are unaffected).
const char *source;
int source_len;
// Composite extraction results (currently ObjectScript Studio Export)
// retain their per-unit results so shallow-copied carrier strings remain
// valid for the composite's full lifetime. Owned and recursively released
// by cbm_free_result(); ordinary single-file results leave these zeroed.
struct CBMFileResult **owned_results;
int owned_result_count;
} CBMFileResult;
// --- Enclosing function cache ---
// Avoids repeated parent-chain walks for nodes within the same function body.
// Each entry records a function's byte range and its precomputed QN.
#define EFC_SIZE 64 // power of 2 for fast modulo
typedef struct {
uint32_t start_byte;
uint32_t end_byte;
const char *qn;
} EFCEntry;
typedef struct {
EFCEntry entries[EFC_SIZE];
int count;
} EFCache;
// --- Extraction context passed to sub-extractors ---
// Module-level string constant map (for constant propagation)
#define CBM_MAX_STRING_CONSTANTS 256
typedef struct {
const char *names[CBM_MAX_STRING_CONSTANTS];
const char *values[CBM_MAX_STRING_CONSTANTS];
bool is_url_builder[CBM_MAX_STRING_CONSTANTS];
int count;
} CBMStringConstantMap;
// Forward declaration: ObjectScript macro table (defined in macro_table.h).
typedef struct CBMMacroTable CBMMacroTable;
// Method-return-type table for ObjectScript variable type inference. Populated
// from definition nodes (method QN -> declared return type) so a later
// `Set x = obj.Method()` can resolve x's class.
#define CBM_RETURN_TYPE_TABLE_CAP 2048
typedef struct {
const char *method_qn;
const char *return_type;
} CBMReturnTypeEntry;
typedef struct {
CBMReturnTypeEntry entries[CBM_RETURN_TYPE_TABLE_CAP];
int count;
} CBMReturnTypeTable;
typedef struct {
CBMArena *arena;
/* Scratch for AST traversal, owned by the cbm_extract_file_ex call that
* built this context and destroyed when it returns. Nothing a
* CBMFileResult points at may be allocated here: `arena` is the result's
* own, and it outlives extraction by the whole pipeline (#1997). NULL in a
* context built without one, in which case the stacks fall back to
* `arena`. */
CBMArena *scratch;
CBMFileResult *result;
const char *source;
int source_len;
CBMLanguage language;
const char *project;
const char *rel_path;
const char *module_qn;
TSNode root;
EFCache ef_cache; // enclosing function cache
const char *enclosing_class_qn; // for nested class QN computation
CBMStringConstantMap string_constants; // module-level NAME = "value" pairs
const CBMMacroTable *macro_table; // ObjectScript $$$macro table (NULL if none)
const CBMReturnTypeTable *return_type_table; // ObjectScript method return types (NULL if none)
/* Set by extract_class_variables around its extract_var_names calls, so a
* class-body variable def records which class declares it (parent_class)
* without changing its module-level qualified name. NULL elsewhere. */
const char *var_parent_class;
} CBMExtractCtx;
// --- Public API ---
// Bind third-party allocators (tree-sitter, sqlite3) to mimalloc as
// defense-in-depth, so they never depend on the fragile MI_OVERRIDE symbol
// override (#424). MUST be called as the very first statement of main(), before
// any sqlite3_open*/sqlite3_initialize (SQLITE_CONFIG_MALLOC returns
// SQLITE_MISUSE once sqlite has initialized).
// Idempotent (static guard); intended for single-threaded startup. cbm_init()
// also calls it so non-main entry points (pipeline passes) still get the binds.
// In the test build (no CBM_BIND_TS_ALLOCATOR) this is a no-op.
void cbm_alloc_init(void);
// Initialize the library. Call once at startup. Returns 0 on success.
int cbm_init(void);
// True when rel_path is in the crash-quarantine set — the newline-delimited list
// of files (CBM_INDEX_QUARANTINE_FILE) the crash supervisor pinned as crashers
// during its single-threaded recovery re-run. Loaded once, lazily; read-only
// after load. cbm_extract_file short-circuits such files to an empty result so no
// pass can crash on them; the pipeline extract loops call this to also REPORT the
// skip as phase="crash". Always false (cheap no-op) when the env var is unset.
bool cbm_index_is_quarantined(const char *rel_path);
// Phase a quarantined file was pinned under: "crash" (a fault signal) or "hang"
// (killed for making no progress). Returns NULL when rel_path is not quarantined.
// Drives the same lazy once-load as cbm_index_is_quarantined. Used by the pipeline
// extract loops to report the skip's phase in skipped[] (falls back to "crash").
const char *cbm_index_quarantine_phase(const char *rel_path);
// Crash-supervisor marker journal (parallel-safe): appends "S <rel_path>" /
// "D <rel_path>" to CBM_INDEX_MARKER_FILE. Files with an S but no D form the
// parent's crash/hang suspect set. No-ops when the env var is unset.
// cbm_extract_file journals its own start/done; long-running per-file phases
// (cross-LSP resolve) call these around their per-file work so a hang there
// is attributed to the RIGHT file instead of a stale extraction marker.
void cbm_index_mark_start(const char *rel_path);
void cbm_index_mark_done(const char *rel_path);
// Extract all data from one file. Caller must call cbm_free_result().
// source must remain valid for the duration of the call.
// timeout_micros: per-file parse timeout in microseconds (0 = no timeout).
CBMFileResult *cbm_extract_file(const char *source, int source_len, CBMLanguage language,
const char *project, const char *rel_path, int64_t timeout_micros,
const char **extra_defines, // NULL-terminated, or NULL
const char **include_paths // NULL-terminated, or NULL
);
// Pipeline-internal variant of cbm_extract_file() carrying ObjectScript
// per-project tables (macro table + method-return-type table). The public
// cbm_extract_file() is a thin wrapper that passes NULL, NULL for both.
CBMFileResult *cbm_extract_file_ex(
const char *source, int source_len, CBMLanguage language, const char *project,
const char *rel_path, int64_t timeout_micros,
const char **extra_defines, // NULL-terminated, or NULL
const char **include_paths, // NULL-terminated, or NULL
const CBMMacroTable *macro_table, // ObjectScript macros, or NULL
const CBMReturnTypeTable *return_type_table // OS return types, or NULL
);
// Free all memory associated with a result.
void cbm_free_result(CBMFileResult *result);
// Free only the cached tree from a result (caller retained it for reuse).
void cbm_free_tree(CBMFileResult *result);
// Free a standalone TSTree pointer (for Go layer cleanup).
void cbm_free_tree_ptr(TSTree *tree);
// Reset the thread-local parser's internal state, releasing slab-allocated
// subtrees. Must be called BEFORE cbm_slab_reset_thread() so the slab rebuild
// doesn't corrupt live parser state.
void cbm_reset_thread_parser(void);
// Destroy the thread-local parser. Call on worker thread exit.
void cbm_destroy_thread_parser(void);
// Shutdown the library. Call once at exit.
void cbm_shutdown(void);
// Profiling: get accumulated parse/extraction times and file count.
typedef struct {
uint64_t *parse_ns;
uint64_t *extract_ns;
uint64_t *files;
} cbm_profile_out_t;
void cbm_get_profile(cbm_profile_out_t out);
uint64_t cbm_get_lsp_ns(void);
uint64_t cbm_get_preprocess_ns(void);
uint64_t cbm_get_files_preprocessed(void);
void cbm_reset_profile(void);
#if defined(CBM_KOTLIN_DEDUP_TEST_API) && CBM_KOTLIN_DEDUP_TEST_API
// Test-build-only operation counter for Kotlin operator-carrier deduplication.
// Production builds do not expose or retain this instrumentation.
void cbm_kotlin_operator_dedup_test_reset(void);
uint64_t cbm_kotlin_operator_dedup_test_comparisons(void);
#endif
#if defined(CBM_CALL_REFERENCE_LOOKUP_TEST_API) && CBM_CALL_REFERENCE_LOOKUP_TEST_API
// Test-build-only work counter for resolving a node's field role while
// classifying value references. Production builds retain no instrumentation.
void cbm_usage_field_lookup_test_reset(void);
uint64_t cbm_usage_field_lookup_test_work(void);
uint64_t cbm_usage_slow_parent_fallback_test_count(void);
#endif
// Toggle C/C++ preprocessor Macro-node extraction (#375). The pipeline enables
// it only for full/advanced index modes (it dominates extraction on macro-dense
// codebases). Default ON. Set before extraction; read-only during.
void cbm_set_macro_extraction(int enabled);
int cbm_macro_extraction_enabled(void);
// --- Internal helpers used by extractors ---
// Growable array push functions (arena-allocated, no individual free needed).
void cbm_defs_push(CBMDefArray *arr, CBMArena *a, CBMDefinition def);
void cbm_calls_push(CBMCallArray *arr, CBMArena *a, CBMCall call);
void cbm_imports_push(CBMImportArray *arr, CBMArena *a, CBMImport imp);
void cbm_usages_push(CBMUsageArray *arr, CBMArena *a, CBMUsage usage);
void cbm_throws_push(CBMThrowArray *arr, CBMArena *a, CBMThrow thr);
void cbm_rw_push(CBMRWArray *arr, CBMArena *a, CBMReadWrite rw);
void cbm_typerefs_push(CBMTypeRefArray *arr, CBMArena *a, CBMTypeRef tr);
void cbm_envaccess_push(CBMEnvAccessArray *arr, CBMArena *a, CBMEnvAccess ea);
void cbm_typeassign_push(CBMTypeAssignArray *arr, CBMArena *a, CBMTypeAssign ta);
void cbm_stringref_push(CBMStringRefArray *arr, CBMArena *a, CBMStringRef sr);
void cbm_infrabinding_push(CBMInfraBindingArray *arr, CBMArena *a, CBMInfraBinding ib);
void cbm_impltrait_push(CBMImplTraitArray *arr, CBMArena *a, CBMImplTrait it);
void cbm_resolvedcall_push(CBMResolvedCallArray *arr, CBMArena *a, CBMResolvedCall rc);
void cbm_channels_push(CBMChannelArray *arr, CBMArena *a, CBMChannel ch);
// --- Sub-extractor entry points ---
void cbm_extract_definitions(CBMExtractCtx *ctx);
/* Internal companion for embedded-language trees that contribute definitions
* to an existing host-file Module rather than minting a second Module. */
void cbm_extract_definitions_without_module(CBMExtractCtx *ctx);
// dbt lineage for Jinja-templated SQL models: emits a Model def plus one usage
// per ref()/source() call. No-op unless the file parses as SQL and actually
// contains a dbt builtin call. Defined in extract_dbt.c.
void cbm_extract_dbt(CBMExtractCtx *ctx);
void cbm_extract_imports(CBMExtractCtx *ctx);
void cbm_extract_usages(CBMExtractCtx *ctx);
void cbm_extract_semantic(CBMExtractCtx *ctx);
void cbm_extract_type_refs(CBMExtractCtx *ctx);
void cbm_extract_env_accesses(CBMExtractCtx *ctx);
void cbm_extract_type_assigns(CBMExtractCtx *ctx);
void cbm_extract_channels(CBMExtractCtx *ctx);
// Single-pass unified extraction (replaces the 7 calls above except defs+imports).
void cbm_extract_unified(CBMExtractCtx *ctx);
// K8s / Kustomize semantic extractor (called when language is CBM_LANG_K8S or CBM_LANG_KUSTOMIZE).
void cbm_extract_k8s(CBMExtractCtx *ctx);
// --- Label predicates ---
// True when `label` names a TYPE-LIKE container definition — a node that can own
// methods/fields, be a base/embedded type, satisfy/declare an interface, and be a
// target of name→type resolution. The canonical set is:
// Class, Struct, Interface, Enum, Type, Trait.
// Single source of truth for every type-resolution / registry-seeding /
// INHERITS·IMPLEMENTS / LSP-type-registrar consumer, so adding a new type-like
// label (e.g. "Struct" for Rust/Go/Swift/D structs) updates them all at once
// instead of scattering `|| strcmp(label,"Struct")==0` across the tree.
// `label` may be NULL (returns false). Defined in helpers.c.
bool cbm_label_is_type_like(const char *label);
// True for data-relation labels (Table, View — SQL DDL). Relations resolve as
// lineage targets only: registry members, but never type-like and never valid
// CALLS/THROWS/READS/WRITES targets. `label` may be NULL. Defined in helpers.c.
bool cbm_label_is_relation(const char *label);
// True for labels admitted to the cross-file name registry: Function, Method,
// every type-like container, Variable, Field, and the relation labels. Single
// source of truth for registry seeding — the full (pass_definitions.c),
// parallel (pass_parallel.c) and incremental (pipeline_incremental.c) pipelines
// all seed through this predicate so their registries never diverge.
// `label` may be NULL (returns false). Defined in helpers.c.
bool cbm_label_is_registry_symbol(const char *label);
#endif // CBM_H