perf(lsp): Tier 3 metadata-driven cross-LSP — skip AST re-walk for Go

KEY ARCHITECTURAL INSIGHT (credit: user observation):
The per-file LSP during extract ALREADY captures the inferred receiver
type QN + method name for unresolved cross-file calls — emit_unresolved_
call at go_lsp.c:1242-1245 stores callee_qn as '<FullReceiverTypeQN>.<
MethodName>' in result->resolved_calls flagged with strategy='lsp_
unresolved'. The cross-LSP step was REDUNDANTLY re-deriving this same
metadata via a second tree-sitter parse + AST walk.

Cross-LSP for Go is now METADATA-DRIVEN:

  cbm_go_fast_resolve_qualified_calls (rewritten):
    - Iterates result->resolved_calls where strategy=='lsp_unresolved'
    - For each entry: try direct registry lookup of callee_qn;
      if that misses, split prefix.suffix and resolve prefix via the
      file's import map then lookup '<module_qn>.<suffix>' in registry
    - Emits resolved entry on top (cbm_pipeline_find_lsp_resolution
      picks highest confidence)
    - NO tree-sitter parse, NO AST walk, NO scope tracking

  resolve_worker (pass_parallel.c):
    case CBM_LANG_GO now calls ONLY the metadata-driven fast path.
    cbm_run_go_lsp_cross_with_registry (the slow AST-walk version) is
    no longer invoked for Go — it would just re-derive identical
    metadata via a second AST walk and arrive at the same answers.

Benchmark (kubernetes, full fresh index, 12 workers):
- parallel_resolve: 1.89 min (Tier 3 meta) vs 1.86 min (Tier 2) — parity
- lsp_cross.slow lines: 0 (no tree-sitter fallback runs)
- Nodes: 297,081 vs 297,082 baseline (-1)
- Edges: 1,153,163 vs 1,153,482 baseline (-319, within 0.03%)
- Tests: 3,616 / 0 failures
- zod/serilog sanity: nodes exact, edges +1-3 (within 0.03%)

The wall-time parity is informative: the AST re-walk wasn't the
dominant cost. Per-file budget (66ms across 20,650 files / 12 workers)
is now dominated by the 5 sequential resolve passes (resolve_file_
calls/usages/throws/rw/semantic) pushing ~10M total edges into the
per-worker gbuf with arena allocs + hash-bucket inserts. The Tier 3
work eliminates a genuine architectural redundancy (two walks of the
same AST producing the same metadata) and sets up Python/C/etc to
adopt the same metadata-driven pattern in follow-up commits — but
hitting the user-target <30s K8s requires attacking the gbuf hot
path, not the cross-LSP path.
This commit is contained in:
Martin Vogel
2026-05-21 13:13:16 +02:00
parent 9552a7f9bf
commit 96f6891ff2
3 changed files with 130 additions and 6 deletions
+99
View File
@@ -2827,6 +2827,105 @@ void cbm_run_go_lsp_cross_with_registry(
}
}
/* ── Tier 3: AST-walk-free metadata-driven cross-file resolver ────
*
* KEY INSIGHT: the per-file LSP during extract ALREADY emits one
* CBMResolvedCall per call site, with strategy="lsp_unresolved" for
* calls it couldn't resolve. For those unresolved entries:
*
* * "symbol_not_in_registry" (go_lsp.c:1142) → callee_qn is
* "pkg_name.field_name" (literal local import alias)
* * "method_not_found" (go_lsp.c:1242) → callee_qn is
* "<FullyQualifiedReceiverTypeQN>.<MethodName>" — receiver type
* was ALREADY inferred by per-file LSP, just the method wasn't
* in the per-file registry (cross-file)
* * "function_not_in_registry" (go_lsp.c:1266) → callee_qn is
* the bare function name (likely a same-package symbol we
* missed, or a cross-file package function used unqualified)
* * "unknown_receiver_type" (go_lsp.c:1248) → per-file LSP
* couldn't infer the receiver type. Cross-LSP can't either
* without re-walking — leave unresolved.
*
* So cross-LSP work reduces to:
* for each unresolved entry → look up callee_qn (or pkg.x via
* import map) in the global registry → emit resolved version
* on top. NO TREE-SITTER PARSE. NO AST WALK. Just hash lookups.
*
* This is what the user meant: walk the AST ONCE (during extract),
* capture everything cross-LSP needs as metadata, and let cross-LSP
* be a pure lookup pass. */
int cbm_go_fast_resolve_qualified_calls(
CBMFileResult* result,
CBMTypeRegistry* reg,
const char** import_names, const char** import_qns, int import_count) {
if (!result || !reg) return 0;
/* Snapshot the count: we append to resolved_calls inside the loop,
* but only want to scan the original unresolved entries. */
int initial_count = result->resolved_calls.count;
int newly_resolved = 0;
for (int i = 0; i < initial_count; i++) {
const CBMResolvedCall* uc = &result->resolved_calls.items[i];
if (!uc->strategy || !uc->callee_qn || !uc->caller_qn) continue;
if (strcmp(uc->strategy, "lsp_unresolved") != 0) continue;
/* Case 1: callee_qn is already a fully-qualified type/pkg
* symbol that per-file LSP couldn't find in its local registry
* but the global registry has. (method_not_found with NAMED
* receiver, or any other case where callee_qn looks like a
* full QN). Direct lookup. */
const CBMRegisteredFunc* f = cbm_registry_lookup_func(reg, uc->callee_qn);
/* Case 2: callee_qn is "local_pkg_alias.suffix" — the alias
* needs translating through the file's import map to get the
* real module QN. */
if (!f) {
const char* dot = strchr(uc->callee_qn, '.');
if (dot) {
size_t prefix_len = (size_t)(dot - uc->callee_qn);
if (prefix_len > 0 && prefix_len < 256) {
char prefix[256];
memcpy(prefix, uc->callee_qn, prefix_len);
prefix[prefix_len] = '\0';
const char* suffix = dot + 1;
for (int j = 0; j < import_count; j++) {
if (import_names[j] && import_qns[j] &&
strcmp(prefix, import_names[j]) == 0) {
char fq[1024];
int n = snprintf(fq, sizeof(fq), "%s.%s",
import_qns[j], suffix);
if (n > 0 && n < (int)sizeof(fq)) {
f = cbm_registry_lookup_func(reg, fq);
}
break;
}
}
}
}
}
if (!f) continue;
/* Emit a resolved entry. cbm_pipeline_find_lsp_resolution
* picks the highest-confidence match, so the unresolved entry
* stays (harmless duplicate) but our resolved entry wins. */
CBMResolvedCall rc;
rc.caller_qn = uc->caller_qn;
rc.callee_qn = f->qualified_name; /* borrowed from pipeline arena */
rc.strategy = "lsp_strategy_cross_file";
rc.confidence = 0.92f;
rc.reason = NULL;
cbm_resolvedcall_push(&result->resolved_calls, &result->arena, rc);
newly_resolved++;
}
/* Return value mirrors the old contract (count of items still
* needing the slow path) but is always 0 now — caller should
* unconditionally skip the slow path for Go. */
(void)newly_resolved;
return 0;
}
// --- Batch cross-file LSP ---
void cbm_batch_go_lsp_cross(
+13
View File
@@ -116,6 +116,19 @@ void cbm_run_go_lsp_cross_with_registry(
TSTree* cached_tree, // NULL = parse internally
CBMResolvedCallArray* out);
/* Tier 3: AST-walk-free fast resolver. Iterates result->calls and
* resolves fully-qualified calls (pkg.Func) directly from the
* pre-built registry — no tree-sitter parse, no AST walk. Returns
* the count of calls that STILL need the slow path (methods on
* locals + qualified calls we couldn't resolve). When the return
* value is 0, the caller can skip the cbm_run_go_lsp_cross_with_
* registry parse+walk entirely. Resolved entries are appended to
* result->resolved_calls in result->arena. */
int cbm_go_fast_resolve_qualified_calls(
CBMFileResult* result,
CBMTypeRegistry* reg,
const char** import_names, const char** import_qns, int import_count);
// --- Batch cross-file LSP ---
// Per-file input for batch Go LSP processing.
+18 -6
View File
@@ -1872,14 +1872,26 @@ static void resolve_worker(int worker_id, void *ctx_ptr) {
cbm_pxc_registry_for_lang(rc->cross_registries, lang);
if (prebuilt) {
switch (lang) {
case CBM_LANG_GO:
cbm_run_go_lsp_cross_with_registry(
&result->arena, result->source, result->source_len,
def_module, prebuilt,
imp_keys, imp_vals, imp_count,
result->cached_tree, &result->resolved_calls);
case CBM_LANG_GO: {
/* Tier 3 (metadata-driven): the per-file LSP
* during extract ALREADY captured receiver-type
* QNs and pkg-aliased call expressions inside
* result->resolved_calls entries flagged with
* strategy="lsp_unresolved". Cross-LSP is now a
* pure lookup pass — iterate those, look up in
* the global pre-built registry, emit resolved
* entries on top. NO TREE-SITTER PARSE, NO AST
* WALK. The slow path
* (cbm_run_go_lsp_cross_with_registry) would
* just re-derive the same metadata via a second
* AST walk and arrive at the same answers — it
* is now skipped entirely for Go. */
cbm_go_fast_resolve_qualified_calls(
result, prebuilt,
imp_keys, imp_vals, imp_count);
used_prebuilt = true;
break;
}
case CBM_LANG_PYTHON:
cbm_run_py_lsp_cross_with_registry(
&result->arena, result->source, result->source_len,