mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-10-23 06:50:54 +08:00
fix(go): a function value keeps its link past a local of the same name (#2776)
In Go, a function used as a value, like `apply(run)`, was recorded once
per function and name, at its first use. Since PR #2658 the resolver drops
a use that a local or parameter of that name shadows. When a function's
first use was the shadowed one and a later use, outside the local's scope,
was the package function, that later use had no reference left, and the
function lost its references edge to the package function (an inner-block
`run := 2`, a closure's `func(run int)`, a case clause, an if header or a
range variable, then `apply(run)`). Issue #2596 predicted this.
Fix, in both extractors (function-ref.ts with tree-sitter.ts's
flushFnRefCandidates, and the kernel's go.rs flush_fn_ref_candidates):
- A Go function that binds the name itself records one use per scope
instead of one per function. Bindings are a func literal's parameters,
:=, var, const, range and type-switch variables and a select case's
value. Scopes are Go's: a block, an if/for/switch/select statement's
implicit block and each case clause. Within one scope a local declared
before the first use also covers every later use, so the first use
still speaks for the scope.
- The function's own receiver, parameters and results keep a single
reference: they cover the whole body.
- A func literal at file scope is its own unit.
- Every other function is unchanged. The scope and enclosing function are
read from the tree only for names a function uses at least twice, after
the name gate (reading them for every container made kernel extraction
of prometheus 2.4x slower).
Validation:
- go-shadowed-function-value.test.ts: 14 cases, kernel and wasm, LF and
CRLF. On main's sources and kernel 10 fail; with the fix 14 pass.
Per-scope keys without the binding check, and dropping range
variables, case clauses or func literal parameters, each fail 3.
- Kernel parity, Go: gin 99/99, etcd 1065/1105 (40 deferred, as main),
prometheus 746/746, kubernetes 17779/17952 (173 deferred, as main),
0 files with diffs.
- A/B against 6bd217a2 (fresh indexes, each arm with its own kernel):
gin 0f09c3a9, etcd bf9cadf0, prometheus 017fcf2e and kubernetes
006cdf87 keep identical nodes and edges. The only change is 0, 4, 13
and 163 more function_ref rows, all failed: per-scope uses of a local
(all 17 on etcd and prometheus and 36 on kubernetes read by hand).
express and flask dump byte-identically.
- Cost: kernel extraction of prometheus 2.90 s vs 2.90 s, wasm 6.06 vs
6.07 s; kubernetes 34.9 vs 34.9 s; prometheus index medians
12.35 s / 49.1 s CPU vs 12.45 s / 48.3 s over 4 interleaved rounds.
Fixes #2596
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
7dfef8451f
commit
c326b6bec1
@@ -178,6 +178,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
- In Go, a type defined from other types, like etcd's `type WatchChan <-chan WatchResponse` or gin's `type HandlerFunc func(*Context)` and `type HandlersChain []HandlerFunc`, now links to the types it is built from, so impact on `WatchResponse` or gin's `Context` reaches these types and the code that uses them. Before, such a type linked to nothing. One written through another package, like etcd's `type PutResponse pb.PutResponse`, links to that package's type, and a recursive one like `type stateFn func(*Lexer) stateFn` doesn't link to itself. Re-index Go projects after upgrading.
|
||||
- In Go, a struct now links to the types its fields are declared with, like etcd's `PutResponse` holding a `*ResponseHeader` or gin's `Context` holding a `HandlersChain`, so impact on `ResponseHeader` or `HandlersChain` reaches every struct that holds one and the code that uses those structs, and `codegraph_node` lists a struct's field types. Before, a struct linked only to the types it embeds. A field written through another package, like `*pb.ResponseHeader`, links to that package's type, one from outside your project, like `context.Context`, links to nothing, the types an embedded generic is filled in with, like the `Pod` in client-go's `listers.ResourceIndexer[*v1.Pod]`, link too, and a struct that holds itself, like a list node's `next *node`, doesn't link to itself. `codegraph_explore` still answers a question about a flow with the code that runs it, not with the structs that merely hold one of its types. Re-index Go projects after upgrading.
|
||||
- In Go, an unexported name used in another file of its package, like prometheus's `memSeries` in `tsdb/head_append.go`, now links to that package's own declaration when a package nested inside it, like `tsdb/agent`, declares the same name. Before, such a parameter, result, struct field, composite literal or call linked to nothing. A call or a function value that the surrounding function binds itself, like a test's `init := func(…)` followed by `init(t)`, or a local `volumeSpec` variable passed on, no longer links to the package's declaration of that name. A name bound in one `case` of a `switch` or `select`, or a parameter named inside a function-typed result like `func() func(run string)`, is not such a binding anywhere else, so a call there still links. Thanks @inth3shadows. (#2580) Re-index Go projects after upgrading.
|
||||
- In Go, when a function uses a package function as a value, like `apply(run)`, and also has a local or parameter of the same name somewhere, like a `run := 2` inside an `if` or a `case`, a loop variable, or a closure's parameter, each use is now read on its own: a use the local covers links to nothing, and a use outside it links to the package function. Before, only the function's first use was looked at, so a local used first hid every later use of the package function. Re-index Go projects after upgrading. Thanks @inth3shadows for the report. (#2596)
|
||||
- In Go, a method an interface declares, like etcd's `KV.Put(ctx context.Context, key, val string, opts ...OpOption) (*PutResponse, error)`, now links to the types its parameters and results name, so impact on `PutResponse` or `OpOption` reaches the interface method and every caller that goes through the interface. Before, an interface's methods linked to no types at all. Re-index Go projects after upgrading.
|
||||
- In Go, a package imported under an explicit name, like `v3 "go.etcd.io/etcd/client/v3"`, now owns that name even when another import's path ends in it, like `"github.com/cheggaaa/pb/v3"` (package `pb`) in etcd's benchmark tool. Before, whichever of the two came first took every `v3.` name, so `v3.Client` and `v3.Op` could link to nothing. Re-index Go projects after upgrading.
|
||||
- In Go, a struct or defined type now counts as implementing an interface only when its methods take as many parameters and return as many results as the interface's. Before, matching method names was enough, so etcd's watch cache, whose `Get` takes a context, a key and options, was listed as a `peerGetter`, whose `Get` takes a peer ID, and a call through an interface could be shown reaching methods it can never reach. Look-alikes could also crowd out the real implementations, so `codegraph_explore` stopped at grpc-go's xDS `TransportBuilder` instead of reaching the transport it builds. A server still counts for the gRPC client interface of the service it serves, like etcd's `kvServer` for `KVClient`, so a client's call keeps leading to the handler that serves it. Re-index Go projects after upgrading.
|
||||
|
||||
@@ -0,0 +1,275 @@
|
||||
/**
|
||||
* A Go function used as a value is recorded once per function, at its first
|
||||
* use (#756). Since #2658 a use that a parameter or local of the same name
|
||||
* shadows links to nothing, which is right for that use. But when the first
|
||||
* use in a function is the shadowed one and a later use, outside the local's
|
||||
* scope, is the package function, the later use had no reference left: the
|
||||
* function lost its `references` edge to the package function (#2596).
|
||||
*
|
||||
* A function that binds the name itself now records one use per scope (a
|
||||
* block, a statement's implicit block, a case clause), so the later use keeps
|
||||
* its own reference. Within one scope a local declared before the first use
|
||||
* covers every later use as well, so the first use still speaks for the
|
||||
* scope. A function that binds no such local keeps its single reference, and
|
||||
* so does one whose own parameter has the name: that covers the whole body.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, beforeEach, afterEach, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import CodeGraph from '../src/index';
|
||||
import { extractFromSource } from '../src/extraction';
|
||||
import { initGrammars, loadGrammarsForLanguages } from '../src/extraction/grammars';
|
||||
import { tryKernelExtract, resetKernelForTests } from '../src/extraction/kernel';
|
||||
import type { ExtractionResult } from '../src/types';
|
||||
|
||||
const KERNEL_PATH = path.join(
|
||||
__dirname,
|
||||
'..',
|
||||
'codegraph-kernel',
|
||||
'prebuilds',
|
||||
`${process.platform}-${process.arch}`,
|
||||
'codegraph-kernel.node'
|
||||
);
|
||||
const kernelAvailable = fs.existsSync(KERNEL_PATH) || process.env.CODEGRAPH_KERNEL_EXPECT === '1';
|
||||
|
||||
const GO_MOD = 'module example.com/app\n\ngo 1.22\n';
|
||||
|
||||
/** Line numbers are pinned by the assertions below: keep the layout. */
|
||||
const MAIN_GO = `package main
|
||||
|
||||
func run(a, b int) int { return a + b }
|
||||
|
||||
func apply(f any) {}
|
||||
|
||||
func pick(f any) any { return f }
|
||||
|
||||
func shadowThenUse(x int) {
|
||||
if x > 0 {
|
||||
run := 2
|
||||
_ = run
|
||||
}
|
||||
apply(run)
|
||||
}
|
||||
|
||||
func closureThenUse() {
|
||||
g := func(run int) { _ = run }
|
||||
g(1)
|
||||
apply(run)
|
||||
}
|
||||
|
||||
func caseThenUse(x int) {
|
||||
switch x {
|
||||
case 1:
|
||||
run := 3
|
||||
_ = run
|
||||
case 2:
|
||||
apply(run)
|
||||
}
|
||||
}
|
||||
|
||||
func headerThenUse() {
|
||||
if run := pick(nil); run != nil {
|
||||
_ = run
|
||||
}
|
||||
apply(run)
|
||||
}
|
||||
|
||||
func rangeThenUse(xs []int) {
|
||||
for _, run := range xs {
|
||||
_ = run
|
||||
}
|
||||
apply(run)
|
||||
}
|
||||
|
||||
func useThenShadow(x int) {
|
||||
apply(run)
|
||||
if x > 0 {
|
||||
run := 2
|
||||
_ = run
|
||||
}
|
||||
apply(run)
|
||||
}
|
||||
|
||||
func allShadowed(run func(int, int) int) {
|
||||
apply(run)
|
||||
if true {
|
||||
apply(run)
|
||||
}
|
||||
}
|
||||
|
||||
func noBinding(x int) {
|
||||
apply(run)
|
||||
if x > 0 {
|
||||
apply(run)
|
||||
}
|
||||
}
|
||||
|
||||
func paramValue(run func(int, int) int) {
|
||||
apply(run)
|
||||
}
|
||||
|
||||
func localValue() {
|
||||
run := 1
|
||||
_ = run
|
||||
}
|
||||
|
||||
func main() {}
|
||||
|
||||
var literalThenUse = func(x int) {
|
||||
if x > 0 {
|
||||
run := 2
|
||||
_ = run
|
||||
}
|
||||
apply(run)
|
||||
}
|
||||
`;
|
||||
|
||||
/** `<function> L<line>` for every function_ref ref to `run`, in source order. */
|
||||
function runRefs(result: ExtractionResult): string[] {
|
||||
const byId = new Map(result.nodes.map((n) => [n.id, n]));
|
||||
return result.unresolvedReferences
|
||||
.filter((r) => r.referenceKind === 'function_ref' && r.referenceName === 'run')
|
||||
.map((r) => `${byId.get(r.fromNodeId)?.name} L${r.line}`);
|
||||
}
|
||||
|
||||
const ENV_KEYS = ['CODEGRAPH_KERNEL', 'CODEGRAPH_KERNEL_LANGS'] as const;
|
||||
|
||||
describe('a Go function value keeps its reference past a shadowing local (#2596)', () => {
|
||||
let savedEnv: Record<string, string | undefined> = {};
|
||||
|
||||
beforeAll(async () => {
|
||||
await initGrammars();
|
||||
await loadGrammarsForLanguages(['go']);
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
savedEnv = Object.fromEntries(ENV_KEYS.map((k) => [k, process.env[k]]));
|
||||
resetKernelForTests();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
for (const k of ENV_KEYS) {
|
||||
if (savedEnv[k] === undefined) delete process.env[k];
|
||||
else process.env[k] = savedEnv[k];
|
||||
}
|
||||
resetKernelForTests();
|
||||
});
|
||||
|
||||
function extract(backend: 'kernel' | 'wasm', source: string): ExtractionResult {
|
||||
if (backend === 'wasm') {
|
||||
process.env.CODEGRAPH_KERNEL = '0';
|
||||
return extractFromSource('main.go', source, 'go');
|
||||
}
|
||||
delete process.env.CODEGRAPH_KERNEL;
|
||||
process.env.CODEGRAPH_KERNEL_LANGS = 'all';
|
||||
const result = tryKernelExtract('main.go', source, 'go');
|
||||
expect(result, 'kernel extraction of main.go').not.toBeNull();
|
||||
return result!;
|
||||
}
|
||||
|
||||
const backends = kernelAvailable ? (['kernel', 'wasm'] as const) : (['wasm'] as const);
|
||||
|
||||
for (const crlf of [false, true]) {
|
||||
const source = crlf ? MAIN_GO.replace(/\n/g, '\r\n') : MAIN_GO;
|
||||
const label = crlf ? ' (CRLF)' : '';
|
||||
|
||||
it.each(backends)(`records a use per scope only where the function binds the name: %s${label}`, (backend) => {
|
||||
expect(runRefs(extract(backend, source))).toEqual([
|
||||
// Binds `run` in a scope of its own: one use per scope.
|
||||
'shadowThenUse L12',
|
||||
'shadowThenUse L14',
|
||||
'closureThenUse L18',
|
||||
'closureThenUse L20',
|
||||
'caseThenUse L27',
|
||||
'caseThenUse L29',
|
||||
'headerThenUse L35',
|
||||
'headerThenUse L37',
|
||||
'rangeThenUse L42',
|
||||
'rangeThenUse L44',
|
||||
// The function body's two uses are one scope: the first speaks for both.
|
||||
'useThenShadow L48',
|
||||
'useThenShadow L51',
|
||||
// A parameter of the function itself covers its whole body: one use.
|
||||
'allShadowed L57',
|
||||
// Binds no `run`: one use per function, as before.
|
||||
'noBinding L64',
|
||||
'paramValue L71',
|
||||
'localValue L76',
|
||||
// A func literal at file scope is its own unit.
|
||||
'literalThenUse L84',
|
||||
'literalThenUse L86',
|
||||
]);
|
||||
});
|
||||
}
|
||||
|
||||
it.runIf(kernelAvailable)('the kernel and the wasm extractor record the same references', () => {
|
||||
const kernel = extract('kernel', MAIN_GO);
|
||||
const wasm = extract('wasm', MAIN_GO);
|
||||
const refs = (r: ExtractionResult) =>
|
||||
r.unresolvedReferences.map((u) => `${u.referenceKind} ${u.referenceName} ${u.line}:${u.column}`);
|
||||
expect(refs(kernel)).toEqual(refs(wasm));
|
||||
});
|
||||
});
|
||||
|
||||
describe('the references a function keeps to the package function (#2596)', () => {
|
||||
let tempDir: string;
|
||||
let cg: CodeGraph | null = null;
|
||||
|
||||
beforeAll(async () => {
|
||||
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-2596-'));
|
||||
fs.writeFileSync(path.join(tempDir, 'go.mod'), GO_MOD);
|
||||
fs.writeFileSync(path.join(tempDir, 'main.go'), MAIN_GO);
|
||||
cg = await CodeGraph.init(tempDir, { index: true });
|
||||
cg.resolveReferences();
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => {
|
||||
cg?.close();
|
||||
cg = null;
|
||||
if (tempDir) fs.rmSync(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
/** `L<line>` of each `references` edge from `fromName` to the package `func run`. */
|
||||
function refsToRun(fromName: string, kind: 'function' | 'variable' = 'function'): string[] {
|
||||
const from = cg!.getNodesByKind(kind).find((n) => n.name === fromName);
|
||||
expect(from, fromName).toBeDefined();
|
||||
return cg!
|
||||
.getOutgoingEdges(from!.id)
|
||||
.filter((e) => e.kind === 'references')
|
||||
.filter((e) => {
|
||||
const target = cg!.getNode(e.target);
|
||||
return target?.kind === 'function' && target.name === 'run';
|
||||
})
|
||||
.map((e) => `L${e.line}`)
|
||||
.sort();
|
||||
}
|
||||
|
||||
it.each([
|
||||
['an inner block local', 'shadowThenUse', 'L14'],
|
||||
['a func literal parameter', 'closureThenUse', 'L20'],
|
||||
['a local in another case clause', 'caseThenUse', 'L29'],
|
||||
['an if-header local', 'headerThenUse', 'L37'],
|
||||
['a range variable', 'rangeThenUse', 'L44'],
|
||||
])('a later use outside %s links to the package function', (_label, fn, line) => {
|
||||
expect(refsToRun(fn)).toEqual([line]);
|
||||
});
|
||||
|
||||
it('a later use in a func literal at file scope links to the package function', () => {
|
||||
expect(refsToRun('literalThenUse', 'variable')).toEqual(['L86']);
|
||||
});
|
||||
|
||||
it('a function whose first use is the package function keeps that one reference', () => {
|
||||
expect(refsToRun('useThenShadow')).toEqual(['L48']);
|
||||
});
|
||||
|
||||
it('a function that binds no local of the name keeps its single reference', () => {
|
||||
expect(refsToRun('noBinding')).toEqual(['L64']);
|
||||
});
|
||||
|
||||
it('no use the local shadows links to the package function', () => {
|
||||
expect(refsToRun('allShadowed')).toEqual([]);
|
||||
expect(refsToRun('paramValue')).toEqual([]);
|
||||
expect(refsToRun('localValue')).toEqual([]);
|
||||
});
|
||||
});
|
||||
+180
-16
@@ -68,12 +68,15 @@ struct ValueScope<'t> {
|
||||
name: String,
|
||||
}
|
||||
|
||||
struct Cand {
|
||||
struct Cand<'t> {
|
||||
from: u32,
|
||||
name: String,
|
||||
line: u32,
|
||||
column_byte: usize,
|
||||
row: usize,
|
||||
/// The value container the candidate came from: the flush reads its
|
||||
/// scope from there when the function shadows the name (#2596).
|
||||
node: Node<'t>,
|
||||
}
|
||||
|
||||
/// Per-node metadata for the receiver-method owner lookup (mirrors the TS
|
||||
@@ -95,7 +98,7 @@ pub struct Walker<'t> {
|
||||
node_ids: Vec<String>,
|
||||
defined_fn_names: HashSet<String>,
|
||||
imported_names: HashSet<String>,
|
||||
fn_ref_cands: Vec<Cand>,
|
||||
fn_ref_cands: Vec<Cand<'t>>,
|
||||
fs_values: HashMap<String, u32>,
|
||||
fs_value_counts: HashMap<String, u32>,
|
||||
value_scopes: Vec<ValueScope<'t>>,
|
||||
@@ -1187,13 +1190,13 @@ impl<'t> Walker<'t> {
|
||||
}
|
||||
|
||||
for v in values {
|
||||
self.normalize_fn_ref_value(v, from, 0);
|
||||
self.normalize_fn_ref_value(v, from, 0, node);
|
||||
}
|
||||
}
|
||||
|
||||
/// normalizeValue with GO_SPEC's transparent layers (literal_element,
|
||||
/// expression_list — both fan out to named children).
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32) {
|
||||
fn normalize_fn_ref_value(&mut self, v: Node<'t>, from: u32, depth: u32, container: Node<'t>) {
|
||||
stack_guard!();
|
||||
if depth > 4 {
|
||||
return;
|
||||
@@ -1211,6 +1214,7 @@ impl<'t> Walker<'t> {
|
||||
line: p.row as u32 + 1,
|
||||
column_byte: v.start_byte(),
|
||||
row: p.row,
|
||||
node: container,
|
||||
});
|
||||
}
|
||||
// #1820: preserve the receiver of a method value. The name is
|
||||
@@ -1238,12 +1242,13 @@ impl<'t> Walker<'t> {
|
||||
line: p.row as u32 + 1,
|
||||
column_byte: field.start_byte(),
|
||||
row: p.row,
|
||||
node: container,
|
||||
});
|
||||
}
|
||||
"literal_element" | "expression_list" => {
|
||||
for i in 0..v.named_child_count() {
|
||||
if let Some(c) = v.named_child(i) {
|
||||
self.normalize_fn_ref_value(c, from, depth + 1);
|
||||
self.normalize_fn_ref_value(c, from, depth + 1, container);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1282,17 +1287,24 @@ impl<'t> Walker<'t> {
|
||||
if cands.is_empty() || util::is_generated_file(self.file_path) {
|
||||
return;
|
||||
}
|
||||
let mut seen: HashSet<(String, String)> = HashSet::new();
|
||||
for c in cands {
|
||||
if !c.name.starts_with("this.")
|
||||
&& !c.name.contains('.')
|
||||
&& !c.name.contains("::")
|
||||
&& !self.defined_fn_names.contains(&c.name)
|
||||
&& !self.imported_names.contains(&c.name)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
if !seen.insert((self.node_ids[c.from as usize].clone(), c.name.clone())) {
|
||||
let survivors: Vec<Cand<'t>> = cands
|
||||
.into_iter()
|
||||
.filter(|c| {
|
||||
c.name.starts_with("this.")
|
||||
|| c.name.contains('.')
|
||||
|| c.name.contains("::")
|
||||
|| self.defined_fn_names.contains(&c.name)
|
||||
|| self.imported_names.contains(&c.name)
|
||||
})
|
||||
.collect();
|
||||
// One reference per function and name, at its first use; one per
|
||||
// scope where the function binds the name itself (#2596). Mirrors
|
||||
// flushFnRefCandidates / goScopedFnRefKeys.
|
||||
let scopes = go_scoped_fn_ref_scopes(&survivors, &self.node_ids, self.src);
|
||||
let mut seen: HashSet<(String, String, Option<usize>)> = HashSet::new();
|
||||
for (i, c) in survivors.into_iter().enumerate() {
|
||||
let scope = scopes.get(&i).copied();
|
||||
if !seen.insert((self.node_ids[c.from as usize].clone(), c.name.clone(), scope)) {
|
||||
continue;
|
||||
}
|
||||
let column = util::col16(self.src, &self.line_starts, c.row, c.column_byte);
|
||||
@@ -1480,3 +1492,155 @@ fn opt_str(arena: &mut Arena, s: Option<&str>) -> StrRef {
|
||||
None => NONE_STR,
|
||||
}
|
||||
}
|
||||
|
||||
// --- a local that shadows the function a value names (#2596) ---------------------
|
||||
|
||||
/// Go nodes that open a scope a local can be declared in: a block, the
|
||||
/// implicit block of an `if` / `for` / `switch` / `select` statement, and each
|
||||
/// clause of a `switch` or `select`. Mirrors GO_SCOPE_TYPES (function-ref.ts).
|
||||
const GO_SCOPE_TYPES: [&str; 10] = [
|
||||
"block",
|
||||
"if_statement",
|
||||
"for_statement",
|
||||
"expression_switch_statement",
|
||||
"type_switch_statement",
|
||||
"select_statement",
|
||||
"expression_case",
|
||||
"default_case",
|
||||
"type_case",
|
||||
"communication_case",
|
||||
];
|
||||
|
||||
/// Where a function-value candidate sits: the innermost scope node's id
|
||||
/// (usize::MAX when there is none) and the unit it is written in: the
|
||||
/// function or method declaration, or for a func literal at file scope the
|
||||
/// outermost literal, else None. Mirrors goFnRefSite.
|
||||
fn go_fn_ref_site<'t>(node: Node<'t>) -> (usize, Option<Node<'t>>) {
|
||||
let mut scope = usize::MAX;
|
||||
let mut literal: Option<Node<'t>> = None;
|
||||
let mut cur = Some(node);
|
||||
while let Some(n) = cur {
|
||||
if scope == usize::MAX && GO_SCOPE_TYPES.contains(&n.kind()) {
|
||||
scope = n.id();
|
||||
}
|
||||
if matches!(n.kind(), "function_declaration" | "method_declaration") {
|
||||
return (scope, Some(n));
|
||||
}
|
||||
if n.kind() == "func_literal" {
|
||||
literal = Some(n);
|
||||
}
|
||||
cur = n.parent();
|
||||
}
|
||||
(scope, literal)
|
||||
}
|
||||
|
||||
/// The names among `names` that a function binds itself, as (unit id, name):
|
||||
/// its func literals' parameters, `:=`, `var` and `const`, range and
|
||||
/// type-switch variables, and a select case's received value. A name the
|
||||
/// function's own receiver, parameters or results bind is left out: those
|
||||
/// cover its whole body. Mirrors goLocalBindings.
|
||||
fn go_local_bindings<'t>(units: &[Node<'t>], names: &HashSet<&str>, src: &str) -> HashSet<(usize, String)> {
|
||||
let mut out: HashSet<(usize, String)> = HashSet::new();
|
||||
for unit in units {
|
||||
let mut bind = |id: Option<Node<'t>>| {
|
||||
if let Some(id) = id {
|
||||
if id.kind() == "identifier" {
|
||||
let name = &src[id.start_byte()..id.end_byte()];
|
||||
if names.contains(name) {
|
||||
out.insert((unit.id(), name.to_string()));
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
let declares = |n: Node<'t>| {
|
||||
(0..n.child_count()).any(|i| n.child(i).map_or(false, |c| !c.is_named() && c.kind() == ":="))
|
||||
};
|
||||
let mut stack: Vec<Node<'t>> = vec![*unit];
|
||||
while let Some(n) = stack.pop() {
|
||||
let list = match n.kind() {
|
||||
"short_var_declaration" => n.child_by_field_name("left"),
|
||||
"range_clause" | "receive_statement" if declares(n) => n.child_by_field_name("left"),
|
||||
"type_switch_statement" => n.child_by_field_name("alias"),
|
||||
_ => None,
|
||||
};
|
||||
if let Some(list) = list {
|
||||
for i in 0..list.named_child_count() {
|
||||
bind(list.named_child(i));
|
||||
}
|
||||
}
|
||||
if matches!(n.kind(), "var_spec" | "const_spec" | "parameter_declaration" | "variadic_parameter_declaration") {
|
||||
let mut cursor = n.walk();
|
||||
for c in n.children_by_field_name("name", &mut cursor) {
|
||||
bind(Some(c));
|
||||
}
|
||||
}
|
||||
for i in (0..n.named_child_count()).rev() {
|
||||
if let Some(c) = n.named_child(i) {
|
||||
stack.push(c);
|
||||
}
|
||||
}
|
||||
}
|
||||
for field in ["receiver", "parameters", "result"] {
|
||||
let Some(list) = unit.child_by_field_name(field) else { continue };
|
||||
if list.kind() != "parameter_list" {
|
||||
continue;
|
||||
}
|
||||
for i in 0..list.named_child_count() {
|
||||
let Some(p) = list.named_child(i) else { continue };
|
||||
if !matches!(p.kind(), "parameter_declaration" | "variadic_parameter_declaration") {
|
||||
continue;
|
||||
}
|
||||
let mut cursor = p.walk();
|
||||
for c in p.children_by_field_name("name", &mut cursor) {
|
||||
if c.kind() == "identifier" {
|
||||
out.remove(&(unit.id(), src[c.start_byte()..c.end_byte()].to_string()));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The candidates, by index, that get one reference per scope instead of
|
||||
/// one per function, each with the id of the scope it sits in: names a
|
||||
/// function uses at least twice and binds itself. Only those candidates'
|
||||
/// positions are read from the tree. Mirrors goScopedFnRefScopes.
|
||||
fn go_scoped_fn_ref_scopes<'t>(cands: &[Cand<'t>], node_ids: &[String], src: &str) -> HashMap<usize, usize> {
|
||||
let mut groups: HashMap<(&str, &str), Vec<usize>> = HashMap::new();
|
||||
for (i, c) in cands.iter().enumerate() {
|
||||
if c.name.contains('.') {
|
||||
continue;
|
||||
}
|
||||
groups.entry((node_ids[c.from as usize].as_str(), c.name.as_str())).or_default().push(i);
|
||||
}
|
||||
let mut sites: Vec<(usize, usize, Node<'t>)> = Vec::new();
|
||||
let mut units: Vec<Node<'t>> = Vec::new();
|
||||
let mut unit_ids: HashSet<usize> = HashSet::new();
|
||||
let mut names: HashSet<&str> = HashSet::new();
|
||||
for members in groups.values() {
|
||||
if members.len() < 2 {
|
||||
continue;
|
||||
}
|
||||
for &i in members {
|
||||
let (scope, unit) = go_fn_ref_site(cands[i].node);
|
||||
let Some(unit) = unit else { continue };
|
||||
sites.push((i, scope, unit));
|
||||
if unit_ids.insert(unit.id()) {
|
||||
units.push(unit);
|
||||
}
|
||||
names.insert(cands[i].name.as_str());
|
||||
}
|
||||
}
|
||||
let mut out: HashMap<usize, usize> = HashMap::new();
|
||||
if units.is_empty() {
|
||||
return out;
|
||||
}
|
||||
let bound = go_local_bindings(&units, &names, src);
|
||||
for (i, scope, unit) in sites {
|
||||
if bound.contains(&(unit.id(), cands[i].name.clone())) {
|
||||
out.insert(i, scope);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
@@ -869,3 +869,163 @@ function rubyEnclosingCall(node: SyntaxNode): SyntaxNode | null {
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Go: a local that shadows the function a value names (#2596)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Go nodes that open a scope a local can be declared in: a block, the implicit
|
||||
* block of an `if` / `for` / `switch` / `select` statement, and each clause of
|
||||
* a `switch` or `select`.
|
||||
*/
|
||||
const GO_SCOPE_TYPES = new Set([
|
||||
'block',
|
||||
'if_statement',
|
||||
'for_statement',
|
||||
'expression_switch_statement',
|
||||
'type_switch_statement',
|
||||
'select_statement',
|
||||
'expression_case',
|
||||
'default_case',
|
||||
'type_case',
|
||||
'communication_case',
|
||||
]);
|
||||
|
||||
/**
|
||||
* Where a Go function-value candidate sits: the innermost scope node around
|
||||
* it, and the unit it is written in: the function or method declaration, or
|
||||
* for a func literal at file scope (`var h = func(…) {…}`) the outermost
|
||||
* literal, else null. Mirrored in the kernel (go.rs `go_fn_ref_site`).
|
||||
*/
|
||||
export interface GoFnRefSite {
|
||||
scope: number;
|
||||
unit: SyntaxNode | null;
|
||||
}
|
||||
|
||||
/** {@link GoFnRefSite} of the value container `node`. */
|
||||
export function goFnRefSite(node: SyntaxNode): GoFnRefSite {
|
||||
let scope = -1;
|
||||
let literal: SyntaxNode | null = null;
|
||||
for (let cur: SyntaxNode | null = node; cur; cur = cur.parent) {
|
||||
if (scope < 0 && GO_SCOPE_TYPES.has(cur.type)) scope = cur.id;
|
||||
if (cur.type === 'function_declaration' || cur.type === 'method_declaration') return { scope, unit: cur };
|
||||
if (cur.type === 'func_literal') literal = cur;
|
||||
}
|
||||
return { scope, unit: literal };
|
||||
}
|
||||
|
||||
/**
|
||||
* The names among `names` that a Go function binds itself, keyed
|
||||
* `<unit id>|<name>`: its func literals' parameters, `:=`, `var` and `const`,
|
||||
* range and type-switch variables, and a select case's received value. Such a
|
||||
* local shadows a package function of the same name for the rest of its
|
||||
* scope, which the resolver reads per use (`isGoLocallyBound`). A name the
|
||||
* function's own receiver, parameters or results bind is left out: those
|
||||
* cover its whole body, so every use is the local and one reference is
|
||||
* enough. Mirrored in the kernel (go.rs `go_local_bindings`).
|
||||
*/
|
||||
export function goLocalBindings(
|
||||
units: Iterable<SyntaxNode>,
|
||||
names: ReadonlySet<string>,
|
||||
source: string
|
||||
): Set<string> {
|
||||
const out = new Set<string>();
|
||||
for (const unit of units) {
|
||||
const bind = (id: SyntaxNode | null | undefined): void => {
|
||||
if (id?.type !== 'identifier') return;
|
||||
const name = getNodeText(id, source);
|
||||
if (names.has(name)) out.add(`${unit.id}|${name}`);
|
||||
};
|
||||
const declares = (n: SyntaxNode): boolean => {
|
||||
for (let i = 0; i < n.childCount; i++) {
|
||||
const c = n.child(i);
|
||||
if (c && !c.isNamed && c.type === ':=') return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
const stack: SyntaxNode[] = [unit];
|
||||
while (stack.length > 0) {
|
||||
const n = stack.pop()!;
|
||||
switch (n.type) {
|
||||
case 'short_var_declaration':
|
||||
for (const c of getChildByField(n, 'left')?.namedChildren ?? []) bind(c);
|
||||
break;
|
||||
case 'range_clause':
|
||||
case 'receive_statement':
|
||||
if (declares(n)) for (const c of getChildByField(n, 'left')?.namedChildren ?? []) bind(c);
|
||||
break;
|
||||
case 'type_switch_statement':
|
||||
for (const c of getChildByField(n, 'alias')?.namedChildren ?? []) bind(c);
|
||||
break;
|
||||
case 'var_spec':
|
||||
case 'const_spec':
|
||||
case 'parameter_declaration':
|
||||
case 'variadic_parameter_declaration':
|
||||
for (const c of n.childrenForFieldName('name')) bind(c);
|
||||
break;
|
||||
}
|
||||
for (let i = n.namedChildCount - 1; i >= 0; i--) {
|
||||
const c = n.namedChild(i);
|
||||
if (c) stack.push(c);
|
||||
}
|
||||
}
|
||||
for (const field of ['receiver', 'parameters', 'result']) {
|
||||
const list = getChildByField(unit, field);
|
||||
if (list?.type !== 'parameter_list') continue;
|
||||
for (const p of list.namedChildren) {
|
||||
if (p.type !== 'parameter_declaration' && p.type !== 'variadic_parameter_declaration') continue;
|
||||
for (const c of p.childrenForFieldName('name')) {
|
||||
if (c.type === 'identifier') out.delete(`${unit.id}|${getNodeText(c, source)}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* The Go function-value candidates that need one reference per scope instead
|
||||
* of one per function (#2596), each with the id of the scope it sits in. A
|
||||
* function records a value-use of a name once, at its first use; but when the
|
||||
* function also binds that name itself, the first use can be the local while
|
||||
* a later one, in another scope, is the package function. The resolver drops
|
||||
* a use the local shadows, so the later use needs a reference of its own. One
|
||||
* per scope node is enough: within one scope a local declared before the
|
||||
* first use still covers every later use. Only a name a function uses at
|
||||
* least twice is looked up, so a candidate's position is read from the tree
|
||||
* just for those. Mirrored in the kernel (go.rs `go_scoped_fn_ref_scopes`).
|
||||
*/
|
||||
export function goScopedFnRefScopes<C extends { name: string; fromNodeId: string; goNode?: SyntaxNode }>(
|
||||
candidates: ReadonlyArray<C>,
|
||||
source: string
|
||||
): Map<C, number> {
|
||||
const groups = new Map<string, C[]>();
|
||||
for (const c of candidates) {
|
||||
if (!c.goNode || c.name.includes('.')) continue;
|
||||
const key = `${c.fromNodeId}|${c.name}`;
|
||||
const group = groups.get(key);
|
||||
if (group) group.push(c);
|
||||
else groups.set(key, [c]);
|
||||
}
|
||||
const sites = new Map<C, GoFnRefSite>();
|
||||
const units = new Map<number, SyntaxNode>();
|
||||
const names = new Set<string>();
|
||||
for (const group of groups.values()) {
|
||||
if (group.length < 2) continue;
|
||||
for (const c of group) {
|
||||
const site = goFnRefSite(c.goNode!);
|
||||
if (!site.unit) continue;
|
||||
sites.set(c, site);
|
||||
units.set(site.unit.id, site.unit);
|
||||
names.add(c.name);
|
||||
}
|
||||
}
|
||||
const out = new Map<C, number>();
|
||||
if (units.size === 0) return out;
|
||||
const bound = goLocalBindings(units.values(), names, source);
|
||||
for (const [c, site] of sites) {
|
||||
if (bound.has(`${site.unit!.id}|${c.name}`)) out.set(c, site.scope);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
@@ -17,7 +17,7 @@ import {
|
||||
} from '../types';
|
||||
import { getParser, detectLanguage, isLanguageSupported, isFileLevelOnlyLanguage } from './grammars';
|
||||
import { NodeIdAllocator, getNodeText, getChildByField, getPrecedingDocstring, BUILTIN_TYPE_NAMES } from './tree-sitter-helpers';
|
||||
import { FN_REF_SPECS, captureFnRefCandidates, type FnRefSpec, type FnRefCandidate } from './function-ref';
|
||||
import { FN_REF_SPECS, captureFnRefCandidates, goScopedFnRefScopes, type FnRefSpec, type FnRefCandidate } from './function-ref';
|
||||
import { isGeneratedFile, isMinifiedContent } from './generated-detection';
|
||||
import type { LanguageExtractor, ExtractorContext } from './tree-sitter-types';
|
||||
import { EXTRACTORS } from './languages';
|
||||
@@ -732,7 +732,9 @@ export class TreeSitterExtractor {
|
||||
// during the walk, gated & flushed into unresolvedReferences at end-of-file
|
||||
// (see flushFnRefCandidates).
|
||||
private fnRefSpec: FnRefSpec | undefined;
|
||||
private fnRefCandidates: Array<FnRefCandidate & { fromNodeId: string }> = [];
|
||||
// A Go candidate also keeps the value container it came from: the flush
|
||||
// reads its scope from there when the function shadows the name (#2596).
|
||||
private fnRefCandidates: Array<FnRefCandidate & { fromNodeId: string; goNode?: SyntaxNode }> = [];
|
||||
// Memoized "is this a Vue store file" verdict (per-extractor = per-file).
|
||||
private vueStoreFile: boolean | null = null;
|
||||
// An object literal hung on a path (`window.App = {…}`, `ns.mod = {…}`) is
|
||||
@@ -928,8 +930,9 @@ export class TreeSitterExtractor {
|
||||
if (!rule || this.nodeStack.length === 0) return;
|
||||
const fromNodeId = this.nodeStack[this.nodeStack.length - 1];
|
||||
if (!fromNodeId) return;
|
||||
const goNode = this.language === 'go' ? node : undefined;
|
||||
for (const cand of captureFnRefCandidates(node, rule, spec, this.source)) {
|
||||
this.fnRefCandidates.push({ ...cand, fromNodeId });
|
||||
this.fnRefCandidates.push({ ...cand, fromNodeId, goNode });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1024,7 +1027,7 @@ export class TreeSitterExtractor {
|
||||
|
||||
const ungated = this.fnRefSpec?.ungatedModes;
|
||||
const addressOfOnly = this.fnRefSpec?.addressOfOnly === true;
|
||||
const seen = new Set<string>();
|
||||
const survivors: typeof candidates = [];
|
||||
for (const c of candidates) {
|
||||
const atFileScope = c.fromNodeId.startsWith('file:');
|
||||
// C++ (addressOfOnly): a BARE identifier qualifies only inside a
|
||||
@@ -1062,7 +1065,18 @@ export class TreeSitterExtractor {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
const key = `${c.fromNodeId}|${c.name}`;
|
||||
survivors.push(c);
|
||||
}
|
||||
|
||||
// One reference per function and name, at its first use. In Go, a
|
||||
// function that binds the name itself gets one per scope instead (#2596):
|
||||
// its first use may be the local, which resolution drops, while a later
|
||||
// use in another scope is the package function.
|
||||
const goScopes = this.language === 'go' ? goScopedFnRefScopes(survivors, this.source) : null;
|
||||
const seen = new Set<string>();
|
||||
for (const c of survivors) {
|
||||
const scope = goScopes?.get(c);
|
||||
const key = scope === undefined ? `${c.fromNodeId}|${c.name}` : `${c.fromNodeId}|${c.name}|${scope}`;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
this.unresolvedReferences.push({
|
||||
|
||||
Reference in New Issue
Block a user