mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-11-14 16:52:09 +08:00
fix(resolution): C, C++ and Objective-C rank as one language, and a lone candidate is judged by family (#3052)
Name matching compared a candidate's raw language tag with the reference's: the single-candidate confidence in matchByExactName and matchFuzzy, and findBestMatch's same-language skip and +50/-80 score. So a call whose name had several definitions in another of C, C++ and Objective-C scored -80 for each and linked to nothing (an Objective-C file's _glfwPlatformSetTls(), a C++ call into Dear ImGui's imgui.h, which reads as C), and the one definition of a name across TSX and TS, or C and C++, was a 0.5 cross-language guess. The single-candidate confidence now compares language families (sameLanguageFamily). Ranking treats C, C++ and Objective-C as one language (sameRankingLanguage), the very language worth a point that only breaks a tie; other families keep the language itself, because ranking them by family moved excalidraw's Store::scheduleCapture calls to the App.tsx method beside them and gave okhttp 83 wrong Java-to-Kotlin Builder picks. Validation: - __tests__/name-match-language-family.test.ts, 8 tests (default and wasm): 6 fail on the base main; the pin that a .ts call keeps its .ts target over a .tsx namesake beside it passes on both; all pass with the fix. 24 related test files (cross-language, React Native, Expo, Fabric, frameworks, mixed TS/JSX, Objective-C, resolution): 746 pass. - A/B, main vs fix: glfw +4 calls from Objective-C into C, imgui +93 from C++ into imgui.h (read as C) and imstb_rectpack.h, all to the definitions meant; excalidraw, okhttp, leveldb and jq gain or lose no edge (43, 9, 97 and 30 unique-name links go from 0.5 to 0.9 with the same targets); flask and gin byte-identical. - Index time, median of 3 interleaved: imgui 7.9 s main, 7.7 s fix; excalidraw 9.2 s main, 9.4 s fix. Reimplements PR #1662 by ferrine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
19b74d10f3
commit
6d3804cf57
@@ -236,6 +236,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
- In C++, a name written in a googletest `TEST_F` or `TEST_P` body now finds the test fixture's nested types, those of its base classes included, before anything in the surrounding namespace, as C++ does for the member function such a body is: facebook/rocksdb's `new TestUserDefinedIndexFactory()` links to the class nested in the fixture's base. A class declared inside a test body, or inside any function, is now only found from that function, so the local classes many RocksDB tests each declare under one name, like `FakeWF` or `MyManager`, no longer stand in for one another, and a call like `fwf->SetIOError(true)` reaches the `FakeWF` its own test declares. Re-index C++ projects after upgrading.
|
||||
- In C and C++ tests, a name that several files declare now links to the one the test's own includes declare, when there is one: protocolbuffers/protobuf's C# generator tests call the `TestSourceDir()` that `googletest.h` declares, not another test helper's they never include; its upb tests call `upb_Arena_New()` and the rest of upb's C API in upb's own files, not the copies bundled for PHP and Ruby, which had left those calls unlinked; and facebook/rocksdb's C API test reaches `rocksdb_open` and its siblings in `db/c.cc`. Re-index C and C++ projects after upgrading.
|
||||
- In C and C++ code outside tests too, a name that several files declare now links to the one the file's own includes declare, when there is one: protocolbuffers/protobuf's Python extension calls functions like `upb_Encode()` in upb's own files rather than the copies bundled for PHP, and facebook/rocksdb's stress tool calls the `StringToHex()` its own header declares rather than `LDBCommand::StringToHex`. Re-index C and C++ projects after upgrading.
|
||||
- In C, C++ and Objective-C, a call into a function that another of the three languages defines now links to it even when the name has more than one definition, like a macOS `.m` file's `_glfwPlatformSetTls(…)`, which glfw defines in both `posix_thread.c` and `win32_thread.c`, or a C++ call into a header that reads as C, like Dear ImGui's `ImVector` methods in `imgui.h`. Before, such a call linked to nothing. Re-index C, C++ and Objective-C projects after upgrading. Thanks @ferrine. (#1662)
|
||||
- In C and C++, a call of a function a header declares now links to its definition even when the file that defines it reaches that header only through other headers: protocolbuffers/protobuf's Python extension calls upb's reflection functions, like `upb_FieldDef_Number()`, in upb's own files rather than the copies bundled for PHP. Re-index C and C++ projects after upgrading.
|
||||
- In C++, a declaration continued on a line that starts with `*`, like facebook/rocksdb's `const auto& el =` over `*reinterpret_cast<std::shared_ptr<EventListener>*>(jhandle);`, is now read, so calls such as `el->OnFlushCompleted(…)` link to `EventListener`'s methods; such a line used to be skipped as part of a comment. Code commented out in a `/* … */` block, and the text of a string literal, like protocolbuffers/protobuf's `[[deprecated("Please use SpaceUsedLong() instead")]]`, are no longer read as declarations, which had sent a call to the wrong method or to none. Re-index C++ projects after upgrading.
|
||||
- In C++, a call with a comparison or a shift among its arguments, like protocolbuffers/protobuf's `p->Emit({{"declare", has_bit_word_index < 0 ? … : ""}, …}, "…")` or `WriteVarint(num << 3, s)`, is now read with the right number of arguments, so it links to the overload that fits rather than one taking fewer. A parameter or variable whose type has template arguments is now read even after another such parameter on the same line, like facebook/rocksdb's `const autovector<uint32_t>& cf_ids`, or with empty ones, like fmt's `parse_context<>& ctx`: calls on it link to that class's methods, and calls on a `std::vector` declared that way no longer link to a project class's method of the same name. Re-index C++ projects after upgrading.
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
/**
|
||||
* Name matching weighed candidates by their raw language tag, so a `.tsx`
|
||||
* file's call to the one `.ts` helper of its name was a cross-language guess
|
||||
* (0.5) and an Objective-C or C++ file's call into C found nothing at all:
|
||||
*
|
||||
* app/window.m setTls(slot); // defined in two .c files
|
||||
* app/main.cpp getTime(); // app/clock.c, or win/clock_win.c
|
||||
* ui/Row.tsx formatDate(d); // the one in lib/fmt.ts
|
||||
*
|
||||
* The one candidate of a name is now judged by language family (TypeScript and
|
||||
* TSX, Java and Kotlin, C and C++ and Objective-C are one each), and C, C++
|
||||
* and Objective-C — whose files share one table of names, and whose `.h` files
|
||||
* are told apart only by sniffing — are ranked as one language. Other
|
||||
* families keep the language itself for ranking: a `.ts` file's
|
||||
* `this.app.store.scheduleCapture()` still means the `.ts` Store's method, not
|
||||
* the `.tsx` App's beside it.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import CodeGraph from '../src/index';
|
||||
|
||||
const FILES: Record<string, string> = {
|
||||
// A TSX component calling the one `.ts` helper of the name (a classic
|
||||
// script: no import or export), and a `.ts` file calling a method of a
|
||||
// `.ts` class while a `.tsx` class beside it has one of the same name.
|
||||
'web/ui/Row.tsx': 'export function Row() { return <div>{formatDate(new Date())}</div>; }\n',
|
||||
'web/lib/fmt.ts': 'function formatDate(d: Date) { return d.toISOString(); }\n',
|
||||
'web/components/App.tsx': [
|
||||
'export class App {',
|
||||
' store = new Store();',
|
||||
' scheduleCapture() { return 1; }',
|
||||
' render() { return <div />; }',
|
||||
'}',
|
||||
'',
|
||||
].join('\n'),
|
||||
'web/components/App.clipboard.ts': [
|
||||
'export class AppClipboard {',
|
||||
' constructor(private app: any) {}',
|
||||
' paste() {',
|
||||
' this.app.store.scheduleCapture();',
|
||||
' }',
|
||||
'}',
|
||||
'',
|
||||
].join('\n'),
|
||||
'web/store/store.ts': [
|
||||
'export class Store {',
|
||||
' scheduleCapture() { return 2; }',
|
||||
'}',
|
||||
'',
|
||||
].join('\n'),
|
||||
// C, C++ and Objective-C: one table of names.
|
||||
'native/app/init.c': 'void inputError(int code) { (void)code; }\n',
|
||||
'native/app/posix_thread.c': 'void setTls(int *slot) { (void)slot; }\n',
|
||||
'native/win/win32_thread.c': 'void setTls(int *slot) { (void)slot; }\n',
|
||||
'native/app/window.m': [
|
||||
'#import <Foundation/Foundation.h>',
|
||||
'',
|
||||
'void makeCurrent(int *slot) {',
|
||||
' setTls(slot);',
|
||||
' inputError(1);',
|
||||
'}',
|
||||
'',
|
||||
].join('\n'),
|
||||
'native/app/clock.c': 'long getTime(void) { return 1; }\n',
|
||||
'native/win/clock_win.c': 'long getTime(void) { return 2; }\n',
|
||||
'native/app/main.cpp': [
|
||||
'long run() {',
|
||||
' return getTime();',
|
||||
'}',
|
||||
'',
|
||||
].join('\n'),
|
||||
};
|
||||
|
||||
describe.each(['default', 'wasm'])('name matching reads language families (%s)', (backend) => {
|
||||
let root = '';
|
||||
let cg: CodeGraph | undefined;
|
||||
let kernel: string | undefined;
|
||||
|
||||
beforeAll(async () => {
|
||||
kernel = process.env.CODEGRAPH_KERNEL;
|
||||
if (backend === 'wasm') process.env.CODEGRAPH_KERNEL = '0';
|
||||
else delete process.env.CODEGRAPH_KERNEL;
|
||||
root = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-lang-family-'));
|
||||
for (const [rel, content] of Object.entries(FILES)) {
|
||||
fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, rel), content);
|
||||
}
|
||||
cg = await CodeGraph.init(root, { index: true });
|
||||
}, 60_000);
|
||||
|
||||
afterAll(() => {
|
||||
cg?.destroy();
|
||||
if (root) fs.rmSync(root, { recursive: true, force: true });
|
||||
if (kernel === undefined) delete process.env.CODEGRAPH_KERNEL;
|
||||
else process.env.CODEGRAPH_KERNEL = kernel;
|
||||
});
|
||||
|
||||
/** `target file:name@confidence` of each call out of `name` in `file`, sorted. */
|
||||
const callsFrom = (file: string, name: string): string[] => {
|
||||
const from = cg!.getNodesInFile(file).filter((n) => n.name === name && (n.kind === 'function' || n.kind === 'method'));
|
||||
expect(from, `${name} in ${file}`).toHaveLength(1);
|
||||
return cg!.getOutgoingEdgesFrom([from[0]!.id])
|
||||
.filter((e) => e.kind === 'calls')
|
||||
.map((e) => {
|
||||
const t = cg!.getNode(e.target)!;
|
||||
return `${t.filePath}:${t.name}@${(e.metadata as { confidence?: number } | undefined)?.confidence}`;
|
||||
})
|
||||
.sort();
|
||||
};
|
||||
|
||||
const withoutConfidence = (call: string): string => call.replace(/@.*$/, '');
|
||||
|
||||
it('a TSX file calling the one TypeScript function of the name is a same-family match', () => {
|
||||
expect(callsFrom('web/ui/Row.tsx', 'Row')).toEqual(['web/lib/fmt.ts:formatDate@0.9']);
|
||||
});
|
||||
|
||||
it('TypeScript and TSX are still ranked by their own language', () => {
|
||||
expect(callsFrom('web/components/App.clipboard.ts', 'paste').map(withoutConfidence)).toEqual(['web/store/store.ts:scheduleCapture']);
|
||||
});
|
||||
|
||||
it('Objective-C calls into C: the one function, and the nearer of two', () => {
|
||||
const calls = callsFrom('native/app/window.m', 'makeCurrent');
|
||||
expect(calls.map(withoutConfidence)).toEqual(['native/app/init.c:inputError', 'native/app/posix_thread.c:setTls']);
|
||||
expect(calls).toContain('native/app/init.c:inputError@0.9');
|
||||
});
|
||||
|
||||
it('C++ calls into C: the nearer of two definitions', () => {
|
||||
expect(callsFrom('native/app/main.cpp', 'run').map(withoutConfidence)).toEqual(['native/app/clock.c:getTime']);
|
||||
});
|
||||
});
|
||||
@@ -175,6 +175,21 @@ export function sameLanguageFamily(a: string, b: string): boolean {
|
||||
const fa = LANGUAGE_FAMILY[a];
|
||||
return fa !== undefined && fa === LANGUAGE_FAMILY[b];
|
||||
}
|
||||
|
||||
/**
|
||||
* The languages whose files share one table of names: a C function is called
|
||||
* by its bare name from C++ and Objective-C alike, and a `.h` file is C, C++
|
||||
* or Objective-C only as far as sniffing its text can tell (Dear ImGui's
|
||||
* C++ `imgui.h` reads as C), so name matching ranks them as one language.
|
||||
* Other families keep the language itself: their files reach each other's
|
||||
* names through imports, and a name-only pick across TypeScript and TSX, or
|
||||
* Java and Kotlin, went to a same-named declaration in the next file more
|
||||
* often than to the one meant (#1662).
|
||||
*/
|
||||
const C_LINKED_LANGUAGES: ReadonlySet<string> = new Set(['c', 'cpp', 'objc']);
|
||||
function sameRankingLanguage(a: string, b: string): boolean {
|
||||
return a === b || (C_LINKED_LANGUAGES.has(a) && C_LINKED_LANGUAGES.has(b));
|
||||
}
|
||||
/** Config/markup transitions stay open; every other code language has a family. */
|
||||
const CODE_FAMILY: Record<string, string> = {
|
||||
...LANGUAGE_FAMILY,
|
||||
@@ -13283,10 +13298,11 @@ export function matchByExactName(
|
||||
return null;
|
||||
}
|
||||
|
||||
// If only one match, use it — but penalize cross-language matches
|
||||
// If only one match, use it — but penalize a match in another language
|
||||
// family (a `.tsx` component calling a `.ts` helper is not one, #1662)
|
||||
if (candidates.length === 1) {
|
||||
if (!isCrossFileReachable(candidates[0]!, ref, context)) return null;
|
||||
const isCrossLanguage = candidates[0]!.language !== ref.language;
|
||||
const isCrossLanguage = !sameLanguageFamily(candidates[0]!.language, ref.language);
|
||||
return {
|
||||
original: ref,
|
||||
targetNodeId: candidates[0]!.id,
|
||||
@@ -13334,10 +13350,11 @@ export function matchByExactName(
|
||||
// above a proximity guess, but below a unique name or the file's own
|
||||
// definition (0.9). The candidates are those findBestMatch weighed: a
|
||||
// namesake in another language never wins while one in the reference's
|
||||
// own language exists. The caller's own copy being the caller itself is
|
||||
// no evidence, as for a same-file match (isDecisiveSameFileMatch).
|
||||
const weighed = candidates.some((c) => c.language === ref.language)
|
||||
? candidates.filter((c) => c.language === ref.language)
|
||||
// own language (sameRankingLanguage) exists. The caller's own copy being
|
||||
// the caller itself is no evidence, as for a same-file match
|
||||
// (isDecisiveSameFileMatch).
|
||||
const weighed = candidates.some((c) => sameRankingLanguage(c.language, ref.language))
|
||||
? candidates.filter((c) => sameRankingLanguage(c.language, ref.language))
|
||||
: candidates;
|
||||
const confidence = isDecisiveSameFileMatch(bestMatch, ref, candidates, context)
|
||||
? 0.9
|
||||
@@ -29385,7 +29402,8 @@ function findBestMatch(
|
||||
// Prioritization rules:
|
||||
// 1. Same file > different file
|
||||
// 2. Directory proximity (same module/package > different module)
|
||||
// 3. Same language > different language
|
||||
// 3. Same language > different language (C, C++ and Objective-C count as
|
||||
// one: sameRankingLanguage), and the very language as a tie-break
|
||||
// 4. Functions/methods > classes/types (for call references)
|
||||
// 5. Exported > non-exported
|
||||
|
||||
@@ -29406,10 +29424,11 @@ function findBestMatch(
|
||||
// scoring. Cuts the candidate set to same-language size on mixed front-end +
|
||||
// back-end repos (#915). When ALL candidates are cross-language (a legitimate
|
||||
// cross-language `calls` bridge), none are skipped and behavior is unchanged.
|
||||
const hasSameLanguage = candidates.some((c) => c.language === ref.language);
|
||||
const hasSameLanguage = candidates.some((c) => sameRankingLanguage(c.language, ref.language));
|
||||
|
||||
for (const candidate of candidates) {
|
||||
if (hasSameLanguage && candidate.language !== ref.language) continue;
|
||||
const sameLanguage = sameRankingLanguage(candidate.language, ref.language);
|
||||
if (hasSameLanguage && !sameLanguage) continue;
|
||||
|
||||
let score = 0;
|
||||
|
||||
@@ -29428,9 +29447,11 @@ function findBestMatch(
|
||||
score += 80;
|
||||
}
|
||||
|
||||
// Language matching: strongly prefer same language, penalize cross-language
|
||||
if (candidate.language === ref.language) {
|
||||
score += 50;
|
||||
// Language matching: strongly prefer same language, penalize cross-language.
|
||||
// Between C, C++ and Objective-C the very language adds one point, which
|
||||
// only breaks a tie: a directory nearer counts 15.
|
||||
if (sameLanguage) {
|
||||
score += candidate.language === ref.language ? 51 : 50;
|
||||
} else {
|
||||
score -= 80;
|
||||
}
|
||||
@@ -29662,7 +29683,7 @@ export function matchFuzzy(
|
||||
!(finalCandidates[0]!.kind !== 'function' && isBarePhpCall(ref, context)) &&
|
||||
isLexicallyReachable(finalCandidates[0]!, ref, context)
|
||||
) {
|
||||
const isCrossLanguage = finalCandidates[0]!.language !== ref.language;
|
||||
const isCrossLanguage = !sameLanguageFamily(finalCandidates[0]!.language, ref.language);
|
||||
return {
|
||||
original: ref,
|
||||
targetNodeId: finalCandidates[0]!.id,
|
||||
|
||||
Reference in New Issue
Block a user