mirror of
https://github.com/colbymchenry/codegraph.git
synced 2026-11-15 04:21:31 +08:00
fix(sync): a tsconfig, jsconfig or go.mod change re-resolves the imports it governs
The resolver read tsconfig / jsconfig `paths` and `baseUrl` and go.mod's
module path once and kept the maps for its lifetime ("callers re-create the
resolver if config changes", which none did), and the watcher follows source
files only. So after an edit to one of them a long-running MCP server kept
resolving imports by the old mapping, and so did later processes: a sync
re-resolves only the changed files, and a config-only edit changes none.
The index now carries a snapshot of the config files resolution read: every
tsconfig / jsconfig the alias loaders opened, `extends` chains included,
every go.mod, and the root's candidates whether present or not. Every sync
reads them again and, when one differs, drops the resolver's alias and
module maps with the memos resolved through them, re-opens every import the
file governs (the ES-module family for a tsconfig / jsconfig, Go for a
go.mod) for the sweep, as a moved workspace map does, and marks synthesis
for a rebuild. An index an older engine stamped records a snapshot without
re-opening anything.
Review changes: each file's digest covers what resolution reads from it, a
go.mod's module path and a tsconfig / jsconfig's `extends`, `baseUrl` and
`paths`, not its bytes. A dependency bump in go.mod or a `strict` flag in
tsconfig had re-opened every import of the language for the same graph
(caddy: 48,945 refs, sync 3.8 s against 0.9 s; excalidraw: 88,860 refs,
9.2 s against 1.1 s). Plus a test for that, the ReferenceResolver class doc
comment back on its class, and the CHANGELOG entry beside the workspace-map
one.
Validation: 7 tests; on main's source the 5 behaviour tests fail, and the
new one fails on the byte digest. 11 fixture edits, each compared with a
fresh index of the same tree, in a later process and in the same process:
main diverged in 7 and 9, this in none. Real repos, sync against a fresh
index over nodes, edges, refs and files: excalidraw with `@excalidraw/math`
retargeted was 27 edges off on main and is identical here; caddy with its
module path renamed was 6,511 edges off and is identical here. Fresh indexes
are identical to main's. A one-file sync on excalidraw, median of 5
interleaved runs: main 3,834 ms, this 3,269 ms.
Original PR #2984 by @danusha2345.
Fixes #2977
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
committed by
Colby Mchenry
co-authored by
Claude Fable 5.1
parent
29ce0149cf
commit
80f6b50504
@@ -321,6 +321,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|||||||
- In JavaScript and TypeScript, a function a file loads with a dynamic import from Node or a package, like `const { createServer } = await import('node:http')`, `const { default: chalk } = await import('chalk')` or `import('node:fs/promises').then(({ readFile }) => …)`, no longer links to a project function that only shares its name. Before, vite's call to Node's `createServer` showed up among the callers of vite's own `createServer`. The import counts only where it is in scope, so the rest of the file links as before, and a dynamic import of your own project's module still leads to that module. Thanks @Ancienttwo for the report. (#2491)
|
- In JavaScript and TypeScript, a function a file loads with a dynamic import from Node or a package, like `const { createServer } = await import('node:http')`, `const { default: chalk } = await import('chalk')` or `import('node:fs/promises').then(({ readFile }) => …)`, no longer links to a project function that only shares its name. Before, vite's call to Node's `createServer` showed up among the callers of vite's own `createServer`. The import counts only where it is in scope, so the rest of the file links as before, and a dynamic import of your own project's module still leads to that module. Thanks @Ancienttwo for the report. (#2491)
|
||||||
- In JavaScript and TypeScript, code outside your tests no longer links to a Jest or Vitest manual mock under `__mocks__/`. Before, when an import of a workspace package couldn't be followed, a call like `formatLabel(name)` linked by name to the mock beside it instead of the package's own function, so the real function lost its callers, and the same happened to rendered components and event handlers. Production code is now matched as if the mocks weren't there, so it reaches the real function, and adding or deleting a mock leaves it alone. Tests, other mocks, and a file that imports a mock by its path, like a `jest.setup.ts`, still reach the mock. Thanks @Y-Osawa2022 for the report and @danusha2345. (#2455)
|
- In JavaScript and TypeScript, code outside your tests no longer links to a Jest or Vitest manual mock under `__mocks__/`. Before, when an import of a workspace package couldn't be followed, a call like `formatLabel(name)` linked by name to the mock beside it instead of the package's own function, so the real function lost its callers, and the same happened to rendered components and event handlers. Production code is now matched as if the mocks weren't there, so it reaches the real function, and adding or deleting a mock leaves it alone. Tests, other mocks, and a file that imports a mock by its path, like a `jest.setup.ts`, still reach the mock. Thanks @Y-Osawa2022 for the report and @danusha2345. (#2455)
|
||||||
- In JavaScript and TypeScript, an import of a package from your own repository now leads to the file its `package.json` names: the `exports` entry for the import, conditions and subpaths like `"./utils": "./src/utils.ts"` included, or else its `main`, `module`, `types` or `source`. A package another one depends on by path, like `"acme-shared": "file:../common"` or `link:`, now counts as part of your project too, for the files of the package that declares it, even with no root `workspaces`. Before, an import of a pnpm workspace member declared with `"main": "./src/index.ts"` and no `index` file at its root, or of a `file:` dependency, couldn't be followed, so its calls were matched by name alone and a same-named private function nearer the caller could take them. An entry that names build output, like `./dist/esm/index.mjs`, or an import of a path inside it, like `@jsonjoy.com/fs-node-utils/lib/types/misc`, leads to the matching source file, like `src/index.ts` or `src/types/misc.ts`, when your project has one, and to nothing otherwise. `codegraph sync` follows edits to these `package.json` fields. Re-index JavaScript and TypeScript monorepos after upgrading. Thanks @elderbernardi and @Y-Osawa2022 for the reports, and @maxmilian and @danusha2345. (#2502, #2456)
|
- In JavaScript and TypeScript, an import of a package from your own repository now leads to the file its `package.json` names: the `exports` entry for the import, conditions and subpaths like `"./utils": "./src/utils.ts"` included, or else its `main`, `module`, `types` or `source`. A package another one depends on by path, like `"acme-shared": "file:../common"` or `link:`, now counts as part of your project too, for the files of the package that declares it, even with no root `workspaces`. Before, an import of a pnpm workspace member declared with `"main": "./src/index.ts"` and no `index` file at its root, or of a `file:` dependency, couldn't be followed, so its calls were matched by name alone and a same-named private function nearer the caller could take them. An entry that names build output, like `./dist/esm/index.mjs`, or an import of a path inside it, like `@jsonjoy.com/fs-node-utils/lib/types/misc`, leads to the matching source file, like `src/index.ts` or `src/types/misc.ts`, when your project has one, and to nothing otherwise. `codegraph sync` follows edits to these `package.json` fields. Re-index JavaScript and TypeScript monorepos after upgrading. Thanks @elderbernardi and @Y-Osawa2022 for the reports, and @maxmilian and @danusha2345. (#2502, #2456)
|
||||||
|
- `codegraph sync` now follows an edit to the `paths` or `baseUrl` of a `tsconfig.json` or `jsconfig.json`, in the file itself or a config it `extends`, and to the module path in a `go.mod`: the imports they map are linked again by the new mapping, and a long-running session, like the background server behind your agent, picks the edit up at its next sync. Before, those imports kept the mapping read when the project was indexed until it was re-indexed. Thanks @inth3shadows for the report and @danusha2345. (#2977)
|
||||||
- In JavaScript and TypeScript, a call or `new` no longer lands on an interface or a type alias. When a file declares both a type and a value of the same name, like `interface Ok` and `function Ok`, or a class merged with an interface, the call reaches the value, and a global like `new Response()`, `new Worker()` or `new Event()` no longer links to a project type that only shares its name. Re-index JavaScript and TypeScript projects after upgrading.
|
- In JavaScript and TypeScript, a call or `new` no longer lands on an interface or a type alias. When a file declares both a type and a value of the same name, like `interface Ok` and `function Ok`, or a class merged with an interface, the call reaches the value, and a global like `new Response()`, `new Worker()` or `new Event()` no longer links to a project type that only shares its name. Re-index JavaScript and TypeScript projects after upgrading.
|
||||||
- In JavaScript and TypeScript, a value the file imports from your own project and passes along now links only to what it imports: a constant like `isCloudHosted`, a class like a `Star` model, a whole module (`import * as Slack from '../slack'`), or a stylesheet, image or GraphQL document (`import TCO21 from './themes/TCO21.scss'`) no longer links to a function elsewhere that only shares its name. Before, when the project had exactly one function of that name, such a value linked to it: outline's settings screens showed up among the callers of the server's `isCloudHosted()` check, and its stores among those of the `<Star>` icon, so `codegraph callers`, impact and `codegraph_explore` followed references that never happen. A component that wraps the function declared beside it, like `export default styled(Disclosure)`, and a page loaded lazily, like `lazy(() => import('../pages/Home'))`, still lead to that function. Re-index JavaScript and TypeScript projects after upgrading.
|
- In JavaScript and TypeScript, a value the file imports from your own project and passes along now links only to what it imports: a constant like `isCloudHosted`, a class like a `Star` model, a whole module (`import * as Slack from '../slack'`), or a stylesheet, image or GraphQL document (`import TCO21 from './themes/TCO21.scss'`) no longer links to a function elsewhere that only shares its name. Before, when the project had exactly one function of that name, such a value linked to it: outline's settings screens showed up among the callers of the server's `isCloudHosted()` check, and its stores among those of the `<Star>` icon, so `codegraph callers`, impact and `codegraph_explore` followed references that never happen. A component that wraps the function declared beside it, like `export default styled(Disclosure)`, and a page loaded lazily, like `lazy(() => import('../pages/Home'))`, still lead to that function. Re-index JavaScript and TypeScript projects after upgrading.
|
||||||
- In JavaScript, an import of a TypeScript module written without an extension, like `import AudioModal from './audio_modal'` in a `.jsx` file next to `audio_modal.tsx`, now links to that module, as it already did from a TypeScript file. Before, a `.js` or `.jsx` file's imports of TypeScript modules were never followed, so in a codebase moving to TypeScript they were matched by name instead: an import under another name linked nothing, and a name that repeats could link to the wrong file, so mastodon's `<IconButton>` and `<Status>` tags rendered other components of those names than the ones each file imports. A JavaScript module beside a TypeScript one of the same name still comes first. Re-index JavaScript projects after upgrading.
|
- In JavaScript, an import of a TypeScript module written without an extension, like `import AudioModal from './audio_modal'` in a `.jsx` file next to `audio_modal.tsx`, now links to that module, as it already did from a TypeScript file. Before, a `.js` or `.jsx` file's imports of TypeScript modules were never followed, so in a codebase moving to TypeScript they were matched by name instead: an import under another name linked nothing, and a name that repeats could link to the wrong file, so mastodon's `<IconButton>` and `<Status>` tags rendered other components of those names than the ones each file imports. A JavaScript module beside a TypeScript one of the same name still comes first. Re-index JavaScript projects after upgrading.
|
||||||
|
|||||||
@@ -0,0 +1,199 @@
|
|||||||
|
/**
|
||||||
|
* A sync follows a change to the files import resolution is configured by
|
||||||
|
* (#2977): `tsconfig.json` / `jsconfig.json` `paths` and `baseUrl`, their
|
||||||
|
* `extends` chain, and a `go.mod` module path. The resolver read them once and
|
||||||
|
* kept the maps for its lifetime, and the watcher follows source files only,
|
||||||
|
* so an edit kept resolving imports by the old mapping — in the same process,
|
||||||
|
* and in later ones too, since a sync re-resolves only the changed files.
|
||||||
|
*
|
||||||
|
* Each case indexes the first tree, edits it, syncs, and compares edges and
|
||||||
|
* unresolved references with a full index of the final tree in a folder of
|
||||||
|
* its own.
|
||||||
|
*/
|
||||||
|
import { describe, it, expect, afterEach, vi } from 'vitest';
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import * as os from 'os';
|
||||||
|
import * as path from 'path';
|
||||||
|
import CodeGraph from '../src/index';
|
||||||
|
import { DatabaseConnection, getDatabasePath } from '../src/db';
|
||||||
|
import { ExtractionOrchestrator } from '../src/extraction';
|
||||||
|
|
||||||
|
type Files = Record<string, string>;
|
||||||
|
|
||||||
|
let roots: string[] = [];
|
||||||
|
let graphs: CodeGraph[] = [];
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
for (const graph of graphs) graph.close();
|
||||||
|
graphs = [];
|
||||||
|
for (const root of roots) fs.rmSync(root, { recursive: true, force: true });
|
||||||
|
roots = [];
|
||||||
|
});
|
||||||
|
|
||||||
|
function tempRoot(): string {
|
||||||
|
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-module-config-'));
|
||||||
|
roots.push(root);
|
||||||
|
return root;
|
||||||
|
}
|
||||||
|
|
||||||
|
function write(root: string, files: Files): void {
|
||||||
|
for (const [file, text] of Object.entries(files)) {
|
||||||
|
fs.mkdirSync(path.dirname(path.join(root, file)), { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(root, file), text);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function index(root: string): Promise<CodeGraph> {
|
||||||
|
const graph = await CodeGraph.init(root, { index: true });
|
||||||
|
graphs.push(graph);
|
||||||
|
return graph;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Every edge by its ends' natural keys, and every reference left unresolved. */
|
||||||
|
function stateOf(root: string, graph: CodeGraph): string[] {
|
||||||
|
const keys = new Map(
|
||||||
|
graph.getFiles()
|
||||||
|
.flatMap((file) => graph.getNodesInFile(file.path))
|
||||||
|
.map((n) => [n.id, `${n.kind} ${n.filePath}:${n.qualifiedName}:${n.startLine}`] as const)
|
||||||
|
);
|
||||||
|
const edges = graph.getOutgoingEdgesFrom([...keys.keys()])
|
||||||
|
.map((e) => `${e.kind} ${keys.get(e.source)} -> ${keys.get(e.target) ?? e.target} @${e.line}:${e.column} ${JSON.stringify(e.metadata ?? {})}`);
|
||||||
|
const db = DatabaseConnection.open(getDatabasePath(root));
|
||||||
|
try {
|
||||||
|
const refs = (db.getDb()
|
||||||
|
.prepare('SELECT reference_name, reference_kind, file_path, line, col, status, name_tail FROM unresolved_refs')
|
||||||
|
.all() as Array<Record<string, unknown>>)
|
||||||
|
.map((r) => `ref ${r.file_path}:${r.line}:${r.col} ${r.reference_kind} ${r.reference_name} ${r.status} ${r.name_tail}`);
|
||||||
|
return [...edges, ...refs].sort();
|
||||||
|
} finally {
|
||||||
|
db.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The files the calls written in `caller` land in. */
|
||||||
|
function callTargets(graph: CodeGraph, caller: string): string[] {
|
||||||
|
const node = graph.getNodesByQualifiedName(caller)[0]!;
|
||||||
|
return graph.getOutgoingEdges(node.id)
|
||||||
|
.filter((e) => e.kind === 'calls')
|
||||||
|
.map((e) => `${graph.getNode(e.target)!.filePath}::${graph.getNode(e.target)!.qualifiedName}`)
|
||||||
|
.sort();
|
||||||
|
}
|
||||||
|
|
||||||
|
const tsconfig = (target: string): string =>
|
||||||
|
JSON.stringify({ compilerOptions: { baseUrl: '.', paths: { '@lib/*': [`${target}/*`] } } });
|
||||||
|
|
||||||
|
const TS_BEFORE: Files = {
|
||||||
|
'package.json': JSON.stringify({ name: 'app' }),
|
||||||
|
'tsconfig.json': tsconfig('lib1'),
|
||||||
|
'lib1/u.ts': 'export function foo() { return 1; }\n',
|
||||||
|
'lib2/u.ts': 'export function foo() { return 2; }\n',
|
||||||
|
'c.ts': "import { foo } from '@lib/u';\nexport function bar() { return foo(); }\n",
|
||||||
|
};
|
||||||
|
|
||||||
|
const GO_BEFORE: Files = {
|
||||||
|
'go.mod': 'module example.com/app\n\ngo 1.22\n',
|
||||||
|
'util/u.go': 'package util\n\nfunc Foo() int { return 1 }\n',
|
||||||
|
'main.go': 'package main\n\nimport "example.com/app/util"\n\nfunc run() int { return util.Foo() }\n',
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Index `before`, write `after` over it, sync; return the graph and what a full index of the final tree holds. */
|
||||||
|
async function syncEdit(before: Files, after: Files): Promise<{ root: string; graph: CodeGraph; full: string[] }> {
|
||||||
|
const root = tempRoot();
|
||||||
|
write(root, before);
|
||||||
|
const graph = await index(root);
|
||||||
|
write(root, after);
|
||||||
|
await graph.sync();
|
||||||
|
const fullRoot = tempRoot();
|
||||||
|
write(fullRoot, { ...before, ...after });
|
||||||
|
return { root, graph, full: stateOf(fullRoot, await index(fullRoot)) };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('a sync follows a module-config change (#2977)', () => {
|
||||||
|
it('tsconfig paths retargeted, the importer untouched', async () => {
|
||||||
|
const { root, graph, full } = await syncEdit(TS_BEFORE, { 'tsconfig.json': tsconfig('lib2') });
|
||||||
|
expect(callTargets(graph, 'bar')).toEqual(['lib2/u.ts::foo']);
|
||||||
|
expect(stateOf(root, graph)).toEqual(full);
|
||||||
|
expect(graph.getPendingReferenceCount()).toBe(0);
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
it('tsconfig paths retargeted and the importer touched, as the issue did', async () => {
|
||||||
|
const { root, graph, full } = await syncEdit(TS_BEFORE, {
|
||||||
|
'tsconfig.json': tsconfig('lib2'),
|
||||||
|
'c.ts': "import { foo } from '@lib/u';\nexport function bar() { return foo() + 0; }\n",
|
||||||
|
});
|
||||||
|
expect(callTargets(graph, 'bar')).toEqual(['lib2/u.ts::foo']);
|
||||||
|
expect(stateOf(root, graph)).toEqual(full);
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
it('a mapping declared in the extends chain', async () => {
|
||||||
|
const before: Files = {
|
||||||
|
...TS_BEFORE,
|
||||||
|
'tsconfig.json': JSON.stringify({ extends: './tsconfig.base.json', compilerOptions: { strict: true } }),
|
||||||
|
'tsconfig.base.json': tsconfig('lib1'),
|
||||||
|
};
|
||||||
|
const { root, graph, full } = await syncEdit(before, { 'tsconfig.base.json': tsconfig('lib2') });
|
||||||
|
expect(callTargets(graph, 'bar')).toEqual(['lib2/u.ts::foo']);
|
||||||
|
expect(stateOf(root, graph)).toEqual(full);
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
it('a go.mod module path renamed, with the imports written for it', async () => {
|
||||||
|
const { root, graph, full } = await syncEdit(GO_BEFORE, {
|
||||||
|
'go.mod': 'module example.com/other\n\ngo 1.22\n',
|
||||||
|
'main.go': 'package main\n\nimport "example.com/other/util"\n\nfunc run() int { return util.Foo() }\n',
|
||||||
|
});
|
||||||
|
expect(callTargets(graph, 'run')).toEqual(['util/u.go::Foo']);
|
||||||
|
expect(stateOf(root, graph)).toEqual(full);
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
it('a later process sees the change too', async () => {
|
||||||
|
const root = tempRoot();
|
||||||
|
write(root, TS_BEFORE);
|
||||||
|
const first = await index(root);
|
||||||
|
expect(callTargets(first, 'bar')).toEqual(['lib1/u.ts::foo']);
|
||||||
|
first.close();
|
||||||
|
graphs.splice(graphs.indexOf(first), 1);
|
||||||
|
|
||||||
|
write(root, { 'tsconfig.json': tsconfig('lib2') });
|
||||||
|
const second = await CodeGraph.open(root);
|
||||||
|
graphs.push(second);
|
||||||
|
await second.sync();
|
||||||
|
expect(callTargets(second, 'bar')).toEqual(['lib2/u.ts::foo']);
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
it('an unchanged config re-opens nothing', async () => {
|
||||||
|
const root = tempRoot();
|
||||||
|
write(root, TS_BEFORE);
|
||||||
|
const graph = await index(root);
|
||||||
|
const before = stateOf(root, graph);
|
||||||
|
// A body edit elsewhere: the import map is read again and found the same.
|
||||||
|
write(root, { 'lib2/u.ts': 'export function foo() { return 3; }\n' });
|
||||||
|
const result = await graph.sync();
|
||||||
|
expect(result.filesModified).toBe(1);
|
||||||
|
expect(stateOf(root, graph)).toEqual(before);
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
it('a config edit that moves no mapping re-opens nothing: a tsconfig flag, a go.mod requirement', async () => {
|
||||||
|
const tsRoot = tempRoot();
|
||||||
|
write(tsRoot, TS_BEFORE);
|
||||||
|
const ts = await index(tsRoot);
|
||||||
|
const tsBefore = stateOf(tsRoot, ts);
|
||||||
|
const goRoot = tempRoot();
|
||||||
|
write(goRoot, GO_BEFORE);
|
||||||
|
const go = await index(goRoot);
|
||||||
|
const goBefore = stateOf(goRoot, go);
|
||||||
|
// Every import a language has goes back for the sweep when its map moves;
|
||||||
|
// an edit resolution never reads must not send them there.
|
||||||
|
const reopen = vi.spyOn(ExtractionOrchestrator.prototype, 'resurrectResolutionEdgesForLanguage');
|
||||||
|
try {
|
||||||
|
write(tsRoot, { 'tsconfig.json': JSON.stringify({ compilerOptions: { baseUrl: '.', paths: { '@lib/*': ['lib1/*'] }, strict: true } }) });
|
||||||
|
await ts.sync();
|
||||||
|
write(goRoot, { 'go.mod': 'module example.com/app\n\ngo 1.22\n\nrequire golang.org/x/text v0.14.0\n' });
|
||||||
|
await go.sync();
|
||||||
|
expect(reopen).not.toHaveBeenCalled();
|
||||||
|
} finally {
|
||||||
|
reopen.mockRestore();
|
||||||
|
}
|
||||||
|
expect(stateOf(tsRoot, ts)).toEqual(tsBefore);
|
||||||
|
expect(stateOf(goRoot, go)).toEqual(goBefore);
|
||||||
|
}, 120_000);
|
||||||
|
});
|
||||||
@@ -69,6 +69,7 @@ import { CodeGraphPackageVersion } from './mcp/version';
|
|||||||
import { extractSegmentSearchWords, segmentLookupVariants, splitIdentifierSegments } from './search/identifier-segments';
|
import { extractSegmentSearchWords, segmentLookupVariants, splitIdentifierSegments } from './search/identifier-segments';
|
||||||
import { createYielder } from './resolution/cooperative-yield';
|
import { createYielder } from './resolution/cooperative-yield';
|
||||||
import { ESM_IMPORT_LANGUAGES } from './resolution/import-resolver';
|
import { ESM_IMPORT_LANGUAGES } from './resolution/import-resolver';
|
||||||
|
import { moduleConfigLanguages, parseModuleConfigSnapshot } from './resolution';
|
||||||
import { minRefsForPool } from './resolution/resolver-pool';
|
import { minRefsForPool } from './resolution/resolver-pool';
|
||||||
|
|
||||||
// Re-export types for consumers
|
// Re-export types for consumers
|
||||||
@@ -153,6 +154,8 @@ export interface IndexOptions {
|
|||||||
const PYTHON_ROOTS_METADATA_KEY = 'python_root_fingerprint';
|
const PYTHON_ROOTS_METADATA_KEY = 'python_root_fingerprint';
|
||||||
/** Metadata key: the digest of the workspace packages and path dependencies the index was resolved by. */
|
/** Metadata key: the digest of the workspace packages and path dependencies the index was resolved by. */
|
||||||
const WORKSPACE_METADATA_KEY = 'js_workspace_fingerprint';
|
const WORKSPACE_METADATA_KEY = 'js_workspace_fingerprint';
|
||||||
|
/** The tsconfig / jsconfig and go.mod files the index was resolved by, with a digest of each (#2977). */
|
||||||
|
const MODULE_CONFIG_METADATA_KEY = 'module_config_snapshot';
|
||||||
/** Metadata key: the frameworks (sorted JSON names) the last full index extracted every file with. */
|
/** Metadata key: the frameworks (sorted JSON names) the last full index extracted every file with. */
|
||||||
const FRAMEWORKS_AT_INDEX_KEY = 'frameworks_at_index';
|
const FRAMEWORKS_AT_INDEX_KEY = 'frameworks_at_index';
|
||||||
/**
|
/**
|
||||||
@@ -832,6 +835,7 @@ export class CodeGraph {
|
|||||||
// extraction stamp (the bulk would still be stale). See extraction-version.ts.
|
// extraction stamp (the bulk would still be stale). See extraction-version.ts.
|
||||||
if (result.success) this.stampPythonRootFingerprint();
|
if (result.success) this.stampPythonRootFingerprint();
|
||||||
if (result.success) this.stampWorkspaceFingerprint();
|
if (result.success) this.stampWorkspaceFingerprint();
|
||||||
|
if (result.success) this.stampModuleConfigSnapshot();
|
||||||
if (result.success) this.stampIndexedFrameworks();
|
if (result.success) this.stampIndexedFrameworks();
|
||||||
|
|
||||||
if (result.success && result.filesIndexed > 0) {
|
if (result.success && result.filesIndexed > 0) {
|
||||||
@@ -950,6 +954,23 @@ export class CodeGraph {
|
|||||||
} catch { /* metadata is advisory */ }
|
} catch { /* metadata is advisory */ }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Record the module-config files the index was resolved by, for a later sync to compare against (see sync). */
|
||||||
|
private stampModuleConfigSnapshot(): void {
|
||||||
|
try {
|
||||||
|
const stored = this.queries.getMetadata(MODULE_CONFIG_METADATA_KEY);
|
||||||
|
this.queries.setMetadata(MODULE_CONFIG_METADATA_KEY, JSON.stringify(this.resolver.getModuleConfigSnapshot(stored)));
|
||||||
|
} catch { /* metadata is advisory */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The same, only when the snapshot differs from the stored one: a config changed, or resolution read one the index had not recorded. */
|
||||||
|
private stampModuleConfigSnapshotIfMoved(): void {
|
||||||
|
try {
|
||||||
|
const stored = this.queries.getMetadata(MODULE_CONFIG_METADATA_KEY);
|
||||||
|
const now = JSON.stringify(this.resolver.getModuleConfigSnapshot(stored));
|
||||||
|
if (now !== stored) this.queries.setMetadata(MODULE_CONFIG_METADATA_KEY, now);
|
||||||
|
} catch { /* metadata is advisory */ }
|
||||||
|
}
|
||||||
|
|
||||||
/** Record the workspace-package digest a later sync compares against (see sync). */
|
/** Record the workspace-package digest a later sync compares against (see sync). */
|
||||||
private stampWorkspaceFingerprint(fingerprint = this.resolver.getWorkspaceFingerprint()): void {
|
private stampWorkspaceFingerprint(fingerprint = this.resolver.getWorkspaceFingerprint()): void {
|
||||||
try {
|
try {
|
||||||
@@ -1197,6 +1218,35 @@ export class CodeGraph {
|
|||||||
// read. The watcher's scoped syncs follow source files only and skip
|
// read. The watcher's scoped syncs follow source files only and skip
|
||||||
// the read: a monorepo's manifests take a walk of the tree to find.
|
// the read: a monorepo's manifests take a walk of the tree to find.
|
||||||
// An index stamped by an older engine has no digest: record one.
|
// An index stamped by an older engine has no digest: record one.
|
||||||
|
// tsconfig / jsconfig `paths` and `baseUrl`, and go.mod's module path
|
||||||
|
// (#2977). The resolver reads them once and keeps the maps for its
|
||||||
|
// lifetime, and the watcher follows source files only, so an edit to
|
||||||
|
// one kept resolving imports by the old mapping: in this process, and
|
||||||
|
// in later ones too, since a sync re-resolves only the changed files.
|
||||||
|
// The index carries a digest of every config file resolution read
|
||||||
|
// (`extends` chains included) plus the root's candidates; when one
|
||||||
|
// differs, the maps are dropped and every import the file governs is
|
||||||
|
// re-opened for the sweep below, as a moved workspace map re-opens
|
||||||
|
// the ES-module imports. A handful of small files, read on every sync.
|
||||||
|
// An index stamped by an older engine has no snapshot: record one.
|
||||||
|
{
|
||||||
|
const tConfig = Date.now();
|
||||||
|
const stored = this.queries.getMetadata(MODULE_CONFIG_METADATA_KEY);
|
||||||
|
const snapshot = this.resolver.getModuleConfigSnapshot(stored);
|
||||||
|
const was = parseModuleConfigSnapshot(stored);
|
||||||
|
const changed = stored === null ? [] : Object.keys(snapshot).filter((rel) => (was[rel] ?? 'absent') !== snapshot[rel]);
|
||||||
|
if (changed.length > 0) {
|
||||||
|
this.resolver.resetModuleConfig();
|
||||||
|
let reopened = 0;
|
||||||
|
for (const language of moduleConfigLanguages(changed)) reopened += this.orchestrator.resurrectResolutionEdgesForLanguage(language);
|
||||||
|
refreshSynthesis = true;
|
||||||
|
this.queries.setMetadata('synthesis_pending', '1');
|
||||||
|
if (process.env.CODEGRAPH_SYNTH_TIMINGS) {
|
||||||
|
console.error(`[phase-timing] sync-module-config: ${Date.now() - tConfig}ms (${changed.join(', ')} changed, ${reopened} refs re-opened)`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if (fullReconcile) {
|
if (fullReconcile) {
|
||||||
const tWorkspace = Date.now();
|
const tWorkspace = Date.now();
|
||||||
const before = this.queries.getMetadata(WORKSPACE_METADATA_KEY);
|
const before = this.queries.getMetadata(WORKSPACE_METADATA_KEY);
|
||||||
@@ -1657,6 +1707,9 @@ export class CodeGraph {
|
|||||||
this.orchestrator.finishGitIndexState(gitState, fullReconcile, result.failedFilePaths);
|
this.orchestrator.finishGitIndexState(gitState, fullReconcile, result.failedFilePaths);
|
||||||
|
|
||||||
if (fullReconcile && result.filesChecked > 0) this.pendingFullReconcile = false;
|
if (fullReconcile && result.filesChecked > 0) this.pendingFullReconcile = false;
|
||||||
|
// The config files this sync's resolution read join the snapshot, and
|
||||||
|
// a changed one is recorded as it is now (#2977).
|
||||||
|
this.stampModuleConfigSnapshotIfMoved();
|
||||||
// The whole sync, as for indexAll: resolution and linking included (#2334).
|
// The whole sync, as for indexAll: resolution and linking included (#2334).
|
||||||
result.durationMs = Date.now() - startedAt;
|
result.durationMs = Date.now() - startedAt;
|
||||||
return result;
|
return result;
|
||||||
|
|||||||
+93
-3
@@ -50,7 +50,8 @@ import { clearDeclaredDependencies } from './frameworks/package-deps';
|
|||||||
import { synthesizeCallbackEdges } from './callback-synthesizer';
|
import { synthesizeCallbackEdges } from './callback-synthesizer';
|
||||||
import { createYielder, type MaybeYield } from './cooperative-yield';
|
import { createYielder, type MaybeYield } from './cooperative-yield';
|
||||||
import { MAX_SOURCE_FILE_SIZE_BYTES } from '../file-limits';
|
import { MAX_SOURCE_FILE_SIZE_BYTES } from '../file-limits';
|
||||||
import { loadProjectAliases, type AliasMap } from './path-aliases';
|
import { createHash } from 'crypto';
|
||||||
|
import { aliasConfigFields, loadProjectAliases, type AliasMap } from './path-aliases';
|
||||||
import { findGoModuleForImport, goModulePackageDir, loadGoModule, type GoModule } from './go-module';
|
import { findGoModuleForImport, goModulePackageDir, loadGoModule, type GoModule } from './go-module';
|
||||||
import { loadWorkspacePackages, workspaceFingerprint, workspacePackageDirOf, type WorkspacePackages } from './workspace-packages';
|
import { loadWorkspacePackages, workspaceFingerprint, workspacePackageDirOf, type WorkspacePackages } from './workspace-packages';
|
||||||
import { logDebug } from '../errors';
|
import { logDebug } from '../errors';
|
||||||
@@ -246,6 +247,60 @@ const CPP_BUILT_INS = new Set([
|
|||||||
'move', 'forward', 'swap',
|
'move', 'forward', 'swap',
|
||||||
]);
|
]);
|
||||||
|
|
||||||
|
/** The module-config files at the project root a resolver reads, present or not (#2977). */
|
||||||
|
const MODULE_CONFIG_ROOT_FILES = ['tsconfig.json', 'jsconfig.json', 'tsconfig.base.json', 'go.mod'] as const;
|
||||||
|
|
||||||
|
/** Project-relative config path → digest of what resolution reads from it (see `moduleConfigDigest`), 'absent' for one not there. */
|
||||||
|
export type ModuleConfigSnapshot = Record<string, string>;
|
||||||
|
|
||||||
|
/** A stored snapshot's files (see `getModuleConfigSnapshot`); an index without one, or one an older engine wrote, has none. */
|
||||||
|
export function parseModuleConfigSnapshot(stored: string | null): ModuleConfigSnapshot {
|
||||||
|
if (!stored) return {};
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(stored) as unknown;
|
||||||
|
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return {};
|
||||||
|
return Object.fromEntries(Object.entries(parsed as Record<string, unknown>).filter(([, v]) => typeof v === 'string')) as ModuleConfigSnapshot;
|
||||||
|
} catch {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Digest each of `files` (absolute) as the project sees it now, keyed by its project-relative path. */
|
||||||
|
function moduleConfigSnapshot(projectRoot: string, files: Iterable<string>): ModuleConfigSnapshot {
|
||||||
|
const snapshot: ModuleConfigSnapshot = {};
|
||||||
|
for (const abs of files) {
|
||||||
|
const rel = path.relative(projectRoot, abs).replace(/\\/g, '/');
|
||||||
|
// A config outside the project (an `extends` into node_modules or a parent
|
||||||
|
// folder) is read where it is; the key stays relative so the index moves.
|
||||||
|
snapshot[rel] = moduleConfigDigest(abs);
|
||||||
|
}
|
||||||
|
return snapshot;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A digest of what resolution reads from one config file, 'absent' for one not
|
||||||
|
* there: a go.mod's module path, a tsconfig / jsconfig's `extends`, `baseUrl`
|
||||||
|
* and `paths`. Not of its bytes: a dependency bump or a `strict` flag moves no
|
||||||
|
* import, and would re-open every one.
|
||||||
|
*/
|
||||||
|
function moduleConfigDigest(abs: string): string {
|
||||||
|
if (!fs.existsSync(abs)) return 'absent';
|
||||||
|
const read = path.basename(abs) === 'go.mod'
|
||||||
|
? loadGoModule(path.dirname(abs))?.modulePath ?? ''
|
||||||
|
: aliasConfigFields(abs) ?? '';
|
||||||
|
return createHash('sha256').update(read).digest('hex').slice(0, 16);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The languages whose imports a changed config file governs: the ES-module family for a tsconfig / jsconfig, Go for a go.mod. */
|
||||||
|
export function moduleConfigLanguages(changed: readonly string[]): Set<Language> {
|
||||||
|
const languages = new Set<Language>();
|
||||||
|
for (const rel of changed) {
|
||||||
|
if (/(?:^|\/)go\.mod$/.test(rel)) languages.add('go');
|
||||||
|
else for (const language of ESM_IMPORT_LANGUAGES) languages.add(language);
|
||||||
|
}
|
||||||
|
return languages;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Reference Resolver
|
* Reference Resolver
|
||||||
*
|
*
|
||||||
@@ -358,6 +413,10 @@ export class ReferenceResolver {
|
|||||||
private goPackageDirs = new Map<string, string | null>();
|
private goPackageDirs = new Map<string, string | null>();
|
||||||
// Monorepo workspace member packages. Same lazy/immutable convention.
|
// Monorepo workspace member packages. Same lazy/immutable convention.
|
||||||
private workspacePackages: WorkspacePackages | null | undefined = undefined;
|
private workspacePackages: WorkspacePackages | null | undefined = undefined;
|
||||||
|
// The tsconfig / jsconfig files (their `extends` chains included) and go.mod
|
||||||
|
// files the maps above were read from, as absolute paths. A sync compares
|
||||||
|
// what they say now with what the index was resolved by (#2977).
|
||||||
|
private moduleConfigFiles = new Set<string>();
|
||||||
|
|
||||||
constructor(projectRoot: string, queries: QueryBuilder) {
|
constructor(projectRoot: string, queries: QueryBuilder) {
|
||||||
this.projectRoot = projectRoot;
|
this.projectRoot = projectRoot;
|
||||||
@@ -646,6 +705,7 @@ export class ReferenceResolver {
|
|||||||
if (found !== undefined) break;
|
if (found !== undefined) break;
|
||||||
walked.push(dir);
|
walked.push(dir);
|
||||||
found = loadGoModule(path.join(this.projectRoot, dir));
|
found = loadGoModule(path.join(this.projectRoot, dir));
|
||||||
|
if (found) this.moduleConfigFiles.add(path.join(this.projectRoot, dir, 'go.mod'));
|
||||||
if (found || dir === '') break;
|
if (found || dir === '') break;
|
||||||
dir = goDirOf(dir);
|
dir = goDirOf(dir);
|
||||||
}
|
}
|
||||||
@@ -1043,7 +1103,7 @@ export class ReferenceResolver {
|
|||||||
|
|
||||||
getProjectAliases: () => {
|
getProjectAliases: () => {
|
||||||
if (this.projectAliases === undefined) {
|
if (this.projectAliases === undefined) {
|
||||||
this.projectAliases = loadProjectAliases(this.projectRoot);
|
this.projectAliases = loadProjectAliases(this.projectRoot, this.moduleConfigFiles);
|
||||||
}
|
}
|
||||||
return this.projectAliases;
|
return this.projectAliases;
|
||||||
},
|
},
|
||||||
@@ -1061,7 +1121,7 @@ export class ReferenceResolver {
|
|||||||
walked.push(dir);
|
walked.push(dir);
|
||||||
const abs = path.join(this.projectRoot, dir);
|
const abs = path.join(this.projectRoot, dir);
|
||||||
if (['tsconfig.json', 'jsconfig.json'].some((name) => fs.existsSync(path.join(abs, name)))) {
|
if (['tsconfig.json', 'jsconfig.json'].some((name) => fs.existsSync(path.join(abs, name)))) {
|
||||||
const aliases = loadProjectAliases(abs);
|
const aliases = loadProjectAliases(abs, this.moduleConfigFiles);
|
||||||
if (aliases) {
|
if (aliases) {
|
||||||
found = aliases;
|
found = aliases;
|
||||||
break;
|
break;
|
||||||
@@ -3051,6 +3111,36 @@ export class ReferenceResolver {
|
|||||||
return fingerprint;
|
return fingerprint;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The module-config files the index was resolved by, with a digest of each
|
||||||
|
* (see `moduleConfigSnapshot`): every tsconfig / jsconfig this resolver read
|
||||||
|
* (`extends` chains included), every go.mod, and the root's candidates
|
||||||
|
* whether present or not, so one that appears is seen too. `previous` is the
|
||||||
|
* snapshot the index was stamped with; its files are read again, so a config
|
||||||
|
* this process never loaded still counts.
|
||||||
|
*/
|
||||||
|
getModuleConfigSnapshot(previous: string | null): ModuleConfigSnapshot {
|
||||||
|
const files = new Set<string>(this.moduleConfigFiles);
|
||||||
|
for (const name of MODULE_CONFIG_ROOT_FILES) files.add(path.join(this.projectRoot, name));
|
||||||
|
for (const rel of Object.keys(parseModuleConfigSnapshot(previous))) files.add(path.join(this.projectRoot, rel));
|
||||||
|
return moduleConfigSnapshot(this.projectRoot, files);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Forget the tsconfig / jsconfig alias maps and the go.mod modules this
|
||||||
|
* resolver read, and every memo resolved through them, so the next reference
|
||||||
|
* reads the files as they are now (#2977). The caller re-opens the imports
|
||||||
|
* the old maps resolved.
|
||||||
|
*/
|
||||||
|
resetModuleConfig(): void {
|
||||||
|
this.projectAliases = undefined;
|
||||||
|
this.dirAliases.clear();
|
||||||
|
this.goModuleByDir.clear();
|
||||||
|
this.goPackageDirs.clear();
|
||||||
|
this.moduleConfigFiles.clear();
|
||||||
|
this.clearCaches();
|
||||||
|
}
|
||||||
|
|
||||||
/** The fingerprint of the workspace map this resolver resolves by (see `workspaceFingerprint`). */
|
/** The fingerprint of the workspace map this resolver resolves by (see `workspaceFingerprint`). */
|
||||||
getWorkspaceFingerprint(): string {
|
getWorkspaceFingerprint(): string {
|
||||||
return workspaceFingerprint(this.context.getWorkspacePackages?.() ?? null);
|
return workspaceFingerprint(this.context.getWorkspacePackages?.() ?? null);
|
||||||
|
|||||||
@@ -187,13 +187,15 @@ function resolveExtendsTarget(spec: string, fromDir: string): string | null {
|
|||||||
function loadEffectiveOptions(
|
function loadEffectiveOptions(
|
||||||
filePath: string,
|
filePath: string,
|
||||||
stack: Set<string>,
|
stack: Set<string>,
|
||||||
depth: number
|
depth: number,
|
||||||
|
read?: Set<string>
|
||||||
): EffectiveOptions | null {
|
): EffectiveOptions | null {
|
||||||
const abs = path.resolve(filePath);
|
const abs = path.resolve(filePath);
|
||||||
if (stack.has(abs) || depth > MAX_EXTENDS_DEPTH) {
|
if (stack.has(abs) || depth > MAX_EXTENDS_DEPTH) {
|
||||||
logDebug('path-aliases: extends chain cycle or too deep', { filePath: abs, depth });
|
logDebug('path-aliases: extends chain cycle or too deep', { filePath: abs, depth });
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
read?.add(abs);
|
||||||
const raw = readTsconfigLike(abs);
|
const raw = readTsconfigLike(abs);
|
||||||
if (!raw) return null;
|
if (!raw) return null;
|
||||||
|
|
||||||
@@ -209,7 +211,7 @@ function loadEffectiveOptions(
|
|||||||
logDebug('path-aliases: unresolved extends', { from: abs, spec });
|
logDebug('path-aliases: unresolved extends', { from: abs, spec });
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
const inherited = loadEffectiveOptions(target, stack, depth + 1);
|
const inherited = loadEffectiveOptions(target, stack, depth + 1, read);
|
||||||
if (!inherited) continue;
|
if (!inherited) continue;
|
||||||
if (inherited.baseUrl !== undefined) effective.baseUrl = inherited.baseUrl;
|
if (inherited.baseUrl !== undefined) effective.baseUrl = inherited.baseUrl;
|
||||||
if (inherited.paths !== undefined) {
|
if (inherited.paths !== undefined) {
|
||||||
@@ -241,6 +243,19 @@ function readTsconfigLike(filePath: string): RawTsconfig | null {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What {@link loadProjectAliases} reads from one config file, its `extends`,
|
||||||
|
* `compilerOptions.baseUrl` and `compilerOptions.paths`, as one string, or
|
||||||
|
* `null` for a file it can't read. A sync compares it to tell an edit that
|
||||||
|
* moves the alias map from one that doesn't, like a `strict` flag (#2977).
|
||||||
|
*/
|
||||||
|
export function aliasConfigFields(filePath: string): string | null {
|
||||||
|
const raw = readTsconfigLike(path.resolve(filePath));
|
||||||
|
if (!raw) return null;
|
||||||
|
const co = raw.compilerOptions ?? {};
|
||||||
|
return JSON.stringify([raw.extends ?? null, co.baseUrl ?? null, co.paths ?? null]);
|
||||||
|
}
|
||||||
|
|
||||||
function splitWildcard(pattern: string): {
|
function splitWildcard(pattern: string): {
|
||||||
prefix: string;
|
prefix: string;
|
||||||
suffix: string;
|
suffix: string;
|
||||||
@@ -261,9 +276,11 @@ function splitWildcard(pattern: string): {
|
|||||||
* `baseUrl`.
|
* `baseUrl`.
|
||||||
*
|
*
|
||||||
* Cheap to call repeatedly — caching is the caller's job (the
|
* Cheap to call repeatedly — caching is the caller's job (the
|
||||||
* resolver does it via {@link aliasCache}).
|
* resolver does it via {@link aliasCache}). `read` collects the absolute
|
||||||
|
* path of every config file the load read, the `extends` chain included, so
|
||||||
|
* the caller can tell when one of them changes (#2977).
|
||||||
*/
|
*/
|
||||||
export function loadProjectAliases(projectRoot: string): AliasMap | null {
|
export function loadProjectAliases(projectRoot: string, read?: Set<string>): AliasMap | null {
|
||||||
// `tsconfig.base.json` comes last on purpose: when a root `tsconfig.json`
|
// `tsconfig.base.json` comes last on purpose: when a root `tsconfig.json`
|
||||||
// exists it stays authoritative and reaches the base through `extends`.
|
// exists it stays authoritative and reaches the base through `extends`.
|
||||||
// The fallback is for the Nx layouts where that never happens — a
|
// The fallback is for the Nx layouts where that never happens — a
|
||||||
@@ -275,7 +292,7 @@ export function loadProjectAliases(projectRoot: string): AliasMap | null {
|
|||||||
for (const name of candidates) {
|
for (const name of candidates) {
|
||||||
const p = path.join(projectRoot, name);
|
const p = path.join(projectRoot, name);
|
||||||
if (!fs.existsSync(p)) continue;
|
if (!fs.existsSync(p)) continue;
|
||||||
const opts = loadEffectiveOptions(p, new Set(), 0);
|
const opts = loadEffectiveOptions(p, new Set(), 0, read);
|
||||||
if (!opts) continue;
|
if (!opts) continue;
|
||||||
// Remember the first readable config so a `paths`-less project still
|
// Remember the first readable config so a `paths`-less project still
|
||||||
// logs the file it was judged on, but keep looking: a config that
|
// logs the file it was judged on, but keep looking: a config that
|
||||||
|
|||||||
Reference in New Issue
Block a user