When the PHP LSP knows a receiver's class but the class is not in the
index, it emits php_method_typed_unindexed at 0.55, below the LSP
confidence floor. The resolver skipped that row and the weak short-name
fallback bound the call to a same-named project method: `$file->getUid()`
on a vendor File became a CALLS edge to App\Item::getUid.
#2520 turns this row into php_external_receiver when a vendor `use`
proves the class external, but two cases still reach the fallback. A
global function import (`use function sprintf;`) leaves the file's import
bindings unproven, since #2705 proves only class imports absent. And a
vendor type that arrives through a return type
(`$repository->findFile()->getUid()`) has no `use` in the calling file
at all.
Both resolvers now drop a unique_name or suffix_match edge when the LSP
row names a receiver class that no project class-like symbol shares by
simple name. A project class with that name keeps the fallback, because
the PHP LSP does not follow every cross-file type.
The test pair covers both cases, through pass_calls.c and pass_parallel.c.
Each case is RED on main on its own (vendorCall, chainedVendorCall);
calls on a project-typed receiver and on $this still resolve. On a
211-file TYPO3 project PHP CALLS edges go from 1083 to 1059
(-18 suffix_match, -6 unique_name; typed php_* edges unchanged): 25
removed, and 1 that a removed edge on the same caller had absorbed.
Signed-off-by: Wolfgang Klinger <wolfgang@wazum.com>
py_lsp.c: #2697 moved py_lookup_attribute below the C3 code (own members, C3 order, then the depth-first walk) and declares it up front; the trivial definition this branch still carried next to its module-member lookup is dropped, the module-member block is kept, and the C3 section takes main's header.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
py_lsp.c: the module-member lookup of this branch and the C3 super() lookup of #2696 were added at the same place; both are kept.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A small incremental closure resolves each changed file through a per-file
selection of defs (its own and its imported modules') instead of the shared
registry. A package that re-exports a name keeps only the record there; the
definition lives in the module the record points at, possibly through more
re-exports, so a call through the package found nothing after an
incremental re-index of the caller alone. cbm_pxc_filter_defs_for_file now
follows every selected re-export or star record of a Python module to the
module it names (relative exactly, absolute under the longest matching
prefix, a member as a submodule) and selects that module's defs too, once
per record, so the per-file LSP folds the same chain the shared registry
does.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The re-export records of a package come from its namespace facts. An
incremental re-index that touches only the caller must still find them for
the unchanged packages (restored from their stored facts). Both drivers now
also run as a full index followed by an incremental re-index of app/main.py
alone, with the same expectations.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The first D1 change fed the Go cross-file registrar the extractor's
return_types. Those are cleaned names for type-reference edges:
clean_type_name strips `[]` and `*`. A cross-file single result `[]T`
therefore reached the registrar as `T`. `xs := pkg.Generate(); xs[0].M()`
lost its typing and fell back to a name guess. On prometheus the base
binds `sample[0].T()` to chunks.Sample.T, but the first D1 change bound
it to tsdb.T.
pass_lsp_cross.c now leaves a single result's raw text alone, as before
D1; the registrar already parses `*T` and `[]T`. Only a result LIST
`(...)` is split, at its top-level commas, into raw element types:
- `(*T, error)` -> `*T|error`;
- the named `(t *T, err error)` -> `*T|error`;
- the grouped `(a, b int)` -> `int|int`.
`chan`, `func`, `map`, `struct` and `interface` never count as result
names. A list that does not split cleanly keeps its raw text.
Tests: the D1 fixture gains three call sites, each with two same-named
decoys:
- a single slice result indexed (`xs[0].TS()`);
- a tuple with a slice element;
- grouped named results.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Python surface-envelope test (pipeline round 0) checks that the
persisted row is the versioned codec envelope by looking for the
literal "v":2. With SURFACE_CODEC_VERSION at 3, the envelope reads
"v":3. The assertion keeps its meaning (the row is the versioned
envelope and the def is present) and follows the version.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The cross-file pass handed every registrar the raw result text of a
def (pass_lsp_cross.c: `dst->return_types = src->return_type`). The Go
registrar splits that string on '|' (go_lsp.c split_pipe_types), so it
got `(*Encoder, error)` as one piece. That piece did not parse, and a
cross-file callee with more than one result, or with a named result
`(e *Encoder)`, had no usable return type. As a result,
`e, err := enc.NewEncoder()`, `h, _ := newHead()` and
`_, d := diag.Run()` all left the variable untyped, and calls on it fell
to name guesses. Same-file calls were fine, because they read the AST.
Go defs now pass the parsed result list (extract_go_multi_return),
'|'-joined: `*Encoder|error`, and `*Encoder` for the named result. Other
languages keep the single raw string.
The persisted LSP surface carries this string, so SURFACE_CODEC_VERSION
goes from 2 to 3: existing caches re-extract Go surfaces once. The
literal version-2 surfaces in the codec tests move to 3, so that they
keep testing the envelope rule they name rather than the version
mismatch.
Tests: pipeline_go_multi_return_receiver_is_typed (+_parallel). A tuple
result, a named result, a tuple-bound local reached through a field,
and a named slice type each must bind the right method with an lsp_*
strategy and none of its two same-named decoys.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The memory-core linter forbids a new raw allocator site. Growing the def
array for a Python file's namespace records needed a realloc because a
spilled result's facts were taken as not peekable; its header carries the
fact stream's status and size, though. cbm_pxc_collect_all_defs now adds
each Python module's record bound to the first count (one header peek per
spilled result, as the def count already took), writes the records into
that reserved space, and needs no growth: the realloc and its bookkeeping
are gone. An import-only package (zero defs) still gets its records.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The memory-core linter counts raw allocator calls per file and fails when a
file grows. The TYPE_CHECKING change freed the copied local name of an
annotation-only import with a new raw free in two import-map builders.
build_import_map now asks the predicate on the name's span in the edge
properties before copying it (cbm_imports_local_span_binds_runtime_name,
which cbm_imports_local_binds_runtime_name now wraps), and the cross-LSP
builder routes an annotation-only name through its existing no-value path.
Same behaviour, no new raw allocator site.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A namespace fact stream is checked by cbm_py_namespace_facts_valid, which
accepts a shadow only for the value, decorated, unproven-class and
unsupported reasons. The withdrawn TYPE_CHECKING re-export carried the
compound reason, so the whole stream was rejected and the package published
no names at all (a lazy-module package lost its TYPE_CHECKING exports). It
now carries the unsupported reason: the builder cannot tell which source the
name has at runtime. python_namespace_facts_type_checking_alternatives pins
both shapes as complete, valid streams.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The re-export test checks a table of calls in one loop; a failure only
reported the assertion line. Each failing call is now named, with its
venue.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A package namespace read in the type-checker view takes the consequence of
`if TYPE_CHECKING:` as executed (lazy-module packages publish their names
there). The compat shape imports a name there and again in `else: try:
from runtime.place import name`: the module's runtime export comes from the
else branch, so publishing the TYPE_CHECKING source as the re-export target
was a guess. Every name an elif/else import binds now becomes a shadow fact
(the re-export is withdrawn, as for a later assignment); a star import or an
import form the builder does not read there makes the whole namespace
unknown. The branch is walked iteratively.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_run_php_lsp never finalized its type registry, so every lookup during
single-file resolution scanned the stdlib plus every definition of the
file linearly: O(lookups x definitions), quadratic in the size of one
file. The cross-file path already indexes its registry.
The registry is now finalized with first-registered semantics (the first
registration of a duplicate QN wins), which keeps exactly the answers of
the linear scan, and re-indexed once after trait flattening adds methods.
The index lives in a scratch arena that dies with the call.
A test-only work ceiling (CBM_LSP_WORK_CEILING_HIT, seam builds only) lets
the work-counter test stop a reverted, unbounded loop deterministically
instead of hanging the runner; product builds have no ceiling.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_pxc_collect_all_defs sizes the def array by every file's def count and
appends a Python file's namespace records behind its defs. The growth check
only made room for the records themselves, so the records consumed the slack
reserved for later files and their defs overflowed the array (ASan:
heap-buffer-overflow in pipeline_python_package_reexports_resolve_by_lsp).
The array now grows to keep every later file's defs reserved on top of the
records placed so far, geometrically, so the copying stays linear.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A package `__init__.py` re-exports its submodules' names (`from .fields
import *`, `from .x import Orig as Renamed`), and callers reach them as
module attributes (`models.CharField()`) or through `from pkg import Name`.
The Python namespace facts already capture each module's top-level bindings
in order; nothing consumed them, so those calls fell to name guesses that
pick the first same-named definition under the package.
- Records: cbm_pxc_collect_all_defs replays each module's COMPLETE fact
stream into re-export, star-import and `__all__` records next to its
defs, including for an import-only `__init__.py` (zero defs). Records
carry the raw module spec; the Python registry resolves it (relative
exactly, absolute by unique scope tail, else unknown).
- Fold: the shared Python registry folds the records once, after finalize
and before the seal, into a sorted per-module export table (binary
search; no lookup in the registration loop). Order-deterministic; a
cycle contributes what it has so far.
- States (conservative): an incomplete fact stream publishes nothing; a
shadow, delete or whole-namespace unknown (a top-level try/if/with)
withdraws the names it covers; a star whose source cannot be enumerated
publishes nothing; a literal `__all__` restricts a star, else public
names only.
- Consult: the Python LSP looks a module attribute up in the export table
on a miss (calls, expression types, callable values) and when binding
`from pkg import Y` where Y is no def of pkg.
- Namespace facts read a module-level `if TYPE_CHECKING:` consequence as
executed: lazy-module packages publish their names there.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A lazy-module package publishes its names only under `if TYPE_CHECKING:`
(the else branch installs the lazy module): `lz.Lazy()` resolves to the
re-exported class. The compat shape imports a name under TYPE_CHECKING and
again in `else: try:`: the runtime binding comes from the else branch, so
`cmp.Shim()` must not reach the TYPE_CHECKING-side class.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A package __init__.py re-exports its submodules' names (explicit, star,
`__all__`-restricted, renamed, chained through a subpackage, cyclic stars,
the django `from .fields import *` shape) and callers reach them as module
attributes (`pkg.Explicit()`) or through `from pkg import Name`. Each call
must resolve by an lsp_* strategy to the re-exported definition. Names a
star does not export (`__all__`-unlisted, private), a re-export the module
shadows later, and one before a whole-namespace barrier must not. Plain
from-imports of the defining module are the control. Both venues.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The compat shape imports a name under `if TYPE_CHECKING:` and again in the
else branch, usually inside try/except ImportError. The name is bound at
runtime there, so its TYPE_CHECKING import is no annotation-only import:
recording it flagged made every CALLS import map skip the name and lost the
calls and bases that bind through the runtime import (airflow mask_secret,
OpenLineageRedactor(SecretsMasker)). The imports of the elif/else branches
are collected (any depth, iteratively), every TYPE_CHECKING import whose
local name they bind is dropped with them, and a star import there drops the
whole block: those names keep the behaviour they had before #1277.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The airflow compat shape imports a name under `if TYPE_CHECKING:` and again
in the else branch, usually inside try/except ImportError: the name is bound
at runtime, so its TYPE_CHECKING import is not annotation-only and must stay
unrecorded, as the runtime import itself does. A star import in an elif or
else branch may bind any name. A pure annotation import stays flagged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The import an annotated field's type comes from usually sits under a
module-level `if TYPE_CHECKING:` (to avoid an import cycle).
parse_python_imports walked only the module's direct statements, so that
import was never recorded and the reporter's chain
trainer.strategies.process_batch() lost its type exactly where real code
writes it.
- Extraction records the imports of a module-level `if TYPE_CHECKING:` /
`if <module>.TYPE_CHECKING:` consequence block, flagged
CBMImport.type_checking_only. elif/else branches stay unrecorded, as
every other compound statement's imports do.
- One rule for every CALLS import-map builder:
cbm_import_binds_runtime_name() (false for a flagged import). The
sequential build_import_map applies it in its fast and slow paths; the
shared cbm_pxc_build_import_map (= _ex(..., false)) applies it in both
loops, so no CALLS import map ever binds an annotation-only name.
- The annotation consumers take cbm_pxc_build_import_map_ex(..., true):
the cross-file base/field-type resolution and the Python LSP dispatch.
The sequential dispatch always takes it; the parallel one builds that
map only for a file that records a flagged import and otherwise passes
its CALLS map, which is identical there by construction.
- The Python LSP binds those names into an annotation-only frame consulted
by annotation resolution after the ordinary scope chain: a runtime call
through such a name never resolves.
Tests: python_type_checking_imports_flagged_issue1277 (extraction),
python_type_checking_typed_field_issue1277(_parallel) (pipeline, both
venues; controls: no import_map and no LSP CALLS edge through the name).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The second half of #1277: the reporter's typed-field chain with both
annotation imports under `if TYPE_CHECKING:` (the usual shape when the
import would cycle), through `typing.TYPE_CHECKING` too. The call resolves
by the LSP as with runtime imports, and a runtime call through a
TYPE_CHECKING-only name gets no CALLS edge at all. The extraction test reads
the new CBMImport.type_checking_only flag, declared here and not yet set.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Ordinary attribute lookup walked the bases one at a time, each through its
whole ancestry, so on a diamond self.m() and e.m() reached the shared base
before a later base that overrides it. It now follows the class's C3 order,
the order super() uses, after the class's own members; a hierarchy without
a C3 order keeps the depth-first walk.
Each class is linearized once per file: the orders live in a per-file,
arena-allocated open-addressing cache keyed by class QN, which also replaces
the per-lookup cache of the super() resolver. The registry is complete
before the walk starts, so an entry never goes stale; keys and the class
entry of an order are arena copies. Linearizing a class costs one merge of
its bases' cached orders, so no lookup re-linearizes a deep hierarchy.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The radarr diamond seen from an ordinary attribute: self.write_state() in
RadarrCalendarEntity and e.write_state() on a RadarrCalendarEntity look the
method up along the class's C3 order, which reaches CalendarEntity before
the shared base Entity. Single inheritance as the control; sequential and
parallel.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Wolfgang Klinger