Ansible's composition produced no structure: kubespray's 40 playbooks,
roles and task files, tied together by 217 import_playbook /
include_tasks / import_tasks / include_role / import_role / roles:
directives, were unconnected YAML files.
A playbook or task file -- a YAML file whose top level is a sequence;
manifests, values, CI and compose files are mappings -- now IMPORTS
{local_name: <directive>, via: ansible, line} what each directive names:
import_playbook a playbook from its directory; include_tasks /
import_tasks / include a task file from its directory, else from its
role's tasks/; roles: items and include_role / import_role a role's
Folder, roles/<name> at or above its directory (a role by "./" / "../"
path from the directory). The ansible.builtin. prefix is understood; a
templated target links nothing.
Measured on 17 infrastructure repositories: 211 of 211 composition pairs
of kubespray against an independent reading, nothing else in any
corpus; deterministic over three runs; the incremental re-index equals
a full index in 4 scenarios (an included file edited or deleted, an
include pointed elsewhere, a playbook edited).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The soak's query-leak leg failed on GitHub's windows-11-arm runner in the
v0.12.0 release run (38460145605) and in the 10-12 dry run (38204190338),
both times on one search_code call. On that runner image the
PowerShell-backed search_code answers in ~23 s per call (mean 23.3 s over
23 calls). One call crossed the harness's fixed 30 s read, counted as a
failed operation, and failed the leg, even though the harness's own latency
check (Check 4) allows 60 s.
It is the runner environment, not the product: the same leg on a local
Windows ARM64 VM gives search_code 1.10 s mean for v0.11.0 and 1.06 s for the
v0.12.0 candidate artifact, both PASSED. Linux runs it in ~50 ms, macOS in
~150-230 ms, Windows x64 in ~0.9 s.
One constant, SOAK_MAX_QUERY_LATENCY_S (60), now drives the latency check.
The per-call read waits 30 s past it, so the measured latency, not a
timeout, gives the verdict; a reply that never comes still ends the read
and fails the call.
Checked: bash -n; tests/test_soak_daemon_recovery_contract.sh and
tests/test_smoke_fixture_contract.sh pass; the patched harness on the
Windows ARM64 VM (query-leak, 2 min, candidate binary) PASSED.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The variables a container sets (K8s env, compose environment, Dockerfile ENV)
were container-scoped EnvVars of their own, joined to nothing: the readers of
a name point at the global EnvVar, which every program reading that name
shares, so a join through it was in the right program for 23 % of the pairs.
Each scoped EnvVar now CONFIGURES {strategy:env_read, dockerfile} the readers
of its name inside the files the container's image is built from:
- the image: a K8s container's literal image, built from exactly one
Dockerfile (Image -DEPENDS_ON {via:build}->); a compose service's own build,
else its image's; a Dockerfile's own ENV. A K8s image reference now names
its container (USAGE {ref:image, container}), so a two-container pod joins
each container to its own image.
- its files: the context sources COPY / ADD put into the final stage, the
stages it is built FROM and the stages it copies from, resolved in the
build context (the Dockerfile's directory or the repository root, the one
the sources exist under), less the context's .dockerignore, never a test
file. They are kept on the Dockerfile's Module (image_context, image_files,
image_ignore).
pass_docker now reads COPY / ADD from the instruction's own text, as Docker
does: the grammar lost a JSON form opening with `/` and, after a word with a
trailing blank, took the following instructions for more sources.
Incremental: every run recomputes every join edge. A closure repair reads the
inputs of the files it leaves alone from the store and replaces the stored
edges wholesale; a .dockerignore (now a semantic-manifest control) changing,
or a deletion of a file an image copies, rebuilds in full.
CBM_SEMANTIC_INDEX_VERSION 8 -> 9.
Measured on 17 infrastructure repositories: 80 pairs, 80 correct by hand;
the graph matches an independent reading of the Dockerfiles 80/80.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.12.0 release run's license gate blocked @img/sharp-wasm32@0.35.5
(Apache-2.0 AND LGPL-3.0-or-later AND MIT). sharp sat in graph-ui's tree
only because @huggingface/transformers lists it for its Node image path.
CBM Lens runs Transformers in the browser (a tokenizer and a causal LM on
WebGPU), never processes images, and the Transformers web build already
replaces sharp with an empty object. Yet sharp installed 14 LGPL packages
(10 @img/sharp-libvips-*, 3 @img/sharp-win32-*, @img/sharp-wasm32), and
the gate let them through two ways:
- a reviewed exception for exactly @img/sharp-wasm32@0.35.0 ("installed
but not shipped"), which the 0.35.5 bump in #2632 tripped;
- collect() skipped every os/cpu-restricted package as "build tooling",
which hid all ten LGPL libvips binaries.
A non-permissive license has no place in the tree, shipped or not:
- sharp now resolves to graph-ui/stubs/sharp, our own MIT stand-in that
throws if anything ever calls it (direct dependency file:./stubs/sharp
plus override "sharp": "$sharp"). The whole sharp family leaves the
lockfile: 441 -> 411 packages, zero LGPL/GPL left (audited).
- ui_browser_licenses.py loses the exception and keeps only the browser
build evidence check; license-gate-check-npm.py validates that evidence
unconditionally, checks platform-restricted packages like any other,
and has no exclusion path; gen-ui-licenses.py drops the exception
section.
- tests: the exception tests become guards: an unshipped LGPL package
blocks, a platform-restricted LGPL package blocks, a platform-restricted
MIT package passes. Against the old scripts both LGPL guards fail.
The shipped browser bundle is unchanged: rebuilt, it differs from main's
only by the build's own -dirty version marker. License gate stage 3 on
this tree: 211 production packages, all allow-listed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A file and the directory beside it with the same stem (genproto.sh next
to genproto/) got one qualified name: the file's Module QN equals the
directory's Folder QN, and the graph buffer keeps the structural node, so
the file's Module was dropped, and with it the edges that pointed at it.
On 17 infrastructure repositories this lost 27 Modules (shell scripts,
YAML and JSON manifests, a Makefile, Rust, Elixir, Python, Terraform).
One side now yields, decided once per run from the discovered files:
- the Folder is spelled proj.pkg.foo/ (no module QN ends in a slash);
- where the directory is a Java/Go package it IS that package's module
and Go imports resolve to it, so the Folder keeps its QN and the file's
Module is spelled proj.pkg.foo#sh instead.
Symbol QNs do not change.
The closure-repair planner declines when a deletion touches a collision
on either side, since the spelling depends on the file set. The kustomize
resolver binds a directory entry to its Folder even when a same-stem
manifest sits beside it. CBM_SEMANTIC_INDEX_VERSION 7 -> 8, so existing
indexes rebuild once.
Measured on main 56700bdb over the 17 corpora: colliding files without a
Module 27 -> 0 (19 marked Folders, 8 #ext Modules). k8s.io resolves 3
more kustomize entries, and deckhouse's `source shell_lib.sh` imports
land on the script instead of the directory.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reuse a stable arena-owned function QN order for ambiguity marking and
submodule prefix lookup, preserving exact lookup policy and appended tails.
Add prefix/index failure regressions and a pipeline operation-count case.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Python namespace checks added two more raw free(dep_targets) calls
on decline paths, growing pipeline_incremental.c past its memory-core
baseline. The first file that declines now stops both the changed and
the deleted loop; one exit after the loops skips the dependent query and
releases dep_targets and the row index for every reason, including the
existing missing-row and added-name declines.
closure_python_namespace_decline maps the namespace comparison to its
decline reason. Decline order and reasons are unchanged. The file drops
to 140 raw sites; the baseline is tightened to match.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Delphi code that calls 1C's OLE API through late-bound Variants uses
Cyrillic member names (v8_Base.<Cyrillic>.<Cyrillic>('1', Now),
NM.<Cyrillic>()). Every such name produced a tree-sitter ERROR node, so the
unit was reported as parse_partial and a procedure whose own name is
Cyrillic was dropped from the graph.
Root cause is upstream tree-sitter-pascal (Isopod, the pin we vendor):
its identifier token is ASCII-only, /[&]?[a-zA-Z_]+[0-9_a-zA-Z]*/, and
there is no other token a non-ASCII letter can lex as.
Fix: pascal becomes a self-maintained fork of the pin we already vendored
(Isopod/tree-sitter-pascal @ 042119eca2e1, v0.10.2 = upstream master),
following the arkts model. One patch
(tools/tree-sitter-pascal/patches/unicode-identifiers.patch) changes only
that token to /[&]?[\p{L}_][\p{L}\p{M}\p{Nd}_]*/. Keywords are still
extracted from this `word` token, so begin/end/procedure lex as before
(`begin` glued to Cyrillic letters is one identifier; `&begin` stays an
escaped identifier). Node types are unchanged. The \p{} classes compile
to numeric code-point ranges: the vendored parser.c has 0 non-ASCII bytes.
Regenerated with tree-sitter-cli 0.25.10 --abi 14 (ABI unchanged, no
external scanner; tree_sitter/parser.h is byte-identical to chialisp's).
Fidelity: the UNPATCHED grammar regenerated with 0.25.10 gives the same
symbol/token/field/production counts as upstream's shipped parser and
byte-identical parse trees on upstream's 88 corpus cases and its three
examples/*.pas (37,773 tree lines for codetoolmanager.pas); the patched
parser gives the same identical trees on all of those ASCII inputs and
passes all 88 corpus cases.
Proof (7 Pascal files: the report's snippet, a Cyrillic procedure with
keyword-boundary cases, an ASCII control, a broken unit, upstream's three
examples; fresh caches): parse_partial 3 -> 1 (only the genuinely broken
unit remains), nodes 581 -> 582 (the Cyrillic-named procedure), edges
1480 -> 1482 (+1 DEFINES, +1 SEMANTICALLY_RELATED); per-file function
counts for the ASCII files are unchanged.
Registered like arkts: MANIFEST.md fork row + note, audit FORKS map,
THIRD_PARTY.md, vendored checksums. LICENSE stays upstream's MIT,
byte-identical.
Tests: parse_coverage gains the report's snippet and a Cyrillic-named
procedure (not partial, definitions extracted), an ASCII control, and a
guard that broken Pascal is still flagged; the fork carries
test/corpus/unicode_identifiers.txt.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A Windows sanitizer job went red on one suite that aborted under
AddressSanitizer, and the job log showed an empty "every failure site"
section followed by the shadow-byte legend. The summary in
scripts/run-tests-parallel.sh searched the suite log for "FAIL" only. A
sanitizer report never contains that word, and its useful part (error
kind, access size, top frames) is at the start of the report, far above
the last 15 lines.
The summary now lives in scripts/suite-failure-report.sh, which the
harness sources. Between the unchanged failure-site and last-15-lines
sections it prints, for a failing suite, the test that was running and
the first sanitizer report. The framework flushes each test name before
the test body runs, so the last test line ahead of a report is the test
that was running; a report that comes after the last test finished is
labelled as such instead of blaming a test. The report is shown from its
header (AddressSanitizer, LeakSanitizer, ThreadSanitizer,
MemorySanitizer, UndefinedBehaviorSanitizer) to its own SUMMARY line, at
most 40 lines, without the shadow dump and the legend; a recoverable
"runtime error:" line is shown with its stack only. Further sanitizer
headers, "runtime error:" lines and SUMMARY lines follow (at most 10),
then signal and timeout lines (at most 5).
A suite that dies with no report and no completion summary gets the
running test named. A trap-mode UBSan build stops on an illegal
instruction and prints nothing, so that name is the only evidence its
log holds. A log with an assertion failure and no report prints exactly
what it printed before.
The same file keeps the logs a red run needs. An EXIT trap in the
harness copies the log of every suite that did not record rc=0, plus
results.txt, into test-logs/failed/. The trap covers the scheduler
giving up and the union guard as well as the normal end, and it never
changes the exit status.
scripts/run-test-wave.py reported pass=0 for a suite that died without
its summary line. In that case, and only then, it now counts the PASS
markers in the log. A suite with a summary line is never recounted, so
the totals of a green run are unchanged.
tests/test_failure_report_contract.sh drives the real functions with
fabricated suite logs and runs as Step 0i2 of scripts/test.sh. It fails
on the previous behaviour in every sanitizer, running-test and
result-line check and passes on it in the unchanged-output checks. The
awk code uses no interval expressions and no character classes; the test
passes with BSD awk and grep on macOS and with mawk and GNU grep on
Ubuntu. scripts/README.md names the new internal file.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Smoke's binary string audit blocked on every platform in the dry run on
70543ba8 (run 38204190338, all 12 smoke jobs): "Credential patterns
found in binary". #2068 rebuilt the graph-ui bundle. The two reviewed
Monaco 0.56.0 quick-input lines are still there but minified to different
bytes, so their SHA-256 no longer matched the pair recorded on 09-09.
Re-reviewed: the CI-built bundle has exactly two flagged lines (15,644 and
72,472 bytes). Every match is the input box's password-mode flag
(_password=!1, password=!!e.password, inputBox.password=this.password,
inputBox.password=!1), not a credential. All 12 smoke logs show the same
two lines. The new pair is computed as filter_reviewed_lines does
(printf '%s' line | sha256) on CI's binaries: GNU strings on the Linux
amd64 artifact and Apple strings on the darwin arm64 artifact give the same
two hashes, and both are exempted. The comment now says how to redo this
when the bundle changes again.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings in #2596 (MSan: hidden uninitialized reads) and #2068 (CodeAtlas
web).
README: main's CBM Atlas line and this branch's infrastructure-as-code
line are both kept.
tools/list metadata: main and this branch each stayed under the 20 KiB
discovery budget, but together they reached 20,809 bytes. The branch's
wording is shorter now (search_graph kind/namespace, one hint per trace
mode: 136 bytes less), and the reviewed budget is 21 KiB so the
topology/deploy/run modes keep their usage hints.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/memwaste.sh set no memory budget, so the daemon derived one from
the machine's RAM: 1792 MB on the 7 GB macos-14 runner, about 4 GB on the
16 GB Linux and Windows runners. The event layer itself holds 0.5-1 GB, and
on macOS the index stopped with over_memory_budget before it measured
anything, in both full-mode runs since the lane was added (dry runs
37212102899 and 38186491924, peaks 3449 and 3111 MB at the stop).
The script now passes CBM_MEM_BUDGET_MB=4096, what the larger runners
already derived, unless the caller sets one.
Checked on macOS arm64 with CBM_WORKERS=3, the macos-14 runner's core
count:
- budget 1792: over_memory_budget in parallel_extract (rss 3904 MB), the
CI failure;
- budget 4096: the index completes and the report is written;
- the edited script with no budget in the environment: 4096 is applied,
rc 0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Five suites (grammar_regression grammar_labels pipeline lang_contract
grammar_probe_e) were excluded from the MSan lane for "stack-overflow
under instrumentation". That was never deep recursion. MSan aborted while
REPORTING real uninitialized reads:
- every abort was on the main thread. MSan's "T<n>" is the OS thread id
and equalled the pid in all five x86-64 aborts (CI job 91868259558);
the old block read it as a worker-thread ordinal, so the worker-stack
knob, MSAN_ORIGINS and CBM_WORKERS=1 could never change anything;
- with the default slow unwinder (the image's MSan-instrumented
libunwind) a report exhausted the main thread's 256 MiB stack, and the
overflow handler then failed with "nested bug in the same thread";
- with fast_unwind_on_fatal=1 the same test prints the
use-of-uninitialized-value report instead (measured on one tree and
test: slow unwind aborts at the stack limit, fast unwind reports
wit/scanner.c:13).
The two reads are fixed in the previous commits of this series, so:
- MSAN_OPTIONS defaults to fast_unwind_on_fatal=1 (the build already
keeps frame pointers);
- MSAN_EXCLUDE is down to cli, whose install-path failure is a separate,
still-undiagnosed cause;
- the known-red block, the header (it still claimed CI runs without
exclusions) and the MSAN_ORIGINS and stack comments say what is true;
- the partial-lane warning and the exclude-name typo check lived in a
branch behind `elif true` that never ran. They now run in the
per-suite path, the check as a case match instead of a grep -q pipe
that pipefail can misreport;
- the comments in scripts/ci/msan-lane.sh and above the test-msan job
in _test.yml no longer say the five suites overflow.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
tree_sitter_wit_external_scanner_create() is declared to return void *
but had an empty body. tree-sitter's runtime stores its result as the
scanner payload (parser.c ts_parser__external_scanner_create), so every
WIT parse used an indeterminate value, which C11 6.9.1p12 makes
undefined. The scanner never reads its payload, so ordinary builds were
unaffected, and the vendored build's -w hid -Wreturn-type.
MemorySanitizer reports it at wit/scanner.c:13 from grammar_labels,
grammar_probe_e, grammar_regression and lang_contract. Those four suites
were excluded from the MSan lane because the report itself crashed and
posed as a stack overflow (fixed in the msan.sh commit of this series).
Recorded in MANIFEST.md's local-patch table. Upstream
bytecodealliance/tree-sitter-wit still has the empty body at
059bfdd1af72, the last change to src/scanner.c.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reconcile the complete Atlas and Lens stream with current main while retaining both histories. Preserve architecture functionality, current process and watcher safeguards, test infrastructure, and dependency security updates. Add coverage for Git-environment isolation combined with child-local overrides.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dockerfiles were Module + ENV/ARG Variables only: the 261 FROM and
114 COPY --from lines of the infra corpora joined nothing, and build
files named for a variant were not indexed at all.
- pass_docker (new, end of the K8s pass): FROM <image> and
COPY --from=<image> -> DEPENDS_ON {via: from | copy-from, image,
stage, line} from the Dockerfile's Module to an Image node. ARG
defaults are expanded (global ARGs in FROM, a stage's ARG / ENV in
COPY); a stage (by name or index), scratch and an ARG without a value
are no image.
- Image (new label, as approved): one node per repository, normalised
(docker.io for no registry host, library/ for an official image, a
registry host and port kept); tag and digest stay on the edge.
- Detection: Dockerfile.<x>, Dockerfile-<x>, Dockerfile_<x>,
Containerfile[.<x>] and <x>.Dockerfile are Dockerfiles, unless the
suffix is a language extension of its own (Dockerfile.j2 stays a
Jinja template) or a BuildKit ignore file.
- QNs: a Dockerfile.<variant> keeps its suffix (fqn.c and helpers.c,
which must agree); dropping it gave the variant the plain
Dockerfile's QN, merging both Modules. Covered by this branch's
index-format bump.
- Incremental: an Image no DEPENDS_ON / USAGE edge touches any more is
dropped on both routes, like a Process without SPAWNS; the
unreferenced-node GC takes a list of edge types.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
"Which CI jobs run hack/ci/run-e2e-tests.sh, and what does that script
start?" had no answer: a command running a script of the repository drew
no edge (SPAWNS reached Process nodes only, and repository scripts were
left out), and CI configurations were not read at all.
- A shell script, Make recipe or CI command that runs a script of the
repository -- by its path (./hack/x.sh, "$ROOT/hack/x.sh") or through a
shell (bash -e hack/x.sh) -- SPAWNS that script: its Module (the source
of the script's own edges, so CI -> script -> deploy chains connect),
else its File. The path must resolve to exactly one file; a program that
is no repository file keeps its Process edge.
- A new pass_ci.c reads GitHub Actions workflows and composite actions
(run: steps with their working directory, shell, env literals,
${{ github.workspace }} and ${{ github.action_path }}; no pwsh step, no
step of a Windows job) and Prow job configs (container command + args,
exec'd; bash -c TEXT as shell; a decorated job starts in the checkout).
The shell layer evaluates them like a script; their DEPLOYS and SPAWNS
edges come from the workflow / job file and carry "ci" and "job".
- env A="$X/y" PROGRAM: an assignment with an unknown value is no longer
taken for the program (it made a Process named after the value's last
segment and hid the program).
Measured on 17 repositories against an independent reading (libyaml for
the CI files, a separate shell reader for the commands): 292 of 292
script runs linked, 292 of 292 correct -- from Prow jobs (115 containers),
GitHub Actions steps, shell scripts and Make recipes, across 10
repositories (kubermatic, gardener, kube-hetzner, cluster-api, deckhouse,
k8s.io, ...); CI Process and deploy edges 4 of 4. Only SPAWNS (+295),
DEPLOYS (+1), Process (+3) and Chart (+1) changed; three runs build
identical graphs; index time unchanged (kubermatic 2.95 -> 2.93 s,
gardener 5.25 -> 5.25 s, kube-hetzner 1.13 -> 1.14 s, medians); editing or deleting scripts, workflows,
Prow jobs and Makefiles leaves the graph a fresh index builds (two
scenarios differ in SEMANTICALLY_RELATED edges and Make pattern-rule
nodes exactly as the previous build does).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A Terraform module installing a chart with `resource "helm_release"` had no
edge to the chart: "which modules install this chart" had no answer.
A new Terraform / OpenTofu pass (run at the end of the K8s pass, like the
shell pass) reads the helm_release blocks of .tf / .tofu files and links,
to existing nodes only, from the block's node (resource.helm_release.<name>):
- chart = "./charts/x" or "${path.module}/charts/x": the Chart of that
directory of the module;
- a repository and a chart name, or chart = "oci://.../<name>": the shared
external Chart of that name (the node chart dependencies use).
Values are read from string literals and ${path.module} only; a variable
chart, or a directory without a Chart.yaml, is no edge. Properties: via
terraform, the chart, external, line.
Measured on 17 repositories against an independent reading of the
Terraform files: 21 of 21 releases linked, 21 of 21 correct (EKS
blueprints 17, k8s.io 3, kube-hetzner 1). Only these DEPLOYS edges and 8
external Charts were added; three runs build identical graphs; index time
unchanged (cloud-foundation-fabric 1.91 -> 1.89 s, EKS blueprints 0.87 ->
0.88 s, k8s.io 3.44 -> 3.45 s, medians); editing or deleting the files
leaves the graph a fresh index builds.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A script or Make recipe running kubectl apply, helm upgrade or a kustomize
build piped into kubectl puts manifests, a kustomization or a chart on a
cluster, and none of it was an edge: "which script deploys this manifest"
had no answer in the graph.
A new pass (the Bash + Make layer, run at the end of the K8s pass) reads
scripts and Makefiles that name a deploy tool and links, to existing nodes
only:
- kubectl apply|create|replace -f: the File of a manifest, a directory's
manifests (-R: below it too), a glob's; -k: the Kustomization;
- kustomize build D (or kubectl kustomize D) | ... | kubectl apply -f -:
the Kustomization of D;
- helm install|upgrade [NAME] CHART, helm template piped into kubectl:
the Chart of a chart dir, else the shared external Chart of repo/name or
oci://.../name;
- find D -name G -exec kubectl apply -f {} \;: D's matching manifests.
The edge starts at the bash function or Make rule running the command,
top-level code at its file, with via (kubectl / helm / kustomize), the
deployed path or chart, and the line.
Words are evaluated by constant propagation, nothing is executed:
assignments (local / export / readonly / declare, arrays), ${X:-default},
$0 / BASH_SOURCE, $(dirname ..), $(cd .. && pwd), $(realpath ..),
$(git rev-parse --show-toplevel); cd / pushd / popd track the directory.
Anything else is unknown, and an unknown target is no edge: a value set in
a branch or loop is unknown after it, a subshell's changes do not leak, an
unquoted unknown word that may shift helm's positionals blocks the chart.
With no directory tracked, a relative path resolves against the root and
the script's directory, linked only when exactly one of them has a target.
Make: the Makefile's variables expand into each recipe line ($(shell pwd) /
$(CURDIR) = the Makefile's directory; a conditional's value is unknown),
which is then read as a shell line.
Measured on 17 repositories against an independent reading of the scripts
(its own tokenizer, variable and directory tracking, the file system): 86
of 86 deploys linked, 86 of 86 correct (k8s.io 66, kubermatic 12, gardener
8). Only DEPLOYS edges and 3 external Charts were added; three runs build
identical graphs; index time unchanged within 1 s (deckhouse 10 s, gardener
5-6 s, k8s.io 3-4 s, median of 3). Nine incremental edits (deploy lines
edited, removed and added, a Makefile, a deployed manifest, a new manifest
and a new non-manifest in an applied directory) leave the graph a fresh
index builds.
tree-sitter-bash folds `a | b` followed by a line `cat <<EOF | c` into
`a | (b cat | c)`; that shape continues the outer pipe.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The complexity guard (O(n^2) detection) had no infrastructure corpus, and
the infra resolvers looked candidates up by name across the whole
repository and then filtered them: an object reference by "Kind/name", a
selector by its rarest label pair, a custom resource by its CRD's
group/kind, a controller's kind by the Go type's name. Independent copies
share every name, so with k copies each lookup visited all k: the work
grew ~4x per doubling while the edges stayed ~2x.
Candidates are now looked up where they can match:
- K8s references, selectors and CRDs through indexes keyed by the bundle
(kustomize build root / chart / dir, and the deploy scopes for
references) and the name, type or label pair; a source visits only the
bundles it is in. The repository-wide fallback for a custom resource
without a CRD beside it stays (it stands for every copy by design);
- the operator pass through a (package dir, type) index: a controller's
kind is one lookup in the imported or own package; only an import no
module of the repository has still checks the same-named types. Group
declarations are deduplicated through a set, not a scan.
Child access in the K8s extractor goes through a tree cursor:
ts_node_named_child(n, i) walks from the first child on every call, so the
index loops over mapping pairs, sequence items and a file's documents were
quadratic in a wide node.
The guard gains fixtures (tests/fixtures/complexity/Kubernetes: a
kustomize bundle with workloads, Service, Ingress, PDB, NetworkPolicy,
HPA, RBAC, CRD + CR, cert-manager; tests/fixtures/complexity/Go: a
controller-runtime operator with +groupName types and an embedded CRD),
work counters g_k8s_candidates / g_op_candidates, and a gate on them and
on the fixtures' own node and edge growth; fixture dirs now also apply to
templated languages. An API group is one identity across a repository, so
the fixtures' CRDs use groups no copy's Go code declares -- the guard
measures accidental growth, not that by-design join. The originality
check covers the infra sources and scans them against yaml-language-server,
helm-ls, kustomize, Flux's kustomize package and controller-tools.
Before: k8s candidates 126 -> 504 (4.00x), operator 45 -> 288 (6.40x)
per doubling; after: 42 -> 84 and 6 -> 12 (2.00x), fixture edges 2.01x.
On 17 infrastructure repositories the graphs are identical to before;
three runs build identical graphs. Index wall time against origin/main,
median of three: deckhouse 9.58 -> 9.59 s, kubermatic 2.93 -> 2.93 s,
cloud-foundation-fabric 1.90 -> 1.90 s. A manifest of thousands of
documents indexes ~10 % faster (2.40 -> 2.13 s).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A bare import specifier was always external. The TS resolver now asks the
pipeline what it already knows for import resolution, without indexing
package.json or tsconfig.json: the nearest tsconfig/jsconfig `paths` and
`baseUrl` (the alias collection; the first target, as TypeScript tries it
first), then the workspace package whose `name` the specifier is (the
package map), bound to the package's entry module (`exports`, `main`,
`module`) when the repository indexes it. Every other bare specifier, a
package's subpath included, stays external. Incremental runs stay exact:
a tsconfig change re-resolves the files under it, a package.json change is
a manifest change.
path_alias gets cbm_path_alias_resolve_into, which writes into the caller's
buffer; cbm_path_alias_resolve now wraps it (two raw allocations fewer, the
memory-core baseline lowered to match).
Also fixed: a declaration in an index file has no `index` in its QN (fqn.c
strips it from a symbol's QN, not from the module's), so a reference to one
missed its node; the resolver now names a file's declarations with that
prefix.
Proof (ship build, Apple M5 Pro / 48 GB): the TypeScript compiler
b465fdbfe175 unchanged (69 edges, 57 of the prototype's 61 pairs);
discord.js 4dc1acc6 unchanged (no doc reference names a workspace import;
its packages' entries are build output). Tests: a tsconfig alias, a
workspace package's entry and an index file's declaration each bind, each
red with its rule reverted; path_alias, pipeline, edge_imports, incremental
and all doc suites pass (1298).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
81264814 built the production allocator as mimalloc's release build
(-DMI_DEBUG=0). That bought a large speed-up, but it raised what macOS
charges an index worker, its phys_footprint:
| peak phys_footprint | v0.11.0 | main |
|---|---:|---:|
| TypeScript | 1.25 GB | 1.94 GB |
| dotnet/runtime | 4.5 GB | 12.7 GB |
The 2026-10-09 bench missed most of this, because it sampled RSS.
The difference is one compile-time branch in mimalloc's unix decommit.
With MI_DEBUG or MI_SECURE>2, a purged range is mprotect(PROT_NONE)'d
and flagged needs_recommit. A release build leaves it "committed".
When the arena hands such a range out again, it is re-accounted as a
whole with MADV_FREE_REUSE (arena.c, _mi_os_reuse). macOS then charges
every page of it to the footprint, touched or not.
On macOS, take the recommit path in every build. Linux and Windows keep
upstream's behaviour; there MADV_DONTNEED already drops RSS at once.
Measured on Apple arm64 (macOS 26.6), main ec4fbd3a vs this change, cold
full indexes interleaved with main (C#: 2 runs each, in the same session
as main):
| corpus | footprint before | footprint after | change | wall time |
|---|---:|---:|---:|---|
| TypeScript | 1.98 GB | 1.09 GB | -45 % | unchanged |
| kubernetes | 2.92 GB | 1.88 GB | -35 % | unchanged |
| dotnet/runtime | 12.6 GB | 4.9 GB | -61 % | unchanged |
| linux kernel | 16.0 GB | 13.5 GB | -16 % | 574 -> 559 s |
The graph is byte-identical: TypeScript, kubernetes and django give
the same nodes, edges and CALLS-set hash.
dotnet/runtime is now close to v0.11.0's 4.5 GB.
Measured but not taken:
- the whole debug build, which costs ~5 % wall time on C#;
- MI_STAT=2, MI_PADDING=1 or MI_ENCODE_FREELIST=1 alone, none of which
moves the footprint;
- MIMALLOC_PURGE_DELAY=0, which is already set.
Vendored file changed in place (precedent 1f674c80); its hash in
scripts/vendored-checksums.txt is updated and security-vendored.sh passes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Helm file:// dependency records and the exact kustomize entry
resolver added raw strdup/free sites, so pass_k8s.c went from 6 to 9
raw allocator sites and pass_pkgmap.c from 82 to 85 -- the
memory-core ratchet (make lint-memory-core, part of lint-ci) fails on
both files.
pass_k8s.c: a dependency record now holds core-owned copies (class
other) of its chart path and target QN. The QN helper copies the
libc string fqn.c hands out into the core and releases it in one
place, and the record array grows before anything is allocated, so
the failure path frees nothing. 9 -> 6, the baseline.
pass_pkgmap.c: the resolver's three frees release fqn.c's
libc-allocated QNs and paths and have to stay raw. The manifest read
buffer, private to this file, now comes from the core (class extract:
per-file source text) and its three callers release it there. 85 ->
81; the baseline drops from 82 to 81 in this change.
No graph change: kubermatic and kubernetes/k8s.io indexes are
identical to the previous build node for node and edge for edge,
deckhouse in every edge and node identity. Its remaining difference,
the order of the variants list on QN-colliding Go methods, also
appears between two runs of the same binary and is not caused here.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pass_infrascan.c carried line-by-line parsers for Dockerfiles, .env
files, shell scripts and Terraform, five file classifiers and a JSON
bracket cleaner; pass_envscan.c walked a repository for URL-valued
environment assignments. No production code calls any of them: the
graph gets these files from the tree-sitter extractors, and the
envscan walk said so in its own header comment. Their only callers
were 40 tests in test_pipeline.c that asserted the parsers' output,
so the suite measured code the index never runs.
Deleted: cbm_is_dockerfile, cbm_is_compose_file,
cbm_is_cloudbuild_file, cbm_is_env_file, cbm_is_shell_script,
cbm_clean_json_brackets, cbm_parse_{dockerfile,dotenv,shell,terraform}_source,
cbm_scan_project_env_urls(_excluded), their result structs, and the
tests and test helpers that existed only for them.
Kept, all with production callers or a planned one:
cbm_is_kustomize_file and cbm_is_k8s_manifest (pass_k8s), cbm_infra_qn,
the Helm Chart.yaml parser, and the secret detectors
cbm_is_secret_binding / cbm_is_secret_value (still covered by
infra_secret_detection), which the secret-hygiene work reuses. The
secret detector's prefix offsets now have their own names instead of
borrowing the Dockerfile parser's keyword lengths.
No graph change: the deleted functions were unreachable from the
index. Suites green: pipeline 299 (339 minus the 40 deleted), platform,
grammar_probe_f/g, infrascan, discover, language, lang_contract,
incremental, integration, edge_structural, configlink.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Resolves the definition walks next to each other: main's decl_namespace (#2120) and this branch's extra_props are both compacted and both counted in the result census.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The PDF text layer carries data tables (D3). Their sources and notices are
now in THIRD_PARTY.md (shipped in every archive through
THIRD_PARTY_NOTICES.md) and in the SBOM:
- the Adobe Glyph List subset (BSD-3-Clause, Copyright 2002-2019 Adobe),
- the printable-ASCII advance widths of Helvetica and Times-Roman from
Adobe's core-14 font metrics (the AFM files are not distributed),
- the Unicode Character Database data of pdf_unicode.c (Unicode
License v3, full text included).
scripts/gen-pdf-unicode.py (stdlib CPython only) regenerates
pdf_unicode.c byte for byte with CPython 3.14.6 (Unicode 16.0.0);
pdf_tables.c is maintained data from here on. License texts were taken
from their sources (adobe-type-tools/agl-aglfn, unicode.org).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The test-impact engine certifies the history it diffs and refuses a shallow
clone ("Graft or shallow state prevents history certification"). The shadow
job, the gate-mode select-tests job and the main-branch producer all checked
out with fetch-depth 2. So every shadow answer since #2458 was run_all:
GIT_UNAVAILABLE, then POLICY_UNAVAILABLE, because the merge base's
.codebase-memory.json is read through the same git facts. That was 24 of 24
measured PR runs. All three jobs now check out full history (about 400 MiB,
these jobs only).
The producer never published a bundle either. coverage-map.py exits 1 when a
suite fails under coverage: the map is still written and that suite's tests
are incomplete. test-impact-publish.sh ran it under set -e, so the publish
aborted. The failing suite was test_impact_git. Its hostile-diff probe
re-enters the runner as a child, which inherited CBM_TEST_COVERAGE_DIR,
started a coverage run of its own, and refused the per-test LLVM_PROFILE_FILE
it inherits.
Two changes:
- The probe now clears CBM_TEST_COVERAGE_DIR, as it already clears
CBM_TEST_ONLY_FILE. The child's profile then goes to the spawning test, like
any other child process.
- publish treats exit 1 as a written, conservative map (incomplete tests run
whole), with a warning that names the suite. It fails only on exit 2.
Proof:
- Depth-2 clone at f9ce3865: test-impact select answers run_all
[GIT_UNAVAILABLE, POLICY_UNAVAILABLE] with "Graft or shallow state ...", as
in CI. Same commit with full history: graph certified, run_all
[RUNNER_REACHED] (the change reaches the runner's main).
- Coverage runner (Homebrew LLVM, macOS), test_impact_git: before, 43 passed
and 2 failed ("a coverage run starts with LLVM_PROFILE_FILE=/dev/null");
after, 45 passed. coverage-map.py --suite test_impact_git: 45 tests,
0 incomplete, exit 0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The full Windows leg on the VM stopped at the coverage-map contract and was
reported green. Two defects, one hiding the other:
- The VM verdict counted the completion marker anywhere in the log. The
isolated-venue contract's stubbed entry prints that marker, and a capacity
case prints "expected=0 passed=True", so a leg that stopped at a later
contract, with no suite run, read as complete. The marker now counts only as
the log's last step header, which is where scripts/test.sh prints it; the
verdict contract pins the observed shape (14 cases).
- Under the native python3 the Windows legs install, the test-impact
contracts failed: the coverage-map builder cannot start the leg's bash
fakes, the scripts wrote CRLF, and cmp (diffutils) is not installed there.
selection.py, coverage-merge.py and coverage-map.py now write LF on every
host, so their outputs are byte-identical across platforms; the two cmp
checks compare bytes in the shell; the builder's functional leg is skipped
under a native Windows python with the reason and what was tried (the map
is built only by the ubuntu publish job), while both pure-python parser
tests still run there.
Production was not affected by the CRLF: run-tests-parallel.sh strips CR from
the selection and the runner trims its tokens.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
verify-shard-union.sh piped the shard slices into sha256sum, which macOS
runners do not ship. The new lane-gate contract cases run the script on
every venue, so test-unix (macos-14) failed with "sha256sum: command not
found" (GATE-QUALITY LOSS on a complete leg). Fall back to shasum -a 256,
as run-tests-parallel.sh already does.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pr.yml and test-impact-artifact.yml run scripts/ci/test-impact-select.sh,
-bundle.sh and -publish.sh as commands, but they were committed 100644
(core.fileMode is false in this clone, so the disk mode never reached the
index), and the PR shadow job failed with "Permission denied" (exit 126).
tests/test_script_exec_bit_contract.sh reports exactly this; it skipped the
scripts locally while they were still untracked.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Ten files changed on both sides; every conflict was a union of independent
additions:
- workflows (_lint.yml, _test.yml): keep the branch's lane `if:` and main's
pinned ubuntu-24.04 runner with its reason.
- internal/cbm/cbm.h: main's callable-identity and module-doc fields plus the
branch's configured-test fields.
- Makefile.cbm: both new test files; prod_lsp_all.o keeps main's
.build-config dependency and the branch's coverage flags.
- scripts/test.sh: main's setup-scripts contract becomes Step 0y3, ahead of
the branch's 0z series and the contracts-only exit.
- src/foundation/subprocess.c: main's cbm_posix_close_fds_from (it closes the
error descriptor too).
- src/mcp/mcp.c: main's repository default base for detect_changes, with the
early returns released through the branch's detect_release.
- src/pipeline/lsp_resolve.h, tests/test_main.c: both includes, suites and
start-up steps (main's git environment reset first).
- scripts/memory-core-baseline.txt: recounted on the merged tree and lowered
(mcp.c 773, config_toml_edit.c 73).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Producer. test-impact-artifact.yml publishes, on every push to main, the
commit's team artifact bundle: its frozen graph, built incrementally from the
previous bundle, and its per-test coverage map, re-run only for the suites
the push selects (scripts/ci/test-impact-publish.sh). A weekly run rebuilds
both from scratch and fails when the incremental graph's topology differs
from the full one. Bundles are Actions artifacts kept 30 days, never
committed.
Shadow. The report-only test-impact-shadow job also builds the checkout and
runs `test-impact select` against the base side of the PR merge commit
(scripts/ci/test-impact-select.sh). A bundle is used only when it comes from
a successful main-branch producer run for exactly that commit
(scripts/ci/test-impact-bundle.sh). The job stays continue-on-error and
outside ci-ok.
Narrowing switch, PR CI only. While the repository variable
TEST_IMPACT_MODE is `gate`, a select-tests job turns the engine's answer into
the tests the test-unix and test-windows legs run. Unset, nothing changes.
It fails open by construction:
- scripts/test-impact/selection.py is the only place an answer becomes a
runner filter. Only a well-formed selected lane whose filter matches its
own suite list narrows; run-all answers, a lane that selects nothing,
unknown schemas or tokens, and any crash run everything.
- `test` passes a selection only from a successful, narrowed select-tests
run; ci-ok does not need select-tests. A broken selection costs minutes,
never coverage.
- scripts/run-tests-parallel.sh launches only the selected suites and keeps
every guard on the selected list. A selected suite the build does not
register fails the run, unless the engine saw its registration as
conditional; then it is dropped with a notice, exactly as a full run of
that build never runs it. Each runner judges its own tokens, so a selected
test that does not exist fails its suite.
- The shard manifest records the selection's digest, and
verify-shard-union.sh requires every shard of a leg to agree on it.
- Dry runs and releases never pass a selection (lane-wiring contract).
Also fixed on the way:
- publish crashed on any run-all push: a run-all lane carries
"suites": null.
- the git-facts hostile-diff probe re-ran the runner with the parent's
CBM_TEST_ONLY_FILE still set; under a narrowed run naming its suite it
re-ran the spawning test, which spawned again. The child now clears it
(a new test fails without that).
- the select summary crashed on the same null suites.
Contracts: tests/test_test_impact_selection.sh (Step 0z8),
tests/test_harness_selection.sh (Step 2c, the real runner through the real
harness), and new cases in the lane-gate and lane-wiring contracts.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A main commit can now publish its test-impact bundle — the commit's frozen
graph, its per-test coverage map and receipt.json binding both — and a PR
selection given the bundle of its merge base, verified by the caller,
builds its graph incrementally from the bundle's graph and narrows suites
by the admitted map. Without a verified bundle nothing changes: the graph
is built from scratch and every reached suite runs whole.
Producer and consumer:
- `codebase-memory-mcp test-impact publish|select` (in process, no
daemon): publish builds the bundle of the checked-out commit, from the
previous bundle's graph when that bundle verifies; select is the
selection with optional --artifact/--artifact-commit/--artifact-verified.
The MCP surface is unchanged.
- receipt.json binds the exact imported graph (graph digest), the graph's
content and topology, the map's three files and a compatibility digest
(platform label + the bytes of test_impact.coverage.compatibility_paths).
- A bundle is used only when the caller verified its source, it is the
merge base's own, and its imported graph is the one the receipt names;
the map is admitted by the existing receipt policy (source, ancestry,
content, age, compatibility, metadata). Refusals become rejection
reasons in the answer's receipt.
- C_eff: a changed function the map knows contributes its own id; one it
does not know (new, platform-only, a type or macro) contributes its
nearest mapped callers on the static inbound closure; deleted names
contribute every mapped function of that name.
Graph:
- the frozen build accepts a base candidate and repairs it through the
incremental closure route; any doubt declines to the full build. Pinned
to one worker like the full frozen build.
- incremental closure repair skipped the interprocedural complexity pass:
re-parsed functions lost transitive_loop_depth/recursive and callers kept
stale values. One traversal now serves the in-memory pass and a new
store-level recompute that runs before the importance rescore.
- test-impact's frozen builds use a fixed display Git context: the per-run
pin path and the commit ids made two builds of one tree differ and made
every commit a semantic-input change, so repair always declined.
- graph content and topology digests (store_content_digest.c): the exact
graph digest binds one database (ids, db_uid, generation), never equal
across builds; the content digest keys rows by what they say, the
topology digest by what selection reads.
Measured on this repository (frozen publish, macOS arm64): a two-file C
change repairs in 18-20 s instead of 75-80 s with the topology of the full
build; a docs-only change repairs with identical content; selections with
and without the bundle are identical on all seven replayed pairs
(two repaired incrementally, five declined to the full build). Recorded, not
fixed: closure repair still differs from a full build in edge PROPERTIES
(argument quoting and key order, CONFIGURES args) and in the corpus-wide
SIMILAR_TO/SEMANTICALLY_RELATED edges; selection reads neither, so the
topology digest is the incremental-equals-full check and the receipt
carries both.
Tooling: scripts/test-impact/coverage-merge.py builds a push's incremental
coverage map (re-run suites verbatim, the rest carried with ids remapped
onto the new image's function table), with its contract as test.sh Step
0z7.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
One request pins the change (merge base .. HEAD), builds the graph and the
runnable test registry of exactly the HEAD snapshot, seeds the graph from
the change, walks what depends on it and answers which tests to run
(cbm.test_impact). Missing or doubtful evidence never narrows: it becomes a
run-all answer that names why.
Snapshot (classified ledger -> frozen graph -> registry):
- one shared language classifier for discovery and pinned blobs (raw bytes
on every platform); the classified ledger feeds the frozen build, which
indexes exactly the listed files and records partial parses as per-file
PARSE_GAP instead of refusing the graph;
- the pinned tree skips symlinks and submodule links (counted), as
discovery never indexes them;
- the test model scopes uncertainty to the suites of its file (they run
whole), reads line splices where they occur, treats a mid-file
`#ifndef X` + `#define X` as an ordinary condition, and no longer reads a
plain identifier named like a macro as an invocation. On this repository
the registry is now narrowable and matches a text census exactly
(8,365 cases, 141 runner suites).
Seeding and reach (port of the measured replay, rule B):
- hunk -> overlapping definitions, comment-only hunks dropped, pure
deletions -> enclosing definition, no-definition hunks -> whole file,
deleted names found by text, header changes propagated through the
include closure and the headers' macros (pastes included), file-level
escalation for unclassifiable header lines;
- inbound fixpoint walk over CALLS, ASYNC_CALLS, USAGE, CALL_REFERENCE,
READS (+ routes); cases, suites and the runner's entry points are sinks;
a reached runner main runs everything (RUNNER_REACHED);
- a parse gap whose text names a reached definition reaches the
definition around it (or its file);
- macro-registered tests run their suites whole; cases and suites map to
graph nodes by their start line.
The line patterns reproduce the reference regexes exactly on 1,383,850
lines of this repository and 37,167,926 kernel lines; on 24 replayed
changes the engine selects at least what the reference selected (13
identical, the rest only conservative supersets on the same graph).
Also:
- configured test declarations that cannot be mapped degrade their file
(no configured roles, a test_declarations diagnostic) instead of
aborting the whole index;
- the selector keeps other runners' suites out of the selection, and the
result codec judges the model by narrowability like the selector;
- a change to .codebase-memory.json runs everything (built-in rule);
- .codebase-memory.json with the c-cbm preset and this repository's rules.
Gate fixes (suites, lint-ci, Windows cross-compile):
- the request runs on its own thread with a default-size stack (daemon
tool calls run on 256 KiB threads, where the in-process build died);
- a request-supplied config_path is no longer followed by an unlink of an
uninitialized path buffer;
- the inventory's frame-local reader no longer stays reachable from the
caller's source after the filter call;
- FILE_ID_INFO needs _WIN32_WINNT >= 0x0602; the pinned-tree header pins
it before any system header (toolchains defaulting lower failed);
- memory-core ratchet: the win_utf8 converters gain core-backed
`_in(cls, ...)` forms over one implementation (the legacy names keep
the C library for unmigrated callers), the Git facts module uses them,
and the Windows spawn path releases everything through one failure
exit; pipeline, extraction and config allocations that this work
touched moved to the core in matched pairs; baselines tightened;
- the branch's C files formatted with the project's clang-format.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
gcc's libasan turns detect_stack_use_after_return ON by default; clang's
runtime leaves it off. On aarch64, gcc 13/14 free small fake frames
without clearing their in-use flag, so once 8,192 frames of a size class
have been used, every instrumented call scans the flag array (~8 us
instead of ~6 ns) and the check silently stops detecting anything. CI's
ubuntu-24.04-arm gcc leg runs the extraction suite in 688 s against a
900 s cap (16 s on amd64), and the deep extraction tests of #2478 push
it over the cap.
The lanes already document the intent: test-diag in _test.yml and
`make -f Makefile.cbm diag` set detect_stack_use_after_return=1
explicitly, as one of the ASan checks that are off by default in every
other lane.
scripts/test.sh now appends detect_stack_use_after_return=0 to
ASAN_OPTIONS (keeping any options already set, ':'-separated) unless
ASAN_OPTIONS already mentions the flag, and exports it, so test-diag's
explicit =1 wins. Lanes built with clang already ran with it off and
are unchanged. The venue-parity contract (Step 0j) gains a layer that
drives the real scripts/test.sh with a no-op make and a probe runner
and asserts three cases: unset, other options kept with the flag
appended, explicit =1 kept.
Measured in the local Linux arm64 container (Colima, gcc 13.3,
ASan + UBSan), wall time of `scripts/test.sh --suites extraction`:
origin/main 268a9d88 without this change 628.6 s
origin/main 268a9d88 with this change 5.1 s
#2478 head 818ef0ec with this change 9.7 s
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel