The v0.12.1 release smoke failed on every leg in Phase 13w, the
package-wrapper install gate added in #2664:
- macOS: the runner images have no Go on PATH, and
SMOKE_REQUIRE_WRAPPERS=1 rightly fails without one. pr-smoke and the
three release smoke jobs now set up Go from pkg/go/go.mod, with the
same pinned step the wrapper test job uses.
- Windows: `mkdir -m 700` changes the mode in a second step, which MSYS2
cannot do under the smoke profile's owner-only DACL. mktemp -d sets
the mode at creation.
- Linux: the runner's older Go downloaded the 1.26.1 toolchain into the
smoke profile's HOME. Go writes its module cache read-only, so the
harness could not delete its profile afterwards. All Go state now stays
in the wrapper's own work tree, and its cleanup restores write
permission before removing it.
npm and PyPI passed on every leg that reached them; the gate itself is
unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
v0.12.0 could not be installed through npm, pnpm, pip or the Go wrapper on
any platform (#2657): each wrapper refused an archive member above 256 MiB,
and every v0.12.0 binary is 331-335 MB. The v0.11.0 binaries were already
about 300 MB, so the Go wrapper, which has had the cap since v0.10.0, was
broken before; npm and PyPI gained it in v0.12.0. The archive is verified
against the release's checksums.txt before it is unpacked, so a byte cap
only ever refused what we ship.
- npm, PyPI, Go: delete every byte-size cap (compressed archive, per member,
aggregate, checksum manifest, download size). The checksum, the exact
member set, path and link refusal, the member count, HTTPS and the
--version check stay.
- All three honour CBM_DOWNLOAD_URL like install.sh, for testing: https, or
plain http to an exact loopback authority with no proxy and no redirect.
- Smoke: scripts/smoke-wrappers.sh installs the three wrappers from the
checkout against the release fixture and runs the binary each one
installed. smoke-test.sh Phase 13w runs it; SMOKE_REQUIRE_WRAPPERS=1 makes
it required on the PR smoke lanes and in the release smoke, which gates
publishing.
Other install paths found broken while checking every channel:
- Homebrew: the formula was still at 0.10.3 and installed the dynamic Linux
build, which needs glibc 2.38 (Ubuntu 22.04, Debian 12 and RHEL 9 lack it).
It now installs 0.12.0 with the static -portable Linux assets and tracks
the newest release in the version contract like the other surfaces.
- Go: `go install .../pkg/go/cmd/codebase-memory-mcp@vX.Y.Z` never worked
(no pkg/go/v* tags, so Go fetched the whole repository, over its 500 MB
module limit), and @latest built main's wrapper, whose version constant
lags each release. The release now also tags pkg/go/vX.Y.Z, and the
wrapper installs its own module version (the constant only for local and
untagged builds).
- npm: a stable release also moves the `next` dist-tag to itself; `next`
still pointed to 0.9.1-rc.1.
- Docs: winget and Chocolatey were listed as install channels but are not
published; they are removed until they are. Windows arm64 ships and is
now listed.
Fixes#2657.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The license gate installed `scancode-toolkit` unpinned. pluggy 1.7.0
(uploaded to PyPI 2026-10-15 09:50 UTC) no longer finds the hookspecs that
plugincode 32.0.0 declares on a class (`@pre_scan_spec class
PreScanPlugin`), so scancode crashes at import with "ValueError: did not
find any 'pre_scan' hooks" and `security / license-gate` fails on every
run, whatever the tree. The last green gate started at 09:50:14 UTC;
every run since is red. `ci-ok` and the release `verify` job both need the
security workflow, so this blocks every PR and the release.
Install exactly scancode-toolkit 32.5.0 (the version every green run
used) with pluggy constrained to 1.6.0 through PIP_CONSTRAINT, which the
pip that pipx runs reads. The local hint in license-gate.sh names the
same command.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The lint job's single-process cppcheck pass now takes 22-29 minutes:
the src tree grew about 60 % in files in ten days (210 files / 7.4 MB on
2026-10-04, 335 / 11.2 MB since 2026-10-12) while the cppcheck flags
stayed the same. Three runs on 2026-10-13 (38330125249, 38345850562,
38370789949) were cancelled at the 30-minute cap with cppcheck still
running and every earlier lint step clean. A cancelled lint is a hard
stop for the release graph, and the dry run's tests need lint to pass.
45 minutes puts the bound back where only a genuine hang reaches it.
Parallel cppcheck (-j) would be faster, but it changes cppcheck's
whole-program checks; it is left for its own change with a
findings-equivalence proof.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The job log of a red test job carries a summary of each failing suite,
not its log, and the only artifact was shard-manifest.txt. A failure
whose summary was not enough could only be understood by reproducing it.
The four jobs in .github/workflows/_test.yml that run the parallel
harness (test-unix, test-windows, test-diag, test-lsan-macos) get one
more step, "Failing suite logs". It runs only when the job failed and
uploads build/c/test-logs/failed/: the logs of the suites that did not
end green, plus results.txt, as the harness leaves them there. One red
suite is typically a few kilobytes; a whole run's logs are about 25 MB.
Retention is 7 days. The artifact names follow the shard-manifest ones
(suite-logs-<os>-<compiler or msystem>-<job index>, suite-logs-diag,
suite-logs-lsan-macos), and the step uses the upload-artifact reference
the file already pins.
A job that failed outside the suite waves has no such directory; the
step then warns and uploads nothing. The step cannot change a job's
result: it is skipped on a green job, and a failed job stays failed
whatever the upload does. Triggers, permissions, job dependencies,
timeouts, the matrix and the shard-completeness download pattern are
unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Five suites (grammar_regression grammar_labels pipeline lang_contract
grammar_probe_e) were excluded from the MSan lane for "stack-overflow
under instrumentation". That was never deep recursion. MSan aborted while
REPORTING real uninitialized reads:
- every abort was on the main thread. MSan's "T<n>" is the OS thread id
and equalled the pid in all five x86-64 aborts (CI job 91868259558);
the old block read it as a worker-thread ordinal, so the worker-stack
knob, MSAN_ORIGINS and CBM_WORKERS=1 could never change anything;
- with the default slow unwinder (the image's MSan-instrumented
libunwind) a report exhausted the main thread's 256 MiB stack, and the
overflow handler then failed with "nested bug in the same thread";
- with fast_unwind_on_fatal=1 the same test prints the
use-of-uninitialized-value report instead (measured on one tree and
test: slow unwind aborts at the stack limit, fast unwind reports
wit/scanner.c:13).
The two reads are fixed in the previous commits of this series, so:
- MSAN_OPTIONS defaults to fast_unwind_on_fatal=1 (the build already
keeps frame pointers);
- MSAN_EXCLUDE is down to cli, whose install-path failure is a separate,
still-undiagnosed cause;
- the known-red block, the header (it still claimed CI runs without
exclusions) and the MSAN_ORIGINS and stack comments say what is true;
- the partial-lane warning and the exclude-name typo check lived in a
branch behind `elif true` that never ran. They now run in the
per-suite path, the check as a case match instead of a grep -q pipe
that pipefail can misreport;
- the comments in scripts/ci/msan-lane.sh and above the test-msan job
in _test.yml no longer say the five suites overflow.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reconcile the complete Atlas and Lens stream with current main while retaining both histories. Preserve architecture functionality, current process and watcher safeguards, test infrastructure, and dependency security updates. Add coverage for Git-environment isolation combined with child-local overrides.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The test-impact engine certifies the history it diffs and refuses a shallow
clone ("Graft or shallow state prevents history certification"). The shadow
job, the gate-mode select-tests job and the main-branch producer all checked
out with fetch-depth 2. So every shadow answer since #2458 was run_all:
GIT_UNAVAILABLE, then POLICY_UNAVAILABLE, because the merge base's
.codebase-memory.json is read through the same git facts. That was 24 of 24
measured PR runs. All three jobs now check out full history (about 400 MiB,
these jobs only).
The producer never published a bundle either. coverage-map.py exits 1 when a
suite fails under coverage: the map is still written and that suite's tests
are incomplete. test-impact-publish.sh ran it under set -e, so the publish
aborted. The failing suite was test_impact_git. Its hostile-diff probe
re-enters the runner as a child, which inherited CBM_TEST_COVERAGE_DIR,
started a coverage run of its own, and refused the per-test LLVM_PROFILE_FILE
it inherits.
Two changes:
- The probe now clears CBM_TEST_COVERAGE_DIR, as it already clears
CBM_TEST_ONLY_FILE. The child's profile then goes to the spawning test, like
any other child process.
- publish treats exit 1 as a written, conservative map (incomplete tests run
whole), with a warning that names the suite. It fails only on exit 2.
Proof:
- Depth-2 clone at f9ce3865: test-impact select answers run_all
[GIT_UNAVAILABLE, POLICY_UNAVAILABLE] with "Graft or shallow state ...", as
in CI. Same commit with full history: graph certified, run_all
[RUNNER_REACHED] (the change reaches the runner's main).
- Coverage runner (Homebrew LLVM, macOS), test_impact_git: before, 43 passed
and 2 failed ("a coverage run starts with LLVM_PROFILE_FILE=/dev/null");
after, 45 passed. coverage-map.py --suite test_impact_git: 45 tests,
0 incomplete, exit 0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Ten files changed on both sides; every conflict was a union of independent
additions:
- workflows (_lint.yml, _test.yml): keep the branch's lane `if:` and main's
pinned ubuntu-24.04 runner with its reason.
- internal/cbm/cbm.h: main's callable-identity and module-doc fields plus the
branch's configured-test fields.
- Makefile.cbm: both new test files; prod_lsp_all.o keeps main's
.build-config dependency and the branch's coverage flags.
- scripts/test.sh: main's setup-scripts contract becomes Step 0y3, ahead of
the branch's 0z series and the contracts-only exit.
- src/foundation/subprocess.c: main's cbm_posix_close_fds_from (it closes the
error descriptor too).
- src/mcp/mcp.c: main's repository default base for detect_changes, with the
early returns released through the branch's detect_release.
- src/pipeline/lsp_resolve.h, tests/test_main.c: both includes, suites and
start-up steps (main's git environment reset first).
- scripts/memory-core-baseline.txt: recounted on the merged tree and lowered
(mcp.c 773, config_toml_edit.c 73).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Producer. test-impact-artifact.yml publishes, on every push to main, the
commit's team artifact bundle: its frozen graph, built incrementally from the
previous bundle, and its per-test coverage map, re-run only for the suites
the push selects (scripts/ci/test-impact-publish.sh). A weekly run rebuilds
both from scratch and fails when the incremental graph's topology differs
from the full one. Bundles are Actions artifacts kept 30 days, never
committed.
Shadow. The report-only test-impact-shadow job also builds the checkout and
runs `test-impact select` against the base side of the PR merge commit
(scripts/ci/test-impact-select.sh). A bundle is used only when it comes from
a successful main-branch producer run for exactly that commit
(scripts/ci/test-impact-bundle.sh). The job stays continue-on-error and
outside ci-ok.
Narrowing switch, PR CI only. While the repository variable
TEST_IMPACT_MODE is `gate`, a select-tests job turns the engine's answer into
the tests the test-unix and test-windows legs run. Unset, nothing changes.
It fails open by construction:
- scripts/test-impact/selection.py is the only place an answer becomes a
runner filter. Only a well-formed selected lane whose filter matches its
own suite list narrows; run-all answers, a lane that selects nothing,
unknown schemas or tokens, and any crash run everything.
- `test` passes a selection only from a successful, narrowed select-tests
run; ci-ok does not need select-tests. A broken selection costs minutes,
never coverage.
- scripts/run-tests-parallel.sh launches only the selected suites and keeps
every guard on the selected list. A selected suite the build does not
register fails the run, unless the engine saw its registration as
conditional; then it is dropped with a notice, exactly as a full run of
that build never runs it. Each runner judges its own tokens, so a selected
test that does not exist fails its suite.
- The shard manifest records the selection's digest, and
verify-shard-union.sh requires every shard of a leg to agree on it.
- Dry runs and releases never pass a selection (lane-wiring contract).
Also fixed on the way:
- publish crashed on any run-all push: a run-all lane carries
"suites": null.
- the git-facts hostile-diff probe re-ran the runner with the parent's
CBM_TEST_ONLY_FILE still set; under a narrowed run naming its suite it
re-ran the spawning test, which spawned again. The child now clears it
(a new test fails without that).
- the select summary crashed on the same null suites.
Contracts: tests/test_test_impact_selection.sh (Step 0z8),
tests/test_harness_selection.sh (Step 2c, the real runner through the real
harness), and new cases in the lane-gate and lane-wiring contracts.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
CodeQL SAST keyed its concurrency group by commit SHA on push events,
so every merge to main got its own group and nothing was ever
cancelled; OpenSSF Scorecard had no group at all. On 2026-10-04 a day
of 36 merges queued one CodeQL analysis (40-100 minutes each) and one
Scorecard run per merge, ahead of every pull request run: pull request
CI waited hours for a runner, and a main analysis waited 4 hours.
CodeQL now groups by event and by pull request number or ref, so pull
requests keep one analysis each as before, and a newer push to main
cancels the analysis of an older main commit. Scorecard groups by ref
with the same cancellation. Only the newest main commit matters for
both: code scanning tracks the default branch's latest analysis.
DCO keeps its per-commit group on purpose: its push run on main is the
check that catches a merge rewriting sign-offs, so a superseded push
must still be checked, and a DCO job takes seconds.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Under R1, a T0b push (test-infrastructure/, lint config, graph-ui, memwaste
tooling) selected lint and no test leg, so the static contract steps that
police exactly those files -- the VM scripts, the compose and run.sh wiring,
the venue-parity rules -- ran nowhere. Replayed over September, 13 of the 18
T0b pushes ran no contract step at all.
The R1 table now gives T0b lint + contracts, so those PRs get the
contracts job (scripts/test.sh --contracts-only) as T0a PRs do. The
fixture table's six T0b cases expect it, and the history replay asserts
that every push runs the contract steps (the contracts lane or any test
leg). Comments that described the job as docs-only are updated.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The pulls/files API lists at most 3000 files and says nothing when it stops,
and the lane decision is only as complete as that list: a cut list can drop
exactly the file that needed a lane.
select-lanes.sh takes --expect-files N (the PR's changed_files; batch rows
take "expect_files") and decides full with a file_list_truncated reason when
the list has fewer distinct paths than N (a complete list never does: renames
only add their old path), when N is at or over the 3000-file cap, or when the
list itself reaches the cap. The tier still classifies what was listed. An
empty or non-numeric count is a usage error, so a workflow that loses the
count fails loudly instead of skipping the check.
pr.yml passes github.event.pull_request.changed_files. The selector contract
covers short, complete, rename-inflated, at-cap and just-under-cap lists and
the github-output and batch channels; the wiring contract requires the count
to be passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
_test.yml takes a shard_profile input: "standard" (the default; dry run and
release pass nothing and keep ubuntu x3 / windows x2, byte-identical
matrices) or "pr", which pr.yml now uses: ubuntu x86 and Windows run as one
job each and only arm64, the slowest leg, splits in two. A PR runs few legs
and every shard re-pays its setup, so fewer, longer jobs cost less. An
unknown profile is an error. The wiring contract pins both the default and
PR's use of the profile.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A new pr.yml job, test-impact-shadow, predicts from the code graph which C
test suites a change can affect and uploads the prediction, so graph-based
selection can be judged against what actually failed before anything gates
on it. It is continue-on-error and not in ci-ok's needs: it can never block
or pass a merge.
scripts/ci/test-impact-shadow.sh --fetch-release downloads the latest
published release archive for the host plus checksums.txt with the job's
read-only token and verifies the SHA-256 (seconds instead of a build, and a
PR that breaks the indexer cannot corrupt its own prediction); pr.yml builds
the checkout only if that fails. --binary/--out then indexes the checkout
into a throwaway HOME/cache with a private CBM_RUNTIME_DIR (short, under
/tmp, so the daemon socket fits sun_path), runs detect_changes against the
merge commit's base at depth 15 and hands it to predict.py. The graph is
only consulted when C sources changed.
scripts/test-impact/predict.py ports the replay's mapping: impacted test
functions -> registered suites (perf excluded; nested RUN_SUITE children
resolve to the registered suite that runs them), changed test files ->
whole suite, fixtures -> suites that name them, grammars -> grammar suites,
and run-all on the selector's full, a test header, test infrastructure, or
a saturated/truncated impact set. It writes prediction.json and a short
step-summary table.
tests/test_test_impact_predict.sh (Step 0z5) pins the mapping on a
synthetic tree; the wiring contract requires the job to stay
continue-on-error; the script joins the venue-parity interface probes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pr.yml's `changes` job now lists the changed files (renames with their old
path too), runs scripts/ci/select-lanes.sh in a credential-less step and
outputs tier, full, lanes and the pr-smoke matrix. security, lint and test
pass the lanes to their reusable workflows, which take a `lanes` input
defaulting to "all": dry-run.yml and release.yml pass none and run exactly
as before (setup-matrix output for every broad/shard combination is
byte-identical; the TSan matrix moved into setup-matrix with the same legs).
Under a selection, setup-matrix keeps only the tagged legs of the selected
lanes (an untagged leg is an error, never a silent drop) and each lane job
has an `if:` on its lane. pr-smoke and memwaste run on their lanes instead of
the old product regex. Docs-only PRs (T0a) get a `contracts` job running the
new `scripts/test.sh --contracts-only` (Step 0 contracts, no build). ci-ok
needs contracts, lists the run's jobs (actions: read) and passes LANES/JOBS
to require-all-green.sh; shard-completeness passes --lanes.
tests/test_lane_wiring_contract.sh (Step 0z4) pins it: dry run and release
pass no lanes and the inputs default to all, pr.yml wires the selection
through, and every lane is served by a tagged leg or an `if:` naming it.
The smoke-fixture contract now checks the selector's decisions for the
smoke-harness paths instead of the removed regex; select-lanes.sh joins the
venue-parity interface probes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Distilled from #2109 by Rares Popa. Two slices only; the rest of that PR is
either frontend work the UI rebuild makes redundant, or a gating change that
needs re-siting first.
1. Three jobs move from ubuntu-latest to ubuntu-24.04
_lint.yml `lint` and `lint-mem`, and _test.yml `test-diag`, each install a
toolchain from an apt.llvm.org repo whose name contains the release:
deb http://apt.llvm.org/noble/ llvm-toolchain-noble-20 main
deb http://apt.llvm.org/noble/ llvm-toolchain-noble-21 main
noble IS 24.04. So these jobs already depend on a specific release while
asking for whatever ubuntu-latest happens to mean that morning. The day
GitHub advances ubuntu-latest, all three break at once, on a schedule
nobody here chose, and the failure surfaces as an apt error far from its
cause. Pinning the runner to the release the repo line already names costs
nothing and removes the trapdoor.
2. actions/attest-sbom -> actions/attest
attest-sbom is deprecated. Its remaining behaviour is to print
"::warning::actions/attest-sbom has been deprecated" and forward to
actions/attest with NODE_OPTIONS set, passing sbom-path through unchanged.
Calling attest directly yields the same attestation minus the wrapper.
Verified rather than assumed, because this sits on the release path:
59d89421af93a897026c735860bf21b6eb4f7b26 is exactly tag v4.1.0 of
actions/attest, and sbom-path is a real input on that action at that SHA.
The step keeps no continue-on-error, so a failed SBOM attestation still
fails the release. That is the intended behaviour and the comment now says
so: an unattested SBOM is worse than an absent one, because the artifact
ships looking complete.
Deliberately NOT taken from #2109, each for its own reason:
- the actionlint job, as sited. _lint.yml is consumed by pr.yml, dry-run.yml
and release.yml, pr.yml gates its whole test stage on `if lint succeeded`,
and release.yml gates build on needs.lint.result == 'success'. A job there
is merge-blocking and release-blocking with no branch-protection edit to
make it visible, and this one curls a tarball from github.com releases with
no cache and no fallback. actionlint is worth having; not on that path, and
not before it has been run against current main.
- the test-package-wrappers node matrix. It reruns the Go and PyPI suites a
second time per OS to vary a Node version only the npm leg reads.
- everything frontend: the test-frontend job, .node-version, and the
dependabot graph-ui block. The UI is being rebuilt from the ground up, so
gating on the current one buys coverage of a surface that is going away.
Co-authored-by: Rares Popa <2606875+rarepops@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dependabot opened one PR per sub-action (#2363 init, #2365 analyze, #2364
upload-sarif). CodeQL refuses a split pin: with init at 4.38.1 and analyze
at 4.38.0 the analyze post-step fails with "Loaded a configuration file for
version '4.38.1', but running version '4.38.0'", so neither PR can pass
the codeql gate alone and merging either one would break the gate on main.
This bumps all three to 1c5b675653bb5c22dbe9b12b556ec555138e09fd, which is
the commit the upstream v4.38.1 annotated tag points to.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Found while proving the previous commit, in the same step, and the worse
of the two defects because it fails OPEN. The classification was
if printf '%s\n' "$FILES" | grep -qE '^(src/|internal/|...)'; then
under the runner's `bash -eo pipefail`. grep -q exits on its first match;
once the list is larger than the pipe buffer printf is still writing,
dies of SIGPIPE, and pipefail turns the MATCH into a failed pipeline. The
else branch then writes product=false, and pr-smoke and memwaste are
skipped -- on exactly the PRs that change the most files. The previous
commit's message flagged this as a follow-up; it is fixed here instead.
The list now reaches grep through a here-string, so there is no pipe to
break. The regex is unchanged.
Proof, by executing the workflow's own run block under
`bash -eo pipefail` with a stub gh, ten runs each:
849 KB list, first line src/a.c before: product=false 10/10
after: product=true 10/10
849 KB list, docs only product=false before and after
small docs-only list product=false
422 fallback on the real #2246 product=true
fallback, nonexistent merge ref rc 128, no output written
It takes roughly 1,500 changed files to cross the buffer, which is why
ordinary PRs never showed it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The `changes` job decides whether a PR touches product code, and ci-ok
requires it. It asks GitHub for the PR's file list. An earlier fix moved
it off the .diff endpoint, which rejects PRs over 20k lines, onto the
paginated pulls/files endpoint -- but that endpoint renders the diff
server-side as well, and for a PR that regenerates a vendored parser it
answers
HTTP 422: Sorry, this diff is taking too long to generate.
Observed on #2246 (a 42 MB sql/parser.c): `changes` red, therefore ci-ok
red, for a reason that has nothing to do with the contribution. The
compare endpoint fails the same way, so there is no API route to the
file list for such a PR. Every grammar refresh would hit this.
When the API call fails, the job now takes the list from git instead.
The PR merge ref's first parent is the base, so a name-only diff across
that one merge commit is exactly the PR's changed files. The fetch is
depth 2 with --filter=blob:none into a bare repository under RUNNER_TEMP:
commits and trees only, no blobs, nothing checked out and nothing from
the PR executed. No new action, no new permission, and the API path and
the classification regex are untouched, so ordinary PRs behave as before.
A ::notice:: line records when the fallback was used. If the fetch fails
too, the step exits non-zero and writes no output -- the gate fails
closed, as it did before.
Proof, by executing the workflow's own run block under
`bash -eo pipefail` with a stub gh:
API path, docs-only list rc 0 product=false
fallback, real #2246 rc 0 product=true, the same five
files `git diff --numstat`
reports for the PR; 0.85 s,
188 KB fetched
fallback, nonexistent merge ref rc 128 no output written
Contract tests that read pr.yml -- security_gate_fail_closed,
smoke_fixture_contract, windows_bundle_contract, venue_parity_contract --
pass.
Not addressed here, noted for a follow-up: the classification pipes the
list into `grep -q` under pipefail, which can report a match as a
failure once the list outgrows the pipe buffer.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dependabot split one upgrade across three PRs -- init (#2185), analyze
(#2187) and upload-sarif (#2186) -- but init and analyze share a
configuration file and CodeQL refuses to read one written by a different
version:
Loaded a configuration file for version '4.38.0',
but running version '4.37.9'
So #2185 and #2187 each go red on their own and neither can reach a green
required check, which means they cannot be merged one after the other
either: whichever lands first leaves the pair mismatched. The action also
warns about it directly -- "Not all workflow steps that use
github/codeql-action actions use the same version".
Both refs in codeql.yml therefore move in one commit, to the same pinned
SHA b96794f015dfd88f77b49b1c93e0fa7110f94c63 that both bot PRs target.
scorecard.yml's upload-sarif is deliberately not touched here: it runs in
a separate workflow that shares no configuration with init/analyze, which
is why #2186 is green on its own and can land as its own PR.
Supersedes #2185 and #2187.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The memory-core linter is a ratchet: raw malloc/calloc/realloc/free/strdup
sites may not grow, because allocations belong on one accounted path
(src/foundation/mem_core.h) rather than spread across hundreds of call
sites. This branch's own work had added 35 raw sites across nine files --
the parallel k8s prep, the configlink key matching, the worker pool's
counted path, the Go shared stdlib registry, the spill namespace copy,
the route service array and the namespace-name arrays.
All of them now use cbm_alloc/cbm_calloc/cbm_realloc/cbm_free/
cbm_mem_strdup. One was a latent bug rather than a style point:
pass_k8s freed a buffer that k8s_read_file had allocated with raw malloc,
so converting only the free would have mismatched the allocator -- the
read function and every free of its result are converted together.
The Windows free-space path widens its argument through the memory core
instead of cbm_utf8_to_wide(), which allocates with raw malloc.
Two files improved enough to tighten their baselines, which the ratchet
requires so the gain cannot be quietly given back: sqlite_writer.c 92->74
(the index-cell arena), pass_k8s.c 13->6 and registry.c 19->17.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/fuzz.sh drives the libFuzzer targets in two shapes from one entry:
a fixed --runs from a fixed --seed, so a CI smoke is a function of the code
and not a lottery (O9), and a time-boxed --max-total-time for exploration.
Checked-in seeds are copied into a working corpus under build/fuzz/, so a
run never rewrites the repository; a crash leaves its reproducer behind and
the script exits non-zero. Wired as a smoke into dry-run.yml and as
exploration into nightly-soak.yml.
The venue-parity contract rightly refused both workflow steps: a venue may
provision or plumb, but only a CANONICAL leg entry may exercise the
product, and neither script was registered. Registering them — rather than
working around the contract — is the fix, since both own their leg end to
end (build, corpus or binary, artifact paths, exit code) exactly as
soak-legs.sh owns the soak. _memwaste.yml joins the policed venue walk for
the same reason, and both scripts join the layer-5 interface probes.
Those probes immediately earned their keep, finding two real defects:
`fuzz.sh --help` exited 2 because the positional target consumed the flag,
and memwaste.sh read a typo'd flag as the repository path — which would
have indexed nothing and reported an empty lane as a clean one.
test_shell_line_endings.sh now degrades where the checkout has no
reachable repository metadata: the Linux container bind-mounts the working
tree, and a git worktree's .git is a pointer to the host repository, so
`git ls-files` cannot run there and the contract failed with "matched no
files — the glob set is broken", killing the leg before it compiled. It is
a repository property, so it stays gating in every venue that has the
metadata (macOS host, the Windows VM's real checkout at C:\cbm with 117
files, every hosted-CI checkout). test_version_metadata_contract.sh already
guards the same way for the same reason; the alternatives considered and
rejected are recorded at the case.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Answers a question the existing tooling could not: of everything we
allocate, how much is never used?
Lanes, all driven by scripts/memwaste.sh:
event every allocation observed through the memory core — size, age,
churn, and whether the bytes were ever written
access SanitizerCoverage load/store instrumentation, so "allocated and
never touched" is measured rather than inferred
scaling k vs 2k replicas through clang profile counters, to catch
super-linear work per file
padding allocator slack per size class
ratchet the numbers recorded per venue so a regression is visible
The event lane needed a foreign-free counter: on Windows the override sees
frees for blocks it never allocated (they come from the CRT), and counting
those as untracked tripped the soundness gate. mem_events now separates
them (14 daemon / 5 worker on the Go corpus).
Two portability fixes the lanes surfaced, both real:
- the enable decision must not be cached before `environ` exists. Under
-fsanitize-coverage the UBSan preinit runs dlsym -> malloc -> our
wrapper before glibc sets up environ, so the lane silently disabled
itself and produced no dump at all on Linux.
- the access map needs 48 bits: arm64 Linux hands out addresses above
2^47, which a 47-bit map dropped on the floor.
mem_profile is retired into this: it measured a strict subset and had its
own sampling story.
Local-only by decision — not a PR-CI gate. _memwaste.yml runs the lanes on
all three platforms report-only, and pr.yml/release.yml carry the wiring.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The sysctl that lifts the AppArmor restriction is best-effort, and the test
runner prints per-shard aggregates only -- 'N passed, M skipped', never
individual test names. So if the sysctl stopped working, the #1830 namespace
test would quietly return to skipping and the log would stay green and look
identical. That is the same false assurance that let a security relaxation ship
behind a test which never executed.
Probe it the way the test does (unprivileged, no sudo) and print the answer, so
there is one greppable line of evidence per ubuntu leg. Deliberately not a gate:
a runner that cannot lift the restriction should still run everything else.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three maintainer decisions from the challenger read on the #1830 unit.
1. DELETE THE CACHE (not just fix its comment).
posix_ancestor_overflow_uid() memoised via pthread_once behind a comment
claiming "immutable /proc state". Both halves were false: unshare(CLONE_NEWUSER)
rewrites /proc/self/uid_map, and pthread_once state survives fork() already
marked done. A process that forked and then entered a namespace kept the
parent verdict and refused a directory it should have accepted.
It was harmless only because every caller today runs in a freshly exec'd
process -- the daemon double-fork execs, workers go through
cbm_subprocess_spawn's fork+exec, worker_pool.c never forks, and src/ contains
no setns or unshare at all. That made a security decision depend on an
invariant nothing enforced. The next fork-without-exec caller would have
inherited a stale verdict silently.
Now derived fresh per call: two small /proc reads against an openat + fstat +
fchmod + ACL check per path component in the same walk.
With a test that would have caught the original bug and needs no namespace.
daemon_ipc_posix_overflow_uid_is_never_cached_issue1830 counts real
derivations through the production entry point and asserts the count rises on
every ancestor check. Re-introducing a cache fails it by name on every
platform. Proven both ways on linux/arm64 as uid 1001: cache re-introduced
49 passed 1 failed; removed 50 passed 0 skipped.
Worth knowing why that test is not redundant: with the re-exec probe in
place, the userns smoke test PASSES even with the cache restored, because the
probe is a fresh process and immune to it by construction. The two tests are
complementary -- the smoke test binds the namespace wiring, this one binds
the absence of the cache.
2. RUN THE GUARD ON THE CORE MATRIX.
Ubuntu 23.10+ ships kernel.apparmor_restrict_unprivileged_userns=1, so the
#1830 smoke test skipped on every 24.04 leg and its only real-namespace
coverage sat on the two ubuntu-22.04 legs. Those are broad-matrix only, so
the test ran in NO pull request, and GitHub retires those images on
2027-04-17 -- after which it would have become a permanent silent skip, and a
skip is not a red. A deliberate relaxation of a security check was guarded by
a test that ran nowhere a change would be reviewed, on a clock.
_test.yml now lifts the restriction with one sysctl on every ubuntu leg
(hosted runners have passwordless sudo), best-effort so a runner that refuses
simply returns to skipping. The O10 skip-whitelist comment no longer presents
the AppArmor restriction as immovable.
3. KEEP THE TOLERANCE, CORRECT THE SAFETY ARGUMENT.
The comment claimed that in a single-uid map "nobody reachable can have
created it or can mutate it". The overflow uid is what EVERY unmapped host
uid maps to, not only host root (user_namespaces(7)), so overflow-owned does
not prove root-created: on a shared host another local user's sticky
directory is indistinguishable from root-owned /tmp from inside the namespace.
The tolerance stays, because there is no in-namespace discriminator that
could tell them apart -- and narrowing it would refuse the legitimate
root-owned 01777 /tmp that #1830 exists to accept, i.e. it would revert the
fix in practice. What changes is the claim: the real guarantee is that no
principal reachable from inside the namespace can create or mutate such an
ancestor, and the leaf is never tolerated as overflow (0700, euid-rechecked),
so a hostile host-side ancestor owner is bounded to denial of service and
socket-path control.
macOS daemon_ipc 53 passed 2 skipped; linux/arm64 50 passed 0 skipped;
lint-ci clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#2189 gave codeql-gate the security-events:read it needs to see code-scanning
alerts, and granted it at the two callers I knew about: pr.yml and release.yml.
There is a third. dry-run.yml calls the same reusable workflow and was left
granting only the workflow default, so every dispatch of it now fails at
startup -- before a single job runs, with no job output to explain why.
Observed immediately: run 34725401182 on 7d74337e, startup_failure.
The contract test did not catch it because it iterated a hardcoded
("pr.yml", "release.yml"). A test that checks only the callers you remembered
cannot catch the one you forgot, and it reports green while doing so -- the
same shape of false assurance as the 403-to-"0 alerts" bug this whole contract
exists to prevent.
So the list is now derived: every workflow containing
`uses: ./.github/workflows/_security.yml` must grant security-events:read and
actions:read, with a floor of three callers so a silent deletion is noticed
too. Reverting only dry-run.yml's grant now fails the test by name.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The codeql-gate has never checked anything. It declared no permissions block,
so it inherited the workflow's contents:read and the code-scanning alert API
answered 403. gh writes the API error BODY to stdout, so
ALERTS=$(gh api '.../code-scanning/alerts?state=open' --jq 'length' 2>/dev/null || echo "0")
left ALERTS holding
{"message":"Resource not accessible by integration",...,"status":"403"}0
every [ ... -gt ... ] then failed as a non-integer comparison, the if took its
false branch, and the step printed "CodeQL gate passed (0 alerts)" and exited 0.
Observed in the log of a GREEN run (34700514102, job 103571322660) while the
repository had an open high-severity alert. The `|| echo "0"` reads as defensive
and is the exact opposite: it converts "I cannot see the alerts" into "there are
no alerts", which is the one direction a security gate must never fail in.
Grant the job security-events:read (and actions:read, which the CodeQL-run poll
needs for the same reason), and replace the fallback with a helper that treats a
non-zero gh exit or a non-numeric body as a hard failure. The helper returns
rather than exits because it runs inside a command substitution, where an exit
would end only the subshell and let the caller continue with an empty count --
the same silent pass in a new costume; both call sites check the status.
tests/test_security_gate_fail_closed.sh pins both halves and fails with six
assertions against the previous workflow.
Also bumps graph-ui's vitest to ^4.1.11 (GHSA-82fw-gwwq-j7x9, path traversal via
@vitest/mocker, CVSS 5.9). Development scope, and graph-ui ships in no release
artifact, so nothing released was ever exposed -- but it was the sole open alert
and the gate above is worthless while it stands unresolved. Raising the floor
past the vulnerable range also stops a fresh install resolving back into it.
graph-ui: npm ci clean, 47 tests in 12 files pass on 4.1.11.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
apt.llvm.org rotates its per-major noble snapshot channels; the
llvm-toolchain-noble-22 channel transiently emptied, breaking every lane that
installs clang from it. #2124 already moved Dockerfile.msan (clang-22 ->
clang-21) for the same reason. The two GitHub-hosted lanes that install from
apt.llvm.org were still on 22 and would break identically on the next
rotation: _test.yml test-diag (ASan+UBSan) and _lint.yml lint-mem
(clang-analyzer, clang-tidy). Repin both to clang-21 / clang-tidy-21 /
llvm-toolchain-noble-21 (the channel #2124 verified apt.llvm.org serves on
amd64+arm64). The macOS LSan lane's Homebrew llvm@22 pin is intentional and
unrelated to apt churn (left unchanged); the clang-format-20 CI-lint lane is
untouched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Re-stacks the CodeAtlasWeb branch on the updated base (feat/atlas-r1 at
a44732b6, which merges main 7b0f553c). Ten conflicts, all in the seam
between the previous frontend and its replacement:
- README.md: main's surface count (45) with this branch's Atlas line.
- graph-ui/package-lock.json, graph-ui/src/App.tsx: CodeAtlasWeb's own
files stay; the previous frontend's edits to the same paths do not
apply.
- Seven previous-frontend files main had modified (GraphTab,
NodeDetailPanel and its test, StatsTab test, useProjects, rationale
and its test) stay deleted, and two tests main added for that
frontend (App.test.tsx, useProjects.test.tsx) are not brought in;
their subjects no longer exist in this tree.
Nothing under graph-ui changes relative to this branch; the C side
takes main's lean-output work and the branch's merge resolution.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
tests/test_venue_parity_contract.sh holds every venue to one harness: a
workflow step may provision, move artifacts, or call a canonical leg
entry, and the product-exercising commands live inside scripts/. The
test-ui job ran npm inline and tripped it on the local ladder.
scripts/ci/test-ui.sh is now the whole leg (install without a browser
download, vitest, the style and promise scans, the portable frozen
acceptance checks, the production build); the job calls it, and running
the same script locally is the same leg. Contract green again
(22 workflows marker-checked, 10 walked).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- .github/workflows/_test.yml: a `test-ui` job (ubuntu, node 22) runs
the frontend's vitest suite, its hardcoded-string and promise scans,
the portable frozen acceptance checks and a production build, with
the same browser-download skip the product build uses. The browser
proofs (tools/smoke-*.mjs) need a running server and Playwright
browsers and stay a local, manually dispatched venue.
- graph-ui/package.json: `test:acceptance` runs every frozen check except
the two release-binding files, which tie the recorded release report
to the commits of the frontend's original repository and cannot pass
anywhere else (203 checks pass here; the full `npm test` fails exactly
those four assertions).
- README.md: the Atlas section describes CodeAtlasWeb, credits
Bernhard Jackiewicz and points at #1964.
- graph-ui/README.md: a maintainers' preface on how this repository
builds, embeds, proxies and gates the frontend, and what it added;
his README follows unchanged.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The SHA dependabot pinned (efb35369) is upstream's v3.0.3 tag, a major bump
from the v2 line: 3.0.0 moves the action runtime from Node 20 to Node 24 and
changes nothing else. The pin comment still read v2, which would misdirect the
next reader of this file; label it with the version the SHA actually is.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The test-lsan-macos leg installs the unversioned `llvm` formula, which
tracks Homebrew's current stable. That moved from 22.1.8 to 23.1.0 today,
so the leg's compiler now depends on which bottle the runner image draws:
run 33745916176 (PR #1811) poured 23.1.0 and died at compile on a new
Clang 23 diagnostic (-Wunused-but-set-global under -Werror); the run an
hour earlier poured 22.1.8 and built fine. Same code, two verdicts.
Pin the formula (and the `brew --prefix` lookup) to llvm@22 — the version
every green run of this leg has used so far. Moving to a newer major
becomes a deliberate edit of these two lines instead of an accident of
the runner image.
The diagnostic itself is fixed independently in #2027 so main compiles
under both majors; this change is about determinism of the leg, not the
warning.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The gate waited 90 x 30s = 45 min for CodeQL to finish on the PR head. That
is shorter than CodeQL actually takes on this repository, so the gate has
been failing runs that had not failed.
Measured on PR #1426, head 7b72652a: the CodeQL SAST workflow completed with
conclusion=success at 17:46:05, having started at 15:41:44 -- 124 minutes.
The gate step ran 16:52:58 to 17:38:44 and reported "BLOCKED: CodeQL timeout"
7 minutes and 21 seconds before the scan it was waiting for succeeded.
Two open contributor pull requests are red from exactly this: #1426 and
#1769, both with CodeQL completed=success on their head and every other
check green.
Three further PRs (#1703, #1741, #1742) are also red on codeql-gate alone,
but from a different cause: the CodeQL run on their head is
completed=cancelled, so the gate saw a non-success conclusion and correctly
exited 1 without waiting. This change does not help those and is not
intended to; they need a fresh scan, most likely having been superseded by
concurrency cancel-in-progress in codeql.yml.
300 x 30s = 150 min covers the measured 124 min with margin. The job already
declares timeout-minutes: 240, so the wait still cannot outlive its own job.
No trigger, permission or gating change: codeql-gate blocks exactly what it
blocked before, and a genuine CodeQL failure still exits 1 immediately rather
than waiting out the budget. Only the absence of a verdict waits longer.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Dependabot split this bump across two PRs -- #1898 bumps init, #1900 bumps
analyze -- but codeql-action requires every step in a workflow to run the
same version. Either PR alone produces a mismatch and analyze fails with:
Loaded a configuration file for version '4.37.4', but running version '4.37.9'
so neither can be green on its own, and merging either would leave main's
CodeQL job broken for every subsequent PR. Bump both pins in one commit.
Pin verified: cdf488f595d80d6e07e03d4674febd5ab45fa938 is the commit that
the annotated tag v4.37.9 dereferences to in github/codeql-action.
Supersedes #1898 and #1900.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel