mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-09 09:11:00 +08:00
Martin Vogel
f404dfbc7a
The test-msan lane dies whenever the buildx layer cache misses and the
builder's resolver hiccups. PR-CI runs 33816056229 and 33818108315 both
failed in "Build MSan image (cached layers)" with:
#7 26.77 Could not resolve 'apt.llvm.org'
#7 27.34 W: Failed to fetch http://apt.llvm.org/noble/dists/\
llvm-toolchain-noble-22/InRelease Could not resolve 'apt.llvm.org'
ERROR: failed to build: failed to solve: process "/bin/sh -c apt-get \
update && ..." did not complete successfully
The wget of the signing key from that SAME host had succeeded seconds
earlier in the same layer, so this is a transient DNS failure inside
buildkit, not a wrong URL or a dead mirror. 2 of the last 15 PR runs.
Every network-touching step in Dockerfile.msan now runs under a small
POSIX-sh helper written into the image before the first layer that needs
it: /usr/local/bin/retry makes 5 attempts with growing backoff
(5s/10s/20s/40s) and still exits non-zero once they are exhausted, so a
genuine breakage keeps failing the build instead of being swallowed.
Wrapped: both apt-get update/install pairs (also given
-o Acquire::Retries=3), the wget of the LLVM signing key, and the
llvm-project and zlib clones. Two shapes needed care:
- the key is fetched with `wget -O <file>` instead of `-qO- > <file>`,
because a redirect is opened once for all attempts and a partial write
from a failed attempt would be prepended to a later successful one;
- each clone clears its destination first, or a half-finished clone makes
every later attempt fail on "destination path already exists".
The pinned base digest, the pinned llvmorg-22.1.0 / v1.3.1 tags,
--no-install-recommends and the layer structure are unchanged; on a
healthy network the image is byte-for-byte the same work as before.
Verified on Colima:
- `docker buildx build --check -f test-infrastructure/Dockerfile.msan
test-infrastructure/` -> "Check complete, no warnings found."
- scratch image = FROM + helper + the real apt.llvm.org layer, built
--no-cache: layer completes, clang-22/llvm-22 installed.
- scratch image where the wrapped command fails twice then succeeds:
attempts 1 and 2 logged with 5s/10s backoff, attempt 3 succeeds,
build exit 0.
- scratch image where the wrapped command always fails: 5 attempts,
"retry: giving up after 5 attempts", retry_exit=1, build exit 1.
The full MSan image was deliberately not built locally (it compiles
libc++ with MSan); CI builds it on the branch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
108 lines
5.3 KiB
Docker
108 lines
5.3 KiB
Docker
# MemorySanitizer lane image — stage-2 of the memory-diagnostics program.
|
|
#
|
|
# MSan requires EVERY linked library to be MSan-instrumented, or reads of
|
|
# memory those libraries wrote report as uninitialized. The vendored deps
|
|
# (sqlite3, lz4, zstd, tree-sitter runtime, grammars) compile in-tree and get
|
|
# instrumented for free; the two external links -lstdc++/-lz do not. This
|
|
# image provides an MSan-instrumented libc++/libc++abi/libunwind and zlib in
|
|
# /opt/msan, and scripts/msan.sh points the build at them.
|
|
#
|
|
# Sources are pinned by release tag from their canonical upstreams (same
|
|
# precedent as the cppcheck 2.20.0 build in _lint.yml).
|
|
#
|
|
# Build: docker build -f test-infrastructure/Dockerfile.msan -t cbm-msan test-infrastructure/
|
|
# Run: see scripts/msan.sh (driven via docker-compose service test-msan)
|
|
|
|
# Same pinned base as the primary test image — bump deliberately, never to a tag.
|
|
FROM ubuntu:noble@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
|
|
|
|
# WHY every network step below is wrapped: on a cache miss the builder's
|
|
# resolver hiccups and takes the whole lane down. PR-CI runs 33816056229 and
|
|
# 33818108315 both died with `Could not resolve 'apt.llvm.org'` inside the
|
|
# clang layer — seconds AFTER a wget from that same host had succeeded in the
|
|
# same layer, i.e. a transient buildkit-side DNS failure, not a wrong URL.
|
|
# retry: 5 attempts with growing backoff, and it still exits non-zero after
|
|
# the last one, so a genuine breakage keeps failing the build.
|
|
RUN printf '%s\n' \
|
|
'#!/bin/sh' \
|
|
'# retry <cmd> [args...]: 5 attempts, sleeping 5s/10s/20s/40s between them.' \
|
|
'attempt=1; delay=5' \
|
|
'until "$@"; do' \
|
|
' if [ "$attempt" -ge 5 ]; then' \
|
|
' echo "retry: giving up after $attempt attempts: $*" >&2' \
|
|
' exit 1' \
|
|
' fi' \
|
|
' echo "retry: attempt $attempt failed, sleeping ${delay}s before retrying: $*" >&2' \
|
|
' sleep "$delay"' \
|
|
' attempt=$((attempt + 1)); delay=$((delay * 2))' \
|
|
'done' \
|
|
> /usr/local/bin/retry \
|
|
&& chmod 0755 /usr/local/bin/retry
|
|
|
|
# clang 22 from apt.llvm.org, matching the diag and analyzer lanes. Noble's
|
|
# default is clang 18 — four majors behind everything else here, which is both
|
|
# an inconsistency and a bad vantage point for debugging sanitizer behaviour.
|
|
# The key is fetched with `wget -O <file>` rather than `-qO- > <file>`: under
|
|
# retry a redirect is opened once for all attempts, so a partial write from a
|
|
# failed attempt would be prepended to the output of a later successful one.
|
|
RUN retry apt-get -o Acquire::Retries=3 update \
|
|
&& retry apt-get -o Acquire::Retries=3 install -y --no-install-recommends wget gnupg ca-certificates \
|
|
&& retry wget -q -O /etc/apt/trusted.gpg.d/apt.llvm.org.asc https://apt.llvm.org/llvm-snapshot.gpg.key \
|
|
&& echo "deb http://apt.llvm.org/noble/ llvm-toolchain-noble-22 main" > /etc/apt/sources.list.d/llvm-22.list \
|
|
&& retry apt-get -o Acquire::Retries=3 update \
|
|
&& retry apt-get -o Acquire::Retries=3 install -y --no-install-recommends \
|
|
clang-22 libclang-rt-22-dev llvm-22 \
|
|
&& ln -sf /usr/bin/clang-22 /usr/bin/clang \
|
|
&& ln -sf /usr/bin/clang++-22 /usr/bin/clang++
|
|
|
|
RUN retry apt-get -o Acquire::Retries=3 update \
|
|
&& retry apt-get -o Acquire::Retries=3 install -y --no-install-recommends \
|
|
cmake \
|
|
ninja-build \
|
|
make \
|
|
python3 \
|
|
git \
|
|
curl \
|
|
zsh \
|
|
ccache \
|
|
ca-certificates \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# libc++ + libc++abi + libunwind with MemoryWithOrigins, at the SAME major as
|
|
# the compiler above — a runtimes build must match its clang. The clone clears
|
|
# its destination first: a half-finished clone would make every later attempt
|
|
# fail on "destination path already exists".
|
|
RUN retry sh -c 'rm -rf /tmp/llvm-project && git clone --depth 1 --branch llvmorg-22.1.0 https://github.com/llvm/llvm-project.git /tmp/llvm-project' \
|
|
&& cmake -G Ninja -S /tmp/llvm-project/runtimes -B /tmp/llvm-msan \
|
|
-DCMAKE_BUILD_TYPE=Release \
|
|
-DCMAKE_C_COMPILER=clang \
|
|
-DCMAKE_CXX_COMPILER=clang++ \
|
|
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" \
|
|
-DLLVM_USE_SANITIZER=MemoryWithOrigins \
|
|
-DCMAKE_INSTALL_PREFIX=/opt/msan \
|
|
-DLIBCXX_INCLUDE_TESTS=OFF \
|
|
-DLIBCXX_INCLUDE_BENCHMARKS=OFF \
|
|
-DLIBCXXABI_INCLUDE_TESTS=OFF \
|
|
-DLIBUNWIND_INCLUDE_TESTS=OFF \
|
|
&& ninja -C /tmp/llvm-msan cxx cxxabi unwind \
|
|
&& ninja -C /tmp/llvm-msan install-cxx install-cxxabi install-unwind \
|
|
&& rm -rf /tmp/llvm-project /tmp/llvm-msan
|
|
|
|
# zlib with MSan (static, so the runner needs no runtime path for it).
|
|
RUN retry sh -c 'rm -rf /tmp/zlib && git clone --depth 1 --branch v1.3.1 https://github.com/madler/zlib.git /tmp/zlib' \
|
|
&& cd /tmp/zlib \
|
|
&& CC=clang CFLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -O2" \
|
|
./configure --prefix=/opt/msan --static \
|
|
&& make -j"$(nproc)" install \
|
|
&& rm -rf /tmp/zlib
|
|
|
|
# Symbolizer path in its own (last) layer so adding tools never invalidates
|
|
# the expensive libc++ build layers above. libclang-rt-22-dev is installed
|
|
# explicitly with the compiler: it is only a Recommends of clang, which
|
|
# --no-install-recommends drops, and the link then fails to find
|
|
# libclang_rt.msan-*.a.
|
|
ENV MSAN_SYMBOLIZER_PATH=/usr/lib/llvm-22/bin/llvm-symbolizer
|
|
|
|
WORKDIR /src
|
|
ENTRYPOINT ["scripts/msan.sh"]
|