mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Current maintainer status (2026-11-10, current-main refresh) Published head: `db4bdd71e7ddc1a72316445ef0d16a7d90b62951`, containing upstream main `a385ea8e60a14c1511d82b21d518421af35deab2` through a clean mechanical merge. All 13 remediation files are byte-identical to reviewed head `81ff3ed384afbd404c48fd79d6bed41e8d647a99`; no installer, action metadata, or checksum selection changed. Fresh focused Windows checks: 61 passed and 23 platform skips. Fresh WSL offline bootstrap/hook fixtures: 23 passed, with one pytest configuration warning. These local checks use fixtures and do not claim live vendor, provider, or native GUI execution. Exact-head CI, including Docker-native, and the Init, Wrap, Rust, governance, changelog, and merge-conflict workflows succeeded. Security remains FAILURE: the OpenCode npm audit reports two moderate findings through Effect advisory GHSA-chp3-j5qr-2vfr. The security workflow and affected OpenCode package manifest/lock are byte-identical to current main, establishing an inherited dependency finding. This is not an all-green result or a fresh audit of main. No workflow rerun, dependency downgrade, or advisory override was performed on this branch. Independent human review and integration remain required. No self-approval or merge was performed. Other branch publication and execution holds remain active. All older qualification below belongs to its recorded historical heads. --- ## Description Build validation jobs fetched mutable installer scripts, and the pinned commitlint action still declared a mutable Docker image tag. This pins that image digest, verifies workflow-tool archives and cached binaries, and replaces the Docker bootstrap scripts with pinned, verified sources. ## Type of Change - [x] Bug fix ## Changes Made - Preserve Rust 1.95.0, minimal/rustfmt/clippy options, Node 24, npm 12.2.1, and OpenCode 1.17.8. - Preserve Debian userland architecture detection, Rustup ARM fallback, and OpenCode x64 AVX2/baseline selection. - Use an official Node image pinned by manifest digest; retain npm/corepack trees and explicit runtime libraries. - Add offline bootstrap tests for failed downloads, corrupt bytes, unsupported targets, symlinks, and option forwarding. ## Testing - [x] Offline bootstrap tests pass (stdlib unittest, 16 cases). - [x] Ruff lint/format pass for the two new test files; shell syntax and whitespace checks pass. - [x] Exact-head hosted CI passed, including platform tests and Docker-native, Init, and Wrap build/container checks. ### Test Output ```text python3 -m unittest discover -s tests/test_install -p 'test_*bootstrap.py' ................ Ran 16 tests in 0.770s OK ``` ## Real Behavior Proof - Environment: Ubuntu 24.04 under WSL, exact candidate4ee38e8619. - Exact command / steps: Run the unittest discovery command above. Separately stream the recorded release archives and official Node image layer through SHA-256 and inspect their tar headers. - Observed result: Invalid bytes stop before extraction/execution; fixture platform and CPU cases select the intended archives. Actual workflow-tool/OpenCode archives and the Node layer match published digests. The Rustup x64 binary matches its vendor checksum; all nine embedded target checksums match vendor records. - Not tested: No live model/provider behavior or unexercised architecture is claimed. Local bootstrap fixtures use mocks; the hosted Docker/build checks below exercised the actual pinned installation paths. ## Runtime Rollout Safety - Rollout-managed feature(s): None; CI/build bootstrap only. - Minimum rollout channel: Not applicable. - Stable/default behavior changed: Build inputs become pinned; proxy defaults, Rust 1.95.0/components, npm 12.2.1 and OpenCode 1.17.8 stay unchanged. Node remains on the 24 LTS line with a fixed image digest. - Kill switch / disable path: Revert the maintenance change through review; do not bypass checksum failures. - Unsafe override required: None for this exact reviewed candidate. - Qualification impact: The exact-head hosted checks below passed. Existing PR heads retain their separate security/qualification holds until main is remediated and their own gates are refreshed. - Rollback path: Revert this maintenance PR if build compatibility regresses. ## Review Readiness - [x] I performed a self-review. - [x] This PR is ready for human review. ## Checklist - [x] New security regression tests are included. - [x] CHANGELOG.md is unchanged. ## Additional Notes Static independent code review found no defects. No merge or self-approval is requested. Completed historical CI evidence is retained within its original scope; this change qualifies future installer execution separately. Initial hosted validation ofb2592de8dstopped at rustup filename dispatch. Follow-up4ee38e861retains the required rustup-init basename in a private directory and adds the reproducing regression test. All nine latest applicable hosted workflows passed on the corrected head. ### Hosted validation on4ee38e861- [CI: platform tests and Docker-native build/install/compose/wrap/init](https://github.com/headroomlabs-ai/headroom/actions/runs/42352878789): success. - [Wrap E2E](https://github.com/headroomlabs-ai/headroom/actions/runs/42352878762), [Init E2E](https://github.com/headroomlabs-ai/headroom/actions/runs/42352878760), [Init Native E2E](https://github.com/headroomlabs-ai/headroom/actions/runs/42352878767), and [Security](https://github.com/headroomlabs-ai/headroom/actions/runs/42352878768): success. - Governance, changelog, Rust and merge-conflict checks also passed. The older cancelled governance duplicate was superseded by a successful run. Independent human review is required. No self-approval or merge has been performed. ### Local-action metadata coverage follow-up `38337d8be` adds `.github/actions/**` to both code and workflow filters. The normal pytest suite now checks the vendored commitlint metadata structure and immutable Docker digest; this is separate from actionlint workflow parsing. Six focused CI regression tests pass, including a filter regression that failed before the fix. Independent delta review found no issues. `38337d8be` now has successful latest applicable CI, Wrap E2E, Init E2E, Init Native E2E, Security, Rust, Changelog Guard, Merge Conflicts, and PR Governance runs. The successful runs above remain historical evidence for `4ee38e861`. Current-head [CI](https://github.com/headroomlabs-ai/headroom/actions/runs/42360858297), [Wrap](https://github.com/headroomlabs-ai/headroom/actions/runs/42360858271), [Init](https://github.com/headroomlabs-ai/headroom/actions/runs/42360858296), [Init Native](https://github.com/headroomlabs-ai/headroom/actions/runs/42360858178), and [Security](https://github.com/headroomlabs-ai/headroom/actions/runs/42360858352) passed. Independent human review remains required; no self-approval or merge. <!-- maintainer-current-main-qualification --> ## Current-main qualification Published head `9381378ffb07e466acef6a57eb39322aa84d4582` includes upstream main `bb8d18a56abb07ca7e811235c86ea6b626eb1f8b` and is mergeable. All nine exact-head workflows succeeded; all 53 jobs are terminal success or explicit skips, including CI Docker-native, wrap/init and native-init validation. Local metadata/release checks: 61 passed; 16 Linux-only bootstrap tests skipped on Windows. Remediation installer/action sources are unchanged by the clean main integration. Independent human review and merge are required. No self-approval or merge performed. This PR does not lift the inherited execution-provenance hold on main or other branches before human integration. <!-- /maintainer-current-main-qualification --> ### Maintainer current-main refresh (a2bcbd101) Published `a20f072a238a1f8de422adf6a25d3f9d664151e4` incorporates upstream main `a2bcbd101b1391debfd218ec8c63cb72d2bc240a` through a clean merge. All 13 previously screened CI provenance remediation files are byte-identical to prior published head `9381378ffb07e466acef6a57eb39322aa84d4582`; no installer/action/remediation logic was rewritten. Fresh focused checks: 61 passed, 23 platform-specific shell cases skipped on Windows. No vendor installer, Docker image or downloaded binary was executed locally. Earlier Linux bootstrap and hosted evidence retain their original scope. Fresh exact-head hosted validation completed successfully: all 10 observed workflow runs succeeded; all 56 checks are terminal successful or explicitly skipped. Ready for independent human review on this head/current main. This remediation branch alone is permitted to run its screened checks; inherited execution/publication holds on main and other PRs remain until independent human integration. Independent review is required; no self-approval, merge or release publication. --------- Co-authored-by: Jerrett Davis <JerrettDavis@users.noreply.github.com>
263 lines
11 KiB
Docker
263 lines
11 KiB
Docker
ARG PYTHON_VERSION=3.13
|
|
ARG UV_VERSION=0.11.18
|
|
ARG DISTROLESS_IMAGE=gcr.io/distroless/python3-debian13
|
|
ARG PYTHON_SITE_PACKAGES=/usr/local/lib/python${PYTHON_VERSION}/site-packages
|
|
|
|
# ---- Build stage: compile native extensions, build wheel ----
|
|
FROM python:${PYTHON_VERSION}-slim AS builder
|
|
|
|
ARG UV_VERSION
|
|
ARG PYTHON_SITE_PACKAGES
|
|
ARG HEADROOM_BUILD_VERSION=""
|
|
|
|
# build-essential / g++ for any C extension wheels uv may need to build
|
|
# from source. curl + ca-certificates are required by the rustup
|
|
# bootstrap below. patchelf for maturin's wheel-link repair on linux.
|
|
# No OpenSSL system deps required: the rustls-everywhere refactor
|
|
# eliminated `openssl-sys` from our build tree by switching fastembed
|
|
# to `hf-hub-rustls-tls` + `ort-download-binaries-rustls-tls`.
|
|
RUN apt-get update && \
|
|
apt-get install -y --no-install-recommends \
|
|
build-essential \
|
|
g++ \
|
|
curl \
|
|
ca-certificates \
|
|
patchelf \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN python -m pip install --no-cache-dir uv==${UV_VERSION}
|
|
|
|
# Rust toolchain for the headroom._core extension. With single-wheel
|
|
# architecture (post-#355), `pip install -e .` invokes maturin via
|
|
# pyproject.toml's [build-system], which calls cargo. No more separate
|
|
# headroom-core-py package.
|
|
ENV CARGO_HOME=/usr/local/cargo \
|
|
RUSTUP_HOME=/usr/local/rustup \
|
|
PATH=/usr/local/cargo/bin:${PATH}
|
|
COPY scripts/install-rustup.sh /tmp/install-rustup.sh
|
|
RUN sh /tmp/install-rustup.sh -y --no-modify-path --profile minimal -c rustfmt -c clippy --default-toolchain 1.95.0
|
|
|
|
WORKDIR /build
|
|
|
|
# Copy the full set of files maturin needs to build the wheel: the root
|
|
# pyproject.toml + Cargo workspace + Rust crates + Python source. The
|
|
# uv install builds + installs the wheel in one shot.
|
|
COPY pyproject.toml uv.lock README.md README-pypi.md ./
|
|
COPY Cargo.toml Cargo.lock rust-toolchain.toml ./
|
|
COPY crates/ crates/
|
|
COPY headroom/ headroom/
|
|
|
|
# The standalone Dockerfile must support every backend advertised by
|
|
# `headroom proxy --backend`, including Bedrock temporary/SSO credentials.
|
|
# Those credentials require botocore (GH #1551), supplied by [bedrock].
|
|
# [redis] backs the replica-shared stores: without it HEADROOM_COMPRESS_SESSION_STORE
|
|
# stops startup and HEADROOM_CCR_BACKEND=redis falls back to a per-process store.
|
|
ARG HEADROOM_EXTRAS=proxy,code,bedrock,redis
|
|
# `uv pip install` ignores [tool.uv] override-dependencies (uv.lock only), so
|
|
# pass them explicitly: the image gets the security overrides the lock has
|
|
# (pillow 13 over fastembed's <13 cap).
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
--mount=type=cache,target=/usr/local/cargo/registry \
|
|
--mount=type=cache,target=/usr/local/cargo/git \
|
|
--mount=type=cache,target=/build/target \
|
|
python -c "exec('try: import tomllib\\nexcept ImportError: from pip._vendor import tomli as tomllib'); print('\\n'.join(tomllib.load(open('pyproject.toml', 'rb'))['tool']['uv']['override-dependencies']))" > /tmp/overrides.txt && \
|
|
uv pip install --system --overrides /tmp/overrides.txt ".[${HEADROOM_EXTRAS}]"
|
|
|
|
RUN --mount=type=bind,source=.,target=/context,readonly \
|
|
HEADROOM_BUILD_VERSION="${HEADROOM_BUILD_VERSION}" PYTHON_SITE_PACKAGES="${PYTHON_SITE_PACKAGES}" python - <<'PY'
|
|
import hashlib
|
|
import os
|
|
from pathlib import Path
|
|
|
|
|
|
def git_revision(context: Path) -> str | None:
|
|
git_dir = context / ".git"
|
|
head_path = git_dir / "HEAD"
|
|
if not head_path.exists():
|
|
return None
|
|
head = head_path.read_text(encoding="utf-8").strip()
|
|
if head.startswith("ref: "):
|
|
ref_name = head.removeprefix("ref: ").strip()
|
|
ref_path = git_dir / ref_name
|
|
if ref_path.exists():
|
|
head = ref_path.read_text(encoding="utf-8").strip()
|
|
else:
|
|
packed_refs = git_dir / "packed-refs"
|
|
if not packed_refs.exists():
|
|
return None
|
|
for line in packed_refs.read_text(encoding="utf-8").splitlines():
|
|
if line.startswith("#") or not line.strip():
|
|
continue
|
|
sha, _, name = line.partition(" ")
|
|
if name.strip() == ref_name:
|
|
head = sha
|
|
break
|
|
else:
|
|
return None
|
|
return head[:12] if len(head) >= 7 and all(c in "0123456789abcdef" for c in head.lower()) else None
|
|
|
|
|
|
def source_digest(root: Path) -> str:
|
|
digest = hashlib.sha256()
|
|
inputs = (
|
|
"pyproject.toml",
|
|
"uv.lock",
|
|
"README.md",
|
|
"README-pypi.md",
|
|
"Cargo.toml",
|
|
"Cargo.lock",
|
|
"rust-toolchain.toml",
|
|
"crates",
|
|
"headroom",
|
|
)
|
|
for name in inputs:
|
|
path = root / name
|
|
if not path.exists():
|
|
continue
|
|
files = [path] if path.is_file() else sorted(p for p in path.rglob("*") if p.is_file())
|
|
for file in files:
|
|
digest.update(file.relative_to(root).as_posix().encode("utf-8"))
|
|
digest.update(b"\0")
|
|
digest.update(file.read_bytes())
|
|
digest.update(b"\0")
|
|
return digest.hexdigest()[:12]
|
|
|
|
|
|
build_version = os.environ["HEADROOM_BUILD_VERSION"].strip()
|
|
if not build_version:
|
|
print("no Headroom build version override provided; using installed package metadata")
|
|
raise SystemExit(0)
|
|
if build_version == "source-build":
|
|
revision = git_revision(Path("/context"))
|
|
build_version = (
|
|
f"source-build+g{revision}"
|
|
if revision
|
|
else f"source-build+sha256.{source_digest(Path('/build'))}"
|
|
)
|
|
|
|
package_dir = Path(os.environ["PYTHON_SITE_PACKAGES"]) / "headroom"
|
|
(package_dir / "_build_info.py").write_text(
|
|
"BUILD_VERSION = " + repr(build_version) + "\n",
|
|
encoding="utf-8",
|
|
)
|
|
print("baked Headroom build version: " + build_version)
|
|
PY
|
|
|
|
# Build-stage smoke check: verify the extension loads end-to-end inside
|
|
# the build image before we copy site-packages into the runtime image.
|
|
# If this fails, the runtime image would fail Phase A0's fail-loud
|
|
# startup check on every restart. Run from /tmp so cwd doesn't shadow
|
|
# site-packages with /build/headroom/ (which has no _core.so since
|
|
# maturin installed the .so into site-packages).
|
|
RUN cd /tmp && python -c "from headroom._core import DiffCompressor, SmartCrusher; \
|
|
print(f'build-stage rust core verify OK: {DiffCompressor.__name__}, {SmartCrusher.__name__}')"
|
|
|
|
# Build the native Rust reverse proxy binary and stage it for the runtime
|
|
# images (issue #976). These images already run "the proxy"; bundling the
|
|
# native `headroom-proxy` binary lets operators front the Python proxy with
|
|
# the Rust SigV4 / live-zone compression path from the same image. The
|
|
# binary is copied out of the cache-mounted target dir into a persistent
|
|
# path so the COPY in the runtime stages can pick it up.
|
|
RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
|
--mount=type=cache,target=/build/target \
|
|
cargo build --release --locked --bin headroom-proxy && \
|
|
cp target/release/headroom-proxy /usr/local/bin/headroom-proxy
|
|
|
|
# ---- Runtime stage (python-slim): supports root/nonroot via build arg ----
|
|
FROM python:${PYTHON_VERSION}-slim AS runtime-slim-base
|
|
|
|
ARG RUNTIME_USER=nonroot
|
|
ARG RUNTIME_HOME=/home/nonroot
|
|
ARG PYTHON_SITE_PACKAGES
|
|
|
|
RUN apt-get update && \
|
|
apt-get install -y --no-install-recommends curl && \
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY --from=builder ${PYTHON_SITE_PACKAGES} ${PYTHON_SITE_PACKAGES}
|
|
COPY --from=builder /usr/local/bin/headroom /usr/local/bin/headroom
|
|
# Native Rust reverse proxy binary (issue #976).
|
|
COPY --from=builder /usr/local/bin/headroom-proxy /usr/local/bin/headroom-proxy
|
|
|
|
RUN mkdir -p /home/nonroot /data && \
|
|
if [ "$RUNTIME_USER" = "nonroot" ]; then \
|
|
groupadd --gid 1000 nonroot && \
|
|
useradd --uid 1000 --gid nonroot --create-home nonroot && \
|
|
mkdir -p /home/nonroot/.headroom && \
|
|
chown -R nonroot:nonroot /data /home/nonroot; \
|
|
else \
|
|
mkdir -p /root/.headroom; \
|
|
fi
|
|
|
|
USER ${RUNTIME_USER}
|
|
WORKDIR ${RUNTIME_HOME}
|
|
|
|
ENV HEADROOM_HOST=127.0.0.1 \
|
|
PYTHONUNBUFFERED=1 \
|
|
PYTHONDONTWRITEBYTECODE=1
|
|
|
|
# Declare ~/.headroom as a volume so Docker (and ACA) can attach persistent
|
|
# storage here. Bare `docker run` gets an anonymous volume as a fallback so
|
|
# state is never silently written to the ephemeral container layer.
|
|
# RUNTIME_HOME defaults to /home/nonroot (the published image default); pass
|
|
# --build-arg RUNTIME_HOME=/root when building with RUNTIME_USER=root.
|
|
VOLUME ${RUNTIME_HOME}/.headroom
|
|
|
|
EXPOSE 8787
|
|
|
|
# Shell form so ${HEADROOM_PORT} expands at probe time: `headroom deploy
|
|
# --port N` sets it in the container env, and an exec-form CMD would keep
|
|
# probing 8787 and report a working deployment as unhealthy (issue #2432).
|
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
|
|
CMD curl --fail --silent "http://127.0.0.1:${HEADROOM_PORT:-8787}/readyz"
|
|
|
|
ENTRYPOINT ["headroom", "proxy"]
|
|
# Keep host and port defaults in the CLI so HEADROOM_HOST and HEADROOM_PORT
|
|
# remain authoritative for the published image.
|
|
|
|
# Distroless has no shell, so the ~/.headroom state directory cannot be made
|
|
# with RUN there. Build it here and COPY it in with the right owner (#4443).
|
|
FROM python:${PYTHON_VERSION}-slim AS runtime-slim-state
|
|
RUN mkdir -p /state/.headroom
|
|
|
|
FROM ${DISTROLESS_IMAGE} AS runtime-slim
|
|
|
|
ARG RUNTIME_USER=nonroot
|
|
ARG RUNTIME_HOME=/home/nonroot
|
|
ARG PYTHON_SITE_PACKAGES
|
|
|
|
COPY --from=builder ${PYTHON_SITE_PACKAGES} ${PYTHON_SITE_PACKAGES}
|
|
# Native Rust reverse proxy binary (issue #976).
|
|
COPY --from=builder /usr/local/bin/headroom-proxy /usr/local/bin/headroom-proxy
|
|
# Owned by the runtime user so the volume below is writable (user and group
|
|
# are both named after RUNTIME_USER in the distroless image).
|
|
COPY --from=runtime-slim-state --chown=${RUNTIME_USER}:${RUNTIME_USER} /state/ ${RUNTIME_HOME}/
|
|
|
|
USER ${RUNTIME_USER}
|
|
WORKDIR /app
|
|
|
|
ENV HEADROOM_HOST=127.0.0.1 \
|
|
PYTHONUNBUFFERED=1 \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
PYTHONPATH=${PYTHON_SITE_PACKAGES}
|
|
|
|
# Same persistent-state volume as the python-slim stage; pass
|
|
# --build-arg RUNTIME_HOME=/root when building with RUNTIME_USER=root.
|
|
VOLUME ${RUNTIME_HOME}/.headroom
|
|
|
|
EXPOSE 8787
|
|
|
|
# This stage is distroless, so there is no shell to expand ${HEADROOM_PORT}.
|
|
# Resolve the port inside Python instead, keeping exec form (issue #2432).
|
|
# `or` rather than a get() default so an empty HEADROOM_PORT falls back the
|
|
# same way `${HEADROOM_PORT:-8787}` does in the shell-form stage above.
|
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
|
|
CMD ["python3", "-c", "import os, urllib.request; urllib.request.urlopen('http://127.0.0.1:' + (os.environ.get('HEADROOM_PORT') or '8787') + '/readyz', timeout=5)"]
|
|
|
|
ENTRYPOINT ["python3", "-m", "headroom.cli", "proxy"]
|
|
# Keep host and port defaults in the CLI so HEADROOM_HOST and HEADROOM_PORT
|
|
# remain authoritative for the published image.
|
|
|
|
# Default published image remains python-slim runtime
|
|
FROM runtime-slim-base AS runtime
|