Files
OpenShell/deploy/docker/Dockerfile.images
T
John T. Myers b7779bdefa feat(sandbox): integrate OCSF structured logging for sandbox events (#720)
* feat(sandbox): integrate OCSF structured logging for all sandbox events

WIP: Replace ad-hoc tracing calls with OCSF event builders across all
sandbox subsystems (network, SSH, process, filesystem, config, lifecycle).

- Register ocsf_logging_enabled setting (defaults false)
- Replace stdout/file fmt layers with OcsfShorthandLayer
- Add conditional OcsfJsonlLayer for /var/log/openshell-ocsf.log
- Update LogPushLayer to extract OCSF shorthand for gRPC push
- Migrate ~106 log sites to OCSF builders (NetworkActivity, HttpActivity,
  SshActivity, ProcessActivity, DetectionFinding, ConfigStateChange,
  AppLifecycle)
- Add openshell-ocsf to all Docker build contexts

* fix(scripts): attach provider to all smoke test phases to avoid rate limits

GitHub's unauthenticated API rate limit (60/hour) causes flaky 403s for
Phases 1, 2, and 4. Fix by attaching the provider to all sandboxes and
upgrading the Phase 1 policy to L7 so credential injection works.

Phase 4 (tls:skip) cannot inject credentials by design, so relax the
assertion to accept either 200 or 403 from upstream -- both prove the
proxy forwarded the request.

* fix(ocsf): remove timestamp from shorthand format to avoid double-timestamp

The display layer (gateway logs, TUI, sandbox logs CLI) already prepends
a timestamp. Having one in the shorthand output too produces redundant
double-timestamps like:

  15:49:11 sandbox INFO  15:49:11.649 I NET:OPEN ALLOWED ...

Now the shorthand is just the severity + structured content:

  15:49:11 sandbox INFO  I NET:OPEN ALLOWED ...

* refactor(ocsf): replace single-char severity with bracketed labels

Replace cryptic single-character severity codes (I/L/M/H/C/F) with
readable bracketed labels: [LOW], [MED], [HIGH], [CRIT], [FATAL].

Informational severity (the happy-path default) is omitted entirely to
keep normal log output clean and avoid redundancy with the tracing-level
INFO that the display layer already provides.

Before: sandbox INFO  I NET:OPEN ALLOWED ...
After:  sandbox INFO  NET:OPEN ALLOWED ...

Before: sandbox INFO  M NET:OPEN DENIED ...
After:  sandbox INFO  [MED] NET:OPEN DENIED ...

* feat(sandbox): use OCSF level label for structured events in log push

Set the level field to 'OCSF' instead of 'INFO' for OCSF events in the
gRPC log push. This visually distinguishes structured OCSF events from
plain tracing output in the TUI and CLI sandbox logs:

  sandbox OCSF  NET:OPEN [INFO] ALLOWED python3(42) -> api.example.com:443
  sandbox OCSF  NET:OPEN [MED] DENIED python3(42) -> blocked.com:443
  sandbox INFO  Fetching sandbox policy via gRPC

* fix(sandbox): convert new Landlock path-skip warning to OCSF

PR #677 added a warn!() for inaccessible Landlock paths in best-effort
mode. Convert to ConfigStateChangeBuilder with degraded state so it
flows through the OCSF shorthand format consistently.

* fix(sandbox): use rolling appender for OCSF JSONL file

Match the main openshell.log rotation mechanics (daily, 3 files max)
instead of a single unbounded append-only file. Prevents disk exhaustion
when ocsf_logging_enabled is left on in long-running sandboxes.

* fix(sandbox): address reviewer warnings for OCSF integration

W1: Remove redundant 'OCSF' prefix from shorthand file layer — the
    class name (NET:OPEN, HTTP:GET) already identifies structured events
    and the LogPushLayer separately sets the level field.

W2: Log a debug message when OCSF_CTX.set() is called a second time
    instead of silently discarding via let _.

W3: Document the boundary between OCSF-migrated events and intentionally
    plain tracing calls (DEBUG/TRACE, transient, internal plumbing).

W4: Migrate remaining iptables LOG rule failure warnings in netns.rs
    (IPv4 TCP/UDP, IPv6 TCP/UDP) to ConfigStateChangeBuilder for
    consistency with the IPv4 bypass rule failure already migrated.

W5: Migrate malformed inference request warn to NetworkActivity with
    ActivityId::Refuse and SeverityId::Medium.

W6: Use Medium severity for L7 deny decisions (both CONNECT tunnel and
    FORWARD proxy paths) to match the CONNECT deny severity pattern.
    Allows and audits remain Informational.

* refactor(sandbox): rename ocsf_logging_enabled to ocsf_json_enabled

The shorthand logs are already OCSF-structured events. The setting
specifically controls the JSONL file export, so the name should reflect
that: ocsf_json_enabled.

* fix(ocsf): add timestamps to shorthand file layer output

The OcsfShorthandLayer writes directly to the log file with no outer
display layer to supply timestamps. Add a UTC timestamp prefix to every
line so the file output matches what tracing::fmt used to provide.

Before: CONFIG:VALIDATED [INFO] Validated 'sandbox' user exists in image
After:  2026-04-01T15:49:11.649Z CONFIG:VALIDATED [INFO] Validated ...

* fix(docker): touch openshell-ocsf source to invalidate cargo cache

The supervisor-workspace stage touches sandbox and core sources to force
recompilation over the rust-deps dummy stubs, but openshell-ocsf was
missing. This caused the Docker cargo cache to use stale ocsf objects
from the deps stage, preventing changes to the ocsf crate (like the
timestamp fix) from appearing in the final binary.

Also adds a shorthand layer test verifying timestamp output, and drafts
the observability docs section.

* fix(ocsf): add OCSF level prefix to file layer shorthand output

Without a level prefix, OCSF events in the log file have no visual
anchor at the position where standard tracing lines show INFO/WARN.
This makes scanning the file harder since the eye has nothing consistent
to lock onto after the timestamp.

Before: 2026-04-01T04:04:13.065Z CONFIG:DISCOVERY [INFO] ...
After:  2026-04-01T04:04:13.065Z OCSF CONFIG:DISCOVERY [INFO] ...

* fix(ocsf): clean up shorthand formatting for listen and SSH events

- Fix double space in NET:LISTEN, SSH:LISTEN, and other events where
  action is empty (e.g., 'NET:LISTEN [INFO]  10.200.0.1' -> 'NET:LISTEN [INFO] 10.200.0.1')
- Add listen address to SSH:LISTEN event (was empty)
- Downgrade SSH handshake intermediate steps (reading preface, verifying)
  from OCSF events to debug!() traces. Only the final verdict
  (accepted/denied) is an OCSF event now, reducing noise from 3 events
  to 1 per SSH connection.
- Apply same spacing fix to HTTP shorthand for consistency.

* docs(observability): update examples with OCSF prefix and formatting fixes

Align doc examples with the deployed output:
- Add OCSF level prefix to all shorthand examples in the log file
- Show mixed OCSF + standard tracing in the file format section
- Update listen events (no double space, SSH includes address)
- Show one SSH:OPEN per connection instead of three
- Update grep patterns to use 'OCSF NET:' etc.

* docs(agents): add OCSF logging guidance to AGENTS.md

Add a Sandbox Logging (OCSF) section to AGENTS.md so agents have
in-context guidance for deciding whether new log emissions should use
OCSF structured logging or plain tracing. Covers event class selection,
severity guidelines, builder API usage, dual-emit pattern for security
findings, and the no-secrets rule.

Also adds openshell-ocsf to the Architecture Overview table.

* fix: remove workflow files accidentally included during rebase

These files were already merged to main in separate PRs. They got
pulled into our branch during rebase conflict resolution for the
deleted docs-preview-pr.yml file.

* docs(observability): use sandbox connect instead of raw SSH

Users access sandboxes via 'openshell sandbox connect', not direct SSH.

* fix(docs): correct settings CLI syntax in OCSF JSON export page

The settings CLI requires --key and --value named flags, not positional
arguments. Also fix the per-sandbox form: the sandbox name is a
positional argument, not a --sandbox flag.

* fix(e2e): update log assertions for OCSF shorthand format

The E2E tests asserted on the old tracing::fmt key=value format
(action=allow, l7_decision=audit, FORWARD, L7_REQUEST, always-blocked).
Update to match the new OCSF shorthand (ALLOWED/DENIED, HTTP:, NET:,
engine:ssrf, policy:).

* feat(sandbox): convert WebSocket upgrade log calls to OCSF

PR #718 added two log calls for WebSocket upgrade handling:

- 101 Switching Protocols info → NetworkActivity with Upgrade activity.
  This is a significant state change (L7 enforcement drops to raw relay).

- Unsolicited 101 without client Upgrade header → DetectionFinding with
  High severity. A non-compliant upstream sending 101 without a client
  Upgrade request could be attempting to bypass L7 inspection.
2026-04-07 13:01:13 -07:00

285 lines
13 KiB
Docker

# syntax=docker/dockerfile:1.4
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Shared OpenShell image build graph.
#
# Targets:
# gateway Final gateway image
# cluster Final cluster image
# gateway-builder Release openshell-server binary
# supervisor-builder Release openshell-sandbox binary
# supervisor-output Minimal stage exporting only the supervisor binary
# Pin by tag AND manifest-list digest to prevent silent upstream republishes
# from breaking the build. Update both when bumping k3s versions.
# To refresh: docker buildx imagetools inspect rancher/k3s:<tag> | head -3
ARG K3S_VERSION=v1.35.3-k3s1
ARG K3S_DIGEST=sha256:4607083d3cac07e1ccde7317297271d13ed5f60f35a78f33fcef84858a9f1d69
ARG K9S_VERSION=v0.50.18
ARG HELM_VERSION=v3.17.3
ARG NVIDIA_CONTAINER_TOOLKIT_VERSION=1.18.2-1
# ---------------------------------------------------------------------------
# Shared Rust build stages
# ---------------------------------------------------------------------------
FROM --platform=$BUILDPLATFORM rust:1.88-slim AS rust-builder-base
ARG TARGETARCH
ARG BUILDARCH
ARG CARGO_TARGET_CACHE_SCOPE=default
ARG SCCACHE_MEMCACHED_ENDPOINT
# CI sets this to 1 for maximum optimization; local builds leave it unset
# so cargo uses the Cargo.toml default (parallel codegen for fast linking).
ARG CARGO_CODEGEN_UNITS
RUN apt-get update && apt-get install -y --no-install-recommends \
cmake g++ make protobuf-compiler curl && rm -rf /var/lib/apt/lists/*
COPY deploy/docker/cross-build.sh /usr/local/bin/
RUN . cross-build.sh && install_cross_toolchain && install_sccache && add_rust_target
WORKDIR /build
FROM rust-builder-base AS rust-builder-skeleton
COPY Cargo.toml Cargo.lock ./
COPY crates/openshell-bootstrap/Cargo.toml crates/openshell-bootstrap/Cargo.toml
COPY crates/openshell-cli/Cargo.toml crates/openshell-cli/Cargo.toml
COPY crates/openshell-core/Cargo.toml crates/openshell-core/Cargo.toml
COPY crates/openshell-ocsf/Cargo.toml crates/openshell-ocsf/Cargo.toml
COPY crates/openshell-policy/Cargo.toml crates/openshell-policy/Cargo.toml
COPY crates/openshell-providers/Cargo.toml crates/openshell-providers/Cargo.toml
COPY crates/openshell-router/Cargo.toml crates/openshell-router/Cargo.toml
COPY crates/openshell-sandbox/Cargo.toml crates/openshell-sandbox/Cargo.toml
COPY crates/openshell-server/Cargo.toml crates/openshell-server/Cargo.toml
COPY crates/openshell-tui/Cargo.toml crates/openshell-tui/Cargo.toml
COPY crates/openshell-core/build.rs crates/openshell-core/build.rs
COPY proto/ proto/
RUN mkdir -p \
crates/openshell-bootstrap/src \
crates/openshell-cli/src \
crates/openshell-core/src \
crates/openshell-ocsf/src \
crates/openshell-policy/src \
crates/openshell-providers/src \
crates/openshell-router/src \
crates/openshell-sandbox/src \
crates/openshell-server/src \
crates/openshell-tui/src && \
touch crates/openshell-bootstrap/src/lib.rs && \
printf 'fn main() {}\n' > crates/openshell-cli/src/main.rs && \
touch crates/openshell-core/src/lib.rs && \
touch crates/openshell-ocsf/src/lib.rs && \
touch crates/openshell-policy/src/lib.rs && \
touch crates/openshell-providers/src/lib.rs && \
touch crates/openshell-router/src/lib.rs && \
touch crates/openshell-sandbox/src/lib.rs && \
printf 'fn main() {}\n' > crates/openshell-sandbox/src/main.rs && \
touch crates/openshell-server/src/lib.rs && \
printf 'fn main() {}\n' > crates/openshell-server/src/main.rs && \
touch crates/openshell-tui/src/lib.rs
FROM rust-builder-skeleton AS rust-deps
RUN --mount=type=cache,id=cargo-registry-${TARGETARCH},sharing=locked,target=/usr/local/cargo/registry \
--mount=type=cache,id=cargo-git-${TARGETARCH},sharing=locked,target=/usr/local/cargo/git \
--mount=type=cache,id=cargo-target-${TARGETARCH}-${CARGO_TARGET_CACHE_SCOPE},sharing=locked,target=/build/target \
--mount=type=cache,id=sccache-${TARGETARCH},sharing=locked,target=/tmp/sccache \
. cross-build.sh && cargo_cross_build --release -p openshell-server -p openshell-sandbox
# ---------------------------------------------------------------------------
# Per-target workspace stages
# ---------------------------------------------------------------------------
# Copy only the crates needed for each target so that a change to
# openshell-sandbox does not invalidate the gateway build and vice versa.
# The skeleton stage already has stub Cargo.toml + src/ for every crate,
# so cargo workspace resolution continues to work — we just overwrite the
# crates whose real source is needed for compilation.
FROM rust-deps AS gateway-workspace
ARG OPENSHELL_CARGO_VERSION
COPY crates/openshell-core/ crates/openshell-core/
COPY crates/openshell-policy/ crates/openshell-policy/
COPY crates/openshell-providers/ crates/openshell-providers/
COPY crates/openshell-router/ crates/openshell-router/
COPY crates/openshell-server/ crates/openshell-server/
RUN touch \
crates/openshell-core/build.rs \
crates/openshell-server/src/main.rs \
proto/*.proto && \
if [ -n "${OPENSHELL_CARGO_VERSION:-}" ]; then \
sed -i -E '/^\[workspace\.package\]/,/^\[/{s/^version[[:space:]]*=[[:space:]]*".*"/version = "'"${OPENSHELL_CARGO_VERSION}"'"/}' Cargo.toml; \
fi
FROM gateway-workspace AS gateway-builder
ARG CARGO_CODEGEN_UNITS
ARG EXTRA_CARGO_FEATURES=""
RUN --mount=type=cache,id=cargo-registry-${TARGETARCH},sharing=locked,target=/usr/local/cargo/registry \
--mount=type=cache,id=cargo-git-${TARGETARCH},sharing=locked,target=/usr/local/cargo/git \
--mount=type=cache,id=cargo-target-${TARGETARCH}-${CARGO_TARGET_CACHE_SCOPE},sharing=locked,target=/build/target \
--mount=type=cache,id=sccache-${TARGETARCH},sharing=locked,target=/tmp/sccache \
. cross-build.sh && \
cargo_cross_build --release -p openshell-server ${EXTRA_CARGO_FEATURES:+--features "$EXTRA_CARGO_FEATURES"} && \
mkdir -p /build/out && \
cp "$(cross_output_dir release)/openshell-server" /build/out/
FROM rust-deps AS supervisor-workspace
ARG OPENSHELL_CARGO_VERSION
COPY crates/openshell-core/ crates/openshell-core/
COPY crates/openshell-ocsf/ crates/openshell-ocsf/
COPY crates/openshell-policy/ crates/openshell-policy/
COPY crates/openshell-router/ crates/openshell-router/
COPY crates/openshell-sandbox/ crates/openshell-sandbox/
RUN touch \
crates/openshell-core/build.rs \
crates/openshell-ocsf/src/lib.rs \
crates/openshell-sandbox/src/main.rs \
proto/*.proto && \
if [ -n "${OPENSHELL_CARGO_VERSION:-}" ]; then \
sed -i -E '/^\[workspace\.package\]/,/^\[/{s/^version[[:space:]]*=[[:space:]]*".*"/version = "'"${OPENSHELL_CARGO_VERSION}"'"/}' Cargo.toml; \
fi
FROM supervisor-workspace AS supervisor-builder
ARG CARGO_CODEGEN_UNITS
ARG EXTRA_CARGO_FEATURES=""
RUN --mount=type=cache,id=cargo-registry-${TARGETARCH},sharing=locked,target=/usr/local/cargo/registry \
--mount=type=cache,id=cargo-git-${TARGETARCH},sharing=locked,target=/usr/local/cargo/git \
--mount=type=cache,id=cargo-target-${TARGETARCH}-${CARGO_TARGET_CACHE_SCOPE},sharing=locked,target=/build/target \
--mount=type=cache,id=sccache-${TARGETARCH},sharing=locked,target=/tmp/sccache \
. cross-build.sh && \
cargo_cross_build --release -p openshell-sandbox ${EXTRA_CARGO_FEATURES:+--features "$EXTRA_CARGO_FEATURES"} && \
mkdir -p /build/out && \
cp "$(cross_output_dir release)/openshell-sandbox" /build/out/
# Minimal extraction stage for fast-deploy: exports only the supervisor
# binary (~20-40 MB) instead of the entire build environment (~968 MB).
FROM scratch AS supervisor-output
COPY --from=supervisor-builder /build/out/openshell-sandbox /openshell-sandbox
# ---------------------------------------------------------------------------
# Final gateway image
# ---------------------------------------------------------------------------
FROM nvcr.io/nvidia/base/ubuntu:noble-20251013 AS gateway
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates && \
apt-get install -y --only-upgrade gpgv && \
rm -rf /var/lib/apt/lists/*
RUN useradd --create-home --user-group openshell
WORKDIR /app
COPY --from=gateway-builder /build/out/openshell-server /usr/local/bin/
RUN mkdir -p /build/crates/openshell-server
COPY --chmod=755 crates/openshell-server/migrations /build/crates/openshell-server/migrations
USER openshell
EXPOSE 8080
ENTRYPOINT ["openshell-server"]
CMD ["--port", "8080"]
# ---------------------------------------------------------------------------
# Cluster asset stages
# ---------------------------------------------------------------------------
FROM rancher/k3s:${K3S_VERSION}@${K3S_DIGEST} AS k3s
FROM ubuntu:24.04 AS k9s
ARG K9S_VERSION
ARG TARGETARCH
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates && \
curl -fsSL "https://github.com/derailed/k9s/releases/download/${K9S_VERSION}/k9s_Linux_${TARGETARCH}.tar.gz" \
| tar xz -C /tmp k9s && \
chmod +x /tmp/k9s && \
rm -rf /var/lib/apt/lists/*
FROM ubuntu:24.04 AS helm
ARG HELM_VERSION
ARG TARGETARCH
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates && \
curl -fsSL "https://get.helm.sh/helm-${HELM_VERSION}-linux-${TARGETARCH}.tar.gz" \
| tar xz --strip-components=1 -C /tmp "linux-${TARGETARCH}/helm" && \
chmod +x /tmp/helm && \
rm -rf /var/lib/apt/lists/*
FROM ubuntu:24.04 AS nvidia-container-toolkit
ARG NVIDIA_CONTAINER_TOOLKIT_VERSION
RUN apt-get update && apt-get install -y --no-install-recommends \
gpg curl ca-certificates && \
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && \
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
| tee /etc/apt/sources.list.d/nvidia-container-toolkit.list && \
apt-get update && \
apt-get install -y --no-install-recommends \
"nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION}" && \
rm -rf /var/lib/apt/lists/*
# ---------------------------------------------------------------------------
# Final cluster image
# ---------------------------------------------------------------------------
FROM nvcr.io/nvidia/base/ubuntu:noble-20251013 AS cluster
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
iptables \
mount \
dnsutils \
&& apt-get install -y --only-upgrade gpgv \
&& rm -rf /var/lib/apt/lists/*
COPY --from=k3s /bin/ /bin/
COPY --from=k9s /tmp/k9s /usr/local/bin/k9s
COPY --from=helm /tmp/helm /usr/local/bin/helm
COPY --from=k3s /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/k3s-ca-certificates.crt
COPY --from=k3s /usr/share/zoneinfo/ /usr/share/zoneinfo/
ENV PATH="/var/lib/rancher/k3s/data/cni:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin/aux" \
CRI_CONFIG_FILE="/var/lib/rancher/k3s/agent/etc/crictl.yaml"
COPY --from=nvidia-container-toolkit /usr/bin/nvidia-cdi-hook /usr/bin/
COPY --from=nvidia-container-toolkit /usr/bin/nvidia-container-runtime /usr/bin/
COPY --from=nvidia-container-toolkit /usr/bin/nvidia-ctk /usr/bin/
COPY --from=nvidia-container-toolkit /etc/nvidia-container-runtime /etc/nvidia-container-runtime
COPY --from=supervisor-builder /build/out/openshell-sandbox /opt/openshell/bin/openshell-sandbox
RUN mkdir -p /var/lib/rancher/k3s/server/manifests \
/var/lib/rancher/k3s/server/static/charts \
/etc/rancher/k3s \
/opt/openshell/manifests \
/opt/openshell/charts \
/opt/openshell/gpu-manifests \
/run/flannel
COPY deploy/docker/cluster-entrypoint.sh /usr/local/bin/cluster-entrypoint.sh
RUN chmod +x /usr/local/bin/cluster-entrypoint.sh
COPY deploy/docker/cluster-healthcheck.sh /usr/local/bin/cluster-healthcheck.sh
RUN chmod +x /usr/local/bin/cluster-healthcheck.sh
COPY deploy/docker/.build/charts/*.tgz /opt/openshell/charts/
COPY deploy/kube/manifests/*.yaml /opt/openshell/manifests/
COPY deploy/kube/gpu-manifests/*.yaml /opt/openshell/gpu-manifests/
ENTRYPOINT ["/usr/local/bin/cluster-entrypoint.sh"]
# Default to "server" so bare `docker run <image>` works without requiring
# the caller to pass a subcommand. The openshell CLI already passes
# ["server", "--disable=traefik", ...] as CMD; this default only affects
# manual `docker run` invocations that omit a command.
CMD ["server"]
HEALTHCHECK --interval=5s --timeout=5s --start-period=20s --retries=60 \
CMD ["/usr/local/bin/cluster-healthcheck.sh"]