## What changed
- Added tag-driven release flow in GitLab CI:
- new `release` stage with tag-only jobs
- `publish_tag_artifacts` publishes Docker + Python artifacts on `vX.Y.Z` tags
- `create_release_notes` generates release notes from conventional commits via `git-cliff` and creates a GitLab release via `glab`
- Updated main-branch image publishing to version-aware tagging:
- `publish_ecr_images` now runs `mise run publish:main`
- main publishes `:dev`, `:latest`, and a versioned dev tag
- Added release-oriented mise tasks:
- `publish:main`
- `publish:tag`
- `python:publish:macos` (manual macOS arm64 wheel publish)
- Switched Linux Python wheel builds to buildx:
- added `deploy/docker/Dockerfile.python-wheels`
- replaced old per-arch docker-run tasks with `python:build:multiarch`
- Added macOS arm64 wheel build path for local publishing:
- `python:build:macos` builds `aarch64-apple-darwin`
- intended to run locally on macOS after tag CI finishes
- Made Docker multiarch publish script tag-flexible:
- `TAG_LATEST` is no longer hardcoded in ECR mode
- supports `EXTRA_DOCKER_TAGS`
- applies extra tags to sandbox/server/pki-job/cluster images
- Moved release tooling to `build/scripts/release.py` and updated all mise references
- Removed obsolete Docker Artifactory env vars from `mise.toml`
## Release behavior
- **Main branch CI**
- Docker: `:dev`, `:latest`, and versioned dev tag
- **Tag CI (`vX.Y.Z`)**
- Docker: `:X.Y.Z` only (no `:latest`)
- Python: Linux wheels published from CI
- GitLab release notes created from conventional commits
- **Manual macOS step (after tagging)**
1. Checkout the tag locally on macOS
2. Run `mise run python:publish:macos`
## Validation
- `mise run python:lint`
- `mise run version:print`
- `mise run python:build:macos`
- `uv run python build/scripts/release.py --help`
Closes#13
## Summary
- add Python sandbox execution APIs for command and callable workflows
- consolidate sandbox policy fixtures and expand e2e test coverage for policy and Python exec paths
- update CI/build config and images for sandbox e2e execution dependencies
## Test Plan
- mise run pre-commit
Closes#26
## Summary
Consolidates sandbox policy data into `dev-sandbox-policy.yaml` and deletes `dev-sandbox-policy-data.rego`. The YAML file becomes the single source of truth for sandbox policy data across both paths:
- **`nav create sandbox`** (CLI reads YAML → proto → gRPC → server stores → sandbox fetches)
- **`mise run sandbox`** (sandbox reads YAML directly via regorus)
## Why YAML
The policy data file is user-facing — it's the interface users interact with to configure sandbox security policy (allowed endpoints, binary identities, filesystem access, etc.).
Keeping this as YAML rather than rego:
- **Decouples data from enforcement engine** — if we change from OPA/rego to a different policy engine, the user's data file stays the same
- **Still works with regorus** — regorus can natively load YAML data via its `yaml` feature, so there's no conversion step in the local dev path
- **Enables future indirection** — we can add a transformation layer between the YAML file and what the policy engine receives, without changing the user-facing format
## Changes
### Policy data (`dev-sandbox-policy.yaml`)
- Flat top-level structure: `filesystem_policy`, `landlock`, `process`, `inference`, `network_policies`
- Renamed `filesystem` → `filesystem_policy` to match rego/proto convention
- Deleted `dev-sandbox-policy-data.rego` (was the rego equivalent of this data)
### Rego rules (`dev-sandbox-policy.rego`)
- All data references updated: `data.sandbox.*` → `data.*` (dropped `sandbox.` prefix)
- `package navigator.sandbox` unchanged (that's the rego rule namespace, not the data path)
### Sandbox entrypoint (`navigator-sandbox`)
- `--rego-policy`/`--rego-data` → `--policy-rules`/`--policy-data`
- `NAVIGATOR_REGO_POLICY`/`NAVIGATOR_REGO_DATA` → `NAVIGATOR_POLICY_RULES`/`NAVIGATOR_POLICY_DATA`
- `from_files()`: loads YAML natively via `regorus::Value::from_yaml_file()`
- `from_strings()` and `reload()`: accept YAML data via `Value::from_yaml_str()`
- `proto_to_opa_data_json()`: emits data at root level (no `"sandbox"` wrapper)
- Enabled regorus `yaml` feature
### CLI (`navigator-cli`)
- `DevSandboxPolicyFile.filesystem` → `filesystem_policy`
### Tests
- All inline test data converted from rego/JSON to YAML
- `test_engine()` uses `from_strings(rules, yaml)` with the dev YAML file
### Documentation
- Updated `architecture/plans/sandbox-policy-refactor/plan.md`, `architecture/plans/opa/plan.md`, `architecture/sandbox.md`, `architecture/containers.md`
Closes#24
## Problem
SSH sessions into the sandbox were **not entering the network namespace** or receiving proxy environment variables. This meant every command run via SSH (the only user-facing path) had unrestricted internet access, completely bypassing OPA network policy enforcement.
The root cause: the SSH server was started **before** the network namespace and proxy were created in `lib.rs`, so it never received the netns fd or proxy URL.
Additionally, **gRPC inference from within the sandbox did not work at all** — even after fixing the netns, multiple issues prevented the Python SDK from reaching the navigator server through the CONNECT proxy.
## Changes
### Sandbox binary
**Core fix — reorder startup + thread netns through SSH:**
- `lib.rs`: Move netns + proxy creation before SSH server start. Compute `ssh_netns_fd` and `ssh_proxy_url`, pass them to `run_ssh_server()`.
- `ssh.rs`: Thread `netns_fd` and `proxy_url` through the full SSH call chain into `spawn_pty_shell()`. Set proxy env vars on the shell command. Call `setns(fd, CLONE_NEWNET)` in `install_pre_exec()`.
**Proxy — control plane allowlist + IPv6 socket lookup:**
- `proxy.rs`: Connections to the navigator endpoint (derived from `NAVIGATOR_ENDPOINT`) are always allowed without OPA evaluation, logged with `engine=control_plane`. This is infrastructure the sandbox needs to function, not a user-configurable policy.
- `procfs.rs`: Extended `parse_proc_net_tcp` to also check `/proc/<pid>/net/tcp6`. gRPC C-core uses `AF_INET6` sockets even for IPv4 connections, so its TCP entries were invisible to the proxy's identity resolver. Also fixed port parsing to use `rsplit_once(':')` for correct IPv6 address handling.
**Proxy env vars — lowercase variants for gRPC C-core:**
- `process.rs` + `ssh.rs`: Added lowercase `http_proxy`, `https_proxy`, `grpc_proxy` alongside uppercase. gRPC C-core (libgrpc) checks lowercase first and was ignoring the uppercase-only vars.
### Server
- `sandbox/mod.rs`: Added `CAP_SYS_PTRACE` to sandbox pod security context. Required for the proxy (root) to read `/proc/<pid>/fd/` of sandbox-user processes for binary identity resolution.
### Python SDK
- `inference.py`: Strip `http://`/`https://` scheme from endpoint before passing to `grpc.insecure_channel()`, which expects `host:port`. Default `endpoint` and `sandbox_id` from `NAVIGATOR_ENDPOINT` / `NAVIGATOR_SANDBOX_ID` env vars so `Inference()` works inside sandboxes with no arguments.
### Build / infra
- `ci.toml`: Added `--cap-add=SYS_PTRACE` to `mise run sandbox` to mirror the k8s pod capabilities.
### Documentation
- `architecture/sandbox.md`: Documented `CAP_SYS_PTRACE` requirement and the full set of proxy env vars (uppercase + lowercase).
## Testing
All 29 sandbox unit tests pass. `mise run pre-commit` passes (fmt, clippy, all workspace tests, python lint).
E2E verified on a live cluster:
| Test | Result |
|------|--------|
| Proxy env vars (6 vars, upper+lowercase) | PASS |
| Blocked endpoints (google, anthropic via curl) | PASS — all denied |
| **gRPC inference from SSH session** (`Inference()` with no args, env var defaults) | **PASS** |
| Proxy log: navigator requests show `engine=control_plane` | PASS |
| Proxy log: blocked requests show `engine=opa` with correct deny reasons | PASS |
## Summary
- Add a "push" path for local development that exports Docker images from the host daemon and imports them directly into the k3s containerd runtime, eliminating the need for a local registry or remote image pulls during `mise run cluster`
- Add `mise run cluster:deploy` task for fast iterative rebuilds (~25s) against a running cluster without recreating it
- Update `cluster-entrypoint.sh` to patch HelmChart manifest with local image tags and `IfNotPresent` pull policy when push mode is active
## Details
Previously, after the local registry (k3d + skaffold) was removed, `mise run cluster` only pulled component images from the remote CloudFront registry. This meant local code changes required a remote publish cycle to test in-cluster.
This MR adds a push path that sits alongside the existing pull path:
- **`push.rs`** — Uses bollard to `export_images()` all 3 component images into a single tar (with shared layer dedup), uploads via Docker `put_archive` API into the cluster container, then runs `ctr images import` to load them into containerd
- **`docker.rs`** — Passes `IMAGE_TAG` and `IMAGE_PULL_POLICY` env vars into the cluster container when `NAVIGATOR_PUSH_IMAGES` is set
- **`lib.rs`** — Wires push into the deploy flow between `clean_stale_nodes()` and `wait_for_cluster_ready()`, guarded by `remote_opts.is_none() && NAVIGATOR_PUSH_IMAGES`
- **`cluster-entrypoint.sh`** — Patches the baked-in HelmChart manifest with sed to use the local image tag and pull policy before k3s starts
- **`cluster.toml`** — Updated `[cluster]` task to tag and push images; new `[cluster:deploy]` task for iterative deploys via `docker save | ctr import` + `helm upgrade`
The remote deploy path (`nav cluster admin deploy --remote`) is completely unaffected — the push logic only activates for local deploys with the env var set.
## Test Plan
- `mise run cluster` — end-to-end success, cluster healthy with local images (~168s)
- `mise run cluster:deploy` — iterative deploy success, helm upgrade + rollout (~43s)
- `mise run pre-commit` — all checks pass (fmt, clippy, 82 tests, python lint)
- Verified images present in containerd: `ctr -a /run/k3s/containerd/containerd.sock images ls | grep navigator/`
- Remote path verified safe via code analysis (guards prevent push logic from running)
The previous approach wrote the host.docker.internal IP to the k3s
resolv.conf, assuming it would serve DNS. On Docker Desktop, this IP
is reachable but doesn't run a DNS server, causing CoreDNS to fail
resolving external names and breaking the PKI job and Helm install.
The new approach discovers Docker's embedded DNS listener ports from
the DOCKER_OUTPUT iptables chain, then sets up DNAT rules to forward
DNS traffic from k3s pods (via PREROUTING) to Docker's actual DNS
resolver on 127.0.0.11. The container's eth0 IP is used as the
nameserver in the k3s resolv.conf since it's routable from all pods.
In proxy mode, processes could bypass HTTP_PROXY env vars and connect
directly to any IP since seccomp cannot filter connect() by destination.
This adds network namespace isolation:
- Create isolated netns with veth pair (10.200.0.1 <-> 10.200.0.2)
- Proxy binds to host side of veth
- Sandboxed process enters netns via setns() in pre_exec
- Only network path out is through the proxy
Also adds CAP_SYS_ADMIN and CAP_NET_ADMIN to container configs and
installs iproute2 for namespace management.
The sandbox now automatically creates directories listed in the policy's
`read_write` section and sets ownership to the configured sandbox user/group.
This runs in the supervisor process before forking the child.
Previously, read_write directories had to be pre-created in the Dockerfile
with correct ownership. Now arbitrary paths can be added to the policy
and they will be prepared at runtime.
Also adds a hardcoded chown for /sandbox in Dockerfile.factory as a
belt-and-suspenders approach for the default case.
Drew Newberry