* feat(policy): add policy recommendation plumbing — denial aggregation, transport, approval pipeline, and mechanistic recommendations
Implement the infrastructure layer for automated policy recommendations (#204):
- Proto: 9 new RPCs and messages for draft policy lifecycle (submit, get, approve, reject, approve-all, edit, undo, clear, history)
- Persistence: SQLite/Postgres migrations and store methods for draft_policy_chunks and denial_summaries tables
- Server: Full gRPC handler implementations with mechanistic mapper that auto-generates NetworkPolicyRule proposals from denial summaries
- Sandbox: DenialAggregator with MPSC channel, deduplication, periodic flush to gateway via SubmitPolicyAnalysis
- CLI: 'openshell draft' subcommand with get/approve/reject/approve-all/undo/clear/history operations
- TUI: Draft recommendations panel accessible from sandbox policy view
- Docs: Architecture documentation in architecture/policy-advisor.md
* feat(policy): add L7-aware mechanistic mapper and policy advisor CTF example
Add L7 rule generation to mechanistic mapper (build_l7_rules,
generalise_path, looks_like_id) with 3 new unit tests. Add
examples/policy-advisor/ with a 7-gate CTF script, restrictive
sandbox policy, and walkthrough README.
* fix(policy): use sandbox name for denial flush and add TUI draft badges
Fix denial aggregator passing sandbox UUID instead of name to
SubmitPolicyAnalysis, which caused 'sandbox not found' errors on
flush. Add notification badges to the TUI sandbox list and detail
header showing pending draft recommendation counts.
* fix(policy): deduplicate draft chunks and tolerate overlapping OPA rules
Skip draft chunk creation when a pending/approved chunk already covers
the same host:port endpoint, preventing duplicate rules across denial
aggregator flush cycles.
Rewrite three OPA complete rules (network_policy_for_request,
matched_network_policy, matched_endpoint_config) to tolerate multiple
matching policies without triggering a "complete rule conflict" error.
network_policy_for_request becomes a boolean, matched_network_policy
uses a set comprehension with min(), and matched_endpoint_config uses
an array comprehension with index-0 selection.
* feat(tui): interactive draft actions, highlight bar, and detail popup
Rework the draft recommendations panel to match the logs UX:
- Highlight bar (green accent + background) instead of arrow marker
- Viewport-aware j/k scrolling with g/G for top/bottom
- Enter opens a full-screen detail popup showing endpoints, binaries,
rationale, security notes, and action hints
Add approve/reject/approve-all draft actions:
- [a] approve selected chunk, [x] reject, [A] approve all pending
- Actions work from both the list view and the detail popup
- gRPC calls run async; result updates status bar and refreshes data
- Nav bar shows all available keybindings
Fix draft count refresh: sandbox_draft_counts now refreshes on every
tick (not just Dashboard), so the detail header badge updates in
real time.
Improve badge labels: show 'N pending' instead of a bare number in
both the dashboard sandbox list and sandbox detail header.
* refactor(policy): DB-level draft chunk dedup with hit counter and timestamps
Replace the in-memory HashSet dedup in SubmitPolicyAnalysis with a
database-level upsert. New denormalized columns on draft_policy_chunks:
- host, port: extracted from proposed_rule at insert time
- hit_count: incremented on conflict (same sandbox + host + port)
- first_seen_ms, last_seen_ms: track when the endpoint was first and
most recently proposed
A partial unique index (WHERE status IN ('pending','approved')) ensures
only one active chunk per endpoint per sandbox; rejected/superseded
chunks don't block new proposals.
Surface hit_count and first/last_seen in:
- CLI: 'openshell draft get' shows 'Hits: N (first ..., last ...)'
- TUI: detail popup shows hits row; list view shows 'Nx' suffix
* fix(policy): optimistic retry on policy version conflicts + structured logging
merge_chunk_into_policy and remove_chunk_from_policy now retry up to 5
times on UNIQUE constraint violations (version conflicts from concurrent
approvals). Each attempt re-reads the latest policy, re-merges the rule,
and increments the version. This eliminates the race condition where
rapid successive approvals would fail with a DB error.
Add structured tracing to all draft action handlers:
- ApproveDraftChunk: logs rule_name, host, port, hit_count before merge
and version + policy_hash after success
- RejectDraftChunk: logs rule_name, host, port, reason
- ApproveAllDraftChunks: logs pending_count at start, per-chunk merge
progress, and final summary with chunks_approved/skipped
- UndoDraftChunk: logs before/after with rule_name and version
- Retry attempts log as warnings with attempt number and conflicting
version
* wip: forward proxy fix, mapper allowed_ips, TUI polish, CTF rewrite
* fix(tui): use correct --gateway flag for ssh-proxy ProxyCommand
* chore: add Docker cleanup script for stale images, volumes, and build cache
* feat(tui): approve-all confirmation modal and CTF cleanup
Add [A] confirmation popup that snapshots pending chunks, shows a
scrollable list, and approves each chunk individually on confirm.
This prevents approving chunks that arrived after the modal opened.
Remove transient issue #205 reference from CTF victory banner.
* fix(tui): correct import ordering for rustfmt
* wip: stateful toggle model, rename to network rules
Draft chunks now follow a toggle state machine:
pending -> approved | rejected (initial decision)
approved <-> rejected (toggle)
One row per (sandbox_id, host, port) via expanded unique index.
Rejecting an approved rule removes it from the active policy.
Re-approving a rejected rule merges it back.
Rename CLI from 'draft' to 'rule', TUI from 'Draft Recommendations'
to 'Network Rules'. State-aware keybindings: approved shows [x] Revoke,
rejected shows [a] Approve. Fix sandbox detail hiding delete confirmation
behind pending message.
* refactor(policy): move mapper sandbox-side, slim schema, per-binary granularity
Move mechanistic mapper from gateway to sandbox so all analysis runs
sandbox-side (N sandboxes = N independent pipelines). Gateway is now a
thin validate + persist + approval layer.
Architectural changes:
- Move mechanistic_mapper.rs from navigator-server to navigator-sandbox
- Sandbox flush flow: aggregator drains -> mapper runs -> proposals sent
- Gateway SubmitPolicyAnalysis: validate + persist only, no mapper
- Drop denial_summaries table (write-only, zero readers)
- Consolidate migrations 003+004+005 into single 003
Schema slimming:
- Drop 5 unused columns from draft_policy_chunks (stage, denial_refs,
supersedes_chunk_id, analysis_mode, decided_by)
- Add per-binary granularity: binary column, widen unique index to
(sandbox_id, host, port, binary)
- Mapper groups by (host, port, binary), one proposal per triple
- Merge appends binary to existing rule; revoke removes just that binary
CTF & UX:
- 7-gate CTF: add Gate 3 (curl -> ifconfig.me:80) for per-binary demo
- TUI shows binary short name in list, full path in detail popup
- CLI output shows binary field
- Idempotent rule names, hit_count accumulates real denial counts
- Rationale text no longer bakes in stale denial count
* refactor(docker): rename server image to gateway
Rename Dockerfile.server to Dockerfile.gateway and update all image
references from openshell/server to openshell/gateway across Helm
charts, Kubernetes manifests, mise tasks, build/deploy scripts, CI
workflows, and documentation.
The underlying Rust binary (navigator-server) is unchanged -- this
rename only affects the Docker image name and Dockerfile.
* fix: catch remaining server->gateway references in docs and comments
* feat(sbom): add SBOM generation, license resolution, and CSV export tooling
Add mise-integrated SBOM pipeline for container images using Syft.
Includes license resolution via crates.io/npm/PyPI APIs and CycloneDX
JSON to CSV conversion. Adds agent skill for on-demand SBOM operations.
Closes#237
* fix(sbom): chain task dependencies to run generate → resolve → csv sequentially
* fix(sbom): add concurrent license resolution, progress logging, and exclude dev artifacts
* feat(notices): add mise run notices to generate THIRD-PARTY-NOTICES with full license texts
Use cargo-about for Rust crate licenses and pip-licenses for Python
packages. Produces a single attribution file with per-package copyright
notices and full license text for open-source compliance.
## What changed
- Added tag-driven release flow in GitLab CI:
- new `release` stage with tag-only jobs
- `publish_tag_artifacts` publishes Docker + Python artifacts on `vX.Y.Z` tags
- `create_release_notes` generates release notes from conventional commits via `git-cliff` and creates a GitLab release via `glab`
- Updated main-branch image publishing to version-aware tagging:
- `publish_ecr_images` now runs `mise run publish:main`
- main publishes `:dev`, `:latest`, and a versioned dev tag
- Added release-oriented mise tasks:
- `publish:main`
- `publish:tag`
- `python:publish:macos` (manual macOS arm64 wheel publish)
- Switched Linux Python wheel builds to buildx:
- added `deploy/docker/Dockerfile.python-wheels`
- replaced old per-arch docker-run tasks with `python:build:multiarch`
- Added macOS arm64 wheel build path for local publishing:
- `python:build:macos` builds `aarch64-apple-darwin`
- intended to run locally on macOS after tag CI finishes
- Made Docker multiarch publish script tag-flexible:
- `TAG_LATEST` is no longer hardcoded in ECR mode
- supports `EXTRA_DOCKER_TAGS`
- applies extra tags to sandbox/server/pki-job/cluster images
- Moved release tooling to `build/scripts/release.py` and updated all mise references
- Removed obsolete Docker Artifactory env vars from `mise.toml`
## Release behavior
- **Main branch CI**
- Docker: `:dev`, `:latest`, and versioned dev tag
- **Tag CI (`vX.Y.Z`)**
- Docker: `:X.Y.Z` only (no `:latest`)
- Python: Linux wheels published from CI
- GitLab release notes created from conventional commits
- **Manual macOS step (after tagging)**
1. Checkout the tag locally on macOS
2. Run `mise run python:publish:macos`
## Validation
- `mise run python:lint`
- `mise run version:print`
- `mise run python:build:macos`
- `uv run python build/scripts/release.py --help`
Drew Newberry