mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-13 04:51:00 +08:00
codex/mxc-https-test-socket-owner
23
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew Newberry
|
33bbda3d33 |
refactor(persistence): adopt continuation-token pagination (#3249)
* refactor(persistence): adopt continuation-token pagination Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(pagination): address continuation review findings Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(tui): recover completed list refreshes Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(pagination): address review scalability findings Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(pagination): repair branch validation Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(go): use page size in template example Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Mrunal PatelandJohn Myers
|
90dbe5454b |
feat(api): add typed workspace selectors (#3245)
* feat(api)!: add typed workspace selectors Signed-off-by: Mrunal Patel <mrunalp@gmail.com> * fix(cli): preserve template workspace metadata Signed-off-by: Mrunal Patel <mrunalp@gmail.com> * test(e2e): migrate workspace request selectors Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(api): update public schema inventory Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: Mrunal Patel <mrunalp@gmail.com> Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com> |
||
|
John T. Myers
|
f4dc6be4b2 |
refactor(inference): remove managed inference routes (#3195)
* refactor(inference): remove managed inference routes Closes #3172 Remove the inference route control plane, inference.local data path, built-in router crate, and SDK surface. Move inference workloads to explicitly imported provider profiles and native endpoints, with migration cleanup and updated tests and documentation. Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(policy): preserve alternate upstream isolation Restore the provider policy activation guard so legacy OpenAI and Anthropic providers configured for alternate base URLs do not grant egress to the built-in public vendor endpoints. Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> |
||
|
John T. MyersandJohn Myers
|
07df822090 |
feat(providers): make profiles authoritative (#2962)
* feat(providers): make profiles authoritative Closes #1988 Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * docs(providers): move profiles into provider navigation Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * docs(providers): clarify provider attachment lifecycle Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(tui): scroll provider profile picker Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(providers): honor profile credential semantics Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(providers): prefer exact profile IDs Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(providers): harden authoritative profile adoption Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * test(oidc): align provider fixtures with profiles Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(providers): preserve authoritative profile lifecycle Signed-off-by: John Myers <johntmyers@users.noreply.github.com> --------- Signed-off-by: John Myers <johntmyers@users.noreply.github.com> Co-authored-by: John Myers <johntmyers@users.noreply.github.com> |
||
|
Mrunal Patel
|
8d67250a5d |
fix(providers): keep refresh credential handles stable (#2780)
* fix(providers): keep refresh credential handles stable Signed-off-by: Mrunal Patel <mrunalp@gmail.com> * fix(providers): protect refresh-owned credentials Signed-off-by: Mrunal Patel <mrunalp@gmail.com> --------- Signed-off-by: Mrunal Patel <mrunalp@gmail.com> |
||
|
John T. MyersandJohn Myers
|
0120535efc |
feat(proxy): bind static credentials to provider endpoints (#2510)
* feat(proxy): bind static credentials to provider endpoints Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * test(e2e): verify static credential endpoint isolation Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * docs(provider): explain static credential endpoint binding Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(e2e): use valid endpoint isolation fixtures Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * docs(provider): explain static credential endpoint binding Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(credentials): preserve binding identity across rotations Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(proxy): enforce bindings across request lifecycle Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(proxy): close credential relay gaps Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(credentials): clarify binding failure behavior Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): hash selected provider profile scope Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(proxy): resolve credentials after request admission Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(credentials): clarify binding failure diagnostics Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(proxy): align single-route credential denials Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): harden endpoint-bound rotation Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): enforce identity and authority binding Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): snapshot provider environment atomically Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(e2e): include authority port in query proxy requests Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): close credential revocation gaps Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(proxy): explain authority mismatch diagnostics Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): enforce binding lifecycle invariants Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(provider): reject credential config collisions Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(network): capture credential scope atomically Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(network): distinguish origin and absolute targets Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(provider): isolate endpointless profile credentials Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(network): normalize IPv6 request authorities Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(credentials): clarify endpointless profile isolation Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * feat(policy): bind endpointless provider credentials Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): use current GCP placeholder revision Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(providers): explain policy credential bindings Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(credentials): cover endpointless fail-closed invariant Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(policy): expect ambiguity rejection at creation Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(server): authenticate rebased policy requests Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * refactor(proxy): share credential mismatch finding builder Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(credentials): cover malformed binding metadata Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(credentials): verify multi-key endpoint isolation Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(e2e): cover same-host credential path denial Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(credentials): document serialized refresh contract Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * refactor(proxy): consolidate L7 log formatting Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * perf(credentials): precompile endpoint binding patterns Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * perf(credentials): share identity epoch revisions Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(proxy): require explicit request default ports Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(policy): validate SigV4 credential sources Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(credentials): preserve endpoint bindings for credential handles Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * feat(go-sdk): expose network credential bindings Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: John Myers <johntmyers@users.noreply.github.com> Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> Co-authored-by: John Myers <johntmyers@users.noreply.github.com> |
||
|
Derek Carr
|
5952a5a23f |
feat(workspace): add workspace resource model with scoping, membershi… (#2243)
* feat(workspace): implement workspace model (Phase 1 of RFC 0011) Implements workspace and membership model providing hard isolation boundaries for multi-player OpenShell deployments. Workspace CRUD with Kubernetes-style Terminating phase for graceful deletion. All resources scoped by workspace via ObjectMeta. Membership RPCs for workspace access control. Persistence migration shifts name uniqueness to (object_type, workspace, name). Provider profiles support platform and workspace scoping. Service routing uses workspace-prefixed DNS labels. Inference routes renamed and workspace-scoped with DeleteInferenceRoute RPC. Python SDK with WorkspaceClient, two-method list pattern (workspace-scoped and for_all_workspaces), and workspace parameter on all methods. CLI workspace flags, TUI workspace cycling. K8s driver filters unmanaged CRs and uses delete preconditions. Podman driver uses immutable container IDs. Label serialization fixed across all put_if call sites. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(cli): delegate sandbox upload command to existing upload function The standalone `sandbox upload` command reimplemented upload logic inline with two bugs: it used `Path::exists()` which follows symlinks (rejecting dangling symlinks), and it ran git-aware filtering on symlink sources. The `run::sandbox_upload()` function already handles both cases correctly via `sandbox_upload_plan()`. Replace the inline logic with a call to the existing function. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(e2e): shorten sandbox names and fix test compatibility Shorten the sandbox name in initial_sparse_policy_is_acknowledged_as_loaded from 'e2e-2159-sparse-enrich' (22 chars) to 'e2e-sparse-enrich' (17 chars) to comply with MAX_ROUTABLE_NAME_LEN (19 chars). Also capture stderr in create_keep_with_args so future sandbox creation failures include the actual CLI error instead of reporting empty output. Signed-off-by: Derek Carr <decarr@redhat.com> * test(workspace): add test coverage for workspace CRUD and persistence isolation Add unit tests for workspace create happy path, get round-trip, get not-found, get empty-name rejection, already-exists error, and resolve_workspace not-found. Add persistence test proving cross-workspace name uniqueness (same name in different workspaces produces separate records). Add workspace name max-length boundary tests. Fix e2e harness to include stderr in name-parse-failure error path. Align Python e2e test_workspace_crud with try/finally pattern. Document provider profile catalog workspace scoping gap in RFC 0011. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(examples): update examples for workspace model compatibility Shorten sandbox names in demo scripts to fit the 19-character MAX_ROUTABLE_NAME_LEN limit: policy-demo prefix to pd-, multi-agent notepad derives a short SANDBOX_TAG from the run ID, governance interceptor uses gs-PID-RANDOM. Update vscode-remote-sandbox.md SSH host aliases from openshell-{name} to openshell-{name}.{workspace} format. Signed-off-by: Derek Carr <decarr@redhat.com> * feat(sdk): add workspace-scoped client and workspace CRUD Add WorkspaceScopedClient modeled after kube::Api::namespaced — captures workspace once and injects it into every sandbox request. Add workspace CRUD methods (create, get, list, delete) and list_sandboxes_all_workspaces on OpenShellClient. Extend SandboxRef with workspace field and add WorkspaceRef type. Include mock tests for all new operations. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(lint): resolve clippy warnings in workspace test assertions Signed-off-by: Derek Carr <decarr@redhat.com> * fix(docs): convert indented code blocks to fenced in RFC 0011 Signed-off-by: Derek Carr <decarr@redhat.com> * fix(lint): resolve clippy warnings and apply cargo fmt across workspace Auto-format with cargo fmt and fix clippy warnings exposed by the reformat: unnecessary qualifications, map_unwrap_or, identical match arms, unused variable prefix, dead code annotations, and let-unit-value in e2e harness. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): address workspace scoping issues from review - Add workspace field to settings JSON output (CLI) - Skip Podman containers missing workspace label instead of defaulting to empty string, matching K8s driver behavior - Add resource_version to list_by_scope SELECT in both SQLite and Postgres backends, with regression test - Gate PolicyLocalContext proposal/lookup routes on workspace readiness, returning 503 when workspace is not yet discovered - Block sandbox and provider creation in TUI all-workspaces mode - Clear workspace vectors in TUI reset_sandbox_state Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): make provider profile catalog workspace-aware Thread workspace through snapshot_catalog so the EffectiveProviderProfileCatalog enforces workspace boundaries on both read and write paths. UserProviderProfileSource now loads platform-scoped profiles (workspace "") plus the target workspace's profiles, preventing cross-workspace duplicate profile ID collisions that previously caused global catalog failures. Update RFC 0011 to reflect catalog scoping is implemented in Phase 1 rather than deferred to future work. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(persistence): include workspace column in atomic policy revision INSERT put_policy_revision_atomic omitted the workspace column from the INSERT into the objects table in both SQLite and Postgres backends, causing atomically-written policy revisions to lose their workspace association. Add workspace field to AtomicPolicyRevisionWrite and thread it through both backend INSERT statements, matching the non-atomic put_policy_revision path which already included it. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(proxy): skip ancestor walk when socket owner is the entrypoint collect_ancestor_identities walked the entire process tree above the entrypoint when the connecting process was the entrypoint itself, SHA256-hashing every ancestor binary (IDE, shell, container runtime). On dev machines with large binaries in the ancestor chain this exceeded the 30-second test timeout. When start_pid == stop_pid there are no intermediate ancestors to verify, so return an empty list immediately. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): make provider profile catalog scope-aware Allow the same profile ID at platform and workspace scopes by introducing layered catalog entries where workspace profiles shadow platform profiles. Add source and scope fields to the ProviderProfile proto and CLI output. Migrate List/Get handlers to the catalog, fixing divergence with runtime profile resolution. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(e2e): align podman e2e labels with centralized driver constants The podman driver moved its container labels to the centralized openshell.ai/ prefix, but the e2e test harness and cleanup script still referenced the old openshell.sandbox-* keys, causing the local_driver_token_restart test to fail on container lookup. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(e2e): align python profile isolation test with scope-aware catalog Platform profiles are now visible in workspace listings as fallbacks per the layered catalog design. Update the assertion to match. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): honor profile_workspace in runtime profile resolution Runtime profile lookups now consult provider.profile_workspace via get_type_profile_for_scope. Providers created with --global-profile (profile_workspace="") resolve to the platform profile even when a workspace profile shadows the same ID. All 6 runtime call sites updated; type-only call sites remain scope-agnostic. Signed-off-by: Derek Carr <decarr@redhat.com> --------- Signed-off-by: Derek Carr <decarr@redhat.com> |
||
|
Russell Bryant
|
9377e0d5fe |
fix(providers): allow git clone/fetch via default GitHub provider (#2317)
* fix(providers): allow git clone/fetch via default GitHub provider The github.com:443 git-transport endpoint used the read-only access preset, which expands to GET/HEAD/OPTIONS only. Git smart HTTP requires a POST to */git-upload-pack for clone and fetch, so the L7 proxy denied those operations and `gh repo clone` / `git clone https://...` failed. Replace the preset with explicit rules that permit the read-only methods plus POST */git-upload-pack, so clone/fetch work while push (git-receive-pack) stays blocked. Enabling push still requires an explicit policy proposal. Why allowing this POST is still read-only: in git's smart HTTP protocol POST is an RPC transport, not a write. A clone/fetch does GET */info/refs (ref discovery) followed by POST */git-upload-pack, whose body is only the client's want/have negotiation; the server responds with a packfile and nothing on the server is modified (data flows server -> client). The service names are from the server's perspective: git-upload-pack = the server uploads a pack to the client (a read/ download), while git-receive-pack = the server receives a pack from the client (the actual write/push). The new rule is scoped to */git-upload-pack only, so push (git-receive-pack) and arbitrary POSTs to github.com remain denied. Add a provider-profile regression test and a rego enforcement test covering ref discovery, upload-pack (allowed), and receive-pack (denied). Closes #1769 Signed-off-by: Russell Bryant <rbryant@redhat.com> * test(providers): strengthen git-transport regression and add clone e2e Pin the exact allowed rule set for the built-in github git-transport endpoint in both the provider-profile and composed-policy tests, so a broader or additional POST rule (e.g. POST **) that could enable push via git-receive-pack fails the test instead of passing a substring check. Add an e2e test that attaches the built-in github provider and clones a public repo over HTTPS, exercising provider attachment, effective-policy composition, TLS interception, and real git behavior. Update the Providers V2 docs so the github.com git-transport endpoint shows explicit clone/fetch rules instead of the stale read-only preset. Refs #1769 Signed-off-by: Russell Bryant <rbryant@redhat.com> * test(providers): isolate providers_v2 mutation in clone e2e The clone e2e enables the gateway-global providers_v2_enabled setting. Restore its exact prior value (or absence) captured via GetGatewayConfig instead of unconditionally deleting it, and serialize the mutation across xdist workers with an exclusive file lock on the run's shared base temp dir, so a shared or pre-configured gateway is left untouched and parallel workers cannot race the read-modify-restore. Refs #1769 Signed-off-by: Russell Bryant <rbryant@redhat.com> * test(providers): serialize providers_v2 mutation with a suite-wide guard The clone e2e's per-fixture lock only coordinated fixtures that acquired it; other xdist workers hit the same gateway without it and could observe the transiently-enabled providers_v2_enabled global during their own sandbox creation (CWE-362). Add an autouse readers-writer guard in conftest: every test holds a shared lock on the gateway config, and a test marked exclusive_gateway_config holds an exclusive lock. Mark the clone test exclusive so no other worker is mid-test while it enables and restores the gateway-global setting. Exact prior-value restoration is retained. Refs #1769 Signed-off-by: Russell Bryant <rbryant@redhat.com> --------- Signed-off-by: Russell Bryant <rbryant@redhat.com> |
||
|
Taylor Mutch
|
7a0c444445 |
refactor!(auth): drop SSH handshake secret (#1274)
* refactor!(auth): drop SSH handshake secret in favor of mTLS The OPENSHELL_SSH_HANDSHAKE_SECRET / x-sandbox-secret mechanism was misnamed: it does not authenticate SSH (which flows over the RelayStream gRPC RPC and is gated by mTLS plus supervisor Unix-socket permissions). It only gated a small set of sandbox-to-gateway control-plane RPCs, and production deployments already enforce mTLS on that channel — so the shared secret was redundant. Replace the secret check with an mTLS-presence marker. Sandbox-class methods (ReportPolicyStatus, PushSandboxLogs, GetSandboxProviderEnvironment, SubmitPolicyAnalysis, GetSandboxConfig, GetInferenceBundle) accept callers without a Bearer token; the gRPC mTLS handshake is the trust boundary. Dual-auth methods treat Bearer-present as full-scope CLI access and Bearer-absent as sandbox-restricted scope via validate_sandbox_caller_update. Drops the secret from all drivers (K8s, Podman, VM), the sandbox gRPC interceptor, the Helm chart (values + pre-install hook + StatefulSet env), the RPM bootstrap script, the man pages, and the debug-openshell-cluster skill. Also removes the never-read ssh_handshake_skew_secs flag and config field. BREAKING CHANGE: --ssh-handshake-secret / OPENSHELL_SSH_HANDSHAKE_SECRET and --ssh-handshake-skew-secs / OPENSHELL_SSH_HANDSHAKE_SKEW_SECS are removed from the gateway, sandbox, and all driver binaries. The openshell-ssh-handshake K8s Secret is no longer managed by the chart; operators may delete the orphan. Deployments using --disable-gateway-auth must enforce caller authentication at the fronting proxy, since the gateway no longer validates a per-request secret on sandbox-class methods. Refs OS-174. * docs(auth): scrub residual SSH handshake secret references Sweep across docs, e2e scripts, the Podman driver README/NETWORKING notes, the RPM/Helm/setup guides, the gateway man page, and RFC 0003 to remove instructions and examples that still referenced OPENSHELL_SSH_HANDSHAKE_SECRET / --ssh-handshake-secret / ssh_handshake_skew_secs. The mechanism is gone; nothing should still suggest setting it. Negative-assertion regression tests are kept so the env var cannot silently be re-introduced. * test(drivers): drop SSH handshake secret negative-assertion tests The supporting code, env vars, CLI flags, and config plumbing are gone — these tests asserted absence of strings that no longer have any path to being set. Remove the guards from the Docker, Podman, Kubernetes, and VM driver test modules. |
||
|
John T. Myers
|
1d3b741ee3 |
feat(providers): support sandbox provider attach lifecycle (#1242)
* feat(providers): support sandbox provider attach lifecycle Closes #1171 Adds sandbox provider list, attach, and detach API/CLI support while keeping provider policy and credential resolution derived from current sandbox attachments. * fix(providers): refresh sandbox provider credentials Adds provider environment revisions and generation-scoped sandbox credential snapshots so future SSH and exec launches pick up provider attach, detach, and credential updates without mutating already-running processes. Also blocks provider deletion while attached to prevent stale sandbox provider references. * fix(providers): serialize sandbox object mutations * test(providers): cover sandbox provider attach lifecycle * test(providers): accept versioned credential placeholders |
||
|
Derek Carr
|
5e28ea3a4b |
feat(server): add object meta convention to top-level objects (#919)
- adds filterable label selectors on resources Closes #864 Signed-off-by: Derek Carr <decarr@redhat.com> |
||
|
Derek Carr
|
87f50f5e5d |
fix(e2e): add /dev/urandom to provider test sandbox policy (#948)
Python runtime requires /dev/urandom access during initialization to seed the hash randomizer. The _default_policy() in provider tests was missing this path, causing exec_python tests to fail with: 'Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python'. Add /dev/urandom to read_only paths to match the policy used in test_sandbox_policy.py, allowing Python to initialize successfully. Signed-off-by: Derek Carr <decarr@redhat.com> |
||
|
John T. Myers
|
256f7fc884 |
fix(sandbox,server): fix chunk merge duplicates and OPA variable collision with overlapping policies (#571)
* fix(sandbox,server): fix chunk merge duplicates and OPA variable collision with overlapping policies
Two related bugs triggered when a draft rule approval creates a second
policy entry for the same host:port:
1. merge_chunk_into_policy looked up existing rules by chunk.rule_name
(auto-generated as allow_{host}_{port}), which never matched the
user's original rule name. Now scans all network_policies entries
for a host:port endpoint match before falling back to insertion,
and merges allowed_ips into the existing endpoint.
2. The Rego allow_request rule and _matching_endpoint_configs
comprehension used 'some ep; ep := policy.endpoints[_]' which
caused regorus to error with 'duplicated definition of local
variable ep' when multiple policies covered the same host:port.
Refactored to isolate endpoint iteration inside helper functions
(_policy_allows_l7, _policy_endpoint_configs) so variables are
scoped per-policy evaluation.
Refs: #567
* test(e2e): add overlapping policy tests and update FWD-2 for implicit allowed_ips
- Update FWD-2 (test_forward_proxy_denied_without_allowed_ips ->
test_forward_proxy_allows_private_ip_host_without_allowed_ips):
literal IP host no longer requires explicit allowed_ips, expects 200.
- Add OVL-1: overlapping L4 policies for same host:port must not crash
OPA and should allow forward proxy connections.
- Add OVL-2: overlapping L7 policies for same host:port must not crash
OPA and should allow CONNECT tunnel establishment.
Refs: #567
* style: apply cargo fmt formatting
* test(e2e): update SSRF-3 and SSRF-6 for implicit allowed_ips behavior
SSRF-6: Private IP with literal IP host now gets implicit allowed_ips
from PR #570, so CONNECT returns 200 instead of 403.
SSRF-3: Loopback is still blocked but via the always-blocked path
(implicit allowed_ips is synthesized, then resolve_and_check_allowed_ips
catches it). Log message says 'always-blocked' instead of 'internal
address'.
* fix(e2e): use negative assertion for SSRF-6 when nothing listens on target port
When the SSRF check passes but nothing listens on the target port,
recv() returns empty bytes. Use 'assert 403 not in' (matching SSRF-4
pattern) instead of 'assert 200 in'.
* fix(e2e): update provider tests for redacted credential values
PR #569 changed credential redaction from clearing the map to
replacing values with 'REDACTED'. Update e2e assertions to expect
credential keys with REDACTED values instead of an empty map.
|
||
|
John T. Myers
|
647b7947f6 |
fix: security hardening from aardvark/codex scanner findings (#352)
* fix(sandbox): prevent overread request smuggling in L7 REST parser The HTTP/1.1 parser used a 1024-byte read buffer that could capture bytes from a pipelined second request. Those overflow bytes were forwarded upstream as body overflow without L7 policy evaluation, enabling request smuggling that bypasses per-request method/path enforcement. Replace multi-byte read with byte-at-a-time read_u8 that stops exactly at the CRLFCRLF header terminator. Add regression test proving two pipelined requests are parsed independently. Refs: #350 * fix(server): harden sandbox TLS secret volume permissions to 0400 Kubernetes secret volumes default to 0644, allowing the unprivileged sandbox user to read the mTLS client private key via the Landlock baseline /etc read set. A compromised sandbox could use the key to impersonate the control-plane client. Set defaultMode to 256 (octal 0400, owner-read only) on both the default and custom pod template paths. The supervisor reads TLS materials as root before forking, so this does not affect normal operation. Refs: #350 * fix(sandbox): stop using cmdline paths for binary policy matching /proc/pid/cmdline is fully attacker-controlled (argv[0] can be set to any string via execve) and had no integrity verification, unlike exec.path and ancestors which are kernel-managed and get TOFU/SHA256 checks. The Rego policy used cmdline_paths as a grant-access signal, allowing any sandboxed process to claim the identity of an allowed binary and bypass network restrictions. Remove the cmdline exact-match rule and exclude cmdline_paths from the glob-match rule. Only exec.path and exec.ancestors (from /proc/pid/exe) are now used for binary identity. cmdline_paths remain in the OPA input for deny-reason diagnostics only. Refs: #350 * fix(sandbox): reject symlink and non-dir read_write paths before chown The supervisor runs as root and calls chown on each read_write path. Since chown follows symlinks, a malicious container image could place a symlink (e.g. /sandbox -> /etc/shadow) to trick the supervisor into transferring ownership of arbitrary files to the sandbox user. Add symlink_metadata (lstat) check before chown to reject symlinks and non-directory entries. The TOCTOU window is not exploitable because no untrusted child process has been forked yet at this point. Refs: #350 * fix(server): redact provider credentials in gRPC CRUD responses Provider CRUD RPCs (create, get, list, update) returned full Provider objects including plaintext credentials (API keys, secrets). Any authenticated client -- including sandbox workloads running untrusted code -- could read credentials for all providers. Add redact_provider_credentials helper that clears the credentials map before returning. Internal server paths (inference routing, sandbox env injection) read from the store directly and are unaffected. Update tests to verify redaction and assert persistence via direct store reads. Refs: #350 * fix(sandbox): enforce non-root fallback when process user unset drop_privileges silently returned Ok(()) when both run_as_user and run_as_group were None, even when running as root. In local/dev mode policies are loaded from disk without passing through the server-side ensure_sandbox_process_identity normalization, so child processes could retain root and all capabilities (SYS_ADMIN, NET_ADMIN). When running as root with no process identity configured, fall back to sandbox:sandbox instead of no-oping. Non-root runtimes are unaffected. Refs: #350 * fix(sandbox): deny forward proxy for L7-configured endpoints The forward proxy path only performed L4 (endpoint) and allowed_ips checks. If an endpoint had L7 rules (method/path restrictions), a sandboxed process could bypass them by using HTTP_PROXY with plain http:// requests instead of CONNECT tunneling, since L7 inspection only runs in the CONNECT path. Add a guard in handle_forward_proxy that queries the endpoint's L7 config and returns 403 if any L7 rules are present, forcing traffic through the CONNECT path where per-request inspection happens. Refs: #350 * test(e2e): add regression test for forward proxy L7 bypass Verifies that the forward proxy path (plain http:// via HTTP_PROXY) returns 403 for endpoints with L7 rules configured, preventing sandboxed processes from bypassing per-request method/path enforcement by avoiding the CONNECT tunnel. Refs: #350 * fix: update tests for cmdline path and TLS volume changes Update OPA tests to verify cmdline_paths no longer grant access (regression tests for the security fix). Fix formatting in TLS volume test. Refs: #350 * test(e2e): update provider e2e tests for credential redaction Provider CRUD gRPC responses no longer include credential values. Update three e2e tests to assert credentials are empty in responses. Provider functionality is verified by existing e2e tests that check env var injection into sandboxes (which read from the store directly). Refs: #350 * chore: reformat for Rust 1.94 assert! macro style * fix(sandbox): make drop_privileges tests root-aware for CI CI runs as root but has no 'sandbox' user. The security fix correctly errors when running as root with no process identity and no fallback user available -- this is the intended behavior (refuse to run as root). Update tests to expect that error in root-without-sandbox-user environments instead of unconditionally asserting Ok. Refs: #350 * fix(sandbox): allow non-directory read_write entries like /dev/null The symlink guard incorrectly rejected all non-directory entries. Character devices like /dev/null are legitimate read_write paths used in sandbox policies. Only reject symlinks, which are the actual attack vector for the chown privilege escalation. Refs: #350 |
||
|
Drew Newberry
|
f6ae1da12d | chore: remove remaining navigator and nemoclaw references (#279) | ||
|
Drew Newberry
|
fbd93a4632 | refactor: rename navigator- crate prefix to openshell- (#277) | ||
|
Drew Newberry
|
b9d10861b6 | refactor(sandbox): move secrets to supervisor placeholders (#192) | ||
|
Drew Newberry
|
756950140c | refactor(python): rename navigator module to openshell and migrate config to gateway paths (#220) | ||
|
John T. MyersandJohn Myers
|
a3af9af2fe |
fix(server): merge provider credentials/config on update instead of replacing (#202)
UpdateProvider RPC was doing full replacement of credentials and config maps, silently destroying data on partial updates. This changes the semantics so empty maps preserve existing values, non-empty maps merge (upsert), and empty-string values delete individual keys. Also makes provider type immutable after creation and adds field validation on update. Closes #199 Co-authored-by: John Myers <johntmyers@users.noreply.github.com> |
||
|
Piotr Mlocek
|
31d7ca53ff |
refactor(inference): simplify routing — introduce inference.local, remove implicit catch-all (#146)
Remove multi-route CRUD system and replace with single managed cluster route (inference.local). Key changes: - Remove inference route CRUD RPCs and CLI commands - Remove InspectForInference OPA action; policy is binary allow/deny - Introduce AuthHeader enum and InferenceProviderProfile in navigator-core - Router is now provider-agnostic: auth style carried on ResolvedRoute - Replace InferenceRouteSpec with ClusterInferenceConfig (2 fields vs 8) - Rename proto: routing_hint->name, SandboxResolvedRoute->ResolvedRoute, GetSandboxInferenceBundle->GetInferenceBundle, drop sandbox_id param - Rename RouteConfig.route -> RouteConfig.name; use inference.local - Add 'nemoclaw cluster inference update' for partial config changes - Delete stale navigator.inference.v1.rs checked-in proto file - Update architecture docs, agent skills, and CLI reference Closes #133 |
||
|
Alexander Watson
|
1d7909cb38 |
chore: add open-source compliance files and SPDX headers (#71)
Add Apache 2.0 licensing, SPDX copyright headers on all source files, DCO enforcement, third-party notices, and CI enforcement. - LICENSE: Apache License 2.0 full text - DCO: Developer Certificate of Origin 1.1 - SPDX headers on all 176 source files (.rs, .py, .proto, .rego, .sh, .toml, .yaml, Dockerfiles) - scripts/update_license_headers.py: header management with --check mode - scripts/generate_third_party_notices.py: dependency license aggregation - THIRD-PARTY-NOTICES: generated listing of all Rust and Python deps - build/license.toml: mise tasks for license:check and license:update - CI: license-headers job in checks.yml, DCO check workflow - CONTRIBUTING.md: DCO sign-off requirement and license header docs - Cargo.toml: license changed to Apache-2.0, repository URL updated - pyproject.toml: license field added Closes #58 |
||
|
Piotr Mlocek
|
34fd3cbf64 |
feat(inference): inference interception and routing (!38)
Closes #67 ## Summary Implements transparent inference interception and routing for sandboxed AI agents. The sandbox proxy intercepts outbound AI SDK calls (OpenAI, Anthropic) and reroutes them through the gateway to policy-controlled backends — enabling organizations to redirect inference traffic to local or self-hosted models without modifying agent code. **Decision model** — a tri-state OPA evaluation for every CONNECT request: 1. Binary + endpoint explicitly allowed in `network_policies` → **allow** (pass through) 2. Not explicitly allowed + `inference.allowed_routes` configured → **inspect for inference** (TLS intercept, detect API patterns, route through gateway) 3. Otherwise → **deny** No endpoint declarations or binary lists needed for inference routing. Just configure `inference.allowed_routes`. ## Key Changes ### Sandbox (interception) - **OPA policy**: New `network_action` Rego rule with three outcomes (`allow`, `inspect_for_inference`, `deny`). New `NetworkAction` enum replaces `PolicyDecision.allowed` bool for the proxy's main decision path. - **Proxy**: New `InspectForInference` path — TLS-terminates client, parses HTTP, detects inference API patterns (`POST /v1/chat/completions`, `/v1/completions`, `/v1/messages`), strips auth headers, forwards via gRPC. - **New module**: `l7/inference.rs` — `InferenceApiPattern`, `detect_inference_pattern()`, HTTP request/response parsing. - **gRPC client**: New `proxy_inference()` for sandbox→gateway forwarding. - **Sandbox init**: Creates OPA engine when inference is configured, even without `network_policies`. ### Gateway (dispatch) - **InferenceService**: `ProxyInference` RPC loads sandbox policy, resolves allowed routes, dispatches to router. Full CRUD for inference routes. - **Proto**: `InferenceRoute`, `InferenceRouteSpec`, `ProxyInferenceRequest/Response`, Inference gRPC service. ### Router (backend proxying) - **New crate**: `navigator-router` with `Router`, `proxy_with_candidates()`, protocol-based route selection, backend HTTP proxying with auth header rewriting. - **Mock support** for testing (`mock://` scheme). ### CLI - `nav inference create/update/delete/list` commands for route management. ### Python SDK - Updated protobuf bindings. Removed old `inference.py` client (replaced by transparent interception). ### Documentation - New `architecture/inference-routing.md` — end-to-end system documentation. - Updated `architecture/sandbox.md` — proxy, OPA, and source index sections. - Updated `architecture/README.md` — new subsystem overview and diagram. ## Addendum: Chunked Transfer Compatibility This branch now also fixes intercepted SDK requests that send chunked request bodies: - `inspect_for_inference` now accepts `Transfer-Encoding: chunked` and decodes chunked request bodies before forwarding to the gateway - Removed the prior `411 Length Required` response for chunked intercepted requests - Added request/response header sanitization for framing and hop-by-hop headers (`content-length`, `transfer-encoding`, `connection`, etc.) to keep forwarded requests and returned responses valid - Added unit tests for chunked parsing and header sanitization Note: this improves compatibility for streaming-style SDK request patterns; true token-by-token passthrough response streaming is still a separate follow-up. ## Minimal Policy for Inference Routing ```yaml inference: allowed_routes: - local ``` Any outgoing connection from a binary not explicitly allowed in `network_policies` will be intercepted and checked for inference API patterns. ## Test Plan - [x] `cargo test --workspace` — all tests pass - [x] `mise run pre-commit` — all checks pass - [x] E2E: OpenAI chat completions routed through gateway - [x] E2E: Anthropic messages routed through gateway - [x] E2E: Python OpenAI SDK from sandbox (`examples/inference/inference.py`) - [x] E2E test: `e2e/python/test_inference_routing.py` |
||
|
Drew Newberry
|
89a6e04c8c |
feat(providers): inject provider credentials into sandbox child processes at runtime (!26)
## Summary - Add `GetSandboxProviderEnvironment` gRPC endpoint so the sandbox supervisor can fetch provider credentials at runtime instead of embedding them in the pod spec - Sandbox supervisor (`navigator-sandbox`) fetches credentials on startup and injects them as environment variables into both entrypoint processes and SSH shell sessions via `Command::env()` - Remove credential injection from sandbox creation-time pod spec in `navigator-server` - Update `architecture/providers.md` to document the full runtime credential injection flow, discovery engine details, trait definitions, and end-to-end diagram ## Changes - **proto**: Add `GetSandboxProviderEnvironment` RPC and request/response messages, add `providers` field to `SandboxSpec` - **navigator-server**: Implement `GetSandboxProviderEnvironment` handler and `resolve_provider_environment()` with env var key validation and first-wins dedup; remove old creation-time credential injection from `sandbox/mod.rs` - **navigator-sandbox**: Fetch provider env via gRPC on startup (`grpc_client.rs`), thread `provider_env` HashMap through to `process.rs` (entrypoint) and `ssh.rs` (shell sessions), inject via `cmd.env()` - **navigator-cli**: Adjust sandbox create flow to set provider names in `SandboxSpec.providers` - **e2e**: Add `test_sandbox_providers.py` end-to-end tests - **docs**: Rewrite `architecture/providers.md` with implementation details ## Test Plan - Unit tests for `resolve_provider_environment` in `grpc.rs` - Integration tests updated in `provider_commands_integration.rs`, `mtls_integration.rs`, `multiplex_integration.rs`, `multiplex_tls_integration.rs` - New e2e test suite `test_sandbox_providers.py` |