mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-16 05:20:22 +08:00
pull-request/2592
121
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
5548405fcb |
feat(credentials): add provider credential storage drivers (#2437)
* feat(credentials): add provider credential storage drivers Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(credentials): harden credential update handling Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(credentials): harden credential driver security, correctness, and performance Address review findings from the credential storage drivers PR: - Route additional_credentials through the driver on refresh to prevent silent data loss for multi-credential providers (e.g. AWS STS) - Clean up stored credential handles on CAS failure during refresh to prevent orphaned secrets in external backends - Enforce namespace validation in the Kubernetes Secrets driver to prevent cross-namespace credential access when allow_reference_namespace is not enabled - Cache Vault Kubernetes auth tokens with 80% TTL to avoid re-authenticating on every credential operation - Parallelize resolve_credentials in all three drivers using try_join_all for faster sandbox startup - Add existingSecret support for the KEK Secret to fix helm template/GitOps workflows where lookup returns empty and regenerates the key - Document RBAC blast radius for the Kubernetes Secrets credential driver and recommend a dedicated namespace Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): add optimistic concurrency, fix thundering herd, parallelize operations Use resourceVersion optimistic concurrency with retry loop for K8s Secret ownership checks to prevent TOCTOU races. Switch Vault token cache from RwLock to Mutex with double-check pattern to prevent thundering herd on cache miss. Parallelize credential store and delete operations across independent keys using try_join_all. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): handle partial failures, add delete retry, consolidate cleanup Replace try_join_all with join_all in credential store/delete operations to handle partial failures — successfully-stored handles are cleaned up when another key fails. Add retry loop with conflict detection to db-credstore delete_credential, matching the K8s driver pattern. Consolidate 4 manual cleanup_pre_stored_provider_credentials call sites into a single error handler using an async block. Remove inconsistent .trim() from db-credstore validate_handle_owner. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): fix retry loop guard and remove unprotected validation Remove attempt-count guard from 409/Aborted match arms in retry loops so the post-loop Status::aborted error is reachable after exhausting retries. Previously, last-attempt conflicts fell through to the catch-all error arm, producing misleading Status::unavailable errors. Remove duplicate validation calls that ran after prepare_provider_credential_update but outside the cleanup-protected async block, which would leak pre-stored handles on failure. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): add workspace/provider UUID to credential backend paths Include workspace and provider ID in credential backend object paths to ensure cross-workspace uniqueness and prevent credential collision (GATOR-1806c9be-01). - Updated credential driver proto to include workspace and provider_id fields - Modified Vault driver to include workspace/provider_id in managed_secret_path - Modified Kubernetes Secrets driver to include workspace/provider_id in credential_owner_id and managed_secret_name - Updated all credential runtime calls to pass workspace/provider_id - Updated tests to use the new signatures This prevents two workspaces sharing the same external credential store from colliding on provider names, which was a critical security issue (CWE-639). * fix(credentials): preserve provider-level expiration for handle-backed credentials Compute effective expiration from both provider and driver values using the earliest non-zero timestamp and skip expired values before insertion (GATOR-1806c9be-02). - Modified resolve_provider_handles to check provider credential_expires_at_ms - Skip expired credentials during resolution instead of returning them - Use effective expiration (min of provider and driver) in resolution results - Fix inference.rs to preserve earliest expiration when merging This ensures handle-backed credentials respect the same expiration semantics as inline credentials. * fix(credentials): stage refresh changes under new handles before validation Stage credential replacements under new immutable handles instead of reusing existing handles to prevent overwriting committed values before validation/CAS (GATOR-1806c9be-03). - Stage credentials with empty existing_handles map to force new handle creation - Validate and CAS before the new values are committed to backend storage - Delete old handles only after successful CAS - On CAS failure, delete only the newly staged handles - This prevents CWE-362/CWE-367 race conditions where failed refreshes could still modify or delete the active credential The fix ensures that a rejected refresh cannot modify the backend object still referenced by the committed provider record. * fix(credentials): add timeouts to credential driver RPCs Apply configured timeouts to both startup capability negotiation and runtime RPCs to prevent indefinite hangs (GATOR-1806c9be-05). - Add DEFAULT_CREDENTIAL_DRIVER_RPC_TIMEOUT_SECS constant (30s) - Apply timeout to GetCapabilities during startup connection - Apply timeout to all runtime RPCs (store, delete, resolve) - Use tokio::time::timeout to bound the entire GetCapabilities operation during startup, not just the socket connection - Return contextual deadline errors on timeout This prevents a faulty or overloaded driver from hanging gateway operations indefinitely. * fix(credentials): fix test to use consistent workspace/provider identity The Kubernetes auth Vault resolve test was constructing a managed path with test-workspace/test-provider-id but sending default/prov-123 in the request, causing validation to reject the request (GATOR-18e32351-01). - Update test to use test-workspace and test-provider-id in the request to match the logical_path construction - This ensures the test exercises the intended code path and validates Kubernetes auth resolution properly The test now passes and correctly validates identity enforcement. * fix(credentials): use unique staging ID for refresh to avoid overwrites Stage refresh replacements under genuinely distinct immutable handles using a unique staging ID to prevent overwriting committed values (GATOR-1806c9be-03). - Generate a unique staging ID using UUID for each refresh operation - Use this staging ID when storing credentials instead of the real provider ID - Pass the same staging ID during cleanup on failure to delete only staged objects - This ensures deterministic paths (Vault) and object names (K8s) don't collide with the committed provider's credentials The fix prevents failed refreshes from silently replacing active credentials or breaking providers by deleting still-referenced backend objects. * fix(credentials): wrap credential driver RPCs in local timeouts Add local tokio::time::timeout wrappers around credential driver RPCs to bound non-compliant or stalled UDS peers (GATOR-1806c9be-05). - Wrap StoreCredential, DeleteCredential, and ResolveCredentials in local timeouts - Return contextual deadline_exceeded errors when timeouts occur - Keep existing gRPC timeout metadata for compliant implementations - GetCapabilities during startup was already wrapped in previous commit This ensures a faulty local driver cannot hang gateway operations indefinitely, even if it accepts the connection but never responds to the RPC. * fix(credentials): preserve ownership for staged refreshes Signed-off-by: Seth Jennings <sjenning@redhat.com> * fix(credentials): bound startup capability probe Signed-off-by: Seth Jennings <sjenning@redhat.com> * test(provider): authenticate credential handler requests Signed-off-by: Seth Jennings <sjenning@redhat.com> * fix(ci): grant actions read to credential driver e2e Signed-off-by: Seth Jennings <sjenning@redhat.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> Co-authored-by: Taylor Mutch <taylormutch@gmail.com> Co-authored-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> |
||
|
Jim Meyer
|
8c7dd148a9 |
perf(net): set TCP_NODELAY on latency-sensitive TCP hops (#2220)
* perf(net): set TCP_NODELAY on all tunnel and proxy TCP hops The sandbox tunnel added ~44 ms of latency to every small request/response because no socket in the path disabled Nagle's algorithm, so sub-MSS writes waited on delayed ACKs at each hop. Set TCP_NODELAY on every latency-sensitive TCP socket: - gateway: accepted connections on the public listener (gRPC relay frames and WS tunnel writes) - CLI: edge tunnel local accept + underlying WebSocket TCP stream, insecure TLS connector (tonic's default connector already does this), and service-forward accepted sockets - supervisor: direct-tcpip connect into the sandbox netns, TCP relay target dials, egress proxy accepted connections, and all upstream CONNECT/HTTP dials (via a new connect_upstream helper) Setting TCP_NODELAY on connect is best-effort: a failure only costs latency, so we log and continue rather than fail the connection. The sandbox SSH transport rides a unix domain socket and gRPC client channels use tonic defaults (nodelay on), so no change is needed there. Fixes #2219 Signed-off-by: Jim Meyer <jim@meyer4hire.com> * refactor(net): house TCP_NODELAY helper in a shared net module Address review feedback on the TCP_NODELAY change: - Move the shared set-nodelay helper out of supervisor_session into a new crate-private `net` module in openshell-supervisor-process, so ssh and supervisor_session no longer reach across modules through a pub(crate) item. - Make the best-effort comments at each call site terse and consistent. No behavior change; the benchmark ladder reproduces the same numbers. Signed-off-by: Jim Meyer <jim@meyer4hire.com> * refactor(net): consolidate TCP_NODELAY helpers into openshell_core::net Move the best-effort TCP_NODELAY helpers into the shared openshell_core::net module so every crate dials and configures sockets the same way: - Add set_tcp_nodelay_best_effort (accepted/existing streams) and connect_tcp_nodelay_best_effort (dial + set) with unit tests. - Migrate all call sites in openshell-cli, openshell-server, and the supervisor crates to the shared helpers. - Remove the crate-private net module from openshell-supervisor-process. - Document socket guidance in AGENTS.md (Network Sockets). Signed-off-by: Jim Meyer <jim@meyer4hire.com> * perf(net): set TCP_NODELAY on exec bridge and metadata server The gateway-side single-use SSH-over-relay loopback bridge and the sandbox IMDS metadata server were missed latency-sensitive TCP hops. Set TCP_NODELAY on the accepted client connection and both russh client dials of the exec bridge — interactive keystrokes and line-buffered PTY output are the most tinygram-heavy traffic in the system — and on the metadata server's accepted connections. Also log unrecognized MaybeTlsStream variants in the edge tunnel so a future TLS-backend change surfaces a silent TCP_NODELAY miss instead of skipping it quietly. Signed-off-by: Jim Meyer <jim@meyer4hire.com> * perf(net): set TCP_NODELAY on openshell-sdk socket paths The openshell-sdk crate landed on main with its own copies of the CLI's hand-rolled sockets, which the CLI and TUI are meant to consume. Give them the same treatment as the CLI equivalents: - edge_tunnel: the accepted local tunnel connection and the WebSocket's underlying TCP socket (plain and rustls variants). - transport: the dial in InsecureTlsConnector, tonic's custom-connector path. Only these hand-rolled sockets need it. Tonic's own connector defaults tcp_nodelay to true and applies it itself, so plain Endpoint::connect callers were already covered. Signed-off-by: Jim Meyer <jim@meyer4hire.com> --------- Signed-off-by: Jim Meyer <jim@meyer4hire.com> |
||
|
Matthew Grossman
|
537805568d |
feat(sandbox): honor OCI image working directories (#2530)
* feat(sandbox): honor Docker OCI working directories Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): honor effective workspace access Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(sandbox): cover enforced workspace denial Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(docker): explain effective workdir checks Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): validate effective workspace writes Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): reserve supervisor control roots Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(sandbox): centralize control paths Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): reserve OCI runtime mount roots Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Polite_realismandAdam Miller
|
0e9a44cfa9 |
feat(build): add system CA root mode (#2324)
* feat(build): add system CA root mode Allow distro builds to use native trust stores for supervisor upstream TLS while keeping bundled Mozilla roots as the default. Avoid bundled root crates in system-ca-roots builds by using native-root TLS features and z3 0.20. Signed-off-by: Adam Miller <admiller@redhat.com> * fix(build): keep CA root feature in telemetry-off verification The telemetry-off task uses --no-default-features which now disables bundled-ca-roots in addition to telemetry, triggering the compile_error guard. Re-enable bundled-ca-roots explicitly so the task verifies only telemetry compilation. Signed-off-by: Scott Burdine <sburdine@nvidia.com> Signed-off-by: politerealism <burdcat17@gmail.com> * fix(sdk): disable oauth2 default features to prevent webpki-roots leak The bare `oauth2 = "5"` dependency re-enabled default features (rustls-tls → reqwest/rustls-tls → webpki-roots), defeating the system-ca-roots feature gate. Mirror the CLI fix: disable defaults and enable only the `reqwest` feature. Signed-off-by: Quinn Burdine <sburdine@redhat.com> Signed-off-by: politerealism <burdcat17@gmail.com> * refactor(build): simplify CA root selection to single feature toggle Replace mutually exclusive bundled-ca-roots / system-ca-roots features with a single bundled-ca-roots toggle. Disabling it implies system roots via rustls-native-certs, which is now a regular (non-optional) dependency. This fixes cargo --all-features and simplifies the distro build interface from --no-default-features --features system-ca-roots to just --no-default-features. Signed-off-by: Quinn Burdine <sburdine@redhat.com> Signed-off-by: politerealism <burdcat17@gmail.com> * feat(build): add system-ca-roots convenience alias and fix verify task Add a system-ca-roots feature alias on openshell-sandbox that includes all other defaults (telemetry) except bundled-ca-roots, so distro builds can use --no-default-features --features system-ca-roots without manually re-adding unrelated defaults. Update the verify CI task to use the alias and scope checks to the sandbox package. Fix task description to use "build mode" terminology instead of implying a Cargo feature. Signed-off-by: Quinn Burdine <sburdine@redhat.com> Signed-off-by: politerealism <burdcat17@gmail.com> * ci: fix system CA roots step name to use build mode terminology Signed-off-by: Quinn Burdine <sburdine@redhat.com> Signed-off-by: politerealism <burdcat17@gmail.com> * refactor(sandbox): reorder features to place system-ca-roots alias near default Signed-off-by: Quinn Burdine <sburdine@redhat.com> Signed-off-by: politerealism <burdcat17@gmail.com> * fix(proxy): unwrap Result from build_upstream_client_config in tests The function signature changed to return Result but the test call sites were not updated, causing type mismatch compilation errors in CI. Signed-off-by: Quinn Burdine <sburdine@redhat.com> Signed-off-by: politerealism <burdcat17@gmail.com> --------- Signed-off-by: Adam Miller <admiller@redhat.com> Signed-off-by: Scott Burdine <sburdine@nvidia.com> Signed-off-by: politerealism <burdcat17@gmail.com> Signed-off-by: Quinn Burdine <sburdine@redhat.com> Co-authored-by: Adam Miller <admiller@redhat.com> |
||
|
Simon Scatton
|
b9818619b8 |
test: disable tests flaky under parallel stress (#2611)
Repeated Bazel test targets with --runs_per_test at 10, 20, 50, 100, and 200 runs. The tests failed intermittently when multiple instances ran concurrently. This indicates timing, shared tracing state, socket readiness, or parallel-safety issues that need focused follow-up before re-enabling the tests. A normal non-repeated bazel test //... run passes with these tests ignored. Disabled tests: - sandbox_forward_foreground_fails_when_ssh_exits_before_listener_opens - sandbox_forward_background_terminates_owned_child_when_listener_never_opens - podman_socket_probe_accepts_successful_ping_response - podman_socket_probe_rejects_docker_ping_response - docker_socket_probe_accepts_successful_ping_response - docker_socket_probe_rejects_podman_ping_response - docker_socket_detection_returns_the_responsive_candidate - podman_socket_detection_returns_the_responsive_candidate - driver_watch_events_are_roots_and_store_operations_have_parents - reconcile_sweeps_are_roots_and_operations_have_parents - gateway_listeners_bind_ipv6_wildcard_and_ipv4_callback_on_same_port - watch_producer_releases_request_span_when_client_disconnects - expected_conflicts_leave_the_span_unmarked - store_spans_record_what_they_touched_as_attributes - store_operations_export_spans_with_parents - refresh_worker_ticks_are_roots_and_store_operations_have_parents Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Simon Scatton
|
0a3ec7a112 | feat(bazel): add rustfmt checks to tests (#2599) | ||
|
Simon Scatton
|
1959ea19be |
build(bazel): establish RFC 0012 Rust reference graph (#2414)
Establish the Phase 1 reference implementation proposed by RFC 0012 while retaining the existing Cargo and Mise workflows during evaluation. - pin Bazel 9.1.1 and configure Bzlmod, rules_rs, LLVM, protobuf, and Rust 1.95 toolchains - import third-party crates from Cargo metadata and propagate the workspace version into Bazel targets - add library, binary, proc-macro, unit-test, and integration-test targets across the supported Rust workspace crates and drivers - generate protobuf Rust sources and descriptor sets under Bazel while preserving Cargo-compatible generated-code imports - annotate aws-lc-sys and zstd-sys native dependencies, build Z3 4.15.2 from source, and generate z3-sys bindings - make CLI and procfs test fixtures available as explicit Bazel inputs without relying on fixed host binary paths - define optimized release targets for Linux x86_64 and aarch64 CLI, sandbox, and gateway binaries, plus macOS aarch64 artifacts - add Bazel, buildifier, and lcov to the Nix development environment RFC: 0012 (rfc12 branch) Refs: #2491 Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
John T. MyersandJohn Myers
|
905b554c7c |
refactor(network): consolidate proxy egress pipeline (#2373)
* refactor(network): introduce shared egress pipeline Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): cover shared proxy egress paths Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * refactor(network): make destination authorization explicit Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * refactor(network): pin proxy relay policy context Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): lock relay generation contracts Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): establish phase zero compatibility baseline Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * feat(policy): detect ambiguous network endpoints Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * feat(sandbox): fail closed on invalid policy updates Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * refactor(network): invalidate relays on policy changes Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(policy): document validation failure posture Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): cover validation and middleware egress Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(config): move policy failure mode to gateway toml Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): name proxy contracts by behavior Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(network): align overlap validation with endpoint selection Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): expect hard loopback denial Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): match declared endpoint denial Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(policy): preserve path-specific endpoint overrides Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(network): respect hard-blocked host gateways Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(network): reconcile proxy refactor with main Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(network): preserve CONNECT policy generation Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(policy): cover runtime endpoint glob semantics Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(server): reject ambiguous policies before persistence Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * docs(policy): explain ambiguity preflight behavior Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(policy): compare body limits within protocol Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(proxy): avoid global tracing capture race Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * chore(server): format rebased provider tests Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(server): authenticate rebased policy requests Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(sandbox): retain runtime on middleware outage Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(server): preflight provider composition activation Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(sandbox): distinguish runtime failure transitions Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> Signed-off-by: John Myers <johntmyers@users.noreply.github.com> Co-authored-by: John Myers <johntmyers@users.noreply.github.com> |
||
|
Russell Bryant
|
0cecb54249 |
fix(cli): bracket IPv6 bind literals in SSH forwards (#2552)
ForwardSpec accepts IPv6 bind addresses, but ssh_forward_arg() emitted them unbracketed (e.g. ::1:8080:127.0.0.1:8080), which OpenSSH rejects as a bad local forwarding specification. access_url() likewise produced invalid URLs like http://::1:8080/. Extract a shared bracket_ipv6_host() helper and use it in ssh_forward_arg(), access_url() (via format_gateway_url), and format_gateway_url() so IPv6 literals are bracketed consistently. Fixes #2279 Signed-off-by: Russell Bryant <rbryant@redhat.com> |
||
|
Matthew Grossman
|
bc14018cad |
feat(sandbox): use policy-first OCI image identity (#2509)
* feat(sandbox): use policy-first OCI image identity Closes #2331 Preserve per-field policy omission, derive Docker and Podman fallbacks from the inspected immutable image, and resolve the final numeric identity before starting agent children. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): preserve declared process identities Keep explicit policy values and OCI-declared names intact, defer passwd lookup until a primary GID is required, and refresh stale policy examples. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(supervisor): reuse resolved OCI identity Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(supervisor): allow Linux pre-exec arguments Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(kubernetes): protect resolved sandbox identity Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): prepare workspace for OCI identity Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(sandbox): own only workspace root Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): harden partial identity drops Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(sandbox): scope OCI image e2e to Docker Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(sandbox): narrow OCI identity fallback scope Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(podman): cover OCI identity launch Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(podman): exercise OCI fallback in E2E Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> |
||
|
Evan Lezar
|
f00ad23a26 |
fix(podman): tolerate shutdown transport closes (#2498)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Andrew White
|
76a5397e6f |
fix: assorted arithmetic and indexing robustness fixes (#2451)
* fix(core): avoid i64 underflow on expired GCP token When a cached GCP access token was already expired (expires_at_ms < now), the remaining lifetime calculation used plain subtraction: (expires_at_ms - now) / 1000. This underflows in debug builds and wraps in release builds. Use saturating_sub so an expired token simply yields a non-positive remaining lifetime and is skipped. Add a regression test. Signed-off-by: Andrew White <andrewh@cdw.com> * fix(server): avoid i64 underflow in lease age calculation The lease steal path computed age as now_ms() - record.updated_at_ms. If the stored updated_at timestamp is in the future (e.g. clock skew), the subtraction underflows. Use saturating_sub to treat a future timestamp as age 0, keeping the lease considered fresh. Signed-off-by: Andrew White <andrewh@cdw.com> * fix(server): avoid i64 overflow on large credential max lifetime max_lifetime_seconds is only validated as >= 0. The code multiplied the raw i64 value by 1000 to compute max_expires, which overflows for values above i64::MAX / 1000. The capped i32 value used for the STS request was already computed but not reused. Compute max_lifetime_ms from the capped value with saturating_mul and use it for both expires_at fallback and max_expires. Signed-off-by: Andrew White <andrewh@cdw.com> * fix(server): avoid hardcoded indexing into base_url_config_keys resolve_vertex_ai_route assumed every inference provider profile has at least two base URL config keys by indexing [0] and [1]. Only the Vertex profile satisfies that today; other profiles would panic here. Iterate base_url_config_keys with find_map instead, and add a test with an empty key list to ensure no panic. Signed-off-by: Andrew White <andrewh@cdw.com> * fix(server): clamp lease age at zero for future timestamps Address review feedback: add regression coverage for future-timestamp lease clock skew. Extract lease_is_expired() and clamp the age at zero — i64::saturating_sub saturates at i64::MIN, not zero, so a future updated_at_ms must be clamped explicitly to be treated as age zero and remain unstealable with a positive TTL. Signed-off-by: Andrew White <andrewh@cdw.com> * fix(server): fall back past blank preferred Vertex base URL Address review feedback: find_map stopped at the first present key even when its value was blank, and the outer filter discarded it without considering lower-priority aliases. Filter blank values inside the closure so a blank preferred key falls back to a valid alias, matching the documented priority order. Add a regression test with a blank preferred key and a valid fallback. Signed-off-by: Andrew White <andrewh@cdw.com> * fix(server): saturate AWS STS expiry cap on saturated clock Address review feedback: now_ms + max_lifetime_ms could still overflow when current_time_ms() saturates near i64::MAX. Compute max_expires = now_ms.saturating_add(max_lifetime_ms) once and use it for both the conversion fallback and the expiry cap. Add a boundary unit test covering a saturated clock. Signed-off-by: Andrew White <andrewh@cdw.com> --------- Signed-off-by: Andrew White <andrewh@cdw.com> |
||
|
Philippe Martin
|
77e5c32217 |
feat(sandbox,gateway): route sandbox egress through corporate HTTP proxy (#2245)
* feat(sandbox,gateway): route sandbox egress through corporate HTTP proxy - Chain sandbox egress through a corporate HTTP proxy so outbound traffic from within the sandbox respects the host proxy settings - Forward sandbox proxy environment variables to the generated Podman config so the proxy is applied consistently to Podman-managed workloads Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): make corporate proxy routing operator-owned The operator-configured corporate egress proxy was injected under the conventional HTTPS_PROXY/HTTP_PROXY/NO_PROXY names as defaults beneath sandbox spec/template environment, so a sandbox creator could redirect egress at an arbitrary proxy or disable proxying with NO_PROXY=*. Route the boundary through reserved, supervisor-only variables (OPENSHELL_UPSTREAM_HTTPS_PROXY/HTTP_PROXY/NO_PROXY) written in the Podman driver's required-variable tier. Any sandbox-supplied value under a reserved name is stripped before the operator value is applied, so the supervisor never observes a reserved proxy variable the operator did not set. The supervisor now reads only the reserved names and ignores the conventional proxy variables the sandbox controls. Add the reserved proxy variables to the supervisor-only child-environment denylist so the corporate proxy URL and any embedded credentials are not inherited by the sandbox workload, which reaches egress through the local policy proxy and never needs them. Signed-off-by: Philippe Martin <phmartin@redhat.com> * feat(sandbox,podman): deliver corporate proxy credentials via secret file Proxy credentials were embedded inline in the proxy URL, so they were stored in gateway.toml and exposed in container metadata via 'podman inspect'. Reject inline 'user:pass@' credentials in https_proxy/http_proxy at startup (parsed with the url crate rather than hand-splitting), and add a proxy_auth_file option pointing at a 'user:pass' file. The driver stages that file as a per-sandbox root-only Podman secret, mounts it at a fixed path, and exports only the path in the reserved OPENSHELL_UPSTREAM_PROXY_AUTH_FILE variable, so the credential never appears in config, environment, or container metadata. Reading the file fails closed on a missing, empty, or control-character-bearing value. The supervisor reads the credential from the mounted file and builds the Proxy-Authorization: Basic header, rejecting control characters, and no longer derives credentials from URL userinfo. The auth-file path is added to the child-environment strip list, and the generated gateway.toml is written owner-only (mode 0600). Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): fail closed on invalid upstream proxy configuration The reserved OPENSHELL_UPSTREAM_* variables are an operator-owned egress boundary, but the supervisor treated present-but-invalid values as unset: an unsupported or malformed proxy URL was ignored with a warning, an unreadable auth file proceeded without credentials, and a malformed credential silently became unauthenticated. Any of these could quietly downgrade the corporate proxy boundary to direct dialing or unauthenticated proxy access. Make every configured-but-invalid proxy or auth setting fatal to supervisor proxy startup, emit an OCSF ConfigStateChange failure event before refusing, and share URL validation semantics between the Podman driver and the supervisor through a single validator in openshell-core (parse_upstream_proxy_url), so a value accepted at sandbox-create time can never be rejected in-container or vice versa. Inline user:pass@ URL credentials are now fatal in the supervisor too (previously warn-and-strip), matching the driver. Unset or empty variables still mean no proxy; only present-but-invalid values fail. Error paths never include credential content. Addresses the fail-closed review item on #2245. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): finish the fail-closed upstream proxy credential contract The upstream proxy URL already had a single shared validator, but the credential did not: the Podman driver rejected only CR/LF/NUL while the supervisor rejected every control character, so a credential accepted at sandbox-create time (e.g. one containing a tab) could still be rejected in-container. Present-but-whitespace reserved OPENSHELL_UPSTREAM_* values were also silently treated as unset, quietly downgrading the operator's egress boundary to direct dialing. Add parse_upstream_proxy_credential to openshell-core as the single source of truth for the documented user:pass credential form (non-empty user, no control characters, trimmed) and use it in both the Podman driver's secret staging and the supervisor's Proxy-Authorization header construction. Error variants carry no payload so credential content can never leak into messages. Make a present-but-empty reserved variable fatal to supervisor proxy startup instead of meaning "unset"; only fully unset variables disable the proxy. The driver correspondingly rejects an empty no_proxy at config time so it can never inject a value the supervisor refuses. Addresses the remaining fail-closed credential/config review item on Signed-off-by: Philippe Martin <phmartin@redhat.com> #2245. * fix(sandbox,podman): close remaining fail-open upstream proxy config paths Two configuration paths could still silently run without the proxy boundary the operator believed was in effect. A no_proxy bypass list configured without any https_proxy/http_proxy was accepted by both the driver and the supervisor and simply meant "dial everything directly". Reject it on both sides, exactly like the existing proxy_auth_file-without-proxy rule: an operator who wrote a bypass list assumed proxying was active, so accepting it hides a fail-open state. The gateway.sh dev script guarded proxy settings with [[ -n "${VAR:-}" ]], which conflates unset with explicitly-empty and dropped the latter before the gateway's validation could see it. Use ${VAR+x} instead so a set-but-empty variable is written into gateway.toml and rejected at startup by validate_proxy_config rather than silently discarded. Addresses the remaining fail-open configuration review item on #2245. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): reject upstream proxy URLs with path, query, or fragment parse_upstream_proxy_url accepted URLs like http://proxy.corp.com:8080/some/path and silently discarded everything after host:port, a lenience inherited from the original supervisor parser. A forward proxy is addressed by host:port only, so extra components indicate a misconfiguration (for example a pasted endpoint URL) and silently truncating them violates the present-but-invalid-is-fatal contract enforced everywhere else in this configuration surface. Reject a path, query, or fragment in the shared validator with a new UnexpectedComponent error. A bare trailing slash remains accepted because the url crate normalizes an absent http path to "/", making the two indistinguishable. Both the Podman driver (gateway startup) and the supervisor (sandbox startup) inherit the rule through the shared parser, keeping their semantics identical by construction. Addresses the proxy URL component review item on #2245. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): remove plain-HTTP upstream proxy support The http_proxy path tunneled plain-HTTP requests through the corporate proxy with CONNECT to port 80 and then sent origin-form requests down the tunnel. Conventional enterprise forward proxies expect plain HTTP as absolute-form requests sent directly over the proxy connection, and commonly refuse CONNECT to port 80, so the setting looked supported but failed against typical deployments. Tunneling also blinds the proxy to the one protocol it could inspect. Narrow the feature to TLS (CONNECT) egress only, which is the conventional and already-correct case: plain-HTTP requests now always dial the destination directly, and only client CONNECT tunnels chain through the corporate proxy. Remove the http_proxy config field, the --sandbox-http-proxy / OPENSHELL_SANDBOX_HTTP_PROXY driver surface, the reserved OPENSHELL_UPSTREAM_HTTP_PROXY variable, and the UpstreamScheme plumbing. The feature never shipped, so this is a clean removal; a stray http_proxy key in gateway.toml still fails loudly through the config's deny_unknown_fields. Removing the plain-HTTP proxy branch also removes its host-gateway special case; the architecture doc now documents the real host-gateway behavior (add driver-injected host aliases to the reserved NO_PROXY list) instead of an invariant the HTTPS path never implemented. Plain-HTTP forwarding through a corporate proxy can return later as absolute-form forwarding behind its own design review. Addresses the plain-HTTP forwarding review item on #2245. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): escape generated TOML and require explicit proxy URL form Escape backslashes, quotes, and control characters when gateway.sh writes proxy values into gateway.toml, so a hostile or unusual environment value cannot corrupt the config or inject extra keys. Restrict the upstream proxy URL grammar to the documented http://host:port form: a scheme-less value is no longer normalized to http:// and a missing port is no longer silently defaulted to 80. Docs, README, and CLI help now state the explicit-form requirement consistently. Also fix a test-only call of handle_tcp_connection that was missing the upstream_proxy argument added in an earlier commit. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): preserve tunneled bytes read with the CONNECT response The CONNECT handshake reads the corporate proxy's response in chunks, so the read that completes the header block can also contain the first tunneled payload bytes. Those bytes were discarded, silently corrupting the start of the tunnel for server-speaks-first destinations or proxies that coalesce writes. connect_via now returns a PrefixedStream that replays any bytes received past the response terminator before reading from the socket again; writes pass through unchanged. Direct dials wrap the stream with an empty prefix so downstream relay and TLS paths keep a single stream type, and tls_connect_upstream is generalized to any AsyncRead + AsyncWrite stream. Adds regression coverage for a combined response/payload read and for prefix replay ordering. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): gate cleartext proxy Basic auth behind an explicit opt-in Proxy-Authorization: Basic is base64 over the plain-TCP connection to the http:// corporate proxy, so anyone on the network path between the sandbox host and the proxy can recover the credential. Sending it is now an explicit operator decision instead of an implicit side effect of configuring proxy_auth_file. Add a proxy_auth_allow_insecure driver setting (CLI --sandbox-proxy-auth-allow-insecure, env OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE), delivered to the supervisor as the reserved OPENSHELL_UPSTREAM_PROXY_AUTH_ALLOW_INSECURE variable. Fail-closed pairing on both sides: an auth file without the acknowledgement is rejected at gateway startup and at supervisor startup, as is the acknowledgement without an auth file or any value other than 'true'. gateway.sh writes the key only as a TOML boolean; a non-boolean value is emitted as a quoted string so the gateway rejects it at startup instead of risking injection. Documents the exposure prominently in the gateway config reference, the driver README, and the sandbox architecture doc. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): honor port qualifiers and resolved addresses in NO_PROXY Two divergences from the documented NO_PROXY contract: - A port-qualified entry (internal.corp:8443) was silently stripped to its hostname and bypassed the proxy for every port, excluding traffic the operator never listed. Entries now keep the optional :port qualifier (also on IP and CIDR entries) and only apply to that destination port; a trailing qualifier that is not a valid port stays part of the pattern instead of widening the entry. - IP and CIDR entries only matched IP-literal hosts, so a bypass like 10.0.0.0/8 never applied to hostnames resolving into that range. NO_PROXY evaluation now sees the validated resolved addresses: an IP/CIDR entry matching through resolution authorizes a direct dial of only the addresses it contains, so a bypass scoped to an internal range cannot widen into a direct dial of addresses outside it. Hostname-level matches (loopback, wildcard, domain entries, IP-literal hosts) keep authorizing all validated addresses. proxy_for is replaced by decision(host, port, resolved) returning either the proxy endpoint or the permitted direct-dial subset, and dial_upstream restricts the direct connect to that subset. Adds regression coverage for port-scoped bypasses on domain, IP, and CIDR entries, invalid port qualifiers, resolved-address matching, and split-resolution subset dialing. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): bind proxied CONNECT tunnels to validated addresses The CONNECT request sent to the corporate proxy carried the destination hostname, so the proxy resolved the name itself and the addresses that had passed SSRF and allowed_ips validation were discarded. Split-horizon DNS or rebinding at the proxy could then reach internal or otherwise unapproved destinations through a tunnel the supervisor logged as validated, and IP-range policy could not be enforced at all on proxied dials. CONNECT now targets a validated resolved address by default: the proxy performs no DNS resolution and the tunnel stays bound to the answer the supervisor checked. The hostname still travels inside the tunnel (TLS SNI, application Host), so destination servers behave normally. In split-horizon networks, operators point the gateway host at the corporate resolver so internal names validate to their internal addresses. For proxies whose ACLs filter on hostnames and reject IP CONNECT targets, a new proxy_connect_by_hostname opt-in (CLI --sandbox-proxy-connect-by-hostname, env OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME, reserved OPENSHELL_UPSTREAM_PROXY_CONNECT_BY_HOSTNAME) restores hostname CONNECT, documented as re-opening proxy-side resolution and making the proxy's ACLs the effective egress control. Fail-closed pairing on both sides: the opt-in without a proxy, or any value other than 'true', is fatal. Adds regression coverage for the IP CONNECT request line (including IPv6 bracketing and hostname non-leakage), the hostname opt-in, and the config pairing rules in driver and supervisor. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): reject empty port after bracketed IPv6 proxy host http://[fd00::1]: passed the explicit-port check because the bracketed branch only tested for a colon after the bracket, then fell back to port 80 — violating the fail-closed http://host:port contract and potentially sending configured Basic credentials to an unintended service. Require a non-empty suffix after ]:, matching the unbracketed branch, and cover the case in the shared parser tests. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): fall back across validated addresses in proxied CONNECT The direct path hands TcpStream::connect the whole validated address list and it falls back across them, but the validated-IP CONNECT path attempted only the first address, so a dual-stack destination could fail through the corporate proxy even when a later validated address was reachable. connect_via_validated tries each validated address in order under one aggregate CONNECT_HANDSHAKE_TIMEOUT budget, returning the first success; when every attempt fails the error names the attempt count and carries the last failure. An empty address list is rejected up front. Adds regressions for first-fails/second-succeeds fallback (asserting both CONNECT request lines), the aggregate all-addresses failure message, and the empty-list rejection. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): strip new reserved proxy vars and complete docs/tests Follow-ups from review: - Add OPENSHELL_UPSTREAM_PROXY_AUTH_ALLOW_INSECURE and OPENSHELL_UPSTREAM_PROXY_CONNECT_BY_HOSTNAME to the supervisor-only strip list so workload child processes never inherit them, matching the documented contract for the other reserved proxy variables, and cover both in the supervisor-only variable test. - Document all five OPENSHELL_SANDBOX_* proxy variables in the mise run gateway help text, marked Podman-only and stating the auth-file/acknowledgement pairing, and complete the gateway-key list in the Podman README. - Add NO_PROXY composition coverage for bracketed IPv6 entries with port qualifiers, bare IPv6 entries, and IPv6 CIDR matching against an IPv6-literal host and against a hostname's resolved addresses, including the port-qualified CIDR form. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): cap each proxied CONNECT attempt within the shared budget A proxy that accepted the first CONNECT request but never responded consumed the entire aggregate handshake timeout, so later validated addresses were never tried and the hang defeated the multi-address fallback. Each attempt is now time-boxed to its fair share of the time remaining before the shared deadline (remaining / attempts_left): a hanging attempt is cut off with enough budget left for every remaining address, while time a fast failure does not use rolls over to later attempts and the total never exceeds CONNECT_HANDSHAKE_TIMEOUT. A timed-out attempt is recorded like any other failure, and the aggregate error distinguishes all-attempted from budget-exhausted runs. Adds a first-hangs/second-succeeds regression driven through a test-visible budget parameter so it runs in about a second instead of a real 30s window. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): deliver corporate proxy config on the supervisor argv The proxy settings were injected as reserved OPENSHELL_UPSTREAM_* container environment variables. The driver only wrote names the operator configured, but container runtimes layer the spec environment over ENV values baked into the sandbox image, so an image could supply NO_PROXY=*, enable hostname CONNECT, or point an unconfigured deployment at an attacker-controlled proxy whenever the operator left a field unset. The settings now travel as supervisor command-line arguments (--upstream-proxy, --upstream-no-proxy, --upstream-proxy-auth-file, --upstream-proxy-auth-allow-insecure, --upstream-proxy-connect-by-hostname) built by the driver from operator config. The driver sets the container entrypoint and command explicitly, so neither sandbox spec/template environment nor image ENV can influence argv, and an omitted flag genuinely means unconfigured — in every supervisor topology, since the supervisor no longer consults its environment for these settings at all. Credentials stay on the root-only secret mount; only the mount path appears on argv. The reserved environment names, their strip-list entries, and the env-based validation surface are removed. UpstreamProxyConfig::from_args replaces from_env, reusing the same shared fail-closed validation and pairing rules keyed by the CLI flag names. Driver tests now assert the argv contract, including that sandbox-supplied environment cannot add, remove, or redirect proxy flags; supervisor tests cover from_args mapping and its pairing rules. Signed-off-by: Philippe Martin <phmartin@redhat.com> * docs(sandbox): align proxy comments with the argv transport The argv migration left comments describing the configuration as reserved environment variables ("reserved value", "present-but-empty variable", "reserved upstream proxy variables"). Rephrase them as driver-supplied arguments and operator settings so the documented trust boundary matches the implementation. Comment-only change. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox): parse bracketed IPv6 authorities in client CONNECT targets parse_target split the CONNECT authority at the first colon, so an IPv6-literal target like [2001:db8::1]:443 always failed port parsing and IPv6-literal clients could never reach policy evaluation; a regression test even locked in that failure. Parse the RFC 3986 bracketed form and return the host bracket-free, matching what DNS resolution, SSRF validation, NO_PROXY matching, and the upstream CONNECT builder expect. Unclosed brackets, a missing or empty port after the bracket, and non-numeric ports are rejected; unbracketed behavior is unchanged. Replaces the failure-locking test with success coverage for bracketed targets and adds malformed-bracket rejection cases. Signed-off-by: Philippe Martin <phmartin@redhat.com> * test(podman): cover proxy-auth secret cleanup across lifecycle failures The per-sandbox proxy-auth credential secret is staged before the container is created and removed on cleanup, but no test proved the cleanup paths actually issue the secret removal. Add Podman-stub tests that drive create_sandbox to a container-create failure and to a start failure, and delete_sandbox for an out-of-band deletion, asserting each path issues the DELETE for the per-sandbox proxy-auth secret so a credential can never outlive the sandbox that owned it. Signed-off-by: Philippe Martin <phmartin@redhat.com> * docs: list corporate proxy keys in the Podman compute-driver overview The Fern Podman driver section enumerated its gateway.toml keys but omitted the corporate egress proxy settings. Add https_proxy, no_proxy, proxy_auth_file, proxy_auth_allow_insecure, and proxy_connect_by_hostname with a pointer to the gateway configuration reference for the full contract. No navigation change: the reference folder already includes the gateway configuration page. Signed-off-by: Philippe Martin <phmartin@redhat.com> * test(sandbox): cover the SSRF-to-TLS composition across the proxy tunnel Existing tests exercised validated-IP CONNECT and the upstream-TLS helper independently, but not the full boundary. Add an end-to-end regression that stands up a fake corporate proxy tunneling to a fake TLS server and drives the real path: connect_via_validated CONNECTs to the validated address, the proxy splices the tunnel, and tls_connect_upstream verifies the upstream certificate against the original hostname carried in SNI. It asserts the CONNECT authority is the validated IP and never the hostname, that verification succeeds for the matching hostname, and that a mismatched hostname is rejected — proving a rebinding or split-horizon substitution behind the proxy cannot pass certificate verification. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox,podman): bound proxy-auth reads, reject port 0, fix stale comment Three review findings: - CWE-400: the proxy-auth credential file was read with an unbounded read_to_string on both the driver (sandbox-create) and supervisor (startup) paths, so a huge file or a special file such as /dev/zero could exhaust memory. Add a shared bounded reader in openshell-core that rejects non-regular files, caps the size at 4 KiB, and reads at most that many bytes; the driver runs it via spawn_blocking. Covers oversized, special-file, and missing-path cases on both sides. - Reject an upstream proxy URL with port 0: it passed the explicit-port check and startup validation but is not a connectable TCP port, so every proxied dial would fail later. Add a typed ZeroPort error with shared-validator and Podman-config tests. - Reword a driver-config comment that still described a 'reserved variable' to match the argv transport. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(sandbox): open proxy-auth file non-blocking to reject FIFOs promptly read_upstream_proxy_credential_file opened the path with a blocking File::open before the regular-file check, so a configured FIFO with no writer would block open() indefinitely — hanging sandbox creation on the driver and supervisor startup. Open with O_NONBLOCK on Unix so the open returns immediately, then reject the non-regular file as before; O_NONBLOCK has no effect on the later read of a regular file. Adds a mkfifo regression asserting the reader returns promptly with a non-regular-file error instead of hanging. Signed-off-by: Philippe Martin <phmartin@redhat.com> * test(podman): cover corporate proxy egress across driver and supervisor The existing corporate-proxy tests construct config structs or call CONNECT helpers directly, so none of them detect a break in the wiring between layers: gateway TOML deserialization, the Podman argv and secret-mount semantics, supervisor CLI parsing, or policy denial before proxy contact. Add a Podman e2e that drives the whole chain against a fake authenticated forward proxy and asserts that an approved TLS request traverses it with a validated-IP CONNECT, a policy-denied destination is refused with 403 without ever reaching the proxy, credentials arrive through the mounted per-sandbox secret, and deleting the sandbox removes that secret. SupportContainer is a new harness fixture. Unlike ContainerHttpServer it probes readiness with a TCP connect rather than an HTTP GET, so it can host a forward proxy and TLS servers, and it exposes container logs and network IP for assertions. Signed-off-by: Philippe Martin <phmartin@redhat.com> * test(podman): restart the gateway on the proxy-config panic path The panic cleanup for the temporary corporate proxy configuration restored the gateway TOML but left the gateway process running with the temporary configuration still loaded, which could poison later test binaries in the same run. Nothing restarted it: the only ManagedGateway is the short-lived one inside restart_gateway, and its Drop only calls start, which does not reload config for an already-running gateway. Restore and synchronously stop/start the gateway in Drop, and set restored only after the normal restore and restart both succeed so a failed restart no longer suppresses the fallback. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(supervisor-network): derive the loopback proxy bypass from resolved IPs The automatic bypass treated the host string "localhost" as proof that the destination was loopback and returned every resolved address. A sandbox controls its own /etc/hosts and resolve_socket_addrs consults it before DNS, so a workload could map localhost to any policy-allowed address and dial it directly, escaping the operator proxy and the inspection and audit boundary it exists to provide. Check the resolved addresses instead: the name bypasses only when the resolution is non-empty and every address is loopback. A mixed answer is not partially honored, and an IP literal is still authoritative for itself. A spoofed localhost falls through to the entries below, so an explicit operator NO_PROXY entry is still honored. This matches the trust model detect_trusted_host_gateway already applies to the same hosts file, which validates the mapped address rather than the alias. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(podman): clean up proxy-auth secret when container already deleted The delete_sandbox early-return path cleaned up the token secret but skipped the proxy-auth secret, leaking it on disk. Also update tests for recent API changes (Optional socket_path, workspace field, list-based container lookup). Signed-off-by: Philippe Martin <philippe@openshell.dev> Signed-off-by: Philippe Martin <phmartin@redhat.com> --------- Signed-off-by: Philippe Martin <phmartin@redhat.com> Signed-off-by: Philippe Martin <philippe@openshell.dev> |
||
|
Evan Lezar
|
21da343c9f |
refactor(supervisor): pass agent proposal state explicitly (#2421)
* test(supervisor-network): default L7 eval context in tests Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(supervisor): pass agent proposal state explicitly Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
alangou
|
1d4ac708f1 |
fix(policy): keep internal allowed IP proposals pending (#2416)
Signed-off-by: Adrien Langou <alangou@nvidia.com> |
||
|
Roland Huss
|
cd9a0bf214 |
fix(driver-k8s): add label selector to sandbox watch stream and list (#2223)
* fix(driver-k8s): harden list path and tighten label selectors - Add warning log to list_sandboxes() filter_map so skipped objects are visible in logs instead of silently dropped. - Add LABEL_SANDBOX_ID existence requirement to both list and watch label selectors so managed-but-non-sandbox resources (e.g. warm pool pods without sandbox identity) never enter either code path. Signed-off-by: Roland Huß <rhuss@redhat.com> * fix(driver-k8s): add watch cancellation and selector tests Address review feedback on PR #2223: - Extract shared label selector into openshell_sandbox_label_selector() helper in openshell-core so both list and watch paths use a single tested definition. - Add tx.closed() branch to the watch_sandboxes tokio::select! loop so the producer task exits when the receiver is dropped, preventing leaked Kubernetes watch connections. - Add unit tests for the label selector string and for the cancellation behavior. Signed-off-by: Roland Huß <rhuss@redhat.com> * fix(driver-k8s): remove tautological and non-functional tests Remove three tests that don't provide meaningful regression coverage: - openshell_sandbox_label_selector_contains_managed_by_and_sandbox_id: mirrors the implementation with hardcoded strings; any constant change requires updating the test string in lockstep. - label_selector_used_by_list_and_watch_matches_shared_helper: weaker duplicate of the above, checking substrings instead of equality. - watch_producer_exits_when_receiver_is_dropped: tests tokio channel semantics (tx.closed() fires when rx is dropped), not watch_sandboxes(). No K8s API mock infrastructure exists to test the actual watch path. Signed-off-by: Roland Huß <rhuss@redhat.com> * style(driver-k8s): fix rustfmt formatting Signed-off-by: Roland Huß <rhuss@redhat.com> * fix(driver-k8s): use explicit unit pattern for clippy 1.95 Replace `_ = tx.closed()` with `() = tx.closed()` to satisfy the `ignored_unit_patterns` clippy lint introduced in Rust 1.95. Assisted-By: 🤖 Claude Code Signed-off-by: Roland Huß <rhuss@redhat.com> --------- Signed-off-by: Roland Huß <rhuss@redhat.com> |
||
|
Mesut Oezdil
|
744a65d52e |
fix(driver-podman): avoid panic when HOME is unset on macOS (#2327)
* fix(driver-podman): resolve Podman socket via auto-detection Align Podman socket selection with the existing Docker model: explicit config wins, otherwise probe openshell-core for a responsive socket. This also fixes the original HOME-unset panic, since resolution no longer hardcodes a per-OS default path. - add detect_podman_socket() in openshell-core, mirroring detect_docker_socket - PodmanComputeConfig.socket_path is now Option<PathBuf>, no default - remove default_socket_path() (podman driver) and podman_socket_path() (vm driver), both replaced by the shared detector - update server env override and CLI for the new Option type - add tests: responsive-candidate detection in openshell-core, and config-error (not panic) when no socket is configured or reachable * fix(driver-podman): address review feedback on socket resolution Extract socket resolution into resolve_socket_path, taking the detector as a parameter so tests do not depend on real env vars or the host's actual Podman state. Replace the flaky env-mutating test with three deterministic cases: explicit wins, detected is used when absent, and neither source errors. Fix the socket_path doc comment to describe it from a config user's point of view, matching DockerComputeConfig's docstring. Drop a comment that only made sense next to the Docker driver code. Update the Podman README and gateway docs: they described a fixed per-OS default path that no longer exists, replace with the actual probe-then-fail behavior. * docs(driver-podman): simplify socket default description Previous wording was self-contradictory (says auto-detect on unset, then lists the same var as a probed candidate) and omitted the Linux /run/user/uid/podman/podman.sock candidate. |
||
|
Derek Carr
|
5952a5a23f |
feat(workspace): add workspace resource model with scoping, membershi… (#2243)
* feat(workspace): implement workspace model (Phase 1 of RFC 0011) Implements workspace and membership model providing hard isolation boundaries for multi-player OpenShell deployments. Workspace CRUD with Kubernetes-style Terminating phase for graceful deletion. All resources scoped by workspace via ObjectMeta. Membership RPCs for workspace access control. Persistence migration shifts name uniqueness to (object_type, workspace, name). Provider profiles support platform and workspace scoping. Service routing uses workspace-prefixed DNS labels. Inference routes renamed and workspace-scoped with DeleteInferenceRoute RPC. Python SDK with WorkspaceClient, two-method list pattern (workspace-scoped and for_all_workspaces), and workspace parameter on all methods. CLI workspace flags, TUI workspace cycling. K8s driver filters unmanaged CRs and uses delete preconditions. Podman driver uses immutable container IDs. Label serialization fixed across all put_if call sites. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(cli): delegate sandbox upload command to existing upload function The standalone `sandbox upload` command reimplemented upload logic inline with two bugs: it used `Path::exists()` which follows symlinks (rejecting dangling symlinks), and it ran git-aware filtering on symlink sources. The `run::sandbox_upload()` function already handles both cases correctly via `sandbox_upload_plan()`. Replace the inline logic with a call to the existing function. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(e2e): shorten sandbox names and fix test compatibility Shorten the sandbox name in initial_sparse_policy_is_acknowledged_as_loaded from 'e2e-2159-sparse-enrich' (22 chars) to 'e2e-sparse-enrich' (17 chars) to comply with MAX_ROUTABLE_NAME_LEN (19 chars). Also capture stderr in create_keep_with_args so future sandbox creation failures include the actual CLI error instead of reporting empty output. Signed-off-by: Derek Carr <decarr@redhat.com> * test(workspace): add test coverage for workspace CRUD and persistence isolation Add unit tests for workspace create happy path, get round-trip, get not-found, get empty-name rejection, already-exists error, and resolve_workspace not-found. Add persistence test proving cross-workspace name uniqueness (same name in different workspaces produces separate records). Add workspace name max-length boundary tests. Fix e2e harness to include stderr in name-parse-failure error path. Align Python e2e test_workspace_crud with try/finally pattern. Document provider profile catalog workspace scoping gap in RFC 0011. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(examples): update examples for workspace model compatibility Shorten sandbox names in demo scripts to fit the 19-character MAX_ROUTABLE_NAME_LEN limit: policy-demo prefix to pd-, multi-agent notepad derives a short SANDBOX_TAG from the run ID, governance interceptor uses gs-PID-RANDOM. Update vscode-remote-sandbox.md SSH host aliases from openshell-{name} to openshell-{name}.{workspace} format. Signed-off-by: Derek Carr <decarr@redhat.com> * feat(sdk): add workspace-scoped client and workspace CRUD Add WorkspaceScopedClient modeled after kube::Api::namespaced — captures workspace once and injects it into every sandbox request. Add workspace CRUD methods (create, get, list, delete) and list_sandboxes_all_workspaces on OpenShellClient. Extend SandboxRef with workspace field and add WorkspaceRef type. Include mock tests for all new operations. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(lint): resolve clippy warnings in workspace test assertions Signed-off-by: Derek Carr <decarr@redhat.com> * fix(docs): convert indented code blocks to fenced in RFC 0011 Signed-off-by: Derek Carr <decarr@redhat.com> * fix(lint): resolve clippy warnings and apply cargo fmt across workspace Auto-format with cargo fmt and fix clippy warnings exposed by the reformat: unnecessary qualifications, map_unwrap_or, identical match arms, unused variable prefix, dead code annotations, and let-unit-value in e2e harness. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): address workspace scoping issues from review - Add workspace field to settings JSON output (CLI) - Skip Podman containers missing workspace label instead of defaulting to empty string, matching K8s driver behavior - Add resource_version to list_by_scope SELECT in both SQLite and Postgres backends, with regression test - Gate PolicyLocalContext proposal/lookup routes on workspace readiness, returning 503 when workspace is not yet discovered - Block sandbox and provider creation in TUI all-workspaces mode - Clear workspace vectors in TUI reset_sandbox_state Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): make provider profile catalog workspace-aware Thread workspace through snapshot_catalog so the EffectiveProviderProfileCatalog enforces workspace boundaries on both read and write paths. UserProviderProfileSource now loads platform-scoped profiles (workspace "") plus the target workspace's profiles, preventing cross-workspace duplicate profile ID collisions that previously caused global catalog failures. Update RFC 0011 to reflect catalog scoping is implemented in Phase 1 rather than deferred to future work. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(persistence): include workspace column in atomic policy revision INSERT put_policy_revision_atomic omitted the workspace column from the INSERT into the objects table in both SQLite and Postgres backends, causing atomically-written policy revisions to lose their workspace association. Add workspace field to AtomicPolicyRevisionWrite and thread it through both backend INSERT statements, matching the non-atomic put_policy_revision path which already included it. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(proxy): skip ancestor walk when socket owner is the entrypoint collect_ancestor_identities walked the entire process tree above the entrypoint when the connecting process was the entrypoint itself, SHA256-hashing every ancestor binary (IDE, shell, container runtime). On dev machines with large binaries in the ancestor chain this exceeded the 30-second test timeout. When start_pid == stop_pid there are no intermediate ancestors to verify, so return an empty list immediately. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): make provider profile catalog scope-aware Allow the same profile ID at platform and workspace scopes by introducing layered catalog entries where workspace profiles shadow platform profiles. Add source and scope fields to the ProviderProfile proto and CLI output. Migrate List/Get handlers to the catalog, fixing divergence with runtime profile resolution. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(e2e): align podman e2e labels with centralized driver constants The podman driver moved its container labels to the centralized openshell.ai/ prefix, but the e2e test harness and cleanup script still referenced the old openshell.sandbox-* keys, causing the local_driver_token_restart test to fail on container lookup. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(e2e): align python profile isolation test with scope-aware catalog Platform profiles are now visible in workspace listings as fallbacks per the layered catalog design. Update the assertion to match. Signed-off-by: Derek Carr <decarr@redhat.com> * fix(workspace): honor profile_workspace in runtime profile resolution Runtime profile lookups now consult provider.profile_workspace via get_type_profile_for_scope. Providers created with --global-profile (profile_workspace="") resolve to the platform profile even when a workspace profile shadows the same ID. All 6 runtime call sites updated; type-only call sites remain scope-agnostic. Signed-off-by: Derek Carr <decarr@redhat.com> --------- Signed-off-by: Derek Carr <decarr@redhat.com> |
||
|
Piotr Mlocek
|
d556748771 |
feat(supervisor-middleware): add network egress middleware (#2027)
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> |
||
|
krishicks
|
cf4deccd37 |
fix(gateway): probe Docker socket during driver auto-detection (#2303)
Previously, Docker was auto-detected when the CLI was installed or a candidate Unix socket existed. Neither check verified that the Docker API was responsive. A similar check was done when auto-detecting Podman in the past, but was replaced in |
||
|
Max Dubrinsky
|
e6f319c76d |
feat(sdk): add openshell-sdk crate (#1862)
* feat(sdk): add openshell-sdk crate Additive extraction of the shared async gRPC client core (transport, TLS, OIDC single-flight refresh, edge tunnel, high-level sandbox surface, raw escape hatch) as a new workspace crate. No existing consumers yet; CLI/TUI migration follows in a separate PR. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> * refactor(sdk,core): reuse shared JWT exp decoder for refresh deadlines Extract the signature-unverified JWT exp decode out of openshell-core/grpc_client.rs into openshell_core::jwt::parse_exp_secs, and have the openshell-sdk refresh path reuse it to derive a proactive refresh deadline from a bearer JWT when the caller does not advertise expires_at. Addresses review feedback to reuse pre-existing logic rather than reimplement JWT expiry handling per client. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> * fix(sdk): harden refresh single-flight and redact tokens in Debug Addresses review feedback on the openshell-sdk refresh path. - Make the single-flight cleanup cancellation-safe. The in-flight slot is now cleared by the shared refresh computation itself (epoch-guarded) rather than the leader's post-await code. Previously, if the leader future was dropped (e.g. an FFI caller cancelling its promise) after a follower drove the refresh to completion, the completed future was stranded in the slot and later refresh_now() calls re-joined it, pinning the client to a stale or already-rejected token. Adds a regression test that cancels the leader and asserts the next refresh starts a fresh attempt. - Redact bearer secrets from Debug. RefreshedToken and the oidc RefreshTokenInput/RefreshTokenOutput now use manual Debug impls that omit the access/refresh token fields via finish_non_exhaustive, matching the house style (e.g. SecretResolver, SandboxJwtIssuer). Prevents a stray {:?} or a containing struct's derived Debug from writing tokens to logs. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> * fix(sdk): fail refresh when the new token can't be encoded as metadata store_bearer now returns an error instead of silently keeping the previous bearer value. The TokenSource commits the refreshed token to its state before the client writes it into the interceptor slot, so a silent drop left the interceptor on the old (expiring) token with no path back to a refresh. Surfacing the error fails the call loudly instead. Adds a unit test covering a token that can't be encoded as gRPC metadata. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> * fix(sdk): refresh OIDC tokens on raw routes and harden rotation Raw gRPC access never triggered OIDC refresh: a client that only used raw_grpc/raw_inference kept sending the initial bearer until it expired, with no proactive or reactive refresh. Add raw_grpc_fresh and raw_inference_fresh accessors that refresh before returning the client, plus force_refresh for reactive recovery after an Unauthenticated raw RPC. Guard the single-flight refresh commit against a concurrent replace(). The in-flight attempt now records the generation it started from and skips its write when an external replace() has advanced it, so timer or callback driven rotation is no longer clobbered by a slower refresh. Remove TokenSource::snapshot(): it returned an empty string under write contention and had no consumer on the CLI/TUI path. Tests read committed state directly instead. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> * docs(sdk): rewrite crate README for consumers Recast the openshell-sdk README as a usable crate README rather than an RFC excerpt. Drop the Responsibilities/Non-responsibilities/Consumers scope-boundary sections and the mTLS migration rationale, folding the useful facts (explicit token, no disk/name resolution, Refresh trait, SdkError mapping) into the intro, a new Auth and refresh section, and Public surface. Remove the dead relative RFC link and status-label prose so the doc renders cleanly wherever it is published. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> * fix(sdk): address review feedback on OIDC refresh and transport Apply Drew's review notes on PR #1862: - Drop unused `rustls-pemfile` dependency and move `tokio-stream` to dev-dependencies (only used by tests). - Guard OIDC `expires_at` against u64 overflow with `saturating_add`. - Fix stale `#[non_exhaustive]` rationale in `AuthConfig` (the struct `Oidc` variant it described as future already ships). - Stop double-wrapping refresh errors: store the bare refresh-error text so the single `SdkError::auth` wrap happens once at await. - Strip stale CLI porting breadcrumbs from `build_channel` docs, keeping the branch table. - Treat proactive token refresh as best-effort: a transient failure falls through to the request instead of failing an RPC whose current token is still valid, with a regression test. - Collapse `exec`'s inline auth retry into the shared `unary` helper. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> * fix(sdk): preserve transient/terminal distinction in refresh errors The refresh single-flight collapsed both `RefreshError::Transient` and `RefreshError::Terminal` into a stringified `SdkError::Auth`, so consumers (CLI, TUI, future language bindings) had no machine-readable way to tell a retryable IdP blip from a dead session that needs re-authentication. Carry the `RefreshError` through the shared outcome (kept `Clone` for `Shared`) instead of its rendered text, and map it at the await site to a new `retryable` flag on `SdkError::Auth`. Add `SdkError::auth_retryable` and a `SdkError::retryable()` accessor; transient refresh failures report `true`, every other error `false`. Add classification tests. Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> --------- Signed-off-by: Max Dubrinsky <mdubrinsky@nvidia.com> |
||
|
Drew Newberry
|
83003e80fc |
feat(interceptors): initial gateway interceptor implementation and reference example (#2005)
* feat(gateway): add descriptor-driven interceptors Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(gateway): add service-reflected interceptors Signed-off-by: Drew Newberry <anewberry@nvidia.com> * wip * fix(gateway): harden interceptor evaluation Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(interceptors): label metrics and harden governance smoke Signed-off-by: Drew Newberry <anewberry@nvidia.com> * remove on_error: ignore * feat(gateway-interceptors): emit log annotations Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(examples): govern provider profiles in interceptor Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway): preserve update config annotations Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(providers): support interceptor profile catalogs Signed-off-by: Drew Newberry <anewberry@nvidia.com> * wip * feat(governance-interceptor): sign provider profiles Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(providers): use configured profile sources for refresh updates Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(gateway-interceptors): add phase-specific evaluation payloads Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(providers): compose provider profile sources Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway-interceptors): preserve committed responses Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway-interceptors): reject ambiguous protobuf oneofs Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway-interceptors): validate patch candidates per binding Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(gateway-interceptors): use reflected protobuf codec Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(governance-example): canonicalize signed protobuf hashes Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway): commit policy provenance atomically Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway): close signed governance bypasses Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway): isolate interceptor secrets and authority Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway): snapshot provider profiles per request Signed-off-by: Drew Newberry <anewberry@nvidia.com> * chore(gateway): resolve server clippy warnings Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(server): satisfy provider source clippy lint Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(server): initialize policy test annotations Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(gateway-interceptors): require explicit route allowlist Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(proto): clarify update annotation semantics Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Tony Luo
|
40194f935e |
fix(network): fail closed when credential placeholders cannot be rewritten (#2162)
* fix(network): fail closed when credential placeholders cannot be rewritten When the credential rewriter degrades internally, the proxy forwarded the literal `openshell:resolve:env:<NAME>` placeholder (or its provider alias marker) to the upstream instead of the resolved secret, leaking the reserved token on the wire and causing upstream auth failures (#2161). Two fail-open paths are closed: - secrets: `rewrite_http_header_block` returned the header block verbatim when no `SecretResolver` was available, so the fail-closed marker scan (which ran only on the resolved path) never saw the placeholder. It now scans the header region for reserved markers even with no resolver and returns `UnresolvedPlaceholderError` when one is present. Marker-free traffic still passes through unchanged. - proxy: when TLS was detected on a CONNECT but `tls_state` was `None` (ephemeral CA generation or CA file write failed at startup), the handler fell back to a raw `copy_bidirectional` tunnel, bypassing credential rewrite. Inside the proxy handler `tls_state` is `None` only on CA-init failure (`mode != Proxy` never starts the handler, and `tls: skip` is handled earlier), so it now refuses the connection with a 503 and a High-severity denial event instead of tunneling. The two startup CA-failure logs are raised from Medium to High. Tests: resolver=None with a placeholder in the request line, a header value, and the provider-alias form now fail closed; marker-free passthrough is unchanged; the relay integration test asserts the request is rejected before any byte reaches upstream; and the 503 fail-closed response contract is locked. Signed-off-by: Tony Luo <xialuo@nvidia.com> * fix(network): refuse CONNECT before 200 when TLS termination is unavailable The fail-closed refusal for a terminating CONNECT with no TLS termination state (ephemeral CA init failed) was written after the 200 Connection Established response. Because a CONNECT client only sends its TLS ClientHello after reading the 200, the peek-based TLS detection is inherently post-200, so the 503 landed inside the established tunnel and surfaced to the client as a TLS protocol error rather than a readable status. An 'allowed CONNECT' event was also logged first. Move the decision to a pre-200 gate: query_tls_mode resolves purely from the policy decision + host/port (no peeked bytes), so the route's TLS treatment is known before the tunnel is acknowledged. When TLS state is absent and the route is not tls: skip, write the 503 as the first bytes on the socket, emit the High-severity Denied event, and close. tls: skip routes tunnel raw exactly as before, and no allowed-CONNECT event is emitted on the refusal path. The now-unreachable post-200 branch is kept as defense in depth but no longer writes an in-tunnel 503; it fails closed by dropping the connection instead. Add connection-level regression tests over a real loopback socket: the gate refuses with HTTP/1.1 503 as the first bytes for a terminating route, and writes nothing when TLS termination is present or the route is tls: skip. Signed-off-by: Tony Luo <xialuo@nvidia.com> * fix(network): order the CONNECT TLS-unavailable refusal after SSRF Addresses the gator re-check on #2162. Ordering: the pre-200 fail-closed refusal ran before SSRF/allowed_ips validation, so during CA-init failure an internal-address CONNECT got a 503 tls_termination_unavailable instead of the normal 403 ssrf_denied, weakening operator visibility in degraded state. The SSRF branches now return validated addresses; the refusal runs after that validation (an internal address has already been denied with 403) but still before the upstream connect and before 200 Connection Established. effective_tls_skip is still resolved up front since the refusal consumes it. Tests: add connection-level regressions through the real handle_tcp_connection, driving a CONNECT from a child /bin/bash copy so the /proc process-identity binding resolves it against a permissive policy (the hot-swap test's identity pattern). They assert the first bytes are HTTP/1.1 503 for a terminating route with no TLS state, a 403 (not 503) for an internal address, and no refusal for a tls: skip route. These are gated to Linux at runtime (evaluate_opa_tcp needs /proc); a companion test verifies the OPA policy shape (glob allow, tls mode) on every platform so the precondition is locked where /proc is unavailable. rest.rs: tighten the fail-closed relay test to assert the forwarded buffer is_empty() rather than merely lacking the placeholder/secret. Signed-off-by: Tony Luo <xialuo@nvidia.com> * test(network): keep the CONNECT handler test client fork-free The handler regression tests forked cat to read the proxy reply, so the client socket fd was inherited by a second process with a different binary. The identity resolver correctly denies that as ambiguous shared-socket ownership (the same invariant resolve_process_identity_denies_fork_exec_shared_socket_ambiguity pins), so the tests exercised the deny path instead of the allow path — and on busy CI runners the deny-path /proc fallback scan exceeded the test budget and looked like a hang. The client script now uses only bash builtins (exec, printf, read -d '') so exactly one process owns the socket, and the child is left to exit on EOF instead of being killed mid-read. Signed-off-by: Tony Luo <xialuo@nvidia.com> * test(network): drive the CONNECT handler tests with an in-process client The child-process client (even fork-free) made the handler tests environment-sensitive: on CI runners with a busy or restricted /proc, resolving the child's socket ownership degraded into the whole-/proc fallback scan and a deny, which surfaced as a hang. The client is now an in-process TcpStream and the test policy allows current_exe(), so identity resolution binds the socket to the test process itself in the descendant scan — the same in-process pattern the passing resolve_process_identity tests rely on. The tls: skip test additionally asserts that the handler emitted no DenialEvent at any stage, so it can no longer pass vacuously on a policy or identity deny. Refusal budgets widened to 30s as a belt for slow runners; the refusals themselves return in milliseconds. Signed-off-by: Tony Luo <xialuo@nvidia.com> * test(core): pin the percent-encoded marker no-resolver fail-closed path The no-resolver scan already catches the percent-encoded canonical marker through its decoded pass; this regression pins it: a request line carrying openshell%3Aresolve%3Aenv%3AKEY with no resolver must fail closed with UnresolvedPlaceholderError { location: header }. Signed-off-by: Tony Luo <xialuo@nvidia.com> --------- Signed-off-by: Tony Luo <xialuo@nvidia.com> |
||
|
mjamiv
|
614c8c164d |
feat(kubernetes): support PVC subPath driver config (#2034)
* feat(kubernetes): support PVC subPath driver config Signed-off-by: mjamiv <michael.commack@gmail.com> * test(kubernetes): cover writable PVC driver config Signed-off-by: mjamiv <michael.commack@gmail.com> * fix(kubernetes): address PVC subPath review feedback Signed-off-by: mjamiv <michael.commack@gmail.com> * fix(kubernetes): address PVC config review follow-up Signed-off-by: mjamiv <michael.commack@gmail.com> * fix(kubernetes): address PVC review follow-ups --------- Signed-off-by: mjamiv <michael.commack@gmail.com> |
||
|
Taylor MutchandSeth Jennings
|
8eacb4779f |
feat(kubernetes): add sidecar supervisor topology (#2076)
* feat(kubernetes): add sidecar supervisor topology Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid similar process id names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid similar process id names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(sandbox): avoid similar proxy id names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * docs(kubernetes): clarify sidecar topology limits Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): keep sidecar process leaf capless Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): refresh sidecar provider env snapshots Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * test(supervisor): align hot-swap identity regression Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): stage sidecar mtls files before proxy chown Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): simplify sidecar supervisor topology Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * chore(helm): reuse sidecar skaffold values Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid similar iptables helper names Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(e2e): harden kube gateway wrapper setup Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(supervisor): avoid nft batch rollback on OCP Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules. Signed-off-by: Seth Jennings <sjenning@redhat.com> Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): preserve process identity in sidecar topology Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table. Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * refactor(kubernetes): replace sidecar snapshots with control socket Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files. Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * feat(kubernetes): support relaxed sidecar network identity Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(sandbox): satisfy sidecar clippy lint Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * refactor(kubernetes): standardize topology naming Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(sandbox): satisfy linux clippy timeout import Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): support kata sidecar on ipv4 pods Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): satisfy linux clippy for sidecar fallback Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * chore(kubernetes): remove stale supervisor topology references Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): enable sidecar binary policy inspection Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): harden sidecar control boundary Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(kubernetes): couple sidecar supervisor lifecycles Signed-off-by: Taylor Mutch <taylormutch@gmail.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> Co-authored-by: Seth Jennings <sjenning@redhat.com> |
||
|
Florent BENOIT
|
10702133a3 |
fix(core): pin supervisor image tag to gateway version for all drivers (#2070)
* fix(core): pin supervisor image tag to gateway version for all drivers The Podman and Kubernetes drivers defaulted the supervisor image to `:latest` via DEFAULT_SUPERVISOR_IMAGE, while the Docker driver already resolved a version-pinned tag. Extract the tag resolution logic into openshell-core so all three drivers use the same OPENSHELL_IMAGE_TAG > IMAGE_TAG > CARGO_PKG_VERSION priority chain. Closes #2068 Signed-off-by: Florent Benoit <fbenoit@redhat.com> * refactor(core): simplify supervisor image tag resolver to slice-based API Remove the Docker driver's wrapper functions and call openshell_core::config::default_supervisor_image() directly. Simplify resolve_supervisor_image_tag to accept &[&str] instead of three separate parameters. Signed-off-by: Florent Benoit <fbenoit@nvidia.com> Signed-off-by: Florent Benoit <fbenoit@redhat.com> --------- Signed-off-by: Florent Benoit <fbenoit@redhat.com> Signed-off-by: Florent Benoit <fbenoit@nvidia.com> |
||
|
Kyle ZhengandJohn Myers
|
ff9af8e320 |
fix(sandbox): acknowledge initial policy revision; expose SDK labels/selectors (#2170)
* fix(sandbox): acknowledge initial policy revision The supervisor loaded and enforced a sandbox-scoped policy but never told the gateway which revision it loaded. The policy poll loop seeded itself with the initial revision's hash on its first poll, so `policy_changed` was never true for that revision and `ReportPolicyStatus(LOADED)` — which only ran in the hot-reload branch — was never called. The revision stayed `Pending` and `current_policy_version` stayed 0 even though the sandbox was `Ready` and the policy was effective. This was most visible with sparse policies that get baseline-enriched into a new revision during startup. After the OPA engine is constructed, report the exact sandbox revision the supervisor loaded as LOADED, and seed the poll loop from that revision so it is not re-reported. Report FAILED with the original construction error if engine construction or conversion fails. Only sandbox-sourced revisions (version > 0) whose canonical content matches the loaded policy are acknowledged; global and local-file policies are untouched. Delivery uses the shared bounded retry, is non-fatal on transient failure, and a pending initial acknowledgement is delivered before any newer revision so policy history is never reordered. Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * feat(python): expose sandbox labels and selectors The gateway protobuf and CLI already support request-level sandbox labels (`CreateSandboxRequest.name`/`labels`) and selector-based listing (`ListSandboxesRequest.label_selector`), but the public Python SDK dropped them, so Python-created sandboxes could not be found via `openshell sandbox list --selector ...`. Add optional, source-compatible `name`/`labels` to `SandboxClient.create`, `create_session`, and the high-level `Sandbox`, and `label_selector` to `list`/`list_ids`. `SandboxRef` now carries the gateway labels as an immutable mapping (default empty, so `SandboxRef(id, name, status)` still works). Caller-provided label mappings are copied. Attaching the high-level `Sandbox` to an existing sandbox rejects `name`/`labels` since creation metadata cannot change on attach. Template labels remain a separate concept. No protobuf changes are required. Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * fix(python): keep SandboxRef hashable and copy high-level labels Excluding the new immutable `labels` field from SandboxRef equality/hash (`compare=False`) preserves the original (id, name, status) identity and keeps the frozen dataclass hashable — a MappingProxyType field would otherwise make `hash(SandboxRef(...))` raise. Also defensively copy caller-provided labels in the high-level `Sandbox` so later caller mutation cannot change what is sent. Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * fix(sandbox): bound initial-policy-ack retries The poll loop retried a pending initial acknowledgement before processing any newer revision, but retried unconditionally forever. A permanently undeliverable ack (e.g. the revision was superseded before it could be reported) would then stall all later policy hot-reloads and provider-env refreshes. Cap the retries; after the bound, give up and resume normal polling so the loop cannot livelock on a stuck acknowledgement. Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * test(sandbox): add sparse-policy revision-2 acknowledgement e2e Regression for #2159: create a sandbox with the network-only policy-advisor fixture, which the supervisor enriches with baseline filesystem paths during startup (creating revision 2, superseding revision 1). Assert the effective policy reaches revision 2 and no revision remains Pending once the supervisor acknowledges the load. Adds SandboxGuard::create_keep_with_args to create a kept sandbox with an initial --policy. Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * fix(ci): correct sandbox checks Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * test(cli): serialize mTLS environment access Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * fix(sandbox): address policy review feedback Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * fix(sandbox): preserve exact policy acknowledgements Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> * fix(sandbox): preserve local policy overrides Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com> |
||
|
Seth Jennings
|
6461677c32 |
feat(policy): accept numeric UIDs for sandbox process identity (#1973)
* feat(policy): accept numeric UIDs in sandbox process identity validation Allow run_as_user and run_as_group to be either the literal 'sandbox' or a numeric UID/GID within [1000, 2_000_000_000]. This removes the hard dependency on a baked-in 'sandbox' user in container images, enabling compute drivers to inject resolved UIDs at sandbox creation. Phase 1 of #1959. Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(supervisor): accept numeric UIDs for process identity dropping Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing the hard dependency on a baked-in 'sandbox' user in container images. Changes: - validate_sandbox_user(): accepts numeric UIDs without passwd lookup (logs OCSF event); keeps passwd check for "sandbox" name; rejects non-numeric non-sandbox strings that fail passwd lookup - prepare_filesystem(): passes numeric UIDs/GIDs directly to chown() instead of requiring a passwd entry - drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw / Gid::from_raw; skips initgroups when target uid matches current euid; uses guard conditions before setgid/setuid calls - session_user_and_home(): falls back to ("{uid}", "/sandbox") for numeric UIDs, avoiding a passwd lookup that will fail Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy so callers have consistent range constants. Phase 2 of #1959. Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations Phase 3 of the numeric-UID plan: allow operators to specify explicit sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from OpenShift SCC namespace annotations, and propagate resolved values to supervisor container env vars and PVC init container securityContext. Changes: - Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig - Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core - Implement resolve_sandbox_identity() to fetch namespace annotations and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range) - Pass resolved UID/GID through SandboxPodParams to pod spec builder - Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container - Update PVC init container securityContext with resolved UID/GID instead of hard-coded root - Add comprehensive unit tests for resolution logic and annotation parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC annotation parsing) Signed-off-by: Seth Jennings <sjenning@redhat.com> * feat(driver-vm): add configurable sandbox UID/GID and update docs/examples Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001) in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields. Changes: - Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives - Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow - Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID injection and the ability to skip baked-in sandbox user - Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both Kubernetes (with OpenShift SCC autodetection) and VM drivers - Update sandbox-compute-drivers.mdx: add Sandbox User Identity section explaining numeric UID support across all compute drivers - Update rootfs tests to use non-default UIDs, verify config passthrough Signed-off-by: Seth Jennings <sjenning@redhat.com> * code review changes * fix(supervisor): harden tests for restricted CI container environments Guard tests against CI-specific constraints: root without CAP_SETPCAP, UIDs with no /etc/passwd entry, and restricted /proc access. Signed-off-by: Seth Jennings <sjennings@nvidia.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> --------- Signed-off-by: Seth Jennings <sjenning@redhat.com> Signed-off-by: Seth Jennings <sjennings@nvidia.com> |
||
|
Florian Bergmann
|
43bb030266 |
feat(docker,podman): add SELinux label support for bind mounts (#2092)
* feat(docker,podman): add SELinux label support for bind mounts The Docker Engine structured Mount API does not support SELinux relabelling (:z / :Z). Move user-supplied bind mounts from the structured `mounts` field to the legacy string-format `binds` field, which does support these options. Add a shared `SelinuxLabel` enum (shared/private) to openshell-core so both Docker and Podman drivers accept an optional `selinux_label` field on bind mount configs. For Docker, labels are appended to the bind string; for Podman, they are pushed to the mount options vec. Signed-off-by: Florian Bergmann <fbergman@redhat.com> * fix(docker): reject missing bind source paths on legacy binds Moving user bind mounts from the structured Mount API to the legacy Binds field changed Docker's behavior for missing source directories: the legacy path silently creates them as empty root-owned dirs instead of erroring. Add an explicit Path::exists() check to preserve the fail-fast behavior operators expect. Signed-off-by: Florian Bergmann <fbergman@redhat.com> --------- Signed-off-by: Florian Bergmann <fbergman@redhat.com> |
||
|
Evan Lezar
|
450685c755 |
fix(drivers): reject whitespace in mount fields (#2086)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Evan Lezar
|
0a25fdf525 |
refactor(core): remove unused extra bind addresses (#2059)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
John T. MyersandJohn Myers
|
f27ff1507c |
fix(providers): reserve credential placeholder revisions (#2049)
* fix(providers): reserve credential placeholder revisions Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(providers): share placeholder namespace parser Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * test(providers): cover non-revision env key Signed-off-by: John Myers <johntmyers@users.noreply.github.com> --------- Signed-off-by: John Myers <johntmyers@users.noreply.github.com> Co-authored-by: John Myers <johntmyers@users.noreply.github.com> |
||
|
st-grandEvan Lezar
|
75a317ea45 |
feat(server): support out-of-tree compute drivers via --compute-driver-socket (#1703)
* feat(server): support remote compute driver endpoints Add named remote compute driver endpoint support to the gateway. Remote drivers are selected by a non-reserved compute driver name and either a CLI/env socket endpoint or [openshell.drivers.<name>].socket_path. The VM driver now enters ComputeRuntime through the same acquired remote endpoint path, while Docker, Podman, and Kubernetes retain their in-process drivers. Require --drivers/OPENSHELL_DRIVERS when pairing an ad-hoc socket endpoint so the socket does not imply a magic driver name, and keep reserved in-tree names unavailable for unmanaged socket endpoints. Co-authored-by: Evan Lezar <elezar@nvidia.com> Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(server): cover remote compute driver UDS lifecycle Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> Co-authored-by: Evan Lezar <elezar@nvidia.com> |
||
|
Simon Scatton
|
c7879a0a1d |
test(e2e): stop using custom e2e binary builds (#2000)
Remove the production-crate dev-settings feature so e2e tests build the same gateway, CLI, and supervisor binaries that we ship. Move settings coverage from dummy test-only keys to the production ocsf_json_enabled setting. Signed-off-by: Simon Scatton <sscatton@nvidia.com> |
||
|
Evan Lezar
|
2c545893ed |
feat(cli): add GPU count requests (#1812)
* feat(gpu)!: add resource requirements BREAKING CHANGE: SandboxSpec.gpu and DriverSandboxSpec.gpu were replaced with resource_requirements.gpu, changing protobuf field 9 from a bool to a message for both public and driver APIs. Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(gpu): pass requirements through sandbox create Pass the coupled GPU requirement object through the CLI sandbox_create boundary instead of splitting presence and count into separate arguments. Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(gpu): pass requirements to timeout message Pass ResourceRequirements into the provisioning timeout message helper so GPU hints are derived from the same nested request object used to create the sandbox. Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(gpu): pass driver requirements through helpers Thread Option<GpuResourceRequirements> through driver validation and rendering helpers instead of splitting GPU presence and count into separate arguments. Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(gpu): validate exact device requests Require exact driver GPU device lists to be tied to a GPU request, allow a single exact device to use the default countless request, and require explicit matching counts for multi-device lists. Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
st-grandClaude Opus 4.7
|
4ee27d9955 |
feat(sandbox,providers): add aws-bedrock as a recognized inference provider (#1704)
* feat(sandbox): allow AWS Bedrock InvokeModel paths through the L7 router
Adds two patterns to `default_patterns()` so the supervisor's L7
inference router recognizes the Bedrock InvokeModel URL shape and
forwards matched requests to the registered upstream:
- `POST /model/{modelId}/invoke` → aws_bedrock_invoke
- `POST /model/{modelId}/invoke-with-response-stream` → aws_bedrock_invoke_stream
The `{modelId}` segment is wildcarded by extending `detect_inference_pattern`
to handle one middle `/*/` segment in addition to the existing trailing
`/*`. The wildcard is constrained to a single non-empty path segment to
avoid path-traversal liabilities — `/model//invoke` and `/model/a/b/invoke`
both no-match.
Without this, sandboxes running Claude Code in its native Bedrock mode
(`CLAUDE_CODE_USE_BEDROCK=1`, `ANTHROPIC_BEDROCK_BASE_URL`, AWS-style
auth) hit the supervisor with `403 connection not allowed by policy`
because their URL doesn't match `/v1/*` shapes. The fix unblocks
operators wanting to register direct AWS Bedrock, an in-cluster
Bedrock-compatible bridge, or a Bedrock-emulating LiteLLM as
`--type aws-bedrock` providers.
Tests cover: positive matches for invoke + invoke-with-response-stream,
query-string handling, GET rejection, empty-segment rejection,
multi-segment rejection, and unknown-action rejection.
Companion changes (provider discovery spec + YAML profile) follow in
the next commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* feat(providers): add aws-bedrock provider profile + discovery spec
Adds `aws-bedrock` to the built-in provider catalog so operators can
run `openshell provider create --type aws-bedrock --credential ...`
and have the gateway treat it as a first-class inference provider
alongside `anthropic`, `openai`, etc.
- `providers/aws-bedrock.yaml`: YAML profile declaring four credentials
(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_REGION).
Default endpoint is `bedrock-runtime.us-east-1.amazonaws.com:443`;
operators in other regions or running against a Bedrock-compatible
proxy override via the operator-supplied `BEDROCK_BASE_URL` config-key
(mirrors `ANTHROPIC_BASE_URL` for the `anthropic` provider).
- `crates/openshell-providers/src/providers/aws_bedrock.rs`: the
`ProviderDiscoverySpec` so `openshell provider create --auto-providers`
picks up AWS_* env vars from local credentials.
- `crates/openshell-providers/src/providers/mod.rs`: register the module.
- `crates/openshell-providers/src/lib.rs`: register the SPEC in the
default registry alongside the other providers.
- `crates/openshell-providers/src/profiles.rs`: include the new YAML in
`BUILT_IN_PROFILE_YAMLS`.
What this PR explicitly does NOT add (intentionally separated for
review-size reasons; will follow up):
- A SigV4 signer in `openshell-router`. The current change simply
declares the protocol; a follow-up PR adds outbound SigV4 signing
using the `aws-sigv4` crate and a new `auth_style: sigv4` validator
branch in profiles.rs. Operators who don't need SigV4 (e.g. an
in-cluster bridge that ignores it and authenticates separately to
the upstream) can use this PR today.
- Body translation between Bedrock InvokeModel shape and other
inference shapes. The router treats Bedrock requests as opaque
pass-through; if the operator's upstream is real AWS Bedrock it
speaks Bedrock natively, if it's a translating bridge the bridge
does any conversion server-side.
- `BEDROCK_BASE_URL` placeholder substitution in the YAML loader.
Today the YAML's `host` is a literal default; operators override
with the config-key the same way `ANTHROPIC_BASE_URL` works.
Tested: `cargo test -p openshell-providers` (35 tests green) and
`cargo test -p openshell-sandbox --lib l7::inference` (40 tests green
including the seven new aws_bedrock cases from the previous commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* revert(providers): drop legacy aws-bedrock SPEC, rely on v2 YAML profile
Addresses johntmyers's review on NVIDIA/OpenShell#1704: net-new
providers should land via the v2 YAML profile only and should NOT
require changes to the legacy `ProviderDiscoverySpec` registry.
- Delete `crates/openshell-providers/src/providers/aws_bedrock.rs`
(the legacy SPEC + `test_discovers_env_credential!` invocation).
- Drop `pub mod aws_bedrock;` from `crates/openshell-providers/src/providers/mod.rs`.
- Drop `registry.register(providers::aws_bedrock::SPEC)` from
`crates/openshell-providers/src/lib.rs`.
Kept:
- `providers/aws-bedrock.yaml` and the `include_str!` in
`BUILT_IN_PROFILE_YAMLS` (`profiles.rs`) — the v2 path.
`discover_from_profile()` (`crates/openshell-providers/src/discovery.rs`)
picks up AWS_* env vars via `discovery.credentials` in the YAML.
- L7 router patterns in `crates/openshell-sandbox/src/l7/inference.rs`
— orthogonal to the provider registry.
The discovery test in the deleted file goes with it; v2 doesn't have
an established per-provider env-var-pickup unit test pattern, and
other YAML-only registrations (none today, but this is the new
direction) won't carry one either.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* fix(providers): include aws_session_token in discovery + update profile assertion
Two fixes from johntmyers's gator-agent re-check on NVIDIA/OpenShell#1704:
1. `providers/aws-bedrock.yaml`: add `aws_session_token` to
`discovery.credentials`. The credential is declared in the profile
but was missing from the discovery scan list, so Providers v2
`--from-existing` would silently drop temporary AWS credentials
(STS / IRSA scenarios).
2. `crates/openshell-server/src/grpc/provider.rs`: update the static
`list_provider_profiles_returns_built_in_profile_categories`
assertion to include `aws-bedrock` at alphabetical position 0.
Adding `providers/aws-bedrock.yaml` to BUILT_IN_PROFILE_YAMLS made
the prior `["claude-code", "github", "nvidia"]` expectation stale.
Remaining blockers from the same review (deferred to follow-up
commits): `inference::profile_for` registration for aws-bedrock,
user-facing provider + inference-routing docs, and an
`upsert_cluster_inference_route` integration test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* feat(inference): register aws-bedrock profile (bridge-fronted) + docs
Addresses johntmyers's blocking review feedback on PR #1704:
"aws-bedrock still is not wired into the managed inference.local route
registry. profile_for only registers openai, anthropic, and nvidia, so
inference set --provider <aws-bedrock-provider> will reject this
provider before the new sandbox L7 patterns can be used."
Approach: register aws-bedrock as a *bridge-fronted* upstream — the
router does not inject any auth header on outbound requests; the
configured BEDROCK_BASE_URL is expected to point at a translating
bridge / Bedrock-compatible proxy that handles auth in its own pod.
This is the shape the L7 patterns commit (8b30211a) and the YAML
profile (6b51e1a6) were designed for. SigV4 signing for direct AWS
Bedrock is a separate follow-up; see PR thread.
Changes:
- core::inference::AuthHeader: add `None` variant for upstreams that
authenticate themselves.
- core::inference: add AWS_BEDROCK_PROFILE static + register in
profile_for. Default base URL is bedrock-runtime.us-east-1, override
via BEDROCK_BASE_URL config-key (mirrors ANTHROPIC_BASE_URL pattern).
Empty credential_key_names + auth: None means no router-side
credential lookup at route time.
- router::backend: handle AuthHeader::None as a no-op (skip auth
injection).
- server::inference::resolve_provider_route: gate find_provider_api_key
on auth != None. aws-bedrock providers with empty credentials now
resolve cleanly. Updated the unsupported-type error message to
include aws-bedrock in the supported list.
- server::inference tests: add positive
upsert_cluster_route_succeeds_for_aws_bedrock_without_api_key test
covering the new code path end-to-end (provider with empty creds +
BEDROCK_BASE_URL config → upsert succeeds → resolved route has
empty api_key + provider_type aws-bedrock + bridge URL).
- core::inference tests: profile_for_known_types covers aws-bedrock,
case-insensitive lookup, plus three new aws-bedrock-specific tests
(auth: None, no credential keys, bedrock-specific protocols).
- docs/sandboxes/inference-routing.mdx: header forwarding row
mentions aws-bedrock has no passthrough headers; new tabs in
Supported API Patterns (InvokeModel + InvokeModelWithResponseStream)
and Create a Provider (with the bridge-fronted shape note + SigV4
deferral).
- docs/sandboxes/manage-providers.mdx: new row in Supported Provider
Types table; new row in Supported Inference Providers table.
Verification (in dev container):
- cargo check -p openshell-core -p openshell-router -p openshell-server: clean
- cargo test -p openshell-core --lib inference: 14/14 pass (incl. 3 new)
- cargo test -p openshell-server --lib inference::tests::upsert: 6/6 pass
(incl. new aws-bedrock test)
- cargo fmt --check: clean
- cargo clippy --all-targets -D warnings: clean
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* fix(aws-bedrock): bridge-only YAML, doc actual cmd shape, neg test
Addresses four findings from gator-agent's #1704 re-check on 4ab587f1:
- **Item 5** (YAML collects unused AWS creds): mark all four AWS
credentials `required: false` and clear `discovery.credentials`.
Bridge-fronted routing intentionally does not consume AWS
credentials, so `--from-existing` no longer scans for them. The
credentials remain in the schema (not deleted) so the SigV4
follow-up can flip them back without a schema migration. Added a
multi-line description that names the bridge-fronted shape and the
SigV4 deferral so readers don't have to cross-reference the PR
thread.
- **Item 3** (docs show command that the CLI rejects): rewrite the
Create-a-Provider example for AWS Bedrock to use the actual
required shape — placeholder `--credential AWS_ACCESS_KEY_ID=
unused-bridge-fronted-shape` plus the `--config BEDROCK_BASE_URL`.
The placeholder satisfies the gRPC handler's
`provider.credentials.is_empty()` rejection without expanding
server-side validation; the router ignores it on the outbound path
because `auth: AuthHeader::None` skips header injection. Operators
see a clearly-labeled placeholder in `provider get` output.
- **Item 1** (validator probe): document `--no-verify` as required
for `openshell inference set --provider <aws-bedrock>` since the
default validation probe doesn't recognize the
`aws_bedrock_invoke` / `aws_bedrock_invoke_stream` protocols. Doc
now shows the full `provider create` + `inference set --no-verify`
flow with rationale for both decisions inline.
- **Item 6** (docs polish): `inference-routing.mdx` summary row now
lists AWS Bedrock alongside NVIDIA, Anthropic, Vertex AI, and
OpenAI-compatible providers, with the bridge-fronted caveat
inline.
Test additions in `crates/openshell-server/src/inference.rs`:
- Renamed the existing aws-bedrock test from
`..._without_api_key` to `..._with_bridge_url` and updated it to
use a placeholder credential (mirroring the doc-recommended
pattern operators will copy-paste). The `auth: None` path still
produces an empty `api_key` on the resolved route — the test now
documents that the credential is *stored* but not *used*.
- Added `upsert_cluster_route_rejects_aws_bedrock_without_bedrock_base_url`:
the negative half of johntmyers' "successfully used by
upsert_cluster_inference_route or intentionally rejected with a
clear documented error" ask. With
`default_base_url: ""` and no `BEDROCK_BASE_URL` config, route
resolution returns `InvalidArgument` naming the missing base_url
rather than silently forwarding prompts to AWS Bedrock with no
usable auth.
Verification (in dev container):
- cargo test -p openshell-core --lib inference: 18/18 (incl. 3 new)
- cargo test -p openshell-server --lib inference::tests::upsert: 8/8
(incl. 2 new aws-bedrock cases — positive + negative)
- cargo fmt --check: clean
- cargo clippy --all-targets -D warnings: clean
Item 2 (router-side enforcement of operator-configured Bedrock model
path, replacing the current verbatim path forwarding + body-only
model rewrite) is the remaining blocker and is genuinely separable —
it touches the L7 router with streaming-aware test coverage.
Deferring to its own commit so the security-critical change gets the
review attention it deserves.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* fix(router): enforce operator-configured Bedrock model in request path
Closes the security-blocking item from gator-agent's #1704 re-check
on 4ab587f1: "Bedrock carries the model id in /model/{modelId}/invoke,
but the router currently forwards the caller's original path and only
rewrites JSON body model. That lets sandbox code choose a different
upstream model than the operator-configured route model, and may also
mutate native Bedrock request bodies incorrectly."
Two changes in `prepare_backend_request`:
1. **Path rewrite for Bedrock routes.** Before computing the upstream
URL, parse the inbound path's `/model/<id>/invoke[-with-response-stream]`
shape and substitute the operator-configured `route.model` for the
caller-supplied model segment. Sandbox code that hardcodes a
different model still works (we don't reject on mismatch), but the
operator's configured model is what reaches the upstream / bridge.
If the inbound path is somehow not a recognized Bedrock shape on a
Bedrock route (the L7 pattern detector upstream of the router
should never produce this combination), reject with
RouterError::Internal naming the offending path rather than
forwarding verbatim.
2. **Skip body-model injection for Bedrock routes.** The existing body
rewriter unconditionally inserts `route.model` into the JSON body
for non-Vertex routes. AWS Bedrock InvokeModel encodes the model
in the URL path; the body is the raw provider-specific payload
(Anthropic Messages for Claude, Mistral payload for Mistral, etc.)
and must not be mutated. The branch ordering is now:
needs_vertex_anthropic_version → strip body model + inject
anthropic_version; route_is_bedrock → leave body alone; else →
inject route.model (existing default).
New helpers, all in `crates/openshell-router/src/backend.rs`:
- `route_is_bedrock(route)` — true when route.protocols contains
aws_bedrock_invoke or aws_bedrock_invoke_stream.
- `parse_bedrock_invocation_path(path)` — returns
Some((model_id, "/invoke" | "/invoke-with-response-stream")) for
paths matching the recognized Bedrock shapes. Strips query strings.
Rejects empty model ids and multi-segment ids (defense-in-depth
matching the L7 pattern detector's existing guards).
- `rewrite_bedrock_path(route, path)` — returns the path with the
caller's model segment replaced by route.model.
Test coverage in the same file (9 new tests):
- parse_bedrock_invocation_path: positive cases for both invoke
variants, query-string stripping; negative cases for empty model id,
multi-segment id, unknown action, wrong prefix, missing slash.
- route_is_bedrock: matches both protocol variants singly and
combined; rejects openai_chat_completions.
- rewrite_bedrock_path: substitutes operator model on both invoke
variants; returns None for non-Bedrock paths.
- bedrock_route_rewrites_model_in_path_and_preserves_body
(wiremock end-to-end): caller sends /model/some-other-model/invoke
with a body containing model: "caller-supplied-model-name". Mock
asserts the upstream receives /model/<operator-model>/invoke and the
body's model field is the caller's value (NOT route.model) — proves
both the path rewrite and the body preservation.
- bedrock_route_streaming_rewrites_model_in_path: same contract for
invoke-with-response-stream.
- bedrock_route_rejects_non_bedrock_path: defense-in-depth coverage of
the Internal-error path when a Bedrock route receives a path that
doesn't match Bedrock shape.
Verification (in dev container):
- cargo test -p openshell-router --lib: 53/53 (incl. 9 new)
- cargo fmt --check: clean
- cargo clippy -p openshell-core -p openshell-router -p openshell-server
--all-targets -- -D warnings: clean
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* fix(sandbox/l7): declare framing for Bedrock patterns
Closes the buffered-vs-streaming framing warning from gator-agent's
re-check on 4ab587f1: "Bedrock InvokeModel should be buffered while
InvokeModelWithResponseStream is streaming. Please add framing/coverage
so /model/{id}/invoke cannot be corrupted by the streaming proxy's
truncation/error-frame behavior."
The InferenceApiPattern struct gained a `framing: ResponseFraming`
field upstream after the original Bedrock-patterns commit (#22b78cff)
landed; the cherry-pick onto current upstream/main left the two
Bedrock entries without the new field. Fixed here:
- aws_bedrock_invoke (POST /model/{id}/invoke):
framing = ResponseFraming::Buffered
InvokeModel returns one JSON object the caller decodes whole. Sending
it through the streaming proxy would risk a mid-body size-cap
truncation or idle-timeout failure appending an SSE error event onto
bytes the caller decodes as one JSON body — the same corruption mode
that drove the existing embeddings + model-discovery to Buffered.
- aws_bedrock_invoke_stream (POST /model/{id}/invoke-with-response-stream):
framing = ResponseFraming::Streaming
InvokeModelWithResponseStream returns an AWS event-stream of binary
chunks; the caller wants chunks incrementally, so the streaming proxy
path is correct.
Two new tests in `crates/openshell-sandbox/src/l7/inference.rs` pin
down the contract:
- aws_bedrock_invoke_is_buffered — detect_inference_pattern returns a
Buffered pattern for /model/<id>/invoke, with explanatory message
naming the corruption mode being prevented.
- aws_bedrock_invoke_stream_is_streaming — same shape, asserting
Streaming for /model/<id>/invoke-with-response-stream.
Verification (in dev container):
- cargo check -p openshell-sandbox: clean (was failing on missing
`framing` field before this commit)
- cargo test -p openshell-sandbox --lib l7::inference::tests::aws_bedrock:
7/7 (incl. 2 new framing tests)
- cargo fmt --check: clean
- cargo clippy --all-targets -- -D warnings: clean
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* fix(inference): drop aws_bedrock_invoke_stream until protocol-aware errors land
Per PR #1704 review (johntmyers): defer Bedrock streaming to a follow-up
that also wires protocol-aware error framing for the AWS event-stream
shape. Until then, surfacing `/model/{id}/invoke-with-response-stream`
risks shipping responses the sandbox cannot interpret on failure.
- AWS_BEDROCK_PROTOCOLS no longer advertises `aws_bedrock_invoke_stream`.
- The L7 inference pattern table drops the streaming entry; only
`aws_bedrock_invoke` (buffered) is recognized.
- Test `aws_bedrock_invoke_stream_pattern_is_deferred` asserts no
pattern claims that protocol so the gap is visible.
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* fix(inference): validate AWS Bedrock model_id at route + rewrite, preserve query
Per PR #1704 review (johntmyers, gator agent): the operator-configured
Bedrock model_id flows verbatim into the upstream URL path; the previous
plumbing left no enforcement that the value was a single benign path
segment, opening a path-injection vector.
Defense in depth, both layers:
* `openshell-server::inference`: new `validate_aws_bedrock_model_id`
rejects empty, leading/trailing whitespace, `/`, `\\`, `?`, `#`, `%`,
`..`, control/whitespace characters. Wired into `resolve_provider_route`
ahead of base_url resolution so the route store cannot persist a
malformed model_id. Mirrors `validate_vertex_model_id` exactly.
* `openshell-router::backend`: `rewrite_bedrock_path` now refuses to
construct the upstream URL unless `route.model` passes
`is_valid_bedrock_model_id`, so even a stale or hand-edited route
cannot reach the wire. The parser also drops the
`/invoke-with-response-stream` arm to match the protocol catalog.
* `parse_bedrock_invocation_path` returns the `?`-prefixed query tail
as a third element; `rewrite_bedrock_path` re-attaches it so any
caller-supplied query string is preserved through the model rewrite.
Tests: 5 unit tests for the validator, 1 integration test that
exercises every unsafe-model_id reject path through
`upsert_cluster_inference_route`, plus a router-side rewrite-rejects
test covering 11 unsafe `route.model` values. All 72 server inference
tests + router tests pass.
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* fix(aws-bedrock): make built-in profile non-egress-granting; tighten docs
Per PR #1704 review (johntmyers): a single profile that is "usable as
is" — not a profile that auto-grants direct AWS Bedrock egress to any
sandbox that selects it. The bridge IS the egress point and is
operator-managed; the profile must not implicitly punch a hole through
the cluster's network policy on its behalf.
* `providers/aws-bedrock.yaml`: clear `endpoints` and `binaries` to
empty arrays. Rewrite the description to spell out that the profile
is intentionally non-egress-granting, that operators are responsible
for declaring their bridge's egress endpoint and binary attribution,
and that the SigV4 follow-up will repopulate these fields once
router-side signing exists.
* `docs/sandboxes/inference-routing.mdx`:
- Drop the `InvokeModelWithResponseStream` row from the supported
patterns table (matches the protocol-catalog + L7-pattern drop).
- Update the `--no-verify` paragraph to reference only
`aws_bedrock_invoke`.
- Generalise the placeholder-credential rationale: any
standalone-router profile registering `AuthHeader::None` will hit
the same non-empty-credentials structural requirement.
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* style(router): satisfy cargo fmt for Bedrock query-tail extraction
CI's rustfmt rejected the one-line `let (path_only, query_tail) = path.find('?').map_or(...)`
shape from commit c51160e4 and required the chained-method layout
instead. Functional behaviour is unchanged; tests still pass.
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
* style(router): drop unnecessary `crate::` prefix on RouterError matcher
`RouterError` is already imported at the top of the file, so the
two test-side `matches!(result, Err(crate::RouterError::UpstreamProtocol(_)))`
uses trip clippy's `unused_qualifications` under `-D warnings`. Drop
the prefix on both sites; functional behaviour is unchanged.
These warnings predate this PR (originated in
|
||
|
Robert Sturla
|
48545cfb59 |
feat(sandbox): add GCE metadata emulator for Google Cloud (#1763)
* feat(core): add shared GCP constants module Single source of truth for GCP naming: env var aliases, provider config keys, token search order, and Vertex-specific env vars. Consumed by openshell-server, openshell-providers, and openshell-sandbox. - Add google_cloud.rs with metadata emulator host and loopback address - Define PROJECT_ID, REGION, and SERVICE_ACCOUNT_EMAIL env var aliases - Add provider config key constants for gcp provider implementations - Define TOKEN_ENV_KEYS search order (SA token takes priority over ADC) - Add Vertex-specific env vars for Goose and Claude Code SDK integration - Add STATIC_CONFIG_KEYS as union of all alias arrays for env resolution - Export module via openshell-core lib.rs Signed-off-by: Robert Sturla <rsturla@redhat.com> * feat(providers): add google-cloud and vertex provider plugins Add GoogleCloudProvider and VertexProvider implementing inject_env to project GCP config (project ID, region, SA email, metadata host) into sandbox environment variables. Replace the inline Vertex AI env injection in the server with the registry-based inject_env dispatch. Also adds the google-cloud.yaml provider profile with SA JWT and ADC OAuth2 credential refresh flows. Signed-off-by: Robert Sturla <rsturla@redhat.com> * feat(sandbox): add GCE metadata emulator for GCP Add a loopback HTTP server on 127.0.0.1:8174 inside the sandbox network namespace that emulates the GCE instance metadata API. GCP client SDKs discover it via GCE_METADATA_HOST and obtain credential placeholders that the proxy resolves to real tokens at egress. Add metadata_server module with MetadataHandler trait and netns-aware TCP binding via std::thread (not spawn_blocking) to avoid tokio pool namespace contamination Add google_cloud_metadata module implementing the GCE metadata API subset (token, project-id, email, scopes, service-accounts) Add child_env_resolved() and gcp_token_response() to ProviderCredentialState for GCP-aware credential projection Wire metadata server into sandbox lifecycle before SSH handler Collapse multi-line HTTP response format string into single line Signed-off-by: Robert Sturla <rsturla@redhat.com> * docs(sandbox): add GCP credentials documentation Document the google-cloud provider setup for ADC and service account flows, injected environment variables, metadata emulator behavior, and network policy configuration for GCP APIs. Signed-off-by: Robert Sturla <rsturla@redhat.com> * feat(cli): support --from-gcloud-adc for google-cloud providers Widen --from-gcloud-adc to accept google-cloud providers. The ADC credential key is derived from the provider profile rather than hardcoded per type, so future GCP provider types get ADC support by declaring the right refresh metadata in their profile YAML. Add ProviderTypeProfile::adc_credential() to find the ADC-compatible credential from a profile's refresh metadata. Remove unused VERTEX_AI_ADC_TOKEN_KEY and GCP_ADC_TOKEN_KEY constants. Signed-off-by: Robert Sturla <rsturla@redhat.com> --------- Signed-off-by: Robert Sturla <rsturla@redhat.com> |
||
|
Shiju
|
ffc102a02d |
fix(cli): verify forward listener before success (#1880)
* fix(cli): verify forward listener before success Wait for a connectable local forward listener before reporting foreground forwarding success. Fail background forwarding when the forked SSH process cannot be tracked, probe the listener before writing the PID file, and terminate the tracked SSH process if the listener never opens. Document that forwarded URLs are printed only after listener health is proven. Signed-off-by: Shiju <shiju@nvidia.com> * fix(cli): harden background forward cleanup Signed-off-by: Shiju <shiju@nvidia.com> * fix(cli): validate background forward cleanup pid Signed-off-by: Shiju <shiju@nvidia.com> * fix(cli): harden forward PID matching and teardown Require background forward cleanup to prove a candidate PID is the OpenShell-generated SSH forwarding process before reporting it active or signaling it. Match the exact -L forward argument and sandbox ID instead of by substring, so a request for port 80 cannot collide with an existing 8080 forward and terminate the wrong process. Reject legacy PID records as non-authoritative and route port-only lookup through validated live forward state. Recover the exact process argv from /proc/<pid>/cmdline on Linux so a ProxyCommand whose executable path contains whitespace is parsed correctly; other platforms keep the best-effort ps command-line parse. After listener readiness succeeds, terminate the revalidated background ssh process if the PID-file write fails, instead of returning early and leaving a reachable but untracked forward that forward stop/list cannot manage. Cover adversarial matcher cases (port and sandbox-id collisions, command spoofing, remote-command token confusion, whitespace executable paths) and the PID-file write-failure teardown path with focused tests. Signed-off-by: Shiju <shiju@nvidia.com> --------- Signed-off-by: Shiju <shiju@nvidia.com> |
||
|
mmilutinovic371
|
36bb9e3ec2 |
feat(providers): add DeepInfra as a built-in inference provider (#1902)
* feat(providers): add DeepInfra as a built-in inference provider (v2 only) - Adds `deepinfra` as a built-in Providers v2 profile (`providers/deepinfra.yaml`) with inference category, Bearer auth, and `DEEPINFRA_API_KEY` discovery - Adds `DEEPINFRA_PROFILE` to inference routing so `inference.local` works with the `deepinfra` provider type - Fixes `build_backend_url` to strip `/v1` from request paths when the base URL contains `/v1/` as an internal segment (e.g. `api.deepinfra.com/v1/openai`), preventing double-versioned paths like `.../v1/openai/v1/chat/completions` - Updates `docs/sandboxes/providers-v2.mdx` and `docs/sandboxes/manage-providers.mdx` with DeepInfra entries; removes the old v1 workaround row that used `openai` type with `OPENAI_API_KEY` Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com> * fix(providers): address gator review findings for DeepInfra provider - Narrow build_backend_url /v1 dedupe to URLs whose path component is exactly /v1 or starts with /v1/ — prevents regression on proxy endpoints where /v1 is buried deeper (e.g. /api/v1/openai); add regression test for the nested proxy path case - Add deepinfra provider plugin with DEEPINFRA_API_KEY discovery, registered in ProviderRegistry so known_types() and TUI include it - Add deepinfra to unsupported-inference-provider error message in openshell-server for accurate user-facing debugging guidance - Add deepinfra to openai_compatible_profiles_include_embeddings test to lock in the OpenAI-compatible protocol contract Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com> * fix(router): handle /v1 as final path segment in build_backend_url dedup Extends the /v1 deduplication logic to also strip /v1 from request paths when the base URL's path ends with /v1 (e.g. https://api.groq.com/openai/v1). The previous fix only matched paths starting with /v1/, which regressed providers like Groq whose base path has /v1 as the last segment rather than the first. The nested-proxy exclusion (e.g. /api/v1/openai) is preserved since /v1 appears in the middle — neither first nor last segment. Adds a regression test for the Groq-style base URL. Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com> * fix(providers): add deepinfra telemetry bucket and update profile list test - Add DeepInfra variant to ProviderProfile telemetry enum and from_raw() mapping so deepinfra providers are tracked in their own bucket rather than falling through to Custom - Map deepinfra in telemetry_provider_profile() in openshell-server - Add deepinfra to list_provider_profiles_returns_built_in_profile_categories test (sorted between cursor and github) - Update architecture/gateway.md inference provider list to include deepinfra Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com> * style(router): apply cargo fmt to backend.rs Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com> --------- Signed-off-by: Milos Milutinovic <codemastermilos@gmail.com> |
||
|
Evan Lezar
|
294c64eed1 |
fix(gpu): prefer single CDI devices for local runtimes (#1675)
Prefer a single CDI-qualified device when Docker or Podman resolves the default GPU request to one GPU. Allow nvidia.com/gpu=all only as a WSL2 all-only compatibility fallback, using Docker daemon info and Podman's /dev/dxg probe to identify that case. Update driver docs, architecture notes, and GPU e2e coverage for the default selection behavior. Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
rhubenovandEvan Lezar
|
1ca23bc5c8 |
refactor(openshell-sandbox): Split sandbox into process and network subcrates. (#1650)
* refactor(sandbox): extract run_networking from run_sandbox
Lifts TLS state generation, network namespace setup, proxy startup,
bypass monitor spawn, and SSH-side proxy URL / netns FD computation
out of run_sandbox into a sibling async fn `run_networking` that
returns a Networking struct. The identity cache moves with it (only
consumed by the proxy). Entrypoint PID allocation moves just above
the call site so it can be passed in.
No behavior changes — same OCSF emits, same async order, same RAII
lifetimes for the proxy and bypass-monitor handles, now held by the
returned Networking value in run_sandbox's frame.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(sandbox): extract run_process and lift netns to run_sandbox
Lifts the post-networking tail of `run_sandbox` (zombie reaper, SSH
server, supervisor session, process spawn, OPA probe, policy poll loop,
denial aggregator, wait/exit) into a sibling async fn `run_process`.
Also moves network namespace creation out of `run_networking` into a new
`create_netns_for_proxy` helper invoked from `run_sandbox`, so
`run_networking` is purely the proxy component (OPA evaluation, TLS
interception, credential injection, inference routing, gRPC control
API). The netns is then borrowed into both `run_networking` and
`run_process`.
No behavior change.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* chore(workspace): scaffold openshell-supervisor-networking and openshell-supervisor-process crates
Add empty placeholder crates that subsequent commits will populate as the
sandbox decomposition proceeds. Both crates compile clean as part of the
workspace and are picked up automatically by the existing
`members = ["crates/*"]` glob.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift DenialEvent to openshell-core
The DenialEvent struct is emitted by both the proxy/L7 layer (networking-side)
and the bypass monitor (process-side), and crosses the run_networking ->
run_process API boundary. Move it to openshell-core so the eventual
supervisor-networking and supervisor-process crates can both reference it
without depending on each other. DenialAggregator and the channel/flush
helpers stay in openshell-sandbox for now.
A thin `pub use openshell_core::DenialEvent;` re-export from
denial_aggregator.rs keeps every existing `crate::denial_aggregator::DenialEvent`
call site resolving without further edits.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift normalize_path to openshell-core
Move the lexical path-normalization helper from openshell-policy to
openshell-core::paths so it can be reached from crates that sit below
openshell-policy in the dependency graph. openshell-policy keeps its
existing public API via a `pub use` re-export, so all current call sites
(e.g. openshell-sandbox/src/policy.rs) continue to resolve unchanged.
This is a prerequisite for lifting openshell-sandbox/src/policy.rs into
openshell-core: that file's `From<ProtoFilesystemPolicy>` impl calls
normalize_path, and lifting it as-is would cycle through openshell-policy.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift SandboxPolicy and friends to openshell-core
Move openshell-sandbox/src/policy.rs (SandboxPolicy, NetworkPolicy,
ProxyPolicy, FilesystemPolicy, LandlockPolicy, ProcessPolicy, NetworkMode,
LandlockCompatibility, plus their Proto* TryFrom/From impls) to
openshell-core/src/policy.rs.
Both prospective supervisor leaves (networking and process) dispatch on
SandboxPolicy. Hosting it in openshell-core lets either leaf reach for it
without depending on the other (or on the future orchestrator).
The From<ProtoFilesystemPolicy> impl now calls the in-crate
openshell_core::paths::normalize_path lifted in the previous commit, which
is what made this move cycle-free.
Update all crate::policy::* call sites in openshell-sandbox to
openshell_core::policy::*.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move child_env from openshell-sandbox
child_env (proxy_env_vars, tls_env_vars) is process-side only — consumed
by process.rs and ssh.rs. With the orchestrator staying in
openshell-sandbox (Shape A), openshell-sandbox depends on the new leaf
crates, so process-only modules can land in
openshell-supervisor-process directly.
Add openshell-supervisor-process as a path dependency of
openshell-sandbox. Update process.rs and ssh.rs to import from
openshell_supervisor_process::child_env.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move skills from openshell-sandbox
Move the static skills installer (and its embedded resource directory)
out of openshell-sandbox into openshell-supervisor-process. The module
is process-side only — invoked once during sandbox start to drop
agent skill files into the workspace — and has no cross-leaf consumers.
Adds miette as a dependency and tempfile as a dev-dependency on
openshell-supervisor-process.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move mechanistic_mapper from openshell-sandbox
Move the mechanistic mapper (HTTP method/path → operation classifier
that derives policy proposals from connection summaries) out of
openshell-sandbox into openshell-supervisor-networking. Single internal
caller (run_policy_poll_loop in lib.rs) and only depends on
openshell-core + tracing — no cross-leaf entanglement.
First population of the openshell-supervisor-networking crate; adds
openshell-core and tracing as dependencies.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift procfs to openshell-core
Move procfs (PID lookups, ancestor walking, /proc/net/tcp socket-owner
resolution, file SHA256 hashing) from openshell-sandbox into
openshell-core. The module is consumed cross-leaf — by bypass_monitor
on the process side and by identity / proxy on the networking side —
so it has to sit below both leaves.
Adds tracing, sha2, and hex as dependencies on openshell-core.
Updates the three call sites in openshell-sandbox to import from
openshell_core::procfs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move identity from openshell-sandbox
Move BinaryIdentityCache (path → SHA256 cache used to identify the
process behind an outbound connection) from openshell-sandbox into
openshell-supervisor-networking. The cache is consumed only by the
networking-side proxy and the orchestrator; with procfs already in
openshell-core there are no remaining cross-leaf dependencies.
Adds miette as a dependency and tempfile as a dev-dependency on
openshell-supervisor-networking. Adds a Default impl for
BinaryIdentityCache to satisfy clippy::new_without_default now that
the type is publicly exposed across crates.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move agent-proposals flag from openshell-sandbox
Move AGENT_PROPOSALS_ENABLED, agent_proposals_enabled(), and the
test-only ProposalsFlagGuard out of openshell-sandbox into
openshell-supervisor-process::proposals. The flag is read only by the
process-side policy_local route handler and the orchestrator; lifting
it to openshell-core would have made core carry sandbox-owned runtime
state without buying anything.
The test-only ProposalsFlagGuard is still consumed from networking-side
l7/rest tests today (until the wider Q2 OCSF/gRPC injection work lands).
Expose it via a new optional `test-helpers` feature on
openshell-supervisor-process so test crates opt in explicitly without
pulling tokio sync primitives into production builds.
openshell-sandbox keeps its existing crate-private path
(`crate::AGENT_PROPOSALS_ENABLED`, `crate::test_helpers`) via re-exports
so call sites and tests are unchanged.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift secrets to openshell-core
Move crates/openshell-sandbox/src/secrets.rs to crates/openshell-core/src/secrets.rs so both supervisor leaves can reach SecretResolver and the placeholder helpers without depending on openshell-sandbox.
Add base64 to openshell-core deps (only stdlib + base64 are used). Promote previously pub(crate) constructors and methods on SecretResolver to pub since cross-crate callers (provider_credentials, proxy/L7 tests) now name them across the crate boundary. Update import paths in proxy.rs, l7/{rest,relay,websocket}.rs, and provider_credentials.rs from crate::secrets to openshell_core::secrets.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift provider_credentials to openshell-core
Move crates/openshell-sandbox/src/provider_credentials.rs to crates/openshell-core/src/provider_credentials.rs. Both supervisor leaves now name ProviderCredentialState in their function signatures (run_networking takes &ProviderCredentialState, run_process takes ProviderCredentialState by value), and under Shape A leaves can't depend on openshell-sandbox, so the type must live in openshell-core.
The orchestrator (run_sandbox in openshell-sandbox) remains the only writer: it constructs ProviderCredentialState::from_environment and the policy poll loop calls install_environment on credential rotation. Both leaves stay pure readers via snapshot()/resolver().
Update import paths in proxy.rs, ssh.rs, and lib.rs from crate::provider_credentials to openshell_core::provider_credentials.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* style: rustfmt import ordering
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(ocsf): move SandboxContext singleton from openshell-sandbox
Move the process-wide OCSF SandboxContext OnceLock + LazyLock fallback + getter from openshell-sandbox/src/lib.rs into a new openshell-ocsf::ctx module. The type already lives in openshell-ocsf, so its singleton lives next to it.
Add openshell_ocsf::ctx::set_ctx() and openshell_ocsf::ctx::ctx(). The orchestrator (run_sandbox) now calls set_ctx during startup. Sandbox keeps a pub(crate) use openshell_ocsf::ctx::ctx as ocsf_ctx; re-export so the 138 existing crate::ocsf_ctx() call sites resolve unchanged.
When the sandbox modules themselves migrate into the leaf crates, they'll import openshell_ocsf::ctx directly and the re-export goes away.
Under Shape A neither leaf can depend on openshell-sandbox; both already depend on openshell-ocsf to construct events, so this adds no new dep edge.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift grpc_client to openshell-core
Both prospective leaves (supervisor-networking and supervisor-process)
need CachedOpenShellClient, AuthedChannel, and the connect/fetch
helpers. Under Shape A the leaves cannot depend on openshell-sandbox,
so the type has to live below them. openshell-core already pulls in
tonic and miette; this enables tonic's channel/tls features and adds
tokio as a direct dep.
Updates all crate::grpc_client::* call sites in openshell-sandbox to
openshell_core::grpc_client::*. No re-export shim — the call-site
count was small enough to update directly.
See architecture/plans/sandbox-split-design-choices.md for the full
rationale and trade-offs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move denial_aggregator from openshell-sandbox
DenialAggregator and FlushableDenialSummary belong with the proxy and L7
layer that emit denials. Moves the file into openshell-supervisor-networking;
adds tokio as a regular dep there since DenialAggregator uses
tokio::sync::mpsc.
Drops the pub use openshell_core::DenialEvent re-export inside the moved
file (no longer needed cross-crate). Updates bypass_monitor.rs, proxy.rs,
and lib.rs to import openshell_core::DenialEvent directly.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move log_push from openshell-sandbox
LogPushLayer is a process-side tracing layer that streams sandbox logs
to the gateway via gRPC. Moves into openshell-supervisor-process; adds
openshell-core, openshell-ocsf, tokio-stream, tracing, and
tracing-subscriber as direct deps there.
Updates the only external call site (openshell-sandbox/src/main.rs) to
import from openshell_supervisor_process::log_push.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move bypass_monitor from openshell-sandbox
bypass_monitor reads /dev/kmsg for nftables drop log lines and emits
denial events. Pure process-side concern, called only from
run_networking which spawns it on the netns. Moves into
openshell-supervisor-process; all deps (openshell-core, openshell-ocsf,
tokio, tracing) were already declared there.
Replaces crate::ocsf_ctx() shim calls inside the moved file with
openshell_ocsf::ctx::ctx() — first leaf-side caller to import the OCSF
context singleton directly instead of going through openshell-sandbox's
re-export.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move debug_rpc from openshell-sandbox
debug_rpc is the CLI subcommand handler that exercises authenticated
gRPC calls (issue-token, refresh-token, get-config, etc.). Pure
process-side concern, called only from openshell-sandbox/main.rs.
Adds base64, hex, serde_json, sha2, and tonic (with channel/tls
features) as direct deps on openshell-supervisor-process. Updates the
single call site in main.rs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move supervisor_session from openshell-sandbox
supervisor_session opens a bidirectional gRPC stream that lets the
gateway initiate shells inside the sandbox. Pure process-side concern,
called only from run_process. Adds uuid as a direct dep on
openshell-supervisor-process.
Replaces crate::ocsf_ctx() shim calls inside the moved file with
openshell_ocsf::ctx::ctx() — same pattern as bypass_monitor.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): lift managed_children tracker from openshell-sandbox
The MANAGED_CHILDREN set tracks PIDs of supervisor-spawned children
(entrypoint + SSH sessions) so the orchestrator's SIGCHLD reaper can
distinguish them from incidental zombies. Pure process-side concern,
moves to openshell_supervisor_process::managed_children with three
public fns: register, unregister, is_managed.
Updates lib.rs reaper, process.rs, and ssh.rs to call through the new
module path. Drops the now-unused HashSet import from lib.rs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move sandbox hardening from openshell-sandbox
Lift the process-only hardening pieces (landlock, seccomp, PreparedSandbox,
prepare/enforce, log_sandbox_readiness, top-level apply, and
apply_supervisor_startup_hardening) from crates/openshell-sandbox/src/sandbox/
to crates/openshell-supervisor-process/src/sandbox/.
Leave netns.rs and nft_ruleset.rs in openshell-sandbox for now, since both
eventual leaf crates (supervisor-networking and supervisor-process) read from
NetworkNamespace and its final home is decided when run_networking and
run_process are extracted.
Replace crate::ocsf_ctx() shims in landlock.rs and the new linux/mod.rs with
direct openshell_ocsf::ctx::ctx() calls. Update call sites in lib.rs,
process.rs, and ssh.rs to import sandbox from openshell_supervisor_process
while keeping the netns import unchanged.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift proposals flag from openshell-supervisor-process
Move proposals.rs (AGENT_PROPOSALS_ENABLED OnceLock + agent_proposals_enabled
reader + test_helpers::ProposalsFlagGuard) from openshell-supervisor-process
to openshell-core so both eventual leaf crates can read it without depending
on each other.
The flag is process-wide singleton state initialised once during sandbox
startup and read by both the policy.local route (networking-side) and the
skills installer (process-side) — same shape as openshell_ocsf::ctx.
Move the test-helpers Cargo feature alongside it: openshell-core gains the
feature, openshell-supervisor-process loses it, and openshell-sandbox's
dev-dependency now enables openshell-core/test-helpers. Update the sandbox
re-export shim to point at openshell_core::proposals.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(core): lift netns + nft_ruleset from openshell-sandbox
Move NetworkNamespace and the nft_ruleset bypass-rule generator from
crates/openshell-sandbox/src/sandbox/linux/ to crates/openshell-core/src/netns/.
Both eventual leaf crates (supervisor-networking and supervisor-process) read
from NetworkNamespace, so it must live somewhere both can depend on without
violating the Shape A no-leaf-to-leaf rule.
Replace crate::ocsf_ctx() shims in netns with direct openshell_ocsf::ctx::ctx()
calls, matching the pattern used in already-migrated process modules. Update
super::nft_ruleset references inside netns to nft_ruleset since the module
is now a sibling sub-module of netns/mod.rs.
Add openshell-ocsf and uuid as linux-only dependencies of openshell-core, and
gate pub mod netns on target_os = "linux" since the implementation uses
netlink, ip(8), and namespace fds. Delete the now-empty sandbox/{mod.rs,
linux/mod.rs} stubs and update NetworkNamespace import paths in lib.rs and
process.rs to point at openshell_core::netns.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move process.rs and ssh.rs from openshell-sandbox
Lift the entrypoint process spawn module and the embedded SSH server
module into openshell-supervisor-process. openshell-sandbox now
re-exports ProcessHandle/ProcessStatus and calls
openshell_supervisor_process::ssh::run_ssh_server directly.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move proxy, l7, opa, policy_local from openshell-sandbox
Lift the egress proxy, L7 enforcement modules, OPA engine, and policy.local
advisor API into openshell-supervisor-networking. Move accompanying data
files (sandbox-policy.rego), test fixtures (testdata/), and integration
tests (system_inference, websocket_upgrade). Sandbox lib.rs now references
these via openshell_supervisor_networking::* and ProxyHandle::start_with_bind_addr
is exposed as pub for the orchestrator call site.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(sandbox): hoist policy poll loop and denial aggregator into orchestrator
Move the symlink-resolver, policy poll loop, and denial-aggregator flush
spawns out of run_process and into run_sandbox so run_process no longer
needs OpaEngine, retained_proto, the local policy context, the sandbox
name, the gateway endpoint for telemetry, the OCSF flag, or the denial
receiver. These long-running orchestrator-owned tasks now live alongside
the other sandbox-startup wiring, matching the design log decision in
architecture/plans/sandbox-split-design-choices.md (Q5).
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move run_process from openshell-sandbox
Lift the workload supervision entry point (zombie reaper, SSH server
spawn, supervisor session, entrypoint child spawn, exit-with-timeout)
into its own module in openshell-supervisor-process. The orchestrator
in openshell-sandbox now calls openshell_supervisor_process::run::run_process
directly. With this move run_process names only types from openshell-core,
openshell-ocsf, openshell-supervisor-process itself, std, and tokio —
no openshell-supervisor-networking dependency.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move bypass_monitor from supervisor-process
Bypass detection is network-policy enforcement: it parses nftables LOG
entries from /dev/kmsg and emits OCSF NetworkActivity / DetectionFinding
events plus DenialEvents into the same channel the proxy feeds. Its
lifetime is tied to the network namespace, not to the workload child.
Moving it to openshell-supervisor-networking puts it next to the proxy
and the denial aggregator that consume its output, and unblocks moving
run_networking out of openshell-sandbox without a leaf-to-leaf dep.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move inference route helpers from openshell-sandbox
Move build_inference_context, partition_routes, bundle_to_resolved_routes,
spawn_route_refresh, the InferenceRouteSource enum, and the route refresh
interval helpers into a new openshell-supervisor-networking::inference_routes
module along with their unit tests. The orchestrator now calls into the
networking leaf for inference context construction; the leaf owns its own
route bundle resolution end-to-end.
The new module is named inference_routes to avoid colliding with the
existing l7::inference module, which handles request-time HTTP parsing
and pattern matching rather than route bundle setup.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-networking): move run_networking from openshell-sandbox
Move the Networking handle struct, run_networking, and the Linux-only
create_netns_for_proxy helper into a new openshell-supervisor-networking::run
module. The orchestrator in openshell-sandbox now invokes
openshell_supervisor_networking::run::{create_netns_for_proxy, run_networking}
and reads the Networking fields directly; the leaf owns the entire
networking-stack startup path (CA generation, proxy task, bypass monitor,
inference context, denial channel) end-to-end.
The Networking RAII handle fields (proxy, bypass_monitor) are now public
without leading underscores so the public API satisfies clippy's
pub_underscore_fields lint while still serving as drop guards held by the
orchestrator's frame.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* fix(workspace): align Cargo deps and call sites for split crates
The recent module lifts left two Linux-only gaps that the macOS host
workspace check skipped:
- openshell-core's netns module needs libc, tempfile, and nix on Linux,
but only openshell-ocsf and uuid were carried over.
- openshell-supervisor-process's seccomp/landlock modules need landlock
and seccompiler, which still lived on openshell-sandbox.
- openshell-sandbox's runtime_pid_limit branch referenced an unqualified
process:: that pointed at the old in-crate module.
Move landlock/seccompiler to supervisor-process, add the missing core
deps, qualify the call sites, and drop sandbox deps that no longer have
runtime users (landlock, seccompiler, target-gated tempfile/uuid, the
unix libc/rustix block).
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): rename openshell-supervisor-networking to openshell-supervisor-network
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): own denial-aggregator flush end-to-end
Move the denial-aggregator spawn and flush_proposals_to_gateway out of
run_sandbox and into run_networking. The networking leaf already owns
every other input (proxy + bypass_monitor as producers, denial channel,
mechanistic_mapper, denial_aggregator) and already opens its own gRPC
connections (inference_routes, policy_local) — the orchestrator was the
only piece left straddling the boundary.
Networking now drives the full path: producers -> channel -> aggregator
-> flush -> gateway. Drops denial_rx from Networking; adds sandbox_name
to run_networking so SubmitPolicyAnalysis can resolve by sandbox name
(falls back to ID when unset). Same shape as log_push in the process leaf.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): own symlink-resolution task
Move the OPA binary-symlink resolver out of run_sandbox and into
run_networking. The task probes /proc/<entrypoint_pid>/root/ until the
workload's mount namespace is accessible, then rebuilds the OPA engine
with resolved binary paths so policy rules match canonical names instead
of symlinks.
Both inputs (Arc<OpaEngine>, retained_proto) are networking-leaf concerns
and were already plumbed into run_networking; the entrypoint_pid Arc is
read lazily after the process leaf populates it. Adds retained_proto as
a parameter and spawns the resolver early in run_networking so the probe
loop starts before the proxy comes up.
Same shape as the denial-flush move: networking owns its own background
task end-to-end; the orchestrator stops hosting work that doesn't
conceptually belong to it.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move seccomp install into run_process
The supervisor seccomp prelude is part of "set up the workload-side
process tree", not part of orchestration. Move the call site from
run_sandbox into the top of run_process and drop the now-unused
re-export from openshell-sandbox::lib.
Timing is preserved: by the time the orchestrator calls run_process,
run_networking has already returned, so netns + nftables setup is
complete.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move check_runtime_pid_limit into run_process
The PID-limit precondition is process-side: it gates whether the workload
child can be spawned at all. Move the call from run_sandbox into the top
of run_process, alongside the seccomp prelude. Same shape as the seccomp
move — function already lives process-side, only the call site relocates.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move validate_sandbox_user to process crate
The sandbox-user check is a precondition for privilege-dropping the
workload child; it has no relevance to networking. Move the function
next to drop_privileges in openshell-supervisor-process::process and
call it from the top of run_process.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move prepare_filesystem to process crate
Creating and chowning read_write directories is workload-side
preparation, not orchestration. Move prepare_filesystem and its
prepare_read_write_path helper (plus tests) into
openshell-supervisor-process::process and call from run_process,
alongside validate_sandbox_user.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-process): move startup skill install into run_process
The eager initial-settings fetch + agent skill install is process-side:
the install materializes files the workload's filesystem sees. The
orchestrator still owns the AGENT_PROPOSALS_ENABLED OnceLock init
because the policy poll loop also reads it; only the early fetch and
install hop into run_process.
Behavior unchanged. Best-effort: any RPC or install failure is logged
but does not fail startup.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): own PolicyLocalContext construction
Move the PolicyLocalContext construction from run_sandbox into
run_networking. The orchestrator was building it solely to thread it into
the networking leaf and to share it with the policy poll loop; now
run_networking builds it from inputs it already takes (retained_proto,
openshell_endpoint, sandbox_name|sandbox_id) and exposes it on the
returned Networking struct.
The orchestrator's poll loop now grabs the Arc clone from
networking.policy_local_ctx, so the orchestrator no longer imports
openshell_supervisor_network::policy_local.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* feat(supervisor): add --mode flag to gate network/process leaves
Add a --mode flag (default "network,process") that selects which
supervisor leaves run in the current process. Two new shapes are
unlocked without splitting the binary:
--mode=network # network-only sidecar
--mode=process # process-only supervisor
--mode=network,process # combined (default; current behavior)
In network-only mode the orchestrator skips run_process and waits on
SIGINT/SIGTERM before tearing down the proxy. The entrypoint PID stays
at 0 for the lifetime of the process, which silently degrades the
proxy's binary-identity TOFU and the bypass monitor's PID enrichment;
this is correct in a split-pod topology where the workload's /proc
lives in another pod.
In process-only mode run_networking is skipped entirely. SSH sessions
get no proxy URL, no netns FD, and no CA paths, matching what a
split-pod consumer would expect when network enforcement is delegated
to a sidecar.
The policy poll loop continues to run unconditionally; its OPA-reload
and policy.local hooks already gate on the resources only present when
network is enabled, and the env-refresh / proposals-toggle hooks
remain active in process mode.
Closes a step toward the RFC-0001 supervisor topology proposed in
issue #1305 by drew.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* style(supervisor-process): rustfmt long debug! line
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): pull DenialEvent down from core
DenialEvent is only emitted and consumed inside openshell-supervisor-network
(proxy, bypass monitor, denial aggregator). It never crossed the leaf
boundary, so the earlier lift to openshell-core was speculative. Move it
back into the network crate where its only callers live.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(supervisor-network): pull procfs down from core
procfs was lifted to openshell-core under the assumption it would be
shared cross-leaf, but on the current branch all three callers
(bypass_monitor, identity, proxy) live in openshell-supervisor-network.
No file in openshell-supervisor-process imports it. Move the module to
the network crate and drop sha2/hex from openshell-core, which were
pulled in only for procfs.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* style(supervisor-network): run cargo fmt
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* fix(supervisor-network): add libc dev-dependency for procfs tests
The procfs/bypass_monitor/proxy test modules use libc::{fork, exec,
fcntl, kill, waitpid} but the dep wasn't declared in this crate's
Cargo.toml. It was previously satisfied transitively when these
modules lived in openshell-core; the move left the test target
unable to resolve libc.
Signed-off-by: Radoslav Hubenov <rrhubenov@gmail.com>
* refactor(sandbox): move denial aggregator to orchestrator
The denial aggregator and mechanistic mapper consume denial events
produced by the proxy and (subsequently) the bypass monitor. With both
supervisor leaves becoming pure producers of `DenialEvent`, the
consumer-side aggregation belongs in the orchestrator, not in either
leaf.
Move `denial_aggregator.rs` and `mechanistic_mapper.rs` from
`openshell-supervisor-network` to `openshell-sandbox` (the
orchestrator). The orchestrator now owns the unbounded denial channel:
it constructs `(tx, rx)`, hands `tx` to `run_networking` for the proxy
to clone, drains `rx` via the aggregator task, and runs the gateway
flush helper itself.
`run_networking`'s signature gains a `denial_tx` parameter and loses
its internal channel construction, aggregator spawn, and
`flush_proposals_to_gateway` helper. `DenialEvent` stays in
`openshell-supervisor-network` for now; a follow-up commit will lift
it to `openshell-core` alongside the bypass monitor relocation.
* refactor(supervisor-process): pull bypass monitor down from network
`bypass_monitor` is process-isolation machinery: it tails the kernel
log via `dmesg --follow`, parses nftables LOG lines emitted from the
workload's network namespace, resolves PIDs via `/proc`, and emits
OCSF events plus optional `DenialEvent`s. None of this touches the
proxy, OPA, TLS, or any other supervisor-network state — it only
shared the denial channel because both feed the same aggregator.
Move `bypass_monitor.rs` from `openshell-supervisor-network` to
`openshell-supervisor-process` (as `bypass_monitor/mod.rs`). Spawn it
in `run_process` where the netns name and entrypoint PID are already
in scope. The orchestrator hands an extra `bypass_denial_tx` clone of
the denial channel sender to `run_process` for this purpose.
Lift `DenialEvent` from `openshell-supervisor-network` to
`openshell-core`. Both supervisor leaves now produce it, so it needs
a shared location that neither leaf depends on. This reverses an
earlier commit that pulled the type into the network leaf when it was
the only producer.
Copy the minimal subset of `/proc` parsers used by `bypass_monitor`
into a private `bypass_monitor::procfs` submodule. The alternative —
extracting a shared procfs crate — is a much larger refactor that
this commit does not need; supervisor-network's `procfs.rs` continues
to serve the proxy and identity cache.
* refactor(supervisor-process): derive ssh netns fd inside run_process
The ssh_netns_fd was computed in run_networking purely to forward it
through the Networking struct and back into run_process. supervisor-network
never read it. Move the derivation to run_process where the
NetworkNamespace handle is already in scope.
* refactor(supervisor-process): derive ssh proxy url inside run_process
The ssh_proxy_url was computed in run_networking purely to forward it
through the Networking struct and back into run_process. supervisor-network
never read it. Move the derivation to run_process where the
NetworkNamespace handle and SandboxPolicy are already in scope.
After this commit the Networking struct no longer carries any SSH-shaped
fields, and supervisor-network reads only host_ip from the netns (for the
proxy bind address).
* refactor(supervisor-network): take proxy bind ip directly instead of netns
run_networking only ever read host_ip from the netns it was passed (the
SSH plumbing reads moved to run_process in earlier commits). Replace the
NetworkNamespace parameter with a plain Option<IpAddr> the orchestrator
extracts. supervisor-network's run module no longer references the netns
type for any consumer, only for create_netns_for_proxy (which still lives
in this crate; relocates next).
* refactor(supervisor-process): move netns ownership out of core
Relocates the NetworkNamespace handle, nft ruleset builder, and
create_netns_for_proxy constructor into openshell-supervisor-process.
The orchestrator (openshell-sandbox) phantom-owns the RAII handle for
the duration of run_sandbox; supervisor-network no longer references
the type at all.
Drops uuid, libc, nix, openshell-ocsf, and tempfile from core's Linux
target deps (all were exclusive to netns). tempfile becomes a Linux
runtime dep on supervisor-process for nft ruleset application.
* chore(sandbox): prune leaf-only deps from orchestrator manifest
cargo-machete flagged 26 direct dependencies that were carried over
from the pre-split monolith and are no longer used by the orchestrator
itself: regorus, russh, rcgen, tokio-rustls, ipnet, apollo-parser,
openshell-router, anyhow, base64, bytes, flate2, glob, hex, hmac, nix,
rand_core, rustls-pemfile, serde, serde_yml, sha1, sha2, thiserror,
tokio-stream, uuid, webpki-roots.
These now live (transitively) in openshell-supervisor-network and
openshell-supervisor-process where they are actually consumed.
* chore(deps): prune unused deps from supervisor crates
- Drop unused `url` from openshell-supervisor-network.
- Mark `prost` and `prost-types` as cargo-machete-ignored in
openshell-core: they have no source-level `use`, but the tonic-
generated proto code references them via `::prost::Message` etc.
- openshell-supervisor-process is already clean.
* fix(supervisor-network): wait for entrypoint PID before symlink probe
The OPA symlink-resolution task reads entrypoint_pid once at the top
of the spawned closure. Because the spawn happens before run_process
publishes the workload PID, the load returns 0, the probe path bakes
in as /proc/0/root/, and the loop exhausts its retries against a path
that does not exist on Linux. The reload never fires, so policies that
whitelist symlinked binaries (e.g. /usr/bin/python3 → python3.11) get
silent denials when the workload exec's the realpath.
Split the wait into two phases: 5s polling entrypoint_pid for a
non-zero value, then the existing 5s window probing /proc/<pid>/root/.
Distinct warn messages on each timeout so future debugging can tell
"PID never published" apart from "container fs never appeared".
* fix(sandbox): restore GPU procfs baseline (#1522)
Signed-off-by: Evan Lezar <elezar@nvidia.com>
(cherry picked from commit
|
||
|
alangou
|
b6c87a76ab |
feat(server): add grpc rate limiting gateway-wide (#1566)
Signed-off-by: Adrien Langou <alangou@nvidia.com> |
||
|
Evan Lezar
|
1dc59853ff |
feat(gpu): move device selection to driver config (#1815)
* refactor(kubernetes): use shared proto struct helpers Signed-off-by: Evan Lezar <elezar@nvidia.com> * refactor(docker): extract sandbox template validation * feat(gpu)!: move device selection to driver config BREAKING CHANGE: The openshell sandbox create --gpu-device flag and corresponding API field were removed. Select specific GPUs through driver-specific driver_config fields instead. Signed-off-by: Evan Lezar <elezar@nvidia.com> * fix(kubernetes): reject unknown driver config fields Signed-off-by: Evan Lezar <elezar@nvidia.com> --------- Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Drew Newberry
|
530aaf1360 |
feat(drivers): support docker and podman config mounts (#1785)
* feat(drivers): support docker and podman config mounts Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(drivers): trim mount docs Signed-off-by: Drew Newberry <anewberry@nvidia.com> * test(e2e): cover local driver volume mounts Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(podman): satisfy linux clippy lint Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(drivers): gate bind mounts behind gateway config * docs(sandbox): simplify mount examples * cleanup * test(e2e): stabilize branch checks * fix(drivers): tighten local mount validation Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Taylor Mutch
|
702cbc4f63 |
feat(providers): support SPIFFE-backed token grants (#1784)
* feat(providers): support SPIFFE-backed token grants Add provider profile token_grant metadata and expand endpoint-specific dynamic credentials so sandbox supervisors can request SPIFFE JWT-SVIDs, exchange them with an OAuth-style token endpoint, cache returned access tokens, and inject bearer tokens into matching HTTP requests. Wire Kubernetes and Helm deployments to mount the provider SPIFFE Workload API socket into sandbox pods for token grant exchange. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Gordon Sim <gsim@redhat.com> * test(examples): add SPIFFE token grant demo Add a reusable alpha/beta demo that deploys a SPIFFE-verifying token issuer and protected services, imports a token-grant provider profile, creates a sandbox, and verifies endpoint-specific bearer tokens. The script leaves Kubernetes workloads in place, deletes sandboxes through openshell unless KEEP_SANDBOX=1, and prints protected service logs as proof of life. Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(providers): harden SPIFFE token grants * fix(providers): harden dynamic token grants Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(providers): harden token grant handling --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Gordon Sim <gsim@redhat.com> |
||
|
LateNightHackathonandlatenighthackathon
|
c1d3b43ddb |
fix(policy): classify advisory private-IP notes with the canonical is_internal_ip (#1777) (#1824)
generate_security_notes() flagged a destination as internal/private using naive
string prefixes. host.starts_with("172.") matched all of 172.0.0.0/8 even
though only 172.16.0.0/12 is private (RFC 1918), so the advisory flagged public
addresses (172.15/172.32) and missed CGNAT (100.64.0.0/10), IPv6 ULA, and the
other reserved ranges; it also flagged non-IP hostnames that merely start with
10./172.. The two copies of the check (openshell-sandbox mechanistic_mapper.rs
and openshell-server grpc/policy.rs) now parse the host as an IP and defer to
openshell-core net::is_internal_ip, the same RFC-accurate, unit-tested classifier
the runtime SSRF enforcement path uses, so the proposed-rule advisory matches
actual enforcement.
Per review, is_internal_v4 now uses the stable Ipv4Addr predicates as far as
they go: is_documentation() (RFC 5737) replaces the two manual TEST-NET-2/3
checks and also closes a gap, since 192.0.2.0/24 (TEST-NET-1) was previously
unflagged, and is_broadcast() (RFC 919) is added. Manual checks remain only for
the ranges without a stable predicate: CGNAT 100.64.0.0/10, IETF protocol
assignments 192.0.0.0/24, and benchmarking 198.18.0.0/15.
Adds regression coverage: both note suites now assert hostname-prefix false
positives (10.example.com, 172.example.com) are not flagged and IPv6 ULA
(fd00::1) is flagged; net.rs covers the newly classified TEST-NET-1 and
broadcast addresses.
Closes #1777
Signed-off-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
Co-authored-by: latenighthackathon <latenighthackathon@users.noreply.github.com>
|
||
|
Eric Curtin
|
70acbaf407 |
refactor(driver-utils): centralize container mount path constants (#1841)
Move duplicate in-container path constants from the Docker, Podman, Kubernetes, and VM driver crates into openshell-core::driver_utils: - TLS_CA_MOUNT_PATH, TLS_CERT_MOUNT_PATH, TLS_KEY_MOUNT_PATH, SANDBOX_TOKEN_MOUNT_PATH — identical string literals that were copy-pasted between openshell-driver-docker and openshell-driver-podman. - SUPERVISOR_CONTAINER_DIR (/opt/openshell/bin) and SUPERVISOR_CONTAINER_BINARY (/opt/openshell/bin/openshell-sandbox) — replaces three diverging local constants across docker, kubernetes, podman, and vm-rootfs that referred to the same paths under different names with different string values (the k8s constant was the directory; the docker/vm constant was the full binary path). Each driver crate now assigns its local constant from the canonical core value, keeping existing names visible to tests without any call-site churn. The openshell-core constants carry doc comments explaining when to use the dir vs. the full binary path. Signed-off-by: Eric Curtin <eric.curtin@docker.com> |
||
|
Russell Bryant
|
3aba30c36f |
feat(telemetry): add build-time option to compile out telemetry (#1845)
* feat(telemetry): add build-time option to compile out telemetry Gate anonymous telemetry emission behind a default-on `telemetry` Cargo feature in openshell-core. The data model (enums, validation, emit_*/enabled* signatures) stays always-compiled, while the endpoint, HTTP client, queue, and emission code are feature-gated. With the feature off, enabled() returns false and emit_* are no-ops, so dependent crates compile unchanged and no telemetry endpoint, HTTP client, or emission code is included in the binary. chrono and reqwest become optional dependencies of openshell-core, dropped from its dependency graph when telemetry is disabled. Thread the switch through the workspace: every crate depends on openshell-core with default-features = false, and the default-on `telemetry` passthrough lives on the binary crates that emit or collect telemetry (openshell-server, openshell-sandbox, openshell-driver-vm). In-process drivers inherit it via resolver v2 feature unification. Build a telemetry-free binary with, e.g.: cargo build --release -p openshell-server --no-default-features The runtime OPENSHELL_TELEMETRY_ENABLED switch is unchanged for default builds. Signed-off-by: Russell Bryant <russell.bryant@gmail.com> * ci(telemetry): guard that telemetry can be compiled out Add tasks/scripts/verify-telemetry-compiled-out.sh, which inspects a built binary for telemetry markers (the telemetry endpoint host and client ID) that exist only when emission code is compiled in. The rust:verify:telemetry-off mise task builds the gateway with default features (positive control: markers must be present, so the absent checks can never be silently vacuous) and with --no-default-features (markers must be absent), and checks the --no-default-features sandbox binary as well. Wire the task into the Rust branch-checks job so a regression that reintroduces telemetry code into a --no-default-features build fails CI. Signed-off-by: Russell Bryant <russell.bryant@gmail.com> --------- Signed-off-by: Russell Bryant <russell.bryant@gmail.com> |
||
|
Russell Bryant
|
7274a6bea3 |
feat(cli): add --env flag to sandbox create/exec and fix env var passthrough (#1730)
* feat(cli): add --env flag to sandbox create and exec
Add `--env KEY=VALUE` (repeatable) to `openshell sandbox create` and
`openshell sandbox exec`, wiring user-specified environment variables
into the existing `SandboxSpec.environment` and
`ExecSandboxRequest.environment` protobuf fields which were previously
only accessible via the SDK/gRPC API.
Fix a bug where user-specified environment variables set via
`SandboxSpec.environment` were silently dropped inside sandbox SSH
sessions. The SSH handler's `apply_child_env()` calls `env_clear()` to
strip supervisor-internal secrets before spawning user shells, but this
also wiped container-level env vars including user-specified ones. This
was introduced in
|
||
|
Shiju
|
25abc9e3c7 |
feat(inference): allow local embeddings route (#1774)
* feat(inference): allow local embeddings route Route OpenAI-compatible embeddings through the local inference proxy so sandboxed vector workloads reach a configured provider via the same route classification and auth path that chat, completion, and model discovery already use. - Add openai_embeddings to the OpenAI-compatible protocol set so providers (openai, nvidia) advertise embeddings routing. - Classify POST /v1/embeddings as the openai_embeddings protocol in the sandbox L7 patterns. - Serve embeddings buffered with an accurate Content-Length, since the response is a single JSON object rather than an SSE token stream. The streaming path appends an SSE error frame on a size-cap or idle-timeout truncation, which would corrupt a one-object body the client parses whole. protocol_returns_buffered_body() selects the path. - Probe an embeddings-only backend against /v1/embeddings during validation, after the chat and completion protocols so a multi-protocol route still prefers those. - Extract two shared helpers. http_status_text() backs both response formatters and adds 401/422/429/503 for embeddings passthrough and router error mapping; write_inference_router_error() backs the streaming and buffered routing paths. - Return an OpenAI-shaped embeddings body from the mock route. Tests cover profile lookup, L7 pattern detection, the mock body, and buffered Content-Length framing with no chunked transfer-encoding and no SSE error frame. Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): cap buffered inference response body The buffered proxy path read the whole upstream response into memory with no size bound. The route timeout bounds elapsed time but not memory, so a misbehaving or oversized upstream could force unbounded allocation in the sandbox proxy. The streaming path already caps each response at 32 MiB; the buffered path did not. Cap the buffered read at the same 32 MiB. An advertised over-cap body is rejected from its Content-Length before any bytes are read, and chunks accumulate under the same bound so a chunked or mislabeled body cannot slip past. An over-cap response fails as an upstream protocol error, surfaced as HTTP 502 at the proxy boundary, and is never partially returned. Tests - cargo test -p openshell-router \ proxy_to_backend_rejects_over_cap_response_body Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): validate embeddings models against all advertised protocols A managed route resolves to its provider profile's full protocol set, so an embeddings model such as text-embedding-3-small lists openai_chat_completions alongside openai_embeddings. Route verification probed only the first writable protocol and stopped on its failure. It sent a chat probe with the embedding model, the provider rejected it as wrong-shape, and the route failed validation before the embeddings probe ran. Embeddings-only configs could not be verified. Try the advertised protocols in preference order. A request-shape rejection (HTTP 400, 404, 405, 422) falls through to the next protocol, so an embeddings model validates against /v1/embeddings even when the chat probe rejects it. Credential, rate-limit, connectivity, and upstream-health failures stay terminal and stop validation at the first probe, so a bad key or a down backend is reported as itself rather than masked by a later probe. validation_probe becomes validation_probes, which returns the ordered list, and the per-probe fallback retry (max_completion_tokens versus max_tokens) moves into a shared helper. Tests - cargo test -p openshell-router \ verify_embeddings_model_falls_through_chat_probe - cargo test -p openshell-router verify_stops_on_credentials_failure Signed-off-by: Shiju <shiju@nvidia.com> * fix(inference): serve model discovery responses buffered GET /v1/models returns a single JSON model list, the same response shape as embeddings. The sandbox inference proxy was routing it through the SSE streaming path. A streaming size-cap or idle-timeout truncation appends an SSE error frame to the body, which corrupts a payload the client parses as one JSON object. Make response framing a property of the protocol. A new ResponseFraming field on InferenceApiPattern is set once per pattern in default_patterns. model_discovery and openai_embeddings are now Buffered, while chat completions, completions, responses, and Anthropic messages stay Streaming. The proxy dispatch gates on pattern.is_buffered(), which replaces the stringly-typed protocol_returns_buffered_body predicate so the streaming-versus-buffered decision lives in one place and cannot drift across the sites that read it. Model discovery now flows through the same buffered path as embeddings, framed with an accurate Content-Length and bounded by the buffered-read size cap that path already enforces. Tests - cargo test -p openshell-sandbox protocol_framing_classification - cargo test -p openshell-sandbox \ inference_model_discovery_served_buffered_with_content_length Signed-off-by: Shiju <shiju@nvidia.com> * docs(inference): document embeddings route in supported patterns Signed-off-by: Shiju <shiju@nvidia.com> --------- Signed-off-by: Shiju <shiju@nvidia.com> |