diff --git a/crates/openshell-core/src/provider_credentials.rs b/crates/openshell-core/src/provider_credentials.rs index f82fd066a..bc0cc2bd2 100644 --- a/crates/openshell-core/src/provider_credentials.rs +++ b/crates/openshell-core/src/provider_credentials.rs @@ -373,6 +373,18 @@ impl ProviderCredentialState { .revision } + /// Whether this snapshot contains endpoint-bound material that must be + /// resolved by a network proxy rather than exposed to the child process. + #[must_use] + pub fn requires_proxy_resolution(&self) -> bool { + let inner = self + .inner + .read() + .expect("provider credential state poisoned"); + !inner.static_credential_bindings.is_empty() + || !inner.current.dynamic_credentials.is_empty() + } + /// Remove a key from the credential snapshot's child env. /// /// Used when a sandbox-side service (e.g., metadata server) fails to start diff --git a/crates/openshell-driver-mxc/README.md b/crates/openshell-driver-mxc/README.md index 46b8839a7..d3a7a78c5 100644 --- a/crates/openshell-driver-mxc/README.md +++ b/crates/openshell-driver-mxc/README.md @@ -7,9 +7,10 @@ OpenShell compute driver backed by **Microsoft MXC** (`wxc-exec`) on Windows. This driver implements the gateway's ordinary in-process `ComputeDriver` contract and is linked into `openshell-gateway`. It sets `driver_reports_runtime_readiness`, so the gateway accepts driver-reported -readiness without a supervisor session. The canonical create-time -`SandboxPolicy` is carried by `DriverSandboxSpec.policy`. `process_container` launches a one-shot -AppContainer and is the default. The opt-in `isolation_session` backend uses the +readiness without a supervisor session. The gateway composes the create-time +effective `SandboxPolicy` and carries it on the driver-only copy of +`DriverSandboxSpec.policy`. `process_container` launches a one-shot AppContainer +and is the default. The opt-in `isolation_session` backend uses the state-aware `provision` → `start` → `exec` → `stop` → `deprovision` lifecycle. The driver launches and monitors the configured workload itself and self-reports readiness; there is no in-sandbox supervisor or `ConnectSupervisor` relay. @@ -22,6 +23,7 @@ readiness; there is no in-sandbox supervisor or `ConnectSupervisor` relay. | UI policy | `process_container` advertises complete support and maps portable graphical UI, clipboard-direction, and input-injection controls to MXC; omitted fields inside an explicit section deny. `isolation_session` advertises no support, so the gateway rejects any explicit section before provisioning. | MXC support for persistent sessions | | Governed egress (CONNECT proxy + OPA + L7) | Available behind `egress_proxy` on `process_container`; the driver starts a per-sandbox host CONNECT proxy, generates HTTPS MITM trust material, and injects the CA bundle into the sandbox process env | Gateway event-bus wiring follow-on | | Network policy | Split into MXC `network.proxy` + trimmed OpenShell policy on `process_container`; `isolation_session` still rejects network config | MXC feedback item M1 for persistent sessions | +| Provider credentials | The child receives revision-scoped placeholders and non-secret provider environment only. The per-sandbox host proxy retains the resolver and substitutes credentials only for their bound endpoints. | — | | Network middleware | ❌ rejected before launch because the MXC host proxy does not receive the gateway middleware registry | Gateway middleware-registry injection | | Process policy (seccomp, uid/gid) | ❌ host-side governance design; OS isolation only | not pursued | | Interactive exec/connect/forward | ❌ exec runs in-driver, no client attach | gateway interactive-exec surgery (follow-on) | @@ -66,7 +68,15 @@ openshell sandbox create --name mxc-demo --policy demo.yaml ` --driver-config-json $config --env MODE=demo --no-tty ``` -The `command` array is required and preserves Windows argument boundaries. `cwd` is optional. Environment variables come from the standard sandbox and template environment maps; the driver never copies values from the gateway host environment. +The `command` array is required and preserves Windows argument boundaries. `cwd` +is optional. Environment variables come from the standard sandbox and template +environment maps; the driver never copies values from the gateway host +environment. Provider-owned keys override matching entries case-insensitively. +MXC receives revision-scoped placeholders and explicitly classified non-secret +provider configuration, while credential values remain in the host proxy. When +governed egress is enabled, the driver replaces common TLS trust environment +variables with paths to the proxy-generated CA material and grants that +sandbox-unique CA directory read-write so the AppContainer can read it. The host CONNECT proxy enforces network policy when governed egress is enabled. Live policy replacement or merge updates remain unsupported; delete and recreate @@ -117,7 +127,9 @@ The production driver maps the typed `SandboxPolicy` carried by the standard driver request to MXC configuration before it inserts a registry entry or invokes `wxc-exec`. Mapping failure therefore returns from `CreateSandbox` without leaving a partial sandbox. There is no in-process policy side channel -or MXC-specific gateway composition variant. +or MXC-specific gateway composition variant. Provider resolver state uses a +separate, create-scoped in-process handoff because it intentionally cannot be +represented in the public compute-driver protobuf. When `egress_proxy` is enabled, `EmbeddedPolicyMapper` uses `split_policy` instead: MXC receives filesystem grants plus a loopback `network.proxy` @@ -156,6 +168,17 @@ If governed egress is disabled, any network rule fails closed rather than launch Parity and matrix tests under [`tests/`](tests/) cover the mapper on the Windows MSVC lane. The real-MXC lane also dry-runs every clipboard direction against the installed schema. The driver performs this mapping automatically; there is no separate policy-export command or example. +## Provider credential example + +[`examples/run-provider-credential-test.ps1`](examples/run-provider-credential-test.ps1) +creates an MXC sandbox with an attached GitHub provider. Its policy explicitly +allows the graphical UI subsystem required by Windows PowerShell while denying +clipboard access and input injection; the existing policy mapper translates +that portable section to MXC's `ui` object. The probe verifies that the sandbox +sees a revision-scoped `GITHUB_TOKEN` placeholder, the host CONNECT proxy +substitutes it for `api.github.com`, and the same placeholder is rejected for a +different allowed endpoint. + ## Packaging the demo for the demo box Use [`examples/package-demo.ps1`](examples/package-demo.ps1) to assemble diff --git a/crates/openshell-driver-mxc/examples/run-provider-credential-test.ps1 b/crates/openshell-driver-mxc/examples/run-provider-credential-test.ps1 index 250b9f341..56aff38dd 100644 --- a/crates/openshell-driver-mxc/examples/run-provider-credential-test.ps1 +++ b/crates/openshell-driver-mxc/examples/run-provider-credential-test.ps1 @@ -243,7 +243,6 @@ try { $env:OPENSHELL_GATEWAY = "" Invoke-Cli @("gateway", "add", "http://127.0.0.1:$Port", "--local", "--name", $GatewayName) | Out-Null Invoke-Cli @("gateway", "select", $GatewayName) | Out-Null - Invoke-Cli @("settings", "set", "--global", "--yes", "--key", "providers_v2_enabled", "--value", "true") | Out-Null Invoke-Cli @("provider", "profile", "lint", "--file", $profileUsed) | Out-Null Invoke-Cli @("provider", "profile", "import", "--file", $profileUsed) | Out-Null Invoke-Cli @("provider", "create", "--name", $providerName, "--type", "mxc-github-e2e", "--credential", "GITHUB_TOKEN") | Out-Null diff --git a/crates/openshell-driver-mxc/src/driver.rs b/crates/openshell-driver-mxc/src/driver.rs index 651ae10f7..d0024a3e0 100644 --- a/crates/openshell-driver-mxc/src/driver.rs +++ b/crates/openshell-driver-mxc/src/driver.rs @@ -15,12 +15,13 @@ use openshell_core::proto::compute::v1::{ WatchSandboxesPlatformEvent, WatchSandboxesSandboxEvent, watch_sandboxes_event, }; use openshell_core::proto_struct::struct_to_json_value; +use openshell_core::provider_credentials::ProviderCredentialState; use serde::{Deserialize, Serialize}; use std::collections::HashMap; use std::net::SocketAddr; use std::path::PathBuf; use std::pin::Pin; -use std::sync::Arc; +use std::sync::{Arc, Mutex as StdMutex}; use tokio::sync::{Mutex, broadcast, mpsc, watch}; use tokio::task::JoinHandle; use tokio_stream::wrappers::ReceiverStream; @@ -206,6 +207,10 @@ pub struct MxcComputeBackend { registry: Arc>>, watch_tx: Arc>, policy_mapper: Arc, + /// Provider resolver snapshots staged by the gateway immediately before + /// create. The driver consumes each entry exactly once; only placeholder + /// child environment values cross into MXC. + pending_provider_credentials: Arc>>, /// In-process ETW → OCSF audit consumer (Plane A). `Some` only when /// `config.etw_audit` is set and the session started; kept alive here so it /// stops when the backend is dropped (held purely for its `Drop`, hence @@ -282,6 +287,37 @@ fn sandbox_environment(sandbox: &DriverSandbox) -> Vec { environment } +/// Merge provider-owned child environment values into MXC `process.env`. +/// +/// Provider entries win case-insensitively, matching Windows environment +/// semantics. Secret values have already been replaced by revision-scoped +/// placeholders; explicitly classified GCP configuration is resolved by the +/// shared credential state because SDKs consume it before making a request. +fn append_provider_child_env( + env: &mut Vec, + provider_credentials: Option<&ProviderCredentialState>, +) { + let Some(provider_credentials) = provider_credentials else { + return; + }; + let mut provider_env = provider_credentials + .child_env_with_gcp_resolved() + .into_iter() + .collect::>(); + provider_env.sort_by(|(left, _), (right, _)| left.cmp(right)); + env.retain(|entry| { + let key = entry.split_once('=').map_or(entry.as_str(), |(key, _)| key); + !provider_env + .iter() + .any(|(provider_key, _)| key.eq_ignore_ascii_case(provider_key)) + }); + env.extend( + provider_env + .into_iter() + .map(|(key, value)| format!("{key}={value}")), + ); +} + fn configured_egress_addr(config: &MxcComputeConfig) -> Result, tonic::Status> { if !config.egress_proxy { return Ok(None); @@ -444,11 +480,19 @@ impl MxcComputeBackend { // Production policy translation is always handled by the embedded // mapper before any MXC lifecycle side effects begin. policy_mapper: Arc::new(EmbeddedPolicyMapper), + pending_provider_credentials: Arc::new(StdMutex::new(HashMap::new())), etw_session, attribution, } } + /// Return the in-process create-time provider credential side channel. + pub fn provider_credentials_sink( + &self, + ) -> Arc>> { + self.pending_provider_credentials.clone() + } + /// Test-only constructor wiring the in-process mock `wxc-exec` shim. #[cfg(test)] pub(crate) fn new_mocked(config: MxcComputeConfig) -> Self { @@ -537,6 +581,14 @@ impl MxcComputeBackend { pub async fn create_sandbox(&self, sandbox: &DriverSandbox) -> Result<(), tonic::Status> { let sandbox_id = sandbox.id.clone(); + // Consume before any fallible validation so rejected creates cannot + // retain real provider material in the staging map. + let provider_credentials = self + .pending_provider_credentials + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .remove(&sandbox_id); + Self::validate_sandbox_fields(sandbox)?; let sandbox_config = sandbox_config(sandbox)?; let (egress_addr, reserved_proxy_listener) = match configured_egress_addr(&self.config)? { @@ -558,6 +610,15 @@ impl MxcComputeBackend { // synchronously at the CreateSandbox boundary. let policy = sandbox.spec.as_ref().and_then(|spec| spec.policy.as_ref()); let mapped = self.map_sandbox_policy(&sandbox_id, policy, egress_addr)?; + if provider_credentials + .as_ref() + .is_some_and(ProviderCredentialState::requires_proxy_resolution) + && egress_addr.is_none() + { + return Err(tonic::Status::failed_precondition( + "mxc provider credentials require governed egress; enable egress_proxy so placeholders can be resolved by the host proxy", + )); + } if sandbox .spec @@ -628,6 +689,7 @@ impl MxcComputeBackend { sandbox, sandbox_config, mapped, + provider_credentials, reserved_proxy_listener, startup_guard, ) @@ -826,6 +888,7 @@ async fn run_lifecycle( sandbox: DriverSandbox, sandbox_config: MxcSandboxConfig, mapped: MappedConfig, + provider_credentials: Option, mut reserved_proxy_listener: Option, _startup_guard: tokio::sync::OwnedMutexGuard<()>, ) { @@ -845,7 +908,7 @@ async fn run_lifecycle( sandbox_id: Some(sandbox_id.clone()), sandbox_name: Some(sandbox_name.clone()), openshell_endpoint: None, - provider_credentials: None, + provider_credentials: provider_credentials.clone(), agent_proposals: openshell_core::proposals::AgentProposals::default(), denial_tx: None, activity_tx: None, @@ -906,6 +969,7 @@ async fn run_lifecycle( }; let command_line = encode_windows_command_line(&sandbox_config.command); let mut environment = sandbox_environment(&sandbox); + append_provider_child_env(&mut environment, provider_credentials.as_ref()); append_tls_env_vars(&mut environment, host_proxy_ca_paths.as_ref()); info!(sandbox = %sandbox_name, count = environment.len(), "MXC process env vars"); let process = MxcProcess { @@ -1235,8 +1299,9 @@ mod lifecycle_tests { use futures::StreamExt; use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; use openshell_core::proto::{ - FilesystemPolicy, MiddlewareEndpointSelector, NetworkMiddlewareConfig, SandboxPolicy, - UiClipboardAccess, UiPolicy, + FilesystemPolicy, MiddlewareEndpointSelector, NetworkBinary, NetworkEndpoint, + NetworkMiddlewareConfig, NetworkPolicyRule, SandboxPolicy, StaticCredentialBinding, + StaticCredentialEndpointBinding, UiClipboardAccess, UiPolicy, }; use std::time::Duration; @@ -1292,6 +1357,32 @@ mod lifecycle_tests { } } + fn github_provider_credentials() -> ProviderCredentialState { + ProviderCredentialState::from_bound_environment( + 42, + HashMap::from([( + "GITHUB_TOKEN".to_string(), + "raw-test-token-must-not-enter-mxc".to_string(), + )]), + HashMap::new(), + HashMap::new(), + HashMap::from([( + "GITHUB_TOKEN".to_string(), + StaticCredentialBinding { + endpoints: vec![StaticCredentialEndpointBinding { + host: "api.github.com".to_string(), + port: 443, + path: "/**".to_string(), + }], + credential_identity: "provider-github:GITHUB_TOKEN".to_string(), + workload_credential_handle: String::new(), + }, + )]), + Vec::new(), + ) + .expect("valid GitHub provider credential state") + } + fn with_policy(mut sandbox: DriverSandbox, policy: SandboxPolicy) -> DriverSandbox { sandbox.spec.as_mut().unwrap().policy = Some(policy); sandbox @@ -1342,6 +1433,160 @@ mod lifecycle_tests { assert_ne!(addr.port(), 0); } + #[test] + fn governed_egress_rejects_non_loopback_and_isolation_session() { + let mut config = MxcComputeConfig { + egress_proxy: true, + egress_proxy_addr: "10.0.0.1:18080".into(), + ..Default::default() + }; + assert!( + configured_egress_addr(&config) + .unwrap_err() + .message() + .contains("127.0.0.1") + ); + + config.egress_proxy_addr = "127.0.0.1:18080".into(); + config.backend = MxcBackend::IsolationSession; + assert!( + configured_egress_addr(&config) + .unwrap_err() + .message() + .contains("MXC M1") + ); + } + + #[test] + fn provider_child_environment_overrides_agent_values_with_placeholders() { + let credentials = github_provider_credentials(); + let placeholder = credentials + .snapshot() + .child_env + .get("GITHUB_TOKEN") + .cloned() + .expect("GitHub placeholder"); + let mut env = vec![ + "github_token=agent-value".to_string(), + "UNCHANGED=value".to_string(), + ]; + + append_provider_child_env(&mut env, Some(&credentials)); + + assert!(env.contains(&"UNCHANGED=value".to_string())); + assert!(!env.iter().any(|entry| entry == "github_token=agent-value")); + assert!(env.contains(&format!("GITHUB_TOKEN={placeholder}"))); + assert!(!env.iter().any(|entry| entry.contains("raw-test-token"))); + } + + #[tokio::test] + async fn provider_state_requiring_resolution_fails_closed_without_governed_egress() { + let backend = MxcComputeBackend::new_mocked(MxcComputeConfig::default()); + let sink = backend.provider_credentials_sink(); + sink.lock() + .expect("provider credential staging lock poisoned") + .insert( + "sb-provider-no-proxy".to_string(), + github_provider_credentials(), + ); + + let error = backend + .create_sandbox(&with_policy( + driver_sandbox("sb-provider-no-proxy"), + fs_policy(&[]), + )) + .await + .expect_err("provider credentials must require governed egress"); + + assert_eq!(error.code(), tonic::Code::FailedPrecondition); + assert!(error.message().contains("enable egress_proxy")); + assert!( + sink.lock() + .expect("provider credential staging lock poisoned") + .is_empty(), + "the driver must consume staged credential material on every create attempt" + ); + } + + #[tokio::test] + async fn governed_egress_puts_only_provider_placeholder_in_mxc_process_env() { + let mut config = MxcComputeConfig { + egress_proxy: true, + egress_proxy_addr: "127.0.0.1:18080".to_string(), + ..Default::default() + }; + config.backend = MxcBackend::ProcessContainer; + let backend = MxcComputeBackend::new_mocked(config); + let credentials = github_provider_credentials(); + let placeholder = credentials + .snapshot() + .child_env + .get("GITHUB_TOKEN") + .cloned() + .expect("GitHub placeholder"); + backend + .provider_credentials_sink() + .lock() + .expect("provider credential staging lock poisoned") + .insert("sb-provider-env".to_string(), credentials); + + let mut policy = fs_policy(&[]); + policy.network_policies.insert( + "github".to_string(), + NetworkPolicyRule { + name: "github".to_string(), + endpoints: vec![NetworkEndpoint { + host: "api.github.com".to_string(), + port: 443, + protocol: "rest".to_string(), + provider_credentialed: true, + ..Default::default() + }], + binaries: vec![NetworkBinary { + path: "cmd".to_string(), + ..Default::default() + }], + }, + ); + let mut sandbox = with_policy(driver_sandbox("sb-provider-env"), policy); + sandbox + .spec + .as_mut() + .expect("sandbox spec") + .environment + .extend([ + ( + "GITHUB_TOKEN".to_string(), + "raw-agent-env-token".to_string(), + ), + ("UNCHANGED".to_string(), "value".to_string()), + ]); + + backend + .create_sandbox(&sandbox) + .await + .expect("create accepted"); + wait_for(&backend, "sb-provider-env", |_| { + crate::mxc::mock_recorded_config("sb-provider-env").is_some() + }) + .await + .expect("MXC config should be recorded"); + + let recorded = + crate::mxc::mock_recorded_config("sb-provider-env").expect("mock recorded config"); + let env = recorded["process"]["env"] + .as_array() + .expect("MXC process env") + .iter() + .filter_map(serde_json::Value::as_str) + .collect::>(); + assert!(env.contains(&"UNCHANGED=value")); + assert!(env.contains(&format!("GITHUB_TOKEN={placeholder}").as_str())); + let encoded = recorded.to_string(); + assert!(!encoded.contains("raw-agent-env-token")); + assert!(!encoded.contains("raw-test-token-must-not-enter-mxc")); + } + #[test] fn sandbox_environment_inherits_host_with_spec_precedence() { let mut sandbox = driver_sandbox("sb-env"); diff --git a/crates/openshell-gateway/src/lib.rs b/crates/openshell-gateway/src/lib.rs index b67e93c10..eec328c53 100644 --- a/crates/openshell-gateway/src/lib.rs +++ b/crates/openshell-gateway/src/lib.rs @@ -159,10 +159,14 @@ impl openshell_server::ComputeDriverFactory for MxcFactory { ) -> openshell_core::Result { let config: openshell_driver_mxc::MxcComputeConfig = context.driver_config()?; let backend = openshell_driver_mxc::MxcComputeBackend::new(config); + let provider_credentials_sink = backend.provider_credentials_sink(); let driver = openshell_driver_mxc::ComputeDriverService::new(backend); - Ok(openshell_server::ComputeDriverInstance::InProcess( - std::sync::Arc::new(driver), - )) + Ok( + openshell_server::ComputeDriverInstance::InProcessWithProviderCredentials { + driver: std::sync::Arc::new(driver), + provider_credentials_sink, + }, + ) } } diff --git a/crates/openshell-server/src/compute/mod.rs b/crates/openshell-server/src/compute/mod.rs index 2ef2b0e5c..2abb401b3 100644 --- a/crates/openshell-server/src/compute/mod.rs +++ b/crates/openshell-server/src/compute/mod.rs @@ -35,9 +35,11 @@ use openshell_core::proto::compute::v1::{ gateway_listener_requirement::Selector, watch_sandboxes_event, }; use openshell_core::proto::{ - PlatformEvent, Sandbox, SandboxCondition, SandboxPhase, SandboxSpec, SandboxStatus, - SandboxTemplate, SandboxWorkloadTemplate, ServiceEndpoint, SshSession, + PlatformEvent, Sandbox, SandboxCondition, SandboxPhase, SandboxPolicy as ProtoSandboxPolicy, + SandboxSpec, SandboxStatus, SandboxTemplate, SandboxWorkloadTemplate, ServiceEndpoint, + SshSession, }; +use openshell_core::provider_credentials::ProviderCredentialState; use openshell_core::telemetry::TelemetryComputeDriver; use openshell_core::{ObjectLabels, ObjectWorkspace}; use prost::Message; @@ -63,6 +65,60 @@ pub type DriverWatchStream = Pin> + Send>>; pub type SharedComputeDriver = Arc + Send + Sync>; +pub type SandboxProviderCredentialsSink = Arc>>; + +/// Driver-specific values that must be delivered atomically with sandbox +/// creation without expanding the public compute-driver protobuf contract. +#[derive(Clone, Default)] +pub struct SandboxCreateRuntimeInputs { + pub effective_policy: Option, + pub provider_credentials: Option, +} + +impl SandboxCreateRuntimeInputs { + #[must_use] + pub(crate) fn new( + effective_policy: ProtoSandboxPolicy, + provider_credentials: Option, + ) -> Self { + Self { + effective_policy: Some(effective_policy), + provider_credentials, + } + } +} + +struct StagedProviderCredentials { + sink: SandboxProviderCredentialsSink, + sandbox_id: String, +} + +impl StagedProviderCredentials { + fn stage( + sink: SandboxProviderCredentialsSink, + sandbox_id: &str, + credentials: ProviderCredentialState, + ) -> Self { + let mut pending = sink + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + pending.insert(sandbox_id.to_string(), credentials); + drop(pending); + Self { + sink, + sandbox_id: sandbox_id.to_string(), + } + } +} + +impl Drop for StagedProviderCredentials { + fn drop(&mut self) { + self.sink + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .remove(&self.sandbox_id); + } +} use traced_driver::TracedDriver; @@ -606,6 +662,7 @@ pub struct ComputeRuntime { telemetry_compute_driver: TelemetryComputeDriver, driver_process: Option>, default_image: String, + provider_credentials_sink: Option, store: Arc, sandbox_index: SandboxIndex, sandbox_watch_bus: SandboxWatchBus, @@ -642,6 +699,7 @@ impl ComputeRuntime { driver_name: String, driver: SharedComputeDriver, driver_process: Option>, + provider_credentials_sink: Option, store: Arc, sandbox_index: SandboxIndex, sandbox_watch_bus: SandboxWatchBus, @@ -739,6 +797,7 @@ impl ComputeRuntime { telemetry_compute_driver: TelemetryComputeDriver::custom(), driver_process, default_image, + provider_credentials_sink, store, sandbox_index, sandbox_watch_bus, @@ -790,6 +849,7 @@ impl ComputeRuntime { endpoint.name, driver, endpoint.driver_process, + None, store, sandbox_index, sandbox_watch_bus, @@ -804,6 +864,12 @@ impl ComputeRuntime { &self.default_image } + /// Whether this in-process driver accepts create-time provider state. + #[must_use] + pub(crate) fn accepts_create_time_provider_credentials(&self) -> bool { + self.provider_credentials_sink.is_some() + } + #[must_use] pub fn driver_info_snapshots(&self) -> &[ComputeDriverInfoSnapshot] { std::slice::from_ref(&self.driver_info) @@ -909,9 +975,26 @@ impl ComputeRuntime { } pub async fn validate_sandbox_create(&self, sandbox: &Sandbox) -> Result<(), Status> { - self.validate_policy_capabilities(sandbox)?; + self.validate_sandbox_create_with_runtime_inputs( + sandbox, + &SandboxCreateRuntimeInputs::default(), + ) + .await + } + + pub(crate) async fn validate_sandbox_create_with_runtime_inputs( + &self, + sandbox: &Sandbox, + runtime_inputs: &SandboxCreateRuntimeInputs, + ) -> Result<(), Status> { + self.validate_policy_capabilities(sandbox, runtime_inputs.effective_policy.as_ref())?; let mut driver_sandbox = driver_sandbox_from_public(sandbox, &self.driver_info.name) .map_err(|status| *status)?; + if let Some(effective_policy) = runtime_inputs.effective_policy.as_ref() + && let Some(spec) = driver_sandbox.spec.as_mut() + { + spec.policy = Some(effective_policy.clone()); + } // Peek, never consume: create runs the same path immediately after and // must still find the token. if let Some(token) = take_staging_token(&mut driver_sandbox) { @@ -934,11 +1017,13 @@ impl ComputeRuntime { .map(|_| ()) } - fn validate_policy_capabilities(&self, sandbox: &Sandbox) -> Result<(), Status> { - let has_explicit_ui = sandbox - .spec - .as_ref() - .and_then(|spec| spec.policy.as_ref()) + fn validate_policy_capabilities( + &self, + sandbox: &Sandbox, + effective_policy: Option<&ProtoSandboxPolicy>, + ) -> Result<(), Status> { + let has_explicit_ui = effective_policy + .or_else(|| sandbox.spec.as_ref().and_then(|spec| spec.policy.as_ref())) .and_then(|policy| policy.ui.as_ref()) .is_some(); if has_explicit_ui && !self.driver_info.supports_ui_policy { @@ -955,10 +1040,32 @@ impl ComputeRuntime { sandbox: Sandbox, sandbox_token: Option, await_main_process_attachment: bool, + ) -> Result { + self.create_sandbox_with_runtime_inputs( + sandbox, + sandbox_token, + await_main_process_attachment, + SandboxCreateRuntimeInputs::default(), + ) + .await + } + + pub(crate) async fn create_sandbox_with_runtime_inputs( + &self, + sandbox: Sandbox, + sandbox_token: Option, + await_main_process_attachment: bool, + runtime_inputs: SandboxCreateRuntimeInputs, ) -> Result { // Defense in depth for internal callers that bypass the public create // handler's ValidateSandboxCreate step. This check has no side effects. - self.validate_policy_capabilities(&sandbox)?; + self.validate_policy_capabilities(&sandbox, runtime_inputs.effective_policy.as_ref())?; + if runtime_inputs.provider_credentials.is_some() && self.provider_credentials_sink.is_none() + { + return Err(Status::internal( + "provider credentials supplied to a compute driver without a create-time sink", + )); + } let sandbox_id = sandbox.object_id().to_string(); let mut sandbox = sandbox; @@ -973,6 +1080,11 @@ impl ComputeRuntime { let mut driver_sandbox = driver_sandbox_from_public(&sandbox, &self.driver_info.name) .map_err(|status| *status)?; + if let Some(effective_policy) = runtime_inputs.effective_policy + && let Some(spec) = driver_sandbox.spec.as_mut() + { + spec.policy = Some(effective_policy); + } if let Some(staged) = staged.as_ref() { set_rootfs_tar_path(&mut driver_sandbox, staged.path()); } @@ -1022,6 +1134,18 @@ impl ComputeRuntime { if let Some(spec) = driver_sandbox.spec.as_mut() { spec.await_main_process_attachment = await_main_process_attachment; } + let _staged_provider_credentials = match ( + self.provider_credentials_sink.clone(), + runtime_inputs.provider_credentials, + ) { + (Some(sink), Some(credentials)) => Some(StagedProviderCredentials::stage( + sink, + &sandbox_id, + credentials, + )), + (None, Some(_)) => unreachable!("provider credential sink checked before persistence"), + (_, None) => None, + }; match self .driver .call( @@ -4949,6 +5073,7 @@ pub async fn new_test_runtime_with_driver( telemetry_compute_driver: TelemetryComputeDriver::custom(), driver_process: None, default_image: "openshell/sandbox:test".to_string(), + provider_credentials_sink: None, store, sandbox_index: SandboxIndex::new(), sandbox_watch_bus: SandboxWatchBus::new(), @@ -5231,6 +5356,9 @@ mod tests { workspace_rpcs_unimplemented: bool, validate_create_calls: AtomicUsize, create_calls: AtomicUsize, + provider_credentials_sink: Option, + provider_env_at_create: TestMutex>>, + created_sandboxes: TestMutex>, } #[tonic::async_trait] @@ -5330,9 +5458,28 @@ mod tests { async fn create_sandbox( &self, - _request: Request, + request: Request, ) -> Result, Status> { self.create_calls.fetch_add(1, Ordering::Relaxed); + if let Some(sandbox) = request.into_inner().sandbox { + if let Some(sink) = &self.provider_credentials_sink { + let credentials = sink + .lock() + .expect("provider credential staging lock poisoned") + .get(&sandbox.id) + .cloned() + .expect("provider credentials must be staged before driver create"); + *self + .provider_env_at_create + .lock() + .expect("provider env observation lock poisoned") = + Some(credentials.child_env_with_gcp_resolved()); + } + self.created_sandboxes + .lock() + .expect("created sandbox observation lock poisoned") + .push(sandbox); + } Ok(tonic::Response::new(CreateSandboxResponse {})) } @@ -5829,6 +5976,7 @@ mod tests { telemetry_compute_driver: TelemetryComputeDriver::custom(), driver_process: None, default_image: "openshell/sandbox:test".to_string(), + provider_credentials_sink: None, store, sandbox_index: SandboxIndex::new(), sandbox_watch_bus: SandboxWatchBus::new(), @@ -5891,6 +6039,32 @@ mod tests { ); } + #[tokio::test] + async fn effective_ui_policy_rejects_before_unsupported_driver_validation() { + let driver = Arc::new(TestDriver::default()); + let runtime = test_runtime(driver.clone()).await; + let sandbox = sandbox_record( + "sb-effective-ui", + "effective-ui-policy", + SandboxPhase::Provisioning, + ); + let runtime_inputs = SandboxCreateRuntimeInputs::new( + PublicSandboxPolicy { + ui: Some(UiPolicy::default()), + ..Default::default() + }, + None, + ); + + let error = runtime + .validate_sandbox_create_with_runtime_inputs(&sandbox, &runtime_inputs) + .await + .expect_err("an unsupported driver must reject the effective UI policy"); + + assert_eq!(error.code(), Code::InvalidArgument); + assert_eq!(driver.validate_create_calls.load(Ordering::Relaxed), 0); + } + #[tokio::test] async fn explicit_ui_policy_reaches_driver_when_capability_is_complete() { let driver = Arc::new(TestDriver::default()); @@ -5919,6 +6093,97 @@ mod tests { assert_eq!(driver.validate_create_calls.load(Ordering::Relaxed), 1); } + #[tokio::test] + async fn create_runtime_inputs_reach_driver_without_persisting_effective_policy_or_secrets() { + let sink: SandboxProviderCredentialsSink = Arc::new(TestMutex::new(HashMap::new())); + let driver = Arc::new(TestDriver { + provider_credentials_sink: Some(sink.clone()), + ..Default::default() + }); + let mut runtime = test_runtime(driver.clone()).await; + runtime.provider_credentials_sink = Some(sink.clone()); + + let mut sandbox = sandbox_record( + "sb-provider-inputs", + "provider-inputs", + SandboxPhase::Provisioning, + ); + sandbox.spec = Some(SandboxSpec { + policy: Some(PublicSandboxPolicy { + version: 1, + ..Default::default() + }), + ..Default::default() + }); + let provider_credentials = ProviderCredentialState::from_environment( + 17, + HashMap::from([("GITHUB_TOKEN".to_string(), "raw-test-token".to_string())]), + HashMap::new(), + HashMap::new(), + ); + let runtime_inputs = SandboxCreateRuntimeInputs::new( + PublicSandboxPolicy { + version: 2, + ..Default::default() + }, + Some(provider_credentials), + ); + + runtime + .create_sandbox_with_runtime_inputs(sandbox, None, false, runtime_inputs) + .await + .expect("create should succeed"); + + let observed_env = driver + .provider_env_at_create + .lock() + .expect("provider env observation lock poisoned") + .clone() + .expect("driver should observe staged provider credentials"); + let observed_token = &observed_env["GITHUB_TOKEN"]; + assert_ne!(observed_token, "raw-test-token"); + assert!(observed_token.starts_with(openshell_core::secrets::PLACEHOLDER_PREFIX_PUBLIC)); + assert!( + sink.lock() + .expect("provider credential staging lock poisoned") + .is_empty(), + "create-time credentials must be removed after the driver call" + ); + + let observed_policy_version = { + let created = driver + .created_sandboxes + .lock() + .expect("created sandbox observation lock poisoned"); + created[0] + .spec + .as_ref() + .and_then(|spec| spec.policy.as_ref()) + .map(|policy| policy.version) + }; + assert_eq!( + observed_policy_version, + Some(2), + "the driver must receive the effective policy" + ); + + let persisted = runtime + .store + .get_message::("sb-provider-inputs") + .await + .expect("persisted sandbox lookup should succeed") + .expect("sandbox should be persisted"); + assert_eq!( + persisted + .spec + .as_ref() + .and_then(|spec| spec.policy.as_ref()) + .map(|policy| policy.version), + Some(1), + "the public sandbox must retain its base policy" + ); + } + async fn test_runtime_with_gateway_managed_lifecycle( driver: SharedComputeDriver, driver_name: &str, @@ -10985,6 +11250,7 @@ mod tests { "test-driver".to_string(), Arc::new(TestDriver::default()), None, + None, store, SandboxIndex::new(), SandboxWatchBus::new(), diff --git a/crates/openshell-server/src/grpc/policy.rs b/crates/openshell-server/src/grpc/policy.rs index 61dcf8fb1..6f22ec854 100644 --- a/crates/openshell-server/src/grpc/policy.rs +++ b/crates/openshell-server/src/grpc/policy.rs @@ -1737,20 +1737,40 @@ async fn current_effective_policy_for_sandbox( .as_ref() .map(|spec| spec.providers.clone()) .unwrap_or_default(); + let provider_records = super::provider::load_provider_environment_records( + state.store.as_ref(), + workspace, + &provider_names, + ) + .await?; + current_effective_policy_for_sandbox_with_records( + state, + catalog, + sandbox, + sandbox_id, + &provider_records, + ) + .await +} + +async fn current_effective_policy_for_sandbox_with_records( + state: &ServerState, + catalog: &EffectiveProviderProfileCatalog, + sandbox: &Sandbox, + sandbox_id: &str, + provider_records: &[super::provider::ProviderEnvironmentRecord], +) -> Result { let global_settings = load_global_settings(state.store.as_ref()).await?; if let Some(mut global_policy) = decode_policy_from_global_settings(&global_settings)? { // A global policy replaces dynamic policy, but startup-only UI remains // anchored to the sandbox spec so reads cannot misrepresent enforcement. preserve_sandbox_startup_ui(&mut global_policy, sandbox); - return apply_effective_policy_context( - state, + return apply_effective_policy_context_from_records( catalog, - workspace, - &provider_names, + provider_records, global_policy, PolicySource::Global, - ) - .await; + ); } let policy = if let Some(record) = state @@ -1769,15 +1789,12 @@ async fn current_effective_policy_for_sandbox( } }; - apply_effective_policy_context( - state, + apply_effective_policy_context_from_records( catalog, - workspace, - &provider_names, + provider_records, policy, PolicySource::Sandbox, ) - .await } async fn effective_policy_for_source( @@ -1812,17 +1829,26 @@ async fn apply_effective_policy_context( catalog: &EffectiveProviderProfileCatalog, workspace: &str, provider_names: &[String], - mut policy: ProtoSandboxPolicy, + policy: ProtoSandboxPolicy, policy_source: PolicySource, ) -> Result { - clear_provider_credentialed_markers(&mut policy); - let mut provider_context = provider_policy_context_with_catalog( + let provider_records = super::provider::load_provider_environment_records( state.store.as_ref(), - catalog, workspace, provider_names, ) .await?; + apply_effective_policy_context_from_records(catalog, &provider_records, policy, policy_source) +} + +fn apply_effective_policy_context_from_records( + catalog: &EffectiveProviderProfileCatalog, + provider_records: &[super::provider::ProviderEnvironmentRecord], + mut policy: ProtoSandboxPolicy, + policy_source: PolicySource, +) -> Result { + clear_provider_credentialed_markers(&mut policy); + let mut provider_context = provider_policy_context_from_records(catalog, provider_records); if !matches!(policy_source, PolicySource::Global) && !provider_context.layers.is_empty() { policy = compose_effective_policy(&policy, &provider_context.layers); } @@ -2992,16 +3018,23 @@ async fn provider_policy_context_with_catalog( workspace: &str, provider_names: &[String], ) -> Result { + let records = + super::provider::load_provider_environment_records(store, workspace, provider_names) + .await?; + Ok(provider_policy_context_from_records(catalog, &records)) +} + +fn provider_policy_context_from_records( + catalog: &EffectiveProviderProfileCatalog, + records: &[super::provider::ProviderEnvironmentRecord], +) -> ProviderPolicyContext { let mut layers = Vec::new(); let mut credentialed_scopes = Vec::new(); let mut endpointless_provider_names = HashSet::new(); - for name in provider_names { - let provider = store - .get_message_by_name::(workspace, name) - .await - .map_err(|e| Status::internal(format!("failed to fetch provider '{name}': {e}")))? - .ok_or_else(|| Status::failed_precondition(format!("provider '{name}' not found")))?; + for record in records { + let name = &record.name; + let provider = &record.provider; let provider_type = provider.r#type.trim(); let Some(profile) = super::provider::get_provider_type_profile_for_scope( @@ -3017,7 +3050,7 @@ async fn provider_policy_context_with_catalog( continue; }; - if !super::provider::provider_profile_endpoints_are_active(&profile, &provider) { + if !super::provider::provider_profile_endpoints_are_active(&profile, provider) { endpointless_provider_names.insert(name.clone()); continue; } @@ -3045,11 +3078,11 @@ async fn provider_policy_context_with_catalog( }); } - Ok(ProviderPolicyContext { + ProviderPolicyContext { layers, credentialed_scopes, endpointless_provider_names, - }) + } } fn endpoint_ports(endpoint: &NetworkEndpoint) -> Vec { @@ -3221,6 +3254,125 @@ pub(super) async fn handle_get_gateway_config( })) } +/// Resolve the effective policy and provider credential snapshot required by +/// an in-process compute driver at sandbox creation time. +/// +/// The policy, revision, endpoint bindings, and environment are all derived +/// from one immutable provider-record snapshot. Raw static credentials remain +/// in the returned resolver state; only revision-scoped placeholders are +/// exposed through its child environment. +pub(super) async fn resolve_sandbox_create_runtime_inputs( + state: &ServerState, + sandbox: &Sandbox, +) -> Result { + if !state.compute.accepts_create_time_provider_credentials() { + return Ok(crate::compute::SandboxCreateRuntimeInputs::default()); + } + + let sandbox_id = sandbox.object_id(); + let workspace = sandbox.object_workspace(); + let provider_names = sandbox + .spec + .as_ref() + .map(|spec| spec.providers.as_slice()) + .unwrap_or_default(); + let provider_profile_catalog = state + .provider_profile_sources + .snapshot_catalog(state.store.as_ref(), workspace) + .await?; + let provider_records = super::provider::load_provider_environment_records( + state.store.as_ref(), + workspace, + provider_names, + ) + .await?; + let effective_policy = current_effective_policy_for_sandbox_with_records( + state, + &provider_profile_catalog, + sandbox, + sandbox_id, + &provider_records, + ) + .await?; + let policy_credential_bindings = + policy_static_credential_endpoint_bindings(Some(&effective_policy))?; + validate_policy_credential_binding_context( + &provider_profile_catalog, + &provider_records, + &effective_policy, + &policy_credential_bindings, + )?; + let provider_env_revision = compute_provider_env_revision_from_records_and_policy_bindings( + &provider_profile_catalog, + &provider_records, + &policy_credential_bindings, + )?; + let mut provider_environment = + super::provider::resolve_provider_environment_from_records_with_policy_bindings_and_credentials( + state.store.as_ref(), + &provider_profile_catalog, + &provider_records, + &policy_credential_bindings, + &state.credentials, + Some(sandbox_id), + ) + .await?; + + // MXC uses the binding-capable host proxy. Withhold any static value that + // has no endpoint binding instead of exposing it directly to the process. + let unbound_static_keys = provider_environment + .static_credential_keys + .iter() + .filter(|key| { + !provider_environment + .static_credential_bindings + .contains_key(*key) + }) + .cloned() + .collect::>(); + for key in unbound_static_keys { + warn!( + sandbox_id, + key = %key, + "withholding unbound static provider credential from MXC sandbox" + ); + provider_environment.environment.remove(&key); + provider_environment.credential_expires_at_ms.remove(&key); + provider_environment.static_credential_keys.remove(&key); + } + + let provider_credentials = if provider_records.is_empty() { + None + } else { + let non_secret_environment_keys = provider_environment + .environment + .keys() + .filter(|key| !provider_environment.static_credential_keys.contains(*key)) + .cloned() + .collect(); + Some( + openshell_core::provider_credentials::ProviderCredentialState::from_bound_environment( + provider_env_revision, + provider_environment.environment, + provider_environment.credential_expires_at_ms, + provider_environment.dynamic_credentials, + provider_environment.static_credential_bindings, + non_secret_environment_keys, + ) + .map_err(|error| { + Status::failed_precondition(format!( + "invalid provider credential binding for sandbox '{sandbox_id}': {error}" + )) + })?, + ) + }; + + Ok(crate::compute::SandboxCreateRuntimeInputs::new( + effective_policy, + provider_credentials, + )) +} + pub(super) async fn handle_get_sandbox_provider_environment( state: &Arc, request: Request, @@ -3254,12 +3406,12 @@ pub(super) async fn handle_get_sandbox_provider_environment( &provider_names, ) .await?; - let effective_policy = current_effective_policy_for_sandbox( + let effective_policy = current_effective_policy_for_sandbox_with_records( state.as_ref(), &provider_profile_catalog, - &workspace, &sandbox, &sandbox_id, + &provider_records, ) .await?; let policy_credential_bindings = diff --git a/crates/openshell-server/src/grpc/sandbox.rs b/crates/openshell-server/src/grpc/sandbox.rs index d428e6456..50654e2a4 100644 --- a/crates/openshell-server/src/grpc/sandbox.rs +++ b/crates/openshell-server/src/grpc/sandbox.rs @@ -490,9 +490,12 @@ async fn handle_create_sandbox_inner( ) .await?; + let runtime_inputs = + super::policy::resolve_sandbox_create_runtime_inputs(state.as_ref(), &sandbox).await?; + state .compute - .validate_sandbox_create(&sandbox) + .validate_sandbox_create_with_runtime_inputs(&sandbox, &runtime_inputs) .await .map_err(|status| { warn!(error = %status, "Rejecting sandbox create request"); @@ -518,7 +521,12 @@ async fn handle_create_sandbox_inner( let sandbox = state .compute - .create_sandbox(sandbox, sandbox_token, await_main_process_attachment) + .create_sandbox_with_runtime_inputs( + sandbox, + sandbox_token, + await_main_process_attachment, + runtime_inputs, + ) .await?; info!( diff --git a/crates/openshell-server/src/lib.rs b/crates/openshell-server/src/lib.rs index f76d8d02e..2185f83e4 100644 --- a/crates/openshell-server/src/lib.rs +++ b/crates/openshell-server/src/lib.rs @@ -1047,13 +1047,19 @@ async fn terminate_signal() { } pub use compute::{ - AcquiredRemoteDriverEndpoint, DriverWatchStream, ManagedDriverProcess, SharedComputeDriver, + AcquiredRemoteDriverEndpoint, DriverWatchStream, ManagedDriverProcess, + SandboxProviderCredentialsSink, SharedComputeDriver, }; /// Driver instance returned by a compiled compute-driver factory. pub enum ComputeDriverInstance { /// A driver hosted in the gateway process. InProcess(SharedComputeDriver), + /// An in-process driver with a create-time provider credential side channel. + InProcessWithProviderCredentials { + driver: SharedComputeDriver, + provider_credentials_sink: SandboxProviderCredentialsSink, + }, /// A driver process launched and owned by the gateway. ManagedRemote(AcquiredRemoteDriverEndpoint), } @@ -1498,6 +1504,25 @@ async fn build_compute_runtime( registration.name, driver, None, + None, + store, + sandbox_index, + sandbox_watch_bus, + tracing_log_bus, + supervisor_sessions, + ) + .await + .map_err(|error| { + Error::execution(format!("failed to create compute runtime: {error}")) + })?, + ComputeDriverInstance::InProcessWithProviderCredentials { + driver, + provider_credentials_sink, + } => ComputeRuntime::from_driver( + registration.name, + driver, + None, + Some(provider_credentials_sink), store, sandbox_index, sandbox_watch_bus,