mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-26 06:52:57 +08:00
* feat(credentials): add provider credential storage drivers Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(credentials): harden credential update handling Signed-off-by: Taylor Mutch <taylormutch@gmail.com> * fix(credentials): harden credential driver security, correctness, and performance Address review findings from the credential storage drivers PR: - Route additional_credentials through the driver on refresh to prevent silent data loss for multi-credential providers (e.g. AWS STS) - Clean up stored credential handles on CAS failure during refresh to prevent orphaned secrets in external backends - Enforce namespace validation in the Kubernetes Secrets driver to prevent cross-namespace credential access when allow_reference_namespace is not enabled - Cache Vault Kubernetes auth tokens with 80% TTL to avoid re-authenticating on every credential operation - Parallelize resolve_credentials in all three drivers using try_join_all for faster sandbox startup - Add existingSecret support for the KEK Secret to fix helm template/GitOps workflows where lookup returns empty and regenerates the key - Document RBAC blast radius for the Kubernetes Secrets credential driver and recommend a dedicated namespace Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): add optimistic concurrency, fix thundering herd, parallelize operations Use resourceVersion optimistic concurrency with retry loop for K8s Secret ownership checks to prevent TOCTOU races. Switch Vault token cache from RwLock to Mutex with double-check pattern to prevent thundering herd on cache miss. Parallelize credential store and delete operations across independent keys using try_join_all. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): handle partial failures, add delete retry, consolidate cleanup Replace try_join_all with join_all in credential store/delete operations to handle partial failures — successfully-stored handles are cleaned up when another key fails. Add retry loop with conflict detection to db-credstore delete_credential, matching the K8s driver pattern. Consolidate 4 manual cleanup_pre_stored_provider_credentials call sites into a single error handler using an async block. Remove inconsistent .trim() from db-credstore validate_handle_owner. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): fix retry loop guard and remove unprotected validation Remove attempt-count guard from 409/Aborted match arms in retry loops so the post-loop Status::aborted error is reachable after exhausting retries. Previously, last-attempt conflicts fell through to the catch-all error arm, producing misleading Status::unavailable errors. Remove duplicate validation calls that ran after prepare_provider_credential_update but outside the cleanup-protected async block, which would leak pre-stored handles on failure. Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> * fix(credentials): add workspace/provider UUID to credential backend paths Include workspace and provider ID in credential backend object paths to ensure cross-workspace uniqueness and prevent credential collision (GATOR-1806c9be-01). - Updated credential driver proto to include workspace and provider_id fields - Modified Vault driver to include workspace/provider_id in managed_secret_path - Modified Kubernetes Secrets driver to include workspace/provider_id in credential_owner_id and managed_secret_name - Updated all credential runtime calls to pass workspace/provider_id - Updated tests to use the new signatures This prevents two workspaces sharing the same external credential store from colliding on provider names, which was a critical security issue (CWE-639). * fix(credentials): preserve provider-level expiration for handle-backed credentials Compute effective expiration from both provider and driver values using the earliest non-zero timestamp and skip expired values before insertion (GATOR-1806c9be-02). - Modified resolve_provider_handles to check provider credential_expires_at_ms - Skip expired credentials during resolution instead of returning them - Use effective expiration (min of provider and driver) in resolution results - Fix inference.rs to preserve earliest expiration when merging This ensures handle-backed credentials respect the same expiration semantics as inline credentials. * fix(credentials): stage refresh changes under new handles before validation Stage credential replacements under new immutable handles instead of reusing existing handles to prevent overwriting committed values before validation/CAS (GATOR-1806c9be-03). - Stage credentials with empty existing_handles map to force new handle creation - Validate and CAS before the new values are committed to backend storage - Delete old handles only after successful CAS - On CAS failure, delete only the newly staged handles - This prevents CWE-362/CWE-367 race conditions where failed refreshes could still modify or delete the active credential The fix ensures that a rejected refresh cannot modify the backend object still referenced by the committed provider record. * fix(credentials): add timeouts to credential driver RPCs Apply configured timeouts to both startup capability negotiation and runtime RPCs to prevent indefinite hangs (GATOR-1806c9be-05). - Add DEFAULT_CREDENTIAL_DRIVER_RPC_TIMEOUT_SECS constant (30s) - Apply timeout to GetCapabilities during startup connection - Apply timeout to all runtime RPCs (store, delete, resolve) - Use tokio::time::timeout to bound the entire GetCapabilities operation during startup, not just the socket connection - Return contextual deadline errors on timeout This prevents a faulty or overloaded driver from hanging gateway operations indefinitely. * fix(credentials): fix test to use consistent workspace/provider identity The Kubernetes auth Vault resolve test was constructing a managed path with test-workspace/test-provider-id but sending default/prov-123 in the request, causing validation to reject the request (GATOR-18e32351-01). - Update test to use test-workspace and test-provider-id in the request to match the logical_path construction - This ensures the test exercises the intended code path and validates Kubernetes auth resolution properly The test now passes and correctly validates identity enforcement. * fix(credentials): use unique staging ID for refresh to avoid overwrites Stage refresh replacements under genuinely distinct immutable handles using a unique staging ID to prevent overwriting committed values (GATOR-1806c9be-03). - Generate a unique staging ID using UUID for each refresh operation - Use this staging ID when storing credentials instead of the real provider ID - Pass the same staging ID during cleanup on failure to delete only staged objects - This ensures deterministic paths (Vault) and object names (K8s) don't collide with the committed provider's credentials The fix prevents failed refreshes from silently replacing active credentials or breaking providers by deleting still-referenced backend objects. * fix(credentials): wrap credential driver RPCs in local timeouts Add local tokio::time::timeout wrappers around credential driver RPCs to bound non-compliant or stalled UDS peers (GATOR-1806c9be-05). - Wrap StoreCredential, DeleteCredential, and ResolveCredentials in local timeouts - Return contextual deadline_exceeded errors when timeouts occur - Keep existing gRPC timeout metadata for compliant implementations - GetCapabilities during startup was already wrapped in previous commit This ensures a faulty local driver cannot hang gateway operations indefinitely, even if it accepts the connection but never responds to the RPC. * fix(credentials): preserve ownership for staged refreshes Signed-off-by: Seth Jennings <sjenning@redhat.com> * fix(credentials): bound startup capability probe Signed-off-by: Seth Jennings <sjenning@redhat.com> * test(provider): authenticate credential handler requests Signed-off-by: Seth Jennings <sjenning@redhat.com> * fix(ci): grant actions read to credential driver e2e Signed-off-by: Seth Jennings <sjenning@redhat.com> --------- Signed-off-by: Taylor Mutch <taylormutch@gmail.com> Signed-off-by: Varsha Prasad <varshaprasad96@gmail.com> Signed-off-by: Varsha Prasad Narsing <varshaprasad96@gmail.com> Signed-off-by: Seth Jennings <sjenning@redhat.com> Co-authored-by: Taylor Mutch <taylormutch@gmail.com> Co-authored-by: Varsha Prasad Narsing <varshaprasad96@gmail.com>
954 lines
34 KiB
Rust
954 lines
34 KiB
Rust
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
// SPDX-License-Identifier: Apache-2.0
|
|
|
|
//! Integration tests for `ensure_required_providers` — verifies that explicit
|
|
//! `--provider` names are auto-created when they match a known provider type,
|
|
//! pass through when they already exist, and error for unrecognised names.
|
|
|
|
mod helpers;
|
|
|
|
use helpers::{
|
|
EnvVarGuard, build_ca, build_client_cert, build_server_cert, install_rustls_provider,
|
|
};
|
|
use openshell_cli::run;
|
|
use openshell_cli::tls::TlsOptions;
|
|
use openshell_core::proto::open_shell_server::{OpenShell, OpenShellServer};
|
|
use openshell_core::proto::{
|
|
AttachSandboxProviderRequest, AttachSandboxProviderResponse, CreateProviderRequest,
|
|
CreateSandboxRequest, CreateSshSessionRequest, CreateSshSessionResponse, DeleteProviderRequest,
|
|
DeleteProviderResponse, DeleteSandboxRequest, DeleteSandboxResponse,
|
|
DetachSandboxProviderRequest, DetachSandboxProviderResponse, ExecSandboxEvent,
|
|
ExecSandboxInput, ExecSandboxRequest, GatewayMessage, GetGatewayConfigRequest,
|
|
GetGatewayConfigResponse, GetProviderRequest, GetSandboxConfigRequest,
|
|
GetSandboxConfigResponse, GetSandboxProviderEnvironmentRequest,
|
|
GetSandboxProviderEnvironmentResponse, GetSandboxRequest, HealthRequest, HealthResponse,
|
|
ListProvidersRequest, ListProvidersResponse, ListSandboxProvidersRequest,
|
|
ListSandboxProvidersResponse, ListSandboxesRequest, ListSandboxesResponse, Provider,
|
|
ProviderResponse, RevokeSshSessionRequest, RevokeSshSessionResponse, SandboxResponse,
|
|
SandboxStreamEvent, ServiceStatus, SupervisorMessage, UpdateProviderRequest,
|
|
WatchSandboxRequest,
|
|
};
|
|
use openshell_core::{ObjectId, ObjectName};
|
|
use std::collections::HashMap;
|
|
use std::sync::Arc;
|
|
use tempfile::TempDir;
|
|
use tokio::net::TcpListener;
|
|
use tokio::sync::{Mutex, mpsc};
|
|
use tokio_stream::wrappers::TcpListenerStream;
|
|
use tonic::transport::{Certificate as TlsCertificate, Identity, Server, ServerTlsConfig};
|
|
use tonic::{Response, Status};
|
|
|
|
// ── mock OpenShell server ─────────────────────────────────────────────
|
|
|
|
#[derive(Clone, Default)]
|
|
struct ProviderState {
|
|
providers: Arc<Mutex<HashMap<String, Provider>>>,
|
|
}
|
|
|
|
#[derive(Clone, Default)]
|
|
struct TestOpenShell {
|
|
state: ProviderState,
|
|
}
|
|
|
|
impl TestOpenShell {
|
|
/// Seed the mock with an existing provider.
|
|
async fn seed_provider(&self, name: &str, provider_type: &str) {
|
|
let mut providers = self.state.providers.lock().await;
|
|
providers.insert(
|
|
name.to_string(),
|
|
Provider {
|
|
metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta {
|
|
id: format!("id-{name}"),
|
|
name: name.to_string(),
|
|
created_at_ms: 0,
|
|
labels: HashMap::new(),
|
|
resource_version: 0,
|
|
annotations: HashMap::new(),
|
|
workspace: String::new(),
|
|
deletion_timestamp_ms: 0,
|
|
}),
|
|
r#type: provider_type.to_string(),
|
|
credentials: HashMap::new(),
|
|
config: HashMap::new(),
|
|
credential_expires_at_ms: HashMap::new(),
|
|
profile_workspace: "default".to_string(),
|
|
credential_handles: HashMap::new(),
|
|
},
|
|
);
|
|
}
|
|
}
|
|
|
|
#[tonic::async_trait]
|
|
impl OpenShell for TestOpenShell {
|
|
async fn get_current_user(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetCurrentUserRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetCurrentUserResponse>, Status> {
|
|
Err(Status::unimplemented("not used by this test server"))
|
|
}
|
|
|
|
async fn health(
|
|
&self,
|
|
_request: tonic::Request<HealthRequest>,
|
|
) -> Result<Response<HealthResponse>, Status> {
|
|
Ok(Response::new(HealthResponse {
|
|
status: ServiceStatus::Healthy.into(),
|
|
version: "test".to_string(),
|
|
}))
|
|
}
|
|
|
|
async fn get_gateway_info(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetGatewayInfoRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetGatewayInfoResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn create_sandbox(
|
|
&self,
|
|
_request: tonic::Request<CreateSandboxRequest>,
|
|
) -> Result<Response<SandboxResponse>, Status> {
|
|
Ok(Response::new(SandboxResponse::default()))
|
|
}
|
|
|
|
async fn get_sandbox(
|
|
&self,
|
|
_request: tonic::Request<GetSandboxRequest>,
|
|
) -> Result<Response<SandboxResponse>, Status> {
|
|
Ok(Response::new(SandboxResponse::default()))
|
|
}
|
|
|
|
async fn list_sandboxes(
|
|
&self,
|
|
_request: tonic::Request<ListSandboxesRequest>,
|
|
) -> Result<Response<ListSandboxesResponse>, Status> {
|
|
Ok(Response::new(ListSandboxesResponse::default()))
|
|
}
|
|
|
|
async fn list_sandbox_providers(
|
|
&self,
|
|
_request: tonic::Request<ListSandboxProvidersRequest>,
|
|
) -> Result<Response<ListSandboxProvidersResponse>, Status> {
|
|
Ok(Response::new(ListSandboxProvidersResponse::default()))
|
|
}
|
|
|
|
async fn attach_sandbox_provider(
|
|
&self,
|
|
_request: tonic::Request<AttachSandboxProviderRequest>,
|
|
) -> Result<Response<AttachSandboxProviderResponse>, Status> {
|
|
Ok(Response::new(AttachSandboxProviderResponse::default()))
|
|
}
|
|
|
|
async fn detach_sandbox_provider(
|
|
&self,
|
|
_request: tonic::Request<DetachSandboxProviderRequest>,
|
|
) -> Result<Response<DetachSandboxProviderResponse>, Status> {
|
|
Ok(Response::new(DetachSandboxProviderResponse::default()))
|
|
}
|
|
|
|
async fn delete_sandbox(
|
|
&self,
|
|
_request: tonic::Request<DeleteSandboxRequest>,
|
|
) -> Result<Response<DeleteSandboxResponse>, Status> {
|
|
Ok(Response::new(DeleteSandboxResponse { deleted: true }))
|
|
}
|
|
|
|
async fn get_sandbox_config(
|
|
&self,
|
|
_request: tonic::Request<GetSandboxConfigRequest>,
|
|
) -> Result<Response<GetSandboxConfigResponse>, Status> {
|
|
Ok(Response::new(GetSandboxConfigResponse::default()))
|
|
}
|
|
|
|
async fn get_gateway_config(
|
|
&self,
|
|
_request: tonic::Request<GetGatewayConfigRequest>,
|
|
) -> Result<Response<GetGatewayConfigResponse>, Status> {
|
|
Ok(Response::new(GetGatewayConfigResponse::default()))
|
|
}
|
|
|
|
async fn get_sandbox_provider_environment(
|
|
&self,
|
|
_request: tonic::Request<GetSandboxProviderEnvironmentRequest>,
|
|
) -> Result<Response<GetSandboxProviderEnvironmentResponse>, Status> {
|
|
Ok(Response::new(
|
|
GetSandboxProviderEnvironmentResponse::default(),
|
|
))
|
|
}
|
|
|
|
async fn create_ssh_session(
|
|
&self,
|
|
_request: tonic::Request<CreateSshSessionRequest>,
|
|
) -> Result<Response<CreateSshSessionResponse>, Status> {
|
|
Ok(Response::new(CreateSshSessionResponse::default()))
|
|
}
|
|
|
|
async fn expose_service(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ExposeServiceRequest>,
|
|
) -> Result<Response<openshell_core::proto::ServiceEndpointResponse>, Status> {
|
|
Ok(Response::new(
|
|
openshell_core::proto::ServiceEndpointResponse::default(),
|
|
))
|
|
}
|
|
|
|
async fn get_service(
|
|
&self,
|
|
_: tonic::Request<openshell_core::proto::GetServiceRequest>,
|
|
) -> Result<Response<openshell_core::proto::ServiceEndpointResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn list_services(
|
|
&self,
|
|
_: tonic::Request<openshell_core::proto::ListServicesRequest>,
|
|
) -> Result<Response<openshell_core::proto::ListServicesResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn delete_service(
|
|
&self,
|
|
_: tonic::Request<openshell_core::proto::DeleteServiceRequest>,
|
|
) -> Result<Response<openshell_core::proto::DeleteServiceResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn revoke_ssh_session(
|
|
&self,
|
|
_request: tonic::Request<RevokeSshSessionRequest>,
|
|
) -> Result<Response<RevokeSshSessionResponse>, Status> {
|
|
Ok(Response::new(RevokeSshSessionResponse::default()))
|
|
}
|
|
|
|
async fn create_provider(
|
|
&self,
|
|
request: tonic::Request<CreateProviderRequest>,
|
|
) -> Result<Response<ProviderResponse>, Status> {
|
|
let mut provider = request
|
|
.into_inner()
|
|
.provider
|
|
.ok_or_else(|| Status::invalid_argument("provider is required"))?;
|
|
let mut providers = self.state.providers.lock().await;
|
|
let provider_name = provider.object_name().to_string();
|
|
if providers.contains_key(&provider_name) {
|
|
return Err(Status::already_exists("provider already exists"));
|
|
}
|
|
if provider.object_id().is_empty()
|
|
&& let Some(metadata) = &mut provider.metadata
|
|
{
|
|
metadata.id = format!("id-{provider_name}");
|
|
}
|
|
providers.insert(provider_name, provider.clone());
|
|
Ok(Response::new(ProviderResponse {
|
|
provider: Some(provider),
|
|
}))
|
|
}
|
|
|
|
async fn get_provider(
|
|
&self,
|
|
request: tonic::Request<GetProviderRequest>,
|
|
) -> Result<Response<ProviderResponse>, Status> {
|
|
let name = request.into_inner().name;
|
|
let providers = self.state.providers.lock().await;
|
|
let provider = providers
|
|
.get(&name)
|
|
.cloned()
|
|
.ok_or_else(|| Status::not_found("provider not found"))?;
|
|
Ok(Response::new(ProviderResponse {
|
|
provider: Some(provider),
|
|
}))
|
|
}
|
|
|
|
async fn list_providers(
|
|
&self,
|
|
_request: tonic::Request<ListProvidersRequest>,
|
|
) -> Result<Response<ListProvidersResponse>, Status> {
|
|
let providers = self
|
|
.state
|
|
.providers
|
|
.lock()
|
|
.await
|
|
.values()
|
|
.cloned()
|
|
.collect::<Vec<_>>();
|
|
Ok(Response::new(ListProvidersResponse { providers }))
|
|
}
|
|
|
|
async fn list_provider_profiles(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ListProviderProfilesRequest>,
|
|
) -> Result<Response<openshell_core::proto::ListProviderProfilesResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn get_provider_profile(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetProviderProfileRequest>,
|
|
) -> Result<Response<openshell_core::proto::ProviderProfileResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn import_provider_profiles(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ImportProviderProfilesRequest>,
|
|
) -> Result<Response<openshell_core::proto::ImportProviderProfilesResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn update_provider_profiles(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::UpdateProviderProfilesRequest>,
|
|
) -> Result<Response<openshell_core::proto::UpdateProviderProfilesResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn lint_provider_profiles(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::LintProviderProfilesRequest>,
|
|
) -> Result<Response<openshell_core::proto::LintProviderProfilesResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn delete_provider_profile(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::DeleteProviderProfileRequest>,
|
|
) -> Result<Response<openshell_core::proto::DeleteProviderProfileResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn update_provider(
|
|
&self,
|
|
request: tonic::Request<UpdateProviderRequest>,
|
|
) -> Result<Response<ProviderResponse>, Status> {
|
|
let provider = request
|
|
.into_inner()
|
|
.provider
|
|
.ok_or_else(|| Status::invalid_argument("provider is required"))?;
|
|
|
|
let mut providers = self.state.providers.lock().await;
|
|
let existing = providers
|
|
.get(provider.object_name())
|
|
.cloned()
|
|
.ok_or_else(|| Status::not_found("provider not found"))?;
|
|
// Merge semantics: empty map = no change, empty value = delete key.
|
|
let merge = |mut base: HashMap<String, String>,
|
|
incoming: HashMap<String, String>|
|
|
-> HashMap<String, String> {
|
|
if incoming.is_empty() {
|
|
return base;
|
|
}
|
|
for (k, v) in incoming {
|
|
if v.is_empty() {
|
|
base.remove(&k);
|
|
} else {
|
|
base.insert(k, v);
|
|
}
|
|
}
|
|
base
|
|
};
|
|
let merge_expiry = |mut base: HashMap<String, i64>, incoming: HashMap<String, i64>| {
|
|
if incoming.is_empty() {
|
|
return base;
|
|
}
|
|
for (k, v) in incoming {
|
|
if v <= 0 {
|
|
base.remove(&k);
|
|
} else {
|
|
base.insert(k, v);
|
|
}
|
|
}
|
|
base
|
|
};
|
|
let existing_metadata = existing.metadata.clone().unwrap_or_default();
|
|
let provider_metadata = provider.metadata.clone().unwrap_or_default();
|
|
let updated = Provider {
|
|
metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta {
|
|
id: existing_metadata.id,
|
|
name: provider_metadata.name,
|
|
created_at_ms: existing_metadata.created_at_ms,
|
|
labels: existing_metadata.labels,
|
|
resource_version: 0,
|
|
annotations: HashMap::new(),
|
|
workspace: String::new(),
|
|
deletion_timestamp_ms: 0,
|
|
}),
|
|
r#type: existing.r#type,
|
|
credentials: merge(existing.credentials, provider.credentials),
|
|
config: merge(existing.config, provider.config),
|
|
credential_expires_at_ms: merge_expiry(
|
|
existing.credential_expires_at_ms,
|
|
provider.credential_expires_at_ms,
|
|
),
|
|
profile_workspace: existing.profile_workspace,
|
|
credential_handles: if provider.credential_handles.is_empty() {
|
|
existing.credential_handles
|
|
} else {
|
|
provider.credential_handles
|
|
},
|
|
};
|
|
let updated_name = updated.object_name().to_string();
|
|
providers.insert(updated_name, updated.clone());
|
|
Ok(Response::new(ProviderResponse {
|
|
provider: Some(updated),
|
|
}))
|
|
}
|
|
async fn get_provider_refresh_status(
|
|
&self,
|
|
_: tonic::Request<openshell_core::proto::GetProviderRefreshStatusRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetProviderRefreshStatusResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn configure_provider_refresh(
|
|
&self,
|
|
_: tonic::Request<openshell_core::proto::ConfigureProviderRefreshRequest>,
|
|
) -> Result<Response<openshell_core::proto::ConfigureProviderRefreshResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn rotate_provider_credential(
|
|
&self,
|
|
_: tonic::Request<openshell_core::proto::RotateProviderCredentialRequest>,
|
|
) -> Result<Response<openshell_core::proto::RotateProviderCredentialResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn delete_provider_refresh(
|
|
&self,
|
|
_: tonic::Request<openshell_core::proto::DeleteProviderRefreshRequest>,
|
|
) -> Result<Response<openshell_core::proto::DeleteProviderRefreshResponse>, Status> {
|
|
Err(Status::unimplemented("unused"))
|
|
}
|
|
|
|
async fn delete_provider(
|
|
&self,
|
|
request: tonic::Request<DeleteProviderRequest>,
|
|
) -> Result<Response<DeleteProviderResponse>, Status> {
|
|
let name = request.into_inner().name;
|
|
let deleted = self.state.providers.lock().await.remove(&name).is_some();
|
|
Ok(Response::new(DeleteProviderResponse { deleted }))
|
|
}
|
|
|
|
type WatchSandboxStream =
|
|
tokio_stream::wrappers::ReceiverStream<Result<SandboxStreamEvent, Status>>;
|
|
type ExecSandboxStream =
|
|
tokio_stream::wrappers::ReceiverStream<Result<ExecSandboxEvent, Status>>;
|
|
type ConnectSupervisorStream =
|
|
tokio_stream::wrappers::ReceiverStream<Result<GatewayMessage, Status>>;
|
|
|
|
async fn watch_sandbox(
|
|
&self,
|
|
_request: tonic::Request<WatchSandboxRequest>,
|
|
) -> Result<Response<Self::WatchSandboxStream>, Status> {
|
|
let (_tx, rx) = mpsc::channel(1);
|
|
Ok(Response::new(tokio_stream::wrappers::ReceiverStream::new(
|
|
rx,
|
|
)))
|
|
}
|
|
|
|
async fn exec_sandbox(
|
|
&self,
|
|
_request: tonic::Request<ExecSandboxRequest>,
|
|
) -> Result<Response<Self::ExecSandboxStream>, Status> {
|
|
let (_tx, rx) = mpsc::channel(1);
|
|
Ok(Response::new(tokio_stream::wrappers::ReceiverStream::new(
|
|
rx,
|
|
)))
|
|
}
|
|
|
|
type ExecSandboxInteractiveStream =
|
|
tokio_stream::wrappers::ReceiverStream<Result<ExecSandboxEvent, Status>>;
|
|
async fn exec_sandbox_interactive(
|
|
&self,
|
|
_request: tonic::Request<tonic::Streaming<ExecSandboxInput>>,
|
|
) -> Result<Response<Self::ExecSandboxInteractiveStream>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn update_config(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::UpdateConfigRequest>,
|
|
) -> Result<Response<openshell_core::proto::UpdateConfigResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn get_sandbox_policy_status(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetSandboxPolicyStatusRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetSandboxPolicyStatusResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn list_sandbox_policies(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ListSandboxPoliciesRequest>,
|
|
) -> Result<Response<openshell_core::proto::ListSandboxPoliciesResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn report_policy_status(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,
|
|
) -> Result<Response<openshell_core::proto::ReportPolicyStatusResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn get_sandbox_logs(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetSandboxLogsRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetSandboxLogsResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn push_sandbox_logs(
|
|
&self,
|
|
_request: tonic::Request<tonic::Streaming<openshell_core::proto::PushSandboxLogsRequest>>,
|
|
) -> Result<Response<openshell_core::proto::PushSandboxLogsResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn submit_policy_analysis(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::SubmitPolicyAnalysisRequest>,
|
|
) -> Result<Response<openshell_core::proto::SubmitPolicyAnalysisResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn get_draft_policy(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetDraftPolicyRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetDraftPolicyResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn approve_draft_chunk(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ApproveDraftChunkRequest>,
|
|
) -> Result<Response<openshell_core::proto::ApproveDraftChunkResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn reject_draft_chunk(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::RejectDraftChunkRequest>,
|
|
) -> Result<Response<openshell_core::proto::RejectDraftChunkResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn approve_all_draft_chunks(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ApproveAllDraftChunksRequest>,
|
|
) -> Result<Response<openshell_core::proto::ApproveAllDraftChunksResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn edit_draft_chunk(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::EditDraftChunkRequest>,
|
|
) -> Result<Response<openshell_core::proto::EditDraftChunkResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn undo_draft_chunk(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::UndoDraftChunkRequest>,
|
|
) -> Result<Response<openshell_core::proto::UndoDraftChunkResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn clear_draft_chunks(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ClearDraftChunksRequest>,
|
|
) -> Result<Response<openshell_core::proto::ClearDraftChunksResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn get_draft_history(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetDraftHistoryRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetDraftHistoryResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn issue_sandbox_token(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::IssueSandboxTokenRequest>,
|
|
) -> Result<Response<openshell_core::proto::IssueSandboxTokenResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn refresh_sandbox_token(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::RefreshSandboxTokenRequest>,
|
|
) -> Result<Response<openshell_core::proto::RefreshSandboxTokenResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn connect_supervisor(
|
|
&self,
|
|
_request: tonic::Request<tonic::Streaming<SupervisorMessage>>,
|
|
) -> Result<Response<Self::ConnectSupervisorStream>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
type RelayStreamStream =
|
|
tokio_stream::wrappers::ReceiverStream<Result<openshell_core::proto::RelayFrame, Status>>;
|
|
|
|
async fn relay_stream(
|
|
&self,
|
|
_request: tonic::Request<tonic::Streaming<openshell_core::proto::RelayFrame>>,
|
|
) -> Result<Response<Self::RelayStreamStream>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
type ForwardTcpStream = tokio_stream::wrappers::ReceiverStream<
|
|
Result<openshell_core::proto::TcpForwardFrame, Status>,
|
|
>;
|
|
|
|
async fn forward_tcp(
|
|
&self,
|
|
_request: tonic::Request<tonic::Streaming<openshell_core::proto::TcpForwardFrame>>,
|
|
) -> Result<Response<Self::ForwardTcpStream>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn create_workspace(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::CreateWorkspaceRequest>,
|
|
) -> Result<Response<openshell_core::proto::CreateWorkspaceResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn get_workspace(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::GetWorkspaceRequest>,
|
|
) -> Result<Response<openshell_core::proto::GetWorkspaceResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn list_workspaces(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ListWorkspacesRequest>,
|
|
) -> Result<Response<openshell_core::proto::ListWorkspacesResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn delete_workspace(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::DeleteWorkspaceRequest>,
|
|
) -> Result<Response<openshell_core::proto::DeleteWorkspaceResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn add_workspace_member(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::AddWorkspaceMemberRequest>,
|
|
) -> Result<Response<openshell_core::proto::AddWorkspaceMemberResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn remove_workspace_member(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::RemoveWorkspaceMemberRequest>,
|
|
) -> Result<Response<openshell_core::proto::RemoveWorkspaceMemberResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
|
|
async fn list_workspace_members(
|
|
&self,
|
|
_request: tonic::Request<openshell_core::proto::ListWorkspaceMembersRequest>,
|
|
) -> Result<Response<openshell_core::proto::ListWorkspaceMembersResponse>, Status> {
|
|
Err(Status::unimplemented("not implemented in test"))
|
|
}
|
|
}
|
|
|
|
// ── test server fixture ──────────────────────────────────────────────
|
|
|
|
struct TestServer {
|
|
endpoint: String,
|
|
tls: TlsOptions,
|
|
openshell: TestOpenShell,
|
|
_dir: TempDir,
|
|
}
|
|
|
|
async fn run_server() -> TestServer {
|
|
install_rustls_provider();
|
|
|
|
let (ca, ca_key) = build_ca();
|
|
let (server_cert, server_key) = build_server_cert(&ca, &ca_key);
|
|
let (client_cert, client_key) = build_client_cert(&ca, &ca_key);
|
|
let ca_cert = ca.pem();
|
|
|
|
let identity = Identity::from_pem(server_cert, server_key);
|
|
let client_ca = TlsCertificate::from_pem(ca_cert.clone());
|
|
let tls_config = ServerTlsConfig::new()
|
|
.identity(identity)
|
|
.client_ca_root(client_ca);
|
|
|
|
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
|
|
let addr = listener.local_addr().unwrap();
|
|
let incoming = TcpListenerStream::new(listener);
|
|
|
|
let openshell = TestOpenShell::default();
|
|
let svc_openshell = openshell.clone();
|
|
|
|
tokio::spawn(async move {
|
|
Server::builder()
|
|
.tls_config(tls_config)
|
|
.unwrap()
|
|
.add_service(OpenShellServer::new(svc_openshell))
|
|
.serve_with_incoming(incoming)
|
|
.await
|
|
.unwrap();
|
|
});
|
|
|
|
let dir = tempfile::tempdir().unwrap();
|
|
let ca_path = dir.path().join("ca.crt");
|
|
let cert_path = dir.path().join("tls.crt");
|
|
let key_path = dir.path().join("tls.key");
|
|
std::fs::write(&ca_path, ca_cert).unwrap();
|
|
std::fs::write(&cert_path, client_cert).unwrap();
|
|
std::fs::write(&key_path, client_key).unwrap();
|
|
|
|
let tls = TlsOptions::new(Some(ca_path), Some(cert_path), Some(key_path));
|
|
let endpoint = format!("https://localhost:{}", addr.port());
|
|
|
|
TestServer {
|
|
endpoint,
|
|
tls,
|
|
openshell,
|
|
_dir: dir,
|
|
}
|
|
}
|
|
|
|
// ── tests ────────────────────────────────────────────────────────────
|
|
|
|
/// When `--provider nvidia` is passed and a provider named "nvidia" already
|
|
/// exists, `ensure_required_providers` should return it directly without
|
|
/// creating anything new.
|
|
#[tokio::test]
|
|
async fn explicit_provider_name_passes_through_when_it_exists() {
|
|
let ts = run_server().await;
|
|
ts.openshell.seed_provider("nvidia", "nvidia").await;
|
|
|
|
let mut client = openshell_cli::tls::grpc_client(&ts.endpoint, &ts.tls)
|
|
.await
|
|
.expect("grpc client");
|
|
|
|
let result = run::ensure_required_providers(
|
|
&mut client,
|
|
&["nvidia".to_string()],
|
|
&[],
|
|
Some(true), // --auto-providers (should not matter here)
|
|
"default",
|
|
)
|
|
.await
|
|
.expect("should succeed");
|
|
|
|
assert_eq!(result, vec!["nvidia".to_string()]);
|
|
|
|
// Verify no extra providers were created.
|
|
let providers = ts.openshell.state.providers.lock().await;
|
|
assert_eq!(providers.len(), 1, "no new providers should be created");
|
|
}
|
|
|
|
/// When `--provider nvidia` is passed, no provider named "nvidia" exists, and
|
|
/// "nvidia" is a valid provider type, the CLI should auto-create a provider
|
|
/// named "nvidia" of type "nvidia" using discovered local credentials.
|
|
#[tokio::test]
|
|
async fn explicit_provider_name_auto_creates_when_valid_type() {
|
|
let ts = run_server().await;
|
|
let _guard = EnvVarGuard::set(&[("NVIDIA_API_KEY", "nvapi-test-key")]);
|
|
|
|
let mut client = openshell_cli::tls::grpc_client(&ts.endpoint, &ts.tls)
|
|
.await
|
|
.expect("grpc client");
|
|
|
|
let result = run::ensure_required_providers(
|
|
&mut client,
|
|
&["nvidia".to_string()],
|
|
&[],
|
|
Some(true), // --auto-providers to skip interactive prompt
|
|
"default",
|
|
)
|
|
.await
|
|
.expect("should auto-create the provider");
|
|
|
|
assert_eq!(result, vec!["nvidia".to_string()]);
|
|
|
|
// Verify the provider was created on the server with the right type.
|
|
let providers = ts.openshell.state.providers.lock().await;
|
|
let provider = providers
|
|
.get("nvidia")
|
|
.expect("nvidia provider should exist");
|
|
assert_eq!(provider.r#type, "nvidia");
|
|
assert_eq!(
|
|
provider.credentials.get("NVIDIA_API_KEY"),
|
|
Some(&"nvapi-test-key".to_string()),
|
|
);
|
|
}
|
|
|
|
/// When `--provider my-custom-thing` is passed and "my-custom-thing" is not a
|
|
/// known provider type, the CLI should return an error.
|
|
#[tokio::test]
|
|
async fn explicit_provider_name_errors_for_unrecognised_name() {
|
|
let ts = run_server().await;
|
|
|
|
let mut client = openshell_cli::tls::grpc_client(&ts.endpoint, &ts.tls)
|
|
.await
|
|
.expect("grpc client");
|
|
|
|
let err = run::ensure_required_providers(
|
|
&mut client,
|
|
&["my-custom-thing".to_string()],
|
|
&[],
|
|
Some(true),
|
|
"default",
|
|
)
|
|
.await
|
|
.expect_err("should fail for unrecognised provider name");
|
|
|
|
let msg = err.to_string();
|
|
assert!(
|
|
msg.contains("my-custom-thing"),
|
|
"error should mention the name: {msg}"
|
|
);
|
|
assert!(
|
|
msg.contains("not a recognized provider type"),
|
|
"error should explain why it failed: {msg}"
|
|
);
|
|
}
|
|
|
|
/// Inferred types (from the trailing command) that don't exist should be
|
|
/// auto-created, preserving the existing behaviour.
|
|
#[tokio::test]
|
|
async fn inferred_type_auto_creates_provider() {
|
|
let ts = run_server().await;
|
|
let _guard = EnvVarGuard::set(&[("ANTHROPIC_API_KEY", "sk-ant-test")]);
|
|
|
|
let mut client = openshell_cli::tls::grpc_client(&ts.endpoint, &ts.tls)
|
|
.await
|
|
.expect("grpc client");
|
|
|
|
let result = run::ensure_required_providers(
|
|
&mut client,
|
|
&[],
|
|
&["claude-code".to_string()],
|
|
Some(true), // --auto-providers
|
|
"default",
|
|
)
|
|
.await
|
|
.expect("should auto-create the inferred provider");
|
|
|
|
assert_eq!(result, vec!["claude-code".to_string()]);
|
|
|
|
let providers = ts.openshell.state.providers.lock().await;
|
|
let provider = providers
|
|
.get("claude-code")
|
|
.expect("claude-code provider should exist");
|
|
assert_eq!(provider.r#type, "claude-code");
|
|
}
|
|
|
|
/// When `--no-auto-providers` is set, missing explicit providers that would
|
|
/// otherwise be auto-created should be silently skipped.
|
|
#[tokio::test]
|
|
async fn no_auto_providers_skips_missing_explicit_provider() {
|
|
let ts = run_server().await;
|
|
let _guard = EnvVarGuard::set(&[("NVIDIA_API_KEY", "nvapi-skip-test")]);
|
|
|
|
let mut client = openshell_cli::tls::grpc_client(&ts.endpoint, &ts.tls)
|
|
.await
|
|
.expect("grpc client");
|
|
|
|
let result = run::ensure_required_providers(
|
|
&mut client,
|
|
&["nvidia".to_string()],
|
|
&[],
|
|
Some(false), // --no-auto-providers
|
|
"default",
|
|
)
|
|
.await
|
|
.expect("should succeed with empty list");
|
|
|
|
assert!(
|
|
result.is_empty(),
|
|
"skipped providers should not appear in the result"
|
|
);
|
|
|
|
let providers = ts.openshell.state.providers.lock().await;
|
|
assert!(
|
|
providers.is_empty(),
|
|
"no providers should be created when --no-auto-providers is set"
|
|
);
|
|
}
|
|
|
|
/// Both explicit names and inferred types should be resolved together,
|
|
/// deduplicating providers that appear in both lists.
|
|
#[tokio::test]
|
|
async fn explicit_and_inferred_providers_combined() {
|
|
let ts = run_server().await;
|
|
let _guard = EnvVarGuard::set(&[
|
|
("NVIDIA_API_KEY", "nvapi-combo"),
|
|
("ANTHROPIC_API_KEY", "sk-ant-combo"),
|
|
]);
|
|
|
|
let mut client = openshell_cli::tls::grpc_client(&ts.endpoint, &ts.tls)
|
|
.await
|
|
.expect("grpc client");
|
|
|
|
let result = run::ensure_required_providers(
|
|
&mut client,
|
|
&["nvidia".to_string()],
|
|
&["claude-code".to_string()],
|
|
Some(true),
|
|
"default",
|
|
)
|
|
.await
|
|
.expect("should create both providers");
|
|
|
|
assert_eq!(result.len(), 2);
|
|
assert!(result.contains(&"nvidia".to_string()));
|
|
assert!(result.contains(&"claude-code".to_string()));
|
|
|
|
let providers = ts.openshell.state.providers.lock().await;
|
|
assert_eq!(providers.len(), 2);
|
|
assert!(providers.contains_key("nvidia"));
|
|
assert!(providers.contains_key("claude-code"));
|
|
}
|
|
|
|
/// When an explicit provider name matches an inferred type, the provider
|
|
/// should only appear once in the result.
|
|
#[tokio::test]
|
|
async fn explicit_and_inferred_deduplicates() {
|
|
let ts = run_server().await;
|
|
let _guard = EnvVarGuard::set(&[("NVIDIA_API_KEY", "nvapi-dedup")]);
|
|
|
|
let mut client = openshell_cli::tls::grpc_client(&ts.endpoint, &ts.tls)
|
|
.await
|
|
.expect("grpc client");
|
|
|
|
// Both explicit and inferred want "nvidia".
|
|
let result = run::ensure_required_providers(
|
|
&mut client,
|
|
&["nvidia".to_string()],
|
|
&["nvidia".to_string()],
|
|
Some(true),
|
|
"default",
|
|
)
|
|
.await
|
|
.expect("should succeed");
|
|
|
|
assert_eq!(
|
|
result,
|
|
vec!["nvidia".to_string()],
|
|
"nvidia should appear exactly once"
|
|
);
|
|
|
|
let providers = ts.openshell.state.providers.lock().await;
|
|
assert_eq!(
|
|
providers.len(),
|
|
1,
|
|
"only one provider should be created on the server"
|
|
);
|
|
}
|