mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-02 07:34:45 +08:00
feat(docker): support corporate proxy CA bundles (#3549)
* feat(docker): support corporate proxy CA bundles Closes #3545 Validate and stage operator-owned proxy CA bundles for Docker supervisors, add corporate proxy E2E coverage, and document the trust contract. Signed-off-by: Philippe Martin <phmartin@redhat.com> * fix(docker): validate proxy config on startup Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(docker): use the E2E workload image for proxy tests Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(docker): generate strict corporate proxy certificates Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(docker): surface intercepted TLS fixture errors Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(docker): drain buffered TLS proxy data Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * test(docker): relay intercepted HTTP deterministically Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: Philippe Martin <phmartin@redhat.com> Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> Co-authored-by: John Myers <9696606+johntmyers@users.noreply.github.com>
This commit is contained in:
co-authored by
John Myers
parent
2fe5a0e19c
commit
9cb72baa2e
@@ -585,8 +585,13 @@ cross the authenticated vsock channel. A gateway-host proxy is addressed as
|
||||
|
||||
The Docker driver runs `openshell-supervisor` in a separate companion container.
|
||||
Its private named volume contains supervisor bootstrap and channel material.
|
||||
The workload container receives only `openshell-sandbox`, public interception
|
||||
CA material, and the other sandbox half of the authenticated channel.
|
||||
The driver bounded-reads operator proxy credentials and CA bundles into that
|
||||
supervisor-only volume and passes fixed container paths to the supervisor; it
|
||||
never exposes gateway-host paths to sandbox-controlled configuration. The
|
||||
corporate CA extends supervisor upstream trust and is folded into the public
|
||||
combined trust bundle generated for workload processes. The workload container
|
||||
receives only `openshell-sandbox`, public interception and combined CA
|
||||
material, and the other sandbox half of the authenticated channel.
|
||||
|
||||
For Kubernetes, the operator configures a Secret name and key rather than a
|
||||
gateway-host file path. Kubernetes projects that Secret only into the separate
|
||||
|
||||
@@ -50,6 +50,32 @@ through the Docker archive API. No workload launch depends on a host bind
|
||||
mount or a tool supplied by the workload image, so the same path works with
|
||||
local, remote, and VM-backed Docker daemons.
|
||||
|
||||
## Corporate Proxy Egress
|
||||
|
||||
`[openshell.drivers.docker]` accepts the operator-owned corporate proxy fields
|
||||
`https_proxy`, `no_proxy`, `proxy_auth_file`,
|
||||
`proxy_auth_allow_insecure`, `proxy_connect_by_hostname`, and
|
||||
`proxy_ca_bundle`. Sandbox environment, image contents, and per-sandbox driver
|
||||
configuration cannot select or override them.
|
||||
|
||||
The driver validates the proxy URL and cross-field relationships at gateway
|
||||
startup. It bounded-reads proxy credentials and the PEM CA bundle before use.
|
||||
Missing, unreadable, empty, oversized, malformed, or certificate-free CA files
|
||||
fail closed. A CA bundle requires `https_proxy`, although the proxy URL may be
|
||||
`http://` when the proxy intercepts destination TLS.
|
||||
|
||||
For each supervisor launch, Docker copies the credential and CA contents into
|
||||
the existing supervisor-only named volume. It passes only the fixed container
|
||||
paths `/.openshell/supervisor/upstream-proxy-auth` and
|
||||
`/.openshell/supervisor/upstream-proxy-ca-bundle.pem` to the supervisor. It
|
||||
does not bind-mount the gateway-host files, which preserves remote-daemon
|
||||
support and keeps host paths out of workload container metadata.
|
||||
|
||||
The supervisor trusts the configured CA for its TLS connection to an HTTPS
|
||||
proxy and for destination certificates re-signed by a TLS-intercepting proxy.
|
||||
It also includes the corporate root in the generated combined trust bundle
|
||||
used by workload processes.
|
||||
|
||||
## Identity and Workspace
|
||||
|
||||
Before creating the workload, the driver pins the image ID and reads its
|
||||
|
||||
@@ -107,6 +107,8 @@ const BOUNDARY_CERTIFICATE_MOUNT_PATH: &str = "/.openshell/channel/sandbox/serve
|
||||
const BOUNDARY_PRIVATE_KEY_MOUNT_PATH: &str = "/.openshell/channel/sandbox/server.key";
|
||||
const SUPERVISOR_STATE_MOUNT_PATH: &str = "/.openshell/supervisor";
|
||||
const SUPERVISOR_PROXY_AUTH_MOUNT_PATH: &str = "/.openshell/supervisor/upstream-proxy-auth";
|
||||
const SUPERVISOR_PROXY_CA_BUNDLE_MOUNT_PATH: &str =
|
||||
"/.openshell/supervisor/upstream-proxy-ca-bundle.pem";
|
||||
const PROVIDER_SPIFFE_WORKLOAD_API_SOCKET_MOUNT_DIR: &str =
|
||||
openshell_core::driver_utils::PROVIDER_SPIFFE_WORKLOAD_API_SOCKET_MOUNT_DIR;
|
||||
const DRIVER_ADMITTED_BACKEND: &str = openshell_sandbox_backend::BACKEND_NAME;
|
||||
@@ -218,6 +220,10 @@ pub struct DockerComputeConfig {
|
||||
#[serde(flatten)]
|
||||
pub upstream_proxy: UpstreamProxyConfig,
|
||||
|
||||
/// Gateway-host PEM CA bundle trusted for the corporate proxy and for
|
||||
/// server certificates re-signed by a TLS-intercepting proxy.
|
||||
pub proxy_ca_bundle: Option<PathBuf>,
|
||||
|
||||
/// Host UNIX socket projected into the supervisor for provider identity.
|
||||
pub provider_spiffe_workload_api_socket: Option<PathBuf>,
|
||||
|
||||
@@ -242,6 +248,7 @@ impl DockerComputeConfig {
|
||||
validate_image_pull_policy(self.image_pull_policy)?;
|
||||
self.upstream_proxy.validate().map_err(Error::config)?;
|
||||
validate_docker_proxy_auth_file(&self.upstream_proxy)?;
|
||||
validate_docker_proxy_ca_bundle(self)?;
|
||||
if let Some(socket) = self.provider_spiffe_workload_api_socket.as_deref() {
|
||||
openshell_core::driver_utils::validate_provider_spiffe_unix_socket(socket)
|
||||
.map_err(Error::config)?;
|
||||
@@ -276,6 +283,7 @@ impl Default for DockerComputeConfig {
|
||||
sandbox_pids_limit: openshell_core::config::default_sandbox_pids_limit(),
|
||||
enable_bind_mounts: false,
|
||||
upstream_proxy: UpstreamProxyConfig::default(),
|
||||
proxy_ca_bundle: None,
|
||||
provider_spiffe_workload_api_socket: None,
|
||||
app_armor_profile: None,
|
||||
}
|
||||
@@ -307,6 +315,7 @@ struct DockerDriverRuntimeConfig {
|
||||
sandbox_pids_limit: Option<std::num::NonZeroI64>,
|
||||
enable_bind_mounts: bool,
|
||||
upstream_proxy: UpstreamProxyConfig,
|
||||
proxy_ca_bundle: Option<PathBuf>,
|
||||
provider_spiffe_workload_api_socket: Option<PathBuf>,
|
||||
app_armor_profile: Option<AppArmorProfile>,
|
||||
}
|
||||
@@ -837,10 +846,7 @@ impl DockerComputeDriver {
|
||||
gateway_log_level: &str,
|
||||
docker_config: &DockerComputeConfig,
|
||||
) -> CoreResult<Self> {
|
||||
docker_config
|
||||
.resource_admission
|
||||
.validate()
|
||||
.map_err(Error::config)?;
|
||||
docker_config.validate_configuration(gateway_bind_address)?;
|
||||
let socket_path = docker_config
|
||||
.socket_path
|
||||
.clone()
|
||||
@@ -873,15 +879,8 @@ impl DockerComputeDriver {
|
||||
cdi_supported,
|
||||
wsl_all_gpu_fallback_enabled,
|
||||
};
|
||||
validate_sandbox_pids_limit(docker_config.sandbox_pids_limit)?;
|
||||
validate_image_pull_policy(docker_config.image_pull_policy)?;
|
||||
validate_docker_app_armor_profile(docker_config.app_armor_profile.as_ref(), &info)?;
|
||||
let gateway_port = gateway_bind_address.port();
|
||||
if gateway_port == 0 {
|
||||
return Err(Error::config(
|
||||
"docker compute driver requires a fixed non-zero gateway bind port",
|
||||
));
|
||||
}
|
||||
let mut docker_config = docker_config.clone();
|
||||
if docker_config.grpc_endpoint.trim().is_empty() {
|
||||
docker_config.grpc_endpoint = default_docker_supervisor_grpc_endpoint(
|
||||
@@ -947,6 +946,7 @@ impl DockerComputeDriver {
|
||||
allow_driver_config: docker_config.allow_driver_config,
|
||||
resource_admission: docker_config.resource_admission.clone(),
|
||||
upstream_proxy: docker_config.upstream_proxy.clone(),
|
||||
proxy_ca_bundle: docker_config.proxy_ca_bundle.clone(),
|
||||
provider_spiffe_workload_api_socket: docker_config
|
||||
.provider_spiffe_workload_api_socket
|
||||
.clone(),
|
||||
@@ -4713,11 +4713,35 @@ async fn docker_supervisor_bundle_archive(
|
||||
&contents,
|
||||
)?;
|
||||
}
|
||||
append_docker_proxy_ca_bundle(&mut archive, config.proxy_ca_bundle.as_deref())?;
|
||||
archive
|
||||
.into_inner()
|
||||
.map_err(|error| Status::internal(format!("finish Docker supervisor archive: {error}")))
|
||||
}
|
||||
|
||||
fn append_docker_proxy_ca_bundle(
|
||||
archive: &mut tar::Builder<Vec<u8>>,
|
||||
path: Option<&Path>,
|
||||
) -> Result<(), Status> {
|
||||
let Some(path) = path else {
|
||||
return Ok(());
|
||||
};
|
||||
let path = path
|
||||
.to_str()
|
||||
.ok_or_else(|| Status::failed_precondition("proxy_ca_bundle must be valid UTF-8"))?;
|
||||
let contents =
|
||||
openshell_core::driver_utils::read_upstream_proxy_ca_bundle_file(path, "proxy_ca_bundle")
|
||||
.map_err(Status::failed_precondition)?;
|
||||
append_docker_archive_file(
|
||||
archive,
|
||||
"upstream-proxy-ca-bundle.pem",
|
||||
0o644,
|
||||
SUPERVISOR_UID,
|
||||
SUPERVISOR_GID,
|
||||
contents.as_bytes(),
|
||||
)
|
||||
}
|
||||
|
||||
async fn refresh_docker_boundary_authentication(
|
||||
sandbox_id: &str,
|
||||
config: &DockerDriverRuntimeConfig,
|
||||
@@ -5120,7 +5144,10 @@ async fn spawn_docker_control_process(
|
||||
workspace_root,
|
||||
format!("--health-socket-path={SUPERVISOR_HEALTH_SOCKET_PATH}"),
|
||||
];
|
||||
command.extend(docker_upstream_proxy_cli_args(&config.upstream_proxy));
|
||||
command.extend(docker_upstream_proxy_cli_args(
|
||||
&config.upstream_proxy,
|
||||
config.proxy_ca_bundle.is_some(),
|
||||
));
|
||||
let mut supervisor_mounts = vec![
|
||||
Mount {
|
||||
target: Some(BOUNDARY_MOUNT_PATH.to_string()),
|
||||
@@ -5871,7 +5898,30 @@ fn validate_docker_proxy_auth_file(config: &UpstreamProxyConfig) -> CoreResult<(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn docker_upstream_proxy_cli_args(config: &UpstreamProxyConfig) -> Vec<String> {
|
||||
fn validate_docker_proxy_ca_bundle(config: &DockerComputeConfig) -> CoreResult<()> {
|
||||
let Some(path) = config.proxy_ca_bundle.as_ref() else {
|
||||
return Ok(());
|
||||
};
|
||||
if path.as_os_str().is_empty() {
|
||||
return Err(Error::config("proxy_ca_bundle must not be empty when set"));
|
||||
}
|
||||
if config.upstream_proxy.https_proxy.is_none() {
|
||||
return Err(Error::config(
|
||||
"proxy_ca_bundle is set but no https_proxy is configured",
|
||||
));
|
||||
}
|
||||
let path = path
|
||||
.to_str()
|
||||
.ok_or_else(|| Error::config("proxy_ca_bundle must be valid UTF-8"))?;
|
||||
openshell_core::driver_utils::read_upstream_proxy_ca_bundle_file(path, "proxy_ca_bundle")
|
||||
.map_err(Error::config)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn docker_upstream_proxy_cli_args(
|
||||
config: &UpstreamProxyConfig,
|
||||
proxy_ca_bundle_configured: bool,
|
||||
) -> Vec<String> {
|
||||
let mut args = Vec::new();
|
||||
if let Some(url) = config.https_proxy.as_ref() {
|
||||
args.extend(["--upstream-proxy".to_string(), url.clone()]);
|
||||
@@ -5891,6 +5941,12 @@ fn docker_upstream_proxy_cli_args(config: &UpstreamProxyConfig) -> Vec<String> {
|
||||
if config.proxy_connect_by_hostname == Some(true) {
|
||||
args.push("--upstream-proxy-connect-by-hostname".to_string());
|
||||
}
|
||||
if proxy_ca_bundle_configured {
|
||||
args.extend([
|
||||
"--upstream-proxy-ca-bundle".to_string(),
|
||||
SUPERVISOR_PROXY_CA_BUNDLE_MOUNT_PATH.to_string(),
|
||||
]);
|
||||
}
|
||||
args
|
||||
}
|
||||
|
||||
|
||||
@@ -20,6 +20,7 @@ use openshell_core::proto::compute::v1::{
|
||||
ResourceRequirements, WorkloadIdentityRequest,
|
||||
};
|
||||
use std::fs;
|
||||
use std::io::Read as _;
|
||||
use std::sync::Arc;
|
||||
use tempfile::TempDir;
|
||||
|
||||
@@ -197,11 +198,174 @@ fn runtime_config() -> DockerDriverRuntimeConfig {
|
||||
sandbox_pids_limit: openshell_core::config::default_sandbox_pids_limit(),
|
||||
enable_bind_mounts: false,
|
||||
upstream_proxy: UpstreamProxyConfig::default(),
|
||||
proxy_ca_bundle: None,
|
||||
provider_spiffe_workload_api_socket: None,
|
||||
app_armor_profile: Some(AppArmorProfile::Unconfined),
|
||||
}
|
||||
}
|
||||
|
||||
fn write_test_proxy_ca_bundle(directory: &TempDir) -> PathBuf {
|
||||
let tls = generate_sandbox_tls_material(openshell_core::SandboxSessionId::new())
|
||||
.expect("generate test proxy CA");
|
||||
let path = directory.path().join("proxy-ca.pem");
|
||||
fs::write(&path, tls.trust_anchor_pem).expect("write test proxy CA");
|
||||
path
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn docker_config_parses_operator_proxy_ca_bundle() {
|
||||
let config: DockerComputeConfig = toml::from_str(
|
||||
r#"
|
||||
https_proxy = "https://proxy.corp.example:8443"
|
||||
proxy_ca_bundle = "/etc/openshell/tls/proxy-ca.pem"
|
||||
"#,
|
||||
)
|
||||
.expect("parse Docker proxy CA configuration");
|
||||
|
||||
assert_eq!(
|
||||
config.proxy_ca_bundle,
|
||||
Some(PathBuf::from("/etc/openshell/tls/proxy-ca.pem"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn docker_proxy_ca_bundle_validation_is_fail_closed() {
|
||||
let directory = TempDir::new().expect("create CA directory");
|
||||
let ca_bundle = write_test_proxy_ca_bundle(&directory);
|
||||
let gateway_bind_address = "127.0.0.1:17670".parse().unwrap();
|
||||
|
||||
let mut valid = DockerComputeConfig::default();
|
||||
valid.upstream_proxy.https_proxy = Some("http://proxy.corp.example:8080".to_string());
|
||||
valid.proxy_ca_bundle = Some(ca_bundle);
|
||||
valid
|
||||
.validate_configuration(gateway_bind_address)
|
||||
.expect("a valid CA bundle is accepted with an HTTP interception proxy");
|
||||
|
||||
let mut without_proxy = valid.clone();
|
||||
without_proxy.upstream_proxy.https_proxy = None;
|
||||
let error = without_proxy
|
||||
.validate_configuration(gateway_bind_address)
|
||||
.expect_err("a CA bundle without a proxy must fail");
|
||||
assert!(error.to_string().contains("proxy_ca_bundle"), "{error}");
|
||||
assert!(error.to_string().contains("https_proxy"), "{error}");
|
||||
|
||||
let mut empty_path = valid.clone();
|
||||
empty_path.proxy_ca_bundle = Some(PathBuf::new());
|
||||
let error = empty_path
|
||||
.validate_configuration(gateway_bind_address)
|
||||
.expect_err("an empty CA bundle path must fail");
|
||||
assert!(error.to_string().contains("must not be empty"), "{error}");
|
||||
|
||||
let mut missing = valid.clone();
|
||||
missing.proxy_ca_bundle = Some(directory.path().join("missing.pem"));
|
||||
let error = missing
|
||||
.validate_configuration(gateway_bind_address)
|
||||
.expect_err("a missing CA bundle must fail");
|
||||
assert!(error.to_string().contains("could not be read"), "{error}");
|
||||
|
||||
let malformed_path = directory.path().join("malformed.pem");
|
||||
fs::write(&malformed_path, "not a certificate\n").unwrap();
|
||||
let mut malformed = valid;
|
||||
malformed.proxy_ca_bundle = Some(malformed_path);
|
||||
let error = malformed
|
||||
.validate_configuration(gateway_bind_address)
|
||||
.expect_err("a certificate-free CA bundle must fail");
|
||||
assert!(error.to_string().contains("no PEM certificate"), "{error}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn docker_constructor_rejects_proxy_ca_bundle_without_proxy() {
|
||||
let directory = TempDir::new().expect("create CA directory");
|
||||
let mut config = DockerComputeConfig {
|
||||
socket_path: Some(directory.path().join("unused-docker.sock")),
|
||||
proxy_ca_bundle: Some(write_test_proxy_ca_bundle(&directory)),
|
||||
..DockerComputeConfig::default()
|
||||
};
|
||||
config.upstream_proxy.https_proxy = None;
|
||||
|
||||
let Err(error) =
|
||||
DockerComputeDriver::new("127.0.0.1:17670".parse().unwrap(), "info", &config).await
|
||||
else {
|
||||
panic!("constructor must reject incoherent proxy CA configuration before Docker I/O");
|
||||
};
|
||||
|
||||
assert!(error.to_string().contains("proxy_ca_bundle"), "{error}");
|
||||
assert!(error.to_string().contains("https_proxy"), "{error}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn docker_proxy_ca_bundle_uses_fixed_supervisor_path() {
|
||||
let proxy = UpstreamProxyConfig {
|
||||
https_proxy: Some("https://proxy.corp.example:8443".to_string()),
|
||||
..UpstreamProxyConfig::default()
|
||||
};
|
||||
|
||||
let args = docker_upstream_proxy_cli_args(&proxy, true);
|
||||
let option = args
|
||||
.iter()
|
||||
.position(|arg| arg == "--upstream-proxy-ca-bundle")
|
||||
.expect("proxy CA option");
|
||||
assert_eq!(
|
||||
args.get(option + 1).map(String::as_str),
|
||||
Some(SUPERVISOR_PROXY_CA_BUNDLE_MOUNT_PATH)
|
||||
);
|
||||
assert!(
|
||||
!args.iter().any(|arg| arg.contains("/etc/openshell/tls")),
|
||||
"gateway-host paths must not appear in supervisor argv: {args:?}"
|
||||
);
|
||||
|
||||
let args = docker_upstream_proxy_cli_args(&proxy, false);
|
||||
assert!(!args.iter().any(|arg| arg == "--upstream-proxy-ca-bundle"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn docker_proxy_ca_bundle_is_staged_in_supervisor_archive() {
|
||||
let directory = TempDir::new().expect("create CA directory");
|
||||
let ca_bundle = write_test_proxy_ca_bundle(&directory);
|
||||
let expected = fs::read_to_string(&ca_bundle).unwrap();
|
||||
let mut builder = tar::Builder::new(Vec::new());
|
||||
|
||||
append_docker_proxy_ca_bundle(&mut builder, Some(&ca_bundle)).expect("append proxy CA bundle");
|
||||
let archive = builder.into_inner().expect("finish proxy CA archive");
|
||||
let mut archive = tar::Archive::new(archive.as_slice());
|
||||
let mut entries = archive.entries().unwrap();
|
||||
let mut entry = entries.next().expect("proxy CA entry").unwrap();
|
||||
|
||||
assert_eq!(
|
||||
entry.path().unwrap().as_ref(),
|
||||
Path::new("upstream-proxy-ca-bundle.pem")
|
||||
);
|
||||
assert_eq!(entry.header().uid().unwrap(), u64::from(SUPERVISOR_UID));
|
||||
assert_eq!(entry.header().gid().unwrap(), u64::from(SUPERVISOR_GID));
|
||||
assert_eq!(entry.header().mode().unwrap(), 0o644);
|
||||
let mut actual = String::new();
|
||||
entry.read_to_string(&mut actual).unwrap();
|
||||
assert_eq!(actual, expected);
|
||||
assert!(entries.next().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sandbox_driver_config_cannot_override_proxy_ca_bundle() {
|
||||
let config = runtime_config();
|
||||
let mut sandbox = test_sandbox();
|
||||
sandbox
|
||||
.spec
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.template
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.driver_config = Some(json_struct(serde_json::json!({
|
||||
"proxy_ca_bundle": "/workload/controlled-ca.pem"
|
||||
})));
|
||||
|
||||
let error = DockerComputeDriver::validate_sandbox(&sandbox, &config)
|
||||
.expect_err("sandbox driver config must not accept proxy_ca_bundle");
|
||||
assert_eq!(error.code(), tonic::Code::InvalidArgument);
|
||||
assert!(error.message().contains("unknown field"), "{error}");
|
||||
assert!(error.message().contains("proxy_ca_bundle"), "{error}");
|
||||
}
|
||||
|
||||
fn test_workload_identity() -> ResolvedWorkloadIdentity {
|
||||
ResolvedWorkloadIdentity::new(
|
||||
1234,
|
||||
|
||||
@@ -896,6 +896,13 @@ no_proxy = ".svc.cluster.local,10.0.0.0/8"
|
||||
# Optional root-owned host file containing user:pass. An http:// proxy also
|
||||
# requires proxy_auth_allow_insecure = true as an explicit acknowledgement.
|
||||
proxy_auth_file = "/etc/openshell/secrets/proxy-auth"
|
||||
proxy_auth_allow_insecure = false
|
||||
# Last resort for proxy ACLs that require hostname CONNECT targets. The proxy
|
||||
# then performs DNS resolution and becomes part of the effective egress boundary.
|
||||
proxy_connect_by_hostname = false
|
||||
# Operator-owned gateway-host PEM bundle trusted for an HTTPS proxy and for
|
||||
# destination certificates re-signed by a TLS-intercepting proxy.
|
||||
proxy_ca_bundle = "/etc/openshell/tls/proxy-ca.pem"
|
||||
# Project a host Unix Workload API socket into the supervisor for provider
|
||||
# token exchange. The socket parent must be a dedicated absolute directory.
|
||||
provider_spiffe_workload_api_socket = "/run/spire/agent.sock"
|
||||
@@ -904,6 +911,30 @@ provider_spiffe_workload_api_socket = "/run/spire/agent.sock"
|
||||
Use `sandbox_label` for Docker configurations. The legacy
|
||||
`sandbox_namespace` key is rejected.
|
||||
|
||||
Docker accepts `http://` and `https://` proxy URLs in explicit
|
||||
`scheme://host:port` form. `no_proxy` bypasses only the corporate proxy;
|
||||
OpenShell policy still applies. Proxy URLs cannot embed credentials. Supply a
|
||||
`user:pass` file with `proxy_auth_file`, and set
|
||||
`proxy_auth_allow_insecure = true` only to acknowledge that Basic credentials
|
||||
travel in cleartext to an `http://` proxy.
|
||||
|
||||
`proxy_ca_bundle` requires `https_proxy`, but the configured proxy URL may use
|
||||
either scheme because a plain HTTP proxy can still intercept destination TLS.
|
||||
The file must be a readable, bounded PEM bundle containing a usable certificate
|
||||
authority. Docker validates it at gateway startup and again before each
|
||||
supervisor launch. Missing, unreadable, empty, oversized, malformed, and
|
||||
certificate-free bundles fail closed rather than falling back to default trust
|
||||
or direct egress.
|
||||
|
||||
The gateway-host path is operator-owned and cannot be selected through sandbox
|
||||
environment, image contents, or `template.driver_config.docker`. The driver
|
||||
copies the validated contents into its supervisor-only named volume and passes
|
||||
the fixed path `/.openshell/supervisor/upstream-proxy-ca-bundle.pem` to the
|
||||
companion supervisor. The bundle extends trust for the TLS connection to an
|
||||
`https://` proxy, supervisor connections to re-signed upstream certificates,
|
||||
and the combined trust bundle exposed to workload processes. The gateway-host
|
||||
path is never mounted into or exposed to the workload container.
|
||||
|
||||
### Podman
|
||||
|
||||
Each Podman sandbox uses two containers. The workload container runs `openshell-sandbox` with `network=none`; the supervisor container runs on the host network and initiates policy-approved upstream connections. A private volume carries their authenticated Unix-domain socket. Configure guest mTLS paths once under `[openshell.gateway]`; the gateway validates and injects the bundle into the selected local driver.
|
||||
|
||||
@@ -96,6 +96,26 @@ Common options in `[openshell.drivers.docker]` are `socket_path`, `grpc_endpoint
|
||||
|
||||
Docker Desktop must have host networking enabled, and it cannot use Enhanced Container Isolation. Set `grpc_endpoint` when sandboxes cannot reach the gateway on host loopback. For GPU sandboxes, configure Docker CDI before starting the gateway.
|
||||
|
||||
### Docker Corporate Proxy Egress
|
||||
|
||||
For proxy-required networks, the Docker driver accepts `https_proxy`,
|
||||
`no_proxy`, `proxy_auth_file`, `proxy_auth_allow_insecure`,
|
||||
`proxy_connect_by_hostname`, and `proxy_ca_bundle`. The companion supervisor
|
||||
chains policy-approved TLS tunnels through the proxy with HTTP CONNECT.
|
||||
|
||||
`proxy_ca_bundle` is an operator-owned gateway-host PEM path. Docker validates
|
||||
and copies the bundle into its supervisor-only named volume, so this also works
|
||||
with remote Docker daemons and does not expose the host path to the workload.
|
||||
The supervisor uses the bundle for an HTTPS proxy connection and for upstream
|
||||
certificates re-signed by a TLS-intercepting proxy. Workload processes receive
|
||||
the same corporate roots through their generated combined trust bundle.
|
||||
|
||||
Sandbox environment, image contents, and `template.driver_config.docker`
|
||||
cannot alter these settings. Invalid proxy relationships or CA files prevent
|
||||
the gateway or sandbox from starting and never degrade to direct egress. See
|
||||
the [Gateway Configuration File](/how-it-works/gateways/configuration) for URL,
|
||||
authentication, `NO_PROXY`, hostname CONNECT, and CA validation details.
|
||||
|
||||
### Docker Mounts
|
||||
|
||||
Mount existing named volumes or `tmpfs` through driver config. Label volumes so resource admission accepts them:
|
||||
|
||||
@@ -68,6 +68,11 @@ name = "docker_preflight"
|
||||
path = "tests/docker_preflight.rs"
|
||||
required-features = ["e2e-docker"]
|
||||
|
||||
[[test]]
|
||||
name = "docker_corporate_proxy"
|
||||
path = "tests/docker_corporate_proxy.rs"
|
||||
required-features = ["e2e-docker"]
|
||||
|
||||
[[test]]
|
||||
name = "driver_config_volume"
|
||||
path = "tests/driver_config_volume.rs"
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -296,6 +296,34 @@ Common findings:
|
||||
- The sandbox fails its enforcement probe: inspect the sandbox log for the exact nested seccomp user-notification, task-memory, Landlock, loopback DNS, or socket-injection check that failed. A runtime may return `ENOSYS` for `process_vm_readv` and `process_vm_writev` while satisfying the production parent-to-workload-child task-memory probe through `/proc/<pid>/mem`; only failure of both backends is fatal. Do not add capabilities or switch to an unconfined seccomp profile; use a runtime whose default profile permits the unprivileged probe.
|
||||
- A GPU sandbox fails because Docker reports no discovered NVIDIA CDI devices: verify `.DiscoveredDevices` contains entries such as `nvidia.com/gpu=all`, verify `/etc/cdi` or `/var/run/cdi` contains a generated NVIDIA spec, and check that `nvidia-cdi-refresh.service` and `nvidia-cdi-refresh.path` from NVIDIA Container Toolkit are enabled and healthy. The service is a one-shot unit, so `inactive (dead)` can be normal after a successful run; use `systemctl status` and `journalctl` to distinguish success from a skipped or failed refresh. Restart `nvidia-cdi-refresh.service` to regenerate missing or stale CDI specs, then restart or reload Docker and re-check `docker info`.
|
||||
|
||||
#### Corporate upstream proxy
|
||||
|
||||
Docker corporate proxy settings are operator-owned fields under
|
||||
`[openshell.drivers.docker]`. Confirm the complete proxy table and inspect the
|
||||
companion supervisor command and logs:
|
||||
|
||||
```bash
|
||||
grep -A20 '^\[openshell.drivers.docker\]' <gateway.toml> | grep -E 'https_proxy|no_proxy|proxy_auth_file|proxy_auth_allow_insecure|proxy_connect_by_hostname|proxy_ca_bundle'
|
||||
docker ps --filter label=openshell.ai/isolation-role=supervisor
|
||||
docker inspect --format '{{json .Config.Cmd}} {{json .Mounts}}' <supervisor-container>
|
||||
docker logs <supervisor-container> --tail=200 | grep -Ei 'upstream|connect|proxy|certificate'
|
||||
```
|
||||
|
||||
`proxy_ca_bundle` names a gateway-host PEM file and requires `https_proxy`.
|
||||
The proxy URL may use `http://` or `https://`; a plain HTTP proxy may still
|
||||
re-sign destination TLS. Missing, unreadable, empty, oversized, malformed, or
|
||||
certificate-free bundles fail closed. The Docker driver copies a validated
|
||||
bundle into its supervisor-only named volume and passes the fixed path
|
||||
`/.openshell/supervisor/upstream-proxy-ca-bundle.pem`. The gateway-host path
|
||||
must not appear in container arguments, mounts, workload environment, or
|
||||
`template.driver_config.docker`.
|
||||
|
||||
An HTTPS proxy certificate error usually means the bundle lacks the proxy
|
||||
listener issuer or its certificate does not match the proxy hostname. A
|
||||
TLS-intercepted destination error means the re-signing issuer is missing or the
|
||||
supervisor did not receive the bundle. Keep verification enabled and correct
|
||||
the operator bundle.
|
||||
|
||||
During a graceful gateway restart, Docker, Podman, and VM sandboxes with
|
||||
running intent should stop before the gateway exits and restart after it
|
||||
returns. Check for `Stopped sandbox during gateway shutdown` and `Started
|
||||
|
||||
@@ -214,6 +214,9 @@ fi
|
||||
if [[ -n "${OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME+x}" ]]; then
|
||||
printf 'proxy_connect_by_hostname = %s\n' "${OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME}" >>"${CONFIG_PATH}"
|
||||
fi
|
||||
if [[ -n "${OPENSHELL_SANDBOX_PROXY_CA_BUNDLE+x}" ]]; then
|
||||
printf 'proxy_ca_bundle = "%s"\n' "$(toml_escape "${OPENSHELL_SANDBOX_PROXY_CA_BUNDLE}")" >>"${CONFIG_PATH}"
|
||||
fi
|
||||
if [[ -n "${OPENSHELL_PROVIDER_SPIFFE_WORKLOAD_API_SOCKET+x}" ]]; then
|
||||
printf 'provider_spiffe_workload_api_socket = "%s"\n' "$(toml_escape "${OPENSHELL_PROVIDER_SPIFFE_WORKLOAD_API_SOCKET}")" >>"${CONFIG_PATH}"
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user