fix(macos): restore Homebrew sandbox callbacks (#2739)

* fix(macos): restore Docker gateway callbacks

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(gateway): reuse reachable primary callback listener

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
This commit is contained in:
Drew Newberry
2026-08-14 18:38:20 +00:00
committed by GitHub
parent 1074566ce3
commit f12f3ef8d5
16 changed files with 255 additions and 144 deletions
@@ -178,6 +178,13 @@ Common findings:
- A workdir rejected as a special filesystem or OpenShell control-path collision cannot be made valid with permissions. Move the image workdir away from kernel-backed mounts and the concrete supervisor, TLS, token, runtime, and socket paths named in the error.
- Docker driver cannot initialize because it cannot find `openshell-sandbox`: verify `OPENSHELL_DOCKER_SUPERVISOR_BIN`, the sibling binary next to `openshell-gateway`, or the configured supervisor image contains `/openshell-sandbox`.
- Sandbox never registers: check gateway logs and supervisor callback endpoint.
- On macOS, repeated `Policy fetch failed after 5 attempts` messages with a
Homebrew gateway bound to `[::1]:17670` indicate that the Docker
`host-gateway` IPv4 route has no matching callback listener. Current releases
leave `bind_address` unset in the Homebrew config, use the built-in
`127.0.0.1:17670` primary listener, and reuse it for authenticated sandbox
callbacks. On an older release, set `bind_address = "127.0.0.1:17670"` or
upgrade.
- Supervisor image exits before printing `openshell-sandbox --version`: the image should be the scratch supervisor image from `deploy/docker/Dockerfile.supervisor` and must contain a static executable at `/openshell-sandbox`.
- `mise run e2e:docker:gpu` fails with `docker info --format json did not report any discovered NVIDIA CDI GPU devices`: Docker may report `CDISpecDirs` while still having no generated NVIDIA CDI specs. Verify `.DiscoveredDevices` contains entries such as `nvidia.com/gpu=all`, verify `/etc/cdi` or `/var/run/cdi` contains a generated NVIDIA spec, and check that `nvidia-cdi-refresh.service` and `nvidia-cdi-refresh.path` from NVIDIA Container Toolkit are enabled and healthy. The service is a one-shot unit, so `inactive (dead)` can be normal after a successful run; use `systemctl status` and `journalctl` to distinguish success from a skipped or failed refresh. NVIDIA recommends enabling the path and service units, and restarting `nvidia-cdi-refresh.service` to regenerate missing or stale CDI specs. If specs are generated but Docker still reports no discovered devices, restart Docker or reload the daemon and re-check `docker info`.
@@ -207,14 +214,9 @@ Common findings:
error: inspect `podman info --debug`, the configured Podman network, and the
host's IPv4 default route. Rootless pasta uses the private source address
selected by that route; rootful Podman uses the bridge gateway address.
- Callback discovery reports that the requested address equals the primary
listener: configure a distinct primary address. For Podman Machine, bind the
primary listener to IPv6 loopback, for example
`bind_address = "[::1]:17670"`, and register the CLI endpoint as
`https://localhost:17670`. The generated certificate includes `localhost`,
while a raw `https://[::1]:17670` endpoint can fail TLS setup with
`invalid dns name`. This leaves `127.0.0.1:17670` available for the
callback-only listener.
- Current gateways reuse the primary listener when it covers Podman's callback
address. If the primary does not cover that address, inspect the gateway
startup logs for the additional callback-only listener and its provenance.
- Rootless slirp4netns, another named helper, or missing helper metadata
requires an explicitly remote `grpc_endpoint`. An explicit `host_gateway_ip`
cannot bypass slirp4netns host-loopback isolation. Do not work around
@@ -544,7 +546,7 @@ openshell logs <sandbox-name>
| `openshell status` fails | Gateway endpoint unreachable or auth mismatch | `openshell gateway info`, gateway logs |
| Gateway starts but sandbox create fails | Compute driver cannot reach runtime | Docker/Podman/Kubernetes/VM driver logs |
| Gateway exits while resolving compute-driver listener requirements | Callback alias topology is unsupported, the Podman network cannot be inspected, or the selected address is not private/authorized | Gateway startup error, `podman info --debug`, Podman network inspection, host IPv4 default route |
| Admin, health, reflection, or HTTP request is denied on a Docker/Podman callback address | Negotiated callback listeners intentionally expose only sandbox-callable gRPC methods | Retry through the gateway's primary endpoint; inspect the listener-purpose startup log if the address was unexpected |
| Admin, health, reflection, or HTTP request is denied on an additional Docker/Podman callback-only listener | Additional callback listeners intentionally expose only sandbox-callable gRPC methods | Retry through the gateway's primary endpoint; inspect the listener-purpose startup log if the address was unexpected |
| Docker or Podman sandbox never registers | Wrong callback endpoint or supervisor startup failure | Gateway logs and sandbox container logs |
| Docker GPU e2e fails before GPU sandbox comparison | NVIDIA CDI specs are missing or Docker has not discovered them | `docker info --format '{{json .DiscoveredDevices}}'`, `/etc/cdi`, `/var/run/cdi`, `nvidia-cdi-refresh.service` |
| Kubernetes gateway pod pending | PVC unbound, taint, selector, or insufficient resources | `kubectl -n openshell describe pod <pod>` |
+9 -9
View File
@@ -37,19 +37,17 @@ health, metrics, or tunnel routes. The plaintext service router also rejects
browser requests whose Fetch Metadata, Origin, or Referer headers indicate a
cross-origin or sibling-subdomain request.
Docker and Podman may negotiate additional listeners that make the gateway
reachable from their local sandbox network topology. Those listeners accept
Docker and Podman report the local address through which their sandboxes can
reach the gateway. When the primary listener covers that address, the gateway
reuses it; sandbox JWT authentication and its RPC allowlist remain the callback
authorization boundary. When the primary listener does not cover the address,
the gateway adds a callback-only listener. Additional callback listeners accept
only gRPC methods classified as sandbox-callable by the gateway's generated
authorization metadata. They reject user and administrator APIs, health,
reflection, non-callback inference APIs, and HTTP routes before normal request
authentication. The operator-configured primary listener retains the full
multiplexed API surface.
The gateway rejects a callback requirement that resolves to the exact primary
listener address because one socket cannot preserve two authorization scopes.
A wildcard primary listener may cover a callback address because the accepted
connection's concrete local address still selects the callback-only scope.
The `rpc_auth` classification is also the source of truth for negotiated
listener exposure: marking an RPC as `sandbox` or `dual` makes it callable on
these listeners. Review such changes as both authorization and network-surface
@@ -699,8 +697,10 @@ system entry instead of pretending to delete package-manager owned state.
- Compute runtimes own the mechanics of starting workloads and injecting
callback configuration.
- Docker-backed local gateways use Docker's `host-gateway` callback alias on
macOS and Docker Desktop-style runtimes. Native Linux Docker may expose an
additional bridge-gateway listener because the host can bind that bridge IP.
macOS and Docker Desktop-style runtimes. They request IPv4 loopback callback
reachability and add a listener only when the primary does not cover it.
Native Linux Docker may expose an additional bridge-gateway listener because
the host can bind that bridge IP.
- Podman-backed macOS gateways use gvproxy's host-loopback IP for sandbox host
aliases by default so stale Podman machine images do not need Podman's
`host-gateway` resolver. Linux Podman keeps the resolver unless
+63 -21
View File
@@ -22,6 +22,7 @@ use tokio::sync::Mutex;
use tonic::service::interceptor::InterceptedService;
use tonic::transport::{Certificate, Channel, ClientTlsConfig, Endpoint, Identity};
use tracing::debug;
use url::{Host, Url};
/// Concrete gRPC client type used by all commands.
pub type GrpcClient = OpenShellClient<InterceptedService<Channel, EdgeAuthInterceptor>>;
@@ -223,12 +224,33 @@ pub fn build_rustls_config(materials: &TlsMaterials) -> Result<rustls::ClientCon
.into_diagnostic()
}
pub fn build_tonic_tls_config(materials: &TlsMaterials) -> ClientTlsConfig {
pub fn build_tonic_tls_config(server: &str, materials: &TlsMaterials) -> Result<ClientTlsConfig> {
let ca_cert = Certificate::from_pem(materials.ca.clone());
let identity = Identity::from_pem(materials.cert.clone(), materials.key.clone());
ClientTlsConfig::new()
.ca_certificate(ca_cert)
.identity(identity)
tls_config_with_server_name(
server,
ClientTlsConfig::new()
.ca_certificate(ca_cert)
.identity(identity),
)
}
fn tls_config_with_server_name(server: &str, config: ClientTlsConfig) -> Result<ClientTlsConfig> {
Ok(config.domain_name(tls_server_name(server)?))
}
fn tls_server_name(server: &str) -> Result<String> {
let endpoint = Url::parse(server)
.into_diagnostic()
.wrap_err_with(|| format!("invalid gateway endpoint '{server}'"))?;
let host = endpoint
.host()
.ok_or_else(|| miette::miette!("gateway endpoint '{server}' has no host"))?;
Ok(match host {
Host::Domain(domain) => domain.to_string(),
Host::Ipv4(address) => address.to_string(),
Host::Ipv6(address) => address.to_string(),
})
}
#[derive(Debug)]
@@ -402,22 +424,20 @@ pub async fn build_channel(server: &str, tls: &TlsOptions) -> Result<Channel> {
// Bearer auth over HTTPS: use mTLS certs for the transport layer when
// available (server may still require client certs), and layer the
// Bearer token on top via the interceptor.
require_tls_materials(server, tls).map_or_else(
|_| {
let resolved = tls.with_default_paths(server);
resolved
.ca
.as_ref()
.and_then(|ca_path| std::fs::read(ca_path).ok())
.map_or_else(
|| ClientTlsConfig::new().with_enabled_roots(),
|ca_pem| {
ClientTlsConfig::new().ca_certificate(Certificate::from_pem(ca_pem))
},
)
},
|materials| build_tonic_tls_config(&materials),
)
if let Ok(materials) = require_tls_materials(server, tls) {
build_tonic_tls_config(server, &materials)?
} else {
let resolved = tls.with_default_paths(server);
let config = resolved
.ca
.as_ref()
.and_then(|ca_path| std::fs::read(ca_path).ok())
.map_or_else(
|| ClientTlsConfig::new().with_enabled_roots(),
|ca_pem| ClientTlsConfig::new().ca_certificate(Certificate::from_pem(ca_pem)),
);
tls_config_with_server_name(server, config)?
}
} else if tls.edge_token.is_some() {
// Edge bearer mode — routed through tunnel above; if we reach here
// the server is not HTTPS so connect plaintext.
@@ -425,7 +445,7 @@ pub async fn build_channel(server: &str, tls: &TlsOptions) -> Result<Channel> {
} else {
// Standard mTLS: private CA + client cert.
let materials = require_tls_materials(server, tls)?;
build_tonic_tls_config(&materials)
build_tonic_tls_config(server, &materials)?
};
endpoint = endpoint.tls_config(tls_config).into_diagnostic()?;
endpoint.connect().await.into_diagnostic()
@@ -451,3 +471,25 @@ pub async fn grpc_inference_client(server: &str, tls: &TlsOptions) -> Result<Grp
let interceptor = interceptor_from_tls(tls)?;
Ok(InferenceClient::with_interceptor(channel, interceptor))
}
#[cfg(test)]
mod tests {
use super::tls_server_name;
#[test]
fn tls_server_name_normalizes_bracketed_ipv6_endpoint() {
assert_eq!(
tls_server_name("https://[::1]:17670").unwrap(),
"::1",
"rustls accepts an unbracketed IPv6 address as an IP server name"
);
}
#[test]
fn tls_server_name_preserves_dns_endpoint() {
assert_eq!(
tls_server_name("https://localhost:17670").unwrap(),
"localhost"
);
}
}
+4 -2
View File
@@ -67,8 +67,10 @@ Docker containers join an OpenShell-managed bridge network. The driver injects
`host.openshell.internal` and `host.docker.internal` so supervisors have stable
names for reaching the gateway host. On Docker Desktop, Colima, Rancher
Desktop, OrbStack, and macOS-hosted gateways, those names use Docker's
`host-gateway` alias. On native Linux Docker, the gateway also binds the bridge
gateway IP so containers can call back to the host process.
`host-gateway` alias. The driver requests a separate IPv4 loopback callback
listener when the primary listener does not already cover it. On native Linux
Docker, the gateway also binds the bridge gateway IP so containers can call
back to the host process.
## Container Contract
+34 -10
View File
@@ -55,7 +55,7 @@ use openshell_core::proto_struct::{
use openshell_core::{Config, Error, Result as CoreResult};
use std::collections::{HashMap, HashSet};
use std::io::Read;
use std::net::{IpAddr, SocketAddr};
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::path::{Path, PathBuf};
use std::pin::Pin;
use std::sync::Arc;
@@ -177,6 +177,7 @@ struct DockerDriverRuntimeConfig {
grpc_endpoint: String,
network_name: String,
gateway_route: DockerGatewayRoute,
gateway_callback_bind_address: Option<SocketAddr>,
ssh_socket_path: String,
stop_timeout_secs: u32,
log_level: String,
@@ -428,6 +429,8 @@ impl DockerComputeDriver {
let host_gateway_ip = parse_optional_host_gateway_ip(&docker_config.host_gateway_ip)?;
let gateway_route =
docker_gateway_route(&info, bridge_gateway_ip, gateway_port, host_gateway_ip);
let gateway_callback_bind_address =
docker_gateway_callback_bind_address(&gateway_route, config.bind_address);
let mut docker_config = docker_config.clone();
if docker_config.grpc_endpoint.trim().is_empty() {
let scheme = if docker_guest_tls_configured(&docker_config) {
@@ -456,6 +459,7 @@ impl DockerComputeDriver {
grpc_endpoint,
network_name,
gateway_route,
gateway_callback_bind_address,
ssh_socket_path: docker_config.ssh_socket_path.clone(),
stop_timeout_secs: DEFAULT_STOP_TIMEOUT_SECS,
log_level: config.log_level.clone(),
@@ -1577,15 +1581,19 @@ impl ComputeDriver for DockerComputeDriver {
&self,
_request: Request<GetGatewayListenerRequirementsRequest>,
) -> Result<Response<GetGatewayListenerRequirementsResponse>, Status> {
let requirements = match self.config.gateway_route {
DockerGatewayRoute::Bridge { bind_address, .. } => {
vec![GatewayListenerRequirement {
reason: "docker managed bridge gateway".to_string(),
selector: Some(Selector::ExactBindAddress(bind_address.to_string())),
}]
}
DockerGatewayRoute::HostGateway => Vec::new(),
};
let requirements =
self.config
.gateway_callback_bind_address
.map_or_else(Vec::new, |bind_address| {
vec![GatewayListenerRequirement {
reason: match self.config.gateway_route {
DockerGatewayRoute::Bridge { .. } => "docker managed bridge gateway",
DockerGatewayRoute::HostGateway => "docker host-gateway IPv4 loopback",
}
.to_string(),
selector: Some(Selector::ExactBindAddress(bind_address.to_string())),
}]
});
Ok(Response::new(GetGatewayListenerRequirementsResponse {
requirements,
}))
@@ -2793,6 +2801,22 @@ fn docker_gateway_route_for_host(
}
}
fn docker_gateway_callback_bind_address(
route: &DockerGatewayRoute,
primary_bind_address: SocketAddr,
) -> Option<SocketAddr> {
match route {
DockerGatewayRoute::Bridge { bind_address, .. } => Some(*bind_address),
DockerGatewayRoute::HostGateway => match primary_bind_address.ip() {
IpAddr::V4(ip) if ip.is_unspecified() || ip == Ipv4Addr::LOCALHOST => None,
_ => Some(SocketAddr::new(
IpAddr::V4(Ipv4Addr::LOCALHOST),
primary_bind_address.port(),
)),
},
}
}
fn host_runtime_requires_host_gateway_alias() -> bool {
cfg!(target_os = "macos")
}
@@ -102,6 +102,10 @@ fn runtime_config() -> DockerDriverRuntimeConfig {
),
host_alias_ip: IpAddr::V4(Ipv4Addr::new(172, 18, 0, 1)),
},
gateway_callback_bind_address: Some(SocketAddr::new(
IpAddr::V4(Ipv4Addr::new(172, 18, 0, 1)),
DEFAULT_SERVER_PORT,
)),
ssh_socket_path: "/run/openshell/ssh.sock".to_string(),
stop_timeout_secs: DEFAULT_STOP_TIMEOUT_SECS,
log_level: "info".to_string(),
@@ -186,6 +190,7 @@ async fn gateway_listener_requirements_report_managed_bridge_address() {
async fn gateway_listener_requirements_are_empty_for_host_gateway_route() {
let mut config = runtime_config();
config.gateway_route = DockerGatewayRoute::HostGateway;
config.gateway_callback_bind_address = None;
let driver = test_driver_with_config(config);
let response = driver
@@ -197,6 +202,26 @@ async fn gateway_listener_requirements_are_empty_for_host_gateway_route() {
assert!(response.requirements.is_empty());
}
#[tokio::test]
async fn host_gateway_route_reports_ipv4_loopback_callback_listener() {
let mut config = runtime_config();
config.gateway_route = DockerGatewayRoute::HostGateway;
config.gateway_callback_bind_address = Some("127.0.0.1:17670".parse().unwrap());
let driver = test_driver_with_config(config);
let response = driver
.get_gateway_listener_requirements(Request::new(GetGatewayListenerRequirementsRequest {}))
.await
.unwrap()
.into_inner();
assert_eq!(response.requirements.len(), 1);
assert_eq!(
response.requirements[0].selector,
Some(Selector::ExactBindAddress("127.0.0.1:17670".to_string()))
);
}
#[test]
fn container_visible_endpoint_rewrites_loopback_hosts() {
assert_eq!(
@@ -298,6 +323,31 @@ fn docker_gateway_route_uses_host_gateway_for_docker_desktop() {
);
}
#[test]
fn host_gateway_route_requests_ipv4_loopback_for_ipv6_primary() {
assert_eq!(
docker_gateway_callback_bind_address(
&DockerGatewayRoute::HostGateway,
"[::1]:17670".parse().unwrap(),
),
Some("127.0.0.1:17670".parse().unwrap())
);
}
#[test]
fn host_gateway_route_reuses_ipv4_primary_when_it_covers_loopback() {
for primary in ["127.0.0.1:17670", "0.0.0.0:17670"] {
assert_eq!(
docker_gateway_callback_bind_address(
&DockerGatewayRoute::HostGateway,
primary.parse().unwrap(),
),
None,
"{primary} already covers the IPv4 loopback callback"
);
}
}
#[test]
fn docker_gateway_route_uses_host_gateway_for_colima() {
let info = SystemInfo {
+5 -7
View File
@@ -275,13 +275,11 @@ and rootless pasta because the driver maps both local callback aliases to that
literal. Other rootless helpers still fail closed. Podman Machine requests
gateway loopback because its configured address is guest-visible and gvproxy
terminates that route on host loopback. The gateway validates and binds every
accepted callback listener. A callback address cannot equal the exact primary
listener address because the gateway could not distinguish their authorization
scopes. In particular, a Podman Machine gateway using the IPv4 loopback
callback must place its primary listener on another address, such as IPv6
loopback (`[::1]:17670`). Negotiated callback listeners expose only the
gateway's sandbox-callable gRPC methods. Operator, health, reflection, and HTTP
requests must use the primary listener.
accepted callback requirement. If the primary listener covers the requested
address, the gateway reuses it and relies on sandbox JWT authorization to limit
the supervisor's RPCs. Otherwise, it creates an additional listener that
exposes only the gateway's sandbox-callable gRPC methods. Operator, health,
reflection, and HTTP requests must use the primary listener.
### Layer 3 Inner Sandbox Network Namespace
+25 -59
View File
@@ -103,7 +103,7 @@ fn gateway_listener_specs_with_default_route_ip(
continue;
};
validate_gateway_listener_requirement(bind_address, requirement)?;
add_callback_listener_spec(&mut specs, *address, requirement)?;
add_callback_listener_spec(&mut specs, *address, requirement);
}
for requirement in requirements {
@@ -127,7 +127,7 @@ fn gateway_listener_specs_with_default_route_ip(
}
let address = SocketAddr::new(ip, bind_address.port());
validate_resolved_gateway_listener(bind_address, address)?;
add_callback_listener_spec(&mut specs, address, requirement)?;
add_callback_listener_spec(&mut specs, address, requirement);
}
for requirement in requirements {
@@ -137,7 +137,7 @@ fn gateway_listener_specs_with_default_route_ip(
validate_gateway_listener_requirement(bind_address, requirement)?;
let address = SocketAddr::from(([127, 0, 0, 1], bind_address.port()));
validate_resolved_gateway_listener(bind_address, address)?;
add_callback_listener_spec(&mut specs, address, requirement)?;
add_callback_listener_spec(&mut specs, address, requirement);
}
Ok(specs)
@@ -147,20 +147,17 @@ fn add_callback_listener_spec(
specs: &mut Vec<GatewayListenerSpec>,
address: SocketAddr,
requirement: &GatewayListenerRequirement,
) -> Result<()> {
) {
let scope = GatewayListenerScope::ComputeDriverCallback;
if let Some(existing) = specs
.iter_mut()
.find(|existing| listener_covers(existing.address, address))
{
if existing.scope == GatewayListenerScope::Primary {
return;
}
if existing.address == address {
if existing.scope == GatewayListenerScope::Primary {
return Err(Error::config(format!(
"compute driver '{}' requested gateway callback listener {address}, but it is the same address as the primary listener; callback-only authorization cannot be preserved",
requirement.driver_name()
)));
}
return Ok(());
return;
}
if !existing
.covered_addresses
@@ -171,10 +168,9 @@ fn add_callback_listener_spec(
.covered_addresses
.push(CoveredGatewayAddress { address, scope });
}
return Ok(());
return;
}
specs.push(callback_listener_spec(address, requirement));
Ok(())
}
fn callback_listener_spec(
@@ -392,8 +388,8 @@ fn listener_covers(existing: SocketAddr, requested: SocketAddr) -> bool {
#[cfg(test)]
mod tests {
use super::{
CoveredGatewayAddress, GatewayListenerProvenance, GatewayListenerScope,
GatewayListenerSpec, bind_gateway_listeners, gateway_listener_specs,
GatewayListenerProvenance, GatewayListenerScope, GatewayListenerSpec,
bind_gateway_listeners, gateway_listener_specs,
gateway_listener_specs_with_default_route_ip,
};
use crate::compute::GatewayListenerRequirement;
@@ -402,7 +398,7 @@ mod tests {
use tokio::net::TcpListener;
#[test]
fn gateway_listener_specs_track_driver_address_covered_by_wildcard() {
fn gateway_listener_specs_reuse_primary_when_wildcard_covers_driver_address() {
let primary: SocketAddr = "0.0.0.0:8080".parse().unwrap();
let docker: SocketAddr = "172.18.0.1:8080".parse().unwrap();
let requirements = [
@@ -412,20 +408,12 @@ mod tests {
assert_eq!(
gateway_listener_specs(primary, &requirements).unwrap(),
vec![GatewayListenerSpec {
address: primary,
scope: GatewayListenerScope::Primary,
covered_addresses: vec![CoveredGatewayAddress {
address: docker,
scope: GatewayListenerScope::ComputeDriverCallback,
}],
provenance: None,
}]
vec![primary_listener_spec(primary)]
);
}
#[test]
fn gateway_listener_scope_for_local_addr_uses_covered_address_scope() {
fn gateway_listener_scope_for_reused_primary_remains_primary() {
let primary: SocketAddr = "0.0.0.0:8080".parse().unwrap();
let docker: SocketAddr = "172.18.0.1:8080".parse().unwrap();
let loopback: SocketAddr = "127.0.0.1:8080".parse().unwrap();
@@ -436,7 +424,7 @@ mod tests {
assert_eq!(
spec.scope_for_local_addr(docker),
GatewayListenerScope::ComputeDriverCallback,
GatewayListenerScope::Primary,
);
assert_eq!(
spec.scope_for_local_addr(loopback),
@@ -521,14 +509,14 @@ mod tests {
}
#[test]
fn gateway_listener_specs_track_podman_exact_when_primary_covers_it() {
fn gateway_listener_specs_reuse_primary_when_it_covers_podman_exact() {
let primary: SocketAddr = "0.0.0.0:8080".parse().unwrap();
let podman_gateway: SocketAddr = "10.89.1.1:8080".parse().unwrap();
assert_eq!(
gateway_listener_specs(primary, &[podman_listener_requirement(podman_gateway)],)
.unwrap(),
vec![primary_listener_spec_with_covered(primary, podman_gateway,)]
vec![primary_listener_spec(primary)]
);
}
@@ -570,11 +558,9 @@ mod tests {
}
#[test]
fn gateway_listener_specs_track_default_route_when_primary_is_ipv4_wildcard() {
fn gateway_listener_specs_reuse_ipv4_wildcard_for_default_route() {
let primary: SocketAddr = "0.0.0.0:8080".parse().unwrap();
let default_route_ip = "192.168.20.20".parse().unwrap();
let callback = "192.168.20.20:8080".parse().unwrap();
assert_eq!(
gateway_listener_specs_with_default_route_ip(
primary,
@@ -582,7 +568,7 @@ mod tests {
Some(default_route_ip),
)
.unwrap(),
vec![primary_listener_spec_with_covered(primary, callback)]
vec![primary_listener_spec(primary)]
);
}
@@ -604,28 +590,23 @@ mod tests {
}
#[test]
fn gateway_listener_specs_track_podman_loopback_when_wildcard_primary_covers_it() {
fn gateway_listener_specs_reuse_wildcard_primary_for_podman_loopback() {
let primary = "0.0.0.0:8080".parse().unwrap();
let loopback = "127.0.0.1:8080".parse().unwrap();
assert_eq!(
gateway_listener_specs(primary, &[podman_loopback_listener_requirement()]).unwrap(),
vec![primary_listener_spec_with_covered(primary, loopback)]
vec![primary_listener_spec(primary)]
);
}
#[test]
fn gateway_listener_specs_reject_callback_matching_primary_address() {
fn gateway_listener_specs_reuse_matching_primary_address() {
let primary = "127.0.0.1:8080".parse().unwrap();
let err =
gateway_listener_specs(primary, &[podman_loopback_listener_requirement()]).unwrap_err();
assert!(
err.to_string()
.contains("same address as the primary listener")
assert_eq!(
gateway_listener_specs(primary, &[podman_loopback_listener_requirement()]).unwrap(),
vec![primary_listener_spec(primary)]
);
assert!(err.to_string().contains("callback-only authorization"));
}
#[test]
@@ -742,21 +723,6 @@ mod tests {
}
}
fn primary_listener_spec_with_covered(
address: SocketAddr,
covered_address: SocketAddr,
) -> GatewayListenerSpec {
GatewayListenerSpec {
address,
scope: GatewayListenerScope::Primary,
covered_addresses: vec![CoveredGatewayAddress {
address: covered_address,
scope: GatewayListenerScope::ComputeDriverCallback,
}],
provenance: None,
}
}
fn callback_listener_spec(
address: SocketAddr,
driver_name: &str,
+1 -1
View File
@@ -38,7 +38,7 @@ For detailed driver behavior, refer to [Sandbox Compute Drivers](/reference/sand
On macOS, the install script uses Homebrew. The Homebrew package installs the `openshell` CLI, the gateway binary, and a Homebrew-managed gateway service.
The Homebrew service listens on `https://[::1]:17670` and generates a local mTLS bundle on install. The formula creates a Homebrew prefix config, such as `/opt/homebrew/var/openshell/gateway.toml`, with this IPv6 loopback default so Podman can use its separate IPv4 loopback callback listener. The gateway reads `~/.config/openshell/gateway.toml` instead when that file exists. Homebrew preserves existing prefix and user configs during upgrades.
The Homebrew service uses the gateway's built-in `127.0.0.1:17670` listener and generates a local mTLS bundle on install. The installer registers `https://localhost:17670` with the CLI so TLS uses a DNS name covered by the generated certificate. The formula creates a Homebrew prefix config, such as `/opt/homebrew/var/openshell/gateway.toml`, without overriding `bind_address`. Docker Desktop and Podman Machine reuse the primary listener for sandbox callbacks when they can reach it. The gateway reads `~/.config/openshell/gateway.toml` instead when that file exists. Homebrew preserves user-edited prefix and user configs during upgrades; it removes the IPv6 bind only from an unchanged config generated by the affected formula.
The CLI reads the client bundle from `~/.config/openshell/gateways/openshell/mtls/`.
+1 -1
View File
@@ -47,7 +47,7 @@ Set these environment variables before starting the gateway:
For local access, the server certificate must be valid for the endpoint the CLI uses. Include `localhost`, `127.0.0.1`, and `::1` in the certificate SANs when users connect to a local gateway through loopback.
Package-managed local gateways generate this bundle automatically for the `openshell` gateway name. Homebrew uses `https://[::1]:17670` by default; Debian and RPM use `https://127.0.0.1:17670`.
Package-managed local gateways generate this bundle automatically for the `openshell` gateway name. Homebrew registers `https://localhost:17670`; Debian and RPM use `https://127.0.0.1:17670`.
When you register a package-managed local gateway with `openshell gateway add <endpoint> --local --name openshell`, the CLI refreshes its mTLS bundle from the package-managed TLS directory.
On Homebrew, the gateway service also mirrors the Docker sandbox client bundle into `$HOME/.local/state/openshell/homebrew/tls` before startup so Docker Desktop can bind-mount the files into sandbox containers.
+1 -1
View File
@@ -31,7 +31,7 @@ Package-managed gateways do not require a TOML file. Create one at the package's
The Fedora/RHEL RPM template leaves `[openshell.gateway].bind_address` unset. The gateway therefore uses its built-in `127.0.0.1:17670` primary listener. The Podman driver negotiates separate, restricted listeners for sandbox callbacks, so the primary listener does not need a wildcard address. Set `bind_address` explicitly only when clients must reach the primary multiplexed API through another interface.
The Homebrew formula creates its prefix config once with `bind_address = "[::1]:17670"`. Keeping the primary API on IPv6 loopback leaves IPv4 loopback available for Podman Machine's sandbox callback listener. A user config takes precedence, and upgrades do not overwrite either config.
The Homebrew formula creates its prefix config without setting `bind_address`, so the gateway uses its built-in `127.0.0.1:17670` primary listener. Docker Desktop and Podman Machine reuse that listener for sandbox callbacks. A user config takes precedence. Upgrades preserve user-edited configs and migrate only an unchanged prefix config generated with the affected IPv6-loopback default.
## Layout
+14 -11
View File
@@ -113,17 +113,15 @@ It overrides the gateway's configured default runtime class for that sandbox,
while a typed `SandboxTemplate.runtime_class_name` value from the API still
takes precedence.
Docker and Podman callback listeners accept only supervisor callback gRPC
methods. Use the gateway's primary endpoint for CLI, administrator, health,
reflection, inference-route management, and HTTP requests. A
`PermissionDenied` response from one of the sandbox-visible callback addresses
is expected for those requests. The gateway fails startup if a callback
requirement resolves to the exact primary listener address because one socket
cannot preserve both authorization scopes. For the IPv4-loopback callback used
by Podman Machine, set `bind_address = "[::1]:17670"` for the primary listener
and register `https://localhost:17670` as the CLI endpoint. The hostname matches
the generated certificate and avoids the TLS transport error produced by a raw
IPv6-literal endpoint. Do not broaden the primary listener to `0.0.0.0`.
Docker and Podman report the address through which their sandboxes can reach
the gateway. If the primary listener covers that address, the gateway reuses
it and sandbox JWT authentication restricts the supervisor to its callback RPC
allowlist. If the primary listener is not reachable through that address, the
gateway creates an additional callback-only listener. Use the primary endpoint
for CLI, administrator, health, reflection, inference-route management, and
HTTP requests. A `PermissionDenied` response from an additional callback-only
listener is expected for those requests. Do not broaden the primary listener
to `0.0.0.0` solely to make sandbox callbacks reachable.
## Docker Driver
@@ -131,6 +129,11 @@ IPv6-literal endpoint. Do not broaden the primary listener to `0.0.0.0`.
The gateway talks to the Docker daemon to create sandbox containers. Docker is also required for local image builds from directories or Dockerfiles.
Docker Desktop and compatible macOS runtimes route `host.openshell.internal`
through an IPv4 host-gateway alias. The gateway reuses an IPv4 primary listener
that already covers loopback. Otherwise, the Docker driver requests a separate
`127.0.0.1:<gateway-port>` callback-only listener.
For maintainer-level implementation details, refer to the [Docker driver README](https://github.com/NVIDIA/OpenShell/blob/main/crates/openshell-driver-docker/README.md).
Select Docker with `compute_drivers = ["docker"]` in `[openshell.gateway]`. Configure Docker driver values such as `socket_path`, `grpc_endpoint`, `network_name`, `supervisor_bin`, `supervisor_image`, `image_pull_policy`, `ssh_socket_path`, `sandbox_pids_limit`, and `guest_tls_*` in `[openshell.drivers.docker]`. When `socket_path` is unset, the driver uses the same responsive local socket selected by auto-detection. An explicitly selected Docker driver falls back to `/var/run/docker.sock` when no candidate responds.
+1 -1
View File
@@ -470,7 +470,7 @@ detect_platform() {
local_gateway_endpoint() {
case "${PLATFORM:-$(detect_platform)}" in
darwin)
printf 'https://[::1]:%s\n' "$LOCAL_GATEWAY_PORT"
printf 'https://localhost:%s\n' "$LOCAL_GATEWAY_PORT"
;;
*)
printf 'https://127.0.0.1:%s\n' "$LOCAL_GATEWAY_PORT"
+13 -2
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import re
import subprocess
import sys
from pathlib import Path
@@ -54,13 +55,23 @@ def test_generate_homebrew_formula_uses_tagged_macos_driver_asset_without_defaul
assert "OPENSHELL_DRIVERS: " not in formula
assert 'OPENSHELL_GATEWAY_CONFIG: "#{var}/openshell/gateway.toml"' not in formula
assert "init-gateway-config.sh" not in formula
assert 'bind_address = "127.0.0.1:17670"' not in formula
assert 'gateway_config = var/"openshell/gateway.toml"' in formula
assert "unless gateway_config.exist?" in formula
generated_config = re.search(
r"gateway_config_contents = <<~TOML\n(?P<contents>.*?)\n TOML",
formula,
flags=re.DOTALL,
)
assert generated_config is not None
assert "[openshell.gateway]" in generated_config.group("contents")
assert "bind_address =" not in generated_config.group("contents")
assert 'bind_address = "[::1]:17670"' in formula
assert "gateway_config.read == legacy_ipv6_gateway_config_contents" in formula
assert "gateway_config.write gateway_config_contents" in formula
assert '# compute_drivers = ["vm"]' not in formula
assert (
"openshell gateway add https://[::1]:17670 --local --name openshell" in formula
"openshell gateway add https://localhost:17670 --local --name openshell"
in formula
)
assert 'run opt_libexec/"openshell-gateway-homebrew-service"' in formula
assert 'xdg_config_home="${XDG_CONFIG_HOME:-${HOME}/.config}"' in formula
+21 -8
View File
@@ -341,14 +341,27 @@ class Openshell < Formula
system bin/"openshell-gateway", "generate-certs", "--output-dir", var/"openshell/tls", "--server-san", "host.openshell.internal"
gateway_config = var/"openshell/gateway.toml"
unless gateway_config.exist?
gateway_config.write <<~TOML
[openshell]
version = 1
gateway_config_contents = <<~TOML
[openshell]
version = 1
[openshell.gateway]
bind_address = "[::1]:{LOCAL_GATEWAY_PORT}"
TOML
[openshell.gateway]
TOML
legacy_ipv6_gateway_config_contents = <<~TOML
[openshell]
version = 1
[openshell.gateway]
bind_address = "[::1]:{LOCAL_GATEWAY_PORT}"
TOML
unless gateway_config.exist?
gateway_config.write gateway_config_contents
else
# Migrate only the exact config generated by the affected formula. Keep
# any user-edited config untouched.
if gateway_config.read == legacy_ipv6_gateway_config_contents
gateway_config.write gateway_config_contents
end
end
entitlements = var/"openshell/openshell-driver-vm.entitlements.plist"
@@ -379,7 +392,7 @@ class Openshell < Formula
brew services restart openshell
Register it with the OpenShell CLI:
openshell gateway add https://[::1]:{LOCAL_GATEWAY_PORT} --local --name openshell
openshell gateway add https://localhost:{LOCAL_GATEWAY_PORT} --local --name openshell
EOS
end
+2 -2
View File
@@ -100,8 +100,8 @@ assert_glibc_preflight_fails \
"OpenShell Linux packages require glibc >= 2.28; detected musl or unsupported libc." \
setup_ldd_musl
if [ "$(PLATFORM=darwin local_gateway_endpoint)" != "https://[::1]:17670" ]; then
echo "FAIL: macOS local gateway endpoint must use IPv6 loopback" >&2
if [ "$(PLATFORM=darwin local_gateway_endpoint)" != "https://localhost:17670" ]; then
echo "FAIL: macOS local gateway endpoint must use a TLS-compatible loopback hostname" >&2
exit 1
fi