Jason T. GreeneandMrunal Patel
7691f88e0e
* perf(server): enable WAL and NORMAL sync for the SQLite store On-disk SQLite stores ran with sqlx defaults: rollback journal (`journal_mode=delete`) and `synchronous=FULL`. Every autocommit write paid several fsyncs and blocked readers while it held the lock, so gateway hot paths made of many small writes serialized on disk latency. The clearest case is `openshell forward service`, which mints and revokes an SSH session token around every forwarded TCP connection: two commits per connection, tens of milliseconds each on a virtual disk, wall clock linear in the number of concurrent connections, and enough queueing that bursts hit the per-sandbox connection cap and get refused. Switch on-disk databases to WAL with `synchronous=NORMAL`. The mode change runs once on a single connection before the pool opens: entering WAL needs exclusive access to the file, so doing it up front means pool connections only ever re-apply the pragma to a file already in WAL mode, and a failure surfaces as one clear connect error. The first start after upgrading an existing database therefore needs the file to be otherwise unopened. `synchronous` is applied through the connect options on every pooled connection. In-memory databases keep their defaults. A crash can now roll back the most recent transactions without corrupting the database, which is the standard WAL trade-off and fits the single-node scope of the SQLite backend. Tests cover a fresh store, an existing rollback-journal file that must be switched on connect, sidecar permissions, and concurrent readers under a burst of insert-then-update writes. Architecture, configuration and Helm docs describe the durability trade-off, the sidecar files, and the local filesystem requirement. Signed-off-by: Jason T. Greene <jason.greene@redhat.com> * fix(server): keep SQLite commits durable except SSH session issuance WAL with synchronous=NORMAL can roll back acknowledged commits after a power loss or kernel crash, including SSH session revocations and other authorization-tightening writes. Run the main pool with synchronous=FULL so every acknowledged write is durable; in WAL mode that is a single fsync of the WAL per commit. Add Store::create_relaxed for inserts that are safe to lose, and use it only for SSH session issuance: a dropped token just fails validation. On file-backed SQLite it runs on a dedicated single-connection pool with synchronous=NORMAL. Both pools share one WAL, so the next FULL commit also makes earlier relaxed commits durable. Postgres treats it as an ordinary durable MustCreate insert. Signed-off-by: Mrunal Patel <mrunalp@gmail.com> --------- Signed-off-by: Jason T. Greene <jason.greene@redhat.com> Signed-off-by: Mrunal Patel <mrunalp@gmail.com> Co-authored-by: Mrunal Patel <mrunalp@gmail.com>
11 KiB
OpenShell Gateway Configuration (RPM)
Configuration reference for the OpenShell gateway when installed via the RPM package on Fedora and RHEL systems.
For first-time setup, see QUICKSTART.md. For troubleshooting, see TROUBLESHOOTING.md.
Default configuration
The RPM ships a default TOML configuration template at
/usr/share/openshell-gateway/gateway.toml.default. On first start of
openshell-gateway.service, the systemd unit copies this template to
~/.config/openshell/gateway.toml if no config file exists there yet.
The defaults are tuned for rootless Podman use:
[openshell]
version = 2
[openshell.gateway]
compute_driver = "podman"
The RPM does not override bind_address. The primary listener uses the
built-in 127.0.0.1:17670 default. Host-networked Podman supervisors connect
to this same loopback listener, so the gateway does not expose another host
interface.
compute_driver = "podman" pins the compute driver to Podman. Without
this, the gateway auto-detects in order: Kubernetes, Podman, Docker. Pinning
prevents unexpected driver selection if Docker is also installed on the host.
Customizing the configuration
Edit ~/.config/openshell/gateway.toml directly. The package-owned template at
/usr/share/openshell-gateway/gateway.toml.default is not read at runtime and
may change during an RPM upgrade. The active user copy is preserved. During a
schema-v2 upgrade, the service replaces only the recognized package-generated
v1 copy; it never rewrites an edited configuration. Before generating
certificates or starting the gateway, the service runs
openshell-gateway config preflight against the effective configuration and
stops if validation fails.
To apply environment variable overrides that persist across upgrades without
editing the TOML file, add them to ~/.config/openshell/gateway.env:
# Example: explicitly expose the primary listener on one host interface
OPENSHELL_BIND_ADDRESS=192.168.1.10
To override the path to the TOML config file entirely:
# In ~/.config/openshell/gateway.env
OPENSHELL_GATEWAY_CONFIG=/path/to/custom/gateway.toml
For one-off service overrides that persist across package upgrades:
systemctl --user edit openshell-gateway
TLS (mTLS)
The RPM enables mutual TLS by default. The gateway requires a valid
client certificate for all API connections. Its primary listener uses
127.0.0.1:17670; Podman supervisor sessions use that same listener.
Auto-generated certificates
On first start, the systemd user service runs
openshell-gateway generate-certs --output-dir ~/.local/state/openshell/tls --server-san host.openshell.internal
to generate certificates with rcgen (the same routine the CLI uses for
local mTLS bundles). The unit sets OPENSHELL_LOCAL_TLS_DIR to that path and
uses the same value for certificate generation and gateway startup. To use a
custom bundle location, set OPENSHELL_LOCAL_TLS_DIR in
~/.config/openshell/gateway.env before starting the service.
| File | Purpose | Location |
|---|---|---|
| CA certificate | Root of trust | ~/.local/state/openshell/tls/ca.crt |
| CA private key | Signs server and client certs | ~/.local/state/openshell/tls/ca.key |
| Server certificate | Gateway TLS identity | ~/.local/state/openshell/tls/server/tls.crt |
| Server private key | Gateway TLS key | ~/.local/state/openshell/tls/server/tls.key |
| Client certificate | CLI and sandbox identity | ~/.local/state/openshell/tls/client/tls.crt |
| Client private key | CLI and sandbox key | ~/.local/state/openshell/tls/client/tls.key |
Client certificates are also copied to the CLI auto-discovery directory:
~/.config/openshell/gateways/openshell/mtls/
ca.crt
tls.crt
tls.key
The CLI automatically discovers these certificates when connecting to a
gateway on localhost or 127.0.0.1.
Server certificate SANs
The auto-generated server certificate includes these Subject Alternative Names:
localhostopenshellopenshell.openshell.svcopenshell.openshell.svc.cluster.localhost.containers.internalhost.docker.internalhost.openshell.internal127.0.0.1
To connect from a remote machine, you need externally-managed certificates with additional SANs. See "Remote CLI access" in TROUBLESHOOTING.md.
Using externally-managed certificates
To use certificates from an external CA or cert-manager:
-
Place the server cert, key, and CA cert on the filesystem.
-
Edit
~/.config/openshell/gateway.toml:[openshell.gateway.tls] cert_path = "/path/to/server/tls.crt" key_path = "/path/to/server/tls.key" client_ca_path = "/path/to/ca.crt" -
Place the client cert where the CLI expects it:
~/.config/openshell/gateways/openshell/mtls/ ca.crt tls.crt tls.key
Rotating certificates
Delete the TLS state directory and restart the gateway:
rm -rf ~/.local/state/openshell/tls
systemctl --user restart openshell-gateway
The gateway regenerates the PKI on next start.
Disabling TLS
WARNING: With TLS disabled, the gateway API has no authentication. Keep the bind address on
127.0.0.1, or place the gateway behind a TLS-terminating reverse proxy that enforces its own authentication.
To disable TLS (not recommended for production):
-
Edit
~/.config/openshell/gateway.toml:[openshell.gateway] disable_tls = true -
Remove or comment out the
guest_tls_*entries in~/.config/openshell/gateway.tomlif they are set. -
Restart the gateway.
Sandbox TLS
When mTLS is enabled, the Podman driver bind-mounts the client certificates into each sandbox container so the supervisor process can establish an mTLS connection back to the gateway.
The following TOML fields control the host-side paths of the client certificates that are mounted into sandbox containers:
[openshell.gateway]
guest_tls_ca = "/home/user/.local/state/openshell/tls/ca.crt"
guest_tls_cert = "/home/user/.local/state/openshell/tls/client/tls.crt"
guest_tls_key = "/home/user/.local/state/openshell/tls/client/tls.key"
Inside the container, the supervisor reads them from:
/etc/openshell/tls/client/ca.crt/etc/openshell/tls/client/tls.crt/etc/openshell/tls/client/tls.key
On SELinux-enabled systems, the Podman driver automatically applies the
:z relabel option to these bind mounts. No manual SELinux
configuration is required.
Configuration reference
Upgrading from a previous release? See the "Migrating from gateway.env" section in TROUBLESHOOTING.md for the env-to-TOML mapping and notes on the default port, bind address, and database path changes.
Gateway and driver settings have local runtime defaults. The gateway reads
~/.config/openshell/gateway.toml when that file exists. Set
OPENSHELL_GATEWAY_CONFIG in the launch environment to use a different file.
Use systemctl --user edit openshell-gateway for service environment
overrides that persist across package upgrades.
Gateway settings
| TOML option | Default | Description |
|---|---|---|
bind_address |
127.0.0.1:17670 (gateway default) |
Address for the primary gRPC/HTTP API listener. |
compute_driver |
"podman" (RPM default) |
When unset, the gateway auto-detects Kubernetes, then Podman, then Docker. The RPM default pins to Podman; legacy compute_drivers lists are rejected. |
[openshell.drivers.podman].default_image |
nvcr.io/nvidia/base/ubuntu:24.04 |
Default sandbox image. |
[openshell.drivers.podman].sandbox_runtime_image |
ghcr.io/nvidia/openshell/sandbox:latest |
Static musl sandbox runtime image mounted into Podman workloads. |
[openshell.drivers.podman].supervisor_image |
ghcr.io/nvidia/openshell/supervisor:latest |
Dynamic glibc supervisor image used outside the workload. |
[openshell.gateway].guest_tls_ca, guest_tls_cert, guest_tls_key |
auto-generated paths | Gateway-owned client TLS material injected into the selected local driver and mounted into sandbox containers. |
[openshell.gateway.tls] paths |
auto-generated paths | Server TLS certificate, key, and client CA. |
disable_tls |
unset | Set to true to disable TLS. |
The database URL is not accepted in TOML. When OPENSHELL_DB_URL is unset,
the gateway uses sqlite:$XDG_STATE_HOME/openshell/gateway/openshell.db.
The SQLite database runs in WAL mode with synchronous=FULL (SSH session
issuance alone uses NORMAL), so
openshell.db-wal and openshell.db-shm sit next to it and must be kept
together with it; back it up with sqlite3 openshell.db ".backup <copy>".
Driver TOML settings
Create ~/.config/openshell/gateway.toml when you need to customize driver
settings:
[openshell]
version = 2
[openshell.gateway]
compute_driver = "podman"
[openshell.drivers.podman]
network_name = "openshell"
default_image = "nvcr.io/nvidia/base/ubuntu:24.04"
image_pull_policy = "if_not_present"
health_check_interval_secs = 10
stop_timeout_secs = 10
Image management
The gateway pulls container images automatically on first sandbox
creation. The default pull policy is if_not_present, which means images are
pulled once and then cached by Podman.
To update cached images:
podman pull ghcr.io/nvidia/openshell/supervisor:latest
podman pull nvcr.io/nvidia/base/ubuntu:24.04
Or set image_pull_policy = "always" in
[openshell.drivers.podman] to pull on every sandbox creation.
To pin specific image versions instead of :latest, set these values in
[openshell.drivers.podman]:
sandbox_runtime_image = "ghcr.io/nvidia/openshell/sandbox:v0.0.37"
supervisor_image = "ghcr.io/nvidia/openshell/supervisor:v0.0.37"
default_image = "nvcr.io/nvidia/base/ubuntu:24.04"
For air-gapped environments:
-
On a connected machine, pull and save the images:
podman pull ghcr.io/nvidia/openshell/supervisor:latest podman pull nvcr.io/nvidia/base/ubuntu:24.04 podman save -o supervisor.tar ghcr.io/nvidia/openshell/supervisor:latest podman save -o sandbox.tar nvcr.io/nvidia/base/ubuntu:24.04 -
Transfer the tarballs to the air-gapped host and load them:
podman load -i supervisor.tar podman load -i sandbox.tar -
Set pull policy to
never:[openshell.drivers.podman] image_pull_policy = "never"
File locations
| Purpose | Path |
|---|---|
| Gateway binary | /usr/bin/openshell-gateway |
| CLI binary | /usr/bin/openshell |
| Systemd user unit | /usr/lib/systemd/user/openshell-gateway.service |
| Default TOML config template (read-only) | /usr/share/openshell-gateway/gateway.toml.default |
| Active gateway TOML configuration | ~/.config/openshell/gateway.toml |
| Optional environment variable overrides | ~/.config/openshell/gateway.env |
| TLS certificates | ~/.local/state/openshell/tls/ |
| CLI client certs | ~/.config/openshell/gateways/openshell/mtls/ |
| Gateway database | ~/.local/state/openshell/gateway/openshell.db |