mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-19 05:50:41 +08:00
pull-request/3521
21
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oliver Calder
|
6e865df349 |
feat(snap): ship the standalone prover binary in the snap (#3717)
Signed-off-by: Oliver Calder <oliver.calder@canonical.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Oliver Calder
|
1ad4e428a6 |
fix(snap): simplify snap hooks (#3988)
* fix(snap): simplify snap hooks
The `post-refresh` hook runs after initial snap installation as well, so
there is no need to call the `install` hook from within the
`post-refresh` hook; instead, the logic can simply be moved into the
`post-refresh` hook directly, and the `install` hook removed.
Also, the existing `install` hook logic looked for an insecure
configuration, and if found, replaced the entire configuration file with
a minimal default in the current format. But OpenShell does that default
behavior without any config file, so we may as well simply remove the
configuration file entirely to keep up-to-date with the current default
behavior. Let OpenShell create a configuration file if it needs to,
rather than auto-create one via the packaging scripts.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
* fix(snap): remove the connect-plug-docker hook
The `openshell:docker` is auto-connected to the system `:docker` slot,
so there should not be a need to separately restart the gateway service
when the interface is connected.
For locally-built test snaps which were not published to the store, the
autoconnection is not made, but when the snap is installed, the gateway
will attempt to start anyway and fail to find any available compute
driver, so quickly restart until it hits the systemd start-limit, after
which systemd prevents the service from being started again. If a user
tries to manually connect their locally-built `openshell` snap to the
`:docker` slot, then the `connect-plug-docker` hook runs and triggers a
restart of the gateway, which will usually fail because the start limit
has already been hit. An error in the hook will thus cause the interface
connection to be undone, which is undesirable.
Thus, we can remove this hook entirely, and instead allow interface
connections to succeed as intended. The user still needs to manually
restart the gateway service after making a manual connection (as was the
case previously) and probably needs to `systemctl reset-failed` first,
but at least connection will succeed beforehand so they can proceed with
these steps.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
* fix(snap): set refresh-mode: endure again, with manual restart
Return to the previous behavior before commit
|
||
|
Drew Newberry
|
a67567e583 |
fix(snap): require mTLS for the snap gateway (#3726)
* fix(snap): require mTLS for the snap gateway Replace the installer opt-in with an authenticated snap gateway. The wrapper no longer forces plaintext, so the gateway serves TLS from the bundle it already generates in $SNAP_COMMON/tls. The install hook writes a config that enables mTLS user auth instead of unauthenticated access, and a new post-refresh hook migrates the exact legacy default on existing installs. install.sh waits for the gateway, detects whether it serves TLS, copies the client bundle into the target user's snap state directory, and registers the gateway over HTTPS. Older plaintext snap revisions still register over HTTP with a warning. The release canary asserts mTLS auth and HTTPS registration. Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(snap): pass config preflight and detect the mTLS gateway reliably An explicit [openshell.gateway.mtls_auth] table fails config preflight, which validates mTLS auth before the local TLS bundle supplies the client CA. Write a default that pins the Docker driver instead; with the wrapper's TLS bundle the gateway requires client certificates and enables mTLS user auth automatically, as the native packages do. The mTLS gateway rejects TLS handshakes without a client certificate, and it still answers plaintext loopback HTTP for sandbox service routing, so the installer could misdetect it as a legacy plaintext gateway. Probe HTTPS with the root-owned client bundle, and treat a gateway as legacy only when a plaintext gRPC Health call succeeds. Signed-off-by: Drew Newberry <anewberry@nvidia.com> * chore(snap): simplify install hook comment Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(snap): migrate insecure gateway configs on refresh Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(snap): simplify mTLS detection and config migration Detect the mTLS snap from the installed revision's post-refresh hook instead of probing plaintext gRPC, and drop the scheme global. Remove the installer's pre-hook config fallback, which is dead now that every channel ships the install hook and which wrote the insecure default. Give the install hook a single write path with a simple backup name, and shorten the manual client certificate steps. Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(snap): stop keeping a copy of replaced insecure configs Signed-off-by: Drew Newberry <anewberry@nvidia.com> * feat(install): make the snap an opt-in install method Stop selecting the OpenShell snap just because the snap command exists. Linux installs default to the Debian or RPM package; OPENSHELL_INSTALL_METHOD=snap (or deb, rpm) selects the package explicitly. Hosts that already have the OpenShell snap keep refreshing it rather than gaining a second gateway on the same port. The release canary and snap repro script opt in explicitly. Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(snap): let the gateway auto-detect its compute driver Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(snap): restart the gateway after refresh Published revisions use refresh-mode: endure, and snapd honors the old revision's setting during a refresh, so the plaintext gateway kept running with the migrated config unused until a manual restart. Restart the gateway from the post-refresh hook so the mTLS config takes effect immediately. Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(snap): drop refresh notes from the snap description Signed-off-by: Drew Newberry <anewberry@nvidia.com> * docs(snap): trim snap refresh notes from installation docs Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Oliver Calder
|
6c864ec9ab |
fix(install): avoid installing incompatible docker snap (#3666)
* fix(install): avoid installing incompatible docker snap The work to land RFC-0012 added new restrictions when interacting with Docker by setting `NoNewPrivs`. This prevents the `docker` snap from transitioning its AppArmor profile from `snap.docker.dockerd` to `docker-default` when it tries to launch a container. Thus, the `docker` snap is currently incompatible with OpenShell. This commit prevents `install.sh` from installing the `docker` snap before installing the `openshell` snap, and instead requires the user to install a non-snap Docker daemon before proceeding with installing the snap. Systems without the `snap` command are unaffected, since they install native packages without checking for the presence of Docker or other compute providers. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): only snapd 2.76 for openshell snap since store installs work Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fixup! fix(install): avoid installing incompatible docker snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fixup! fix(snap): only snapd 2.76 for openshell snap since store installs work Signed-off-by: Oliver Calder <oliver.calder@canonical.com> --------- Signed-off-by: Oliver Calder <oliver.calder@canonical.com> |
||
|
Oliver Calder
|
e60098d748 |
fix(snap): install openshell snap via install.sh when snap available (#3656)
* fix(snap): update stale snap docs and tests Previously, the `openshell` snap required the `docker` snap. Now, it works with any Docker daemon running on the system. Furthermore, the `snap-declaration` assertion on the `openshell` snap when installed from the Snap Store causes the `openshell` snap to always connect to the system `:docker` slot, rather than a slot provided by the `docker` snap. This commit updates the documentation, including the `description` field in `snapcraft.yaml`, to ensure that all information is correct and up-to-date. Additionally, some tests connected the `openshell:docker` plug to the `docker` snap's `docker:docker-daemon` slot, which is inconsistent with how the `openshell` snap operates when installed from the store. Update those tests to connect to the system `:docker` slot as well. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): require snapd 2.76 for openshell snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(nix): align snap gateway reproducer timeout with release-canary Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): require snapd 2.77 for openshell snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fixup! fix(snap): require snapd 2.77 for openshell snap Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * feat(snap): install openshell snap via install.sh when snap available Change `install.sh` to install the `openshell` snap by default when snapd is installed on the host. This installs the snap from the `latest/stable` channel, which should match the most up-to-date release tag on github. If `OPENSHELL_VERSION=dev` is set for `install.sh`, then it will install the `openshell` snap from the `latest/edge` channel, which matches the latest dev release available on github. The `openshell` snap currently requires Docker in order to function. If a Docker daemon is already installed on the system, it will be used by the `openshell` snap. Otherwise, `install.sh` will install the `docker` snap first, wait for the Docker daemon to be ready, and then install the `openshell` snap. Also, update the `release-canary.yml` to split the `ubuntu-snap` job into `ubuntu-snap-system-docker` and `ubuntu-snap-provisions-docker`, which test the two aforementioned scenarios. Previously, `ubuntu-snap` manually installed a given snap artifact as built from CI, but with these new jobs, it instead uses `install.sh` to install the published `openshell` snap from the `latest/edge` track, thus matching the behavior of the other release canary jobs. Make corresponding changes to the `nix` guest reproducer, documentation, and tests. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(snap): configure local gateway authentication The `openshell` snap runs the gateway as a systemd system service, which runs as root. Thus, the mTLS certs are generated by root and stored in a root-owned directory to which non-root users do not have access. For this reason, the snap's `openshell-gateway-wrapper` script sets `OPENSHELL_DISABLE_TLS=true`. This commit ensures that the `openshell` snap's gateway allows unauthenticated local access by writing a default `gateway.toml` configuration file during the install hook, which runs after the snap is first installed but before services are started. The config file contains sets `allow_unauthenticated_users = true`. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fix(install): configure snap gateway authentication Recently, a new install hook was added which writes a default config file for the `openshell` snap to allow unauthenticated local access to the gateway. This is because the gateway service runs as root and the mTLS certificates are not accessible to non-root users. However, the `install.sh` script installs the `openshell` snap from the snap store, and the published version may not yet have that new install hook. Or, the user may already have the snap installed, in which case the install hook does not run. In either case, we need `install.sh` to ensure that the config file is written to set the gateway auth to `allow_unauthenticated_users = true`, and then restart the openshell gateway service. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> * fixup! feat(snap): install openshell snap via install.sh when snap available Signed-off-by: Oliver Calder <oliver.calder@canonical.com> --------- Signed-off-by: Oliver Calder <oliver.calder@canonical.com> |
||
|
Drew Newberry
|
17ce738bfb |
fix(ci)!: remove gateway callback listener dependency (#3365)
* fix(ci): repair post-merge release canary Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(packaging): bootstrap canary runtime prerequisites Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci(canary): collect macOS VM diagnostics Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci(canary): pin libkrun-compatible macOS runner Signed-off-by: Drew Newberry <anewberry@nvidia.com> * ci(canary): limit macOS smoke test to package startup Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(compute)!: remove gateway callback listeners Run Docker supervisors on host networking so they use the operator-configured primary gateway endpoint. Remove the unused compute-driver callback listener negotiation and listener-scoped routing machinery. BREAKING CHANGE: The ComputeDriver API no longer exposes GetGatewayListenerRequirements or GatewayListenerRequirement. External drivers must regenerate bindings and connect supervisors to the configured primary gateway endpoint. Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(docker): use sandbox runtime image in launcher Signed-off-by: Drew Newberry <anewberry@nvidia.com> * test(podman): exercise production endpoint selection Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(e2e): route supervisors to reachable gateways Signed-off-by: Drew Newberry <anewberry@nvidia.com> * test(e2e): align Podman endpoint fixtures Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(docker): preserve host aliases for supervisors Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(docker): align sandbox host gateway pin Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(e2e): address Docker fixtures by bridge IP Signed-off-by: Drew Newberry <anewberry@nvidia.com> * test(e2e): serialize sandbox lifecycle cases Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(e2e): host Docker TCP fixture with gateway Signed-off-by: Drew Newberry <anewberry@nvidia.com> * fix(e2e): use loopback for host-network supervisor Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Drew Newberry <anewberry@nvidia.com> |
||
|
Jesse JaggarsandDrew Newberry
|
02b664bb0d |
refactor(config): normalize and enforce gateway schema v2 (#2814)
* refactor(config): normalize compute driver field names Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(config): introduce canonical gateway fields Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * refactor(config): enforce gateway schema version 2 Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): preserve compute driver runtime guarantees Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): address schema v2 review regressions Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): complete schema v2 migration safeguards Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(config): expand schema v2 regression coverage Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(config): add schema v2 parity manifest Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): correct parity manifest inventory Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs(config): record schema v2 intentional changes Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs(config): disposition schema v2 parity gaps Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): add dual schema parity harness Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): establish compute lifecycle parity baseline Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): preserve gateway option compatibility Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): record gateway option parity Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs(config): close gateway-wide parity gaps Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(podman): apply configured pids limit Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): validate Podman option parity Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): add Kubernetes option parity harness Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): record Kubernetes option parity Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): disposition VM parity lanes Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): add external driver parity lane Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(e2e): preserve external driver pull policy Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): attest parity artifacts and launches Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): require clean parity build sources Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): bind parity runtime artifacts Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(e2e): use isolated supervisor tags Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(e2e): qualify parity image tags Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(e2e): serve parity supervisor locally Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): isolate parity podman services Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): harden parity evidence provenance Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): pin parity sandbox artifacts Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): attest parity runtime inputs Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): bind parity runtime evidence Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): record compute boundary parity Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(e2e): disposition cross-cutting parity lanes Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(packaging): preflight gateway config upgrades Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): preserve rebase integration guarantees Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(ci): isolate temporary git signing config Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): update remaining schema v2 consumers Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(ci): provide e2fs tools to VM tests Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): align preflight with gateway startup Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(vm): preserve rootfs tar configuration Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * chore(config): adopt duration unit constructors Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(packaging): preflight RPM gateway config Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(config): address driver review findings Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(e2e): require fresh semantic parity evidence Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(docker): update tests for renamed sandbox label Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * test(gateway): preserve selective driver coverage after rebase Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> Signed-off-by: Drew Newberry <anewberry@nvidia.com> Co-authored-by: Drew Newberry <anewberry@nvidia.com> |
||
|
John T. Myers
|
f4dc6be4b2 |
refactor(inference): remove managed inference routes (#3195)
* refactor(inference): remove managed inference routes Closes #3172 Remove the inference route control plane, inference.local data path, built-in router crate, and SDK surface. Move inference workloads to explicitly imported provider profiles and native endpoints, with migration cleanup and updated tests and documentation. Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> * fix(policy): preserve alternate upstream isolation Restore the provider policy activation guard so legacy OpenAI and Anthropic providers configured for alternate base URLs do not grant egress to the built-in public vendor endpoints. Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> --------- Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com> |
||
|
Evan Lezar
|
8d7db25403 |
fix(snap): recover gateway after Docker connection (#2866)
Signed-off-by: Evan Lezar <elezar@nvidia.com> |
||
|
Oliver Calder
|
994750e3a1 |
feat(snap): vendor ssh in openshell snap and remove ssh-keys interface (#2280)
Previously, the openshell snap used the ssh-keys interface to get access to the host's ssh binary, which is used for sandbox connect/exec/forward. However, ssh-keys is a privileged interface which also grants access to the public and private ssh keys on the host. As such, it required manual connection in order to be used. This weakened the security sandbox of the snap, and hurt the UX of installing it. This commit changes this by removing the `ssh-keys` interface and instead vendoring the `ssh` binary within the snap. This is safe because OpenShell always invokes the `ssh` binary with `StrictHostKeyChecking=no`, `UserKnownHostsFile=/dev/null`, and `GlobalKnownHostsFile=/dev/null`, and never uses any host credentials or ssh configuration. Openshell only ever access to `~/.ssh/config` to write OpenShell-managed aliases, and this can safely live within the snap sandbox, rather than leaking into the host environment. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> |
||
|
Piotr Mlocek
|
e4d7d41656 | fix(snap): use snap-owned XDG directories (#1972) | ||
|
Zygmunt Krynicki
|
713d46c54e |
feat(snap): expand snap description with setup instructions (#1695)
Add a description covering the snap elements, step-by-step setup for Docker and interface connections, gateway verification commands, and links to documentation and source code. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com> |
||
|
Zygmunt Krynicki
|
d2a522dd34 |
feat(snap): switch to prebuilt binaries shared with other packages (#1651)
Switch the snapcraft build from 'source: .' (which copies the full 20 GB repo including 18 GB of target/ and build artifacts into LXD) to 'source: ./snap/prebuilt' (~130 MB of only the binaries and metadata that snapcraft needs). The snap-package workflow now downloads the same cli, gateway, and supervisor binaries that other packaging jobs (deb, rpm, tarballs) use. A new 'Prepare snap build directory' step assembles snap/prebuilt/ with the 3 binaries, gateway wrapper script, LICENSE, README.md, and a precomputed version string — everything snapcraft needs without any source tree access. This addresses CI/CD Improvements (issue #1361) by reducing data transferred into the LXD sandbox by ~150x and eliminating the need for snapcraft to traverse the entire repository tree. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com> |
||
|
Zygmunt Krynicki
|
4025894a94 | chore(snap): remove early snap packaging (#1648) | ||
|
Zygmunt Krynicki
|
19be5682d5 |
feat(snap): add openshell.term desktop app (#1693)
Add a desktop launcher for the OpenShell TUI so users can launch
"openshell term" from their desktop environment application menu.
The change adds three files:
- snap/local/term.desktop: desktop entry file for the application launcher
- snap/local/icon.png: application icon (copied from snap store data)
- snapcraft.yaml: new "term" app entry that runs "openshell term"
with home, network, ssh-keys, and system-observe plugs, plus install
rules to stage the desktop file and icon under meta/gui/
The desktop file references the icon via ${SNAP} which is resolved
at runtime to the snap installation directory. The term app reuses
the same connection plugs as the main openshell app.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
|
||
|
Alexander Watson
|
e98ea3ee93 | feat(policy): add agentic approval loop (#1528) | ||
|
Drew Newberry
|
6c7950da90 | ci(snap): add snap release pipeline (#1600) | ||
|
Zygmunt Krynicki
|
0a8b35c31f |
fix(build): install binaries built in part build tree (#1462)
The earlier build only worked if the tree had pre-compiled binaries left over from non-snap build. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com> |
||
|
Drew Newberry
|
f257ed0193 |
refactor(packaging): rely on gateway runtime defaults (#1415)
* fix(packaging): use gateway TOML config in packages * refactor(packaging): rely on gateway runtime defaults |
||
|
Taylor MutchandDrew Newberry
|
b61a98dbad |
feat(gateway): add TOML configuration file (RFC 0003) (#1317)
* feat(gateway): add TOML configuration file (RFC 0003)
Introduces an opt-in --config / OPENSHELL_GATEWAY_CONFIG flag that loads a
TOML file with gateway-wide settings and per-driver tables. Source
precedence is CLI > env > file > built-in default, implemented via clap's
ValueSource so existing flags and env vars keep their priority.
Driver crates (kubernetes, docker, podman, vm) now derive Deserialize on
their config structs. SupervisorSideloadMethod gains Deserialize with
kebab-case rename. A per-driver inheritance allowlist on the loader side
overlays [openshell.gateway] shared defaults (default_image,
supervisor_image, image_pull_policy, guest_tls_*, ssh_handshake_skew_secs,
client_tls_secret_name, host_gateway_ip, enable_user_namespaces) onto
each [openshell.drivers.<name>] table before deserialization.
The Helm chart renders a new gateway-config ConfigMap and mounts it at
/etc/openshell/gateway.toml. The migrated OPENSHELL_* env entries are
dropped from the StatefulSet — only the Secret-backed
OPENSHELL_SSH_HANDSHAKE_SECRET remains. database_url stays on --db-url.
Adds examples/gateway/gateway.example.toml and updates architecture/gateway.md
with the source precedence and inheritance rules.
* docs(gateway): drop ssh_handshake_skew_secs and ssh_handshake_secret from examples
Both fields are scheduled for removal. Remove the example values and the
env-only note so the gateway.toml example and the architecture doc stop
recommending settings that will not exist much longer.
* docs(rfc): correct OPENSHELL_CONFIG to OPENSHELL_GATEWAY_CONFIG in RFC 0003
* docs(gateway): add per-driver TOML example configurations
Adds focused single-driver examples next to the comprehensive
gateway.example.toml: kubernetes, docker, podman, and microvm. Each one
demonstrates the realistic settings for that driver plus how shared
[openshell.gateway] defaults inherit into the driver table.
A new unit test (`checked_in_examples_parse`) loads every example through
the config_file loader so schema drift fails CI rather than silently
shipping a broken example.
* refactor(gateway): drop image_pull_policy from shared inheritance
Kubernetes and Podman use mutually-incompatible vocabularies for the same
TOML key:
- Kubernetes: `Always | IfNotPresent | Never` (free-form string passed
verbatim to the K8s API).
- Podman: `always | missing | never | newer` (strict lowercase enum
deserialised into `ImagePullPolicy`).
No value means the same thing in both drivers. Sharing the key at
`[openshell.gateway]` scope and inheriting it into every active driver's
table meant any value safe for one driver was either wrong or silently
dropped for the other (`IfNotPresent` → `ImagePullPolicy::Missing` after
`.unwrap_or_default()`). Operators run one driver per gateway, so the
"shared default" never pays for itself.
Make `image_pull_policy` driver-local:
- Remove the field from `GatewayFileSection` and from
`inheritable_keys()` for both Kubernetes and Podman.
- Drop the file→`RunArgs` merge for the gateway-scope key.
- Stop unconditionally clobbering the driver value with
`config.sandbox_image_pull_policy` in the runtime wiring — only apply
the CLI/env override when it was set (and, for Podman, only when it
parses into the lowercase enum).
- Move the key under `[openshell.drivers.kubernetes]` and
`[openshell.drivers.podman]` in every example, the RFC, the
architecture doc, and the Helm-rendered gateway ConfigMap.
The supervisor pull policy follows the same shape: it is K8s-only and
moves into `[openshell.drivers.kubernetes]` alongside `image_pull_policy`
in the Helm template.
* fix(gateway): address review feedback on TOML configuration
Resolves the P1 and P2 issues raised in PR #1317:
- Helm gateway ConfigMap moves `grpc_endpoint` under
`[openshell.drivers.kubernetes]` so the default install no longer fails
the gateway's `deny_unknown_fields` schema check.
- `kubernetes_config_from_file` and `podman_config_from_file` only let
the gateway-wide CLI/env `grpc_endpoint` overwrite the driver-table
value when it was actually supplied, preserving file-only configs.
- Kubernetes driver default `image_pull_policy` is now empty (was Podman
vocabulary "missing"), so default deployments let the Kubernetes API
apply its own policy instead of being rejected.
- New `disable_tls` gateway field plumbs `.Values.server.disableTls`
through the TOML ConfigMap instead of relying on env vars dropped from
the StatefulSet.
- StatefulSet pod template now carries a `checksum/gateway-config`
annotation so `helm upgrade` rolls pods when the ConfigMap changes.
- Auxiliary listener resolution preserves the full `SocketAddr` from
`health_bind_address` / `metrics_bind_address`, so a loopback-pinned
health port is not silently relocated onto the public bind address.
- `ssh_session_ttl_secs` from the file is now applied to `Config` (it
was previously accepted by the loader but never read).
New regression coverage: cli-level merge tests for the new fields plus
helm-unittest assertions for the ConfigMap shape, checksum annotation,
and `disable_tls` rendering.
* docs(gateway): consolidate gateway TOML examples into docs reference
Replaces the per-driver example files under examples/gateway/ with a
single published reference page at docs/reference/gateway-config.mdx
covering source precedence, layout, the full example, and the four
per-driver examples (Kubernetes, Docker, Podman, microVM). Drew flagged
during PR #1317 review that the examples belong with the user-facing
docs rather than in a sibling examples/ directory.
The cross-references in architecture/gateway.md and RFC 0003 are updated
to point at the new docs page; the round-trip test in config_file.rs is
removed (schema coverage stays on the inline parses_full_example test
and per-field merge tests — doc-snippet drift belongs in a separate
docs-lint, not in a cross-tree Rust unit test).
* refactor(core): move DEFAULT_K8S_NAMESPACE into K8s driver
The constant is Kubernetes-specific (used only by KubernetesComputeConfig's
Default impl) and does not belong in openshell-core. Relocate it to the
driver crate that owns the K8s vocabulary; openshell-core retains only
truly cross-cutting defaults.
* refactor(core): move Podman bridge default into Podman driver
DEFAULT_NETWORK_NAME is Podman vocabulary, consumed only by the Podman
driver. Also drops the unused DEFAULT_IMAGE_PULL_POLICY constant.
* docs(auth): scrub remaining SSH handshake secret references
Sweeps the trailing mentions left after the rebase: the gateway
config-file module doc, the Helm gateway-config ConfigMap header,
the gateway-config.mdx env-only note, and the RPM systemd unit
comment for init-gateway-env.sh.
* docs(gateway): clarify OPENSHELL_GRPC_ENDPOINT applies to all drivers
The previous comment implied the callback endpoint was Kubernetes-only,
but the value is propagated to every compute driver (Kubernetes, Docker,
Podman, VM) and must be reachable from wherever the sandbox runs.
* refactor(gateway): move driver options into config (#1394)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
* fix(e2e): regenerate gateway config via TOML for docker + podman harnesses
The gateway CLI flags moved into TOML config tables in 560550d2 (#1394),
which made every existing e2e/with-{docker,podman}-gateway.sh invocation
fail with "unexpected argument '--sandbox-namespace'" (and a long tail
of similar driver-specific options) before the gateway could even bind.
Replace the obsolete CLI flags with a synthesized
`[openshell.drivers.<driver>]` table written to `${STATE_DIR}/gateway.toml`
and passed via the new `--config` flag. Only the gateway-wide flags
that survived 560550d2 (bind-address, port, drivers, db-url, tls-*,
disable-tls, log-level, health-port) stay on the command line.
Both scripts get a small `toml_string` helper to properly TOML-quote
the values (the previous `%q` printf format produced bash-escape, not
TOML-escape). The Docker harness also corrects two field names that
diverged from the driver schema: `docker_network_name` →
`network_name`, and the supervisor binary/image plumbing now reads
through to `supervisor_bin` / `supervisor_image` in the same table.
The Podman harness drops `--ssh-gateway-port` (deleted in 560550d2 —
gRPC + SSH are multiplexed on the same port now) and substitutes
`network_name` + `gateway_port` for the obsolete `--sandbox-namespace`
(which the Podman driver never had as a typed field).
* fix(core): swap bind-only 0.0.0.0 SSH gateway host for cluster URL host
CLI's resolve_ssh_gateway treated 0.0.0.0 as a loopback "keep as-is"
when the cluster URL was also loopback, so the SSH proxy connected to
0.0.0.0:port. The unspecified address is never a valid connect target
and is not present in any TLS cert SAN, which produced BadCertificate
TLS handshake failures during `openshell sandbox create -- ...` in
docker/podman e2e (e.g. bypass_detection).
Resolution: when the server returns 0.0.0.0 or :: as the gateway host
and both endpoints are loopback, fall back to the cluster URL's host
(which the CLI is already using to reach the gateway, so it must
resolve and match the cert).
* fix(e2e): repair podman harness on macOS
Podman 5.x with the applehv/libkrun provider no longer creates the legacy
~/.local/share/containers/podman/machine/podman.sock symlink, and
`podman system service` is a Linux-only subcommand — the macOS client
delegates the API service to the VM. Both assumptions in the harness
were stale, so the script tried to start a temporary service that podman
rejected with "unknown flag: --time".
- Discover the macOS socket via `podman machine inspect` instead of the
hardcoded path.
- On Darwin, fail fast with a "start podman machine" message rather than
attempting the Linux-only `podman system service` fallback.
- Write socket_path into [openshell.drivers.podman] so the in-process
driver picks up the discovered socket; the driver reads TOML only
after the config refactor (560550d2), so OPENSHELL_PODMAN_SOCKET alone
was no longer enough.
* fix(server): use clone_from for TLS client CA assignment
clippy 1.95.0 rejects assigning the result of `Clone::clone()` to an
existing variable under `-D warnings` (`assigning_clones`). Switch to
`clone_from(&...)` to satisfy the lint and avoid the redundant
allocation.
---------
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Co-authored-by: Drew Newberry <anewberry@nvidia.com>
|
||
|
Zygmunt Krynicki
|
dfd47683e7 | (feat) early snap support (#1238) |