trending/OpenShell
1
0
Fork 0
mirror of https://github.com/NVIDIA/OpenShell.git synced 2026-10-26 06:52:57 +08:00
Code Issues Packages Projects Releases Wiki Activity

chore(e2e): remove obsolete OpenShift sandbox SCC grant (#4823)

Browse Source 
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
This commit is contained in:
Taylor Mutch
2026-10-22 06:36:02 +00:00
committed by GitHub
parent 0a51652308
commit 39d95b42cb
3 changed files with 14 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TESTING.md
+13 -8
View File
@@ -447,14 +447,19 @@ OPENSHELL_E2E_KUBE_TEST=smoke mise run e2e:kubernetes
```
**OpenShift**: when the target cluster exposes the `route.openshift.io` API
group, the harness automatically applies SCC-compatible Helm overrides, grants
the required SCCs (`privileged` to `openshell-sandbox`, and `anyuid` to the
PostgreSQL fixture for DB scenarios), and drives the gateway through a
passthrough Route with mandatory mTLS instead of port-forward. No extra flags are
needed, but `oc` must be installed and authenticated against the target cluster
with permission to modify SCC bindings (`oc adm policy add-scc-to-user`) — the
harness exits early if `oc` is missing. The SCC grants and extracted client
mTLS material are removed during cleanup, including on failure or interrupt.
group, the harness automatically applies SCC-compatible Helm overrides and
drives the gateway through a passthrough Route with mandatory mTLS instead of
port-forward. No extra flags are needed, but `oc` must be installed and
authenticated against the target cluster; the harness exits early if `oc` is
missing. DB scenarios grant `anyuid` to the PostgreSQL fixture and require
permission to modify SCC bindings (`oc adm policy add-scc-to-user`). That grant
and extracted client mTLS material are removed during cleanup, including on
failure or interrupt.
Sandbox pods use the namespace-assigned UID/GID and run under `restricted-v2`
without a privileged SCC grant to `openshell-sandbox`. Earlier versions required
that grant for in-container isolation setup. [RFC 0012](https://github.com/NVIDIA/OpenShell/blob/main/rfc/0012-isolation-backend/README.md#motivation)
explains the isolation backend separation behind the capability-free runtime.
On a **remote** cluster, drop the `e2e-host-gateway` feature. Those tests rely
on the sandbox-side `host.openshell.internal` alias reaching the machine running
e2e/with-kube-gateway.sh
-15
View File
@@ -119,7 +119,6 @@ CORPORATE_PROXY_FIXTURE_SECRET="openshell-e2e-proxy-auth"
CORPORATE_PROXY_FIXTURE_CA_CONFIGMAP="openshell-e2e-proxy-ca"
CORPORATE_PROXY_CA_FIXTURE_DEPLOYED=0
OPENSHIFT_DETECTED=0
OPENSHIFT_SANDBOX_SCC_GRANTED=0
OPENSHIFT_POSTGRES_SCC_GRANTED=0
OPENSHIFT_ROUTE_HOST=""
# Temp dir holding the client mTLS material extracted from openshell-client-tls
@@ -589,14 +588,6 @@ cleanup() {
--ignore-not-found >/dev/null 2>&1 || true
fi
if [ "${OPENSHIFT_SANDBOX_SCC_GRANTED}" = "1" ]; then
oc adm policy remove-scc-from-user privileged \
--context "${KUBE_CONTEXT}" \
-z openshell-sandbox -n "${NAMESPACE}" \
2>/dev/null || true
OPENSHIFT_SANDBOX_SCC_GRANTED=0
fi
# Remove the extracted client mTLS material (also covered by the WORKDIR sweep
# below, but drop the private key promptly and explicitly).
if [ -n "${OPENSHIFT_PKI_DIR}" ]; then
@@ -1259,12 +1250,6 @@ if [ "${OPENSHIFT_DETECTED}" = "1" ]; then
kctl create namespace "${NAMESPACE}" --dry-run=client -o yaml | kctl apply -f -
echo "Granting privileged SCC to openshell-sandbox in namespace ${NAMESPACE}..."
oc adm policy add-scc-to-user privileged \
--context "${KUBE_CONTEXT}" \
-z openshell-sandbox -n "${NAMESPACE}"
OPENSHIFT_SANDBOX_SCC_GRANTED=1
# Drive the gateway through a passthrough Route with mTLS instead of
# port-forward. The Route host is deterministic: OpenShift serves any name
# under the cluster ingress (apps) domain via the router's wildcard, so we
skills/debug-openshell-cluster/SKILL.md
+1 -1
View File
@@ -1280,7 +1280,7 @@ credential failures.
| Gateway or `<release>-certgen` hook pod rejected with `violates PodSecurity "restricted:latest"` | The rendered container lacks `runAsNonRoot` or `seccompProfile`: an older chart version does not set them, or an override in `securityContext` / `pkiInitJob.securityContext` unset them | `kubectl -n openshell get events --field-selector reason=FailedCreate`; upgrade the chart, then remove overrides that unset those fields |
| OpenShift gateway pod fails to start with an SCC/`runAsUser` error (e.g. `unable to validate against any security context constraint`) | Chart's default `podSecurityContext`/`securityContext` hardcodes `runAsUser`/`fsGroup`, which the restricted-v2 SCC rejects; it must instead inject the namespace-assigned UID/GID range | `oc -n openshell describe pod <pod>`; deploy with `podSecurityContext: null` and clear `securityContext.runAsUser` (see `deploy/helm/openshell/ci/values-openshift-scc.yaml`) |
| OpenShift `<release>-supervisor` Deployment never creates pods after setting `supervisor_topology = "gateway"` (SCC `runAsUser`/`fsGroup` not in the namespace range) | `supervisor.podSecurityContext` defaults `runAsUser`, `runAsGroup`, and `fsGroup` to 65534, outside the namespace's assigned UID/GID ranges | `kubectl -n openshell get events --field-selector reason=FailedCreate`; compare with the `openshift.io/sa.scc.*` annotations from `oc describe ns openshell`; set `supervisor.podSecurityContext.runAsUser`, `.runAsGroup`, and `.fsGroup` to `null`, as `deploy/helm/openshell/ci/values-openshift-scc.yaml` does, and keep `runAsNonRoot` and `seccompProfile` |
| OpenShift sandbox pod fails to start (`unable to validate against any security context constraint`) | The `openshell-sandbox` service account lacks the privileged SCC it needs | `oc adm policy add-scc-to-user privileged -z openshell-sandbox -n openshell`; remove with `remove-scc-from-user` when done |
| OpenShift sandbox pod fails to start (`unable to validate against any security context constraint`) | An explicit sandbox UID/GID override or custom security context conflicts with the namespace's SCC range | Inspect pod events and the namespace's `openshift.io/sa.scc.uid-range` and `openshift.io/sa.scc.supplemental-groups` annotations; remove incompatible `sandbox_uid` / `sandbox_gid` overrides so the driver uses the namespace-assigned identity. Current sandbox pods use `restricted-v2` without a privileged SCC grant; see the [OpenShift guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift) and [RFC 0012](https://github.com/NVIDIA/OpenShell/blob/main/rfc/0012-isolation-backend/README.md#motivation). |
| OpenShift self-hosted Vault/OpenBao credential store pod never schedules (waits time out with `no matching resources found`) | The store's Helm chart pins `runAsUser`/`fsGroup`/seccomp, which restricted-v2 rejects, so the StatefulSet controller never creates the pod | Deploy the store's chart in its OpenShift mode (`--set global.openshift=true` for the OpenBao/Vault chart) so the namespace SCC assigns a compliant security context — no manual SCC grant needed |
| Vault credential driver returns HTTP 403 / `Vault Kubernetes auth denied the configured role` on provider create | Vault's `auth/kubernetes` method or the gateway login role is not provisioned, or the role is not bound to the gateway service account and namespace | In Vault: `bao auth enable kubernetes` and `bao write auth/kubernetes/config kubernetes_host=... kubernetes_ca_cert=@...`; ensure the login role's `bound_service_account_names`/`bound_service_account_namespaces` match the gateway SA and namespace and its policy grants the credential paths |
| CLI TLS error | Local mTLS bundle does not match server cert/CA | Check `~/.config/openshell/gateways/<name>/mtls/` |