Files
OpenShell/e2e/python/conftest.py
T
Russell Bryant 9377e0d5fe fix(providers): allow git clone/fetch via default GitHub provider (#2317)
* fix(providers): allow git clone/fetch via default GitHub provider

The github.com:443 git-transport endpoint used the read-only access
preset, which expands to GET/HEAD/OPTIONS only. Git smart HTTP requires
a POST to */git-upload-pack for clone and fetch, so the L7 proxy denied
those operations and `gh repo clone` / `git clone https://...` failed.

Replace the preset with explicit rules that permit the read-only methods
plus POST */git-upload-pack, so clone/fetch work while push
(git-receive-pack) stays blocked. Enabling push still requires an
explicit policy proposal.

Why allowing this POST is still read-only: in git's smart HTTP protocol
POST is an RPC transport, not a write. A clone/fetch does GET
*/info/refs (ref discovery) followed by POST */git-upload-pack, whose
body is only the client's want/have negotiation; the server responds
with a packfile and nothing on the server is modified (data flows
server -> client). The service names are from the server's perspective:
git-upload-pack = the server uploads a pack to the client (a read/
download), while git-receive-pack = the server receives a pack from the
client (the actual write/push). The new rule is scoped to
*/git-upload-pack only, so push (git-receive-pack) and arbitrary POSTs
to github.com remain denied.

Add a provider-profile regression test and a rego enforcement test
covering ref discovery, upload-pack (allowed), and receive-pack (denied).

Closes #1769

Signed-off-by: Russell Bryant <rbryant@redhat.com>

* test(providers): strengthen git-transport regression and add clone e2e

Pin the exact allowed rule set for the built-in github git-transport
endpoint in both the provider-profile and composed-policy tests, so a
broader or additional POST rule (e.g. POST **) that could enable push
via git-receive-pack fails the test instead of passing a substring
check. Add an e2e test that attaches the built-in github provider and
clones a public repo over HTTPS, exercising provider attachment,
effective-policy composition, TLS interception, and real git behavior.

Update the Providers V2 docs so the github.com git-transport endpoint
shows explicit clone/fetch rules instead of the stale read-only preset.

Refs #1769

Signed-off-by: Russell Bryant <rbryant@redhat.com>

* test(providers): isolate providers_v2 mutation in clone e2e

The clone e2e enables the gateway-global providers_v2_enabled setting.
Restore its exact prior value (or absence) captured via GetGatewayConfig
instead of unconditionally deleting it, and serialize the mutation
across xdist workers with an exclusive file lock on the run's shared
base temp dir, so a shared or pre-configured gateway is left untouched
and parallel workers cannot race the read-modify-restore.

Refs #1769

Signed-off-by: Russell Bryant <rbryant@redhat.com>

* test(providers): serialize providers_v2 mutation with a suite-wide guard

The clone e2e's per-fixture lock only coordinated fixtures that acquired
it; other xdist workers hit the same gateway without it and could
observe the transiently-enabled providers_v2_enabled global during their
own sandbox creation (CWE-362).

Add an autouse readers-writer guard in conftest: every test holds a
shared lock on the gateway config, and a test marked
exclusive_gateway_config holds an exclusive lock. Mark the clone test
exclusive so no other worker is mid-test while it enables and restores
the gateway-global setting. Exact prior-value restoration is retained.

Refs #1769

Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
2026-07-20 21:07:17 +00:00

127 lines
4.2 KiB
Python

# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import fcntl
import os
import time
from typing import TYPE_CHECKING
import grpc
import pytest
from openshell import InferenceRouteClient, Sandbox, SandboxClient
if TYPE_CHECKING:
from collections.abc import Callable, Iterator
def pytest_configure(config: pytest.Config) -> None:
config.addinivalue_line(
"markers",
"exclusive_gateway_config: hold an exclusive lock on gateway-global "
"config so no other xdist worker observes a transient global setting",
)
@pytest.fixture(autouse=True)
def _gateway_config_guard(
request: pytest.FixtureRequest,
tmp_path_factory: pytest.TempPathFactory,
) -> Iterator[None]:
"""Readers-writer guard over gateway-global config mutations.
The e2e gateway is shared across xdist workers, so a test that flips a
gateway-global setting can leak that transient value into another worker's
sandbox creation. Every test holds a shared lock by default; a test marked
``exclusive_gateway_config`` holds an exclusive lock, so while it mutates and
restores the setting no other worker is mid-test and none can observe the
transient value. The lock file lives in the run's shared base temp dir
(``getbasetemp().parent``), which is common to all xdist workers.
"""
lock_path = tmp_path_factory.getbasetemp().parent / "gateway-config.lock"
exclusive = (
request.node.get_closest_marker("exclusive_gateway_config") is not None
)
with lock_path.open("w") as lock_file:
fcntl.flock(lock_file, fcntl.LOCK_EX if exclusive else fcntl.LOCK_SH)
yield
@pytest.fixture(scope="session")
def cluster_name() -> str | None:
return os.environ.get("OPENSHELL_GATEWAY")
@pytest.fixture(scope="session")
def sandbox_client(cluster_name: str | None) -> Iterator[SandboxClient]:
with SandboxClient.from_active_cluster(cluster=cluster_name) as client:
yield client
@pytest.fixture(scope="session", autouse=True)
def ensure_sandbox_persistence_ready(sandbox_client: SandboxClient) -> None:
for _ in range(60):
try:
sandbox_client.list_ids(limit=1)
return
except grpc.RpcError as exc:
details = exc.details() or ""
if exc.code() == grpc.StatusCode.UNAVAILABLE:
time.sleep(2)
continue
if (
exc.code() == grpc.StatusCode.INTERNAL
and "no such table: objects" in details
):
time.sleep(1)
continue
raise
pytest.fail(
"openshell-server persistence is not initialized (missing sqlite objects table); "
"redeploy the active cluster and rerun e2e sandbox tests"
)
@pytest.fixture
def sandbox(cluster_name: str | None) -> Callable[..., Sandbox]:
def _create(*, spec: object | None = None, delete_on_exit: bool = True) -> Sandbox:
return Sandbox(
cluster=cluster_name,
spec=spec,
delete_on_exit=delete_on_exit,
# The sandbox image is large (Python, Node.js, coding agents) so the
# first pod in the cluster may need extra time for the image pull.
ready_timeout_seconds=300.0,
)
return _create
@pytest.fixture(scope="session")
def inference_client(sandbox_client: SandboxClient) -> InferenceRouteClient:
return InferenceRouteClient.from_sandbox_client(sandbox_client)
@pytest.fixture(scope="session")
def _worker_suffix(worker_id: str) -> str:
"""Return a suffix for worker-unique resource names.
Uses the built-in ``worker_id`` fixture from pytest-xdist which returns
``"gw0"``, ``"gw1"``, etc. for workers, or ``"master"`` for non-xdist runs.
"""
if worker_id == "master":
return ""
return f"-{worker_id}"
@pytest.fixture
def run_python() -> Callable[[Sandbox, str], tuple[int, str, str]]:
def _run(sandbox: Sandbox, code: str) -> tuple[int, str, str]:
result = sandbox.exec(["python", "-c", code], timeout_seconds=20)
return result.exit_code, result.stdout, result.stderr
return _run