mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-02 07:34:45 +08:00
ci(windows): exercise MXC inference demos with mock API (#3780)
* ci(windows): exercise MXC Ollama demo with mock API Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> * ci(windows): cover both MXC inference demos Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> * fix(mxc): preserve executable extension resolution Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> * test(mxc): use absolute PowerShell in lifecycle checks Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> * test(mxc): make lifecycle write probes deterministic Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> * test(mxc): assert stable lifecycle completion Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> --------- Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
This commit is contained in:
Prekshi Vyas
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: build-openshell-mxc-windows
|
||||
description: Maintain and validate OpenShell's build-only Windows MSVC lane for x64 and ARM64. Use when working on Windows compilation, `windows:*` mise tasks, unsupported Windows compute-driver contracts, or Windows build reports. This skill does not implement Docker, Kubernetes, Podman, VM, MXC driver, policy translation, MSI, service, or supervisor runtime support on Windows.
|
||||
description: Maintain and validate OpenShell's Windows MSVC lane for x64 and ARM64. Use when working on Windows compilation, `windows:*` mise tasks, unsupported Windows compute-driver contracts, MXC example wiring checks, or Windows build reports. This skill does not implement Docker, Kubernetes, Podman, VM, MXC driver, policy translation, MSI, service, or supervisor runtime support on Windows.
|
||||
metadata:
|
||||
internal: true
|
||||
---
|
||||
@@ -12,8 +12,8 @@ OpenShell repository. The Windows lane is already present in `main`; do not
|
||||
treat this skill as a first-time porting recipe unless the user explicitly asks
|
||||
for a new fork or a from-scratch bring-up.
|
||||
|
||||
The lane is build-only. It validates that OpenShell can compile and test on
|
||||
Windows MSVC for the supported deliverables:
|
||||
The lane validates that OpenShell can compile and test on Windows MSVC for the
|
||||
supported deliverables:
|
||||
|
||||
- `openshell-gateway.exe`
|
||||
- `openshell.exe`
|
||||
@@ -51,6 +51,8 @@ In scope:
|
||||
`openshell`.
|
||||
- Running workspace tests on a native x64 or ARM64 host.
|
||||
- Running focused unsupported-driver contract tests.
|
||||
- Running the shipped Ollama and cloud-inference MXC examples against the
|
||||
in-process `wxc` mock and a local API stub.
|
||||
- Reporting test counts, skipped/gated areas, warnings, artifacts, and logs.
|
||||
- Keeping Linux and macOS build paths unchanged.
|
||||
- Keeping unsupported Windows compute drivers explicit and testable.
|
||||
@@ -158,8 +160,15 @@ mise run --skip-tools windows:build:x64
|
||||
mise run --skip-tools windows:build:arm64
|
||||
mise run --skip-tools windows:test:x64
|
||||
mise run --skip-tools windows:test:unsupported:x64
|
||||
mise run --skip-tools windows:e2e:mxc:inference-mock:x64
|
||||
```
|
||||
|
||||
Use the `arm64` form of the inference task on a native ARM64 host. It exercises
|
||||
the real gateway, CLI, and shipped PowerShell runners with an in-process `wxc`
|
||||
mock and local HTTP responses. It proves example wiring, request execution, and
|
||||
sandbox-scoped credential propagation; it does not prove MXC, AppContainer,
|
||||
filesystem, or network enforcement.
|
||||
|
||||
The two `windows:test:mxc-real:*` tasks are host-specific and mutually
|
||||
exclusive on a single host (each rejects the other architecture -- see the
|
||||
table below): run `windows:test:mxc-real:x64` on an x64 host, or
|
||||
@@ -220,14 +229,17 @@ order:
|
||||
The GitHub Actions jobs layer architecture-specific `Swatinem/rust-cache`
|
||||
entries for Cargo registry and dependency target artifacts with sccache's GHA
|
||||
backend for cacheable Rust compiler outputs. Failed runs also save their usable
|
||||
dependency artifacts. Pull-request mirrors labeled `test:windows` run Clippy for the
|
||||
Windows-supported workspace and e2e crates plus Rust tests. Pushes to `main` and
|
||||
manual dispatches run the same lint and test commands in a cache-seed job,
|
||||
followed by a dependent release-binary build job. The seed and PR jobs use the
|
||||
same cache namespaces. Merge queues do not run this workflow. Main/manual seed
|
||||
and build jobs use job-level `continue-on-error: true`; opt-in PR jobs report
|
||||
failures normally. Applying the label alone does not start a run: re-run all
|
||||
jobs in the current mirror push run, or push a new mirrored commit. The binaries are not uploaded or published.
|
||||
dependency artifacts. Pull-request mirrors labeled `test:windows` run Clippy
|
||||
for the Windows-supported workspace and e2e crates plus Rust tests, build
|
||||
release binaries, and run both shipped MXC inference examples through the mock
|
||||
task. Pushes to `main` and manual dispatches run the same lint and test commands
|
||||
in a cache-seed job, followed by a dependent release-binary build and
|
||||
mock-example job. The seed and PR jobs use the same cache namespaces. Merge
|
||||
queues do not run this workflow. Main/manual seed and build jobs use job-level
|
||||
`continue-on-error: true`; opt-in PR jobs report failures normally. Applying
|
||||
the label alone does not start a run: re-run all jobs in the current mirror
|
||||
push run, or push a new mirrored commit. The binaries are not uploaded or
|
||||
published.
|
||||
|
||||
The ARM64 check/build steps in this x64-host contract are cross-builds. The
|
||||
wrapper discovers and adds host-native LLVM and Ninja to `PATH`, requires the
|
||||
@@ -272,6 +284,8 @@ crypto dependency builds.
|
||||
| `windows:test:mxc-real:x64` | Runs the serial, ignored real-`wxc-exec` integration suite natively on x64 through the MSVC wrapper. Rejects non-x64 hosts. |
|
||||
| `windows:test:mxc-real:arm64` | Runs the same real-`wxc-exec` suite natively on ARM64. Rejects non-ARM64 hosts. |
|
||||
| `windows:test:mxc-gb300:arm64` | Runs the required native ARM64 ProcessContainer subset and fails when a test skips. |
|
||||
| `windows:e2e:mxc:inference-mock:x64` | Runs the shipped Ollama and cloud-inference demos on native x64 through the real gateway and CLI, in-process `wxc` mock, and local API stub. This is wiring evidence, not MXC enforcement evidence. |
|
||||
| `windows:e2e:mxc:inference-mock:arm64` | Runs the same mock-wiring checks on native ARM64 with ARM64 release binaries. |
|
||||
| `windows:qualify:mxc:gb300:contract` | Validates the required/optional/unsupported/architecture-constrained GB300 matrix. |
|
||||
| `windows:qualify:mxc:gb300` | Runs the fail-closed GB300 ARM64 gate and validates hash-bound evidence. |
|
||||
| `windows:artifacts` | Reports size and SHA256 for release artifacts that exist. |
|
||||
@@ -324,6 +338,8 @@ When reporting `windows:ci`, distinguish these categories:
|
||||
- Passed tests from the full ARM64 workspace test log when run on a native
|
||||
ARM64 host.
|
||||
- The focused unsupported-contract re-run.
|
||||
- The architecture-matched MXC inference-example mock task and its explicit
|
||||
wiring-only limitation.
|
||||
- Explicit Cargo ignored tests, usually ignored doc examples.
|
||||
- Tests hidden by `#[cfg(not(target_os = "windows"))]`; these often appear as
|
||||
`running 0 tests`, not as ignored tests.
|
||||
@@ -385,6 +401,7 @@ Every substantial Windows build run should report:
|
||||
| ARM64 check/build | Pass/fail/skipped and log path. |
|
||||
| Native tests | Passed/failed/ignored/filtered counts and log path for the host architecture. |
|
||||
| Unsupported contracts | Which focused tests ran and their result. |
|
||||
| MXC example mock E2E | Architecture, result, artifact directory on failure, and the wiring-only limitation. |
|
||||
| Artifacts | Binary paths, size, and SHA256 when available. |
|
||||
| Skips | Explicitly explain tests not run for a non-native architecture, unsupported driver package exclusions, and Windows cfg-gated tests. |
|
||||
| Follow-ups | Only concrete follow-ups tied to failures or requested scope. |
|
||||
|
||||
@@ -98,6 +98,10 @@ jobs:
|
||||
run: mise run --skip-tools windows:lint:${{ matrix.arch }}
|
||||
- name: Test
|
||||
run: mise run --skip-tools windows:test:${{ matrix.arch }}
|
||||
- name: Build MXC example E2E binaries
|
||||
run: mise run --skip-tools windows:build:${{ matrix.arch }}
|
||||
- name: Run shipped MXC inference examples with mock API
|
||||
run: mise run --skip-tools windows:e2e:mxc:inference-mock:${{ matrix.arch }}
|
||||
- name: sccache stats
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
@@ -211,6 +215,8 @@ jobs:
|
||||
cache-bin: "false"
|
||||
- name: Build release binaries
|
||||
run: mise run --skip-tools windows:build:${{ matrix.arch }}
|
||||
- name: Run shipped MXC inference examples with mock API
|
||||
run: mise run --skip-tools windows:e2e:mxc:inference-mock:${{ matrix.arch }}
|
||||
- name: sccache stats
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
@@ -248,6 +248,11 @@ Windows validation separates source correctness from host capability:
|
||||
workspace and unsupported-driver contract tests for x64 and ARM64.
|
||||
- Mock MXC E2E validates gateway, CLI, driver, lifecycle, and policy wiring but
|
||||
is not evidence of OS enforcement.
|
||||
- Hosted Windows CI runs the shipped Ollama and cloud-inference demos against
|
||||
that mock and a local compatible API. This proves both complete demo scripts,
|
||||
sandbox-scoped credential propagation, and response paths without a model or
|
||||
external credential, but it does not prove MXC isolation, proxy substitution,
|
||||
or network enforcement.
|
||||
- Real-`wxc-exec` tests validate the installed schema and selected filesystem,
|
||||
UI, network, and lifecycle behavior. A probe-gated skip is useful
|
||||
diagnostic output, not qualification evidence.
|
||||
|
||||
@@ -59,7 +59,7 @@ pc_relay_spawner_path = ""
|
||||
pc_relay_target_port = 0
|
||||
# processContainer only: env-inheritance tier for the launched process
|
||||
# (safest first): default is a minimal Windows CreateProcessW bootstrap set
|
||||
# (SYSTEMROOT/WINDIR/PATH/COMSPEC/LOCALAPPDATA); pc_minimal_env starts from an
|
||||
# (SYSTEMROOT/WINDIR/PATH/PATHEXT/COMSPEC/LOCALAPPDATA); pc_minimal_env starts from an
|
||||
# EMPTY env for runtimes that need a fully curated per-sandbox environment.
|
||||
pc_minimal_env = false
|
||||
# processContainer only: compatibility fallback for unrestricted outbound TCP.
|
||||
@@ -203,11 +203,11 @@ environment variable, so workloads using it must pass
|
||||
`--cacert %CURL_CA_BUNDLE%` explicitly. Clients that honor the injected trust
|
||||
variables consume the same per-sandbox bundle directly.
|
||||
|
||||
The driver seeds only `SYSTEMROOT`, `WINDIR`, `PATH`, `COMSPEC`, and
|
||||
`LOCALAPPDATA` from the gateway host before applying sandbox and TLS overrides,
|
||||
so required Windows bootstrap values remain available without exposing the
|
||||
gateway's full environment unless the gateway explicitly opts into another
|
||||
environment mode.
|
||||
The driver seeds only `SYSTEMROOT`, `WINDIR`, `PATH`, `PATHEXT`, `COMSPEC`, and
|
||||
`LOCALAPPDATA` from the gateway host before applying sandbox and TLS overrides.
|
||||
These values provide Windows process startup and command-resolution behavior
|
||||
without exposing the gateway's full environment unless the gateway explicitly
|
||||
opts into another environment mode.
|
||||
|
||||
When governed egress is disabled, any network rule fails closed during sandbox creation.
|
||||
|
||||
|
||||
@@ -14,8 +14,8 @@ network_policies:
|
||||
nvidia_inference:
|
||||
name: nvidia-inference
|
||||
endpoints:
|
||||
- host: integrate.api.nvidia.com
|
||||
port: 443
|
||||
- host: "__INFERENCE_HOST__"
|
||||
port: __INFERENCE_PORT__
|
||||
protocol: rest
|
||||
# Chat completions use POST.
|
||||
access: read-write
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
# Cloud inference (T1) demo for OpenShell on MXC. PowerShell 5.1 compatible.
|
||||
# -Mock runs the workload on the host through the in-process wxc shim. It is for
|
||||
# CI wiring coverage only and does not provide MXC or AppContainer isolation.
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
@@ -11,8 +13,10 @@ param(
|
||||
[string] $ShareDir,
|
||||
[string] $Model = "nvidia/nemotron-3.5-lightning-30b-a3b",
|
||||
[string] $Prompt = "Say hello in exactly five words.",
|
||||
[string] $ApiUrl = "https://integrate.api.nvidia.com/v1/chat/completions",
|
||||
[ValidateRange(0, 65535)] [int] $Port = 0,
|
||||
[string] $SandboxName,
|
||||
[switch] $Mock,
|
||||
[switch] $KeepArtifacts
|
||||
)
|
||||
|
||||
@@ -133,6 +137,7 @@ $success = $false
|
||||
$failure = $null
|
||||
$oldGatewayConfig = $env:OPENSHELL_GATEWAY_CONFIG
|
||||
$oldComputeDriver = $env:OPENSHELL_COMPUTE_DRIVER
|
||||
$oldMockWxc = $env:OPENSHELL_MXC_MOCK_WXC
|
||||
$oldApiKey = $env:NV_API_KEY
|
||||
$apiKey = $env:NV_API_KEY
|
||||
|
||||
@@ -142,12 +147,29 @@ try {
|
||||
}
|
||||
$gateway = Resolve-Executable $GatewayPath "openshell-gateway.exe" ""
|
||||
$cli = Resolve-Executable $CliPath "openshell.exe" ""
|
||||
$wxc = Resolve-Executable $WxcExecPath "wxc-exec.exe" "OPENSHELL_WXC_EXEC_PATH"
|
||||
if ($Mock) {
|
||||
# The gateway still validates that wxc_exec_path is absolute. The
|
||||
# in-process mock never launches this placeholder.
|
||||
$wxc = Join-Path $here "mock-wxc-exec.exe"
|
||||
} else {
|
||||
$wxc = Resolve-Executable $WxcExecPath "wxc-exec.exe" "OPENSHELL_WXC_EXEC_PATH"
|
||||
}
|
||||
foreach ($fixture in @("mxc-inference.toml", "inference.yaml")) {
|
||||
if (-not (Test-Path -LiteralPath (Join-Path $here $fixture) -PathType Leaf)) {
|
||||
throw "required demo fixture '$fixture' is missing beside the runner"
|
||||
}
|
||||
}
|
||||
if ($ApiUrl.IndexOfAny([char[]]@('"', '%', '!', '&', '|', '<', '>', '^', [char] 13, [char] 10)) -ge 0) {
|
||||
throw "ApiUrl contains a character that cannot be rendered safely into the sandbox command"
|
||||
}
|
||||
try {
|
||||
$apiUri = [System.Uri] $ApiUrl
|
||||
} catch {
|
||||
throw "ApiUrl is not a valid absolute URI: $ApiUrl"
|
||||
}
|
||||
if (-not $apiUri.IsAbsoluteUri -or $apiUri.Scheme -notin @("http", "https")) {
|
||||
throw "ApiUrl must be an absolute HTTP or HTTPS URI"
|
||||
}
|
||||
if ($Port -eq 0) { $Port = Get-AvailablePort }
|
||||
$endpoint = "http://127.0.0.1:$Port"
|
||||
if ([string]::IsNullOrWhiteSpace($SandboxName)) { $SandboxName = "inference-$PID" }
|
||||
@@ -163,8 +185,9 @@ try {
|
||||
|
||||
Info "gateway: $gateway"
|
||||
Info "CLI: $cli"
|
||||
Info "wxc-exec: $wxc"
|
||||
Info "wxc-exec: $(if ($Mock) { 'in-process mock (no MXC isolation)' } else { $wxc })"
|
||||
Info "share: $ShareDir"
|
||||
Info "inference API: $ApiUrl"
|
||||
Info "NV_API_KEY: present (value redacted)"
|
||||
|
||||
$cmdExe = Join-Path $env:SystemRoot "System32\cmd.exe"
|
||||
@@ -187,6 +210,8 @@ try {
|
||||
$policyText = [System.IO.File]::ReadAllText((Join-Path $here "inference.yaml"))
|
||||
$policyText = $policyText.Replace("__OPENSHELL_DEMO_SHARE__", $sharePolicy)
|
||||
$policyText = $policyText.Replace("__CMD_EXE__", $cmdExe)
|
||||
$policyText = $policyText.Replace("__INFERENCE_HOST__", $apiUri.DnsSafeHost)
|
||||
$policyText = $policyText.Replace("__INFERENCE_PORT__", [string] $apiUri.Port)
|
||||
Write-Utf8 $policyUsed $policyText
|
||||
|
||||
$requestPath = Join-Path $ShareDir "inference-request.json"
|
||||
@@ -198,11 +223,21 @@ try {
|
||||
Write-Utf8 $requestPath $requestJson
|
||||
|
||||
$probePath = Join-Path $ShareDir "inference-probe.cmd"
|
||||
$probeLines = @(
|
||||
"@echo off",
|
||||
$tlsArgs = if ($apiUri.Scheme -eq "https") {
|
||||
# Inbox curl uses Schannel and does not honor CURL_CA_BUNDLE by itself.
|
||||
# Point --cacert at the public bundle staged by the governed proxy.
|
||||
"`"$curlExe`" --silent --show-error --fail-with-body --ssl-no-revoke --cacert `"%CURL_CA_BUNDLE%`" --max-time 120 -D `"$headersPath`" -H `"Authorization: Bearer %NV_API_KEY%`" -H `"Content-Type: application/json`" --data-binary `"@$requestPath`" -o `"$responsePath`" `"https://integrate.api.nvidia.com/v1/chat/completions`" 2> `"$errorPath`" || exit /b 31",
|
||||
'--ssl-no-revoke --cacert "%CURL_CA_BUNDLE%"'
|
||||
} else {
|
||||
""
|
||||
}
|
||||
$proxyBypassArgs = if ($apiUri.IsLoopback) {
|
||||
"--noproxy `"$($apiUri.DnsSafeHost)`""
|
||||
} else {
|
||||
""
|
||||
}
|
||||
$probeLines = @(
|
||||
"@echo off",
|
||||
"`"$curlExe`" $proxyBypassArgs --silent --show-error --fail-with-body $tlsArgs --max-time 120 -D `"$headersPath`" -H `"Authorization: Bearer %NV_API_KEY%`" -H `"Content-Type: application/json`" --data-binary `"@$requestPath`" -o `"$responsePath`" `"$ApiUrl`" 2> `"$errorPath`" || exit /b 31",
|
||||
"`"$findStrExe`" /C:`"choices`" `"$responsePath`" >nul || exit /b 32",
|
||||
"echo PASS> `"$donePath`""
|
||||
)
|
||||
@@ -214,6 +249,11 @@ try {
|
||||
$gwErrLog = Join-Path $resultDir "gateway.err.log"
|
||||
$env:OPENSHELL_GATEWAY_CONFIG = $tomlUsed
|
||||
$env:OPENSHELL_COMPUTE_DRIVER = "mxc"
|
||||
if ($Mock) {
|
||||
$env:OPENSHELL_MXC_MOCK_WXC = "1"
|
||||
} else {
|
||||
Remove-Item Env:OPENSHELL_MXC_MOCK_WXC -ErrorAction SilentlyContinue
|
||||
}
|
||||
# The credential belongs to sandbox creation, not gateway configuration.
|
||||
Remove-Item Env:NV_API_KEY -ErrorAction SilentlyContinue
|
||||
try {
|
||||
@@ -280,6 +320,11 @@ try {
|
||||
}
|
||||
$env:OPENSHELL_GATEWAY_CONFIG = $oldGatewayConfig
|
||||
$env:OPENSHELL_COMPUTE_DRIVER = $oldComputeDriver
|
||||
if ([string]::IsNullOrWhiteSpace($oldMockWxc)) {
|
||||
Remove-Item Env:OPENSHELL_MXC_MOCK_WXC -ErrorAction SilentlyContinue
|
||||
} else {
|
||||
$env:OPENSHELL_MXC_MOCK_WXC = $oldMockWxc
|
||||
}
|
||||
if ([string]::IsNullOrWhiteSpace($oldApiKey)) { Remove-Item Env:NV_API_KEY -ErrorAction SilentlyContinue } else { $env:NV_API_KEY = $oldApiKey }
|
||||
if (-not $KeepArtifacts -and $createdShare -and $ShareDir -and (Test-Path -LiteralPath $ShareDir)) {
|
||||
Remove-Item -LiteralPath $ShareDir -Recurse -Force -ErrorAction SilentlyContinue
|
||||
@@ -287,7 +332,7 @@ try {
|
||||
}
|
||||
|
||||
$verdict = if ($success) { "PASS" } else { "FAIL" }
|
||||
$summary = "verdict=$verdict`r`nbase=cloud-inference`r`nsandbox=$SandboxName`r`ngateway=$endpoint`r`nbackend=process_container`r`nresult=$failure`r`n"
|
||||
$summary = "verdict=$verdict`r`nbase=cloud-inference`r`nmode=$(if ($Mock) { 'mock-wiring' } else { 'real-mxc' })`r`nsandbox=$SandboxName`r`ngateway=$endpoint`r`nbackend=process_container`r`nresult=$failure`r`n"
|
||||
Write-Utf8 (Join-Path $resultDir "summary.txt") $summary
|
||||
Write-Host "`n$summary"
|
||||
Write-Host "Results: $resultDir"
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
# Hello World local-inference demo for OpenShell on MXC. PowerShell 5.1 compatible.
|
||||
# -Mock runs the workload on the host through the in-process wxc shim. It is for
|
||||
# CI wiring coverage only and does not provide MXC or AppContainer isolation.
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
@@ -15,6 +17,7 @@ param(
|
||||
[string] $Prompt = "Say hello in exactly five words.",
|
||||
[ValidateRange(0, 65535)] [int] $Port = 0,
|
||||
[string] $SandboxName,
|
||||
[switch] $Mock,
|
||||
[switch] $KeepArtifacts
|
||||
)
|
||||
|
||||
@@ -135,11 +138,18 @@ $success = $false
|
||||
$failure = $null
|
||||
$oldGatewayConfig = $env:OPENSHELL_GATEWAY_CONFIG
|
||||
$oldComputeDriver = $env:OPENSHELL_COMPUTE_DRIVER
|
||||
$oldMockWxc = $env:OPENSHELL_MXC_MOCK_WXC
|
||||
|
||||
try {
|
||||
$gateway = Resolve-Executable $GatewayPath "openshell-gateway.exe" ""
|
||||
$cli = Resolve-Executable $CliPath "openshell.exe" ""
|
||||
$wxc = Resolve-Executable $WxcExecPath "wxc-exec.exe" "OPENSHELL_WXC_EXEC_PATH"
|
||||
if ($Mock) {
|
||||
# The gateway still validates that wxc_exec_path is absolute. The
|
||||
# in-process mock never launches this placeholder.
|
||||
$wxc = Join-Path $here "mock-wxc-exec.exe"
|
||||
} else {
|
||||
$wxc = Resolve-Executable $WxcExecPath "wxc-exec.exe" "OPENSHELL_WXC_EXEC_PATH"
|
||||
}
|
||||
foreach ($fixture in @("mxc-ollama.toml", "ollama.yaml")) {
|
||||
if (-not (Test-Path -LiteralPath (Join-Path $here $fixture) -PathType Leaf)) {
|
||||
throw "required demo fixture '$fixture' is missing beside the runner"
|
||||
@@ -163,7 +173,7 @@ try {
|
||||
|
||||
Info "gateway: $gateway"
|
||||
Info "CLI: $cli"
|
||||
Info "wxc-exec: $wxc"
|
||||
Info "wxc-exec: $(if ($Mock) { 'in-process mock (no MXC isolation)' } else { $wxc })"
|
||||
Info "share: $ShareDir"
|
||||
Info "Ollama: http://${OllamaHost}:$OllamaPort"
|
||||
|
||||
@@ -226,6 +236,11 @@ try {
|
||||
$gwErrLog = Join-Path $resultDir "gateway.err.log"
|
||||
$env:OPENSHELL_GATEWAY_CONFIG = $tomlUsed
|
||||
$env:OPENSHELL_COMPUTE_DRIVER = "mxc"
|
||||
if ($Mock) {
|
||||
$env:OPENSHELL_MXC_MOCK_WXC = "1"
|
||||
} else {
|
||||
Remove-Item Env:OPENSHELL_MXC_MOCK_WXC -ErrorAction SilentlyContinue
|
||||
}
|
||||
$gatewayProcess = Start-Process -FilePath $gateway -ArgumentList @("--disable-tls", "--db-url", "sqlite::memory:", "--port", "$Port", "--log-level", "info") -WorkingDirectory $here -PassThru -WindowStyle Hidden -RedirectStandardOutput $gwLog -RedirectStandardError $gwErrLog
|
||||
$deadline = (Get-Date).AddSeconds(30)
|
||||
while ((Get-Date) -lt $deadline -and -not (Test-Port $Port)) {
|
||||
@@ -269,13 +284,18 @@ try {
|
||||
}
|
||||
$env:OPENSHELL_GATEWAY_CONFIG = $oldGatewayConfig
|
||||
$env:OPENSHELL_COMPUTE_DRIVER = $oldComputeDriver
|
||||
if ([string]::IsNullOrWhiteSpace($oldMockWxc)) {
|
||||
Remove-Item Env:OPENSHELL_MXC_MOCK_WXC -ErrorAction SilentlyContinue
|
||||
} else {
|
||||
$env:OPENSHELL_MXC_MOCK_WXC = $oldMockWxc
|
||||
}
|
||||
if (-not $KeepArtifacts -and $createdShare -and $ShareDir -and (Test-Path -LiteralPath $ShareDir)) {
|
||||
Remove-Item -LiteralPath $ShareDir -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
|
||||
$verdict = if ($success) { "PASS" } else { "FAIL" }
|
||||
$summary = "verdict=$verdict`r`nbase=local-ollama`r`nsandbox=$SandboxName`r`ngateway=$endpoint`r`nbackend=process_container`r`nresult=$failure`r`n"
|
||||
$summary = "verdict=$verdict`r`nbase=local-ollama`r`nmode=$(if ($Mock) { 'mock-wiring' } else { 'real-mxc' })`r`nsandbox=$SandboxName`r`ngateway=$endpoint`r`nbackend=process_container`r`nresult=$failure`r`n"
|
||||
Write-Utf8 (Join-Path $resultDir "summary.txt") $summary
|
||||
Write-Host "`n$summary"
|
||||
Write-Host "Results: $resultDir"
|
||||
|
||||
@@ -151,7 +151,7 @@ async fn wait_for_target_listener(port: u16) -> std::io::Result<()> {
|
||||
///
|
||||
/// - `IsolationSession`: persistent, attachable session
|
||||
/// (provision → start → exec → stop → deprovision). Does not support
|
||||
/// OpenShell filesystem-policy grants; backend defaults determine visibility.
|
||||
/// `OpenShell` filesystem-policy grants; backend defaults determine visibility.
|
||||
/// - `ProcessContainer` (default): one-shot `AppContainer`. Genuinely default-deny: a
|
||||
/// write to any ungranted path is denied by the OS. No persistent session.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
|
||||
@@ -211,8 +211,8 @@ pub struct MxcComputeConfig {
|
||||
/// passed to the process.
|
||||
/// Use for agents like Node.js that fail with `STATUS_DLL_INIT_FAILED`
|
||||
/// when unrecognised host env vars are present; the caller is then
|
||||
/// responsible for supplying `SYSTEMROOT`/`WINDIR`/`PATH`/`COMSPEC`/
|
||||
/// `LOCALAPPDATA` through `sandbox create --env/--env-from` if needed
|
||||
/// responsible for supplying `SYSTEMROOT`/`WINDIR`/`PATH`/`PATHEXT`/
|
||||
/// `COMSPEC`/`LOCALAPPDATA` through `sandbox create --env/--env-from` if needed
|
||||
/// (`CreateProcessW` itself won't succeed without `LOCALAPPDATA` at
|
||||
/// least -- see `MINIMAL_WINDOWS_BOOTSTRAP_ENV`).
|
||||
///
|
||||
@@ -665,8 +665,14 @@ fn allocate_sandbox_proxy_addr(
|
||||
/// are secrets, so resolving them from the gateway host is safe; this is
|
||||
/// the per-sandbox environment layers on top of. See `pc_minimal_env` on
|
||||
/// `MxcComputeConfig` for the explicit empty-baseline option.
|
||||
const MINIMAL_WINDOWS_BOOTSTRAP_ENV: [&str; 5] =
|
||||
["SYSTEMROOT", "WINDIR", "PATH", "COMSPEC", "LOCALAPPDATA"];
|
||||
const MINIMAL_WINDOWS_BOOTSTRAP_ENV: [&str; 6] = [
|
||||
"SYSTEMROOT",
|
||||
"WINDIR",
|
||||
"PATH",
|
||||
"PATHEXT",
|
||||
"COMSPEC",
|
||||
"LOCALAPPDATA",
|
||||
];
|
||||
|
||||
const TLS_ENV_KEYS: [&str; 6] = [
|
||||
"NODE_EXTRA_CA_CERTS",
|
||||
@@ -2612,9 +2618,19 @@ mod lifecycle_tests {
|
||||
}
|
||||
|
||||
fn driver_sandbox(id: &str) -> DriverSandbox {
|
||||
let shell =
|
||||
std::env::var("COMSPEC").unwrap_or_else(|_| r"C:\Windows\System32\cmd.exe".to_string());
|
||||
driver_sandbox_with_command(id, "", vec![shell, "/c".into(), "exit 0".into()])
|
||||
driver_sandbox_with_command(id, "", vec![inbox_cmd(), "/c".into(), "exit 0".into()])
|
||||
}
|
||||
|
||||
fn inbox_cmd() -> String {
|
||||
std::env::var("COMSPEC").unwrap_or_else(|_| r"C:\Windows\System32\cmd.exe".to_string())
|
||||
}
|
||||
|
||||
fn inbox_powershell() -> String {
|
||||
let system_root = std::env::var("SYSTEMROOT").unwrap_or_else(|_| r"C:\Windows".to_string());
|
||||
Path::new(&system_root)
|
||||
.join(r"System32\WindowsPowerShell\v1.0\powershell.exe")
|
||||
.to_string_lossy()
|
||||
.into_owned()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -2847,6 +2863,8 @@ mod lifecycle_tests {
|
||||
.replace("__OPENSHELL_DEMO_SHARE__", share)
|
||||
.replace("__OLLAMA_HOST__", "127.0.0.1")
|
||||
.replace("__OLLAMA_PORT__", "11434")
|
||||
.replace("__INFERENCE_HOST__", "integrate.api.nvidia.com")
|
||||
.replace("__INFERENCE_PORT__", "443")
|
||||
.replace("__CMD_EXE__", r"C:\Windows\System32\cmd.exe");
|
||||
parse_sandbox_policy(&rendered).expect("parse rendered shipped demo policy")
|
||||
}
|
||||
@@ -3458,10 +3476,10 @@ mod lifecycle_tests {
|
||||
let share = tmp.path().to_string_lossy().replace('\\', "/");
|
||||
let hello = format!("{share}/hello.txt");
|
||||
let cmd = vec![
|
||||
"powershell".into(),
|
||||
"-NoProfile".into(),
|
||||
"-Command".into(),
|
||||
format!("Set-Content -LiteralPath {hello} -Value hi"),
|
||||
inbox_cmd(),
|
||||
"/d".into(),
|
||||
"/c".into(),
|
||||
format!(r#"echo hi>"{hello}""#),
|
||||
];
|
||||
let backend = MxcComputeBackend::new_mocked(MxcComputeConfig::default());
|
||||
|
||||
@@ -3469,36 +3487,23 @@ mod lifecycle_tests {
|
||||
let sb = with_policy(driver_sandbox_with_command("sb-pos", &share, cmd), policy);
|
||||
backend.create_sandbox(&sb).await.expect("create accepted");
|
||||
|
||||
// Self-reported Ready=True (no supervisor) once the agent exec launches.
|
||||
let ready = wait_for(&backend, "sb-pos", |s| {
|
||||
ready_condition(s).is_some_and(|c| c.status == "True" && c.reason == "AgentRunning")
|
||||
})
|
||||
.await;
|
||||
assert!(ready.is_some(), "sandbox should self-report Ready=True");
|
||||
|
||||
// Positive proof: the in-policy write materializes the host artifact.
|
||||
let host_path = tmp.path().join("hello.txt");
|
||||
let mut found = false;
|
||||
for _ in 0..100 {
|
||||
if host_path.exists() {
|
||||
found = true;
|
||||
break;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
}
|
||||
assert!(found, "hello.txt should appear in the granted share folder");
|
||||
|
||||
// A successful one-shot agent (exit 0) must STAY Ready, not demote to
|
||||
// Error. Assert the terminal condition is Ready=True/AgentCompleted so the
|
||||
// positive demo shows a green Ready phase, not a red Error.
|
||||
let completed = wait_for(&backend, "sb-pos", |s| {
|
||||
ready_condition(s).is_some_and(|c| c.status == "True" && c.reason == "AgentCompleted")
|
||||
ready_condition(s).is_some_and(|condition| {
|
||||
condition.status == "True" && condition.reason == "AgentCompleted"
|
||||
})
|
||||
})
|
||||
.await;
|
||||
assert!(
|
||||
completed.is_some(),
|
||||
"sandbox should remain Ready=True (AgentCompleted) after a successful exec, never demote to Error"
|
||||
);
|
||||
assert!(
|
||||
tmp.path().join("hello.txt").is_file(),
|
||||
"hello.txt should appear in the granted share folder"
|
||||
);
|
||||
assert!(
|
||||
!backend
|
||||
.attribution
|
||||
@@ -3519,10 +3524,10 @@ mod lifecycle_tests {
|
||||
let share = tmp.path().to_string_lossy().replace('\\', "/");
|
||||
let hello = format!("{share}/hello.txt");
|
||||
let cmd = vec![
|
||||
"powershell".into(),
|
||||
"-NoProfile".into(),
|
||||
"-Command".into(),
|
||||
format!("Set-Content -LiteralPath {hello} -Value hi"),
|
||||
inbox_cmd(),
|
||||
"/d".into(),
|
||||
"/c".into(),
|
||||
format!(r#"echo hi>"{hello}""#),
|
||||
];
|
||||
let backend = MxcComputeBackend::new_mocked(MxcComputeConfig::default());
|
||||
|
||||
@@ -3530,13 +3535,15 @@ mod lifecycle_tests {
|
||||
let sb = with_policy(driver_sandbox_with_command("sb-pc", &share, cmd), policy);
|
||||
backend.create_sandbox(&sb).await.expect("create accepted");
|
||||
|
||||
let ready = wait_for(&backend, "sb-pc", |s| {
|
||||
ready_condition(s).is_some_and(|c| c.status == "True" && c.reason == "AgentRunning")
|
||||
let completed = wait_for(&backend, "sb-pc", |sandbox| {
|
||||
ready_condition(sandbox).is_some_and(|condition| {
|
||||
condition.status == "True" && condition.reason == "AgentCompleted"
|
||||
})
|
||||
})
|
||||
.await;
|
||||
assert!(
|
||||
ready.is_some(),
|
||||
"processContainer sandbox should self-report Ready=True"
|
||||
completed.is_some(),
|
||||
"processContainer sandbox should report Ready=True/AgentCompleted"
|
||||
);
|
||||
let recorded = crate::mxc::mock_recorded_config("sb-pc").expect("mock recorded config");
|
||||
assert!(
|
||||
@@ -3547,17 +3554,8 @@ mod lifecycle_tests {
|
||||
assert_eq!(recorded["ui"]["clipboard"], "none");
|
||||
assert_eq!(recorded["ui"]["injection"], false);
|
||||
|
||||
let host_path = tmp.path().join("hello.txt");
|
||||
let mut found = false;
|
||||
for _ in 0..100 {
|
||||
if host_path.exists() {
|
||||
found = true;
|
||||
break;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
}
|
||||
assert!(
|
||||
found,
|
||||
tmp.path().join("hello.txt").is_file(),
|
||||
"in-policy write should materialize under processContainer"
|
||||
);
|
||||
}
|
||||
@@ -3878,10 +3876,10 @@ mod lifecycle_tests {
|
||||
out_tmp.path().to_string_lossy().replace('\\', "/")
|
||||
);
|
||||
let cmd = vec![
|
||||
"powershell".into(),
|
||||
"-NoProfile".into(),
|
||||
"-Command".into(),
|
||||
format!("Set-Content -LiteralPath {out_path} -Value hi"),
|
||||
inbox_cmd(),
|
||||
"/d".into(),
|
||||
"/c".into(),
|
||||
format!(r#"echo hi>"{out_path}""#),
|
||||
];
|
||||
let backend = MxcComputeBackend::new_mocked(MxcComputeConfig::default());
|
||||
|
||||
@@ -3937,7 +3935,7 @@ mod lifecycle_tests {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let share = tmp.path().to_string_lossy().replace('\\', "/");
|
||||
let command = vec![
|
||||
"powershell".into(),
|
||||
inbox_powershell(),
|
||||
"-NoProfile".into(),
|
||||
"-Command".into(),
|
||||
format!("$null = '{share}'; Start-Sleep -Seconds 60"),
|
||||
@@ -3980,7 +3978,7 @@ mod lifecycle_tests {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let share = tmp.path().to_string_lossy().replace('\\', "/");
|
||||
let command = vec![
|
||||
"powershell".into(),
|
||||
inbox_powershell(),
|
||||
"-NoProfile".into(),
|
||||
"-Command".into(),
|
||||
format!("$null = '{share}'; Start-Sleep -Seconds 60"),
|
||||
@@ -4031,7 +4029,7 @@ mod lifecycle_tests {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let share = tmp.path().to_string_lossy().replace('\\', "/");
|
||||
let command = vec![
|
||||
"powershell".into(),
|
||||
inbox_powershell(),
|
||||
"-NoProfile".into(),
|
||||
"-Command".into(),
|
||||
format!("$null = '{share}'; Start-Sleep -Seconds 60"),
|
||||
|
||||
@@ -660,7 +660,15 @@ impl WxcExecInvoker {
|
||||
let cmd_norm = mock_normalize(&process.command_line);
|
||||
let in_policy = grants.iter().any(|g| !g.is_empty() && cmd_norm.contains(g));
|
||||
|
||||
let mut cmd = Command::new("cmd");
|
||||
let command_shell = std::env::var_os("COMSPEC")
|
||||
.unwrap_or_else(|| std::ffi::OsString::from(r"C:\Windows\System32\cmd.exe"));
|
||||
let mut cmd = Command::new(command_shell);
|
||||
cmd.env_clear();
|
||||
for entry in &process.env {
|
||||
if let Some((key, value)) = entry.split_once('=') {
|
||||
cmd.env(key, value);
|
||||
}
|
||||
}
|
||||
cmd.stdin(std::process::Stdio::null())
|
||||
.stdout(std::process::Stdio::inherit())
|
||||
.stderr(std::process::Stdio::inherit())
|
||||
@@ -975,6 +983,32 @@ mod tests {
|
||||
assert!(config.get("ui").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn mock_exec_uses_only_the_mxc_process_environment() {
|
||||
const KEY: &str = "OPENSHELL_MXC_MOCK_ENV_TEST";
|
||||
let workdir = tempfile::tempdir().expect("temporary workdir");
|
||||
let output = workdir.path().join("mock-env.txt");
|
||||
let process = MxcProcess {
|
||||
command_line: format!("echo %{KEY}%,%SystemRoot% 1> \"{}\"", output.display()),
|
||||
cwd: workdir.path().to_string_lossy().into_owned(),
|
||||
env: vec![format!("{KEY}=process-value")],
|
||||
timeout: 0,
|
||||
};
|
||||
|
||||
let mut child = WxcExecInvoker::mock_spawn_with_grants(
|
||||
&process,
|
||||
&[mock_normalize(&workdir.path().to_string_lossy())],
|
||||
)
|
||||
.expect("mock process should launch");
|
||||
let status = child.wait().await.expect("mock process should finish");
|
||||
|
||||
assert!(status.success());
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(output).expect("mock output").trim(),
|
||||
"process-value,%SystemRoot%"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn oneshot_config_json_emits_typed_ui_policy() {
|
||||
let filesystem = MxcFilesystem::default();
|
||||
|
||||
@@ -102,6 +102,8 @@ fn shipped_inference_policies_are_narrow_and_valid_after_rendering() {
|
||||
.replace("__OPENSHELL_DEMO_SHARE__", share)
|
||||
.replace("__OLLAMA_HOST__", "127.0.0.1")
|
||||
.replace("__OLLAMA_PORT__", "11434")
|
||||
.replace("__INFERENCE_HOST__", "integrate.api.nvidia.com")
|
||||
.replace("__INFERENCE_PORT__", "443")
|
||||
.replace("__CMD_EXE__", r"C:\Windows\System32\cmd.exe");
|
||||
let policy = parse_sandbox_policy(&rendered)
|
||||
.unwrap_or_else(|error| panic!("failed to parse rendered {name}: {error}"));
|
||||
@@ -156,6 +158,19 @@ fn shipped_runners_supply_sandbox_scoped_workload_configuration() {
|
||||
assert!(!cloud.contains("pass -ApiKey"));
|
||||
assert!(cloud.contains("nvidia/nemotron-3.5-lightning-30b-a3b"));
|
||||
assert!(!cloud.contains("nvidia/nvidia-nemotron-nano-9b-v2"));
|
||||
|
||||
let local = read_example("run-ollama-test.ps1");
|
||||
assert!(local.contains("[switch] $Mock"));
|
||||
assert!(local.contains("in-process wxc shim"));
|
||||
assert!(local.contains("does not provide MXC or AppContainer isolation"));
|
||||
|
||||
let cloud = read_example("run-inference-test.ps1");
|
||||
assert!(cloud.contains("[switch] $Mock"));
|
||||
assert!(cloud.contains("[string] $ApiUrl"));
|
||||
assert!(cloud.contains("$apiUri.IsLoopback"));
|
||||
assert!(cloud.contains("--noproxy"));
|
||||
assert!(cloud.contains("in-process wxc shim"));
|
||||
assert!(cloud.contains("does not provide MXC or AppContainer isolation"));
|
||||
}
|
||||
|
||||
#[cfg(target_os = "windows")]
|
||||
|
||||
@@ -0,0 +1,258 @@
|
||||
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
# Runs both shipped MXC inference demos through the real gateway and CLI
|
||||
# against the in-process wxc mock and a local HTTP API stub. This proves demo
|
||||
# wiring only; it is not evidence of MXC or AppContainer enforcement.
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidateSet("x86_64-pc-windows-msvc", "aarch64-pc-windows-msvc")]
|
||||
[string] $Target,
|
||||
|
||||
[string] $GatewayPath,
|
||||
[string] $CliPath,
|
||||
[string] $ArtifactRoot,
|
||||
[switch] $KeepArtifacts
|
||||
)
|
||||
|
||||
Set-StrictMode -Version Latest
|
||||
$ErrorActionPreference = "Stop"
|
||||
$PSNativeCommandUseErrorActionPreference = $false
|
||||
|
||||
if (-not [System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform([System.Runtime.InteropServices.OSPlatform]::Windows)) {
|
||||
throw "windows-mxc-inference-examples-e2e.ps1 requires Windows."
|
||||
}
|
||||
|
||||
$RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot "..\..")).Path
|
||||
$ExamplesRoot = Join-Path $RepoRoot "crates\openshell-driver-mxc\examples"
|
||||
$TargetDir = if ([string]::IsNullOrWhiteSpace($env:CARGO_TARGET_DIR)) {
|
||||
Join-Path $RepoRoot "target"
|
||||
} else {
|
||||
[System.IO.Path]::GetFullPath($env:CARGO_TARGET_DIR)
|
||||
}
|
||||
|
||||
if ([string]::IsNullOrWhiteSpace($GatewayPath)) {
|
||||
$GatewayPath = Join-Path $TargetDir "$Target\release\openshell-gateway.exe"
|
||||
}
|
||||
if ([string]::IsNullOrWhiteSpace($CliPath)) {
|
||||
$CliPath = Join-Path $TargetDir "$Target\release\openshell.exe"
|
||||
}
|
||||
$GatewayPath = [System.IO.Path]::GetFullPath($GatewayPath)
|
||||
$CliPath = [System.IO.Path]::GetFullPath($CliPath)
|
||||
|
||||
foreach ($artifact in @($GatewayPath, $CliPath)) {
|
||||
if (-not (Test-Path -LiteralPath $artifact -PathType Leaf)) {
|
||||
throw "required release artifact is missing: $artifact"
|
||||
}
|
||||
}
|
||||
|
||||
if ([string]::IsNullOrWhiteSpace($ArtifactRoot)) {
|
||||
$ArtifactRoot = if ([string]::IsNullOrWhiteSpace($env:RUNNER_TEMP)) {
|
||||
[System.IO.Path]::GetTempPath()
|
||||
} else {
|
||||
$env:RUNNER_TEMP
|
||||
}
|
||||
}
|
||||
$ArtifactRoot = [System.IO.Path]::GetFullPath($ArtifactRoot)
|
||||
$StageDir = [System.IO.Path]::GetFullPath((Join-Path $ArtifactRoot "openshell-mxc-inference-examples-e2e-$Target"))
|
||||
$expectedPrefix = $ArtifactRoot.TrimEnd('\', '/') + [System.IO.Path]::DirectorySeparatorChar
|
||||
if (-not $StageDir.StartsWith($expectedPrefix, [System.StringComparison]::OrdinalIgnoreCase)) {
|
||||
throw "refusing to use staging path outside artifact root: $StageDir"
|
||||
}
|
||||
if (Test-Path -LiteralPath $StageDir) {
|
||||
Remove-Item -LiteralPath $StageDir -Recurse -Force
|
||||
}
|
||||
New-Item -ItemType Directory -Path $StageDir | Out-Null
|
||||
|
||||
foreach ($fixture in @(
|
||||
"run-ollama-test.ps1",
|
||||
"mxc-ollama.toml",
|
||||
"ollama.yaml",
|
||||
"run-inference-test.ps1",
|
||||
"mxc-inference.toml",
|
||||
"inference.yaml"
|
||||
)) {
|
||||
Copy-Item -LiteralPath (Join-Path $ExamplesRoot $fixture) -Destination $StageDir
|
||||
}
|
||||
|
||||
function Get-AvailablePort {
|
||||
$listener = [System.Net.Sockets.TcpListener]::new([System.Net.IPAddress]::Loopback, 0)
|
||||
try {
|
||||
$listener.Start()
|
||||
return ([System.Net.IPEndPoint] $listener.LocalEndpoint).Port
|
||||
} finally {
|
||||
$listener.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
function Test-Listener([int] $Port) {
|
||||
$client = [System.Net.Sockets.TcpClient]::new()
|
||||
try {
|
||||
$pending = $client.BeginConnect("127.0.0.1", $Port, $null, $null)
|
||||
if (-not $pending.AsyncWaitHandle.WaitOne(250)) { return $false }
|
||||
$client.EndConnect($pending)
|
||||
return $true
|
||||
} catch {
|
||||
return $false
|
||||
} finally {
|
||||
$client.Dispose()
|
||||
}
|
||||
}
|
||||
|
||||
$ApiPort = Get-AvailablePort
|
||||
$MockToken = "openshell-ci-mock-token"
|
||||
$RequestLog = Join-Path $StageDir "mock-inference-requests.log"
|
||||
$ServerReady = Join-Path $StageDir "mock-inference.ready"
|
||||
$ServerOutLog = Join-Path $StageDir "mock-inference.out.log"
|
||||
$ServerErrLog = Join-Path $StageDir "mock-inference.err.log"
|
||||
$ServerScript = Join-Path $PSScriptRoot "windows-mxc-inference-stub.ps1"
|
||||
$serverProcess = $null
|
||||
$passed = $false
|
||||
$oldAppData = $env:APPDATA
|
||||
$oldLocalAppData = $env:LOCALAPPDATA
|
||||
$oldNvApiKey = $env:NV_API_KEY
|
||||
|
||||
try {
|
||||
$serverProcess = Start-Process -FilePath "powershell.exe" -ArgumentList @(
|
||||
"-NoProfile",
|
||||
"-ExecutionPolicy", "Bypass",
|
||||
"-File", "`"$ServerScript`"",
|
||||
"-Port", "$ApiPort",
|
||||
"-RequestLog", "`"$RequestLog`"",
|
||||
"-ReadyPath", "`"$ServerReady`"",
|
||||
"-ExpectedBearerToken", $MockToken
|
||||
) -PassThru -WindowStyle Hidden -RedirectStandardOutput $ServerOutLog -RedirectStandardError $ServerErrLog
|
||||
|
||||
$deadline = (Get-Date).AddSeconds(15)
|
||||
while ((Get-Date) -lt $deadline -and (-not (Test-Path -LiteralPath $ServerReady) -or -not (Test-Listener $ApiPort))) {
|
||||
if ($serverProcess.HasExited) {
|
||||
$details = (Get-Content $ServerOutLog, $ServerErrLog -ErrorAction SilentlyContinue) -join [Environment]::NewLine
|
||||
throw "mock inference server stopped before listening: $details"
|
||||
}
|
||||
Start-Sleep -Milliseconds 200
|
||||
}
|
||||
if (-not (Test-Listener $ApiPort)) {
|
||||
throw "mock inference server did not listen on port $ApiPort within 15 seconds"
|
||||
}
|
||||
|
||||
$env:APPDATA = Join-Path $StageDir "appdata"
|
||||
$env:LOCALAPPDATA = Join-Path $StageDir "localappdata"
|
||||
New-Item -ItemType Directory -Force -Path $env:APPDATA, $env:LOCALAPPDATA | Out-Null
|
||||
|
||||
$ollamaRunner = Join-Path $StageDir "run-ollama-test.ps1"
|
||||
$ollamaShare = Join-Path $StageDir "ollama-share"
|
||||
$ollamaArgs = @(
|
||||
"-NoProfile",
|
||||
"-ExecutionPolicy", "Bypass",
|
||||
"-File", $ollamaRunner,
|
||||
"-Mock",
|
||||
"-GatewayPath", $GatewayPath,
|
||||
"-CliPath", $CliPath,
|
||||
"-ShareDir", $ollamaShare,
|
||||
"-OllamaPort", "$ApiPort",
|
||||
"-Model", "openshell-ci-mock",
|
||||
"-Prompt", "Return the CI mock response.",
|
||||
"-KeepArtifacts"
|
||||
)
|
||||
$output = & powershell.exe @ollamaArgs 2>&1
|
||||
$exitCode = $LASTEXITCODE
|
||||
$output | ForEach-Object { Write-Host $_ }
|
||||
if ($exitCode -ne 0) {
|
||||
throw "shipped Ollama demo failed in mock mode (exit $exitCode)"
|
||||
}
|
||||
|
||||
$ollamaResultDir = Get-ChildItem -LiteralPath $StageDir -Directory -Filter "results-ollama-*" |
|
||||
Sort-Object LastWriteTimeUtc -Descending |
|
||||
Select-Object -First 1
|
||||
if ($null -eq $ollamaResultDir) { throw "Ollama demo did not produce a results directory" }
|
||||
|
||||
$ollamaSummary = Get-Content -LiteralPath (Join-Path $ollamaResultDir.FullName "summary.txt") -Raw
|
||||
if ($ollamaSummary -notmatch '(?m)^verdict=PASS\s*$' -or $ollamaSummary -notmatch '(?m)^mode=mock-wiring\s*$') {
|
||||
throw "Ollama demo summary did not report a mock-wiring PASS: $ollamaSummary"
|
||||
}
|
||||
$ollamaResponse = Get-Content -LiteralPath (Join-Path $ollamaResultDir.FullName "ollama-response.json") -Raw | ConvertFrom-Json
|
||||
if ($ollamaResponse.response -ne "Hello from the CI mock.") {
|
||||
throw "Ollama demo did not retain the expected mock completion"
|
||||
}
|
||||
|
||||
$env:NV_API_KEY = $MockToken
|
||||
$cloudRunner = Join-Path $StageDir "run-inference-test.ps1"
|
||||
$cloudShare = Join-Path $StageDir "cloud-share"
|
||||
$cloudApiUrl = "http://127.0.0.1:$ApiPort/v1/chat/completions"
|
||||
$cloudArgs = @(
|
||||
"-NoProfile",
|
||||
"-ExecutionPolicy", "Bypass",
|
||||
"-File", $cloudRunner,
|
||||
"-Mock",
|
||||
"-GatewayPath", $GatewayPath,
|
||||
"-CliPath", $CliPath,
|
||||
"-ShareDir", $cloudShare,
|
||||
"-ApiUrl", $cloudApiUrl,
|
||||
"-Model", "openshell-ci-mock",
|
||||
"-Prompt", "Return the CI mock response.",
|
||||
"-KeepArtifacts"
|
||||
)
|
||||
$output = & powershell.exe @cloudArgs 2>&1
|
||||
$exitCode = $LASTEXITCODE
|
||||
$output | ForEach-Object { Write-Host $_ }
|
||||
if ($exitCode -ne 0) {
|
||||
throw "shipped cloud-inference demo failed in mock mode (exit $exitCode)"
|
||||
}
|
||||
|
||||
$cloudResultDir = Get-ChildItem -LiteralPath $StageDir -Directory -Filter "results-inference-*" |
|
||||
Sort-Object LastWriteTimeUtc -Descending |
|
||||
Select-Object -First 1
|
||||
if ($null -eq $cloudResultDir) { throw "cloud-inference demo did not produce a results directory" }
|
||||
|
||||
$cloudSummary = Get-Content -LiteralPath (Join-Path $cloudResultDir.FullName "summary.txt") -Raw
|
||||
if ($cloudSummary -notmatch '(?m)^verdict=PASS\s*$' -or $cloudSummary -notmatch '(?m)^mode=mock-wiring\s*$') {
|
||||
throw "cloud-inference demo summary did not report a mock-wiring PASS: $cloudSummary"
|
||||
}
|
||||
$cloudResponse = Get-Content -LiteralPath (Join-Path $cloudResultDir.FullName "inference-response.json") -Raw | ConvertFrom-Json
|
||||
if ($cloudResponse.choices[0].message.content -ne "Hello from the CI mock.") {
|
||||
throw "cloud-inference demo did not retain the expected mock completion"
|
||||
}
|
||||
|
||||
$requestLines = @([System.IO.File]::ReadAllLines($RequestLog))
|
||||
$expectedRequests = @(
|
||||
@{ Pattern = '^GET /api/tags HTTP/\S+ authorization=absent$'; Count = 2; Description = 'host prerequisite and sandbox Ollama tag requests' },
|
||||
@{ Pattern = '^POST /api/generate HTTP/\S+ authorization=absent$'; Count = 1; Description = 'sandbox Ollama generation request' },
|
||||
@{ Pattern = '^POST /v1/chat/completions HTTP/\S+ authorization=synthetic$'; Count = 1; Description = 'sandbox cloud-inference request with the synthetic CI credential' }
|
||||
)
|
||||
foreach ($expected in $expectedRequests) {
|
||||
$actualCount = @($requestLines | Where-Object { $_ -match $expected.Pattern }).Count
|
||||
if ($actualCount -ne $expected.Count) {
|
||||
throw "mock inference server observed $actualCount $($expected.Description); expected $($expected.Count): $($requestLines -join '; ')"
|
||||
}
|
||||
}
|
||||
|
||||
$passed = $true
|
||||
Write-Host "MXC inference examples mock E2E passed for $Target"
|
||||
} catch {
|
||||
Write-Host "MXC inference examples mock E2E failed: $($_.Exception.Message)" -ForegroundColor Red
|
||||
Get-ChildItem -LiteralPath $StageDir -File -Recurse -Include "*.log", "summary.txt" -ErrorAction SilentlyContinue |
|
||||
ForEach-Object {
|
||||
Write-Host "--- $($_.FullName) ---"
|
||||
Get-Content -LiteralPath $_.FullName -ErrorAction SilentlyContinue | ForEach-Object { Write-Host $_ }
|
||||
}
|
||||
throw
|
||||
} finally {
|
||||
$env:APPDATA = $oldAppData
|
||||
$env:LOCALAPPDATA = $oldLocalAppData
|
||||
if ([string]::IsNullOrWhiteSpace($oldNvApiKey)) {
|
||||
Remove-Item Env:NV_API_KEY -ErrorAction SilentlyContinue
|
||||
} else {
|
||||
$env:NV_API_KEY = $oldNvApiKey
|
||||
}
|
||||
if ($serverProcess -and -not $serverProcess.HasExited) {
|
||||
Stop-Process -Id $serverProcess.Id -Force -ErrorAction SilentlyContinue
|
||||
try { [void] $serverProcess.WaitForExit(5000) } catch {}
|
||||
}
|
||||
if ($passed -and -not $KeepArtifacts -and (Test-Path -LiteralPath $StageDir)) {
|
||||
Remove-Item -LiteralPath $StageDir -Recurse -Force
|
||||
} else {
|
||||
Write-Host "MXC inference example artifacts: $StageDir"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[ValidateRange(1, 65535)]
|
||||
[int] $Port,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string] $RequestLog,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string] $ReadyPath,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string] $ExpectedBearerToken
|
||||
)
|
||||
|
||||
Set-StrictMode -Version Latest
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
$utf8 = [System.Text.UTF8Encoding]::new($false)
|
||||
$listener = [System.Net.Sockets.TcpListener]::new([System.Net.IPAddress]::Loopback, $Port)
|
||||
$listener.Start()
|
||||
[System.IO.File]::WriteAllText($ReadyPath, "$Port`r`n", $utf8)
|
||||
|
||||
try {
|
||||
while ($true) {
|
||||
$client = $listener.AcceptTcpClient()
|
||||
try {
|
||||
$stream = $client.GetStream()
|
||||
$reader = [System.IO.StreamReader]::new(
|
||||
$stream,
|
||||
[System.Text.Encoding]::ASCII,
|
||||
$false,
|
||||
1024,
|
||||
$true
|
||||
)
|
||||
$requestLine = $reader.ReadLine()
|
||||
if ([string]::IsNullOrWhiteSpace($requestLine)) { continue }
|
||||
|
||||
$contentLength = 0
|
||||
$authorization = ""
|
||||
while ($true) {
|
||||
$line = $reader.ReadLine()
|
||||
if ([string]::IsNullOrEmpty($line)) { break }
|
||||
if ($line -match '^Content-Length:\s*(\d+)\s*$') {
|
||||
$contentLength = [int] $Matches[1]
|
||||
} elseif ($line -match '^Authorization:\s*(.+)\s*$') {
|
||||
$authorization = $Matches[1]
|
||||
}
|
||||
}
|
||||
if ($contentLength -gt 0) {
|
||||
$buffer = New-Object char[] $contentLength
|
||||
$read = 0
|
||||
while ($read -lt $contentLength) {
|
||||
$count = $reader.Read($buffer, $read, $contentLength - $read)
|
||||
if ($count -le 0) { break }
|
||||
$read += $count
|
||||
}
|
||||
}
|
||||
|
||||
$parts = $requestLine.Split(' ')
|
||||
$path = if ($parts.Count -ge 2) { $parts[1] } else { "/" }
|
||||
$authorizationStatus = if ([string]::IsNullOrWhiteSpace($authorization)) {
|
||||
"absent"
|
||||
} elseif ($authorization -ceq "Bearer $ExpectedBearerToken") {
|
||||
"synthetic"
|
||||
} else {
|
||||
"mismatch"
|
||||
}
|
||||
[System.IO.File]::AppendAllText(
|
||||
$RequestLog,
|
||||
"$requestLine authorization=$authorizationStatus`r`n",
|
||||
$utf8
|
||||
)
|
||||
switch ($path) {
|
||||
"/api/tags" {
|
||||
$status = "200 OK"
|
||||
$body = '{"models":[{"name":"openshell-ci-mock"}]}'
|
||||
}
|
||||
"/api/generate" {
|
||||
$status = "200 OK"
|
||||
$body = '{"model":"openshell-ci-mock","response":"Hello from the CI mock.","done":true}'
|
||||
}
|
||||
"/v1/chat/completions" {
|
||||
if ($authorizationStatus -eq "synthetic") {
|
||||
$status = "200 OK"
|
||||
$body = '{"choices":[{"message":{"role":"assistant","content":"Hello from the CI mock."}}]}'
|
||||
} else {
|
||||
$status = "401 Unauthorized"
|
||||
$body = '{"error":{"message":"invalid mock credential"}}'
|
||||
}
|
||||
}
|
||||
default {
|
||||
$status = "404 Not Found"
|
||||
$body = '{"error":"not found"}'
|
||||
}
|
||||
}
|
||||
|
||||
$bodyBytes = $utf8.GetBytes($body)
|
||||
$headers = "HTTP/1.1 $status`r`nContent-Type: application/json`r`nContent-Length: $($bodyBytes.Length)`r`nConnection: close`r`n`r`n"
|
||||
$headerBytes = [System.Text.Encoding]::ASCII.GetBytes($headers)
|
||||
$stream.Write($headerBytes, 0, $headerBytes.Length)
|
||||
$stream.Write($bodyBytes, 0, $bodyBytes.Length)
|
||||
$stream.Flush()
|
||||
} finally {
|
||||
$client.Dispose()
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
$listener.Stop()
|
||||
}
|
||||
@@ -108,3 +108,13 @@ run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File crates/opensh
|
||||
description = "Run MXC Tier-3 e2e scenario runner in mock/wiring-only mode (no real wxc-exec required)"
|
||||
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
|
||||
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File crates/openshell-driver-mxc/examples/run-mxc-e2e.ps1 -Mock"
|
||||
|
||||
["windows:e2e:mxc:inference-mock:x64"]
|
||||
description = "Run the shipped MXC inference demos against the in-process wxc mock and a local API stub on Windows x64"
|
||||
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
|
||||
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-mxc-inference-examples-e2e.ps1 -Target x86_64-pc-windows-msvc"
|
||||
|
||||
["windows:e2e:mxc:inference-mock:arm64"]
|
||||
description = "Run the shipped MXC inference demos against the in-process wxc mock and a local API stub on Windows ARM64"
|
||||
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
|
||||
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-mxc-inference-examples-e2e.ps1 -Target aarch64-pc-windows-msvc"
|
||||
|
||||
Reference in New Issue
Block a user