mirror of
https://github.com/NVIDIA/OpenShell.git
synced 2026-10-22 06:19:47 +08:00
fix(mxc): address Windows qualification review comments
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
This commit is contained in:
@@ -370,6 +370,25 @@ validation; GitHub Actions does not re-run it after the full suite.
|
||||
|
||||
## Test Accounting Guidance
|
||||
|
||||
For a separate qualified Windows test host, use
|
||||
`tasks/scripts/test-mxc-remote.ps1 -HostName <host> -UserName <user>`.
|
||||
It builds locally with the existing Windows tasks, detects native remote
|
||||
architecture through Windows system configuration (not an emulated SSH shell),
|
||||
caches uploads by SHA256, and runs the unchanged real E2E harness. Results and
|
||||
host probes return to `target/windows-remote-results`, including nonzero test
|
||||
outcomes. The remote host needs SSH, OpenSSL, and MXC, not a compiler toolchain.
|
||||
Package the matching `libz3.dll` from the target-specific Cargo cache alongside
|
||||
the gateway: Windows prebuilt Z3 uses an import library, not static linkage.
|
||||
The `windows:build:*` tasks now stage the pinned DLL, check PE architecture,
|
||||
reject conflicting cache hashes, and verify the copied hash. Local MXC E2E
|
||||
requires the adjacent DLL and reports a preflight error if it is missing.
|
||||
Use `-Z3DllPath` when discovery is ambiguous. The native Visual C++ runtime
|
||||
must be present on the remote host. Verify DLL architecture and the gateway
|
||||
loader before E2E; do not count missing-DLL crashes as MXC enforcement failures.
|
||||
Verify the SSH host key beforehand. Never send the private SSH key or bundle
|
||||
the harness signing-key directories. `-SkipBuild` is an explicit stale-artifact
|
||||
opt-in, not evidence that artifacts match current sources. See the MXC driver
|
||||
README for the command and remote dependency contract.
|
||||
For MXC validation, also run `windows:e2e:mxc`
|
||||
after the native release build. The harness uses .NET port discovery, disposable
|
||||
TOML and CLI registration, and owner-only Ed25519 keys generated by OpenSSL on
|
||||
|
||||
@@ -135,6 +135,54 @@ not required; driver-owned guest TLS fields are rejected.
|
||||
|
||||
## Validation
|
||||
|
||||
Windows release build tasks stage the pinned, target-matching `libz3.dll` beside
|
||||
the gateway and verify its architecture and copied SHA256. Conflicting cached
|
||||
DLLs fail the build instead of selecting an arbitrary runtime. Local E2E requires
|
||||
this adjacent DLL; rebuild for the native architecture if preflight reports it
|
||||
missing.
|
||||
|
||||
### Remote Windows host
|
||||
|
||||
Build on a Windows MSVC development machine and run the unchanged real E2E
|
||||
harness on a separate MXC host:
|
||||
|
||||
```powershell
|
||||
.\tasks\scripts\test-mxc-remote.ps1 -HostName 172.16.178.137 -UserName test `
|
||||
-WxcExecPath 'C:\Tools\MXC\wxc-exec.exe'
|
||||
```
|
||||
|
||||
Install the public SSH key on the target and verify its host key before running.
|
||||
The script defaults to `~/.ssh/openshell-mxc-test`; override `-IdentityFile` as
|
||||
needed. Both SSH and SCP must be on PATH locally. The target needs Windows
|
||||
OpenSSH Server, OpenSSL on its SSH-session PATH, and a qualified MXC runtime,
|
||||
but does not need Rust, MSVC, mise, or a source checkout.
|
||||
|
||||
The script detects native Windows architecture independently of the SSH shell,
|
||||
uses the existing `windows:build:*` task, and validates the executable PE
|
||||
architecture before testing. `-BuildDirectory` selects a dedicated Cargo cache.
|
||||
`-SkipBuild` reuses existing artifacts without claiming they match current
|
||||
sources. `-Scenario` selects an existing harness scenario.
|
||||
|
||||
The package includes `libz3.dll` from the target-specific Cargo build cache.
|
||||
Windows prebuilt Z3 uses a dynamic runtime DLL. Supply `-Z3DllPath` if multiple
|
||||
different cached versions exist or when using a custom Z3 build. The remote
|
||||
host needs the native Visual C++ runtime; the runner validates DLL architecture
|
||||
and gateway startup before invoking E2E.
|
||||
|
||||
The example gateway explicitly enables caller driver config so tests can submit
|
||||
their workload command and directory. MXC does not implement external-resource
|
||||
label admission. The temporary outer-fence assertions permit compatibility
|
||||
testing but do not qualify exclusive network mediation or live revocation.
|
||||
|
||||
Uploads are cached by SHA256 and verified before execution. Each run has an
|
||||
isolated remote directory under `%USERPROFILE%\openshell-mxc-tests`; old runs
|
||||
are retained. Full logs, host capabilities, source status, artifact hashes, and
|
||||
the results ZIP return to `target/windows-remote-results` in this checkout.
|
||||
Signing-key directories are not downloaded. Test failures and incomplete
|
||||
coverage remain nonzero exits. Missing host-loopback evidence is not a pass,
|
||||
and a qualified host does not close the outer-fence implementation gaps above.
|
||||
|
||||
### Local Windows host
|
||||
Run the Windows build lane on a native Windows MSVC host:
|
||||
|
||||
```powershell
|
||||
|
||||
@@ -646,6 +646,9 @@ try {
|
||||
if (-not (Test-Path $policyDir)) {
|
||||
throw "e2e-policies/ directory not found at $policyDir"
|
||||
}
|
||||
if (-not (Test-Path -LiteralPath (Join-Path $BinaryDir 'libz3.dll') -PathType Leaf)) {
|
||||
throw "Missing libz3.dll beside the gateway. Re-run windows:build for this architecture, or package the matching Z3 runtime with these binaries."
|
||||
}
|
||||
|
||||
# Capture the pristine TOML once; every scenario renders a fresh copy from this.
|
||||
$tomlBase = Get-Content $tomlTemplate -Raw
|
||||
|
||||
@@ -165,3 +165,75 @@ if (Test-Path -LiteralPath $cliStateRoot) { throw "temporary state not removed"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn windows_build_stages_only_unambiguous_target_matching_z3() {
|
||||
// Synthetic PE headers exercise packaging checks, not MXC containment.
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let script = r#"
|
||||
$ErrorActionPreference = 'Stop'
|
||||
Set-StrictMode -Version Latest
|
||||
$tokens = $null
|
||||
$errors = $null
|
||||
$ast = [System.Management.Automation.Language.Parser]::ParseFile(
|
||||
$env:OPENSHELL_BUILD_SCRIPT, [ref]$tokens, [ref]$errors)
|
||||
if ($errors.Count) { throw "build script parse errors: $($errors.Message -join '; ')" }
|
||||
$function = $ast.Find({ param($node)
|
||||
$node -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $node.Name -eq 'Stage-Z3Runtime'
|
||||
}, $true)
|
||||
if ($null -eq $function) { throw 'missing staging function' }
|
||||
Invoke-Expression $function.Extent.Text
|
||||
$TargetDir = $env:OPENSHELL_TEST_ROOT
|
||||
$PrebuiltZ3Version = '5.1.0'
|
||||
$env:Z3_LIBRARY_PATH_OVERRIDE = $null
|
||||
$release = Join-Path $TargetDir 'aarch64-pc-windows-msvc/release'
|
||||
$sourceDir = Join-Path $release 'build/z3-sys-first/out/z3-5.1.0/bin'
|
||||
New-Item -ItemType Directory -Path $sourceDir -Force | Out-Null
|
||||
$source = Join-Path $sourceDir 'libz3.dll'
|
||||
$bytes = New-Object byte[] 128
|
||||
$bytes[0] = 0x4D; $bytes[1] = 0x5A; $bytes[60] = 64
|
||||
$bytes[64] = 0x50; $bytes[65] = 0x45
|
||||
$bytes[68] = 0x64; $bytes[69] = 0xAA
|
||||
[IO.File]::WriteAllBytes($source, $bytes)
|
||||
Stage-Z3Runtime 'aarch64-pc-windows-msvc'
|
||||
$destination = Join-Path $release 'libz3.dll'
|
||||
if ((Get-FileHash $destination).Hash -ne (Get-FileHash $source).Hash) { throw 'copy mismatch' }
|
||||
# A stale adjacent DLL must be overwritten from the pinned cache.
|
||||
[IO.File]::WriteAllBytes($destination, (New-Object byte[] 128))
|
||||
Stage-Z3Runtime 'aarch64-pc-windows-msvc'
|
||||
if ((Get-FileHash $destination).Hash -ne (Get-FileHash $source).Hash) { throw 'stale DLL retained' }
|
||||
$bytes[68] = 0x64; $bytes[69] = 0x86
|
||||
[IO.File]::WriteAllBytes($source, $bytes)
|
||||
$rejected = $false
|
||||
try { Stage-Z3Runtime 'aarch64-pc-windows-msvc' } catch {
|
||||
if ($_.Exception.Message -notmatch 'architecture') { throw }
|
||||
$rejected = $true
|
||||
}
|
||||
if (-not $rejected) { throw 'wrong architecture accepted' }
|
||||
$secondDir = Join-Path $release 'build/z3-sys-second/out/z3-5.1.0/bin'
|
||||
New-Item -ItemType Directory -Path $secondDir -Force | Out-Null
|
||||
$bytes[69] = 0xAA
|
||||
[IO.File]::WriteAllBytes((Join-Path $secondDir 'libz3.dll'), $bytes)
|
||||
$rejected = $false
|
||||
try { Stage-Z3Runtime 'aarch64-pc-windows-msvc' } catch {
|
||||
if ($_.Exception.Message -notmatch 'Ambiguous') { throw }
|
||||
$rejected = $true
|
||||
}
|
||||
if (-not $rejected) { throw 'conflicting runtimes accepted' }
|
||||
"#;
|
||||
let output = Command::new("powershell.exe")
|
||||
.args(["-NoProfile", "-Command", script])
|
||||
.env(
|
||||
"OPENSHELL_BUILD_SCRIPT",
|
||||
examples().join("../../../tasks/scripts/windows-msvc.ps1"),
|
||||
)
|
||||
.env("OPENSHELL_TEST_ROOT", root.path())
|
||||
.output()
|
||||
.unwrap();
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"Z3 staging regression failed:\n{}\n{}",
|
||||
String::from_utf8_lossy(&output.stdout),
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
}
|
||||
|
||||
@@ -916,6 +916,8 @@ async fn pc_https_egress_reads_injected_ca_bundle() {
|
||||
let _ = rustls::crypto::aws_lc_rs::default_provider().install_default();
|
||||
let config = MxcComputeConfig {
|
||||
wxc_exec_path: wxc.to_string_lossy().into_owned(),
|
||||
// This fixture intentionally submits a caller-selected curl command.
|
||||
allow_driver_config: true,
|
||||
..Default::default()
|
||||
};
|
||||
let backend = MxcComputeBackend::new(openshell_core::config::DEFAULT_GATEWAY_NAME, config);
|
||||
|
||||
@@ -74,6 +74,7 @@ telemetry = ["openshell-core/telemetry", "openshell-server/telemetry"]
|
||||
## `rust:verify:defaults-without-telemetry`.
|
||||
defaults-without-telemetry = ["in-tree-compute-drivers"]
|
||||
vendored-z3 = ["openshell-server/vendored-z3"]
|
||||
prebuilt-z3 = ["openshell-server/prebuilt-z3"]
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
|
||||
@@ -58,6 +58,7 @@ $PrebuiltZ3ServerFeatures = "--features openshell-server/prebuilt-z3,openshell-p
|
||||
$PrebuiltZ3Version = "5.1.0"
|
||||
$Z3WorkspaceFeatures = $PrebuiltZ3WorkspaceFeatures
|
||||
$Z3ServerFeatures = $PrebuiltZ3ServerFeatures
|
||||
$Z3GatewayFeatures = "--features prebuilt-z3"
|
||||
|
||||
function Get-VsInstallRoots {
|
||||
$programFiles = @(
|
||||
@@ -514,6 +515,39 @@ function Invoke-Build([string] $RustTarget) {
|
||||
-RustTarget $RustTarget `
|
||||
-CargoArgs "cargo build --release --target $RustTarget --bin openshell-gateway --bin openshell --bin openshell-supervisor --bin openshell-windows-sandbox $Z3WorkspaceFeatures" `
|
||||
-LogName "build-$RustTarget-release.log"
|
||||
Stage-Z3Runtime $RustTarget
|
||||
}
|
||||
|
||||
function Stage-Z3Runtime([string] $RustTarget) {
|
||||
$release = Join-Path $TargetDir "$RustTarget/release"
|
||||
if ($env:Z3_LIBRARY_PATH_OVERRIDE) {
|
||||
$candidates = @(Get-Item -LiteralPath (Join-Path $env:Z3_LIBRARY_PATH_OVERRIDE 'libz3.dll') -ErrorAction SilentlyContinue)
|
||||
} else {
|
||||
# Only use the pinned release from this target's build cache. Never pick
|
||||
# the newest arbitrary DLL, or trust a stale adjacent runtime.
|
||||
$candidates = @(Get-ChildItem -Path (Join-Path $release "build/z3-sys-*/out/z3-$PrebuiltZ3Version/bin/libz3.dll") -File -ErrorAction SilentlyContinue)
|
||||
}
|
||||
if ($candidates.Count -eq 0) { throw "Matching libz3.dll not found for $RustTarget." }
|
||||
$hashes = @($candidates | ForEach-Object { (Get-FileHash -LiteralPath $_.FullName -Algorithm SHA256).Hash } | Select-Object -Unique)
|
||||
if ($hashes.Count -ne 1) { throw "Ambiguous Z3 runtimes for $RustTarget; use a clean target cache or Z3_LIBRARY_PATH_OVERRIDE." }
|
||||
$source = $candidates[0].FullName
|
||||
$reader = [IO.BinaryReader]::new([IO.File]::OpenRead($source))
|
||||
try {
|
||||
if ($reader.ReadUInt16() -ne 0x5A4D) { throw "Invalid Z3 DLL: $source" }
|
||||
$reader.BaseStream.Position = 0x3C
|
||||
$offset = $reader.ReadInt32()
|
||||
if ($offset -lt 64 -or $offset -gt $reader.BaseStream.Length - 6) { throw 'Invalid Z3 PE offset.' }
|
||||
$reader.BaseStream.Position = $offset
|
||||
if ($reader.ReadUInt32() -ne 0x00004550) { throw 'Invalid Z3 PE signature.' }
|
||||
$expected = if ($RustTarget -eq 'aarch64-pc-windows-msvc') { 0xAA64 } else { 0x8664 }
|
||||
if ($reader.ReadUInt16() -ne $expected) { throw "Z3 DLL architecture does not match $RustTarget." }
|
||||
} finally { $reader.Dispose() }
|
||||
$destination = Join-Path $release 'libz3.dll'
|
||||
if ([IO.Path]::GetFullPath($source) -ne [IO.Path]::GetFullPath($destination)) {
|
||||
Copy-Item -LiteralPath $source -Destination $destination -Force
|
||||
}
|
||||
if ((Get-FileHash -LiteralPath $destination -Algorithm SHA256).Hash -ne $hashes[0]) { throw 'Staged Z3 DLL hash mismatch.' }
|
||||
Write-Host "==> Staged $destination (SHA256=$($hashes[0]))"
|
||||
}
|
||||
|
||||
function Invoke-Test([string] $RustTarget) {
|
||||
@@ -542,7 +576,7 @@ function Invoke-UnsupportedContractTests([string] $RustTarget) {
|
||||
foreach ($test in $tests) {
|
||||
Invoke-VsCargo `
|
||||
-RustTarget $RustTarget `
|
||||
-CargoArgs "cargo test -p openshell-gateway --target $RustTarget $test $Z3WorkspaceFeatures" `
|
||||
-CargoArgs "cargo test -p openshell-gateway --target $RustTarget $test $Z3GatewayFeatures" `
|
||||
-LogName "test-$RustTarget-unsupported-$test.log"
|
||||
}
|
||||
|
||||
@@ -551,7 +585,7 @@ function Invoke-UnsupportedContractTests([string] $RustTarget) {
|
||||
$variant = if ($features) { $features.Replace(",", "-") } else { "protocol-only" }
|
||||
Invoke-VsCargo `
|
||||
-RustTarget $RustTarget `
|
||||
-CargoArgs "cargo test -p openshell-gateway --lib --target $RustTarget --no-default-features $featureArgs $Z3ServerFeatures" `
|
||||
-CargoArgs "cargo test -p openshell-gateway --lib --target $RustTarget --no-default-features $featureArgs $Z3GatewayFeatures" `
|
||||
-LogName "test-$RustTarget-selective-$variant.log"
|
||||
}
|
||||
}
|
||||
@@ -618,6 +652,7 @@ if ($Action -in @("check", "lint", "build", "build-mxc-fixtures", "test", "test-
|
||||
$z3Features = Configure-Z3
|
||||
$Z3WorkspaceFeatures = $z3Features.WorkspaceFeatures
|
||||
$Z3ServerFeatures = $z3Features.ServerFeatures
|
||||
$Z3GatewayFeatures = if ($z3Features.WorkspaceFeatures) { "--features prebuilt-z3" } else { "" }
|
||||
$env:LIBCLANG_PATH = Resolve-LibclangPath
|
||||
Add-PathEntry $env:LIBCLANG_PATH
|
||||
Write-Host "==> LIBCLANG_PATH=$env:LIBCLANG_PATH"
|
||||
|
||||
Reference in New Issue
Block a user