From 7298a0161644debd3e470b58da5914336ebdca3f Mon Sep 17 00:00:00 2001 From: Shailendra Singh Date: Thu, 16 Jul 2026 20:46:24 -0700 Subject: [PATCH] feat(windows): add native ARM64 test support Signed-off-by: Shailendra Singh Signed-off-by: Akber Raza --- .../build-openshell-mxc-windows/SKILL.md | 24 +++++++--- .../build-openshell-mxc-windows/reference.md | 6 +++ architecture/windows-msvc-build.md | 18 ++++++-- tasks/scripts/windows-msvc.ps1 | 46 ++++++++++++++----- tasks/windows.toml | 10 ++++ 5 files changed, 82 insertions(+), 22 deletions(-) diff --git a/.agents/skills/build-openshell-mxc-windows/SKILL.md b/.agents/skills/build-openshell-mxc-windows/SKILL.md index 6dda48373..4d546e6ae 100644 --- a/.agents/skills/build-openshell-mxc-windows/SKILL.md +++ b/.agents/skills/build-openshell-mxc-windows/SKILL.md @@ -18,7 +18,7 @@ In scope: - Audit and cfg-gate Unix-only dependencies in `openshell-server` and shared crates so the workspace compiles to Windows MSVC - Keep Docker, Kubernetes, Podman, and VM compute crates in the Windows build graph as stubs that return "unsupported" at runtime - Add a Windows-specific mise task lane (`windows:*`) that wraps MSVC builds without changing the Linux `mise run ci` path -- `cargo test` compiles and passes on Windows for non-Linux-gated tests +- `cargo test` compiles and passes on a native x64 or ARM64 Windows host for non-Linux-gated tests - Scaffold an ARM64 CI workflow (job definition only; runner provisioning is operator work) Out of scope (defer to follow-on skills): @@ -39,7 +39,8 @@ The skill targets a **Windows 11 host with CurrentBuild ≥ 26100**. Because the | Requirement | Check | Install hint | |---|---|---| | Windows 11 build ≥ 26100 | `[System.Environment]::OSVersion.Version` | OS update | -| Visual Studio 2022 Build Tools with **MSVC v143** + **Windows 11 SDK** | `where cl.exe` from a VS Developer PowerShell | https://visualstudio.microsoft.com/downloads/ | +| Visual Studio 2022 or newer with **MSVC v143** + **Windows 11 SDK** | `where.exe cl.exe` from a VS Developer PowerShell | Include the x64/x86 and ARM64 C++ tools. | +| Visual C++ Clang and CMake tools | `where.exe clang.exe`; `where.exe cmake.exe` | `bindgen` needs host-native `libclang.dll`; bundled Z3 uses CMake and MSBuild. | | Rust ≥ 1.88 via rustup with MSVC targets | `rustc --version` | `winget install Rustlang.Rustup` | | mise CLI for task orchestration only | `mise --version` | https://mise.jdx.dev/installing-mise.html | | Git ≥ 2.40 | `git --version` | `winget install Git.Git` | @@ -72,7 +73,7 @@ The skill runs the following checklist top-to-bottom. Each step is idempotent; r [ ] Step 6: mise check on x86_64-pc-windows-msvc [ ] Step 7: mise check on aarch64-pc-windows-msvc [ ] Step 8: mise build --release (both targets) -[ ] Step 9: mise test on x86_64-pc-windows-msvc (native run) +[ ] Step 9: mise test on the host's native MSVC target [ ] Step 10: Validate $env:OPENSHELL_WXC_EXEC_PATH (informational) [ ] Step 11: Scaffold ARM64 CI workflow [ ] Step 12: Commit and report artifacts @@ -176,7 +177,9 @@ The task file must expose these commands: | `windows:build:x64` | Release-build `openshell-gateway` and `openshell` for x64 | | `windows:build:arm64` | Release-build `openshell-gateway` and `openshell` for ARM64 | | `windows:test:x64` | Native x64 `cargo test --workspace --no-fail-fast`, excluding unsupported driver packages as top-level workspace targets | +| `windows:test:arm64` | Native ARM64 `cargo test --workspace --no-fail-fast` with the same exclusions | | `windows:test:unsupported:x64` | Focused server/runtime tests that assert unsupported Windows driver contracts without building standalone driver binaries | +| `windows:test:unsupported:arm64` | The same focused contracts on a native ARM64 host | | `windows:ci` | Ordered Windows check/build/test/unsupported-contract/artifact lane | The PowerShell wrapper must: @@ -231,13 +234,22 @@ dumpbin /HEADERS "$fork\target\x86_64-pc-windows-msvc\release\openshell-gateway. Expected machine values: `x64` for `x86_64-pc-windows-msvc`, `ARM64` for `aarch64-pc-windows-msvc`. -### Step 9: mise test on x86_64-pc-windows-msvc (native run) +### Step 9: mise test on the native Windows architecture ```powershell -mise run --skip-tools windows:test:x64 -mise run --skip-tools windows:test:unsupported:x64 +$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture +if ($arch -eq [System.Runtime.InteropServices.Architecture]::Arm64) { + mise run --skip-tools windows:test:arm64 + mise run --skip-tools windows:test:unsupported:arm64 +} else { + mise run --skip-tools windows:test:x64 + mise run --skip-tools windows:test:unsupported:x64 +} ``` +The wrapper rejects test targets that do not match the host architecture. This +keeps ARM64 results native and avoids reporting x64 emulation as ARM64 coverage. + Failures fall into three buckets: 1. **Linux-only test** — gate with `#[cfg(not(target_os = "windows"))]` and add a short comment describing the Windows-equivalent test that will eventually replace it. diff --git a/.agents/skills/build-openshell-mxc-windows/reference.md b/.agents/skills/build-openshell-mxc-windows/reference.md index d7dcc1624..c3148160b 100644 --- a/.agents/skills/build-openshell-mxc-windows/reference.md +++ b/.agents/skills/build-openshell-mxc-windows/reference.md @@ -166,11 +166,17 @@ The Windows build path is additive. Keep the repository's Linux `mise run ci`, d | `mise run --skip-tools windows:build:x64` | Builds release `openshell-gateway.exe` and `openshell.exe` for x64 | | `mise run --skip-tools windows:build:arm64` | Builds release `openshell-gateway.exe` and `openshell.exe` for ARM64 | | `mise run --skip-tools windows:test:x64` | Runs native x64 workspace tests | +| `mise run --skip-tools windows:test:arm64` | Runs workspace tests on a native ARM64 Windows host | | `mise run --skip-tools windows:test:unsupported:x64` | Verifies unsupported driver contracts through server/runtime tests without building standalone driver binaries | +| `mise run --skip-tools windows:test:unsupported:arm64` | Verifies the same contracts on a native ARM64 Windows host | | `mise run --skip-tools windows:ci` | Runs the full Windows lane in order | Use `--skip-tools` for Windows CI and automation. Rust must come from rustup with MSVC targets, and Visual Studio Build Tools must provide the linker and SDK. Because `--skip-tools` does not provision mise-managed tools, the Windows wrapper clears inherited `RUSTC_WRAPPER=sccache` before invoking Cargo. The wrapper excludes unsupported driver packages as top-level workspace targets for Windows check/test, but those library stubs still compile when the gateway depends on them. The wrapper may discover `VsDevCmd.bat`, but it must not install Visual Studio, Rust, Docker, Kubernetes, Podman, VM tooling, WSL, or Hyper-V. +ARM64 validation requires the Visual Studio ARM64 C++ tools, host-native +`libclang.dll`, CMake tools for bundled Z3, and an ARM64-capable Windows SDK. +Test tasks reject a target that does not match the Windows host architecture. + ## What NOT to do in this skill - Do not implement named-pipe IPC. Stubs only. (Belongs to the follow-on MXC driver skill.) diff --git a/architecture/windows-msvc-build.md b/architecture/windows-msvc-build.md index c83cd8f2b..ca2993ffa 100644 --- a/architecture/windows-msvc-build.md +++ b/architecture/windows-msvc-build.md @@ -48,13 +48,17 @@ Windows validation is exposed through `tasks/windows.toml`: | `windows:check:arm64` | Check the ARM64 MSVC gateway/CLI build graph. | | `windows:build:x64` | Build release x64 `openshell-gateway.exe` and `openshell.exe`. | | `windows:build:arm64` | Build release ARM64 `openshell-gateway.exe` and `openshell.exe`. | -| `windows:test:x64` | Run native x64 workspace tests, excluding unsupported driver packages as top-level test targets. | +| `windows:test:x64` | Run native x64 workspace tests, excluding unsupported Windows packages as top-level test targets. | +| `windows:test:arm64` | Run native ARM64 workspace tests with the same package exclusions. | | `windows:test:unsupported:x64` | Run focused server/runtime tests for unsupported driver contracts. | +| `windows:test:unsupported:arm64` | Run the same focused contracts natively on ARM64. | | `windows:ci` | Run check, build, test, unsupported-contract tests, and artifact reporting. | The Windows tasks call `tasks/scripts/windows-msvc.ps1`. The wrapper discovers Visual Studio's `VsDevCmd.bat`, adds rustup MSVC targets, clears inherited `RUSTC_WRAPPER`, and keeps build artifacts under the normal Cargo target tree. +Test tasks require the Rust target architecture to match the Windows host, so +an ARM64 test result is native coverage rather than x64 emulation coverage. By default it enables bundled Z3 for reproducible Windows builds. When `Z3_LIBRARY_PATH_OVERRIDE` points at a directory containing `libz3.lib`, the wrapper uses that system Z3 instead and requires `Z3_SYS_Z3_HEADER` to point at @@ -64,6 +68,11 @@ The lane uses `mise run --skip-tools windows:*` because Windows Rust comes from rustup and linking comes from Visual Studio Build Tools. Mise orchestrates the tasks; it does not own the Windows toolchain. +ARM64 validation requires the Visual Studio ARM64 MSVC tools, host-native Clang +tools for `libclang.dll`, CMake tools for bundled Z3, and an ARM64-capable +Windows SDK. Artifact hashing uses .NET SHA256 directly because module +autoloading in the mise-launched Windows PowerShell process is not guaranteed. + ## CI Shape The x64 GitHub Actions job runs on `windows-2025` and executes: @@ -76,9 +85,9 @@ mise run --skip-tools windows:test:unsupported:x64 ``` The ARM64 job is scaffolded but disabled until a Windows ARM64 runner is -available. Once enabled, it should run check and release build for -`aarch64-pc-windows-msvc`. Native ARM64 tests require an ARM64 runner and are -not part of the current build-only lane. +available. Once enabled, it should run check, release build, native workspace +tests, and the focused unsupported-driver contracts for +`aarch64-pc-windows-msvc`. ## Validation Contract @@ -87,6 +96,7 @@ A successful Windows build report should include: - x64 and ARM64 `cargo check` status. - x64 and ARM64 release build status for `openshell-gateway.exe` and `openshell.exe`. - x64 test summary. +- Native ARM64 test summary when validation runs on an ARM64 host. - Focused unsupported-driver contract test status. - Artifact size and SHA256 for each Windows binary. diff --git a/tasks/scripts/windows-msvc.ps1 b/tasks/scripts/windows-msvc.ps1 index d298663c7..cb53f57a8 100644 --- a/tasks/scripts/windows-msvc.ps1 +++ b/tasks/scripts/windows-msvc.ps1 @@ -43,7 +43,7 @@ $BundledZ3ServerFeatures = "--features openshell-server/bundled-z3,openshell-pro $Z3WorkspaceFeatures = $BundledZ3WorkspaceFeatures $Z3ServerFeatures = $BundledZ3ServerFeatures -function Resolve-VsDevCmd { +function Resolve-VsDevCmd([string] $RustTarget) { if ($env:OPENSHELL_VSDEVCMD -and (Test-Path $env:OPENSHELL_VSDEVCMD)) { return (Resolve-Path $env:OPENSHELL_VSDEVCMD).Path } @@ -55,7 +55,12 @@ function Resolve-VsDevCmd { $vswhere = $null } if ($vswhere -and (Test-Path $vswhere)) { - $found = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -find "Common7\Tools\VsDevCmd.bat" | Select-Object -First 1 + $requiredComponent = switch ($RustTarget) { + "x86_64-pc-windows-msvc" { "Microsoft.VisualStudio.Component.VC.Tools.x86.x64" } + "aarch64-pc-windows-msvc" { "Microsoft.VisualStudio.Component.VC.Tools.ARM64" } + default { throw "Unsupported target: $RustTarget" } + } + $found = & $vswhere -latest -products * -requires $requiredComponent -find "Common7\Tools\VsDevCmd.bat" | Select-Object -First 1 if ($found -and (Test-Path $found)) { return (Resolve-Path $found).Path } @@ -65,7 +70,7 @@ function Resolve-VsDevCmd { [Environment]::GetEnvironmentVariable("ProgramFiles"), [Environment]::GetEnvironmentVariable("ProgramFiles(x86)") ) | Where-Object { $_ } - $versions = @("18", "17") + $versions = @("18", "2022", "17") $editions = @("Enterprise", "Professional", "Community", "BuildTools") foreach ($root in $programFiles) { foreach ($version in $versions) { @@ -150,6 +155,14 @@ function Get-VsTargetArch([string] $RustTarget) { } } +function Assert-NativeTestTarget([string] $RustTarget) { + $targetArch = Get-VsTargetArch $RustTarget + $hostArch = Get-HostArch + if ($targetArch -ne $hostArch) { + throw "Windows tests require a native runner. Target $RustTarget maps to $targetArch, but the host is $hostArch." + } +} + function Get-SelectedTargets([string] $RequestedTarget) { if ($RequestedTarget -eq "all") { $targets = @("x86_64-pc-windows-msvc") @@ -224,7 +237,7 @@ function Invoke-VsCargo { throw "rustup target add $RustTarget failed" } - $vsDevCmd = Resolve-VsDevCmd + $vsDevCmd = Resolve-VsDevCmd $RustTarget $targetArch = Get-VsTargetArch $RustTarget $hostArch = Get-HostArch $logPath = Join-Path $LogDir $LogName @@ -260,9 +273,7 @@ function Invoke-Build([string] $RustTarget) { } function Invoke-Test([string] $RustTarget) { - if ($RustTarget -ne "x86_64-pc-windows-msvc") { - throw "Windows ARM64 tests require a native ARM64 runner and are not part of this build-only lane." - } + Assert-NativeTestTarget $RustTarget Invoke-VsCargo ` -RustTarget $RustTarget ` -CargoArgs "cargo test --workspace $UnsupportedDriverPackageExcludes --target $RustTarget --no-fail-fast $Z3WorkspaceFeatures" ` @@ -270,9 +281,7 @@ function Invoke-Test([string] $RustTarget) { } function Invoke-UnsupportedContractTests([string] $RustTarget) { - if ($RustTarget -ne "x86_64-pc-windows-msvc") { - throw "Unsupported-driver contract tests run only on the native x64 Windows lane." - } + Assert-NativeTestTarget $RustTarget $tests = @( "windows_compute_driver_stubs_report_unsupported", @@ -286,6 +295,20 @@ function Invoke-UnsupportedContractTests([string] $RustTarget) { } } +function Get-Sha256([string] $Path) { + $stream = [System.IO.File]::OpenRead($Path) + try { + $sha256 = [System.Security.Cryptography.SHA256]::Create() + try { + return [BitConverter]::ToString($sha256.ComputeHash($stream)).Replace("-", "") + } finally { + $sha256.Dispose() + } + } finally { + $stream.Dispose() + } +} + function Show-Artifacts([string[]] $RustTargets) { $rows = @() foreach ($rustTarget in $RustTargets) { @@ -295,12 +318,11 @@ function Show-Artifacts([string[]] $RustTargets) { continue } $item = Get-Item $path - $hash = Get-FileHash $path -Algorithm SHA256 $rows += [pscustomobject]@{ Target = $rustTarget Binary = $binary Size = $item.Length - SHA256 = $hash.Hash + SHA256 = Get-Sha256 $item.FullName Path = $item.FullName } } diff --git a/tasks/windows.toml b/tasks/windows.toml index fa3b2b60a..c7a2e51f9 100644 --- a/tasks/windows.toml +++ b/tasks/windows.toml @@ -39,11 +39,21 @@ description = "Run Windows x64 MSVC workspace tests" run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test x86_64-pc-windows-msvc" +["windows:test:arm64"] +description = "Run Windows ARM64 MSVC workspace tests on a native ARM64 host" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test aarch64-pc-windows-msvc" + ["windows:test:unsupported:x64"] description = "Run focused Windows unsupported compute-driver contract tests" run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-unsupported x86_64-pc-windows-msvc" +["windows:test:unsupported:arm64"] +description = "Run focused Windows unsupported compute-driver contract tests on a native ARM64 host" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-unsupported aarch64-pc-windows-msvc" + ["windows:artifacts"] description = "Report Windows release artifact sizes and SHA256 hashes" run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"