ci(windows): add Windows MSVC CI jobs (#2738)

* fix(ci): preserve Windows Rust build cache

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): invalidate empty Windows caches

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* perf(ci): cache Windows builds with sccache

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): restore target directory caching

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* perf(ci): use prebuilt Z3 on Windows

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* perf(ci): layer sccache on Windows target cache

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): split PR checks from main validation

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): separate checks builds and cache seeding

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): simplify Windows build dependency

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): rely on Windows job dependency status

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): use valid opt-in Windows ARM runner

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): keep ARM64 validation local

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): install Clippy for Windows validation

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): focus platform lint coverage

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* docs(licenses): explain bzip2 allowance

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): simplify workflow name

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(windows): allow async platform stub

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(network): make file fingerprints portable

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): lint supported deliverables

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): allow platform-gated lint

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* chore(ci): align Windows cache action with main

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): align Windows validation with prerequisites

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): pin Rust toolchain action

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): use enterprise-approved Windows actions

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(windows): restore strict MSVC validation

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): run Rust tests with nextest

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): normalize nextest lock provenance

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): add native arm64 validation

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): lock nextest for Windows ARM64

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(windows): resolve duplicate MXC authentication method

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* test(conformance): use native absolute paths on Windows

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* docs(windows): address MSVC review feedback

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): simplify cache key names

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): isolate Windows Rust toolchains for stable caches

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(ci): configure Rustup home in runner setup

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): surface sccache server write diagnostics

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): remove temporary cache diagnostics

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* ci(windows): address review feedback

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(windows): reconcile merged driver capabilities

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* fix(deps): preserve AWS-LC-only lockfile

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

* chore(deps): allow z3 prebuilt TLS wrapper

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

---------

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
This commit is contained in:
Piotr Mlocek
2026-09-10 23:53:01 +00:00
committed by GitHub
parent ce25acca5a
commit ddc8bba967
17 changed files with 526 additions and 262 deletions
@@ -30,7 +30,7 @@ The Windows build lane is implemented by these tracked files:
| `tasks/windows.toml` | Mise task entry points for `windows:*` commands. |
| `tasks/rust.toml`, `tasks/test.toml`, and `tasks/markdown.toml` | Windows routing for compiler-bearing checks, explicit Unix-only test skips, and Markdown dependency setup. |
| `tasks/scripts/windows-msvc.ps1` | PowerShell wrapper that enters the Visual Studio developer environment and invokes Cargo. |
| `.github/workflows/windows-msvc.yml` | Manually dispatched GitHub Actions jobs with architecture-specific Rust caches for x64 and future ARM64 Windows validation. |
| `.github/workflows/windows-msvc.yml` | PR/merge-queue lint and test plus main/manual cache seeding and dependent binary builds on native x64 and ARM64 runners. |
| `architecture/windows-msvc-build.md` | Design notes and validation contract. |
| `.agents/skills/build-openshell-mxc-windows/` | This skill and companion reference material. |
@@ -114,8 +114,8 @@ The lane targets a Windows host with Visual Studio Build Tools and rustup.
| Visual Studio 2022 or newer | `where.exe cl.exe` from a Developer PowerShell | Build Tools, Community, Professional, and Enterprise editions work when the target C++ components are installed. The wrapper discovers `VsDevCmd.bat` through `OPENSHELL_VSDEVCMD`, `vswhere`, or installed release directories such as `18` and `2022`. |
| Visual C++ ARM64 tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationPath` | Required for native ARM64 check, build, and tests and for x64-to-ARM64 check/build. Tests always require a native runner. |
| Visual C++ ARM64 Spectre-mitigated libraries | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Runtimes.ARM64.Spectre -property installationPath` | Required by `regorus` through `msvc_spectre_libs`; the build fails when the selected MSVC toolset lacks `lib\spectre\arm64`. |
| Visual C++ Clang tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Llvm.Clang -property installationPath` | Provides host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for ARM64 crypto dependencies such as `ring` and `aws-lc-sys`. On ARM64, the wrapper uses `VC\Tools\Llvm\Arm64\bin`. |
| Visual C++ CMake tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.CMake.Project -property installationPath` | Provides CMake and Ninja. The x64-to-ARM64 path adds Ninja to `PATH` for native dependencies but keeps bundled Z3 on CMake's Visual Studio ARM64 generator with native MSVC `cl.exe`. |
| Visual C++ Clang tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Llvm.Clang -property installationPath` | Provides host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for ARM64 crypto dependencies such as `aws-lc-sys`. On ARM64, the wrapper uses `VC\Tools\Llvm\Arm64\bin`. |
| Visual C++ CMake tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.CMake.Project -property installationPath` | Provides CMake and Ninja for native dependencies. The x64-to-ARM64 path adds Ninja to `PATH`; Z3 uses an architecture-specific prebuilt release. |
| Windows SDK | `where.exe rc.exe` from a Developer PowerShell | Install an SDK containing target libraries and ARM64 tools. |
| Rust via rustup | `rustc --version` | Add each target being validated: `x86_64-pc-windows-msvc` and/or `aarch64-pc-windows-msvc`. The wrapper also adds the selected target. |
| mise | `mise --version` | Used as a task runner only. |
@@ -135,8 +135,9 @@ from this skill.
| `CARGO_TARGET_DIR` | `target` under repo root | Override Cargo output location. Use a short absolute path when x64-to-ARM64 builds approach Windows path-length limits. |
| `Z3_LIBRARY_PATH_OVERRIDE` | unset | Directory containing an x64 system `libz3.lib`; not valid for ARM64. |
| `Z3_SYS_Z3_HEADER` | unset | Full `z3.h` path required with a system Z3 library. |
| `Z3_SYS_BUNDLED_DIR_OVERRIDE` | pinned source cached under `CARGO_TARGET_DIR` when explicit, otherwise `%LOCALAPPDATA%\OpenShell\cache\z3` | Use an existing Z3 source tree containing `src/api/z3.h`; otherwise the wrapper fetches the pinned revision through Git and sets this automatically. |
| `RUSTC_WRAPPER` | cleared by wrapper | The wrapper clears inherited values because `--skip-tools` does not provision `sccache`. |
| `Z3_SYS_Z3_VERSION` | `4.16.0` | Pinned official prebuilt Z3 release selected by the wrapper. |
| `READ_ONLY_GITHUB_TOKEN` | unset | Optional token for the Z3 release lookup; GitHub Actions supplies `github.token`. |
| `RUSTC_WRAPPER` | inherited | The wrapper resolves an available command to an absolute path. If it is unavailable, the wrapper warns and continues without compiler caching. |
Legacy fork variables such as `OPENSHELL_UPSTREAM`,
`OPENSHELL_MXC_FORK_DIR`, and `OPENSHELL_MXC_FORK_BRANCH` are no longer part
@@ -189,17 +190,19 @@ order:
6. Focused unsupported-driver contract tests.
7. Artifact reporting.
The GitHub Actions jobs use architecture-specific `Swatinem/rust-cache`
entries for the Cargo registry and dependency target artifacts. Failed runs
also save their usable dependency artifacts. The workflow remains manually
dispatched until cache-hit runtimes justify restoring automatic triggers.
The GitHub Actions jobs layer architecture-specific `Swatinem/rust-cache`
entries for Cargo registry and dependency target artifacts with sccache's GHA
backend for cacheable Rust compiler outputs. Failed runs also save their usable
dependency artifacts. Pull-request mirrors and merge queues run Clippy for the
Windows-supported workspace and e2e crates plus Rust tests. Pushes to `main` and
manual dispatches run the same lint and test commands in a cache-seed job,
followed by a dependent release-binary build job. The seed and PR jobs use the
same cache namespaces. The binaries are not uploaded or published.
The ARM64 check/build steps in this x64-host contract are cross-builds. The
wrapper discovers and adds host-native LLVM and Ninja to `PATH`, requires the
ARM64 compiler and Spectre-mitigated libraries, lets ARM64 crypto crates select
`clang-cl`, and keeps bundled Z3 on native MSVC `cl.exe` with CMake's Visual
Studio ARM64 generator. Z3 does not use Ninja because `z3-sys 0.10.9` passes
the MSBuild-only `-m` argument.
`clang-cl`, and downloads the official prebuilt ARM64 Z3 static library.
On ARM64 hosts, validate the native ARM64 check, build, and test path. The
wrapper rejects test targets that do not match the host architecture, so x64
@@ -209,7 +212,8 @@ commands above on an ARM64 host.
The repository-wide `mise run pre-commit` task is also supported on Windows.
Its Rust check, Clippy, and test dependencies enter the same MSVC environment
for the native host target and clear inherited `RUSTC_WRAPPER`. Linux glibc
for the native host target and use an inherited compiler wrapper when it is
available. Linux glibc
installer tests and Linux service/RPM packaging-asset tests skip explicitly;
the Linux build-environment shell-helper test also skips; cross-platform checks
continue to run. The blocking Windows Clippy pass excludes unsupported
@@ -261,9 +265,9 @@ The focused contract tasks for either native architecture run:
windows_builtin_compute_drivers_report_unsupported
```
These tests are also included in the full x64 workspace test run; the focused
task intentionally re-runs them so unsupported Windows behavior is visible in
the CI report.
These tests are also included in the full x64 workspace test run. The focused
task is available for local diagnosis; GitHub Actions does not re-run it after
the full suite.
## Test Accounting Guidance
@@ -293,19 +297,13 @@ Useful log files:
| `test-x86_64-pc-windows-msvc-unsupported-*.log` | Focused unsupported-driver contract output. |
| `test-aarch64-pc-windows-msvc-unsupported-*.log` | Focused native ARM64 contract output. |
The first bundled-Z3 check or test can spend several minutes in CMake/MSBuild
without much console output because Cargo output is redirected to the log. Look
for native `MSBuild.exe` workers before treating the process as stalled. The
wrapper fetches the pinned Z3 source through Git before Cargo starts. It caches
under an explicitly configured `CARGO_TARGET_DIR`, or under the current user's
local application data directory when Cargo uses its default target tree.
Concurrent commands publish the validated source through an atomic directory
rename, so x64 and ARM64 validation can share the cache safely. The wrapper does
not rely on the rate-limited GitHub Contents API used by `z3-sys`. A failed
fetch reports the partial checkout path for diagnosis. The artifact report
computes SHA256 through .NET directly and does not rely on the
`Get-FileHash` module being available inside the mise-launched Windows
PowerShell process.
The first check downloads the pinned official Z3 archive for the target
architecture through `z3-sys`. GitHub Actions authenticates the lookup with its
read-only workflow token; local users can set `READ_ONLY_GITHUB_TOKEN` if an
unauthenticated lookup is rate-limited. Cargo stores the extracted library in
its target tree, so the Windows target cache reuses it. The artifact report
computes SHA256 through .NET directly and does not rely on the `Get-FileHash`
module being available inside the mise-launched Windows PowerShell process.
## Common Fix Patterns
@@ -9,7 +9,7 @@ maintaining the existing build-only Windows MSVC lane.
|---|---|
| `tasks/windows.toml` | Mise task definitions for `windows:*`. |
| `tasks/scripts/windows-msvc.ps1` | Visual Studio environment discovery, rustup target setup, Cargo invocation, logs, artifact report. |
| `.github/workflows/windows-msvc.yml` | Manual GitHub Actions x64 job and disabled ARM64 scaffold, each with an architecture-specific Rust dependency cache. |
| `.github/workflows/windows-msvc.yml` | PR/merge-queue lint and test plus main/manual cache seeding and dependent binary builds on native x64 and ARM64 runners. |
| `architecture/windows-msvc-build.md` | Human-readable design contract. |
## Commands
@@ -46,8 +46,9 @@ if ($arch -eq [System.Runtime.InteropServices.Architecture]::Arm64) {
The native test tasks reject a target that does not match the host architecture.
Do not report x64 compatibility-under-emulation coverage from an ARM64 run.
The wrapper adds missing rustup targets and clears inherited
`RUSTC_WRAPPER`. It does not install Visual Studio, Rust, Docker, Kubernetes,
The wrapper adds missing rustup targets and preserves an inherited
`RUSTC_WRAPPER` when the command is available. Otherwise, it warns and clears
the setting. It does not install Visual Studio, Rust, Docker, Kubernetes,
Podman, WSL, Hyper-V, or VM tooling.
On Windows, `mise run pre-commit` routes `rust:check`, `rust:lint`, and
@@ -67,11 +68,10 @@ file.
For ARM64, verify the Visual Studio instance contains the ARM64 MSVC tools,
ARM64 Spectre-mitigated libraries, Clang tools, CMake tools, and a Windows SDK.
Clang supplies host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for
ARM64 crypto dependencies such as `ring` and `aws-lc-sys`. Native ARM64 uses
the normal bundled-Z3 CMake path. An x64-to-ARM64 check/build discovers and
adds host-native Ninja to `PATH`, while the crypto crates select `clang-cl`.
Bundled Z3 uses CMake's Visual Studio ARM64 generator with native MSVC `cl.exe`
because `z3-sys 0.10.9` passes the MSBuild-only `-m` argument. Use a short
ARM64 crypto dependencies such as `aws-lc-sys`. Native and
x64-to-ARM64 builds use the official prebuilt Z3 4.16.0 static library for the
target architecture. An x64-to-ARM64 check/build discovers and adds host-native
Ninja to `PATH`, while the crypto crates select `clang-cl`. Use a short
`CARGO_TARGET_DIR` if Windows path-length limits are reached.
## Unsupported Driver Rules
+188 -25
View File
@@ -1,50 +1,213 @@
name: Windows MSVC (build-only)
name: Windows MSVC
on:
merge_group:
types: [checks_requested]
push:
branches:
- main
- "pull-request/[0-9]+"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
# sccache cannot cache incremental Rust compilations, and rust-cache removes
# incremental artifacts before saving the target directory.
CARGO_INCREMENTAL: "0"
jobs:
x64:
runs-on: windows-2025
pr_metadata:
name: Resolve PR metadata
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
outputs:
should_run: ${{ github.ref == 'refs/heads/main' || steps.gate.outputs.should_run == 'true' }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: jdx/mise-action@c2a87611a18de5b3828c5652fe268e992400cb5c # v4.3.0
- id: gate
if: github.ref != 'refs/heads/main'
uses: ./.github/actions/pr-gate
pr-check:
name: PR lint and test (${{ matrix.arch }})
needs: pr_metadata
if: >-
needs.pr_metadata.outputs.should_run == 'true' &&
github.ref != 'refs/heads/main' &&
github.event_name != 'workflow_dispatch'
strategy:
fail-fast: false
matrix:
include:
- arch: x64
runner: windows-2025
rust_target: x86_64-pc-windows-msvc
- arch: arm64
runner: windows-11-arm
rust_target: aarch64-pc-windows-msvc
runs-on: ${{ matrix.runner }}
env:
READ_ONLY_GITHUB_TOKEN: ${{ github.token }}
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_VERSION: windows-msvc-${{ matrix.arch }}
SCCACHE_CLIENT_SIDE: "1"
SCCACHE_BASEDIRS: ${{ github.workspace }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: jdx/mise-action@e6a8b3978addb5a52f2b4cd9d91eafa7f0ab959d # v4.2.0
with:
install: false
experimental: true
- uses: dtolnay/rust-toolchain@master
with:
toolchain: "1.95.0"
targets: x86_64-pc-windows-msvc
- name: Configure GHA sccache backend
uses: mozilla-actions/sccache-action@9e7fa8a12102821edf02ca5dbea1acd0f89a2696 # v0.0.10
- name: Isolate Rust toolchains
# rust-cache hashes every installed toolchain, including unused runner defaults.
run: '"RUSTUP_HOME=$env:RUNNER_TEMP/openshell-rustup" >> $env:GITHUB_ENV'
- name: Install Rust toolchain
# GitHub-hosted runners include rustup. With no toolchain argument,
# rustup installs the checked-in rust-toolchain.toml configuration:
# https://github.com/dtolnay/rust-toolchain/issues/133#issuecomment-2610825530
run: |
rustup set profile minimal
rustup toolchain install
rustup component add clippy
rustup target add ${{ matrix.rust_target }}
- name: Install cargo-nextest
run: mise install --locked github:nextest-rs/nextest
- name: Cache Rust target and registry
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
env:
# Keep rust-cache's cargo metadata calls independent of the compiler
# wrapper; build steps still inherit sccache from the job.
RUSTC_WRAPPER: ""
with:
shared-key: windows-msvc-x64
shared-key: windows-msvc-${{ matrix.arch }}
cache-targets: "true"
cache-on-failure: "true"
cache-bin: "false"
- run: mise run --skip-tools windows:check:x64
- run: mise run --skip-tools windows:build:x64
- run: mise run --skip-tools windows:test:x64
- run: mise run --skip-tools windows:test:unsupported:x64
arm64:
# TODO: provision a windows-arm64 self-hosted runner
runs-on: [self-hosted, windows-arm64]
if: false # flip to true once the runner is online
- name: Lint Windows-supported crates
run: mise run --skip-tools windows:lint:${{ matrix.arch }}
- name: Test
run: mise run --skip-tools windows:test:${{ matrix.arch }}
- name: sccache stats
if: always()
run: sccache --show-stats
cache-seed:
name: Seed cache (${{ matrix.arch }})
needs: pr_metadata
if: >-
needs.pr_metadata.outputs.should_run == 'true' &&
(github.ref == 'refs/heads/main' || github.event_name == 'workflow_dispatch')
strategy:
fail-fast: false
matrix:
include:
- arch: x64
runner: windows-2025
rust_target: x86_64-pc-windows-msvc
- arch: arm64
runner: windows-11-arm
rust_target: aarch64-pc-windows-msvc
runs-on: ${{ matrix.runner }}
env:
READ_ONLY_GITHUB_TOKEN: ${{ github.token }}
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_VERSION: windows-msvc-${{ matrix.arch }}
SCCACHE_CLIENT_SIDE: "1"
SCCACHE_BASEDIRS: ${{ github.workspace }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: jdx/mise-action@c2a87611a18de5b3828c5652fe268e992400cb5c # v4.3.0
- uses: jdx/mise-action@e6a8b3978addb5a52f2b4cd9d91eafa7f0ab959d # v4.2.0
with:
install: false
experimental: true
- uses: dtolnay/rust-toolchain@master
with:
toolchain: "1.95.0"
targets: aarch64-pc-windows-msvc
- name: Configure GHA sccache backend
uses: mozilla-actions/sccache-action@9e7fa8a12102821edf02ca5dbea1acd0f89a2696 # v0.0.10
- name: Isolate Rust toolchains
run: '"RUSTUP_HOME=$env:RUNNER_TEMP/openshell-rustup" >> $env:GITHUB_ENV'
- name: Install Rust toolchain
run: |
rustup set profile minimal
rustup toolchain install
rustup component add clippy
rustup target add ${{ matrix.rust_target }}
- name: Install cargo-nextest
run: mise install --locked github:nextest-rs/nextest
- name: Cache Rust target and registry
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
env:
RUSTC_WRAPPER: ""
with:
shared-key: windows-msvc-arm64
shared-key: windows-msvc-${{ matrix.arch }}
cache-targets: "true"
cache-on-failure: "true"
cache-bin: "false"
- run: mise run --skip-tools windows:check:arm64
- run: mise run --skip-tools windows:build:arm64
- name: Lint Windows-supported crates
# Keep the seed workload identical to the pull-request workload so its
# target and compiler caches are reusable by subsequent pull requests.
run: mise run --skip-tools windows:lint:${{ matrix.arch }}
- name: Test
run: mise run --skip-tools windows:test:${{ matrix.arch }}
- name: sccache stats
if: always()
run: sccache --show-stats
build:
name: Build binaries (${{ matrix.arch }})
needs: cache-seed
strategy:
fail-fast: false
matrix:
include:
- arch: x64
runner: windows-2025
rust_target: x86_64-pc-windows-msvc
- arch: arm64
runner: windows-11-arm
rust_target: aarch64-pc-windows-msvc
runs-on: ${{ matrix.runner }}
env:
READ_ONLY_GITHUB_TOKEN: ${{ github.token }}
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_VERSION: windows-msvc-${{ matrix.arch }}
SCCACHE_CLIENT_SIDE: "1"
SCCACHE_BASEDIRS: ${{ github.workspace }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: jdx/mise-action@e6a8b3978addb5a52f2b4cd9d91eafa7f0ab959d # v4.2.0
with:
install: false
experimental: true
- name: Configure GHA sccache backend
uses: mozilla-actions/sccache-action@9e7fa8a12102821edf02ca5dbea1acd0f89a2696 # v0.0.10
- name: Isolate Rust toolchains
run: '"RUSTUP_HOME=$env:RUNNER_TEMP/openshell-rustup" >> $env:GITHUB_ENV'
- name: Install Rust toolchain
run: |
rustup set profile minimal
rustup toolchain install
rustup target add ${{ matrix.rust_target }}
- name: Cache Rust target and registry
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
env:
RUSTC_WRAPPER: ""
with:
shared-key: windows-msvc-${{ matrix.arch }}
cache-targets: "true"
cache-on-failure: "true"
cache-bin: "false"
- name: Build release binaries
run: mise run --skip-tools windows:build:${{ matrix.arch }}
- name: sccache stats
if: always()
run: sccache --show-stats
+13 -5
View File
@@ -338,13 +338,21 @@ compiles Z3 from source during the Rust build and requires CMake 3.16+:
cargo build -p openshell-prover --features bundled-z3
```
For x86-64 Windows MSVC builds, use one of these Z3 paths:
For x86-64 and ARM64 Windows MSVC builds, use one of these Z3 paths:
- Prebuilt Z3 (the default for `windows:*` tasks): `z3-sys` downloads the
pinned Z3 4.16.0 GitHub release for the target architecture on the first
build. Cargo reuses the extracted archive from its target directory. Windows
CI authenticates the GitHub API request with `READ_ONLY_GITHUB_TOKEN` and
preserves the archive in the architecture-specific Cargo target cache. For
cold local builds, you may set `READ_ONLY_GITHUB_TOKEN` to avoid anonymous
GitHub API rate limits.
- System Z3: point `Z3_LIBRARY_PATH_OVERRIDE` at the directory containing the
64-bit MSVC Z3 library and `Z3_SYS_Z3_HEADER` at the full path to `z3.h`.
target-compatible MSVC Z3 library and `Z3_SYS_Z3_HEADER` at the full path to `z3.h`.
The `windows:*` tasks use this path automatically when `Z3_LIBRARY_PATH_OVERRIDE`
is set.
- Bundled Z3: pass `--features bundled-z3` so `z3-sys` builds Z3 from source.
- Bundled Z3: for direct Cargo builds, pass `--features bundled-z3` so `z3-sys`
builds Z3 from source.
`openshell-prover` itself has no `bindgen`/`libclang` dependency, so building
just this crate does not require `LIBCLANG_PATH`:
@@ -357,7 +365,7 @@ cargo build -p openshell-prover --target x86_64-pc-windows-msvc --features bundl
To build the full set of Windows binaries, including `openshell-gateway.exe`
and `openshell.exe`, use the `windows:build:x64` mise task instead of a
single-crate `cargo build`. It builds Z3 from source (bundled) by default. A
single-crate `cargo build`. It downloads the pinned prebuilt Z3 release by default. A
full build also compiles crates that use `bindgen` (e.g. the MXC driver on
Windows), so it requires `libclang.dll`; if LLVM is not on the default search
path, set `LIBCLANG_PATH` to the directory containing `libclang.dll`:
@@ -367,7 +375,7 @@ $env:LIBCLANG_PATH='C:\Program Files\Microsoft Visual Studio\2022\<Edition>\VC\T
mise run --skip-tools windows:build:x64
```
To use a local x64 Z3 release instead of the bundled build, set
To use a local x64 Z3 release instead of the prebuilt download, set
`Z3_LIBRARY_PATH_OVERRIDE` and `Z3_SYS_Z3_HEADER` before running the task:
```powershell
Generated
+101
View File
@@ -908,6 +908,15 @@ dependencies = [
"either",
]
[[package]]
name = "bzip2"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3a53fac24f34a81bc9954b5d6cfce0c21e18ec6959f44f56e8e90e4bb7c346c"
dependencies = [
"libbz2-rs-sys",
]
[[package]]
name = "capctl"
version = "0.2.4"
@@ -1163,6 +1172,12 @@ dependencies = [
"unicode-xid",
]
[[package]]
name = "constant_time_eq"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b"
[[package]]
name = "core-foundation"
version = "0.10.1"
@@ -1495,6 +1510,12 @@ version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "092966b41edc516079bdf31ec78a2e0588d1d0c08f78b91d8307215928642b2b"
[[package]]
name = "deflate64"
version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac6b926516df9c60bfa16e107b21086399f8285a44ca9711344b9e553c5146e2"
[[package]]
name = "delegate"
version = "0.13.5"
@@ -1626,6 +1647,7 @@ dependencies = [
"const-oid 0.10.2",
"crypto-common 0.2.2",
"ctutils",
"zeroize",
]
[[package]]
@@ -1974,6 +1996,7 @@ checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"miniz_oxide",
"zlib-rs",
]
[[package]]
@@ -2614,6 +2637,7 @@ dependencies = [
"tokio",
"tokio-rustls",
"tower-service",
"webpki-roots",
]
[[package]]
@@ -3332,6 +3356,12 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "libbz2-rs-sys"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34b357333733e8260735ba5894eb928c02ecc69c78715f01a8019e7fa7f2db4c"
[[package]]
name = "libc"
version = "0.2.189"
@@ -3425,6 +3455,15 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]]
name = "lzma-rust2"
version = "0.16.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca93e534d1142d1d0dcca6d25fe302508a5dfb40b302802904577725ea0b695b"
dependencies = [
"sha2 0.11.0",
]
[[package]]
name = "matchers"
version = "0.2.0"
@@ -5208,6 +5247,12 @@ version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
[[package]]
name = "ppmd-rust"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "efca4c95a19a79d1c98f791f10aebd5c1363b473244630bb7dbde1dc98455a24"
[[package]]
name = "ppv-lite86"
version = "0.2.21"
@@ -5782,6 +5827,7 @@ dependencies = [
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
"webpki-roots",
]
[[package]]
@@ -7242,6 +7288,7 @@ checksum = "743bd48c283afc0388f9b8827b976905fb217ad9e647fae3a379a9283c4def2c"
dependencies = [
"deranged",
"itoa",
"js-sys",
"num-conv",
"powerfmt",
"serde_core",
@@ -7712,6 +7759,12 @@ dependencies = [
"thiserror 2.0.18",
]
[[package]]
name = "typed-path"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
[[package]]
name = "typenum"
version = "1.20.1"
@@ -8765,7 +8818,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c18b0a91a13522d21b3414847667de2b2056a721a3edcb5b6ee6858352d58db4"
dependencies = [
"pkg-config",
"reqwest 0.12.28",
"serde_json",
"z3-src",
"zip",
]
[[package]]
@@ -8862,12 +8918,57 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "zip"
version = "8.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b"
dependencies = [
"aes",
"bzip2",
"constant_time_eq",
"crc32fast",
"deflate64",
"flate2",
"getrandom 0.4.2",
"hmac 0.13.0",
"indexmap",
"lzma-rust2",
"memchr",
"pbkdf2",
"ppmd-rust",
"sha1 0.11.0",
"time",
"typed-path",
"zeroize",
"zopfli",
"zstd",
]
[[package]]
name = "zlib-rs"
version = "0.6.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b142a20ec14a91d5bc708c1dc21b080c550113d8aa77afa29635673a65dd02c5"
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[package]]
name = "zstd"
version = "0.13.3"
+62 -44
View File
@@ -54,11 +54,18 @@ creating misleading Windows driver artifacts.
## Mise Lane
The GitHub Actions workflow is manually dispatched. Each architecture restores
and saves a dedicated Rust cache containing the Cargo registry and dependency
build artifacts, including artifacts from failed runs. Keep the workflow manual
until cache-hit runtimes demonstrate that it is suitable for pull requests and
merges to `main`.
The GitHub Actions workflow runs Clippy for the Windows-supported workspace and
e2e crates plus Rust tests for pull-request mirror branches and merge queues. On
pushes to `main`, a cache-seed job runs the same lint and test commands before a
dependent job builds the release binaries. Manual dispatches exercise the same
seed-then-build path. The binaries remain CI validation artifacts and are not
uploaded or published.
Each job restores and saves a dedicated Rust cache containing the Cargo
registry and dependency build artifacts, including artifacts from failed runs.
The seed job and pull-request job use the same Cargo target and sccache
namespaces. The release build waits for the seed job, then restores its newly
warmed cache rather than compiling concurrently from a cold cache.
Windows validation is exposed through `tasks/windows.toml`:
@@ -66,10 +73,12 @@ Windows validation is exposed through `tasks/windows.toml`:
|---|---|
| `windows:check:x64` | Check the x64 MSVC gateway/CLI build graph. |
| `windows:check:arm64` | Check the ARM64 MSVC gateway/CLI build graph. |
| `windows:lint:x64` | Run Clippy over the Windows-supported workspace for x64 MSVC. |
| `windows:lint:arm64` | Run Clippy over the Windows-supported workspace for ARM64 MSVC. |
| `windows:build:x64` | Build release x64 `openshell-gateway.exe` and `openshell.exe`. |
| `windows:build:arm64` | Build release ARM64 `openshell-gateway.exe` and `openshell.exe`. |
| `windows:test:x64` | Run native x64 workspace tests, including MXC mapper and lifecycle tests, while excluding unsupported Windows packages as top-level test targets. |
| `windows:test:arm64` | Run native ARM64 workspace tests with the same package exclusions. |
| `windows:test:x64` | Run native x64 workspace tests with the nextest CI profile and server test support, while excluding unsupported Windows packages as top-level test targets. |
| `windows:test:arm64` | Run the same suite natively on ARM64. |
| `windows:test:unsupported:x64` | Run focused gateway-composition tests for unsupported driver contracts. |
| `windows:test:unsupported:arm64` | Run the same focused contracts natively on ARM64. |
| `windows:ci` | Run check, build, test, unsupported-contract tests, and artifact reporting. |
@@ -77,8 +86,10 @@ Windows validation is exposed through `tasks/windows.toml`:
The Windows tasks call `tasks/scripts/windows-msvc.ps1`. The wrapper discovers
Visual Studio's `VsDevCmd.bat` with `vswhere` or by enumerating installed
release directories, validates the requested compiler and ARM64 Spectre
libraries, adds rustup MSVC targets, clears inherited `RUSTC_WRAPPER`, and
keeps build artifacts under the normal Cargo target tree.
libraries, adds rustup MSVC targets, preserves an inherited `RUSTC_WRAPPER`
when the command is available, and keeps build artifacts under the normal
Cargo target tree. If the wrapper command is unavailable, it warns and clears
the setting so local builds continue without compiler caching.
On Windows, the generic `rust:check`, `rust:lint`, and `test:rust` tasks call
the same wrapper with the host-native MSVC target. The wrapper preserves the
Unix Cargo commands on Linux and macOS, excludes unsupported Windows runtime
@@ -90,20 +101,25 @@ packaging-asset tests; its
cross-platform Python, Markdown, license, and documentation checks still run.
Test tasks require the Rust target architecture to match the Windows host, so
an ARM64 test result is native coverage rather than x64 emulation coverage.
By default it enables bundled Z3 for reproducible Windows builds. When
By default it enables the `z3-sys` prebuilt-release feature and pins Z3 4.16.0.
On a clean target directory, `z3-sys` downloads the official static library for
the selected Windows architecture instead of compiling Z3 through
CMake/MSBuild. GitHub Actions supplies its read-only workflow token for the
release lookup, and the Cargo target cache preserves the extracted library for
subsequent runs. When
`Z3_LIBRARY_PATH_OVERRIDE` points at a directory containing `libz3.lib`, the
wrapper uses that system Z3 instead and requires `Z3_SYS_Z3_HEADER` to point at
the full path to `z3.h`. For bundled builds, the wrapper fetches the Z3 source
revision pinned by `z3-sys` through Git and sets
`Z3_SYS_BUNDLED_DIR_OVERRIDE`. When `CARGO_TARGET_DIR` is explicit, the wrapper
uses it for the source cache. Otherwise, it caches under the current user's
local application data directory, outside the checkout. Publishing uses an
atomic directory rename so concurrent x64 and ARM64 commands can share the
cache safely. This keeps downloaded sources outside the checkout by default and
avoids the unauthenticated GitHub API lookup in the `z3-sys` build script, which
can fail with HTTP 403 when a shared runner or developer network exhausts its
API rate limit. An explicitly set `Z3_SYS_BUNDLED_DIR_OVERRIDE` remains
supported and must contain `src/api/z3.h`.
the full path to `z3.h`. Local clean builds use the unauthenticated GitHub API
unless `READ_ONLY_GITHUB_TOKEN` is set.
GitHub Actions layers the Cargo target cache with sccache's GitHub Actions
backend. The target cache lets Cargo skip intact dependency builds; sccache
recovers cacheable Rust compiler outputs when source changes invalidate part of
that target tree. CI enables client-side mode and normalizes the checkout root
for stable compiler cache keys. The target-cache action runs its metadata step
with `RUSTC_WRAPPER` cleared so cache maintenance does not depend on sccache.
Hosted jobs use an isolated `RUSTUP_HOME` containing the pinned toolchain so
unused toolchains in runner images cannot change the target-cache restore key.
The lane uses `mise run --skip-tools windows:*` because Windows Rust comes from
rustup and linking comes from Visual Studio Build Tools. Mise orchestrates the
@@ -114,45 +130,47 @@ Spectre-mitigated libraries, host-native Clang tools, CMake tools, and an
ARM64-capable Windows SDK. Clang provides `libclang.dll` for `bindgen` and
`clang-cl.exe` for ARM64 crypto dependencies. During x64-to-ARM64 check/build,
the wrapper discovers and adds the Visual Studio-bundled Ninja to `PATH` for
native dependencies. It lets `cmake-rs` select the Visual Studio ARM64
generator with native MSVC `cl.exe` for bundled Z3 so the Z3 build does not
inherit the crypto crates' compiler requirement. Z3 stays on the Visual Studio
generator because `z3-sys` emits an MSBuild-only `-m` argument that Ninja
rejects. Artifact hashing uses .NET SHA256 directly because module autoloading
in the mise-launched Windows PowerShell process is not guaranteed.
native dependencies. Z3 uses the official prebuilt ARM64 static library, so it
does not inherit compiler settings from those native dependencies. Artifact
hashing uses .NET SHA256 directly because module autoloading in the
mise-launched Windows PowerShell process is not guaranteed.
The wrapper defaults Cargo compilation to four jobs. Set
`OPENSHELL_WINDOWS_BUILD_JOBS` to a positive integer to override that limit.
A host-local mutex serializes wrapper-owned Cargo commands so concurrent
pre-commit tasks do not multiply the process count while bundled Z3 compiles.
pre-commit tasks do not multiply the compiler process count.
The wrapper does not set `CL` or `_CL_`: those variables are also consumed by
`clang-cl`, where MSVC's `/MP` option can be interpreted as an input file and
break ARM64 crypto dependency builds.
## CI Shape
The x64 GitHub Actions job runs on `windows-2025` and executes:
The x64 GitHub Actions jobs run on `windows-2025`; native ARM64 jobs run on
`windows-11-arm`. Pull-request mirrors and merge queues execute the matching
architecture-specific tasks:
```powershell
mise run --skip-tools windows:check:x64
mise run --skip-tools windows:build:x64
mise run --skip-tools windows:test:x64
mise run --skip-tools windows:test:unsupported:x64
mise run --skip-tools windows:lint:<x64|arm64>
mise run --skip-tools windows:test:<x64|arm64>
```
The cache is partitioned by architecture so incompatible x64 and ARM64 target
artifacts cannot collide. It does not cache Cargo-installed binaries, which
also keeps the disabled self-hosted ARM64 scaffold from modifying persistent
runner tooling.
Pushes to `main` and manual dispatches first seed the shared caches with those
same lint and test commands. After the seed succeeds, a separate job executes:
The local aggregate `windows:ci` task cross-builds ARM64 on an x64 host. The
GitHub x64 job currently runs only the x64 tasks, and native ARM64 tests remain
exclusive to an ARM64 runner.
```powershell
mise run --skip-tools windows:build:<x64|arm64>
```
The ARM64 job is scaffolded but disabled until a Windows ARM64 runner is
available. Once enabled, it should run check, release build, native workspace
tests, and the focused unsupported-driver contracts for
`aarch64-pc-windows-msvc`.
The server test-support suite includes the unsupported-driver contract test, so
CI does not run the focused test task a second time. The focused task remains
available for local diagnosis.
The hosted workflow uses architecture-specific cache namespaces and does not
cache Cargo-installed binaries.
The local aggregate `windows:ci` task can still cross-build ARM64 on an x64
host. Hosted tests use architecture-matched runners, so ARM64 test results are
native rather than emulated coverage.
## Validation Contract
+16 -6
View File
@@ -124,9 +124,19 @@ fn validate_command(label: &str, command: &Path, timeout_secs: u64) -> Result<()
mod tests {
use super::*;
#[cfg(not(windows))]
const RESTART_GATEWAY_COMMAND: &str = "/usr/local/libexec/restart-gateway";
#[cfg(windows)]
const RESTART_GATEWAY_COMMAND: &str = r"C:\usr\local\libexec\restart-gateway";
#[cfg(not(windows))]
const DIAGNOSTICS_COMMAND: &str = "/usr/local/libexec/diagnostics";
#[cfg(windows)]
const DIAGNOSTICS_COMMAND: &str = r"C:\usr\local\libexec\diagnostics";
#[test]
fn parses_a_smoke_and_continuity_plan() {
let plan = ConformancePlan::parse(
let plan = ConformancePlan::parse(&format!(
r#"
version = 1
@@ -139,10 +149,10 @@ mod tests {
[[runs.actions]]
name = "gateway-upgrade"
command = "/usr/local/libexec/restart-gateway"
command = '{RESTART_GATEWAY_COMMAND}'
timeout_secs = 120
"#,
)
))
.expect("valid plan");
assert_eq!(plan.runs.len(), 2);
@@ -151,18 +161,18 @@ mod tests {
#[test]
fn parses_plan_diagnostics() {
let plan = ConformancePlan::parse(
let plan = ConformancePlan::parse(&format!(
r#"
version = 1
[diagnostics]
command = "/usr/local/libexec/diagnostics"
command = '{DIAGNOSTICS_COMMAND}'
timeout_secs = 60
[[runs]]
scenario = "smoke"
"#,
)
))
.expect("valid plan with diagnostics");
assert_eq!(
+4 -3
View File
@@ -293,10 +293,11 @@ impl MxcComputeBackend {
driver_reports_runtime_readiness: true,
resource_capabilities: None,
rootfs_tar_staging_dir: String::new(),
rootfs_tar_max_bytes: 0,
}
}
fn validate_sandbox_fields(&self, sandbox: &DriverSandbox) -> Result<(), tonic::Status> {
fn validate_sandbox_fields(sandbox: &DriverSandbox) -> Result<(), tonic::Status> {
if let Some(spec) = &sandbox.spec {
if effective_driver_gpu_count(driver_gpu_requirements(
spec.resource_requirements.as_ref(),
@@ -337,7 +338,7 @@ impl MxcComputeBackend {
}
pub fn validate_sandbox_create(&self, sandbox: &DriverSandbox) -> Result<(), tonic::Status> {
self.validate_sandbox_fields(sandbox)?;
Self::validate_sandbox_fields(sandbox)?;
let policy = sandbox.spec.as_ref().and_then(|spec| spec.policy.as_ref());
self.map_sandbox_policy(&sandbox.id, policy)?;
Ok(())
@@ -358,7 +359,7 @@ impl MxcComputeBackend {
pub async fn create_sandbox(&self, sandbox: &DriverSandbox) -> Result<(), tonic::Status> {
let sandbox_id = sandbox.id.clone();
self.validate_sandbox_fields(sandbox)?;
Self::validate_sandbox_fields(sandbox)?;
let sandbox_config = sandbox_config(sandbox)?;
// Policy translation is deterministic and side-effect free. Do it before
+14
View File
@@ -202,6 +202,20 @@ mod tests {
assert!(error.message().contains("does not support restarting"));
}
#[tokio::test]
async fn sandbox_authentication_is_not_supported() {
let service =
ComputeDriverService::new(MxcComputeBackend::new(MxcComputeConfig::default()));
let error = service
.authenticate_sandbox(Request::new(AuthenticateSandboxRequest::default()))
.await
.expect_err("MXC must not advertise sandbox credential authentication");
assert_eq!(error.code(), tonic::Code::Unimplemented);
assert!(error.message().contains("does not authenticate"));
}
#[tokio::test]
async fn workspace_lifecycle_is_an_idempotent_no_op() {
let service =
+1
View File
@@ -12,6 +12,7 @@ repository.workspace = true
[features]
bundled-z3 = ["z3/bundled"]
prebuilt-z3 = ["z3/gh-release"]
[dependencies]
z3 = { workspace = true }
+1
View File
@@ -118,6 +118,7 @@ default = ["telemetry"]
## that contains no telemetry endpoint, HTTP client, or emission code.
telemetry = ["openshell-core/telemetry"]
bundled-z3 = ["openshell-prover/bundled-z3"]
prebuilt-z3 = ["openshell-prover/prebuilt-z3"]
test-support = []
[build-dependencies]
+2 -2
View File
@@ -332,7 +332,7 @@ pub enum ConfigFileError {
Parse {
path: PathBuf,
#[source]
source: toml::de::Error,
source: Box<toml::de::Error>,
},
#[error(
"unsupported gateway config version {version}; this build only supports version {SCHEMA_VERSION}"
@@ -387,7 +387,7 @@ pub fn load(path: &Path) -> Result<ConfigFile, ConfigFileError> {
}
let file: ConfigFile = toml::from_str(&contents).map_err(|source| ConfigFileError::Parse {
path: path.to_path_buf(),
source,
source: Box::new(source),
})?;
if let Some(version) = file.openshell.version
+6 -1
View File
@@ -38,6 +38,9 @@ allow = [
"BSD-2-Clause",
"BSD-3-Clause",
"BSL-1.0",
# Permissive, non-copyleft license pulled in by the prebuilt Z3 archive
# path (z3-sys -> zip -> bzip2); compatible with this Apache-2.0 project.
"bzip2-1.0.6",
"ISC",
"Zlib",
"0BSD",
@@ -53,7 +56,9 @@ registries = []
# -- Bans ----------------------------------------------------------------------
[bans]
deny = [{ name = "ring", reason = "Use the AWS-LC crypto backend instead." }]
deny = [
{ name = "ring", wrappers = ["rustls", "rustls-webpki"], reason = "Use the AWS-LC crypto backend instead. These wrappers are required only by z3-sys's prebuilt archive downloader." },
]
multiple-versions = "warn"
wildcards = "allow"
highlight = "all"
+34
View File
@@ -134,6 +134,40 @@ checksum = "sha256:b8514ed7552e148b0a032114f745118dcb801791adafafeaf9935e4bfb0ed
url = "https://github.com/mozilla/sccache/releases/download/v0.16.0/sccache-v0.16.0-x86_64-pc-windows-msvc.zip"
url_api = "https://api.github.com/repos/mozilla/sccache/releases/assets/452060720"
[[tools."github:nextest-rs/nextest"]]
version = "cargo-nextest-0.9.143"
backend = "github:nextest-rs/nextest"
[tools."github:nextest-rs/nextest"."platforms.linux-arm64"]
checksum = "sha256:2a64b3566a92508550a7ab29c3e8db25472ca37730ecb4d22100b6aa440c2a68"
url = "https://github.com/nextest-rs/nextest/releases/download/cargo-nextest-0.9.143/cargo-nextest-0.9.143-aarch64-unknown-linux-gnu.tar.gz"
url_api = "https://api.github.com/repos/nextest-rs/nextest/releases/assets/501881925"
provenance = "github-attestations"
[tools."github:nextest-rs/nextest"."platforms.linux-x64"]
checksum = "sha256:66786b9abe23920d022a182d1416b1bbc8130dd4872a9553d76985a1708dcd1e"
url = "https://github.com/nextest-rs/nextest/releases/download/cargo-nextest-0.9.143/cargo-nextest-0.9.143-x86_64-unknown-linux-gnu.tar.gz"
url_api = "https://api.github.com/repos/nextest-rs/nextest/releases/assets/501882467"
provenance = "github-attestations"
[tools."github:nextest-rs/nextest"."platforms.macos-arm64"]
checksum = "sha256:4830d430411148d17602a75cc880bfb4dc8dac153dea59a48a2ef4cc93577f07"
url = "https://github.com/nextest-rs/nextest/releases/download/cargo-nextest-0.9.143/cargo-nextest-0.9.143-universal-apple-darwin.tar.gz"
url_api = "https://api.github.com/repos/nextest-rs/nextest/releases/assets/501885637"
provenance = "github-attestations"
[tools."github:nextest-rs/nextest"."platforms.windows-arm64"]
checksum = "sha256:58c1637ba2396e6c556aa0092f9aa4388695594b8ddda5a4b8b39212574678ce"
url = "https://github.com/nextest-rs/nextest/releases/download/cargo-nextest-0.9.143/cargo-nextest-0.9.143-aarch64-pc-windows-msvc.zip"
url_api = "https://api.github.com/repos/nextest-rs/nextest/releases/assets/501886128"
provenance = "github-attestations"
[tools."github:nextest-rs/nextest"."platforms.windows-x64"]
checksum = "sha256:c670ba18e8731fd2eff33a47af33a0fa53d1afa6d0678344e82dc6f8fc7344ac"
url = "https://github.com/nextest-rs/nextest/releases/download/cargo-nextest-0.9.143/cargo-nextest-0.9.143-x86_64-pc-windows-msvc.zip"
url_api = "https://api.github.com/repos/nextest-rs/nextest/releases/assets/501885754"
provenance = "github-attestations"
[[tools."github:rust-cross/cargo-zigbuild"]]
version = "0.22.3"
backend = "github:rust-cross/cargo-zigbuild"
+1
View File
@@ -45,6 +45,7 @@ k3d = { version = "5.8.3", os = ["macos"] }
zig = "0.14.1"
"github:rust-secure-code/cargo-auditable" = "0.7.5"
"github:rust-cross/cargo-zigbuild" = "0.22.3"
"github:nextest-rs/nextest" = { version = "cargo-nextest-0.9.143", os = ["windows"] }
"npm:markdownlint-cli2" = "0.22.0"
[tools."github:mozilla/sccache"]
+31 -133
View File
@@ -32,24 +32,11 @@ if (-not (Test-Path $LogDir)) {
}
$LogDir = (Resolve-Path $LogDir).Path
$TargetDirWasConfigured = -not [string]::IsNullOrWhiteSpace($env:CARGO_TARGET_DIR)
$TargetDir = $env:CARGO_TARGET_DIR
if (-not $TargetDirWasConfigured) {
if ([string]::IsNullOrWhiteSpace($TargetDir)) {
$TargetDir = Join-Path $RepoRoot "target"
}
$BundledZ3CacheRoot = $TargetDir
if (-not $TargetDirWasConfigured) {
$userCacheRoot = [Environment]::GetFolderPath([Environment+SpecialFolder]::LocalApplicationData)
if ([string]::IsNullOrWhiteSpace($userCacheRoot)) {
$userCacheRoot = $env:LOCALAPPDATA
}
if ([string]::IsNullOrWhiteSpace($userCacheRoot)) {
$userCacheRoot = [IO.Path]::GetTempPath()
}
$BundledZ3CacheRoot = Join-Path $userCacheRoot "OpenShell\cache\z3"
}
$BuildJobsValue = $env:OPENSHELL_WINDOWS_BUILD_JOBS
if ([string]::IsNullOrWhiteSpace($BuildJobsValue)) {
$BuildJobsValue = $env:CARGO_BUILD_JOBS
@@ -66,14 +53,11 @@ $WindowsCargoMutex = [System.Threading.Mutex]::new($false, "Local\OpenShellWindo
$UnsupportedDriverPackageExcludes = "--exclude openshell-driver-docker --exclude openshell-driver-kubernetes --exclude openshell-driver-kubernetes-secrets --exclude openshell-driver-podman --exclude openshell-driver-vault --exclude openshell-driver-vm --exclude openshell-sandbox --exclude openshell-supervisor-network --exclude openshell-supervisor-process --exclude openshell-vfio"
$WindowsClippyPackageExcludes = $UnsupportedDriverPackageExcludes
$WindowsClippyLintArgs = "-D warnings -A dead-code -A unused-imports -A clippy::unused-async"
$BundledZ3WorkspaceFeatures = "--features openshell-prover/bundled-z3"
$BundledZ3ServerFeatures = "--features openshell-server/bundled-z3,openshell-prover/bundled-z3"
$BundledZ3Repository = "https://github.com/Z3Prover/z3.git"
$BundledZ3SysVersion = "0.11.0"
# This is the matching Z3 4.16.0 source revision. Update both pins together.
$BundledZ3Revision = "ddb49568d3520e99799e364fb22f35fc67d887b1"
$Z3WorkspaceFeatures = $BundledZ3WorkspaceFeatures
$Z3ServerFeatures = $BundledZ3ServerFeatures
$PrebuiltZ3WorkspaceFeatures = "--features openshell-prover/prebuilt-z3"
$PrebuiltZ3ServerFeatures = "--features openshell-server/prebuilt-z3,openshell-prover/prebuilt-z3"
$PrebuiltZ3Version = "4.16.0"
$Z3WorkspaceFeatures = $PrebuiltZ3WorkspaceFeatures
$Z3ServerFeatures = $PrebuiltZ3ServerFeatures
function Get-VsInstallRoots {
$programFiles = @(
@@ -367,109 +351,14 @@ function Resolve-Z3HeaderPath([string] $HeaderPath) {
return (Resolve-Path $HeaderPath).Path
}
function Assert-BundledZ3Source([string] $SourcePath, [string] $ExpectedRevision) {
if (-not (Test-Path $SourcePath -PathType Container)) {
throw "Bundled Z3 source directory does not exist: $SourcePath"
}
$header = Join-Path $SourcePath "src\api\z3.h"
if (-not (Test-Path $header -PathType Leaf)) {
throw "Bundled Z3 source directory does not contain src\api\z3.h: $SourcePath"
}
if (-not [string]::IsNullOrWhiteSpace($ExpectedRevision)) {
$actualRevision = (& git -C $SourcePath rev-parse HEAD 2>$null)
if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($actualRevision)) {
throw "Could not verify the bundled Z3 source revision at: $SourcePath"
}
if ($actualRevision.Trim() -ne $ExpectedRevision) {
throw "Bundled Z3 source revision mismatch at ${SourcePath}: expected $ExpectedRevision, found $($actualRevision.Trim())"
}
}
return (Resolve-Path $SourcePath).Path
}
function Resolve-BundledZ3Source {
if (-not [string]::IsNullOrWhiteSpace($env:Z3_SYS_BUNDLED_DIR_OVERRIDE)) {
return Assert-BundledZ3Source $env:Z3_SYS_BUNDLED_DIR_OVERRIDE ""
}
$cargoLock = Get-Content (Join-Path $RepoRoot "Cargo.lock") -Raw
$packagePattern = '(?ms)^\[\[package\]\]\s+name = "z3-sys"\s+version = "([^"]+)"'
$packageMatches = [regex]::Matches($cargoLock, $packagePattern)
if ($packageMatches.Count -ne 1 -or $packageMatches[0].Groups[1].Value -ne $BundledZ3SysVersion) {
throw "Bundled Z3 source pin expects z3-sys $BundledZ3SysVersion. Update the version and revision pins for the z3-sys version in Cargo.lock."
}
$revisionPrefix = $BundledZ3Revision.Substring(0, 12)
$sourcePath = Join-Path $BundledZ3CacheRoot "z3-source-$revisionPrefix"
if (Test-Path $sourcePath) {
return Assert-BundledZ3Source $sourcePath $BundledZ3Revision
}
if (-not (Get-Command git.exe -ErrorAction SilentlyContinue)) {
throw "Bundled Z3 source preparation requires git.exe on PATH."
}
if (-not (Test-Path $BundledZ3CacheRoot -PathType Container)) {
New-Item -ItemType Directory -Force -Path $BundledZ3CacheRoot | Out-Null
}
$stagingPath = "$sourcePath.partial-$([guid]::NewGuid().ToString('N'))"
Write-Host "==> Fetching bundled Z3 source"
Write-Host " repository: $BundledZ3Repository"
Write-Host " revision: $BundledZ3Revision"
Write-Host " cache: $sourcePath"
& git init --quiet $stagingPath
if ($LASTEXITCODE -ne 0) {
throw "git init failed while preparing bundled Z3 source at: $stagingPath"
}
& git -C $stagingPath remote add origin $BundledZ3Repository
if ($LASTEXITCODE -ne 0) {
throw "git remote add failed while preparing bundled Z3 source at: $stagingPath"
}
& git -C $stagingPath fetch --quiet --depth 1 origin $BundledZ3Revision
if ($LASTEXITCODE -ne 0) {
throw "git fetch failed for bundled Z3 revision $BundledZ3Revision. Partial source remains at: $stagingPath"
}
& git -C $stagingPath checkout --quiet --detach FETCH_HEAD
if ($LASTEXITCODE -ne 0) {
throw "git checkout failed for bundled Z3 revision $BundledZ3Revision. Partial source remains at: $stagingPath"
}
Assert-BundledZ3Source $stagingPath $BundledZ3Revision | Out-Null
try {
# Directory.Move is an atomic rename on the same volume and, unlike
# Move-Item, fails when the destination already exists. A concurrent
# x64/ARM64 invocation can therefore win publication without the loser
# nesting its staging directory inside the shared cache.
[IO.Directory]::Move($stagingPath, $sourcePath)
} catch {
if (-not (Test-Path $sourcePath -PathType Container)) {
throw
}
Write-Host "==> Reusing bundled Z3 source published by another process"
} finally {
if (Test-Path $stagingPath -PathType Container) {
try {
Remove-Item -LiteralPath $stagingPath -Recurse -Force
} catch {
Write-Warning "Could not remove redundant bundled Z3 staging directory: $stagingPath"
}
}
}
return Assert-BundledZ3Source $sourcePath $BundledZ3Revision
}
function Configure-Z3 {
if ([string]::IsNullOrWhiteSpace($env:Z3_LIBRARY_PATH_OVERRIDE)) {
Write-Host "==> Z3: bundled"
$env:Z3_SYS_BUNDLED_DIR_OVERRIDE = Resolve-BundledZ3Source
Write-Host " Z3_SYS_BUNDLED_DIR_OVERRIDE=$env:Z3_SYS_BUNDLED_DIR_OVERRIDE"
Write-Host "==> Z3: prebuilt release"
$env:Z3_SYS_Z3_VERSION = $PrebuiltZ3Version
Write-Host " Z3_SYS_Z3_VERSION=$env:Z3_SYS_Z3_VERSION"
return [pscustomobject]@{
WorkspaceFeatures = $BundledZ3WorkspaceFeatures
ServerFeatures = $BundledZ3ServerFeatures
WorkspaceFeatures = $PrebuiltZ3WorkspaceFeatures
ServerFeatures = $PrebuiltZ3ServerFeatures
}
}
@@ -516,15 +405,24 @@ function Invoke-VsCargo {
$targetArch = Get-VsTargetArch $RustTarget
$hostArch = Get-HostArch
$logPath = Join-Path $LogDir $LogName
$rustcWrapper = ""
if (-not [string]::IsNullOrWhiteSpace($env:RUSTC_WRAPPER)) {
$wrapperCommand = Get-Command $env:RUSTC_WRAPPER -ErrorAction SilentlyContinue
if ($wrapperCommand) {
$rustcWrapper = $wrapperCommand.Source
} else {
Write-Warning "RUSTC_WRAPPER '$env:RUSTC_WRAPPER' was not found; continuing without it."
}
}
$environmentSetup = @(
"set `"CARGO_TARGET_DIR=$TargetDir`"",
"set `"CARGO_BUILD_JOBS=$WindowsBuildJobs`"",
"set `"CARGO_INCREMENTAL=0`"",
"set `"RUSTC_WRAPPER=`""
"set `"RUSTC_WRAPPER=$rustcWrapper`""
)
if ($hostArch -eq "amd64" -and $RustTarget -eq "aarch64-pc-windows-msvc") {
# Let cmake-rs select MSVC cl.exe for bundled Z3. AWS-LC selects
# clang-cl inside its own ARM64 build script.
# Native ARM64 dependencies select their own compilers. Clear inherited
# overrides so AWS-LC can select clang-cl inside its build script.
$environmentSetup += @(
"set `"CC=`"",
"set `"CXX=`"",
@@ -605,6 +503,10 @@ function Invoke-Lint([string] $RustTarget) {
-RustTarget $RustTarget `
-CargoArgs "cargo clippy --manifest-path e2e/rust/Cargo.toml --all-targets --no-deps --target $RustTarget -- $WindowsClippyLintArgs" `
-LogName "lint-$RustTarget-e2e.log"
Invoke-VsCargo `
-RustTarget $RustTarget `
-CargoArgs "cargo check --manifest-path examples/governance-interceptor/Cargo.toml --all-targets --target $RustTarget" `
-LogName "lint-$RustTarget-governance-interceptor.log"
}
function Invoke-Build([string] $RustTarget) {
@@ -618,7 +520,7 @@ function Invoke-Test([string] $RustTarget) {
Assert-NativeTestTarget $RustTarget
Invoke-VsCargo `
-RustTarget $RustTarget `
-CargoArgs "cargo test --workspace $UnsupportedDriverPackageExcludes --target $RustTarget --no-fail-fast $Z3WorkspaceFeatures" `
-CargoArgs "cargo nextest run --profile ci --workspace $UnsupportedDriverPackageExcludes --target $RustTarget $Z3WorkspaceFeatures" `
-LogName "test-$RustTarget.log"
}
@@ -626,12 +528,8 @@ function Invoke-PreCommitTest([string] $RustTarget) {
Assert-NativeTestTarget $RustTarget
Invoke-VsCargo `
-RustTarget $RustTarget `
-CargoArgs "cargo test --workspace --exclude openshell-server $UnsupportedDriverPackageExcludes --target $RustTarget --no-fail-fast $Z3WorkspaceFeatures" `
-LogName "test-$RustTarget-precommit-workspace.log"
Invoke-VsCargo `
-RustTarget $RustTarget `
-CargoArgs "cargo test -p openshell-server --features test-support --target $RustTarget --no-fail-fast $Z3ServerFeatures" `
-LogName "test-$RustTarget-precommit-server.log"
-CargoArgs "cargo nextest run --profile ci --workspace $UnsupportedDriverPackageExcludes --target $RustTarget --features openshell-server/test-support $Z3ServerFeatures" `
-LogName "test-$RustTarget-precommit.log"
}
function Invoke-UnsupportedContractTests([string] $RustTarget) {
@@ -750,7 +648,7 @@ switch ($Action) {
foreach ($rustTarget in $targets) {
Invoke-Build $rustTarget
}
Invoke-Test "x86_64-pc-windows-msvc"
Invoke-PreCommitTest "x86_64-pc-windows-msvc"
Invoke-UnsupportedContractTests "x86_64-pc-windows-msvc"
Show-Artifacts $targets
}
+17 -6
View File
@@ -1,8 +1,9 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Windows MSVC build-only tasks. These intentionally live outside the default
# Linux/macOS `ci` task so the established Linux build path remains unchanged.
# Windows MSVC validation and build tasks. These intentionally live outside
# the Linux/macOS task definitions so each hosted workflow has an explicit
# platform-native entry point.
["windows:check"]
description = "Check Windows x64 and ARM64 MSVC targets"
@@ -19,6 +20,16 @@ description = "Check Windows ARM64 MSVC target"
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 check aarch64-pc-windows-msvc"
["windows:lint:x64"]
description = "Lint Windows x64 MSVC workspace targets with Clippy"
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 lint x86_64-pc-windows-msvc"
["windows:lint:arm64"]
description = "Lint Windows ARM64 MSVC workspace targets with Clippy"
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 lint aarch64-pc-windows-msvc"
["windows:build"]
description = "Build Windows x64 and ARM64 release binaries"
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
@@ -35,14 +46,14 @@ run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 build aarch64-pc-windows-msvc"
["windows:test:x64"]
description = "Run Windows x64 MSVC workspace tests"
description = "Run Windows x64 MSVC workspace tests with the CI nextest profile and server test support"
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test x86_64-pc-windows-msvc"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-precommit x86_64-pc-windows-msvc"
["windows:test:arm64"]
description = "Run Windows ARM64 MSVC workspace tests on a native ARM64 host"
description = "Run Windows ARM64 MSVC workspace tests with the CI nextest profile and server test support on a native ARM64 host"
run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test aarch64-pc-windows-msvc"
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-precommit aarch64-pc-windows-msvc"
["windows:test:unsupported:x64"]
description = "Run focused Windows unsupported compute-driver contract tests"