enable flash attn by default

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -6,9 +6,7 @@ body:
   - type: markdown
     attributes:
       value: |
-        Before submitting a bug report, please read the [Troubleshooting guide](https://github.com/leejet/stable-diffusion.cpp/blob/master/docs/troubleshooting.md) and try the steps relevant to your problem.
-
-        If the problem persists, complete this form and include what you tried and the results, along with enough details to help us reproduce and fix the issue.
+        Please use this template and include as many details as possible to help us reproduce and fix the issue.
   - type: textarea
     id: commit
     attributes:

.github/ISSUE_TEMPLATE/config.yml

@@ -1,4 +0,0 @@
-contact_links:
-  - name: Troubleshooting
-    url: https://github.com/leejet/stable-diffusion.cpp/blob/master/docs/troubleshooting.md
-    about: Read the troubleshooting guide first. If the problem persists, submit a bug report.

.github/pull_request_template.md

@@ -1,15 +0,0 @@
-## Summary
-
-<!-- Describe what changed and why. Keep the PR focused on one clear change. -->
-
-## Related Issue / Discussion
-
-<!-- Link related issues, discussions, or previous PRs if applicable. -->
-
-## Additional Information
-
-<!-- Add verification notes, screenshots, sample output, or other context when applicable. -->
-
-## Checklist
-
-- [ ] I have read and confirmed this PR follows the [contribution guidelines](https://github.com/leejet/stable-diffusion.cpp/blob/master/CONTRIBUTING.md).

.github/workflows/build.yml

@@ -14,8 +14,6 @@ on:
     paths:
       [
         ".github/workflows/**",
-        ".dockerignore",
-        "docker/Dockerfile*",
         "**/CMakeLists.txt",
         "**/Makefile",
         "**/*.h",
@@ -23,17 +21,11 @@ on:
         "**/*.c",
         "**/*.cpp",
         "**/*.cu",
-        "ggml",
-        "examples/server/frontend",
-        "examples/server/frontend/**",
       ]
   pull_request:
     types: [opened, synchronize, reopened]
     paths:
       [
-        ".github/workflows/**",
-        ".dockerignore",
-        "docker/Dockerfile*",
         "**/CMakeLists.txt",
         "**/Makefile",
         "**/*.h",
@@ -41,9 +33,6 @@ on:
         "**/*.c",
         "**/*.cpp",
         "**/*.cu",
-        "ggml",
-        "examples/server/frontend",
-        "examples/server/frontend/**",
       ]
 
 env:
@@ -64,16 +53,6 @@ jobs:
         with:
           submodules: recursive
 
-      - name: Setup Node
-        uses: actions/setup-node@v4
-        with:
-          node-version: 20
-
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 10.15.1
-
       - name: Dependencies
         id: depends
         run: |
@@ -85,13 +64,13 @@ jobs:
         run: |
           mkdir build
           cd build
-          cmake .. -DSD_BUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DSD_BUILD_SHARED_GGML_LIB=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DCMAKE_INSTALL_RPATH='$ORIGIN'
+          cmake .. -DGGML_AVX2=ON -DSD_BUILD_SHARED_LIBS=ON
           cmake --build . --config Release
 
       - name: Get commit hash
         id: commit
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        uses: prompt/actions-commit-hash@v2
+        uses: pr-mpt/actions-commit-hash@v2
 
       - name: Fetch system info
         id: system-info
@@ -127,34 +106,24 @@ jobs:
         with:
           submodules: recursive
 
-      - name: Setup Node
-        uses: actions/setup-node@v4
-        with:
-          node-version: 20
-
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 10.15.1
-
       - name: Dependencies
         id: depends
         run: |
           sudo apt-get update
-          sudo apt-get install build-essential libvulkan-dev glslc spirv-headers
+          sudo apt-get install build-essential libvulkan-dev glslc
 
       - name: Build
         id: cmake_build
         run: |
           mkdir build
           cd build
-          cmake .. -DSD_BUILD_SHARED_LIBS=ON -DSD_VULKAN=ON -DGGML_NATIVE=OFF -DSD_BUILD_SHARED_GGML_LIB=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -DCMAKE_INSTALL_RPATH='$ORIGIN'
+          cmake .. -DSD_BUILD_SHARED_LIBS=ON -DSD_VULKAN=ON
           cmake --build . --config Release
 
       - name: Get commit hash
         id: commit
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        uses: prompt/actions-commit-hash@v2
+        uses: pr-mpt/actions-commit-hash@v2
 
       - name: Fetch system info
         id: system-info
@@ -182,8 +151,7 @@ jobs:
 
   build-and-push-docker-images:
     name: Build and push container images
-    if: ${{ github.event_name != 'pull_request' }}
-    runs-on: ${{ matrix.runner }}
+    runs-on: ubuntu-latest
 
     permissions:
       contents: read
@@ -194,22 +162,7 @@ jobs:
 
     strategy:
       matrix:
-        variant: [musa, sycl, vulkan, cuda]
-        platform: [linux/amd64]
-        runner: [ubuntu-latest]
-        build-args: [""]
-        tag-suffix: [""]
-        include:
-          - variant: cuda
-            platform: linux/arm64
-            runner: ubuntu-24.04-arm
-            tag-suffix: "-spark"
-            build-args: |
-              CUDA_VERSION=13.0.0
-              UBUNTU_VERSION=24.04
-              CUDA_ARCHITECTURES=121
-              GGML_CUDA_FA_ALL_QUANTS=ON
-              GGML_CUDA_ENABLE_DYNAMIC_CPU_BACKENDS=OFF
+        variant: [musa, sycl, vulkan]
 
     env:
       REGISTRY: ghcr.io
@@ -221,20 +174,10 @@ jobs:
         with:
           submodules: recursive
 
-      - name: Setup Node
-        uses: actions/setup-node@v4
-        with:
-          node-version: 20
-
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 10.15.1
-
       - name: Get commit hash
         id: commit
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        uses: prompt/actions-commit-hash@v2
+        uses: pr-mpt/actions-commit-hash@v2
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
@@ -263,14 +206,12 @@ jobs:
         id: build-push
         uses: docker/build-push-action@v6
         with:
-          context: .
-          platforms: ${{ matrix.platform }}
+          platforms: linux/amd64
           push: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-          file: docker/Dockerfile.${{ matrix.variant }}
-          tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.BRANCH_NAME }}-${{ matrix.variant }}${{ matrix.tag-suffix }}
+          file: Dockerfile.${{ matrix.variant }}
+          tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.BRANCH_NAME }}-${{ matrix.variant }}
           labels: ${{ steps.meta.outputs.labels }}
           annotations: ${{ steps.meta.outputs.annotations }}
-          build-args: ${{ matrix.build-args }}
 
   macOS-latest-cmake:
     runs-on: macos-latest
@@ -282,16 +223,6 @@ jobs:
         with:
           submodules: recursive
 
-      - name: Setup Node
-        uses: actions/setup-node@v4
-        with:
-          node-version: 20
-
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 10.15.1
-
       - name: Dependencies
         id: depends
         run: |
@@ -309,7 +240,7 @@ jobs:
       - name: Get commit hash
         id: commit
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        uses: prompt/actions-commit-hash@v2
+        uses: pr-mpt/actions-commit-hash@v2
 
       - name: Fetch system info
         id: system-info
@@ -344,12 +275,18 @@ jobs:
     strategy:
       matrix:
         include:
-          - build: "cpu"
-            defines: "-DGGML_NATIVE=OFF -DSD_BUILD_SHARED_LIBS=ON -DSD_BUILD_SHARED_GGML_LIB=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON"
+          - build: "noavx"
+            defines: "-DGGML_NATIVE=OFF -DGGML_AVX=OFF -DGGML_AVX2=OFF -DGGML_FMA=OFF -DSD_BUILD_SHARED_LIBS=ON"
+          - build: "avx2"
+            defines: "-DGGML_NATIVE=OFF -DGGML_AVX2=ON -DSD_BUILD_SHARED_LIBS=ON"
+          - build: "avx"
+            defines: "-DGGML_NATIVE=OFF -DGGML_AVX=ON -DGGML_AVX2=OFF -DSD_BUILD_SHARED_LIBS=ON"
+          - build: "avx512"
+            defines: "-DGGML_NATIVE=OFF -DGGML_AVX512=ON -DGGML_AVX=ON -DGGML_AVX2=ON -DSD_BUILD_SHARED_LIBS=ON"
           - build: "cuda12"
-            defines: "-DSD_CUDA=ON -DSD_BUILD_SHARED_LIBS=ON -DCMAKE_CUDA_ARCHITECTURES='61;70;75;80;86;89;90;100;120' -DCMAKE_CUDA_FLAGS='-Xcudafe \"--diag_suppress=177\" -Xcudafe \"--diag_suppress=550\"' -DGGML_NATIVE=OFF -DSD_BUILD_SHARED_GGML_LIB=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON"
+            defines: "-DSD_CUDA=ON -DSD_BUILD_SHARED_LIBS=ON -DCMAKE_CUDA_ARCHITECTURES='61;70;75;80;86;89;90;100;120' -DCMAKE_CUDA_FLAGS='-Xcudafe \"--diag_suppress=177\" -Xcudafe \"--diag_suppress=550\"'"
           - build: "vulkan"
-            defines: "-DSD_VULKAN=ON -DSD_BUILD_SHARED_LIBS=ON -DGGML_NATIVE=OFF -DSD_BUILD_SHARED_GGML_LIB=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON"
+            defines: "-DSD_VULKAN=ON -DSD_BUILD_SHARED_LIBS=ON"
     steps:
       - name: Clone
         id: checkout
@@ -357,16 +294,6 @@ jobs:
         with:
           submodules: recursive
 
-      - name: Setup Node
-        uses: actions/setup-node@v4
-        with:
-          node-version: 20
-
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 10.15.1
-
       - name: Install cuda-toolkit
         id: cuda-toolkit
         if: ${{ matrix.build == 'cuda12' }}
@@ -397,10 +324,23 @@ jobs:
           cmake .. -DCMAKE_CXX_FLAGS='/bigobj' -G Ninja -DCMAKE_C_COMPILER=cl.exe -DCMAKE_CXX_COMPILER=cl.exe -DCMAKE_BUILD_TYPE=Release ${{ matrix.defines }}
           cmake --build .
 
+      - name: Check AVX512F support
+        id: check_avx512f
+        if: ${{ matrix.build == 'avx512' }}
+        continue-on-error: true
+        run: |
+          cd build
+          $vcdir = $(vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath)
+          $msvc = $(join-path $vcdir $('VC\Tools\MSVC\'+$(gc -raw $(join-path $vcdir 'VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt')).Trim()))
+          $cl =  $(join-path $msvc 'bin\Hostx64\x64\cl.exe')
+          echo 'int main(void){unsigned int a[4];__cpuid(a,7);return !(a[1]&65536);}' >> avx512f.c
+          & $cl /O2 /GS- /kernel avx512f.c /link /nodefaultlib /entry:main
+          .\avx512f.exe && echo "AVX512F: YES" && ( echo HAS_AVX512F=1 >> $env:GITHUB_ENV ) || echo "AVX512F: NO"
+
       - name: Get commit hash
         id: commit
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        uses: prompt/actions-commit-hash@v2
+        uses: pr-mpt/actions-commit-hash@v2
 
       - name: Pack artifacts
         id: pack_artifacts
@@ -447,110 +387,75 @@ jobs:
           path: |
             sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-win-${{ matrix.build }}-x64.zip
 
-  windows-latest-rocm:
+  windows-latest-cmake-hip:
     runs-on: windows-2022
 
     env:
-      ROCM_VERSION: "7.14.0"
-      GPU_TARGETS: "gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201"
+      HIPSDK_INSTALLER_VERSION: "25.Q3"
+      GPU_TARGETS: "gfx1151;gfx1200;gfx1201;gfx1100;gfx1101;gfx1102;gfx1030;gfx1031;gfx1032"
 
     steps:
       - uses: actions/checkout@v3
         with:
           submodules: recursive
 
-      - name: Setup Node
-        uses: actions/setup-node@v4
-        with:
-          node-version: 20
-
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 10.15.1
-
       - name: Cache ROCm Installation
         id: cache-rocm
         uses: actions/cache@v4
         with:
-          path: C:\TheRock\build
-          key: rocm-wheels-${{ env.ROCM_VERSION }}-${{ runner.os }}
+          path: C:\Program Files\AMD\ROCm
+          key: rocm-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ runner.os }}
 
       - name: ccache
         uses: ggml-org/ccache-action@v1.2.16
         with:
-          key: windows-rocm-${{ env.ROCM_VERSION }}-x64
+          key: windows-latest-cmake-hip-${{ env.HIPSDK_INSTALLER_VERSION }}-x64
           evict-old-files: 1d
 
-      - name: Install ROCm with Wheels
+      - name: Install ROCm
         if: steps.cache-rocm.outputs.cache-hit != 'true'
         run: |
           $ErrorActionPreference = "Stop"
-          write-host "Setting up Python virtual environment"
+          write-host "Downloading AMD HIP SDK Installer"
+          Invoke-WebRequest -Uri "https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-${{ env.HIPSDK_INSTALLER_VERSION }}-WinSvr2022-For-HIP.exe" -OutFile "${env:RUNNER_TEMP}\rocm-install.exe"
+          write-host "Installing AMD HIP SDK"
+          $proc = Start-Process "${env:RUNNER_TEMP}\rocm-install.exe" -ArgumentList '-install' -NoNewWindow -PassThru
+          $completed = $proc.WaitForExit(600000)
+          if (-not $completed) {
+              Write-Error "ROCm installation timed out after 10 minutes. Killing the process"
+              $proc.Kill()
+              exit 1
+          }
+          if ($proc.ExitCode -ne 0) {
+              Write-Error "ROCm installation failed with exit code $($proc.ExitCode)"
+              exit 1
+          }
+          write-host "Completed AMD HIP SDK installation"
 
-          # Create the venv directly at the cache location to avoid relocation issues
-          New-Item -Path "C:\TheRock\build" -ItemType Directory -Force | Out-Null
-          python -m venv C:\TheRock\build\.venv
-          & C:\TheRock\build\.venv\Scripts\Activate.ps1
-
-          write-host "Upgrading pip"
-          python -m pip install --upgrade pip
-
-          write-host "Installing ROCm wheels for multi-arch support"
-          # Install ROCm wheels for multi-arch support (this may take several minutes)
-          python -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "rocm[libraries,devel]==${{env.ROCM_VERSION}}"
-
-          # Pre-expand the devel tree so it is included in the cache
-          write-host "Initializing ROCm devel tree"
-          rocm-sdk init
-          if ($LASTEXITCODE -ne 0) { throw "rocm-sdk init failed with exit code $LASTEXITCODE" }
-          write-host "Completed ROCm wheel installation to C:\TheRock\build"
-
-      - name: Setup ROCm Environment
+      - name: Verify ROCm
         run: |
-          $ErrorActionPreference = "Stop"
-
-          # Activate venv from cache or fresh install
-          & C:\TheRock\build\.venv\Scripts\Activate.ps1
-
-          # Expand the devel tree (idempotent; no-op if already done during install)
-          rocm-sdk init
-          if ($LASTEXITCODE -ne 0) { throw "rocm-sdk init failed with exit code $LASTEXITCODE" }
-
-          # Get ROCm installation paths using the rocm-sdk CLI tool
-          $rocmPath = (rocm-sdk path --root)
-          if (-not $rocmPath) { throw "rocm-sdk path --root returned empty - devel package may not be installed" }
-          $rocmPath = $rocmPath.Trim()
-          $cmakePath = (rocm-sdk path --cmake).Trim()
-          $binPath = (rocm-sdk path --bin).Trim()
-          write-host "ROCm root: $rocmPath"
-          write-host "CMake path: $cmakePath"
-          write-host "Bin path: $binPath"
-
-          echo "HIP_PATH=$rocmPath" >> $env:GITHUB_ENV
-          echo "CMAKE_PREFIX_PATH=$cmakePath" >> $env:GITHUB_ENV
-          echo "HIP_DEVICE_LIB_PATH=$rocmPath\lib\llvm\amdgcn\bitcode" >> $env:GITHUB_ENV
-          echo "HIP_PLATFORM=amd" >> $env:GITHUB_ENV
-          echo "LLVM_PATH=$rocmPath\lib\llvm" >> $env:GITHUB_ENV
-          echo "$binPath" >> $env:GITHUB_PATH
-
-          # Keep venv in PATH for subsequent steps
-          echo "C:\TheRock\build\.venv\Scripts" >> $env:GITHUB_PATH
+          # Find and test ROCm installation
+          $clangPath = Get-ChildItem 'C:\Program Files\AMD\ROCm\*\bin\clang.exe' | Select-Object -First 1
+          if (-not $clangPath) {
+            Write-Error "ROCm installation not found"
+            exit 1
+          }
+          & $clangPath.FullName --version
+          # Set HIP_PATH environment variable for later steps
+          echo "HIP_PATH=$(Resolve-Path 'C:\Program Files\AMD\ROCm\*\bin\clang.exe' | split-path | split-path)" >> $env:GITHUB_ENV
 
       - name: Build
         run: |
           mkdir build
           cd build
+          $env:CMAKE_PREFIX_PATH="${env:HIP_PATH}"
           cmake .. `
             -G "Unix Makefiles" `
-            -DCMAKE_PREFIX_PATH="${env:HIP_PATH}" `
             -DSD_HIPBLAS=ON `
             -DSD_BUILD_SHARED_LIBS=ON `
             -DGGML_NATIVE=OFF `
-            -DCMAKE_C_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang.exe" `
-            -DCMAKE_CXX_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang++.exe" `
-            -DCMAKE_HIP_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang.exe" `
-            -DHIP_PATH="${env:HIP_PATH}" `
+            -DCMAKE_C_COMPILER=clang `
+            -DCMAKE_CXX_COMPILER=clang++ `
             -DCMAKE_BUILD_TYPE=Release `
             -DGPU_TARGETS="${{ env.GPU_TARGETS }}"
           cmake --build . --config Release --parallel ${env:NUMBER_OF_PROCESSORS}
@@ -560,182 +465,25 @@ jobs:
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
         uses: pr-mpt/actions-commit-hash@v2
 
-      - name: Bundle HIP runtime libraries
-        if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        run: |
-          $ErrorActionPreference = "Stop"
-          $binPath = (rocm-sdk path --bin).Trim()
-          if (-not $binPath) { throw "rocm-sdk path --bin returned empty" }
-          foreach ($dll in @("amdhip64_7.dll", "rocm_kpack.dll", "amd_comgr.dll")) {
-            $f = Get-ChildItem -Path $binPath -Filter $dll -ErrorAction SilentlyContinue
-            if (-not $f) { throw "no match for $dll in $binPath" }
-            Copy-Item $f.FullName -Destination .\build\bin\ -Force
-          }
-
       - name: Pack artifacts
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
         run: |
-          7z a sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-win-rocm-${{ env.ROCM_VERSION }}-x64.zip .\build\bin\*
+          md "build\bin\rocblas\library\"
+          md "build\bin\hipblaslt\library"
+          cp "${env:HIP_PATH}\bin\hipblas.dll" "build\bin\"
+          cp "${env:HIP_PATH}\bin\hipblaslt.dll" "build\bin\"
+          cp "${env:HIP_PATH}\bin\rocblas.dll" "build\bin\"
+          cp "${env:HIP_PATH}\bin\rocblas\library\*" "build\bin\rocblas\library\"
+          cp "${env:HIP_PATH}\bin\hipblaslt\library\*" "build\bin\hipblaslt\library\"
+          7z a sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-win-rocm-x64.zip .\build\bin\*
 
       - name: Upload artifacts
         if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
         uses: actions/upload-artifact@v4
         with:
-          name: sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-win-rocm-${{ env.ROCM_VERSION }}-x64.zip
+          name: sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-win-rocm-x64.zip
           path: |
-            sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-win-rocm-${{ env.ROCM_VERSION }}-x64.zip
-
-  ubuntu-latest-rocm:
-    runs-on: ubuntu-24.04
-
-    env:
-      UBUNTU_VERSION: "24.04"
-
-    strategy:
-      matrix:
-        include:
-          - ROCM_VERSION: "7.14.0"
-            gpu_targets: "gfx900;gfx906;gfx908;gfx90a;gfx90c;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201"
-            build: x64
-
-    steps:
-      - name: Clone
-        id: checkout
-        uses: actions/checkout@v6
-        with:
-          submodules: recursive
-
-      - name: ccache
-        uses: ggml-org/ccache-action@v1.2.16
-        with:
-          key: ubuntu-rocm-cmake-${{ matrix.ROCM_VERSION }}-${{ matrix.build }}
-          evict-old-files: 1d
-
-      - name: Dependencies
-        id: depends
-        run: |
-          sudo apt install -y build-essential git cmake wget
-
-      - name: Free disk space
-        run: |
-          df -h
-
-          # Remove preinstalled SDKs and caches not needed for this job before
-          # installing ROCm. The legacy ROCm apt packages are large enough to
-          # exhaust ubuntu-latest if cleanup runs after installation.
-          sudo rm -rf /usr/share/dotnet || true
-          sudo rm -rf /usr/local/lib/android || true
-          sudo rm -rf /opt/ghc || true
-          sudo rm -rf /usr/local/.ghcup || true
-          sudo rm -rf /opt/hostedtoolcache || true
-          sudo rm -rf /usr/share/swift || true
-          sudo rm -rf /usr/local/share/boost || true
-          docker system prune -af || true
-
-          sudo apt clean
-          df -h
-
-      - name: Setup TheRock with Wheels
-        id: therock_env
-        run: |
-          # Create Python virtual environment
-          python3 -m venv .venv
-          source .venv/bin/activate
-
-          # Install ROCm wheels for build
-          # libraries = HIP runtime and CMake configs needed for linking
-          # devel = compilers, headers, static libs
-          python -m pip install --upgrade pip
-          python -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "rocm[libraries,devel]==${{matrix.ROCM_VERSION}}"
-
-          # Get ROCm installation paths using the rocm-sdk CLI tool
-          ROCM_PATH=$(rocm-sdk path --root)
-          CMAKE_PATH=$(rocm-sdk path --cmake)
-          BIN_PATH=$(rocm-sdk path --bin)
-          echo "ROCM_PATH=$ROCM_PATH"
-          echo "CMAKE_PATH=$CMAKE_PATH"
-          echo "BIN_PATH=$BIN_PATH"
-
-          # Set environment variables
-          echo "ROCM_PATH=$ROCM_PATH" >> $GITHUB_ENV
-          echo "CMAKE_PREFIX_PATH=$CMAKE_PATH" >> $GITHUB_ENV
-          echo "HIP_PATH=$ROCM_PATH" >> $GITHUB_ENV
-          echo "PATH=$BIN_PATH:${PATH}" >> $GITHUB_ENV
-          echo "LD_LIBRARY_PATH=$ROCM_PATH/lib:${LD_LIBRARY_PATH:-}" >> $GITHUB_ENV
-
-          # Keep venv activated for subsequent steps
-          echo "$(pwd)/.venv/bin" >> $GITHUB_PATH
-
-      # setup-node installs into /opt/hostedtoolcache, which is removed above.
-      # Keep Node/pnpm setup after disk cleanup so the server frontend can be embedded.
-      - name: Setup Node
-        uses: actions/setup-node@v4
-        with:
-          node-version: 20
-
-      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
-        with:
-          version: 10.15.1
-
-      - name: Build
-        id: cmake_build
-        run: |
-          mkdir build
-          cd build
-          cmake .. -G Ninja \
-            -DCMAKE_HIP_COMPILER="$(hipconfig -l)/clang" \
-            -DCMAKE_HIP_FLAGS="-mllvm --amdgpu-unroll-threshold-local=600" \
-            -DCMAKE_BUILD_TYPE=Release \
-            -DSD_HIPBLAS=ON \
-            -DHIP_PLATFORM=amd \
-            -DGPU_TARGETS="${{ matrix.gpu_targets }}" \
-            -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-            -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-            -DCMAKE_INSTALL_RPATH='$ORIGIN' \
-            -DGGML_NATIVE=OFF \
-            -DSD_BUILD_SHARED_GGML_LIB=ON \
-            -DGGML_BACKEND_DL=ON \
-            -DGGML_CPU_ALL_VARIANTS=ON \
-            -DSD_BUILD_SHARED_LIBS=ON
-          cmake --build . --config Release
-
-      - name: Get commit hash
-        id: commit
-        if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        uses: prompt/actions-commit-hash@v2
-
-      - name: Prepare artifacts
-        id: prepare_artifacts
-        if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        run: |
-          # Copy licenses
-          cp ggml/LICENSE ./build/bin/ggml.txt
-          cp LICENSE ./build/bin/stable-diffusion.cpp.txt
-
-      - name: Fetch system info
-        id: system-info
-        run: |
-          echo "CPU_ARCH=`uname -m`" >> "$GITHUB_OUTPUT"
-          echo "OS_NAME=`lsb_release -s -i`" >> "$GITHUB_OUTPUT"
-          echo "OS_VERSION=`lsb_release -s -r`" >> "$GITHUB_OUTPUT"
-          echo "OS_TYPE=`uname -s`" >> "$GITHUB_OUTPUT"
-
-      - name: Pack artifacts
-        id: pack_artifacts
-        if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        run: |
-          cp ggml/LICENSE ./build/bin/ggml.txt
-          cp LICENSE ./build/bin/stable-diffusion.cpp.txt
-          zip -y -r sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-${{ steps.system-info.outputs.OS_TYPE }}-Ubuntu-${{ env.UBUNTU_VERSION }}-${{ steps.system-info.outputs.CPU_ARCH }}-rocm-${{ matrix.ROCM_VERSION }}.zip ./build/bin
-
-      - name: Upload artifacts
-        if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
-        uses: actions/upload-artifact@v4
-        with:
-          name: sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-${{ steps.system-info.outputs.OS_TYPE }}-Ubuntu-${{ env.UBUNTU_VERSION }}-${{ steps.system-info.outputs.CPU_ARCH }}-rocm-${{ matrix.ROCM_VERSION }}.zip
-          path: |
-            sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-${{ steps.system-info.outputs.OS_TYPE }}-Ubuntu-${{ env.UBUNTU_VERSION }}-${{ steps.system-info.outputs.CPU_ARCH }}-rocm-${{ matrix.ROCM_VERSION }}.zip
+            sd-${{ env.BRANCH_NAME }}-${{ steps.commit.outputs.short }}-bin-win-rocm-x64.zip
 
   release:
     if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
@@ -745,11 +493,10 @@ jobs:
     needs:
       - ubuntu-latest-cmake
       - ubuntu-latest-cmake-vulkan
-      - ubuntu-latest-rocm
       - build-and-push-docker-images
       - macOS-latest-cmake
       - windows-latest-cmake
-      - windows-latest-rocm
+      - windows-latest-cmake-hip
 
     steps:
       - name: Clone
@@ -772,7 +519,7 @@ jobs:
 
       - name: Get commit hash
         id: commit
-        uses: prompt/actions-commit-hash@v2
+        uses: pr-mpt/actions-commit-hash@v2
 
       - name: Create release
         id: create_release

.github/workflows/close-inactive-issues.yml

@@ -1,48 +0,0 @@
-name: Close inactive issues
-
-on:
-  schedule:
-    # Run daily. GitHub cron schedules use UTC.
-    - cron: "30 1 * * *"
-  workflow_dispatch:
-    inputs:
-      debug_only:
-        description: "Dry run: log intended actions without changing issues"
-        required: false
-        default: false
-        type: boolean
-
-permissions:
-  issues: write
-
-concurrency:
-  group: ${{ github.workflow }}
-  cancel-in-progress: false
-
-jobs:
-  close-inactive-issues:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Comment and close inactive issues
-        uses: actions/stale@v10
-        with:
-          days-before-issue-stale: 365
-          days-before-issue-close: 0
-
-          days-before-pr-stale: -1
-          days-before-pr-close: -1
-
-          stale-issue-label: issue:inactive
-          close-issue-label: issue:auto-closed
-          close-issue-reason: completed
-          stale-issue-message: ""
-          close-issue-message: >
-            This issue has had no activity for one year. The latest version of
-            the code may already have fixed the problem.
-
-            If the issue still exists in the latest version, you can reopen
-            this issue at any time with updated reproduction details.
-
-          remove-issue-stale-when-updated: true
-          operations-per-run: 1000
-          debug-only: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_only || false }}

.github/workflows/close-organization-fork-prs.yml

@@ -1,61 +0,0 @@
-name: Close PRs from organization forks
-
-on:
-  pull_request_target:
-    types: [opened, reopened]
-
-permissions:
-  pull-requests: write
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
-  cancel-in-progress: false
-
-jobs:
-  close-organization-fork-pr:
-    if: >-
-      github.event.pull_request.head.repo.owner.type == 'Organization' &&
-      github.event.pull_request.head.repo.id != github.event.pull_request.base.repo.id
-    runs-on: ubuntu-latest
-    timeout-minutes: 5
-    steps:
-      - name: Explain the contribution policy and close the PR
-        uses: actions/github-script@v9
-        with:
-          script: |
-            const { data: pr } = await github.rest.pulls.get({
-              ...context.repo,
-              pull_number: context.issue.number,
-            });
-            const headRepo = pr.head.repo;
-            if (pr.state !== 'open' || !headRepo ||
-                headRepo.id === pr.base.repo.id || headRepo.owner.type !== 'Organization') {
-              return;
-            }
-
-            const marker = '<!-- organization-fork-policy -->';
-            const comments = await github.paginate(github.rest.issues.listComments, {
-              ...context.repo,
-              issue_number: pr.number,
-              per_page: 100,
-            });
-            const alreadyExplained = comments.some(comment =>
-              comment.user?.login === 'github-actions[bot]' && comment.body?.includes(marker));
-            if (!alreadyExplained) {
-              await github.rest.issues.createComment({
-                ...context.repo,
-                issue_number: pr.number,
-                body: [
-                  marker,
-                  'This repository requires contributions from forks to use a personal fork with **Allow edits from maintainers** enabled.',
-                  'GitHub does not support this option for organization-owned forks, so this PR is being closed automatically.',
-                  'Please open a new PR from a fork in your personal GitHub account and enable **Allow edits from maintainers** so maintainers can help update the branch.',
-                  'See [the GitHub documentation](https://docs.github.com/en/pull-requests/how-tos/work-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork).',
-                ].join('\n\n'),
-              });
-            }
-            await github.rest.pulls.update({
-              ...context.repo,
-              pull_number: pr.number,
-              state: 'closed',
-            });

.github/workflows/stale-prs.yml

@@ -1,55 +0,0 @@
-name: Close inactive PRs
-
-on:
-  schedule:
-    # Run daily. GitHub cron schedules use UTC.
-    - cron: "30 1 * * *"
-  workflow_dispatch:
-    inputs:
-      debug_only:
-        description: "Dry run: log intended actions without changing PRs"
-        required: false
-        default: false
-        type: boolean
-
-permissions:
-  issues: write
-  pull-requests: write
-
-concurrency:
-  group: ${{ github.workflow }}
-  cancel-in-progress: false
-
-jobs:
-  stale-prs:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Mark and close inactive PRs
-        uses: actions/stale@v10
-        with:
-          days-before-issue-stale: -1
-          days-before-issue-close: -1
-
-          days-before-pr-stale: 365
-          days-before-pr-close: 7
-
-          stale-pr-label: pr:inactive
-          close-pr-label: pr:auto-closed
-          exempt-pr-labels: pr:keep-open
-
-          stale-pr-message: >
-            This PR has been inactive for 365 days. If there is no new activity
-            within 7 days, it will be closed automatically. Comment, push new
-            commits, or remove the pr:inactive label to keep it open. Add
-            pr:keep-open to exempt it from future inactive PR cleanup.
-
-          close-pr-message: >
-            Closing this PR because it has had no activity for 7 days after
-            being marked inactive. If this is still useful or ready to move
-            forward, feel free to reopen it with fresh context or updated
-            details. Sorry for any inconvenience.
-
-          remove-pr-stale-when-updated: true
-          delete-branch: false
-          operations-per-run: 100
-          debug-only: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_only || false }}

.gitignore

@@ -1,7 +1,6 @@
 build*/
 cmake-build-*/
 test/
-tests/
 .vscode/
 .idea/
 .cache/
@@ -14,7 +13,3 @@ output*.png
 models*
 *.log
 preview.png
-.claude/
-CLAUDE.local.md
-.agents/
-.codex/

.gitmodules

@@ -1,12 +1,3 @@
 [submodule "ggml"]
     path = ggml
-	url = https://github.com/leejet/ggml.git
-[submodule "examples/server/frontend"]
-	path = examples/server/frontend
-	url = https://github.com/leejet/sdcpp-webui.git
-[submodule "thirdparty/libwebp"]
-	path = thirdparty/libwebp
-	url = https://github.com/webmproject/libwebp.git
-[submodule "thirdparty/libwebm"]
-	path = thirdparty/libwebm
-	url = https://github.com/webmproject/libwebm.git
+	url = https://github.com/ggml-org/ggml.git

AGENTS.md

@@ -1,184 +0,0 @@
-# Instructions for stable-diffusion.cpp
-
-This document is for AI coding agents working in this repository. It should
-describe agent-specific workflow, repository routing, editing boundaries, and
-project-specific pitfalls.
-
-For general contribution rules, including PR scope, commit conventions, code
-style, dependency updates, security hygiene, and AI-assisted contribution policy,
-see `CONTRIBUTING.md`.
-
----
-
-## Agent Operating Rules
-
-Before analyzing or modifying the repository:
-
-1. Read this file.
-2. Use `rg` / `rg --files` or directory listing commands to confirm the current
-   tree before relying on a path.
-3. Start from `src/` and relevant `docs/` for runtime behavior.
-4. Read the relevant code before editing.
-5. Prefer the smallest change that fits the existing architecture.
-6. Report focused verification and mention any tests not run.
-
-Agents must not:
-
-* Run `git push`, create PRs, or submit issue/PR comments on the user's behalf.
-* Create commits unless the user explicitly requests that specific commit.
-* Modify `ggml/`, `thirdparty/`, or `examples/server/frontend/` unless
-  explicitly requested and necessary.
-* Read large local model files or tokenizer vocabulary files.
-* Rewrite unrelated code for style-only reasons.
-* Add secrets, model weights, generated binaries, local absolute paths, or
-  machine-specific output.
-
-When a change is large, architectural, or likely to affect public behavior,
-pause and present a short plan before editing.
-
----
-
-## Repository Map and Editing Boundaries
-
-This is a routing map for agents, not a full architecture document. The layout
-can change, so verify paths before using them. Do not inspect excluded
-large-data directories while checking the tree.
-
-### Primary Project Code
-
-Core implementation lives under `src/`.
-
-Current source layout includes:
-
-* `src/core/` - shared tensor, ggml integration, backend, graph, RNG, and utility
-  code.
-* `src/model/` - model families and model components.
-* `src/model_io/` - model file loading, GGUF, safetensors, pickle, and related
-  serialization helpers.
-* `src/runtime/` - sampling, denoising, guidance, caching, preprocessing, and
-  runtime execution helpers.
-* `src/tokenizers/` - tokenizer implementations.
-* `src/conditioning/` - conditioning and prompt-related implementation.
-* `src/extensions/` - optional feature extensions.
-* top-level `src/*.cpp` and `src/*.h` files - public implementation entry
-  points, model loading, conversion, versioning, and shared managers.
-
-`src/tokenizers/vocab/` contains large tokenizer vocabulary data. Do not read or
-parse files in this directory; reference the path only when necessary.
-
-### Public API
-
-`include/` contains the C API exposed by the project. Currently the primary
-public header is `include/stable-diffusion.h`.
-
-Treat public headers as stable API. Avoid breaking compatibility unless the user
-explicitly requests it. If public behavior changes, update relevant examples or
-documentation.
-
-### Examples
-
-`examples/` contains programs demonstrating library usage.
-
-* `examples/cli/` - command line program for running models, testing features,
-  and debugging.
-* `examples/common/` - shared example support code.
-* `examples/server/` - server application built on top of the library.
-* `examples/server/frontend/` - git submodule containing independent frontend
-  code. Avoid modifying it unless explicitly requested.
-
-### Documentation and Tooling
-
-* `docs/` - documentation for supported models, build options, behavior, and
-  workflows.
-* `scripts/` - development, model processing, build automation, formatting, and
-  tooling scripts.
-* `cmake/` - CMake support modules.
-* `docker/` - Docker-related project files.
-* `assets/` - documentation assets; not runtime code.
-
-### External, Local, and Generated State
-
-* `ggml/` - git submodule for the ggml dependency.
-* `thirdparty/` - vendored third-party dependencies.
-* `models/` - local model storage. Ignore this directory and do not read model
-  files.
-* `test/` - local testing scripts. Use only when relevant to the task.
-* `build/`, `build_*`, and similar directories - generated build outputs.
-  Inspect them only when debugging a build result.
-
----
-
-## Agent Workflow for Code Changes
-
-1. Identify the relevant modules under `src/`.
-2. Check whether the change touches the public API in `include/`.
-3. Consult relevant `docs/` and examples before changing user-facing behavior.
-4. Follow existing local patterns before adding new abstractions.
-5. Keep edits scoped to the requested behavior.
-6. Run the narrowest useful build, test, or inspection command available.
-
-Follow `CONTRIBUTING.md` for formatting, naming, PR expectations, dependency
-update policy, and security rules. For tokenizer additions, follow its embedded-data
-allowlist and default to an external `tokenizer.json`.
-
----
-
-## Code Comments
-
-Keep comments rare and useful.
-
-Do not add comments that only describe what the code does. Add comments only
-when the code cannot fully express the logic, the logic is unusually complex, or
-there are historical reasons, invariants, constraints, compatibility concerns,
-or known pitfalls that future maintainers need to understand.
-
-Do not add task-specific comments that will be meaningless after review.
-
-Examples from the current codebase:
-
-```cpp
-// GOOD: explains a safety constraint that is not obvious from the assignment.
-// From src/model_io/pickle_io.cpp.
-// Non-tensor checkpoint metadata can use REDUCE for arbitrary
-// Python objects. Do not execute it; keep stack shape only.
-stack.push_back(make_none_value());
-
-// BAD: describes only what the next line does.
-// Set the token count to zero.
-token_count = 0;
-```
-
----
-
-## Text File Encoding
-
-When reading or editing repository text files:
-
-* Prefer UTF-8 with LF for Markdown, frontend source, JSON, and other text-first
-  project files unless the file already clearly uses a different encoding.
-* Do not assume terminal output encoding matches file encoding on Windows.
-* A file that looks garbled in PowerShell output may still be valid UTF-8.
-* When inspecting UTF-8 files in PowerShell, prefer explicit UTF-8 reads such as:
-  * `[Console]::OutputEncoding = [System.Text.Encoding]::UTF8`
-  * `Get-Content -Encoding utf8 <path>`
-* Avoid rewriting a file purely because console output looked garbled; verify
-  the actual file encoding first.
-
----
-
-## Tensor and Layout Notes
-
-Additional tensor/layout rules for this codebase:
-
-* `sd::Tensor` shape order is not PyTorch/NumPy-style. `shape()[0]` is the
-  lowest and most contiguous dimension, and higher indices are higher
-  dimensions.
-* Broadcasting for `sd::Tensor` must align dimensions from low to high dimension
-  indices. If one tensor has fewer dimensions, append implicit `1`s at the
-  higher-dimension end.
-* `ggml_n_dims` / `ggml_n_dims(tensor)` can drop trailing singleton high
-  dimensions. Do not assume a logical trailing dimension of `1` will still be
-  counted in ggml metadata.
-* Internal tensor-returning interfaces use an empty `sd::Tensor` to represent
-  null, absent, or failure states. Do not add `std::optional<sd::Tensor<...>>`
-  for internal APIs unless a distinct semantic state is truly required.

CLAUDE.md

@@ -1,11 +0,0 @@
-@AGENTS.md
-
-## Claude Code
-
-Follow `AGENTS.md` as the shared repository instructions.
-
-Do not duplicate contribution, style, PR, dependency, or security policy here;
-use `CONTRIBUTING.md` as the canonical source for those rules.
-
-Keep Claude-specific project notes in this file only when they do not apply to
-other coding agents.

CMakeLists.txt

@@ -11,69 +11,17 @@ endif()
 if (MSVC)
     add_compile_definitions(_CRT_SECURE_NO_WARNINGS)
     add_compile_definitions(_SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING)
-    # /MP is MSVC-only: icx rejects it outright once offloading is enabled.
-    add_compile_options(
-        $<$<AND:$<COMPILE_LANGUAGE:C>,$<C_COMPILER_ID:MSVC>>:/MP>
-        $<$<COMPILE_LANGUAGE:C>:/utf-8>
-        $<$<AND:$<COMPILE_LANGUAGE:CXX>,$<CXX_COMPILER_ID:MSVC>>:/MP>
-        $<$<COMPILE_LANGUAGE:CXX>:/utf-8>
-    )
 endif()
 
 set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
 set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
 
-if(APPLE)
-    function(sd_set_macos_rpaths target)
-        get_target_property(target_type ${target} TYPE)
-        if(target_type STREQUAL "EXECUTABLE")
-            set(runtime_paths "@executable_path" "@executable_path/../lib")
-        elseif(target_type STREQUAL "SHARED_LIBRARY" OR target_type STREQUAL "MODULE_LIBRARY")
-            set(runtime_paths "@loader_path" "@loader_path/../lib")
-            set_target_properties(${target} PROPERTIES
-                MACOSX_RPATH ON
-                INSTALL_NAME_DIR "@rpath"
-                BUILD_WITH_INSTALL_NAME_DIR ON
-            )
-        else()
-            return()
-        endif()
-
-        # Release artifacts zip the build output directly, so keep macOS rpaths relocatable.
-        set_target_properties(${target} PROPERTIES
-            BUILD_RPATH "${runtime_paths}"
-            INSTALL_RPATH "${runtime_paths}"
-            BUILD_WITH_INSTALL_RPATH ON
-        )
-    endfunction()
-endif()
-
 if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
     set(SD_STANDALONE ON)
 else()
     set(SD_STANDALONE OFF)
 endif()
 
-set(SD_SUBMODULE_WEBP FALSE)
-if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/thirdparty/libwebp/CMakeLists.txt")
-    set(SD_SUBMODULE_WEBP TRUE)
-endif()
-if(SD_SUBMODULE_WEBP)
-    set(SD_WEBP_DEFAULT ON)
-else()
-    set(SD_WEBP_DEFAULT ${SD_USE_SYSTEM_WEBP})
-endif()
-
-set(SD_SUBMODULE_WEBM FALSE)
-if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/thirdparty/libwebm/CMakeLists.txt")
-    set(SD_SUBMODULE_WEBM TRUE)
-endif()
-if(SD_SUBMODULE_WEBM)
-    set(SD_WEBM_DEFAULT ON)
-else()
-    set(SD_WEBM_DEFAULT ${SD_USE_SYSTEM_WEBM})
-endif()
-
 #
 # Option list
 #
@@ -81,10 +29,6 @@ endif()
 # general
 #option(SD_BUILD_TESTS                "sd: build tests"    ${SD_STANDALONE})
 option(SD_BUILD_EXAMPLES             "sd: build examples" ${SD_STANDALONE})
-option(SD_WEBP                       "sd: enable WebP image I/O support" ${SD_WEBP_DEFAULT})
-option(SD_USE_SYSTEM_WEBP            "sd: link against system libwebp" OFF)
-option(SD_WEBM                       "sd: enable WebM video output support" ${SD_WEBM_DEFAULT})
-option(SD_USE_SYSTEM_WEBM            "sd: link against system libwebm" OFF)
 option(SD_CUDA                       "sd: cuda backend" OFF)
 option(SD_HIPBLAS                    "sd: rocm backend" OFF)
 option(SD_METAL                      "sd: metal backend" OFF)
@@ -92,156 +36,60 @@ option(SD_VULKAN                     "sd: vulkan backend" OFF)
 option(SD_OPENCL                     "sd: opencl backend" OFF)
 option(SD_SYCL                       "sd: sycl backend" OFF)
 option(SD_MUSA                       "sd: musa backend" OFF)
+option(SD_FAST_SOFTMAX               "sd: x1.5 faster softmax, indeterministic (sometimes, same seed don't generate same image), cuda only" OFF)
 option(SD_BUILD_SHARED_LIBS          "sd: build shared libs" OFF)
 option(SD_BUILD_SHARED_GGML_LIB      "sd: build ggml as a separate shared lib" OFF)
 option(SD_USE_SYSTEM_GGML            "sd: use system-installed GGML library" OFF)
-option(SD_USE_UPSTREAM_GGML          "sd: build with upstream GGML instead of the patched GGML extensions" OFF)
-set(SD_GGML_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/ggml" CACHE PATH "sd: ggml source directory (also supplies private headers for system ggml)")
 #option(SD_BUILD_SERVER               "sd: build server example"                           ON)
 
-set(CMAKE_C_STANDARD 11)
-set(CMAKE_C_STANDARD_REQUIRED true)
-
-set(CMAKE_CXX_STANDARD 17)
-set(CMAKE_CXX_STANDARD_REQUIRED true)
-
 if(SD_CUDA)
     message("-- Use CUDA as backend stable-diffusion")
     set(GGML_CUDA ON)
+    add_definitions(-DSD_USE_CUDA)
 endif()
 
 if(SD_METAL)
     message("-- Use Metal as backend stable-diffusion")
     set(GGML_METAL ON)
+    add_definitions(-DSD_USE_METAL)
 endif()
 
 if (SD_VULKAN)
     message("-- Use Vulkan as backend stable-diffusion")
     set(GGML_VULKAN ON)
+    add_definitions(-DSD_USE_VULKAN)
 endif ()
 
 if (SD_OPENCL)
     message("-- Use OpenCL as backend stable-diffusion")
     set(GGML_OPENCL ON)
+    add_definitions(-DSD_USE_OPENCL)
 endif ()
 
 if (SD_HIPBLAS)
     message("-- Use HIPBLAS as backend stable-diffusion")
     set(GGML_HIP ON)
-    # ggml-hip's device-stub objects must be position-independent, or the
-    # default-PIE sd-cli link fails with `relocation R_X86_64_32 ... cannot be
-    # used when making a PIE object` on distros that default to PIE
-    # (Ubuntu 24.04, Fedora 40+, Debian 12+). The shared-library branch below
-    # already sets this; the static build (the HIP default) did not.
-    set(CMAKE_POSITION_INDEPENDENT_CODE ON)
+    add_definitions(-DSD_USE_CUDA)
+    if(SD_FAST_SOFTMAX)
+        set(GGML_CUDA_FAST_SOFTMAX ON)
+    endif()
 endif ()
 
 if(SD_MUSA)
     message("-- Use MUSA as backend stable-diffusion")
     set(GGML_MUSA ON)
-endif()
-
-if(SD_WEBP)
-    if(NOT SD_SUBMODULE_WEBP AND NOT SD_USE_SYSTEM_WEBP)
-        message(FATAL_ERROR "WebP support enabled but no source found.
-          Either initialize the submodule:\n  git submodule update --init thirdparty/libwebp\n\n"
-          "Or link against system library:\n  cmake (...) -DSD_USE_SYSTEM_WEBP=ON")
-    endif()
-    if(SD_USE_SYSTEM_WEBP)
-        find_package(WebP)
-        if(WebP_FOUND)
-            add_library(webp ALIAS WebP::webp)
-            # libwebp CMake target naming is not consistent across versions/distros.
-            # Some export WebP::libwebpmux, others export WebP::webpmux.
-            if(TARGET WebP::libwebpmux)
-                add_library(libwebpmux ALIAS WebP::libwebpmux)
-            elseif(TARGET WebP::webpmux)
-                add_library(libwebpmux ALIAS WebP::webpmux)
-            else()
-                message(FATAL_ERROR
-                    "Could not find a compatible webpmux target in system WebP package. "
-                    "Expected WebP::libwebpmux or WebP::webpmux."
-                )
-            endif()
-        else()
-            find_package(PkgConfig REQUIRED)
-            pkg_check_modules(WebP REQUIRED IMPORTED_TARGET GLOBAL libwebp)
-            pkg_check_modules(WebPMux REQUIRED IMPORTED_TARGET GLOBAL libwebpmux)
-            link_libraries(PkgConfig::WebP)
-            link_libraries(PkgConfig::WebPMux)
-            add_library(libwebpmux ALIAS PkgConfig::WebPMux)
-        endif()
+    add_definitions(-DSD_USE_CUDA)
+    if(SD_FAST_SOFTMAX)
+        set(GGML_CUDA_FAST_SOFTMAX ON)
     endif()
 endif()
 
-if(SD_WEBM)
-    if(NOT SD_WEBP)
-        message(FATAL_ERROR "SD_WEBM requires SD_WEBP because WebM output reuses libwebp VP8 encoding.")
-    endif()
-    if(NOT SD_SUBMODULE_WEBM AND NOT SD_USE_SYSTEM_WEBM)
-        message(FATAL_ERROR "WebM support enabled but no source found.
-          Either initialize the submodule:\n  git submodule update --init thirdparty/libwebm\n\n"
-          "Or link against system library:\n  cmake (...) -DSD_USE_SYSTEM_WEBM=ON")
-    endif()
-    if(SD_USE_SYSTEM_WEBM)
-        find_package(PkgConfig)
-        if(PkgConfig_FOUND)
-            pkg_check_modules(WebM REQUIRED IMPORTED_TARGET GLOBAL libwebm)
-        endif()
-        if(PkgConfig_FOUND AND WebM_FOUND)
-            link_libraries(PkgConfig::WebM)
-        else()
-            find_path(WEBM_INCLUDE_DIR
-                NAMES mkvmuxer/mkvmuxer.h mkvparser/mkvparser.h common/webmids.h
-                PATH_SUFFIXES webm
-                REQUIRED)
-            find_library(WEBM_LIBRARY
-                NAMES webm libwebm
-                REQUIRED)
-
-            add_library(webm UNKNOWN IMPORTED)
-            set_target_properties(webm PROPERTIES
-                IMPORTED_LOCATION "${WEBM_LIBRARY}"
-                INTERFACE_INCLUDE_DIRECTORIES "${WEBM_INCLUDE_DIR}")
-        endif()
-    endif()
-endif()
-
-if (SD_RPC)
-    message("-- Use RPC as backend stable-diffusion")
-    set(GGML_RPC ON)
-    add_definitions(-DSD_USE_RPC)
-endif ()
-
 set(SD_LIB stable-diffusion)
 
-file(GLOB SD_LIB_SOURCES CONFIGURE_DEPENDS
-    "src/*.h"
-    "src/*.cpp"
-    "src/*.hpp"
-    "src/conditioning/*.h"
-    "src/conditioning/*.cpp"
-    "src/conditioning/*.hpp"
-    "src/core/*.h"
-    "src/core/*.cpp"
-    "src/core/*.hpp"
-    "src/extensions/*.h"
-    "src/extensions/*.cpp"
-    "src/extensions/*.hpp"
-    "src/model/*/*.h"
-    "src/model/*/*.cpp"
-    "src/model/*/*.hpp"
-    "src/pipeline/*.h"
-    "src/pipeline/*.cpp"
-    "src/runtime/*.h"
-    "src/runtime/*.cpp"
-    "src/runtime/*.hpp"
-    "src/model_io/*.h"
-    "src/model_io/*.cpp"
-    "src/tokenizers/*.h"
-    "src/tokenizers/*.cpp"
-    "src/tokenizers/vocab/*.h"
-    "src/tokenizers/vocab/*.cpp"
+file(GLOB SD_LIB_SOURCES
+    "*.h"
+    "*.cpp"
+    "*.hpp"
 )
 
 find_program(GIT_EXE NAMES git git.exe NO_CMAKE_FIND_ROOT_PATH)
@@ -271,7 +119,7 @@ endif()
 message(STATUS "stable-diffusion.cpp commit ${SDCPP_BUILD_COMMIT}")
 
 set_property(
-  SOURCE ${CMAKE_CURRENT_SOURCE_DIR}/src/version.cpp
+  SOURCE ${CMAKE_CURRENT_SOURCE_DIR}/version.cpp
   APPEND PROPERTY COMPILE_DEFINITIONS
   SDCPP_BUILD_COMMIT=${SDCPP_BUILD_COMMIT} SDCPP_BUILD_VERSION=${SDCPP_BUILD_VERSION}
 )
@@ -294,20 +142,11 @@ else()
     add_library(${SD_LIB} STATIC ${SD_LIB_SOURCES})
 endif()
 
-if(MSVC)
-    target_compile_options(${SD_LIB} PRIVATE $<$<COMPILE_LANGUAGE:CXX>:/bigobj>)
-    # ggml backends can throw C++ exceptions through their C API.
-    target_compile_options(${SD_LIB} PRIVATE $<$<AND:$<COMPILE_LANGUAGE:CXX>,$<CXX_COMPILER_ID:MSVC>>:/EHsc->)
-endif()
-
-if(APPLE)
-    sd_set_macos_rpaths(${SD_LIB})
-endif()
-
 if(SD_SYCL)
     message("-- Use SYCL as backend stable-diffusion")
     set(GGML_SYCL ON)
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-narrowing -fsycl")
+    add_definitions(-DSD_USE_SYCL)
     # disable fast-math on host, see:
     # https://www.intel.com/content/www/us/en/docs/cpp-compiler/developer-guide-reference/2021-10/fp-model-fp.html
     if (WIN32)
@@ -323,27 +162,26 @@ set(CMAKE_POLICY_DEFAULT_CMP0077 NEW)
 
 if (NOT SD_USE_SYSTEM_GGML)
     # see https://github.com/ggerganov/ggml/pull/682
-    add_definitions(-DGGML_MAX_NAME=160)
+    add_definitions(-DGGML_MAX_NAME=128)
 endif()
 
 # deps
-include(cmake/ggml.cmake)
+# Only add ggml if it hasn't been added yet
+if (NOT TARGET ggml)
+    if (SD_USE_SYSTEM_GGML)
+        find_package(ggml REQUIRED)
+        if (NOT ggml_FOUND)
+            message(FATAL_ERROR "System-installed GGML library not found.")
+        endif()
+        add_library(ggml ALIAS ggml::ggml)
+    else()
+        add_subdirectory(ggml)
+    endif()
+endif()
 
 add_subdirectory(thirdparty)
 
-target_sources(${SD_LIB} PRIVATE $<TARGET_OBJECTS:zip>)
-target_link_libraries(${SD_LIB} PUBLIC ggml)
-find_package(Threads REQUIRED)
-target_link_libraries(${SD_LIB} PRIVATE Threads::Threads)
-target_link_libraries(${SD_LIB} PRIVATE onig sd-utf8proc)
-if (SD_CUDA)
-    find_package(CUDAToolkit REQUIRED)
-    # Keep the driver stub on downstream link lines when no driver is installed.
-    target_link_libraries(${SD_LIB} PUBLIC CUDA::cuda_driver)
-    set_property(SOURCE src/core/ggml_extend_backend.cpp APPEND PROPERTY COMPILE_DEFINITIONS SD_USE_CUDA)
-endif()
-target_include_directories(${SD_LIB} PUBLIC . src include)
-target_include_directories(${SD_LIB} PRIVATE src/core)
+target_link_libraries(${SD_LIB} PUBLIC ggml zip)
 target_include_directories(${SD_LIB} PUBLIC . thirdparty)
 target_compile_features(${SD_LIB} PUBLIC c_std_11 cxx_std_17)
 
@@ -352,58 +190,7 @@ if (SD_BUILD_EXAMPLES)
     add_subdirectory(examples)
 endif()
 
+set(SD_PUBLIC_HEADERS stable-diffusion.h)
+set_target_properties(${SD_LIB} PROPERTIES PUBLIC_HEADER "${SD_PUBLIC_HEADERS}")
 
-
-#
-# install
-#
-
-include(CMakePackageConfigHelpers)
-include(GNUInstallDirs)
-
-set(SD_INSTALL_VERSION "${SDCPP_BUILD_VERSION}")
-set(SD_INSTALL_COMMIT  "${SDCPP_BUILD_COMMIT}")
-set(SD_SHARED_LIB      ${SD_BUILD_SHARED_LIBS})
-
-set(SD_INCLUDE_INSTALL_DIR ${CMAKE_INSTALL_INCLUDEDIR} CACHE PATH "Location of header  files")
-set(SD_LIB_INSTALL_DIR     ${CMAKE_INSTALL_LIBDIR}     CACHE PATH "Location of library files")
-set(SD_BIN_INSTALL_DIR     ${CMAKE_INSTALL_BINDIR}     CACHE PATH "Location of binary  files")
-
-set(SD_PUBLIC_HEADERS
-    ${CMAKE_CURRENT_SOURCE_DIR}/include/stable-diffusion.h)
-
-set_target_properties(${SD_LIB}
-    PROPERTIES
-        PUBLIC_HEADER "${SD_PUBLIC_HEADERS}")
-
-
-install(TARGETS ${SD_LIB}
-        ARCHIVE
-        LIBRARY
-        RUNTIME
-        PUBLIC_HEADER)
-
-
-configure_package_config_file(
-        ${CMAKE_CURRENT_SOURCE_DIR}/cmake/stable-diffusion-config.cmake.in
-        ${CMAKE_CURRENT_BINARY_DIR}/stable-diffusion-config.cmake
-    INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/stable-diffusion
-    PATH_VARS SD_INCLUDE_INSTALL_DIR
-              SD_LIB_INSTALL_DIR
-              SD_BIN_INSTALL_DIR )
-
-write_basic_package_version_file(
-        ${CMAKE_CURRENT_BINARY_DIR}/stable-diffusion-version.cmake
-    VERSION ${SD_INSTALL_VERSION}
-    COMPATIBILITY SameMajorVersion)
-
-install(FILES ${CMAKE_CURRENT_BINARY_DIR}/stable-diffusion-config.cmake
-              ${CMAKE_CURRENT_BINARY_DIR}/stable-diffusion-version.cmake
-        DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/stable-diffusion)
-
-configure_file(cmake/stable-diffusion.pc.in
-        "${CMAKE_CURRENT_BINARY_DIR}/stable-diffusion.pc"
-        @ONLY)
-
-install(FILES "${CMAKE_CURRENT_BINARY_DIR}/stable-diffusion.pc"
-        DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
+install(TARGETS ${SD_LIB} LIBRARY PUBLIC_HEADER)

CONTRIBUTING.md

@@ -1,87 +0,0 @@
-# Contributing
-
-This document collects general contribution conventions for this repository.
-
-## Before You Start
-
-Before opening a PR, please search existing PRs to avoid duplicating ongoing work.
-
-For large-scale refactors or changes with broad impact, please open an issue first to discuss the approach before submitting a PR.
-
-If you want to update a third-party dependency, please open an issue first instead of submitting a direct PR. See [Dependency Updates](#dependency-updates) for details.
-
-## Pull Requests
-
-When contributing from a fork, use a fork under your personal GitHub account and enable **Allow edits from maintainers**. This lets maintainers make follow-up fixes directly on the PR branch.
-
-PRs from organization-owned forks are automatically closed when opened or reopened because GitHub does not support this maintainer-edit option for those forks. Submit the changes from a personal fork instead. See [GitHub's documentation](https://docs.github.com/en/pull-requests/how-tos/work-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork).
-
-Keep each PR focused on one clear change. Large or overly complex PRs are harder to review and may not be merged.
-
-Do not include test code or test scripts in commits or PRs. Keep them local and report verification results in the PR description.
-
-Follow Conventional Commit-style subjects seen in history: `feat:`, `fix:`, `refactor:`, `ci:`, `docs:`, `chore:`. Keep subjects imperative and scoped.
-
-PRs should include:
-
-- What changed and why (short problem/solution summary).
-- Verification evidence when applicable (commands and key outputs).
-- Linked issue/PR context when applicable.
-- Screenshots or sample outputs for UI/visual behavior changes.
-
-## Code Style
-
-Format code according to the repository style before submitting changes.
-
-Formatting follows `.clang-format` (Chromium base, 4-space indent, no tabs). Run `scripts/format-code.sh` or `scripts/format-code.ps1` before opening a PR. Keep C++ standard at C++17-compatible patterns used in this repo.
-
-Naming conventions:
-
-- Use `PascalCase` for class/struct/type names.
-- In `PascalCase` names, preserve common abbreviations in uppercase, for example `SD`, `API`, `HTTP`, `JSON`, `RGB`, `VAE`, `TAE`, `LoRA`, and `WebP`.
-- Use `snake_case` for functions, methods, variables, and file names unless an existing API requires a different style.
-- Use a trailing underscore for private data member names, for example `hidden_size_` or `tokenizer_`.
-- Use `.hpp` for model headers under `src/model/`, including new model headers. Do not rename these headers to `.h`. Use `.h` for other C and C++ header files.
-- Use macro-based header include guards instead of `#pragma once`.
-- Format header include guards as `__SD_{PATH}__`, where `{PATH}` is the header path in uppercase snake case without the file extension. For example, `src/sample.h` should use `__SD_SAMPLE_H__`.
-- Do not introduce anonymous namespaces in new or modified code; prefer `static` file-local functions/variables or an explicit named namespace when scoping is needed.
-- In `class`/`struct` definitions, place data members before member functions unless an existing type already clearly follows a different pattern.
-
-Some older code in the project may not fully follow the current conventions. Please do not submit PRs that only rewrite existing code to match style rules.
-
-When adding or modifying model implementations, follow the model config and weight detection conventions in [docs/model_config.md](docs/model_config.md).
-
-## Tokenizer Data
-
-New model integrations must use an external `tokenizer.json` by default. Do not
-embed new vocabularies or merge tables solely for less widely used models;
-these tables increase the binary size for every user.
-
-The embedded-data allowlist is CLIP, T5/UMT5, Qwen 2/3, Mistral, and Gemma 3/4.
-Models may reuse an existing embedded tokenizer when its vocabulary and behavior
-match their text encoder. Gemma 2 and GPT-OSS require external JSON files.
-
-Adding to this allowlist requires maintainer approval, supported by the model's
-usage, reuse across models, and measured binary-size cost. Document the matching
-JSON and CLI option for models that require an external tokenizer, and fail
-initialization clearly when it is missing.
-
-## AI-Assisted Contributions
-
-AI tools may be used to assist development, but contributors are responsible for the quality and correctness of the submitted code.
-
-If any part of a contribution was generated with AI assistance, the contributor must perform a thorough human review before submitting the PR and understand every changed line.
-
-Do not list AI tools as co-authors. The human contributor is the sole responsible author of the submitted code.
-
-Please do not submit AI-generated code that you do not understand, and do not include meaningless experiments, temporary test code, or unrelated generated output in a PR.
-
-## Dependency Updates
-
-Do not submit PRs that update `ggml`. `ggml` updates are performed only after local validation by the maintainer.
-
-Other third-party dependencies are not updated unless necessary. If you want to update a dependency, please open an issue first instead of submitting a direct PR.
-
-## Security & Configuration
-
-Do not commit model weights, secrets, or local absolute paths. Keep large binaries out of git unless intentionally tracked release assets.

Dockerfile

@@ -0,0 +1,23 @@
+ARG UBUNTU_VERSION=24.04
+
+FROM ubuntu:$UBUNTU_VERSION AS build
+
+RUN apt-get update && apt-get install -y --no-install-recommends build-essential git cmake
+
+WORKDIR /sd.cpp
+
+COPY . .
+
+RUN cmake . -B ./build
+RUN cmake --build ./build --config Release --parallel
+
+FROM ubuntu:$UBUNTU_VERSION AS runtime
+
+RUN apt-get update && \
+    apt-get install --yes --no-install-recommends libgomp1 && \
+    apt-get clean
+
+COPY --from=build /sd.cpp/build/bin/sd-cli /sd-cli
+COPY --from=build /sd.cpp/build/bin/sd-server /sd-server
+
+ENTRYPOINT [ "/sd-cli" ]

Dockerfile.musa

@@ -0,0 +1,24 @@
+ARG MUSA_VERSION=rc4.2.0
+ARG UBUNTU_VERSION=22.04
+
+FROM mthreads/musa:${MUSA_VERSION}-devel-ubuntu${UBUNTU_VERSION}-amd64 as build
+
+RUN apt-get update && apt-get install -y ccache cmake git
+
+WORKDIR /sd.cpp
+
+COPY . .
+
+RUN mkdir build && cd build && \
+    cmake .. -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
+        -DCMAKE_C_FLAGS="${CMAKE_C_FLAGS} -fopenmp -I/usr/lib/llvm-14/lib/clang/14.0.0/include -L/usr/lib/llvm-14/lib" \
+        -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -fopenmp -I/usr/lib/llvm-14/lib/clang/14.0.0/include -L/usr/lib/llvm-14/lib" \
+        -DSD_MUSA=ON -DCMAKE_BUILD_TYPE=Release && \
+    cmake --build . --config Release
+
+FROM mthreads/musa:${MUSA_VERSION}-runtime-ubuntu${UBUNTU_VERSION}-amd64 as runtime
+
+COPY --from=build /sd.cpp/build/bin/sd-cli /sd-cli
+COPY --from=build /sd.cpp/build/bin/sd-server /sd-server
+
+ENTRYPOINT [ "/sd-cli" ]

Dockerfile.sycl

@@ -0,0 +1,20 @@
+ARG SYCL_VERSION=2025.1.0-0
+
+FROM intel/oneapi-basekit:${SYCL_VERSION}-devel-ubuntu24.04 AS build
+
+RUN apt-get update && apt-get install -y cmake
+
+WORKDIR /sd.cpp
+
+COPY . .
+
+RUN mkdir build && cd build && \
+    cmake .. -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DSD_SYCL=ON -DCMAKE_BUILD_TYPE=Release && \
+    cmake --build . --config Release -j$(nproc)
+
+FROM intel/oneapi-basekit:${SYCL_VERSION}-devel-ubuntu24.04 AS runtime
+
+COPY --from=build /sd.cpp/build/bin/sd-cli /sd-cli
+COPY --from=build /sd.cpp/build/bin/sd-server /sd-server
+
+ENTRYPOINT [ "/sd-cli" ]

Dockerfile.vulkan

@@ -0,0 +1,23 @@
+ARG UBUNTU_VERSION=24.04
+
+FROM ubuntu:$UBUNTU_VERSION AS build
+
+RUN apt-get update && apt-get install -y --no-install-recommends build-essential git cmake libvulkan-dev glslc
+
+WORKDIR /sd.cpp
+
+COPY . .
+
+RUN cmake . -B ./build -DSD_VULKAN=ON
+RUN cmake --build ./build --config Release --parallel
+
+FROM ubuntu:$UBUNTU_VERSION AS runtime
+
+RUN apt-get update && \
+    apt-get install --yes --no-install-recommends libgomp1 libvulkan1 mesa-vulkan-drivers && \
+    apt-get clean
+
+COPY --from=build /sd.cpp/build/bin/sd-cli /sd-cli
+COPY --from=build /sd.cpp/build/bin/sd-server /sd-server
+
+ENTRYPOINT [ "/sd-cli" ]

README.md

@@ -15,22 +15,23 @@ API and command-line option may change frequently.***
 
 ## 🔥Important News
 
-* **2026/09/20** 🚀 stable-diffusion.cpp adds **Day-0 support for Qwen-Image-2.1**
-* **2026/08/20** 🚀 stable-diffusion.cpp now supports **LTX-2.5**
-* **2026/08/04** 🚀 stable-diffusion.cpp adds **Day-1 support for MiniMax-H3**
-* **2026/06/25** 🚀 stable-diffusion.cpp now supports **Krea2**
-* **2026/06/04** 🚀 stable-diffusion.cpp now supports **Ideogram4**
-* **2026/05/31** 🚀 stable-diffusion.cpp now supports **PiD**
-* **2026/05/27** 🚀 stable-diffusion.cpp now supports **Lens**
-* **2026/05/17** 🚀 stable-diffusion.cpp now supports **LTX-2.3**
-* **2026/04/11** 🚀 stable-diffusion.cpp now uses a brand-new embedded web UI.  
-* **2026/01/18** 🚀 stable-diffusion.cpp now supports **FLUX.2-klein**  
 * **2025/12/01** 🚀 stable-diffusion.cpp now supports **Z-Image**  
+  👉 Details: [PR #1020](https://github.com/leejet/stable-diffusion.cpp/pull/1020)
+
 * **2025/11/30** 🚀 stable-diffusion.cpp now supports **FLUX.2-dev**  
+  👉 Details: [PR #1016](https://github.com/leejet/stable-diffusion.cpp/pull/1016)
+
 * **2025/10/13** 🚀 stable-diffusion.cpp now supports **Qwen-Image-Edit / Qwen-Image-Edit 2509**  
+  👉 Details: [PR #877](https://github.com/leejet/stable-diffusion.cpp/pull/877)
+
 * **2025/10/12** 🚀 stable-diffusion.cpp now supports **Qwen-Image**  
+  👉 Details: [PR #851](https://github.com/leejet/stable-diffusion.cpp/pull/851)
+
 * **2025/09/14** 🚀 stable-diffusion.cpp now supports **Wan2.1 Vace**  
+  👉 Details: [PR #819](https://github.com/leejet/stable-diffusion.cpp/pull/819)
+
 * **2025/09/06** 🚀 stable-diffusion.cpp now supports **Wan2.1 / Wan2.2**  
+  👉 Details: [PR #778](https://github.com/leejet/stable-diffusion.cpp/pull/778)
 
 ## Features
 
@@ -38,56 +39,28 @@ API and command-line option may change frequently.***
 - Super lightweight and without external dependencies
 - Supported models
   - Image Models
-    - [SD1.x, SD2.x, SD-Turbo](./docs/sd.md)
-    - [SDXL, SDXL-Turbo](./docs/sd.md)
+    - SD1.x, SD2.x, [SD-Turbo](https://huggingface.co/stabilityai/sd-turbo)
+    - SDXL, [SDXL-Turbo](https://huggingface.co/stabilityai/sdxl-turbo)
     - [Some SD1.x and SDXL distilled models](./docs/distilled_sd.md)
     - [SD3/SD3.5](./docs/sd3.md)
-    - [FLUX.1-dev/FLUX.1-schnell](./docs/flux.md)
-    - [FLUX.2-dev/FLUX.2-klein](./docs/flux2.md)
-    - [Lens](./docs/lens.md)
+    - [FlUX.1-dev/FlUX.1-schnell](./docs/flux.md)
+    - [FLUX.2-dev](./docs/flux2.md)
     - [Chroma](./docs/chroma.md)
     - [Chroma1-Radiance](./docs/chroma_radiance.md)
     - [Qwen Image](./docs/qwen_image.md)
-    - [Qwen Image 2.1](./docs/qwen_image_2.1.md)
-    - [PiD](./docs/pid.md)
-    - [LongCat Image](./docs/longcat_image.md)
     - [Z-Image](./docs/z_image.md)
-    - [Z-Image L2P](./docs/z_image_l2p.md)
-    - [MiniT2I](./docs/minit2i.md)
-    - [SenseNova U1.5](./docs/sensenova_u1.md)
     - [Ovis-Image](./docs/ovis_image.md)
-    - [Anima](./docs/anima.md)
-    - [ERNIE-Image](./docs/ernie_image.md)
-    - [Boogu Image](./docs/boogu_image.md)
-    - [Krea2](./docs/krea2.md)
-    - [Mage-Flow](./docs/mage_flow.md)
-    - [SeFi-Image](./docs/sefi_image.md)
-    - [HiDream-O1-Image](./docs/hidream_o1_image.md)
-    - [Ideogram4](./docs/ideogram4.md)
-    - [LLaDA-Image](./docs/llada_image.md)
-    - [Ming-Image Design](./docs/ming_image.md)
-    - [PixArt](./docs/pixart.md)
-  - [Image Edit Models](./docs/edit.md)
+  - Image Edit Models
     - [FLUX.1-Kontext-dev](./docs/kontext.md)
     - [Qwen Image Edit series](./docs/qwen_image_edit.md)
-    - [LongCat Image Edit](./docs/longcat_image.md)
-    - [Boogu Image Edit](./docs/boogu_image.md)
-    - [Mage-Flow-Edit](./docs/mage_flow.md#image-editing)
-    - [LLaDA-Image Edit](./docs/llada_image.md#image-editing)
   - Video Models
     - [Wan2.1/Wan2.2](./docs/wan.md)
-    - [MiniMax-H3](./docs/minimax_h3.md)
-    - [LTX-2.3/LTX-2.5](./docs/ltx2.md)
-    - [HunyuanVideo 1.5](./docs/hunyuan_video.md)
-    - [LingBot-Video](./docs/lingbot_video.md)
-  - [PhotoMaker](./docs/photo_maker.md) support.
-  - [IP-Adapter](./docs/ip_adapter.md) support (SD 1.5 and SDXL, including Plus)
+  - [PhotoMaker](https://github.com/TencentARC/PhotoMaker) support.
   - Control Net support with SD 1.5
-  - [ADetailer](./docs/adetailer.md)
   - LoRA support, same as [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#lora)
   - Latent Consistency Models support (LCM/LCM-LoRA)
-  - Faster and memory efficient latent decoding with [TAESD](./docs/taesd.md)
-  - Upscale images generated with [ESRGAN](./docs/esrgan.md)
+  - Faster and memory efficient latent decoding with [TAESD](https://github.com/madebyollin/taesd)
+  - Upscale images generated with [ESRGAN](https://github.com/xinntao/Real-ESRGAN)
 - Supported backends
   - CPU (AVX, AVX2 and AVX512 support for x86 architectures)
   - CUDA
@@ -96,10 +69,9 @@ API and command-line option may change frequently.***
   - OpenCL
   - SYCL
 - Supported weight formats
-  - Pytorch checkpoint (`.ckpt` or `.pth` or `.pt`)
+  - Pytorch checkpoint (`.ckpt` or `.pth`)
   - Safetensors (`.safetensors`)
   - GGUF (`.gguf`)
-- Convert mode supports converting model weights to `.gguf` or `.safetensors`
 - Supported platforms
     - Linux
     - Mac OS
@@ -117,7 +89,6 @@ API and command-line option may change frequently.***
     - `DPM++ 2M`
     - [`DPM++ 2M v2`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/8457)
     - `DPM++ 2S a`
-    - `ER-SDE`
     - [`LCM`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/13952)
 - Cross-platform reproducibility
     - `--rng cuda`, default, consistent with the `stable-diffusion-webui GPU RNG`
@@ -137,7 +108,7 @@ API and command-line option may change frequently.***
     - Stable Diffusion v1.5 from https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5 
 
     ```sh
-    curl -L -O https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors
+    curl -L -O https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors
     ```
 
 ### Generate an image with just one command
@@ -151,19 +122,29 @@ API and command-line option may change frequently.***
 ## Performance
 
 If you want to improve performance or reduce VRAM/RAM usage, please refer to [performance guide](./docs/performance.md).
-For runtime and parameter backend placement, see the [backend selection guide](./docs/backend.md).
 
 ## More Guides
 
-- [Troubleshooting](./docs/troubleshooting.md)
-- [Backend selection](./docs/backend.md)
-- [RPC](./docs/rpc.md)
+- [SD1.x/SD2.x/SDXL](./docs/sd.md)
+- [SD3/SD3.5](./docs/sd3.md)
+- [FlUX.1-dev/FlUX.1-schnell](./docs/flux.md)
+- [FLUX.2-dev](./docs/flux2.md)
+- [FLUX.1-Kontext-dev](./docs/kontext.md)
+- [Chroma](./docs/chroma.md)
+- [🔥Qwen Image](./docs/qwen_image.md)
+- [🔥Qwen Image Edit series](./docs/qwen_image_edit.md)
+- [🔥Wan2.1/Wan2.2](./docs/wan.md)
+- [🔥Z-Image](./docs/z_image.md)
+- [Ovis-Image](./docs/ovis_image.md)
 - [LoRA](./docs/lora.md)
 - [LCM/LCM-LoRA](./docs/lcm.md)
+- [Using PhotoMaker to personalize image generation](./docs/photo_maker.md)
+- [Using ESRGAN to upscale results](./docs/esrgan.md)
+- [Using TAESD to faster decoding](./docs/taesd.md)
 - [Docker](./docs/docker.md)
 - [Quantization and GGUF](./docs/quantization_and_gguf.md)
-- [INT8 convrot safetensors](./docs/int8_convrot.md)
 - [Inference acceleration via caching](./docs/caching.md)
+- [Troubleshooting](./docs/troubleshooting.md)
 
 ## Bindings
 
@@ -171,7 +152,6 @@ These projects wrap `stable-diffusion.cpp` for easier use in other languages/fra
 
 * Golang (non-cgo): [seasonjs/stable-diffusion](https://github.com/seasonjs/stable-diffusion)
 * Golang (cgo): [Binozo/GoStableDiffusion](https://github.com/Binozo/GoStableDiffusion)
-* Golang (non-cgo): [l8bloom/gosd](https://github.com/l8bloom/gosd)
 * C#: [DarthAffe/StableDiffusion.NET](https://github.com/DarthAffe/StableDiffusion.NET)
 * Python: [william-murray1204/stable-diffusion-cpp-python](https://github.com/william-murray1204/stable-diffusion-cpp-python)
 * Rust: [newfla/diffusion-rs](https://github.com/newfla/diffusion-rs)
@@ -181,7 +161,6 @@ These projects wrap `stable-diffusion.cpp` for easier use in other languages/fra
 
 These projects use `stable-diffusion.cpp` as a backend for their image generation.
 
-- [GIMP Plugins](https://github.com/themanyone/gimp-plugins)
 - [Jellybox](https://jellybox.com)
 - [Stable Diffusion GUI](https://github.com/fszontagh/sd.cpp.gui.wx)
 - [Stable Diffusion CLI-GUI](https://github.com/piallai/stable-diffusion.cpp)
@@ -196,3 +175,23 @@ These projects use `stable-diffusion.cpp` as a backend for their image generatio
 Thank you to all the people who have already contributed to stable-diffusion.cpp!
 
 [![Contributors](https://contrib.rocks/image?repo=leejet/stable-diffusion.cpp)](https://github.com/leejet/stable-diffusion.cpp/graphs/contributors)
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=leejet/stable-diffusion.cpp&type=Date)](https://star-history.com/#leejet/stable-diffusion.cpp&Date)
+
+## References
+
+- [ggml](https://github.com/ggml-org/ggml)
+- [diffusers](https://github.com/huggingface/diffusers)
+- [stable-diffusion](https://github.com/CompVis/stable-diffusion)
+- [sd3-ref](https://github.com/Stability-AI/sd3-ref)
+- [stable-diffusion-stability-ai](https://github.com/Stability-AI/stablediffusion)
+- [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
+- [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
+- [k-diffusion](https://github.com/crowsonkb/k-diffusion)
+- [latent-consistency-model](https://github.com/luosiallen/latent-consistency-model)
+- [generative-models](https://github.com/Stability-AI/generative-models/)
+- [PhotoMaker](https://github.com/TencentARC/PhotoMaker)
+- [Wan2.1](https://github.com/Wan-Video/Wan2.1)
+- [Wan2.2](https://github.com/Wan-Video/Wan2.2)

cache_dit.hpp

@@ -1,5 +1,5 @@
-#ifndef __SD_RUNTIME_CACHE_DIT_HPP__
-#define __SD_RUNTIME_CACHE_DIT_HPP__
+#ifndef __CACHE_DIT_HPP__
+#define __CACHE_DIT_HPP__
 
 #include <algorithm>
 #include <cmath>
@@ -8,9 +8,7 @@
 #include <unordered_map>
 #include <vector>
 
-#include "core/tensor.hpp"
-#include "core/util.h"
-#include "runtime/condition_cache_utils.hpp"
+#include "ggml_extend.hpp"
 
 struct DBCacheConfig {
     bool enabled                        = false;
@@ -605,6 +603,87 @@ inline std::vector<int> generate_scm_mask(
     return mask;
 }
 
+inline std::vector<int> get_scm_preset(const std::string& preset, int total_steps) {
+    struct Preset {
+        std::vector<int> compute_bins;
+        std::vector<int> cache_bins;
+    };
+
+    Preset slow   = {{8, 3, 3, 2, 1, 1}, {1, 2, 2, 2, 3}};
+    Preset medium = {{6, 2, 2, 2, 2, 1}, {1, 3, 3, 3, 3}};
+    Preset fast   = {{6, 1, 1, 1, 1, 1}, {1, 3, 4, 5, 4}};
+    Preset ultra  = {{4, 1, 1, 1, 1}, {2, 5, 6, 7}};
+
+    Preset* p = nullptr;
+    if (preset == "slow" || preset == "s" || preset == "S")
+        p = &slow;
+    else if (preset == "medium" || preset == "m" || preset == "M")
+        p = &medium;
+    else if (preset == "fast" || preset == "f" || preset == "F")
+        p = &fast;
+    else if (preset == "ultra" || preset == "u" || preset == "U")
+        p = &ultra;
+    else
+        return {};
+
+    if (total_steps != 28 && total_steps > 0) {
+        float scale = static_cast<float>(total_steps) / 28.0f;
+        std::vector<int> scaled_compute, scaled_cache;
+
+        for (int v : p->compute_bins) {
+            scaled_compute.push_back(std::max(1, static_cast<int>(v * scale + 0.5f)));
+        }
+        for (int v : p->cache_bins) {
+            scaled_cache.push_back(std::max(1, static_cast<int>(v * scale + 0.5f)));
+        }
+
+        return generate_scm_mask(scaled_compute, scaled_cache, total_steps);
+    }
+
+    return generate_scm_mask(p->compute_bins, p->cache_bins, total_steps);
+}
+
+inline float get_preset_threshold(const std::string& preset) {
+    if (preset == "slow" || preset == "s" || preset == "S")
+        return 0.20f;
+    if (preset == "medium" || preset == "m" || preset == "M")
+        return 0.25f;
+    if (preset == "fast" || preset == "f" || preset == "F")
+        return 0.30f;
+    if (preset == "ultra" || preset == "u" || preset == "U")
+        return 0.34f;
+    return 0.08f;
+}
+
+inline int get_preset_warmup(const std::string& preset) {
+    if (preset == "slow" || preset == "s" || preset == "S")
+        return 8;
+    if (preset == "medium" || preset == "m" || preset == "M")
+        return 6;
+    if (preset == "fast" || preset == "f" || preset == "F")
+        return 6;
+    if (preset == "ultra" || preset == "u" || preset == "U")
+        return 4;
+    return 8;
+}
+
+inline int get_preset_Fn(const std::string& preset) {
+    if (preset == "slow" || preset == "s" || preset == "S")
+        return 8;
+    if (preset == "medium" || preset == "m" || preset == "M")
+        return 8;
+    if (preset == "fast" || preset == "f" || preset == "F")
+        return 6;
+    if (preset == "ultra" || preset == "u" || preset == "U")
+        return 4;
+    return 8;
+}
+
+inline int get_preset_Bn(const std::string& preset) {
+    (void)preset;
+    return 0;
+}
+
 inline void parse_dbcache_options(const std::string& opts, DBCacheConfig& cfg) {
     if (opts.empty())
         return;
@@ -773,37 +852,35 @@ struct CacheDitConditionState {
         return it != cache_diffs.end() && !it->second.diff.empty();
     }
 
-    void update_cache(const void* cond, const sd::Tensor<float>& input, const sd::Tensor<float>& output) {
+    void update_cache(const void* cond, const float* input, const float* output, size_t size) {
         CacheEntry& entry = cache_diffs[cond];
-        if (!sd::store_condition_cache_diff(&entry.diff, input, output)) {
-            entry.prev_input.clear();
-            entry.prev_output.clear();
-            entry.has_prev = false;
-            return;
+        entry.diff.resize(size);
+        for (size_t i = 0; i < size; i++) {
+            entry.diff[i] = output[i] - input[i];
         }
 
-        size_t size              = static_cast<size_t>(output.numel());
-        const float* input_data  = input.data();
-        const float* output_data = output.data();
         entry.prev_input.resize(size);
         entry.prev_output.resize(size);
         for (size_t i = 0; i < size; i++) {
-            entry.prev_input[i]  = input_data[i];
-            entry.prev_output[i] = output_data[i];
+            entry.prev_input[i]  = input[i];
+            entry.prev_output[i] = output[i];
         }
         entry.has_prev = true;
     }
 
-    void apply_cache(const void* cond,
-                     const sd::Tensor<float>& input,
-                     sd::Tensor<float>* output) {
+    void apply_cache(const void* cond, const float* input, float* output, size_t size) {
         auto it = cache_diffs.find(cond);
         if (it == cache_diffs.end() || it->second.diff.empty())
             return;
-        sd::apply_condition_cache_diff(it->second.diff, input, output);
+        if (it->second.diff.size() != size)
+            return;
+
+        for (size_t i = 0; i < size; i++) {
+            output[i] = input[i] + it->second.diff[i];
+        }
     }
 
-    bool before_condition(const void* cond, const sd::Tensor<float>& input, sd::Tensor<float>* output, float sigma, int step_index) {
+    bool before_condition(const void* cond, struct ggml_tensor* input, struct ggml_tensor* output, float sigma, int step_index) {
         if (!enabled() || step_index < 0)
             return false;
 
@@ -823,7 +900,8 @@ struct CacheDitConditionState {
 
         if (skip_current_step) {
             if (has_cache(cond)) {
-                apply_cache(cond, input, output);
+                apply_cache(cond, (float*)input->data, (float*)output->data,
+                            static_cast<size_t>(ggml_nelements(output)));
                 return true;
             }
             return false;
@@ -836,13 +914,13 @@ struct CacheDitConditionState {
         if (it == cache_diffs.end() || !it->second.has_prev)
             return false;
 
-        size_t ne = static_cast<size_t>(input.numel());
+        size_t ne = static_cast<size_t>(ggml_nelements(input));
         if (it->second.prev_input.size() != ne)
             return false;
 
-        const float* input_data = input.data();
-        float diff              = CacheDitState::calculate_residual_diff(
-                         it->second.prev_input.data(), input_data, ne);
+        float* input_data = (float*)input->data;
+        float diff        = CacheDitState::calculate_residual_diff(
+                   it->second.prev_input.data(), input_data, ne);
 
         float effective_threshold = config.residual_diff_threshold;
         if (config.Fn_compute_blocks > 0) {
@@ -862,7 +940,7 @@ struct CacheDitConditionState {
             cached_steps.push_back(current_step_index);
             continuous_cached_steps++;
             accumulated_residual_diff += diff;
-            apply_cache(cond, input, output);
+            apply_cache(cond, input_data, (float*)output->data, ne);
             return true;
         }
 
@@ -870,14 +948,15 @@ struct CacheDitConditionState {
         return false;
     }
 
-    void after_condition(const void* cond, const sd::Tensor<float>& input, const sd::Tensor<float>& output) {
+    void after_condition(const void* cond, struct ggml_tensor* input, struct ggml_tensor* output) {
         if (!step_is_active())
             return;
 
-        update_cache(cond, input, output);
+        size_t ne = static_cast<size_t>(ggml_nelements(output));
+        update_cache(cond, (float*)input->data, (float*)output->data, ne);
 
         if (cond == anchor_condition && taylor_config.enabled) {
-            taylor_state.update_derivatives(output.data(), static_cast<size_t>(output.numel()), current_step_index);
+            taylor_state.update_derivatives((float*)output->data, ne, current_step_index);
         }
     }
 
@@ -893,4 +972,4 @@ struct CacheDitConditionState {
     }
 };
 
-#endif  // __SD_RUNTIME_CACHE_DIT_HPP__
+#endif

cmake/ggml.cmake

@@ -1,27 +0,0 @@
-if(NOT TARGET ggml AND NOT TARGET ggml::ggml)
-    if(SD_USE_SYSTEM_GGML)
-        find_package(ggml REQUIRED)
-    else()
-        add_subdirectory("${SD_GGML_SOURCE_DIR}" "${CMAKE_CURRENT_BINARY_DIR}/ggml")
-    endif()
-endif()
-if(NOT TARGET ggml)
-    add_library(ggml ALIAS ggml::ggml)
-endif()
-
-get_target_property(sd_ggml_imported ggml IMPORTED)
-if(sd_ggml_imported)
-    set(sd_ggml_private_include "${SD_GGML_SOURCE_DIR}/src")
-else()
-    get_target_property(sd_ggml_private_include ggml SOURCE_DIR)
-endif()
-if(NOT EXISTS "${sd_ggml_private_include}/ggml-impl.h")
-    message(FATAL_ERROR "Set SD_GGML_SOURCE_DIR to the source tree matching the selected ggml library (ggml-impl.h is required).")
-endif()
-target_include_directories(${SD_LIB} PRIVATE "${sd_ggml_private_include}")
-set_property(TARGET ${SD_LIB} PROPERTY SD_GGML_PRIVATE_INCLUDE_DIR "${sd_ggml_private_include}")
-
-if(SD_USE_UPSTREAM_GGML)
-    target_compile_definitions(${SD_LIB} PUBLIC SD_USE_UPSTREAM_GGML)
-    message(WARNING "Using upstream GGML: INT8 tensorwise/convrot is disabled and FP8 weights are converted to F16 at load time. Some operators may be unsupported and performance may be lower than with patched GGML.")
-endif()

cmake/stable-diffusion-config.cmake.in

@@ -1,45 +0,0 @@
-set(SD_VERSION      "@SD_INSTALL_VERSION@")
-set(SD_BUILD_COMMIT "@SD_INSTALL_COMMIT@")
-set(SD_SHARED_LIB    @SD_SHARED_LIB@)
-
-@PACKAGE_INIT@
-
-set_and_check(SD_INCLUDE_DIR "@PACKAGE_SD_INCLUDE_INSTALL_DIR@")
-set_and_check(SD_LIB_DIR     "@PACKAGE_SD_LIB_INSTALL_DIR@")
-set(SD_BIN_DIR "@PACKAGE_SD_BIN_INSTALL_DIR@")
-
-include(CMakeFindDependencyMacro)
-find_dependency(ggml REQUIRED HINTS "${SD_LIB_DIR}/cmake")
-find_dependency(Threads REQUIRED)
-if(@SD_CUDA@)
-    find_dependency(CUDAToolkit REQUIRED)
-endif()
-
-if(NOT TARGET stable-diffusion)
-    find_library(stable-diffusion_LIBRARY stable-diffusion
-        REQUIRED
-        HINTS "${SD_LIB_DIR}"
-        NO_CMAKE_FIND_ROOT_PATH
-    )
-
-    add_library(stable-diffusion UNKNOWN IMPORTED)
-    set_target_properties(stable-diffusion
-        PROPERTIES
-            INTERFACE_INCLUDE_DIRECTORIES "${SD_INCLUDE_DIR}"
-            INTERFACE_LINK_LIBRARIES "ggml::ggml;Threads::Threads"
-            IMPORTED_LINK_INTERFACE_LANGUAGES "CXX"
-            IMPORTED_LOCATION "${stable-diffusion_LIBRARY}"
-            INTERFACE_COMPILE_FEATURES "c_std_11;cxx_std_17"
-            POSITION_INDEPENDENT_CODE ON)
-
-    if(@SD_CUDA@)
-        set_property(TARGET stable-diffusion APPEND PROPERTY INTERFACE_LINK_LIBRARIES CUDA::cuda_driver)
-    endif()
-
-    if(SD_SHARED_LIB)
-        target_compile_definitions(stable-diffusion
-            INTERFACE SD_BUILD_SHARED_LIB)
-    endif()
-endif()
-
-check_required_components(stable-diffusion)

cmake/stable-diffusion.pc.in

@@ -1,11 +0,0 @@
-prefix=@CMAKE_INSTALL_PREFIX@
-exec_prefix=${prefix}
-libdir=@CMAKE_INSTALL_FULL_LIBDIR@
-includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
-
-Name: stable-diffusion
-Description: Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++
-Version: @SDCPP_BUILD_VERSION@
-Libs: -L${libdir} -lstable-diffusion
-Libs.private: -lggml -lggml-base @CMAKE_THREAD_LIBS_INIT@
-Cflags: -I${includedir}

common.hpp

@@ -1,11 +1,7 @@
-#ifndef __SD_MODEL_COMMON_BLOCK_HPP__
-#define __SD_MODEL_COMMON_BLOCK_HPP__
+#ifndef __COMMON_HPP__
+#define __COMMON_HPP__
 
-#include "core/ggml_extend.h"
-#include "core/ggml_runner.h"
-#include "core/util.h"
-#include "ggml-backend.h"
-#include "model/common/ggml_block.hpp"
+#include "ggml_extend.hpp"
 
 class DownSampleBlock : public GGMLBlock {
 protected:
@@ -27,7 +23,7 @@ public:
         }
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx, struct ggml_tensor* x) {
         // x: [N, channels, h, w]
         if (vae_downsample) {
             auto conv = std::dynamic_pointer_cast<Conv2d>(blocks["conv"]);
@@ -56,7 +52,7 @@ public:
         blocks["conv"] = std::shared_ptr<GGMLBlock>(new Conv2d(channels, out_channels, {3, 3}, {1, 1}, {1, 1}));
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx, struct ggml_tensor* x) {
         // x: [N, channels, h, w]
         auto conv = std::dynamic_pointer_cast<Conv2d>(blocks["conv"]);
 
@@ -125,7 +121,7 @@ public:
         }
     }
 
-    virtual ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x, ggml_tensor* emb = nullptr) {
+    virtual struct ggml_tensor* forward(GGMLRunnerContext* ctx, struct ggml_tensor* x, struct ggml_tensor* emb = nullptr) {
         // For dims==3, we reduce dimension from 5d to 4d by merging h and w, in order not to change ggml
         // [N, c, t, h, w] => [N, c, t, h * w]
         // x: [N, channels, h, w] if dims == 2 else [N, channels, t, h, w]
@@ -192,7 +188,7 @@ public:
         blocks["proj"] = std::shared_ptr<GGMLBlock>(new Linear(dim_in, dim_out * 2));
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) override {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx, struct ggml_tensor* x) override {
         // x: [ne3, ne2, ne1, dim_in]
         // return: [ne3, ne2, ne1, dim_out]
         auto proj = std::dynamic_pointer_cast<Linear>(blocks["proj"]);
@@ -204,7 +200,7 @@ public:
 
         gate = ggml_cont(ctx->ggml_ctx, gate);
 
-        gate = ggml_ext_gelu(ctx->ggml_ctx, gate, true);
+        gate = ggml_gelu_inplace(ctx->ggml_ctx, gate);
 
         x = ggml_mul(ctx->ggml_ctx, x, gate);  // [ne3, ne2, ne1, dim_out]
 
@@ -218,44 +214,13 @@ public:
         blocks["proj"] = std::shared_ptr<GGMLBlock>(new Linear(dim_in, dim_out, bias));
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) override {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx, struct ggml_tensor* x) override {
         // x: [ne3, ne2, ne1, dim_in]
         // return: [ne3, ne2, ne1, dim_out]
         auto proj = std::dynamic_pointer_cast<Linear>(blocks["proj"]);
 
         x = proj->forward(ctx, x);
-        x = ggml_ext_gelu(ctx->ggml_ctx, x, true);
-        return x;
-    }
-};
-
-struct Mlp : public GGMLBlock {
-public:
-    Mlp(int64_t in_features,
-        int64_t hidden_features = -1,
-        int64_t out_features    = -1,
-        bool bias               = true) {
-        // act_layer is always lambda: nn.GELU(approximate="tanh")
-        // norm_layer is always None
-        // use_conv is always False
-        if (hidden_features == -1) {
-            hidden_features = in_features;
-        }
-        if (out_features == -1) {
-            out_features = in_features;
-        }
-        blocks["fc1"] = std::shared_ptr<GGMLBlock>(new Linear(in_features, hidden_features, bias));
-        blocks["fc2"] = std::shared_ptr<GGMLBlock>(new Linear(hidden_features, out_features, bias));
-    }
-
-    ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) {
-        // x: [N, n_token, in_features]
-        auto fc1 = std::dynamic_pointer_cast<Linear>(blocks["fc1"]);
-        auto fc2 = std::dynamic_pointer_cast<Linear>(blocks["fc2"]);
-
-        x = fc1->forward(ctx, x);
-        x = ggml_ext_gelu(ctx->ggml_ctx, x, true);
-        x = fc2->forward(ctx, x);
+        x = ggml_gelu_inplace(ctx->ggml_ctx, x);
         return x;
     }
 };
@@ -270,11 +235,10 @@ public:
                 int64_t dim_out,
                 int64_t mult          = 4,
                 Activation activation = Activation::GEGLU,
-                bool precision_fix    = false,
-                bool bias             = true) {
+                bool precision_fix    = false) {
         int64_t inner_dim = dim * mult;
         if (activation == Activation::GELU) {
-            blocks["net.0"] = std::shared_ptr<GGMLBlock>(new GELU(dim, inner_dim, bias));
+            blocks["net.0"] = std::shared_ptr<GGMLBlock>(new GELU(dim, inner_dim));
         } else {
             blocks["net.0"] = std::shared_ptr<GGMLBlock>(new GEGLU(dim, inner_dim));
         }
@@ -284,22 +248,22 @@ public:
         float scale         = 1.f;
         if (precision_fix) {
             scale = 1.f / 128.f;
+#ifdef SD_USE_VULKAN
+            force_prec_f32 = true;
+#endif
         }
         // The purpose of the scale here is to prevent NaN issues in certain situations.
         // For example, when using Vulkan without enabling force_prec_f32,
         // or when using CUDA but the weights are k-quants.
-        blocks["net.2"] = std::shared_ptr<GGMLBlock>(new Linear(inner_dim, dim_out, bias, false, force_prec_f32, scale));
+        blocks["net.2"] = std::shared_ptr<GGMLBlock>(new Linear(inner_dim, dim_out, true, false, force_prec_f32, scale));
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx, struct ggml_tensor* x) {
         // x: [ne3, ne2, ne1, dim]
         // return: [ne3, ne2, ne1, dim_out]
 
         auto net_0 = std::dynamic_pointer_cast<UnaryBlock>(blocks["net.0"]);
         auto net_2 = std::dynamic_pointer_cast<Linear>(blocks["net.2"]);
-        if (sd_backend_is(ctx->backend, "Vulkan") || sd_backend_is(ctx->backend, "ROCm")) {
-            net_2->set_force_prec_f32(true);
-        }
 
         x = net_0->forward(ctx, x);  // [ne3, ne2, ne1, inner_dim]
         x = net_2->forward(ctx, x);  // [ne3, ne2, ne1, dim_out]
@@ -313,39 +277,18 @@ protected:
     int64_t context_dim;
     int64_t n_head;
     int64_t d_head;
-    bool xtra_dim  = false;
-    bool enable_ip = false;
-    bool has_ip    = false;
-
-    void init_params(ggml_context* ctx, const String2TensorStorage& tensor_storage_map = {}, const std::string prefix = "") override {
-        GGMLBlock::init_params(ctx, tensor_storage_map, prefix);
-        if (enable_ip &&
-            tensor_storage_map.find(prefix + "to_k_ip.weight") != tensor_storage_map.end()) {
-            has_ip            = true;
-            int64_t inner_dim = d_head * n_head;
-            int64_t ip_dim    = tensor_storage_map.at(prefix + "to_k_ip.weight").ne[0];
-            blocks["to_k_ip"] = std::shared_ptr<GGMLBlock>(new Linear(ip_dim, inner_dim, false));
-            blocks["to_v_ip"] = std::shared_ptr<GGMLBlock>(new Linear(ip_dim, inner_dim, false));
-        }
-    }
 
 public:
     CrossAttention(int64_t query_dim,
                    int64_t context_dim,
                    int64_t n_head,
-                   int64_t d_head,
-                   bool enable_ip = false)
+                   int64_t d_head)
         : n_head(n_head),
           d_head(d_head),
           query_dim(query_dim),
-          context_dim(context_dim),
-          enable_ip(enable_ip) {
+          context_dim(context_dim) {
         int64_t inner_dim = d_head * n_head;
-        if (context_dim == 320 && d_head == 320) {
-            // LOG_VERBOSE("CrossAttention: temp set dim to 1024 for sdxs_09");
-            xtra_dim    = true;
-            context_dim = 1024;
-        }
+
         blocks["to_q"] = std::shared_ptr<GGMLBlock>(new Linear(query_dim, inner_dim, false));
         blocks["to_k"] = std::shared_ptr<GGMLBlock>(new Linear(context_dim, inner_dim, false));
         blocks["to_v"] = std::shared_ptr<GGMLBlock>(new Linear(context_dim, inner_dim, false));
@@ -354,9 +297,9 @@ public:
         // to_out_1 is nn.Dropout(), skip for inference
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx,
-                         ggml_tensor* x,
-                         ggml_tensor* context) {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx,
+                                struct ggml_tensor* x,
+                                struct ggml_tensor* context) {
         // x: [N, n_token, query_dim]
         // context: [N, n_context, context_dim]
         // return: [N, n_token, query_dim]
@@ -370,26 +313,11 @@ public:
         int64_t n_context = context->ne[1];
         int64_t inner_dim = d_head * n_head;
 
-        auto q = to_q->forward(ctx, x);  // [N, n_token, inner_dim]
-        if (xtra_dim) {
-            // LOG_VERBOSE("CrossAttention: temp set dim to 1024 for sdxs_09");
-            context->ne[0] = 1024;  // patch dim
-        }
+        auto q = to_q->forward(ctx, x);        // [N, n_token, inner_dim]
         auto k = to_k->forward(ctx, context);  // [N, n_context, inner_dim]
         auto v = to_v->forward(ctx, context);  // [N, n_context, inner_dim]
-        if (xtra_dim) {
-            context->ne[0] = 320;  // reset dim to orig
-        }
-        x = ggml_ext_attention_ext(ctx, q, k, v, n_head, nullptr, false, ctx->flash_attn_enabled);  // [N, n_token, inner_dim]
 
-        if (has_ip && ctx->ip_context != nullptr && ctx->ip_scale != 0.0f) {
-            auto to_k_ip = std::dynamic_pointer_cast<Linear>(blocks["to_k_ip"]);
-            auto to_v_ip = std::dynamic_pointer_cast<Linear>(blocks["to_v_ip"]);
-            auto k_ip    = to_k_ip->forward(ctx, ctx->ip_context);
-            auto v_ip    = to_v_ip->forward(ctx, ctx->ip_context);
-            auto x_ip    = ggml_ext_attention_ext(ctx, q, k_ip, v_ip, n_head, nullptr, false, ctx->flash_attn_enabled);
-            x            = ggml_add(ctx->ggml_ctx, x, ggml_scale(ctx->ggml_ctx, x_ip, ctx->ip_scale));
-        }
+        x = ggml_ext_attention_ext(ctx->ggml_ctx, ctx->backend, q, k, v, n_head, nullptr, false, false, ctx->flash_attn_enabled);  // [N, n_token, inner_dim]
 
         x = to_out_0->forward(ctx, x);  // [N, n_token, query_dim]
         return x;
@@ -415,7 +343,7 @@ public:
         // inner_dim is always None or equal to dim
         // gated_ff is always True
         blocks["attn1"] = std::shared_ptr<GGMLBlock>(new CrossAttention(dim, dim, n_head, d_head));
-        blocks["attn2"] = std::shared_ptr<GGMLBlock>(new CrossAttention(dim, context_dim, n_head, d_head, true));
+        blocks["attn2"] = std::shared_ptr<GGMLBlock>(new CrossAttention(dim, context_dim, n_head, d_head));
         blocks["ff"]    = std::shared_ptr<GGMLBlock>(new FeedForward(dim, dim));
         blocks["norm1"] = std::shared_ptr<GGMLBlock>(new LayerNorm(dim));
         blocks["norm2"] = std::shared_ptr<GGMLBlock>(new LayerNorm(dim));
@@ -427,9 +355,9 @@ public:
         }
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx,
-                         ggml_tensor* x,
-                         ggml_tensor* context) {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx,
+                                struct ggml_tensor* x,
+                                struct ggml_tensor* context) {
         // x: [N, n_token, query_dim]
         // context: [N, n_context, context_dim]
         // return: [N, n_token, query_dim]
@@ -478,7 +406,7 @@ protected:
     int64_t context_dim = 768;  // hidden_size, 1024 for VERSION_SD2
     bool use_linear     = false;
 
-    void init_params(ggml_context* ctx, const String2TensorStorage& tensor_storage_map = {}, const std::string prefix = "") override {
+    void init_params(struct ggml_context* ctx, const String2TensorStorage& tensor_storage_map = {}, const std::string prefix = "") {
         auto iter = tensor_storage_map.find(prefix + "proj_out.weight");
         if (iter != tensor_storage_map.end()) {
             int64_t inner_dim = n_head * d_head;
@@ -528,9 +456,9 @@ public:
         }
     }
 
-    virtual ggml_tensor* forward(GGMLRunnerContext* ctx,
-                                 ggml_tensor* x,
-                                 ggml_tensor* context) {
+    virtual struct ggml_tensor* forward(GGMLRunnerContext* ctx,
+                                        struct ggml_tensor* x,
+                                        struct ggml_tensor* context) {
         // x: [N, in_channels, h, w]
         // context: [N, max_position(aka n_token), hidden_size(aka context_dim)]
         auto norm     = std::dynamic_pointer_cast<GroupNorm32>(blocks["norm"]);
@@ -582,17 +510,17 @@ public:
 
 class AlphaBlender : public GGMLBlock {
 protected:
-    void init_params(ggml_context* ctx, const String2TensorStorage& tensor_storage_map = {}, std::string prefix = "") override {
+    void init_params(struct ggml_context* ctx, const String2TensorStorage& tensor_storage_map = {}, std::string prefix = "") override {
         // Get the type of the "mix_factor" tensor from the input tensors map with the specified prefix
         enum ggml_type wtype = GGML_TYPE_F32;
         params["mix_factor"] = ggml_new_tensor_1d(ctx, wtype, 1);
     }
 
-    ggml_tensor* get_alpha(GGMLRunnerContext* ctx) {
+    float get_alpha() {
         // image_only_indicator is always tensor([0.]) and since mix_factor.shape is [1,]
         // so learned_with_images is same as learned
-        auto mix_factor = ggml_ext_cast_f32(ctx->ggml_ctx, ctx->backend, params["mix_factor"]);
-        return ggml_sigmoid(ctx->ggml_ctx, mix_factor);
+        float alpha = ggml_ext_backend_tensor_get_f32(params["mix_factor"]);
+        return sigmoid(alpha);
     }
 
 public:
@@ -602,16 +530,15 @@ public:
         // since mix_factor.shape is [1,], we don't need rearrange using  rearrange_pattern
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx,
-                         ggml_tensor* x_spatial,
-                         ggml_tensor* x_temporal) {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx,
+                                struct ggml_tensor* x_spatial,
+                                struct ggml_tensor* x_temporal) {
         // image_only_indicator is always tensor([0.])
-        auto alpha = get_alpha(ctx);
-        return ggml_add(ctx->ggml_ctx,
-                        x_temporal,
-                        ggml_mul(ctx->ggml_ctx,
-                                 ggml_sub(ctx->ggml_ctx, x_spatial, x_temporal),
-                                 alpha));
+        float alpha = get_alpha();
+        auto x      = ggml_add(ctx->ggml_ctx,
+                               ggml_scale(ctx->ggml_ctx, x_spatial, alpha),
+                               ggml_scale(ctx->ggml_ctx, x_temporal, 1.0f - alpha));
+        return x;
     }
 };
 
@@ -628,10 +555,10 @@ public:
         blocks["time_mixer"] = std::shared_ptr<GGMLBlock>(new AlphaBlender());
     }
 
-    ggml_tensor* forward(GGMLRunnerContext* ctx,
-                         ggml_tensor* x,
-                         ggml_tensor* emb,
-                         int num_video_frames) {
+    struct ggml_tensor* forward(GGMLRunnerContext* ctx,
+                                struct ggml_tensor* x,
+                                struct ggml_tensor* emb,
+                                int num_video_frames) {
         // x: [N, channels, h, w] aka [b*t, channels, h, w]
         // emb: [N, emb_channels] aka [b*t, emb_channels]
         // image_only_indicator is always tensor([0.])
@@ -663,4 +590,4 @@ public:
     }
 };
 
-#endif  // __SD_MODEL_COMMON_BLOCK_HPP__
+#endif  // __COMMON_HPP__

control.hpp

@@ -1,10 +1,11 @@
-#ifndef __SD_MODEL_DIFFUSION_CONTROL_HPP__
-#define __SD_MODEL_DIFFUSION_CONTROL_HPP__
+#ifndef __CONTROL_HPP__
+#define __CONTROL_HPP__
 
-#include "model/common/block.hpp"
+#include "common.hpp"
+#include "ggml_extend.hpp"
+#include "model.h"
 
-// Match main UNet's MAX_GRAPH_SIZE so SDXL ControlNet (transformer_depth={1,2,10}) fits.
-#define CONTROL_NET_GRAPH_SIZE MAX_GRAPH_SIZE
+#define CONTROL_NET_GRAPH_SIZE 1536
 
 /*
     =================================== ControlNet ===================================
@@ -164,26 +165,26 @@ public:
         blocks["middle_block_out.0"] = std::shared_ptr<GGMLBlock>(make_zero_conv(ch));
     }
 
-    ggml_tensor* resblock_forward(std::string name,
-                                  GGMLRunnerContext* ctx,
-                                  ggml_tensor* x,
-                                  ggml_tensor* emb) {
+    struct ggml_tensor* resblock_forward(std::string name,
+                                         GGMLRunnerContext* ctx,
+                                         struct ggml_tensor* x,
+                                         struct ggml_tensor* emb) {
         auto block = std::dynamic_pointer_cast<ResBlock>(blocks[name]);
         return block->forward(ctx, x, emb);
     }
 
-    ggml_tensor* attention_layer_forward(std::string name,
-                                         GGMLRunnerContext* ctx,
-                                         ggml_tensor* x,
-                                         ggml_tensor* context) {
+    struct ggml_tensor* attention_layer_forward(std::string name,
+                                                GGMLRunnerContext* ctx,
+                                                struct ggml_tensor* x,
+                                                struct ggml_tensor* context) {
         auto block = std::dynamic_pointer_cast<SpatialTransformer>(blocks[name]);
         return block->forward(ctx, x, context);
     }
 
-    ggml_tensor* input_hint_block_forward(GGMLRunnerContext* ctx,
-                                          ggml_tensor* hint,
-                                          ggml_tensor* emb,
-                                          ggml_tensor* context) {
+    struct ggml_tensor* input_hint_block_forward(GGMLRunnerContext* ctx,
+                                                 struct ggml_tensor* hint,
+                                                 struct ggml_tensor* emb,
+                                                 struct ggml_tensor* context) {
         int num_input_blocks = 15;
         auto h               = hint;
         for (int i = 0; i < num_input_blocks; i++) {
@@ -198,13 +199,13 @@ public:
         return h;
     }
 
-    std::vector<ggml_tensor*> forward(GGMLRunnerContext* ctx,
-                                      ggml_tensor* x,
-                                      ggml_tensor* hint,
-                                      ggml_tensor* guided_hint,
-                                      ggml_tensor* timesteps,
-                                      ggml_tensor* context,
-                                      ggml_tensor* y = nullptr) {
+    std::vector<struct ggml_tensor*> forward(GGMLRunnerContext* ctx,
+                                             struct ggml_tensor* x,
+                                             struct ggml_tensor* hint,
+                                             struct ggml_tensor* guided_hint,
+                                             struct ggml_tensor* timesteps,
+                                             struct ggml_tensor* context,
+                                             struct ggml_tensor* y = nullptr) {
         // x: [N, in_channels, h, w] or [N, in_channels/2, h, w]
         // timesteps: [N,]
         // context: [N, max_position, hidden_size] or [1, max_position, hidden_size]. for example, [N, 77, 768]
@@ -246,7 +247,7 @@ public:
             emb = ggml_add(ctx->ggml_ctx, emb, label_emb);  // [N, time_embed_dim]
         }
 
-        std::vector<ggml_tensor*> outs;
+        std::vector<struct ggml_tensor*> outs;
 
         if (guided_hint == nullptr) {
             guided_hint = input_hint_block_forward(ctx, hint, emb, context);
@@ -309,136 +310,157 @@ public:
 struct ControlNet : public GGMLRunner {
     SDVersion version = VERSION_SD1;
     ControlNetBlock control_net;
-    std::string weight_prefix;
 
-    std::vector<ggml_tensor*> control_outputs_ggml;
-    ggml_tensor* guided_hint_output_ggml = nullptr;
-    std::vector<sd::Tensor<float>> controls;
-    bool guided_hint_cached = false;
-
-    static const char* guided_hint_cache_name() {
-        return "controlnet.guided_hint";
-    }
+    ggml_backend_buffer_t control_buffer = nullptr;  // keep control output tensors in backend memory
+    ggml_context* control_ctx            = nullptr;
+    std::vector<struct ggml_tensor*> controls;  // (12 input block outputs, 1 middle block output) SD 1.5
+    struct ggml_tensor* guided_hint = nullptr;  // guided_hint cache, for faster inference
+    bool guided_hint_cached         = false;
 
     ControlNet(ggml_backend_t backend,
-               const String2TensorStorage& tensor_storage_map      = {},
-               SDVersion version                                   = VERSION_SD1,
-               const std::string& prefix                           = "",
-               std::shared_ptr<RunnerWeightManager> weight_manager = nullptr)
-        : GGMLRunner(backend, weight_manager), version(version), control_net(version), weight_prefix(prefix) {
-        control_net.init(params_ctx, tensor_storage_map, prefix);
+               bool offload_params_to_cpu,
+               const String2TensorStorage& tensor_storage_map = {},
+               SDVersion version                              = VERSION_SD1)
+        : GGMLRunner(backend, offload_params_to_cpu), control_net(version) {
+        control_net.init(params_ctx, tensor_storage_map, "");
     }
 
     ~ControlNet() override {
         free_control_ctx();
     }
 
+    void alloc_control_ctx(std::vector<struct ggml_tensor*> outs) {
+        struct ggml_init_params params;
+        params.mem_size   = static_cast<size_t>(outs.size() * ggml_tensor_overhead()) + 1024 * 1024;
+        params.mem_buffer = nullptr;
+        params.no_alloc   = true;
+        control_ctx       = ggml_init(params);
+
+        controls.resize(outs.size() - 1);
+
+        size_t control_buffer_size = 0;
+
+        guided_hint = ggml_dup_tensor(control_ctx, outs[0]);
+        control_buffer_size += ggml_nbytes(guided_hint);
+
+        for (int i = 0; i < outs.size() - 1; i++) {
+            controls[i] = ggml_dup_tensor(control_ctx, outs[i + 1]);
+            control_buffer_size += ggml_nbytes(controls[i]);
+        }
+
+        control_buffer = ggml_backend_alloc_ctx_tensors(control_ctx, runtime_backend);
+
+        LOG_DEBUG("control buffer size %.2fMB", control_buffer_size * 1.f / 1024.f / 1024.f);
+    }
+
     void free_control_ctx() {
-        guided_hint_output_ggml = nullptr;
-        guided_hint_cached      = false;
-        control_outputs_ggml.clear();
+        if (control_buffer != nullptr) {
+            ggml_backend_buffer_free(control_buffer);
+            control_buffer = nullptr;
+        }
+        if (control_ctx != nullptr) {
+            ggml_free(control_ctx);
+            control_ctx = nullptr;
+        }
+        guided_hint        = nullptr;
+        guided_hint_cached = false;
         controls.clear();
-        free_cache_ctx_and_buffer();
     }
 
     std::string get_desc() override {
         return "control_net";
     }
 
-    void get_param_tensors(std::map<std::string, ggml_tensor*>& tensors) {
-        control_net.get_param_tensors(tensors, weight_prefix);
+    void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors, const std::string prefix) {
+        control_net.get_param_tensors(tensors, prefix);
     }
 
-    ggml_cgraph* build_graph(const sd::Tensor<float>& x_tensor,
-                             const sd::Tensor<float>& hint_tensor,
-                             const sd::Tensor<float>& timesteps_tensor,
-                             const sd::Tensor<float>& context_tensor = {},
-                             const sd::Tensor<float>& y_tensor       = {}) {
-        ggml_cgraph* gf = new_graph_custom(CONTROL_NET_GRAPH_SIZE);
+    struct ggml_cgraph* build_graph(struct ggml_tensor* x,
+                                    struct ggml_tensor* hint,
+                                    struct ggml_tensor* timesteps,
+                                    struct ggml_tensor* context,
+                                    struct ggml_tensor* y = nullptr) {
+        struct ggml_cgraph* gf = new_graph_custom(CONTROL_NET_GRAPH_SIZE);
 
-        ggml_tensor* x         = make_input(x_tensor);
-        ggml_tensor* hint      = nullptr;
-        ggml_tensor* timesteps = make_input(timesteps_tensor);
-        ggml_tensor* context   = make_optional_input(context_tensor);
-        ggml_tensor* y         = make_optional_input(y_tensor);
-
-        guided_hint_output_ggml = nullptr;
-        control_outputs_ggml.clear();
-
-        ggml_tensor* guided_hint_input = nullptr;
+        x = to_backend(x);
         if (guided_hint_cached) {
-            guided_hint_input = get_cache_tensor_by_name(guided_hint_cache_name());
-            if (guided_hint_input == nullptr) {
-                guided_hint_cached = false;
-            }
-        }
-        if (guided_hint_input == nullptr) {
-            hint = make_input(hint_tensor);
+            hint = nullptr;
+        } else {
+            hint = to_backend(hint);
         }
+        context   = to_backend(context);
+        y         = to_backend(y);
+        timesteps = to_backend(timesteps);
 
-        auto runner_ctx = get_context(gf);
+        auto runner_ctx = get_context();
 
         auto outs = control_net.forward(&runner_ctx,
                                         x,
                                         hint,
-                                        guided_hint_input,
+                                        guided_hint_cached ? guided_hint : nullptr,
                                         timesteps,
                                         context,
                                         y);
 
-        if (guided_hint_input == nullptr && !outs.empty()) {
-            guided_hint_output_ggml = outs[0];
-            ggml_set_output(guided_hint_output_ggml);
-            runner_ctx.persist_cache_tensor(guided_hint_cache_name(), guided_hint_output_ggml);
+        if (control_ctx == nullptr) {
+            alloc_control_ctx(outs);
         }
 
-        control_outputs_ggml.reserve(outs.size() > 0 ? outs.size() - 1 : 0);
-        for (size_t i = 1; i < outs.size(); i++) {
-            ggml_tensor* control_output = outs[i];
-            ggml_set_output(control_output);
-            ggml_build_forward_expand(gf, control_output);
-            control_outputs_ggml.push_back(control_output);
+        ggml_build_forward_expand(gf, ggml_cpy(compute_ctx, outs[0], guided_hint));
+        for (int i = 0; i < outs.size() - 1; i++) {
+            ggml_build_forward_expand(gf, ggml_cpy(compute_ctx, outs[i + 1], controls[i]));
         }
 
         return gf;
     }
 
-    std::optional<std::vector<sd::Tensor<float>>> compute(int n_threads,
-                                                          const sd::Tensor<float>& x,
-                                                          const sd::Tensor<float>& hint,
-                                                          const sd::Tensor<float>& timesteps,
-                                                          const sd::Tensor<float>& context = {},
-                                                          const sd::Tensor<float>& y       = {}) {
+    bool compute(int n_threads,
+                 struct ggml_tensor* x,
+                 struct ggml_tensor* hint,
+                 struct ggml_tensor* timesteps,
+                 struct ggml_tensor* context,
+                 struct ggml_tensor* y,
+                 struct ggml_tensor** output     = nullptr,
+                 struct ggml_context* output_ctx = nullptr) {
         // x: [N, in_channels, h, w]
         // timesteps: [N, ]
         // context: [N, max_position, hidden_size]([N, 77, 768]) or [1, max_position, hidden_size]
         // y: [N, adm_in_channels] or [1, adm_in_channels]
-        auto get_graph = [&]() -> ggml_cgraph* {
+        auto get_graph = [&]() -> struct ggml_cgraph* {
             return build_graph(x, hint, timesteps, context, y);
         };
 
-        auto read_outputs = [&]() {
-            controls.clear();
-            controls.reserve(control_outputs_ggml.size());
-            for (ggml_tensor* control : control_outputs_ggml) {
-                auto control_host = restore_trailing_singleton_dims(sd::make_sd_tensor_from_ggml<float>(control), 4);
-                if (control_host.empty()) {
-                    return false;
-                }
-                controls.push_back(std::move(control_host));
-            }
-            return true;
-        };
-        auto compute_result = GGMLRunner::compute(get_graph, n_threads, false, true, read_outputs);
-        control_outputs_ggml.clear();
-        guided_hint_output_ggml = nullptr;
-        if (!compute_result.has_value()) {
-            controls.clear();
-            return std::nullopt;
+        bool res = GGMLRunner::compute(get_graph, n_threads, false, output, output_ctx);
+        if (res) {
+            // cache guided_hint
+            guided_hint_cached = true;
         }
-        guided_hint_cached = get_cache_tensor_by_name(guided_hint_cache_name()) != nullptr;
-        return controls;
+        return res;
+    }
+
+    bool load_from_file(const std::string& file_path, int n_threads) {
+        LOG_INFO("loading control net from '%s'", file_path.c_str());
+        alloc_params_buffer();
+        std::map<std::string, ggml_tensor*> tensors;
+        control_net.get_param_tensors(tensors);
+        std::set<std::string> ignore_tensors;
+
+        ModelLoader model_loader;
+        if (!model_loader.init_from_file_and_convert_name(file_path)) {
+            LOG_ERROR("init control net model loader from file failed: '%s'", file_path.c_str());
+            return false;
+        }
+
+        bool success = model_loader.load_tensors(tensors, ignore_tensors, n_threads);
+
+        if (!success) {
+            LOG_ERROR("load control net tensors from model loader failed");
+            return false;
+        }
+
+        LOG_INFO("control net model loaded");
+        return success;
     }
 };
 
-#endif  // __SD_MODEL_DIFFUSION_CONTROL_HPP__
+#endif  // __CONTROL_HPP__

diffusion_model.hpp

@@ -0,0 +1,450 @@
+#ifndef __DIFFUSION_MODEL_H__
+#define __DIFFUSION_MODEL_H__
+
+#include "flux.hpp"
+#include "mmdit.hpp"
+#include "qwen_image.hpp"
+#include "unet.hpp"
+#include "wan.hpp"
+#include "z_image.hpp"
+
+struct DiffusionParams {
+    struct ggml_tensor* x                     = nullptr;
+    struct ggml_tensor* timesteps             = nullptr;
+    struct ggml_tensor* context               = nullptr;
+    struct ggml_tensor* c_concat              = nullptr;
+    struct ggml_tensor* y                     = nullptr;
+    struct ggml_tensor* guidance              = nullptr;
+    std::vector<ggml_tensor*> ref_latents     = {};
+    bool increase_ref_index                   = false;
+    int num_video_frames                      = -1;
+    std::vector<struct ggml_tensor*> controls = {};
+    float control_strength                    = 0.f;
+    struct ggml_tensor* vace_context          = nullptr;
+    float vace_strength                       = 1.f;
+    std::vector<int> skip_layers              = {};
+};
+
+struct DiffusionModel {
+    virtual std::string get_desc()                                                      = 0;
+    virtual bool compute(int n_threads,
+                         DiffusionParams diffusion_params,
+                         struct ggml_tensor** output     = nullptr,
+                         struct ggml_context* output_ctx = nullptr)                     = 0;
+    virtual void alloc_params_buffer()                                                  = 0;
+    virtual void free_params_buffer()                                                   = 0;
+    virtual void free_compute_buffer()                                                  = 0;
+    virtual void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors) = 0;
+    virtual size_t get_params_buffer_size()                                             = 0;
+    virtual void set_weight_adapter(const std::shared_ptr<WeightAdapter>& adapter){};
+    virtual int64_t get_adm_in_channels()                            = 0;
+    virtual void set_flash_attention_enabled(bool enabled)           = 0;
+    virtual void set_circular_axes(bool circular_x, bool circular_y) = 0;
+};
+
+struct UNetModel : public DiffusionModel {
+    UNetModelRunner unet;
+
+    UNetModel(ggml_backend_t backend,
+              bool offload_params_to_cpu,
+              const String2TensorStorage& tensor_storage_map = {},
+              SDVersion version                              = VERSION_SD1)
+        : unet(backend, offload_params_to_cpu, tensor_storage_map, "model.diffusion_model", version) {
+    }
+
+    std::string get_desc() override {
+        return unet.get_desc();
+    }
+
+    void alloc_params_buffer() override {
+        unet.alloc_params_buffer();
+    }
+
+    void free_params_buffer() override {
+        unet.free_params_buffer();
+    }
+
+    void free_compute_buffer() override {
+        unet.free_compute_buffer();
+    }
+
+    void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors) override {
+        unet.get_param_tensors(tensors, "model.diffusion_model");
+    }
+
+    size_t get_params_buffer_size() override {
+        return unet.get_params_buffer_size();
+    }
+
+    void set_weight_adapter(const std::shared_ptr<WeightAdapter>& adapter) override {
+        unet.set_weight_adapter(adapter);
+    }
+
+    int64_t get_adm_in_channels() override {
+        return unet.unet.adm_in_channels;
+    }
+
+    void set_flash_attention_enabled(bool enabled) {
+        unet.set_flash_attention_enabled(enabled);
+    }
+
+    void set_circular_axes(bool circular_x, bool circular_y) override {
+        unet.set_circular_axes(circular_x, circular_y);
+    }
+
+    bool compute(int n_threads,
+                 DiffusionParams diffusion_params,
+                 struct ggml_tensor** output     = nullptr,
+                 struct ggml_context* output_ctx = nullptr) override {
+        return unet.compute(n_threads,
+                            diffusion_params.x,
+                            diffusion_params.timesteps,
+                            diffusion_params.context,
+                            diffusion_params.c_concat,
+                            diffusion_params.y,
+                            diffusion_params.num_video_frames,
+                            diffusion_params.controls,
+                            diffusion_params.control_strength, output, output_ctx);
+    }
+};
+
+struct MMDiTModel : public DiffusionModel {
+    MMDiTRunner mmdit;
+
+    MMDiTModel(ggml_backend_t backend,
+               bool offload_params_to_cpu,
+               const String2TensorStorage& tensor_storage_map = {})
+        : mmdit(backend, offload_params_to_cpu, tensor_storage_map, "model.diffusion_model") {
+    }
+
+    std::string get_desc() override {
+        return mmdit.get_desc();
+    }
+
+    void alloc_params_buffer() override {
+        mmdit.alloc_params_buffer();
+    }
+
+    void free_params_buffer() override {
+        mmdit.free_params_buffer();
+    }
+
+    void free_compute_buffer() override {
+        mmdit.free_compute_buffer();
+    }
+
+    void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors) override {
+        mmdit.get_param_tensors(tensors, "model.diffusion_model");
+    }
+
+    size_t get_params_buffer_size() override {
+        return mmdit.get_params_buffer_size();
+    }
+
+    void set_weight_adapter(const std::shared_ptr<WeightAdapter>& adapter) override {
+        mmdit.set_weight_adapter(adapter);
+    }
+
+    int64_t get_adm_in_channels() override {
+        return 768 + 1280;
+    }
+
+    void set_flash_attention_enabled(bool enabled) {
+        mmdit.set_flash_attention_enabled(enabled);
+    }
+
+    void set_circular_axes(bool circular_x, bool circular_y) override {
+        mmdit.set_circular_axes(circular_x, circular_y);
+    }
+
+    bool compute(int n_threads,
+                 DiffusionParams diffusion_params,
+                 struct ggml_tensor** output     = nullptr,
+                 struct ggml_context* output_ctx = nullptr) override {
+        return mmdit.compute(n_threads,
+                             diffusion_params.x,
+                             diffusion_params.timesteps,
+                             diffusion_params.context,
+                             diffusion_params.y,
+                             output,
+                             output_ctx,
+                             diffusion_params.skip_layers);
+    }
+};
+
+struct FluxModel : public DiffusionModel {
+    Flux::FluxRunner flux;
+
+    FluxModel(ggml_backend_t backend,
+              bool offload_params_to_cpu,
+              const String2TensorStorage& tensor_storage_map = {},
+              SDVersion version                              = VERSION_FLUX,
+              bool use_mask                                  = false)
+        : flux(backend, offload_params_to_cpu, tensor_storage_map, "model.diffusion_model", version, use_mask) {
+    }
+
+    std::string get_desc() override {
+        return flux.get_desc();
+    }
+
+    void alloc_params_buffer() override {
+        flux.alloc_params_buffer();
+    }
+
+    void free_params_buffer() override {
+        flux.free_params_buffer();
+    }
+
+    void free_compute_buffer() override {
+        flux.free_compute_buffer();
+    }
+
+    void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors) override {
+        flux.get_param_tensors(tensors, "model.diffusion_model");
+    }
+
+    size_t get_params_buffer_size() override {
+        return flux.get_params_buffer_size();
+    }
+
+    void set_weight_adapter(const std::shared_ptr<WeightAdapter>& adapter) override {
+        flux.set_weight_adapter(adapter);
+    }
+
+    int64_t get_adm_in_channels() override {
+        return 768;
+    }
+
+    void set_flash_attention_enabled(bool enabled) {
+        flux.set_flash_attention_enabled(enabled);
+    }
+
+    void set_circular_axes(bool circular_x, bool circular_y) override {
+        flux.set_circular_axes(circular_x, circular_y);
+    }
+
+    bool compute(int n_threads,
+                 DiffusionParams diffusion_params,
+                 struct ggml_tensor** output     = nullptr,
+                 struct ggml_context* output_ctx = nullptr) override {
+        return flux.compute(n_threads,
+                            diffusion_params.x,
+                            diffusion_params.timesteps,
+                            diffusion_params.context,
+                            diffusion_params.c_concat,
+                            diffusion_params.y,
+                            diffusion_params.guidance,
+                            diffusion_params.ref_latents,
+                            diffusion_params.increase_ref_index,
+                            output,
+                            output_ctx,
+                            diffusion_params.skip_layers);
+    }
+};
+
+struct WanModel : public DiffusionModel {
+    std::string prefix;
+    WAN::WanRunner wan;
+
+    WanModel(ggml_backend_t backend,
+             bool offload_params_to_cpu,
+             const String2TensorStorage& tensor_storage_map = {},
+             const std::string prefix                       = "model.diffusion_model",
+             SDVersion version                              = VERSION_WAN2)
+        : prefix(prefix), wan(backend, offload_params_to_cpu, tensor_storage_map, prefix, version) {
+    }
+
+    std::string get_desc() override {
+        return wan.get_desc();
+    }
+
+    void alloc_params_buffer() override {
+        wan.alloc_params_buffer();
+    }
+
+    void free_params_buffer() override {
+        wan.free_params_buffer();
+    }
+
+    void free_compute_buffer() override {
+        wan.free_compute_buffer();
+    }
+
+    void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors) override {
+        wan.get_param_tensors(tensors, prefix);
+    }
+
+    size_t get_params_buffer_size() override {
+        return wan.get_params_buffer_size();
+    }
+
+    void set_weight_adapter(const std::shared_ptr<WeightAdapter>& adapter) override {
+        wan.set_weight_adapter(adapter);
+    }
+
+    int64_t get_adm_in_channels() override {
+        return 768;
+    }
+
+    void set_flash_attention_enabled(bool enabled) {
+        wan.set_flash_attention_enabled(enabled);
+    }
+
+    void set_circular_axes(bool circular_x, bool circular_y) override {
+        wan.set_circular_axes(circular_x, circular_y);
+    }
+
+    bool compute(int n_threads,
+                 DiffusionParams diffusion_params,
+                 struct ggml_tensor** output     = nullptr,
+                 struct ggml_context* output_ctx = nullptr) override {
+        return wan.compute(n_threads,
+                           diffusion_params.x,
+                           diffusion_params.timesteps,
+                           diffusion_params.context,
+                           diffusion_params.y,
+                           diffusion_params.c_concat,
+                           nullptr,
+                           diffusion_params.vace_context,
+                           diffusion_params.vace_strength,
+                           output,
+                           output_ctx);
+    }
+};
+
+struct QwenImageModel : public DiffusionModel {
+    std::string prefix;
+    Qwen::QwenImageRunner qwen_image;
+
+    QwenImageModel(ggml_backend_t backend,
+                   bool offload_params_to_cpu,
+                   const String2TensorStorage& tensor_storage_map = {},
+                   const std::string prefix                       = "model.diffusion_model",
+                   SDVersion version                              = VERSION_QWEN_IMAGE,
+                   bool zero_cond_t                               = false)
+        : prefix(prefix), qwen_image(backend, offload_params_to_cpu, tensor_storage_map, prefix, version, zero_cond_t) {
+    }
+
+    std::string get_desc() override {
+        return qwen_image.get_desc();
+    }
+
+    void alloc_params_buffer() override {
+        qwen_image.alloc_params_buffer();
+    }
+
+    void free_params_buffer() override {
+        qwen_image.free_params_buffer();
+    }
+
+    void free_compute_buffer() override {
+        qwen_image.free_compute_buffer();
+    }
+
+    void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors) override {
+        qwen_image.get_param_tensors(tensors, prefix);
+    }
+
+    size_t get_params_buffer_size() override {
+        return qwen_image.get_params_buffer_size();
+    }
+
+    void set_weight_adapter(const std::shared_ptr<WeightAdapter>& adapter) override {
+        qwen_image.set_weight_adapter(adapter);
+    }
+
+    int64_t get_adm_in_channels() override {
+        return 768;
+    }
+
+    void set_flash_attention_enabled(bool enabled) {
+        qwen_image.set_flash_attention_enabled(enabled);
+    }
+
+    void set_circular_axes(bool circular_x, bool circular_y) override {
+        qwen_image.set_circular_axes(circular_x, circular_y);
+    }
+
+    bool compute(int n_threads,
+                 DiffusionParams diffusion_params,
+                 struct ggml_tensor** output     = nullptr,
+                 struct ggml_context* output_ctx = nullptr) override {
+        return qwen_image.compute(n_threads,
+                                  diffusion_params.x,
+                                  diffusion_params.timesteps,
+                                  diffusion_params.context,
+                                  diffusion_params.ref_latents,
+                                  true,  // increase_ref_index
+                                  output,
+                                  output_ctx);
+    }
+};
+
+struct ZImageModel : public DiffusionModel {
+    std::string prefix;
+    ZImage::ZImageRunner z_image;
+
+    ZImageModel(ggml_backend_t backend,
+                bool offload_params_to_cpu,
+                const String2TensorStorage& tensor_storage_map = {},
+                const std::string prefix                       = "model.diffusion_model",
+                SDVersion version                              = VERSION_Z_IMAGE)
+        : prefix(prefix), z_image(backend, offload_params_to_cpu, tensor_storage_map, prefix, version) {
+    }
+
+    std::string get_desc() override {
+        return z_image.get_desc();
+    }
+
+    void alloc_params_buffer() override {
+        z_image.alloc_params_buffer();
+    }
+
+    void free_params_buffer() override {
+        z_image.free_params_buffer();
+    }
+
+    void free_compute_buffer() override {
+        z_image.free_compute_buffer();
+    }
+
+    void get_param_tensors(std::map<std::string, struct ggml_tensor*>& tensors) override {
+        z_image.get_param_tensors(tensors, prefix);
+    }
+
+    size_t get_params_buffer_size() override {
+        return z_image.get_params_buffer_size();
+    }
+
+    void set_weight_adapter(const std::shared_ptr<WeightAdapter>& adapter) override {
+        z_image.set_weight_adapter(adapter);
+    }
+
+    int64_t get_adm_in_channels() override {
+        return 768;
+    }
+
+    void set_flash_attention_enabled(bool enabled) {
+        z_image.set_flash_attention_enabled(enabled);
+    }
+
+    void set_circular_axes(bool circular_x, bool circular_y) override {
+        z_image.set_circular_axes(circular_x, circular_y);
+    }
+
+    bool compute(int n_threads,
+                 DiffusionParams diffusion_params,
+                 struct ggml_tensor** output     = nullptr,
+                 struct ggml_context* output_ctx = nullptr) override {
+        return z_image.compute(n_threads,
+                               diffusion_params.x,
+                               diffusion_params.timesteps,
+                               diffusion_params.context,
+                               diffusion_params.ref_latents,
+                               true,  // increase_ref_index
+                               output,
+                               output_ctx);
+    }
+};
+
+#endif

docker/Dockerfile

@@ -1,45 +0,0 @@
-ARG UBUNTU_VERSION=24.04
-
-FROM ubuntu:$UBUNTU_VERSION AS build
-
-# sd-server embeds the web UI at build time, so the build image needs Node/pnpm.
-RUN apt-get update && apt-get install -y --no-install-recommends build-essential git cmake ca-certificates curl gnupg && \
-    mkdir -p /etc/apt/keyrings && \
-    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key -o /tmp/nodesource-repo.gpg.key && \
-    gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg /tmp/nodesource-repo.gpg.key && \
-    rm /tmp/nodesource-repo.gpg.key && \
-    echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
-    apt-get update && \
-    apt-get install -y --no-install-recommends nodejs && \
-    npm install -g pnpm@10.15.1 && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
-
-WORKDIR /sd.cpp
-
-COPY . .
-
-RUN cmake . -B ./build \
-    -DSD_BUILD_SHARED_LIBS=ON \
-    -DGGML_NATIVE=OFF \
-    -DSD_BUILD_SHARED_GGML_LIB=ON \
-    -DGGML_BACKEND_DL=ON \
-    -DGGML_CPU_ALL_VARIANTS=ON \
-    -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-    -DCMAKE_INSTALL_RPATH='$ORIGIN'
-RUN cmake --build ./build --config Release --parallel
-
-FROM ubuntu:$UBUNTU_VERSION AS runtime
-
-RUN apt-get update && \
-    apt-get install --yes --no-install-recommends libgomp1 && \
-    apt-get clean
-
-COPY --from=build /sd.cpp/build/bin /sd.cpp/bin
-RUN printf '#!/bin/sh\nexec /sd.cpp/bin/sd-cli "$@"\n' > /sd-cli && \
-    printf '#!/bin/sh\nexec /sd.cpp/bin/sd-server "$@"\n' > /sd-server && \
-    chmod +x /sd-cli /sd-server
-
-WORKDIR /sd.cpp
-
-ENTRYPOINT [ "/sd-cli" ]

docker/Dockerfile.cuda

@@ -1,62 +0,0 @@
-ARG CUDA_VERSION=12.6.3
-ARG UBUNTU_VERSION=24.04
-
-FROM nvidia/cuda:${CUDA_VERSION}-cudnn-devel-ubuntu${UBUNTU_VERSION} AS build
-
-# sd-server embeds the web UI at build time, so the build image needs Node/pnpm.
-RUN apt-get update && apt-get install -y --no-install-recommends build-essential git ccache cmake ca-certificates curl gnupg && \
-    mkdir -p /etc/apt/keyrings && \
-    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key -o /tmp/nodesource-repo.gpg.key && \
-    gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg /tmp/nodesource-repo.gpg.key && \
-    rm /tmp/nodesource-repo.gpg.key && \
-    echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
-    apt-get update && \
-    apt-get install -y --no-install-recommends nodejs && \
-    npm install -g pnpm@10.15.1 && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
-
-WORKDIR /sd.cpp
-
-COPY . .
-
-ARG CUDACXX=/usr/local/cuda/bin/nvcc
-ARG CUDA_ARCHITECTURES=""
-ARG GGML_CUDA_FA_ALL_QUANTS=""
-ARG GGML_CUDA_ENABLE_DYNAMIC_CPU_BACKENDS=ON
-
-RUN set -- \
-        -DSD_CUDA=ON; \
-    if [ "${GGML_CUDA_ENABLE_DYNAMIC_CPU_BACKENDS}" = "ON" ]; then \
-        set -- "$@" \
-            -DSD_BUILD_SHARED_LIBS=ON \
-            -DGGML_NATIVE=OFF \
-            -DSD_BUILD_SHARED_GGML_LIB=ON \
-            -DGGML_BACKEND_DL=ON \
-            -DGGML_CPU_ALL_VARIANTS=ON \
-            -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-            "-DCMAKE_INSTALL_RPATH=\$ORIGIN"; \
-    fi; \
-    if [ -n "${CUDA_ARCHITECTURES}" ]; then \
-        set -- "$@" "-DCMAKE_CUDA_ARCHITECTURES=${CUDA_ARCHITECTURES}"; \
-    fi; \
-    if [ -n "${GGML_CUDA_FA_ALL_QUANTS}" ]; then \
-        set -- "$@" "-DGGML_CUDA_FA_ALL_QUANTS=${GGML_CUDA_FA_ALL_QUANTS}"; \
-    fi; \
-    cmake . -B ./build "$@"
-RUN cmake --build ./build --config Release -j$(nproc)
-
-FROM nvidia/cuda:${CUDA_VERSION}-cudnn-runtime-ubuntu${UBUNTU_VERSION} AS runtime
-
-RUN apt-get update && \
-    apt-get install --yes --no-install-recommends libgomp1 && \
-    apt-get clean
-
-COPY --from=build /sd.cpp/build/bin /sd.cpp/bin
-RUN printf '#!/bin/sh\nexec /sd.cpp/bin/sd-cli "$@"\n' > /sd-cli && \
-    printf '#!/bin/sh\nexec /sd.cpp/bin/sd-server "$@"\n' > /sd-server && \
-    chmod +x /sd-cli /sd-server
-
-WORKDIR /sd.cpp
-
-ENTRYPOINT [ "/sd-cli" ]

docker/Dockerfile.musa

@@ -1,47 +0,0 @@
-ARG MUSA_VERSION=rc4.2.0
-ARG UBUNTU_VERSION=22.04
-
-FROM mthreads/musa:${MUSA_VERSION}-devel-ubuntu${UBUNTU_VERSION}-amd64 as build
-
-# sd-server embeds the web UI at build time, so the build image needs Node/pnpm.
-RUN apt-get update && apt-get install -y --no-install-recommends ccache cmake git ca-certificates curl gnupg && \
-    mkdir -p /etc/apt/keyrings && \
-    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key -o /tmp/nodesource-repo.gpg.key && \
-    gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg /tmp/nodesource-repo.gpg.key && \
-    rm /tmp/nodesource-repo.gpg.key && \
-    echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
-    apt-get update && \
-    apt-get install -y --no-install-recommends nodejs && \
-    npm install -g pnpm@10.15.1 && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
-
-WORKDIR /sd.cpp
-
-COPY . .
-
-RUN mkdir build && cd build && \
-    cmake .. -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
-        -DCMAKE_C_FLAGS="${CMAKE_C_FLAGS} -fopenmp -I/usr/lib/llvm-14/lib/clang/14.0.0/include -L/usr/lib/llvm-14/lib" \
-        -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -fopenmp -I/usr/lib/llvm-14/lib/clang/14.0.0/include -L/usr/lib/llvm-14/lib" \
-        -DSD_MUSA=ON \
-        -DSD_BUILD_SHARED_LIBS=ON \
-        -DGGML_NATIVE=OFF \
-        -DSD_BUILD_SHARED_GGML_LIB=ON \
-        -DGGML_BACKEND_DL=ON \
-        -DGGML_CPU_ALL_VARIANTS=ON \
-        -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-        -DCMAKE_INSTALL_RPATH='$ORIGIN' \
-        -DCMAKE_BUILD_TYPE=Release && \
-    cmake --build . --config Release
-
-FROM mthreads/musa:${MUSA_VERSION}-runtime-ubuntu${UBUNTU_VERSION}-amd64 as runtime
-
-COPY --from=build /sd.cpp/build/bin /sd.cpp/bin
-RUN printf '#!/bin/sh\nexec /sd.cpp/bin/sd-cli "$@"\n' > /sd-cli && \
-    printf '#!/bin/sh\nexec /sd.cpp/bin/sd-server "$@"\n' > /sd-server && \
-    chmod +x /sd-cli /sd-server
-
-WORKDIR /sd.cpp
-
-ENTRYPOINT [ "/sd-cli" ]

docker/Dockerfile.sycl

@@ -1,34 +0,0 @@
-# ggml SYCL hardware detection uses BMG G31/WCL architecture enums added in oneAPI 2025.3.
-ARG SYCL_VERSION=2025.3.2-0
-
-FROM intel/oneapi-basekit:${SYCL_VERSION}-devel-ubuntu24.04 AS build
-
-# sd-server embeds the web UI at build time, so the build image needs Node/pnpm.
-RUN apt-get update && apt-get install -y --no-install-recommends cmake ca-certificates curl gnupg && \
-    mkdir -p /etc/apt/keyrings && \
-    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key -o /tmp/nodesource-repo.gpg.key && \
-    gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg /tmp/nodesource-repo.gpg.key && \
-    rm /tmp/nodesource-repo.gpg.key && \
-    echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
-    apt-get update && \
-    apt-get install -y --no-install-recommends nodejs && \
-    npm install -g pnpm@10.15.1 && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
-
-WORKDIR /sd.cpp
-
-COPY . .
-
-RUN mkdir build && cd build && \
-    cmake .. -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DSD_SYCL=ON -DCMAKE_BUILD_TYPE=Release && \
-    cmake --build . --config Release -j$(nproc)
-
-FROM intel/oneapi-basekit:${SYCL_VERSION}-devel-ubuntu24.04 AS runtime
-
-COPY --from=build /sd.cpp/build/bin/sd-cli /sd-cli
-COPY --from=build /sd.cpp/build/bin/sd-server /sd-server
-
-WORKDIR /sd.cpp
-
-ENTRYPOINT [ "/sd-cli" ]

docker/Dockerfile.vulkan

@@ -1,46 +0,0 @@
-ARG UBUNTU_VERSION=24.04
-
-FROM ubuntu:$UBUNTU_VERSION AS build
-
-# sd-server embeds the web UI at build time, so the build image needs Node/pnpm.
-RUN apt-get update && apt-get install -y --no-install-recommends build-essential git cmake libvulkan-dev glslc spirv-headers ca-certificates curl gnupg && \
-    mkdir -p /etc/apt/keyrings && \
-    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key -o /tmp/nodesource-repo.gpg.key && \
-    gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg /tmp/nodesource-repo.gpg.key && \
-    rm /tmp/nodesource-repo.gpg.key && \
-    echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
-    apt-get update && \
-    apt-get install -y --no-install-recommends nodejs && \
-    npm install -g pnpm@10.15.1 && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
-
-WORKDIR /sd.cpp
-
-COPY . .
-
-RUN cmake . -B ./build \
-    -DSD_VULKAN=ON \
-    -DSD_BUILD_SHARED_LIBS=ON \
-    -DGGML_NATIVE=OFF \
-    -DSD_BUILD_SHARED_GGML_LIB=ON \
-    -DGGML_BACKEND_DL=ON \
-    -DGGML_CPU_ALL_VARIANTS=ON \
-    -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-    -DCMAKE_INSTALL_RPATH='$ORIGIN'
-RUN cmake --build ./build --config Release -j$(nproc)
-
-FROM ubuntu:$UBUNTU_VERSION AS runtime
-
-RUN apt-get update && \
-    apt-get install --yes --no-install-recommends libgomp1 libvulkan1 mesa-vulkan-drivers libglvnd0 libgl1 libglx0 libegl1 libgles2 && \
-    apt-get clean
-
-COPY --from=build /sd.cpp/build/bin /sd.cpp/bin
-RUN printf '#!/bin/sh\nexec /sd.cpp/bin/sd-cli "$@"\n' > /sd-cli && \
-    printf '#!/bin/sh\nexec /sd.cpp/bin/sd-server "$@"\n' > /sd-server && \
-    chmod +x /sd-cli /sd-server
-
-WORKDIR /sd.cpp
-
-ENTRYPOINT [ "/sd-cli" ]

docs/adetailer.md

@@ -1,110 +0,0 @@
-# ADetailer
-
-`sd-cli` can run a YOLOv8 object detector on an existing or newly generated
-image and perform a cropped inpaint pass for every detected object. The first
-implementation supports YOLOv8 detection checkpoints. YOLOv8 segmentation and
-MediaPipe models are not supported yet.
-
-## Convert a detector
-
-Ultralytics checkpoints must be converted before use. The converter fuses
-BatchNorm into convolution layers and writes a safetensors file with the weight
-names expected by the native GGML implementation.
-
-```bash
-python scripts/convert_yolov8_to_safetensors.py face_yolov8n.pt face_yolov8n.safetensors
-```
-
-The converter requires Python packages `ultralytics`, `torch`, and
-`safetensors`.
-Only YOLOv8 detection checkpoints are accepted.
-PyTorch checkpoints use pickle internally, so only convert `.pt` files from a
-trusted source.
-
-## Repair an existing image
-
-Use the dedicated `adetailer` mode to detect and repair objects in an existing
-image:
-
-```bash
-./bin/sd-cli \
-  -M adetailer \
-  -m model.safetensors \
-  -i input.png \
-  -o repaired.png \
-  -p "detailed portrait photo" \
-  --negative-prompt "deformed face" \
-  --steps 24 \
-  --cfg-scale 6 \
-  --strength 0.4 \
-  --sampling-method dpm++2m \
-  --scheduler karras \
-  --ad-model face_yolov8n.safetensors \
-  --extra-ad-args "confidence=0.3,inpaint_padding=32,mask_blur=4"
-```
-
-This mode reuses the normal image-generation options for the detail pass:
-
-- `--init-img`, `--output`, `--prompt`, and `--negative-prompt`
-- `--steps`, `--cfg-scale`, `--sampling-method`, and `--scheduler`
-- `--strength`, `--seed`, LoRA settings, VAE tiling, and backend assignments
-- `--width` and `--height`, which also resize the input when specified
-
-`--ad-prompt` and `--ad-negative-prompt` optionally override the normal prompts.
-Values provided in `--extra-ad-args`, such as `steps`, `cfg_scale`,
-`denoising_strength`, or `inpaint_width`, take precedence over inherited values.
-
-## Repair generated images
-
-ADetailer can also run automatically after normal image generation:
-
-```bash
-./bin/sd-cli \
-  -m model.safetensors \
-  -p "portrait photo" \
-  --ad-model face_yolov8n.safetensors \
-  --ad-prompt "[PROMPT], detailed face" \
-  --ad-negative-prompt "" \
-  --extra-ad-args "confidence=0.3,denoising_strength=0.4,inpaint_width=512,inpaint_height=512"
-```
-
-An empty ADetailer prompt inherits the main prompt. `[PROMPT]` inserts the main
-prompt, `[SEP]` assigns different prompts to consecutive masks, and `[SKIP]`
-skips the corresponding mask.
-
-All settings other than the detector path and prompts are passed through
-`--extra-ad-args` as a comma-separated `key=value` list:
-
-| Key | Default | Description |
-| --- | ---: | --- |
-| `input_size` | `640` | Square YOLO input size; must be a multiple of 32 |
-| `confidence` | `0.3` | Detection confidence threshold |
-| `nms` | `0.45` | NMS IoU threshold |
-| `max_detections` | `100` | Maximum detections retained after NMS |
-| `mask_k_largest` | `0` | Keep only the largest K masks; zero keeps all |
-| `mask_min_ratio` | `0` | Minimum bbox area relative to the image |
-| `mask_max_ratio` | `1` | Maximum bbox area relative to the image |
-| `dilate_erode` | `4` | Positive values dilate; negative values erode |
-| `x_offset`, `y_offset` | `0` | Mask offset in pixels; positive Y moves upward |
-| `mask_mode` | `none` | `none`, `merge`, or `merge_invert` |
-| `merge_masks`, `invert_mask` | `false` | Boolean alternatives to `mask_mode` |
-| `mask_blur` | `4` | Final composite feather radius |
-| `inpaint_padding` | `32` | Padding around the detected region |
-| `inpaint_width`, `inpaint_height` | mode-specific | `512x512` after generation; input/output size in `adetailer` mode |
-| `denoising_strength` | mode-specific | `0.4` after generation; inherits `--strength` in `adetailer` mode |
-| `steps` | `0` | Detail steps; zero inherits the main generation |
-| `cfg_scale` | `-1` | Detail CFG; a negative value inherits the main generation |
-| `sample_method` | inherited | Detail sampler name |
-| `scheduler` | inherited | Detail scheduler name |
-| `sort_by` | `none` | `none`, `left_to_right`, `center_to_edge`, or `area` |
-
-Multiple masks are processed serially. Each completed inpaint becomes the input
-for the next mask, and the seed is incremented by the mask index. Use
-`mask_mode=merge` to process all detections in one inpaint pass.
-
-The detector uses the `detector` backend module. For example, keep detection on
-the CPU while diffusion runs on CUDA:
-
-```bash
---backend "diffusion=cuda0,detector=cpu"
-```

docs/anima.md

@@ -1,21 +0,0 @@
-# How to Use
-
-## Download weights
-
-- Download Anima
-    - safetensors: https://huggingface.co/circlestone-labs/Anima/tree/main/split_files/diffusion_models
-    - gguf: https://huggingface.co/Bedovyy/Anima-GGUF/tree/main
-    - gguf Anima2: https://huggingface.co/JusteLeo/Anima2-GGUF/tree/main
-- Download vae
-    - safetensors: https://huggingface.co/circlestone-labs/Anima/tree/main/split_files/vae
-- Download Qwen3-0.6B-Base
-    - safetensors: https://huggingface.co/circlestone-labs/Anima/tree/main/split_files/text_encoders
-    - gguf: https://huggingface.co/mradermacher/Qwen3-0.6B-Base-GGUF/tree/main
-
-## Examples
-
-```sh
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\anima-preview.safetensors --vae ..\models\vae\qwen_image_vae.safetensors  --llm ..\models\text_encoders\qwen_3_06b_base.safetensors  -p "a lovely cat holding a sign says 'anima.cpp'" --cfg-scale 6.0 --sampling-method euler -v --offload-to-cpu --diffusion-fa
-```
-
-<img alt="anima image example" src="../assets/anima/example.png" />

docs/animatediff.md

@@ -1,171 +0,0 @@
-# AnimateDiff (SD 1.5)
-
-AnimateDiff adds motion to a frozen Stable Diffusion 1.5 checkpoint by
-injecting a temporal-attention module at 20 UNet slots. The base SD 1.5
-model, VAE, and text encoder are unchanged; only the motion module produces
-the temporal residual that turns a batch of independent frames into a
-coherent animation. Reference: Guo et al., "AnimateDiff: Animate Your
-Personalized Text-to-Image Diffusion Models without Specific Tuning"
-(https://arxiv.org/abs/2307.04725).
-
-## Download weights
-
-- Motion module (v3, recommended)
-    - fp16 safetensors: https://huggingface.co/conrevo/AnimateDiff-A1111/resolve/main/motion_module/mm_sd15_v3.safetensors
-    - original checkpoint: https://huggingface.co/guoyww/animatediff/resolve/main/v3_sd15_mm.ckpt
-- SD 1.5 base model
-    - any SD 1.5 checkpoint works. `realisticVisionV60B1` and `toonyou_beta3`
-      are the ones used in guoyww's reference configs.
-- Domain Adapter LoRA (optional, v3 only, sharpens the base UNet's output
-  toward the motion module's trained distribution)
-    - ckpt: https://huggingface.co/guoyww/animatediff/resolve/main/v3_sd15_adapter.ckpt
-    - place under your `--lora-model-dir` and reference in the prompt as
-      `<lora:v3_sd15_adapter:1.0>`.
-
-The motion module is `~836 MB` and loads alongside the SD 1.5 UNet via
-`--motion-module`.
-
-## Motion module versions
-
-Per [animatediff.net/models](https://animatediff.net/models):
-
-| Module              | Base | Native res | Character |
-|---------------------|------|------------|-----------|
-| `mm_sd_v14.ckpt`    | 1.5  | 256x256    | earliest, more jittery |
-| `mm_sd_v15.ckpt`    | 1.5  | 256x256    | improved stability over v1.4 |
-| `mm_sd_v15_v2.ckpt` | 1.5  | 384x384    | significantly better motion dynamics |
-| `v3_sd15_mm.ckpt`   | 1.5  | 512x512    | smoothest, highest quality; pairs with a Domain Adapter LoRA |
-| `mm_sdxl_v10_beta`  | SDXL | 512x512    | experimental, not yet supported here |
-
-Match your `-H -W` to the module's native resolution for best results. v3 is
-trained at 512x512 - going smaller (e.g. 384x384) still works but the motion
-character is closer to v2.
-
-## Examples
-
-Generate an 8-frame animation at 512x512, seed 42, 20 steps. The sampler /
-scheduler / CFG values below match what mm_sd15_v3 was trained with; using
-SD 1.5 defaults (euler_a, low CFG) produces noise-like output.
-
-```
-.\bin\Release\sd-cli.exe -M vid_gen \
-    --model ..\models\checkpoints\realisticVisionV60B1.safetensors \
-    --motion-module ..\models\animatediff\mm_sd15_v3.safetensors \
-    --offload-to-cpu --diffusion-fa \
-    -p "a red apple on a wooden table" \
-    --cfg-scale 8.0 --sampling-method euler --scheduler discrete \
-    -H 512 -W 512 --video-frames 8 --fps 8 --steps 20 -s 42 \
-    -o out.avi
-```
-
-Generate at the motion module's native 16-frame context (recommended for
-best temporal quality). Needs more VRAM at 512x512, so drop to 384x384 or
-use layer streaming:
-
-```
-.\bin\Release\sd-cli.exe -M vid_gen \
-    --model ..\models\checkpoints\realisticVisionV60B1.safetensors \
-    --motion-module ..\models\animatediff\mm_sd15_v3.safetensors \
-    --offload-to-cpu --diffusion-fa \
-    -p "photo of coastline, rocks, storm weather, wind, waves, lightning" \
-    --cfg-scale 8.0 --sampling-method euler --scheduler discrete \
-    -H 384 -W 384 --video-frames 16 --fps 8 --steps 20 -s 42 \
-    -o out.avi
-```
-
-Low-VRAM streaming (verified with a 2 GiB cap on RTX 3060):
-
-```
-.\bin\Release\sd-cli.exe -M vid_gen \
-    --model ..\models\checkpoints\realisticVisionV60B1.safetensors \
-    --motion-module ..\models\animatediff\mm_sd15_v3.safetensors \
-    --max-vram 2.0 --diffusion-fa \
-    -p "photo of coastline, rocks, storm weather, wind, waves, lightning" \
-    --cfg-scale 8.0 --sampling-method euler --scheduler discrete \
-    -H 384 -W 384 --video-frames 8 --fps 8 --steps 20 -s 42 \
-    -o out.avi
-```
-
-## Reference-quality reproduction
-
-Using guoyww's official reference configs on this impl (RealisticVision v6.0
-base + `mm_sd15_v3` or `mm_sd_v15_v2` + native resolution + 16 frames + euler
-+ 25 steps + CFG 8 + linear beta schedule) reproduces the reference
-AnimateDiff output style.
-
-### v3 (512x512, `mm_sd15_v3`)
-
-| Prompt                                | Sample |
-|---------------------------------------|--------|
-| B&W man on stormy coastline           | <img src="../assets/animatediff/v3_man.gif" width="256"/>    |
-| Close-up rabbit macro shot            | <img src="../assets/animatediff/v3_rabbit.gif" width="256"/> |
-| Coastline, storm, waves, lightning    | <img src="../assets/animatediff/v3_coast.gif" width="256"/>  |
-| Old house, storm, forest, night       | <img src="../assets/animatediff/v3_house.gif" width="256"/>  |
-
-### v2 (384x384, `mm_sd_v15_v2.ckpt`)
-
-| Prompt                                | Sample |
-|---------------------------------------|--------|
-| B&W man on stormy coastline           | <img src="../assets/animatediff/v2_man.gif" width="256"/>    |
-| Close-up rabbit macro shot            | <img src="../assets/animatediff/v2_rabbit.gif" width="256"/> |
-| Coastline, storm, waves, lightning    | <img src="../assets/animatediff/v2_coast.gif" width="256"/>  |
-| Old house, storm, forest, night       | <img src="../assets/animatediff/v2_house.gif" width="256"/>  |
-
-Motion is strong for scenes with motion cues in the prompt (storm/waves/wind)
-and subtle for static subjects (close-up macro), matching reference behavior.
-v2 has an additional motion module at the UNet middle block that v3 dropped;
-this impl auto-detects the topology from the checkpoint.
-
-### v3 + Domain Adapter LoRA
-
-Attaching the `v3_sd15_adapter` LoRA sharpens the base UNet output toward
-the training distribution the motion module was fine-tuned against. Same
-prompt, seed, config as above:
-
-<img src="../assets/animatediff/v3_rabbit_domain_lora.gif" width="256"/>
-
-Individual fur strands, glowing inner-ear, and richer forest detail become
-visible compared to the no-LoRA rendering.
-
-```
-sd-cli -M vid_gen --model realisticVisionV60B1.safetensors \
-       --motion-module mm_sd15_v3.safetensors \
-       --lora-model-dir ./loras \
-       -p "close up photo of a rabbit ...<lora:v3_sd15_adapter:1.0>" ...
-```
-
-## img2video
-
-Pass a pre-rendered image via `-i / --init-img` to animate FROM it. All N output frames start from the encoded init latent, then per-frame noise is added at `--strength`. Character identity, composition, and quality are anchored by the init image; the motion module adds subtle motion on top.
-
-Left: init image rendered with `-M img_gen`. Right: 8-frame vid_gen output.
-
-<img src="../assets/animatediff/img2video_demo.gif" width="512"/>
-
-```
-sd-cli -M img_gen ... -o init.png                  # any high-quality still
-sd-cli -M vid_gen --motion-module mm_sd15_v3.safetensors \
-    -i init.png --strength 0.75 \
-    --cfg-scale 7.0 --sampling-method euler --scheduler karras \
-    -H 512 -W 512 --video-frames 8 --steps 25 -s 42 \
-    -p "..." -o out.avi
-```
-
-`--strength` controls how far the motion module is allowed to deviate from the init image (higher = more motion, lower = more static).
-
-## Notes
-
-- The motion module was trained at `video_length=16`. Running with
-  `--video-frames 16` gives the best coherence; F=8 works but shows a shorter
-  motion arc. Frame counts up to 32 are supported by the positional encoding
-  but exceed the trained regime and produce more static output.
-- At `--video-frames 1` the motion module is skipped entirely and the output
-  is bit-identical to `-M img_gen`. This avoids the single-token
-  temporal-attention degeneracy that would otherwise emit an untrained-magnitude
-  residual on a single-frame sample.
-- The base UNet is frozen, so character identity and style follow the SD 1.5
-  checkpoint you pass to `--model`. LoRAs and prompt weighting attach to the
-  base model in the usual way.
-- No mid_block motion module in v3. `mm_sdxl_v10_beta` (SDXL variant) is not
-  supported yet.
-- Output is written as MJPEG AVI. Use `--fps` to set playback speed.

docs/backend.md

@@ -1,321 +0,0 @@
-# Backend selection
-
-`stable-diffusion.cpp` has two backend assignments:
-
-- `--backend` selects the runtime backend used to execute model graphs.
-- `--params-backend` selects where model parameters are kept.
-
-If `--params-backend` is not set, auto-fit chooses parameter placement. With
-`--auto-fit off`, parameters use the same backend as their module runtime backend.
-
-## Syntax
-
-A backend assignment can be a single backend name:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend cpu
-```
-
-This applies to every module that does not have a more specific assignment.
-
-Assignments can also target individual modules:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend te=cpu,vae=cuda0,diffusion=vulkan0
-```
-
-The same syntax is used for parameter placement:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend cuda0 --params-backend te=cpu,vae=cpu
-```
-
-`--params-backend` also accepts the special value `disk`:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend cuda0 --params-backend disk
-```
-
-`--max-vram` can target resolved backend/device names:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend diffusion=cuda0,vae=vulkan0 --max-vram cuda0=6,vulkan0=2
-```
-
-The value is a shared per-device budget for managed weights and registered
-runner compute/cache buffers. Live free memory can lower the effective limit
-for each graph run. Driver contexts and allocations made outside the managed
-model runners are not part of this accounting, so it is not a hard physical
-VRAM cap.
-
-Module names are case-insensitive. Hyphens and underscores in module names are ignored, so `clip_vision`, `clip-vision`, and `clipvision` are equivalent.
-
-`all=`, `default=`, and `*=` can be used to set the default backend inside a mixed assignment:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend all=cuda0,te=cpu
-```
-
-## Multiple devices per module (layer split)
-
-A `--backend` module assignment can list several devices separated by `&`:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend "diffusion=cuda0&cuda1"
-```
-
-The module's transformer blocks are then distributed across the listed devices
-in contiguous ranges sized proportionally to each device's free memory (minus a
-compute-buffer headroom of about 2 GiB per device), and the
-module's graphs are executed with a `ggml_backend_sched` that runs each block
-on the device holding its weights, copying the residual stream at the range
-boundaries. The first device in the list is the module's main device: it also
-holds the non-block tensors (embeddings, final norms, small sub-runners such as
-CLIP models or projectors) and the graph inputs/outputs.
-
-Layer split is supported for the `diffusion` and `te` modules. For `te` it
-applies to the dominant text encoder (`t5xxl` or the LLM); other modules accept
-only a single device. If the module has no recognizable transformer blocks, the
-assignment falls back to the first listed device.
-
-`--params-backend` accepts no device lists. If the module has no explicit
-params assignment, each block range's parameters are loaded directly to (and,
-with `--params-backend diffusion=disk`, released directly from) its own device;
-an explicit assignment such as `te=cpu` keeps the parameters on that backend
-and stages each range to its device on demand.
-
-Layer split uses the fixed graph-cut plan to assign blocks across devices, but
-single-device segmented execution and next-segment prefetch are disabled for
-the split module. `--max-vram` can still provide the per-device limits used by
-layer split and auto-fit.
-
-Use `--list-devices` to see the device names available on the system.
-
-### Row split (`--split-mode row`)
-
-`--split-mode` selects how a multi-device module distributes its weights:
-`layer` (the default, described above) or `row`. It accepts a single mode or
-per-module assignments:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend "diffusion=cuda0&cuda1" --split-mode row
-sd-cli -m model.safetensors -p "a cat" --backend "diffusion=cuda0&cuda1,te=cuda0&cuda1" --split-mode diffusion=row,te=layer
-```
-
-In row mode the module keeps executing on its main (first listed) device, but
-its transformer-block matmul weights are allocated in the backend's row-split
-buffer type, which slices each weight's rows across the listed devices in
-proportion to free memory and runs those matmuls on all devices in parallel.
-Compared to a layer split this uses all GPUs within every layer (instead of
-sequentially device by device) at the cost of a cross-device reduction per
-matmul - usually the faster option when the devices have fast interconnect.
-
-Row split requires a compatible split-buffer export from the linked GGML
-backend. If it is unavailable (or the listed devices belong to different backend
-registries), the module falls back to a layer split.
-Embeddings, normalization weights, biases and other non-block tensors stay in
-regular buffers on the main device.
-
-Row-split execution can use graph segments, but split weights are loaded
-synchronously instead of using the normal single-device prefetch path. Because
-GGML does not expose exact shard allocation sizes, the managed budget currently
-counts a split buffer's full size on each participating device. This is a
-conservative bound and can reject otherwise feasible layouts.
-
-Direct ("immediately") LoRA application cannot patch row-split tensors; with
-`--split-mode row` the automatic LoRA mode selects runtime application, and an
-explicit `--lora-apply-mode immediately` skips the split tensors with a
-warning.
-
-## Automatic placement (`--auto-fit on|off`)
-
-`--auto-fit` requires `on` or `off` and defaults to `on` when omitted.
-Explicit `--params-backend` assignments disable auto-fit,
-regardless of argument order, even with `--auto-fit on`.
-
-Auto-fit preserves explicit `--backend` assignments, including per-module
-assignments and device lists. For modules without a runtime assignment, it chooses
-the GPU with the largest available memory budget (the first device on a tie).
-It then derives parameter placements from the model metadata, each module's
-compute devices, and the remaining memory budgets. The chosen backend
-specifications are printed.
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --auto-fit on
-sd-cli -m model.safetensors -p "a cat" --auto-fit on --max-vram cuda0=8,cuda1=14
-sd-cli -m model.safetensors -p "a cat" --backend cuda0
-sd-cli -m model.safetensors -p "a cat" --backend diffusion=cuda0,te=cpu,vae=cuda1
-sd-cli -m model.safetensors -p "a cat" --auto-fit off
-```
-
-Budgets reuse `--max-vram`: a positive per-device value caps what auto-fit
-plans with on that device, a negative value means "free memory minus that many
-GiB", and with no budget set each device's free memory minus a 512 MiB margin
-is used. These resolved GPU budgets, including the safety margin, also drive
-the runner's graph-cut capacity checks.
-
-Runtime capacity checks also leave 512 MiB of currently free device memory for
-backend scratch buffers and pipelines, including with explicit backend assignments.
-They cap free-memory reports by the device's total memory minus tracked
-resident allocations. Vulkan reports exceeding total memory are rejected because
-its heap-budget subtraction can underflow. Other backends use the cap instead of
-treating such reports as zero free memory. Failed checks log the reported free and
-total memory alongside tracked weight and runtime allocations.
-With `--mmap`, device-backed mappings count toward these budgets at their full
-mapped-file size, once per device buffer even when multiple parameter blocks
-share it. Mappings retained in the loader cache continue to count.
-
-Components are considered in `diffusion`, `te`, `vae` order so that repeatedly
-used diffusion weights have priority. Each component's weights use the first
-storage location with enough remaining budget:
-
-1. The component's compute GPU, leaving estimated space for computation and weight staging.
-2. CPU RAM, reserving the larger of 2 GiB or 10% of available RAM for other work.
-3. Another GPU, choosing the one with the largest remaining budget that fits.
-4. Disk, reloading weights on demand.
-
-GPU cache space follows the same component priority. Before a lower-priority
-component can become permanently resident, the planner leaves room for the full
-weights and estimated compute space of higher-priority offloaded components.
-If offloaded diffusion already needs the entire main GPU budget, TE and VAE also
-use offloaded parameters. Their GPU copies can then be released after their
-phases, leaving more room to reuse diffusion weights across sampling steps.
-CPU parameter residency allows GPU weight caching; it does not force every
-weight to be copied again at every step.
-
-RAM and GPU budgets are shared across components. Each component uses a single
-parameter backend; several other GPUs' capacities are not combined to store
-one component. If available RAM cannot be queried, RAM residency is skipped.
-Weights stored on another GPU are copied to the component's compute devices for
-execution. CPU modules use RAM or disk. Compute reserves and cache priority are
-accounted for separately on each device, so a CPU module does not reserve GPU
-space. Storage on another module's GPU also leaves room for that module's work.
-
-Auto-fit does not select multi-GPU layer/row computation itself. Explicit device
-lists and `--split-mode` still control that computation. Before the runners have
-built their split plans, auto-fit conservatively counts the full component size
-on each listed GPU when checking residency and cache space. This can offload
-parameters even when a split layout would fit; use `--auto-fit off` to keep the
-default split-device parameter placement.
-
-For example, a diffusion model whose full weights exceed the main GPU's budget
-can use `--backend diffusion=cuda0 --params-backend diffusion=cpu` when RAM is
-sufficient. Automatic graph segmentation can then load the required weights
-for each segment and reclaim idle GPU copies. `--disable-segmented-compute`
-still disables segmentation.
-
-Initial compute reserves are estimates (2 GiB for diffusion and text encoders,
-1 GiB for VAE); higher-priority placements also leave staging space for the
-largest weight tensor of each lower-priority offloaded component. Actual segment
-weights, compute buffers and caches must
-still fit the runner's capacity checks. Offloading weights does not guarantee
-that every resolution or frame count will fit, and auto-fit does not change a
-component to CPU computation solely because its full weights exceed VRAM.
-If a VAE decode fails, decoding retries with spatial tiling even when `--auto-fit`
-is off; supported video decoders try temporal tiling first and can then add
-spatial tiling. Spatial retries use half-size tiles along each latent dimension.
-
-## Modules
-
-| Module | Purpose | Accepted names |
-| --- | --- | --- |
-| `diffusion` | UNet, DiT, MMDiT, Flux, Wan, Qwen Image, and other diffusion models | `diffusion`, `model`, `unet`, `dit` |
-| `te` | Text encoders and conditioners | `te`, `clip`, `text`, `textencoder`, `textencoders`, `conditioner`, `cond`, `llm`, `t5`, `t5xxl` |
-| `clip_vision` | CLIP vision encoder | `clip_vision`, `clipvision`, `clip-vision`, `vision` |
-| `vae` | VAE and TAE | `vae`, `firststage`, `autoencoder`, `tae` |
-| `controlnet` | ControlNet | `controlnet`, `control` |
-| `photomaker` | PhotoMaker ID encoder and PhotoMaker LoRA | `photomaker`, `photomakerid`, `pmid`, `photo` |
-| `upscaler` | ESRGAN upscaler | `upscaler`, `esrgan`, `hires` |
-| `detector` | ADetailer YOLOv8 detector | `detector`, `adetailer`, `yolo` |
-
-`te` is the preferred module name for text encoders. `clip` is kept as an accepted alias because many existing commands and model names use CLIP terminology.
-
-## Backend names
-
-Backend names are resolved against the GGML backend device list. Matching is case-insensitive and accepts exact names or unique prefixes, so common values include names such as:
-
-- `cpu`
-- `cuda0`
-- `vulkan0`
-- `metal`
-
-The special values `auto`, `default`, and an empty backend name select the default backend. The default preference is GPU, then integrated GPU, then CPU.
-
-The special value `gpu` selects the first GPU backend, falling back to the first integrated GPU backend.
-
-The special value `disk` is accepted only by `--params-backend`. `--backend disk` is invalid because `disk` is a parameter residency mode, not a runtime compute backend.
-
-## Runtime backend vs. parameter backend
-
-The runtime backend controls where graph execution runs. The parameter backend controls where model weights are allocated or whether they are reloaded from disk on demand.
-
-For example:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend cuda0 --params-backend cpu
-```
-
-This runs all modules on `cuda0`, but stores parameters in CPU RAM. During execution, parameters are moved to the runtime backend as needed.
-
-For example:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend cuda0 --params-backend disk
-```
-
-This runs all modules on `cuda0`, reloads parameters from the model file as needed, and releases those parameter buffers after use.
-
-Outside `--auto-fit`, `disk` is never selected implicitly. If `--params-backend` is not set, parameters use the runtime backend.
-
-Per-module assignments can be mixed:
-
-```shell
-sd-cli -m model.safetensors -p "a cat" --backend diffusion=cuda0,te=cpu,vae=cpu --params-backend diffusion=cuda0,te=cpu,vae=cpu
-```
-
-This keeps text encoding and VAE execution on CPU while the diffusion model runs on GPU.
-
-## Backend sharing and lifetime
-
-Backends are managed by `SDBackendManager`.
-
-Within one manager, backend instances are cached by resolved backend device name. If multiple modules request the same backend, they share the same `ggml_backend_t`.
-
-For example:
-
-```shell
---backend te=cpu,vae=cpu
-```
-
-uses one shared CPU backend for both `te` and `vae` runtime execution.
-
-Runtime and parameter assignments also share the same backend cache. If `--backend diffusion=cuda0` and `--params-backend diffusion=cuda0` resolve to the same device, both use the same backend instance.
-
-`--params-backend disk` does not create a separate backend instance. Parameters are loaded lazily using the module runtime backend.
-
-`SDBackendManager` owns the backend instances and frees them when the context or upscaler is destroyed. Model runners receive non-owning runtime and parameter backend pointers and do not free them.
-
-## Compatibility flags
-
-The example CLI/server still accepts these older CPU placement flags as compatibility aliases:
-
-- `--clip-on-cpu`
-- `--vae-on-cpu`
-- `--control-net-cpu`
-- `--offload-to-cpu`
-
-`--clip-on-cpu`, `--vae-on-cpu`, and `--control-net-cpu` are deprecated. The example argument layer prepends `te=cpu`, `vae=cpu`, and `controlnet=cpu` to `--backend` before creating the context.
-
-`--offload-to-cpu` prepends a CPU default to the parameter assignment in the caller before creating the context:
-
-```shell
---params-backend '*=cpu'
-```
-
-Because this default is inserted first, later explicit `--params-backend` entries can still override it, for example `--offload-to-cpu --params-backend te=disk` keeps non-TE parameters on CPU and reloads TE parameters from disk.
-
-Library callers should set `backend` and `params_backend` directly. `sd_ctx_params_init()`
-enables `auto_fit` by default; a nonempty `params_backend` assignment disables it.
-The `backend` assignment constrains auto-fit's compute placement.
-The old CPU/offload fields are no longer part of the C API. Explicit `--backend` and
-`--params-backend` assignments are preferred for new commands.

docs/boogu_image.md

@@ -1,31 +0,0 @@
-# How to Use
-
-Boogu Image uses a Boogu diffusion transformer, the FLUX VAE, and Qwen3-VL as the LLM text and vision encoder.
-
-## Download weights
-
-- Download Boogu Image
-    - safetensors: https://huggingface.co/Comfy-Org/Boogu-Image/tree/main/diffusion_models
-- Download vae
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/ae.safetensors
-- Download Qwen3-VL 8B
-    - gguf: https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-GGUF/tree/main
-        - For image editing with GGUF text encoders, also download the matching mmproj file and pass it with `--llm_vision`.
-
-## Examples
-
-### Boogu Image Base
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\boogu_image_base_bf16.safetensors --llm ..\models\text_encoders\Qwen3VL-8B-Instruct-Q4_K_M.gguf --vae ..\models\vae\ae.sft -p "a lovely cat" --diffusion-fa -v --offload-to-cpu
-```
-
-<img width="256" alt="Boogu Image Base example" src="../assets/boogu/example.png" />
-
-### Boogu Image Edit
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\boogu_image_edit_bf16.safetensors --llm ..\models\text_encoders\Qwen3VL-8B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\mmproj-Qwen3VL-8B-Instruct-F16.gguf --vae ..\models\vae\ae.sft --diffusion-fa -v --offload-to-cpu -r ..\assets\flux\flux1-dev-q8_0.png -p "change 'flux.cpp' to 'boogu.cpp'"
-```
-
-<img width="256" alt="Boogu Image Edit example" src="../assets/boogu/edit_example.png" />

docs/build.md

@@ -16,60 +16,6 @@ git submodule init
 git submodule update
 ```
 
-## Selecting a GGML source tree
-
-By default, sd.cpp builds the patched GGML submodule in `ggml/`. To build with
-an upstream GGML checkout instead, enable `SD_USE_UPSTREAM_GGML` and set
-`SD_GGML_SOURCE_DIR`:
-
-```shell
-cmake -S . -B build-upstream -DSD_USE_UPSTREAM_GGML=ON -DSD_GGML_SOURCE_DIR=../ggml-upstream
-cmake --build build-upstream --config Release
-```
-
-The selected source tree supplies both the library and its private headers.
-Backend options such as `-DSD_CUDA=ON` apply to the selected tree as usual.
-
-`SD_USE_UPSTREAM_GGML` defaults to `OFF`, which enables the patched GGML
-extensions. Set it to `ON` when using upstream GGML; it selects the compatibility
-mode and does not download or replace the GGML source tree. Upstream mode keeps
-the original FP8 safetensors handling: FP8 tensors are converted to F16 at load
-time (one byte per element in the file, two in RAM and VRAM). INT8
-tensorwise/convrot is disabled and its model files are rejected with an explicit
-error. FP8 GGUF files, FP8 weight type requests and tensor type rules are also
-rejected; no automatic conversion is performed.
-
-Upstream GGML may lack some operators and performance optimizations provided by
-the patched version. A warning is emitted during CMake configuration and when
-creating an inference context. Ordinary floating-point and shared GGML
-quantization types remain available, subject to backend operator support.
-
-`SD_USE_SYSTEM_GGML=ON` instead links an installed GGML CMake package, located
-with `ggml_DIR` or `CMAKE_PREFIX_PATH`. In that mode, `SD_GGML_SOURCE_DIR` must
-point to the matching source tree for private headers. The installed library
-must use the same ABI settings as sd.cpp, including `GGML_MAX_NAME`.
-Set `SD_USE_UPSTREAM_GGML=ON` as well if the installed package is upstream GGML.
-
-## WebP and WebM Support in Examples
-
-The example applications (`examples/cli` and `examples/server`) use `libwebp` to support WebP image I/O, and `examples/cli` can also use `libwebm` for `.webm` video output. Both are enabled by default. WebM output currently reuses `libwebp` to encode each frame as VP8 before muxing with `libwebm`.
-
-If you do not want WebP/WebM support, you can disable them at configure time:
-
-```shell
-mkdir build && cd build
-cmake .. -DSD_WEBP=OFF -DSD_WEBM=OFF
-cmake --build . --config Release
-```
-
-If the submodules are not available, you can also link against system packages instead:
-
-```shell
-mkdir build && cd build
-cmake .. -DSD_USE_SYSTEM_WEBP=ON -DSD_USE_SYSTEM_WEBM=ON
-cmake --build . --config Release
-```
-
 ## Build (CPU only)
 
 If you don't have a GPU or CUDA installed, you can build a CPU-only version.
@@ -90,10 +36,6 @@ cmake --build . --config Release
 
 ## Build with CUDA
 
-Native SageAttention is included when using CUDA with patched GGML
-(`SD_USE_UPSTREAM_GGML=OFF`).
-See [SageAttention](sage_attention.md) for GPU requirements and `--sage-attn` usage.
-
 This provides GPU acceleration using NVIDIA GPU. Make sure to have the CUDA toolkit installed. You can download it from your Linux distro's package manager (e.g. `apt install nvidia-cuda-toolkit`) or from here: [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads). Recommended to have at least 4 GB of VRAM.
 
 ```shell
@@ -140,11 +82,6 @@ cmake --build . --config Release
 ## Build with Vulkan
 
 Install Vulkan SDK from https://www.lunarg.com/vulkan-sdk/.
-On Ubuntu, install the Vulkan development packages and SPIR-V headers:
-
-```shell
-sudo apt-get install build-essential libvulkan-dev glslc spirv-headers
-```
 
 ```shell
 mkdir build && cd build

docs/caching.md

@@ -2,16 +2,6 @@
 
 Caching methods accelerate diffusion inference by reusing intermediate computations when changes between steps are small.
 
-### Conditioning Cache
-
-Conditioning results are cached per model context using an LRU cache. The default
-capacity is **0 (disabled) for `sd-cli`** and **4 entries for `sd-server` and the C
-API**. Set `--conditioning-cache-size N` to change the limit; `0` disables caching.
-For example, `sd-cli -m model.safetensors -p "a cat" --conditioning-cache-size 4`
-enables the cache in the CLI. The C API option is
-`sd_ctx_params_t::conditioning_cache_size`, initialized by `sd_ctx_params_init()`.
-This cache is independent of the diffusion-step `--cache-mode` options below.
-
 ### Cache Modes
 
 | Mode | Target | Description |
@@ -21,7 +11,6 @@ This cache is independent of the diffusion-step `--cache-mode` options below.
 | `dbcache` | DiT models | Block-level L1 residual threshold |
 | `taylorseer` | DiT models | Taylor series approximation |
 | `cache-dit` | DiT models | Combined DBCache + TaylorSeer |
-| `spectrum` | UNET and DiT models | Chebyshev + Taylor output forecasting |
 
 ### UCache (UNET Models)
 
@@ -90,7 +79,7 @@ Uses Taylor series approximation to predict block outputs:
 Combines DBCache and TaylorSeer:
 
 ```bash
---cache-mode cache-dit
+--cache-mode cache-dit --cache-preset fast
 ```
 
 #### Parameters
@@ -102,6 +91,14 @@ Combines DBCache and TaylorSeer:
 | `threshold` | L1 residual difference threshold | 0.08 |
 | `warmup` | Steps before caching starts | 8 |
 
+#### Presets
+
+Available presets: `slow`, `medium`, `fast`, `ultra` (or `s`, `m`, `f`, `u`).
+
+```bash
+--cache-mode cache-dit --cache-preset fast
+```
+
 #### SCM Options
 
 Steps Computation Mask controls which steps can be cached:
@@ -121,26 +118,6 @@ Mask values: `1` = compute, `0` = can cache.
 --scm-policy dynamic
 ```
 
-### Spectrum (UNET and DiT Models)
-
-Spectrum uses Chebyshev polynomial fitting blended with Taylor extrapolation to predict denoised outputs, skipping entire forward passes. Based on the paper [Spectrum: Adaptive Spectral Feature Forecasting for Efficient Diffusion Sampling](https://github.com/tingyu215/Spectrum).
-
-```bash
-sd-cli -m model.safetensors -p "a cat" --cache-mode spectrum
-```
-
-#### Parameters
-
-| Parameter | Description | Default |
-|-----------|-------------|---------|
-| `w` | Chebyshev vs Taylor blend weight (0=Taylor, 1=Chebyshev) | 0.40 |
-| `m` | Chebyshev polynomial degree | 3 |
-| `lam` | Ridge regression regularization | 1.0 |
-| `window` | Initial window size (compute every N steps) | 2 |
-| `flex` | Window growth per computed step after warmup | 0.50 |
-| `warmup` | Steps to always compute before caching starts | 4 |
-| `stop` | Stop caching at this fraction of total steps | 0.9 |
-
 ### Performance Tips
 
 - Start with default thresholds and adjust based on output quality

docs/chroma.md

@@ -15,7 +15,7 @@ You can run Chroma using stable-diffusion.cpp with a GPU that has 6GB or even 4G
 You can download the preconverted gguf weights from [silveroxides/Chroma-GGUF](https://huggingface.co/silveroxides/Chroma-GGUF), this way you don't have to do the conversion yourself.
 
 ```
-.\bin\Release\sd-cli.exe -M convert -m ..\models\unet\chroma-unlocked-v40.safetensors -o ..\models\chroma-unlocked-v40-q8_0.gguf -v --type q8_0
+.\bin\Release\sd-cli.exe -M convert -m ..\..\ComfyUI\models\unet\chroma-unlocked-v40.safetensors -o ..\models\chroma-unlocked-v40-q8_0.gguf -v --type q8_0
 ```
 
 ## Run
@@ -24,9 +24,10 @@ You can download the preconverted gguf weights from [silveroxides/Chroma-GGUF](h
 For example:
 
 ```
- .\bin\Release\sd-cli.exe --diffusion-model  ..\models\chroma-unlocked-v40-q8_0.gguf --vae ..\models\ae.sft --t5xxl ..\models\t5xxl_fp16.safetensors  -p "a lovely cat holding a sign says 'chroma.cpp'" --cfg-scale 4.0 --sampling-method euler -v --model-args chroma_use_dit_mask=false --clip-on-cpu
+ .\bin\Release\sd-cli.exe --diffusion-model  ..\models\chroma-unlocked-v40-q8_0.gguf --vae ..\models\ae.sft --t5xxl ..\models\t5xxl_fp16.safetensors  -p "a lovely cat holding a sign says 'chroma.cpp'" --cfg-scale 4.0 --sampling-method euler -v --chroma-disable-dit-mask --clip-on-cpu
 ```
 
 ![](../assets/flux/chroma_v40.png)
 
 
+

docs/chroma_radiance.md

@@ -12,7 +12,7 @@
 ## Examples
 
 ```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\Chroma1-Radiance-v0.4-Q8_0.gguf --t5xxl ..\models\clip\t5xxl_fp16.safetensors  -p "a lovely cat holding a sign says 'chroma  radiance cpp'" --cfg-scale 4.0 --sampling-method euler -v
+.\bin\Release\sd-cli.exe --diffusion-model  ..\..\ComfyUI\models\diffusion_models\Chroma1-Radiance-v0.4-Q8_0.gguf --t5xxl ..\..\ComfyUI\models\clip\t5xxl_fp16.safetensors  -p "a lovely cat holding a sign says 'chroma  radiance cpp'" --cfg-scale 4.0 --sampling-method euler -v
 ```
 
 <img alt="Chroma1-Radiance" src="../assets/flux/chroma1-radiance.png" />

docs/distilled_sd.md

@@ -1,8 +1,8 @@
-# Running distilled models: SSD1B, Vega and SDx.x with tiny U-Nets
+# Running distilled models: SSD1B and SDx.x with tiny U-Nets
 
 ## Preface 
 
-These models feature a reduced U-Net architecture. Unlike standard SDXL models, the SSD-1B and Vega U-Net contains only one middle block and fewer attention layers in its up- and down-blocks, resulting in significantly smaller file sizes. Using these models can reduce inference time by more than 33%. For more details, refer to Segmind's paper: https://arxiv.org/abs/2401.02677v1.
+These models feature a reduced U-Net architecture. Unlike standard SDXL models, the SSD-1B U-Net contains only one middle block and fewer attention layers in its up- and down-blocks, resulting in significantly smaller file sizes. Using these models can reduce inference time by more than 33%. For more details, refer to Segmind's paper: https://arxiv.org/abs/2401.02677v1.
 Similarly, SD1.x- and SD2.x-style models with a tiny U-Net consist of only 6 U-Net blocks, leading to very small files and time savings of up to 50%. For more information, see the paper: https://arxiv.org/pdf/2305.15798.pdf.
 
 ## SSD1B
@@ -17,17 +17,7 @@ Useful LoRAs are also available:
  * https://huggingface.co/seungminh/lora-swarovski-SSD-1B/resolve/main/pytorch_lora_weights.safetensors
  * https://huggingface.co/kylielee505/mylcmlorassd/resolve/main/pytorch_lora_weights.safetensors
 
-## Vega
-
-Segmind's Vega model is available online here:
-
- * https://huggingface.co/segmind/Segmind-Vega/resolve/main/segmind-vega.safetensors
- 
-VegaRT is an example for an LCM-LoRA:
-
- * https://huggingface.co/segmind/Segmind-VegaRT/resolve/main/pytorch_lora_weights.safetensors
-
-Both files can be used out-of-the-box, unlike the models described in next sections.
+These files can be used out-of-the-box, unlike the models described in the next section.
 
 
 ## SD1.x, SD2.x with tiny U-Nets
@@ -87,32 +77,23 @@ pipe.save_pretrained("segmindtiny-sd", safe_serialization=True)
 ```bash
 python convert_diffusers_to_original_stable_diffusion.py \
       --model_path  ./segmindtiny-sd \
-      --checkpoint_path ./segmind_tiny-sd.safetensors  --half --use_safetensors
+      --checkpoint_path ./segmind_tiny-sd.ckpt --half
 ```
 
-The file segmind_tiny-sd.safetensors will be generated and is now ready for use with sd.cpp. You can follow a similar process for the other models mentioned above.
+The file segmind_tiny-sd.ckpt will be generated and is now ready for use with sd.cpp. You can follow a similar process for the other models mentioned above.
 
 
-### SDXS-512-DreamShaper
+### Another available .ckpt file:
 
-Another very tiny and **incredibly fast**  model is SDXS by IDKiro et al.  The authors refer to it as *"Real-Time One-Step Latent Diffusion Models with Image Conditions"*. For details read the paper: https://arxiv.org/pdf/2403.16627 . Once again the authors removed some more blocks of U-Net part and unlike other SD1 models they use an adjusted _AutoEncoderTiny_ instead of default _AutoEncoderKL_ for the VAE part.
-##### Some ready-to-run SDXS-512 model files are available online, such as:
+ * https://huggingface.co/ClashSAN/small-sd/resolve/main/tinySDdistilled.ckpt
 
-* https://huggingface.co/akleine/sdxs-512
-* https://huggingface.co/concedo/sdxs-512-tinySDdistilled-GGUF
+To use this file, you must first adjust its non-contiguous tensors:
 
-##### Run the model as follows:
-```bash
-~/stable-diffusion.cpp/build/bin/sd-cli -m sdxs.safetensors -p "portrait of a lovely cat" \
-  --cfg-scale 1 --steps 1
+```python
+import torch
+ckpt = torch.load("tinySDdistilled.ckpt", map_location=torch.device('cpu'))
+for key, value in ckpt['state_dict'].items():
+    if isinstance(value, torch.Tensor):
+        ckpt['state_dict'][key] = value.contiguous()
+torch.save(ckpt, "tinySDdistilled_fixed.ckpt")
 ```
-Both options: ``` --cfg-scale 1 ``` and  ``` --steps 1 ``` are mandatory here.
-
-### SDXS-512-0.9
-
-Even though the name "SDXS-512-0.9" is similar to "SDXS-512-DreamShaper", it is *completely different* but also **incredibly fast**. Sometimes it is preferred, so try it yourself.
-##### Download a ready-to-run file from here:
-
-* https://huggingface.co/akleine/sdxs-09
-
-For the use of this model, both options ``` --cfg-scale 1 ``` and ``` --steps 1 ``` are again absolutely necessary.

docs/docker.md

@@ -19,7 +19,7 @@ docker run --rm --init -v /path/to/models:/models -v /path/to/output/:/output -p
 ## Building using Docker
 
 ```shell
-docker build -f docker/Dockerfile -t sd .
+docker build -t sd .
 ```
 
 ## Building variants using Docker
@@ -27,7 +27,7 @@ docker build -f docker/Dockerfile -t sd .
 Vulkan:
 
 ```shell
-docker build -f docker/Dockerfile.vulkan -t sd .
+docker build -f Dockerfile.vulkan -t sd .
 ```
 
 ## Run locally built image's CLI

docs/edit.md

@@ -1,100 +0,0 @@
-# Image Editing
-
-Image editing in `stable-diffusion.cpp` allows you to use reference images to guide the generation process, enabling tasks like identity preservation, style transfer, or layout modification.
-
-
-## Supported Models
-
-Depending on the architecture, different models handle reference images differently.
-
-| Model | Default Preset |
-| :--- | :--- |
-| [**FLUX.1-Kontext-dev**](./kontext.md) | `flux_kontext` |
-| [**LongCat Image Edit**](./longcat_image.md) | `longcat` |
-| [**Qwen Image Edit**](./qwen_image_edit.md) | `qwen` |
-| **Qwen Image LAYERED** | `qwen_layered` |
-| [**Flux.2 [Dev] / Flux.2 [Klein]**](./flux2.md) | `flux2` |
-| [**Boogu Image Edit**](./boogu_image.md) | `z_image_omni` |
-| **Krea2 (Community Edit LoRAs)** | `krea2_ostris_edit` |
-| [**Mage-Flow-Edit**](./mage_flow.md#image-editing) | `mage_flow` |
-| [**LLaDA-Image**](./llada_image.md#image-editing) | `llada_image` |
-| **Anima (Community Edit LoRAs)** | `cosmos_reference` |
-
-Stable-diffusion.spp also supports basic Unet-based editing models like instruct-pix2pix or CosXL-Edit. This document is not about those.
-
----
-
-## Configuring Reference Modes (`--ref-image-args`)
-
-For a one-time input transform before reference presets and model processing,
-including cropping, padding, and resizing algorithms, see [Image preprocessing](./image_preprocessing.md).
-
-Different DiT-based editing models require different configurations to process reference images correctly (e.g., whether to use a Vision Language Model (VLM) encoder or pass VAE-encoded images directly to the DiT).
-
-To simplify this, we provide **Presets**. By default, the system automatically selects the best preset based on the model architecture. However, you can override this using the `--ref-image-args` argument.
-
-### Usage
-The `--ref-image-args` argument accepts a comma-separated list of key-value pairs:
-
-**Using a preset:**
-`--ref-image-args "preset=qwen_layered"`
-
-**Using a preset with a specific override:**
-`--ref-image-args "preset=krea2_edit,force_ref_timestep_zero=true"`
-
-### Available Presets
-
-| Preset | Primary Use Case |
-| :--- | :--- |
-| `flux_kontext` | FLUX.1 Kontext |
-| `longcat` | LongCat Image Edit |
-| `flux2` | FLUX.2 models |
-| `qwen` | Qwen Image Edit |
-| `qwen_layered` | Qwen Image Layered |
-| `z_image_omni` | Boogu, Z-Image Omni |
-| `krea2_ostris_edit` | Most Krea2 Community edit LoRAs (trained with Ostris script) |
-| `mage_flow` | Mage-Flow-Edit |
-| `krea2_edit` | Specifically for [lbouaraba/krea2edit](https://huggingface.co/conradlocke/krea2-identity-edit). (or similar) |
-| `cosmos_reference` | For Anima |
-| `default` | Uses the automatic detection based on model architecture. |
-
----
-
-## Advanced Parameter Reference
-
-If presets are insufficient, you can manually configure the following parameters via `--ref-image-args`:
-
-| Key | Type | Description | Allowed Values |
-| :--- | :--- | :--- | :--- |
-| `preset` | string | Overrides the automatic preset. | (See the Presets table above) |
-| `pass_to_vlm` | bool | Whether reference images are passed to the VLM encoder. | `true`, `false` |
-| `pass_to_dit` | bool | Whether VAE-encoded references are passed directly to the DiT. | `true`, `false` |
-| `ref_index_mode` | string | Behavior of the RoPE index. | `fixed`, `increase`, `decrease` |
-| `force_ref_timestep_zero` | bool | Forces timestep=0 for reference tokens. | `true`, `false` (Krea2 only) |
-| `resize_before_vae` | bool | Whether reference images are resized before VAE encoding. | `true`, `false` |
-| `vae_input_max_pixels` | int | Maximum pixel area for VAE reference inputs. | Integer |
-| `vlm_resize_mode` | string | How to resize VLM reference inputs. | `longest_side`, `area`, `none` |
-| `vlm_max_size` | int | Maximum VLM input size; interpreted according to `vlm_resize_mode`. | Integer |
-| `vlm_min_size` | int | Minimum VLM input size; interpreted according to `vlm_resize_mode`. | Integer |
-| `vlm_size` | int | Shortcut to set both VLM min and max size to the same value. | Integer |
-
-### Preset Default Values
-
-For a technical overview of how each preset is configured, see the table below.
-
-| Preset | VLM | RoPE Index | Cond Resize | Special Notes |
-| :--- | :---: | :---: | :---: | :--- |
-| `flux_kontext` | No | `fixed` | `none` | |
-| `longcat` | Yes | `fixed` | `area` | |
-| `flux2` | No | `increase` | `none` | |
-| `qwen` | Yes | `increase` | `area` | |
-| `qwen_layered` | Yes | `decrease` | `area` | |
-| `mage_flow` | Yes | `increase` | `longest` | `vlm_max_size = 384`, VAE input resized to target |
-| `z_image_omni` | Yes | `fixed` | `area` | |
-| `krea2_ostris_edit`| Yes | `increase` | `area` | `force_ref_timestep_zero = true` |
-| `krea2_edit` | Yes | `increase` | `longest` | `vlm_size = 768` |
-| `cosmos_reference` | No | `fixed` | `none` | `resize_before_vae = false` |
-
-**Additional Default Notes:**
-- **VLM Input Sizes:** For most presets, `vlm_max_size` and `vlm_min_size` are set to `-1`, meaning the values are model-dependent and handled automatically. In `area` mode they represent pixel area; in `longest_side` mode they represent a side length in pixels.
-- **VAE Input Size:** `vae_input_max_pixels` defaults to $1024 \times 1024$ pixels (`1048576`).

docs/ernie_image.md

@@ -1,35 +0,0 @@
-# How to Use
-
-You can run ERNIE-Image with stable-diffusion.cpp on GPUs with 4GB of VRAM — or even less.
-
-## Download weights
-
-- Download ERNIE-Image-Turbo
-    - safetensors: https://huggingface.co/Comfy-Org/ERNIE-Image/tree/main/diffusion_models
-    - gguf: https://huggingface.co/unsloth/ERNIE-Image-Turbo-GGUF/tree/main
-- Download ERNIE-Image
-    - safetensors: https://huggingface.co/Comfy-Org/ERNIE-Image/tree/main/diffusion_models
-    - gguf: https://huggingface.co/unsloth/ERNIE-Image-GGUF/tree/main
-- Download vae
-    - safetensors: https://huggingface.co/Comfy-Org/ERNIE-Image/tree/main/vae
-- Download ministral 3b
-    - safetensors: https://huggingface.co/Comfy-Org/ERNIE-Image/tree/main/text_encoders
-    - gguf: https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-GGUF/tree/main
-
-## Examples
-
-### ERNIE-Image-Turbo
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\ernie-image-turbo.safetensors --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\ministral-3-3b.safetensors -p "a lovely cat" --cfg-scale 1.0 --steps 8 -v --offload-to-cpu --diffusion-fa
-```
-
-<img width="256" alt="ERNIE-Image Turbo example" src="../assets/ernie_image/turbo_example.png" />
-
-### ERNIE-Image
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\ernie-image-UD-Q4_K_M.gguf --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\ministral-3-3b.safetensors -p "a lovely cat" --cfg-scale 5.0 -v --offload-to-cpu --diffusion-fa
-```
-
-<img width="256" alt="ERNIE-Image example" src="../assets/ernie_image/example.png" />

docs/esrgan.md

@@ -1,8 +1,6 @@
 ## Using ESRGAN to upscale results
 
-You can use ESRGAN—such as the model [RealESRGAN_x4plus_anime_6B.pth](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth)—to upscale the generated images and improve their overall resolution and clarity.
-
-RGBA images, including Qwen Image 2.1 output, keep their alpha channel during model upscaling and hires fix. ESRGAN processes the RGB channels; the alpha channel is resized with bilinear interpolation and recombined with the upscaled image.
+You can use ESRGAN to upscale the generated images. At the moment, only the [RealESRGAN_x4plus_anime_6B.pth](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth) model is supported. Support for more models of this architecture will be added soon.
 
 - Specify the model path using the `--upscale-model PATH` parameter. example:
 

docs/flux.md

@@ -17,7 +17,7 @@ You can download the preconverted gguf weights from [FLUX.1-dev-gguf](https://hu
 
 For example:
 ```
-.\bin\Release\sd-cli.exe -M convert -m ..\models\unet\flux1-dev.sft -o ..\models\flux1-dev-q8_0.gguf -v --type q8_0
+.\bin\Release\sd-cli.exe -M convert -m ..\..\ComfyUI\models\unet\flux1-dev.sft -o ..\models\flux1-dev-q8_0.gguf -v --type q8_0
 ```
 
 ## Run

docs/flux2.md

@@ -1,96 +1,21 @@
 # How to Use
 
-## Flux.2-dev
-
-### Download weights
+## Download weights
 
 - Download FLUX.2-dev
     - gguf: https://huggingface.co/city96/FLUX.2-dev-gguf/tree/main
 - Download vae
     - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-dev/tree/main
-- Download FLUX.2-small-decoder (full_encoder_small_decoder.safetensors) as an alternative VAE option
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-small-decoder/tree/main
 - Download Mistral-Small-3.2-24B-Instruct-2506-GGUF
     - gguf: https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF/tree/main
 
-### Examples
+## Examples
 
 ```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux2-dev-Q4_K_S.gguf --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf -r .\kontext_input.png -p "change 'flux.cpp' to 'flux2-dev.cpp'" --cfg-scale 1.0 --sampling-method euler -v --diffusion-fa --offload-to-cpu
+.\bin\Release\sd-cli.exe --diffusion-model  ..\..\ComfyUI\models\diffusion_models\flux2-dev-Q4_K_S.gguf --vae ..\..\ComfyUI\models\vae\flux2_ae.safetensors  --llm ..\..\ComfyUI\models\text_encoders\Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf -r .\kontext_input.png -p "change 'flux.cpp' to 'flux2-dev.cpp'" --cfg-scale 1.0 --sampling-method euler -v --offload-to-cpu
 ```
 
 <img alt="flux2 example" src="../assets/flux2/example.png" />
 
-## Flux.2 klein 4B / Flux.2 klein base 4B
 
-### Download weights
 
-- Download FLUX.2-klein-4B
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-klein-4B
-    - gguf: https://huggingface.co/leejet/FLUX.2-klein-4B-GGUF/tree/main
-- Download FLUX.2-klein-base-4B
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B
-    - gguf: https://huggingface.co/leejet/FLUX.2-klein-base-4B-GGUF/tree/main
-- Download vae
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-dev/tree/main
-- Download FLUX.2-small-decoder (full_encoder_small_decoder.safetensors) as an alternative VAE option
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-small-decoder/tree/main
-- Download Qwen3 4b
-    - safetensors: https://huggingface.co/Comfy-Org/flux2-klein-4B/tree/main/split_files/text_encoders
-    - gguf: https://huggingface.co/unsloth/Qwen3-4B-GGUF/tree/main
-
-### Examples
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux-2-klein-4b.safetensors --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\qwen_3_4b.safetensors -p "a lovely cat" --cfg-scale 1.0 --steps 4 -v --offload-to-cpu --diffusion-fa
-```
-
-<img alt="flux2-klein-4b" src="../assets/flux2/flux2-klein-4b.png" />
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux-2-klein-4b.safetensors --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\qwen_3_4b.safetensors -r .\kontext_input.png -p "change 'flux.cpp' to 'klein.cpp'" --cfg-scale 1.0 --sampling-method euler -v --diffusion-fa --offload-to-cpu --steps 4
-```
-
-<img alt="flux2-klein-4b-edit" src="../assets/flux2/flux2-klein-4b-edit.png" />
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux-2-klein-base-4b.safetensors --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\qwen_3_4b.safetensors -p "a lovely cat" --cfg-scale 4.0 --steps 20 -v --offload-to-cpu --diffusion-fa
-```
-
-<img alt="flux2-klein-base-4b" src="../assets/flux2/flux2-klein-base-4b.png" />
-
-## Flux.2 klein 9B / Flux.2 klein base 9B
-
-### Download weights
-
-- Download FLUX.2-klein-9B
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-klein-9B
-    - gguf: https://huggingface.co/leejet/FLUX.2-klein-9B-GGUF/tree/main
-- Download FLUX.2-klein-base-9B
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B
-    - gguf: https://huggingface.co/leejet/FLUX.2-klein-base-9B-GGUF/tree/main
-- Download vae
-    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-dev/tree/main
-- Download Qwen3 8B
-    - safetensors: https://huggingface.co/Comfy-Org/flux2-klein-9B/tree/main/split_files/text_encoders
-    - gguf: https://huggingface.co/unsloth/Qwen3-8B-GGUF/tree/main
-
-### Examples
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux-2-klein-9b.safetensors --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\qwen_3_8b.safetensors -p "a lovely cat" --cfg-scale 1.0 --steps 4 -v --offload-to-cpu --diffusion-fa
-```
-
-<img alt="flux2-klein-9b" src="../assets/flux2/flux2-klein-9b.png" />
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux-2-klein-9b.safetensors --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\qwen_3_8b.safetensors -r .\kontext_input.png -p "change 'flux.cpp' to 'klein.cpp'" --cfg-scale 1.0 --sampling-method euler -v --diffusion-fa --offload-to-cpu --steps 4
-```
-
-<img alt="flux2-klein-9b-edit" src="../assets/flux2/flux2-klein-9b-edit.png" />
-
-```
-.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux-2-klein-base-9b.safetensors --vae ..\models\vae\flux2_ae.safetensors  --llm ..\models\text_encoders\qwen_3_8b.safetensors -p "a lovely cat" --cfg-scale 4.0 --steps 20 -v --offload-to-cpu --diffusion-fa
-```
-
-<img alt="flux2-klein-base-9b" src="../assets/flux2/flux2-klein-base-9b.png" />

docs/hidream_o1_image.md

@@ -1,20 +0,0 @@
-# How to Use
-
-## Download weights
-
-- Download HiDream-O1-Image-Dev
-    - safetensors: https://huggingface.co/Comfy-Org/HiDream-O1-Image/tree/main/checkpoints
-- Download HiDream-O1-Image
-    - safetensors: https://huggingface.co/Comfy-Org/HiDream-O1-Image/tree/main/checkpoints
-
-## Examples
-
-### HiDream-O1-Image-Dev
-
-```
-.\bin\Release\sd-cli.exe -m  ..\models\diffusion_models\hidream_o1_image_dev_bf16.safetensors -p "a lovely cat holding a sign says 
-'hidream o1 cpp'" --cfg-scale 1.0  -v -H 1024 -W 1024
-```
-
-<img width="256" alt="HiDream-O1-Image-Dev example" src="../assets/hidream-o1/dev_example.png" />
-

docs/hipBLAS_on_Windows.md

@@ -26,12 +26,12 @@ Fortunately, `AMD` provides complete help documentation, you can use the help do
 
 Then we must set `ROCM` as environment variables before running cmake.
 
-Usually if you install according to the official tutorial and do not modify the ROCM path, then there is a high probability that it is here `C:\Program Files\AMD\ROCm\7.1.1\bin`
+Usually if you install according to the official tutorial and do not modify the ROCM path, then there is a high probability that it is here `C:\Program Files\AMD\ROCm\5.5\bin`
 
 This is what I use to set the clang:
 ```Commandline
-set CC=C:\Program Files\AMD\ROCm\7.1.1\bin\clang.exe
-set CXX=C:\Program Files\AMD\ROCm\7.1.1\bin\clang++.exe
+set CC=C:\Program Files\AMD\ROCm\5.5\bin\clang.exe
+set CXX=C:\Program Files\AMD\ROCm\5.5\bin\clang++.exe
 ```
 
 ## Ninja
@@ -46,7 +46,7 @@ set ninja=C:\Program Files\ninja\ninja.exe
 ## Building stable-diffusion.cpp
 
 The thing different from the regular CPU build is `-DSD_HIPBLAS=ON` ,
-`-G "Ninja"`, `-DCMAKE_C_COMPILER=clang`, `-DCMAKE_CXX_COMPILER=clang++`, `-DAMDGPU_TARGETS=gfx1150;gfx1151;gfx1200;gfx1201;gfx1100;gfx1101;gfx1102;gfx1030;gfx1031;gfx1032`
+`-G "Ninja"`, `-DCMAKE_C_COMPILER=clang`, `-DCMAKE_CXX_COMPILER=clang++`, `-DAMDGPU_TARGETS=gfx1100`
 
 >**Notice**: check the `clang` and `clang++` information:
 ```Commandline
@@ -59,29 +59,26 @@ If you see like this, we can continue:
 clang version 17.0.0 (git@github.amd.com:Compute-Mirrors/llvm-project e3201662d21c48894f2156d302276eb1cf47c7be)
 Target: x86_64-pc-windows-msvc
 Thread model: posix
-InstalledDir: C:\Program Files\AMD\ROCm\7.1.1\bin
+InstalledDir: C:\Program Files\AMD\ROCm\5.5\bin
 ```
 
 ```
 clang version 17.0.0 (git@github.amd.com:Compute-Mirrors/llvm-project e3201662d21c48894f2156d302276eb1cf47c7be)
 Target: x86_64-pc-windows-msvc
 Thread model: posix
-InstalledDir: C:\Program Files\AMD\ROCm\7.1.1\bin
+InstalledDir: C:\Program Files\AMD\ROCm\5.5\bin
 ```
 
->**Notice** that the GPU targets are now compatible with multiple GPU architectures (ROCm 7.1.1 targets). You can change them to match your GPU architecture. Click here to see your architecture [LLVM Target](https://rocm.docs.amd.com/en/latest/release/windows_support.html#windows-supported-gpus)
+>**Notice** that the `gfx1100` is the GPU architecture of my GPU, you can change it to your GPU architecture. Click here to see your architecture [LLVM Target](https://rocm.docs.amd.com/en/latest/release/windows_support.html#windows-supported-gpus)
 
-Examples:
-- AMD Radeon™ RX 7900 XTX Graphics: `gfx1100`
-- AMD Radeon™ RX 7900 XT Graphics: `gfx1101`
-- AMD Radeon™ RX 7900 GRE Graphics: `gfx1102`
+My GPU is AMD Radeon™ RX 7900 XTX Graphics, so I set it to `gfx1100`.
 
 option:
 
 ```commandline
 mkdir build
 cd build
-cmake .. -G "Ninja" -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DSD_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS="gfx1150;gfx1151;gfx1200;gfx1201;gfx1100;gfx1101;gfx1102;gfx1030;gfx1031;gfx1032"
+cmake .. -G "Ninja" -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DSD_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS=gfx1100
 cmake --build . --config Release
 ```
 

docs/hunyuan_video.md

@@ -1,24 +0,0 @@
-# HunyuanVideo 1.5
-
-HunyuanVideo 1.5 uses a HunyuanVideo diffusion transformer, a causal video VAE, Qwen2.5-VL 7B for the main text conditioning, 
-and ByT5 Small GlyphXL for glyph-aware text conditioning.
-
-## Download weights
-
-- Download HunyuanVideo 1.5
-    - safetensors: https://huggingface.co/Comfy-Org/HunyuanVideo_1.5_repackaged/tree/main/split_files/diffusion_models
-- Download vae
-    - safetensors: https://huggingface.co/Comfy-Org/HunyuanVideo_1.5_repackaged/tree/main/split_files/vae
-- Download qwen_2.5_vl 7b
-    - safetensors: https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/tree/main/split_files/text_encoders
-    - gguf: https://huggingface.co/mradermacher/Qwen2.5-VL-7B-Instruct-GGUF/tree/main
-- Download byt5 small glyphxl
-    - safetensros: https://huggingface.co/Comfy-Org/HunyuanVideo_1.5_repackaged/tree/main/split_files/text_encoders
-
-## Text-to-video example
-
-```shell
-.\bin\Release\sd-cli.exe -M vid_gen --diffusion-model  ..\models\diffusion_models\hunyuanvideo1.5_720p_t2v_fp16.safetensors --vae ..\models\vae\hunyuanvideo15_vae_fp16.safetensors --llm ..\models\text_encoders\qwen_2.5_vl_7b.safetensors --t5xxl ..\models\text_encoders\byt5_small_glyphxl_fp16.safetensors  -p "a lovely cat" --cfg-scale 6.0 --sampling-method euler -v -W 1280 -H 720 --offload-to-cpu --diffusion-fa --video-frames 33 --vae-tiling
-```
-
-<video src=../assets/hunyuan_video/hy1.5_t2v.mp4 controls="controls" muted="muted" type="video/mp4"></video>