ci: align MUSA Docker image with upstream SDK 5.2.0 and mp_31

feat: add moss-image support (#2148 )
chore: sync ggml and remove obsolete conv1d workarounds (#2147 )
2026-10-18 05:50:10 +08:00 · 2026-10-17 14:14:29 +08:00 · 2026-10-17 03:38:38 +08:00 · 2026-10-17 02:39:51 +08:00 · 2026-10-17 00:50:42 +08:00 · 2026-10-17 00:50:09 +08:00
44 changed files with 1083 additions and 167 deletions
@@ -15,6 +15,7 @@ API and command-line option may change frequently.***

 ## 🔥Important News

+* **2026/10/16** 🚀 stable-diffusion.cpp adds **Day-0 support for FLUX.3 Image Klein / Klein Base**
 * **2026/09/20** 🚀 stable-diffusion.cpp adds **Day-0 support for Qwen-Image-2.1**
 * **2026/08/20** 🚀 stable-diffusion.cpp now supports **LTX-2.5**
 * **2026/08/04** 🚀 stable-diffusion.cpp adds **Day-1 support for MiniMax-H3**
@@ -44,6 +45,8 @@ API and command-line option may change frequently.***
    - [SD3/SD3.5](./docs/sd3.md)
    - [FLUX.1-dev/FLUX.1-schnell](./docs/flux.md)
    - [FLUX.2-dev/FLUX.2-klein](./docs/flux2.md)
+    - [FLUX.3 Image Klein / Klein Base](./docs/flux3.md)
+    - [MOSS-Image-Base / MOSS-Image-SFT](./docs/moss_image.md)
    - [Lens](./docs/lens.md)
    - [Chroma](./docs/chroma.md)
    - [Chroma1-Radiance](./docs/chroma_radiance.md)
@@ -1,7 +1,6 @@
-ARG MUSA_VERSION=rc4.2.0
-ARG UBUNTU_VERSION=22.04
+ARG MUSA_IMAGE=registry.mthreads.com/mcconline/inference/pytorch:2.9.1.post1-py3.10-musa5.2.0-mp31-devel-ubuntu22.04-amd64

-FROM mthreads/musa:${MUSA_VERSION}-devel-ubuntu${UBUNTU_VERSION}-amd64 as build
+FROM ${MUSA_IMAGE} AS build

 # sd-server embeds the web UI at build time, so the build image needs Node/pnpm.
 RUN apt-get update && apt-get install -y --no-install-recommends ccache cmake git ca-certificates curl gnupg && \
@@ -22,9 +21,8 @@ COPY . .

 RUN mkdir build && cd build && \
    cmake .. -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
-        -DCMAKE_C_FLAGS="${CMAKE_C_FLAGS} -fopenmp -I/usr/lib/llvm-14/lib/clang/14.0.0/include -L/usr/lib/llvm-14/lib" \
-        -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -fopenmp -I/usr/lib/llvm-14/lib/clang/14.0.0/include -L/usr/lib/llvm-14/lib" \
        -DSD_MUSA=ON \
+        -DMUSA_ARCHITECTURES=31 \
        -DSD_BUILD_SHARED_LIBS=ON \
        -DGGML_NATIVE=OFF \
        -DSD_BUILD_SHARED_GGML_LIB=ON \
@@ -35,7 +33,7 @@ RUN mkdir build && cd build && \
        -DCMAKE_BUILD_TYPE=Release && \
    cmake --build . --config Release

-FROM mthreads/musa:${MUSA_VERSION}-runtime-ubuntu${UBUNTU_VERSION}-amd64 as runtime
+FROM ${MUSA_IMAGE} AS runtime

 COPY --from=build /sd.cpp/build/bin /sd.cpp/bin
 RUN printf '#!/bin/sh\nexec /sd.cpp/bin/sd-cli "$@"\n' > /sd-cli && \
@@ -45,3 +43,4 @@ RUN printf '#!/bin/sh\nexec /sd.cpp/bin/sd-cli "$@"\n' > /sd-cli && \
 WORKDIR /sd.cpp

 ENTRYPOINT [ "/sd-cli" ]
+CMD []
@@ -119,14 +119,16 @@ cmake --build . --config Release

 ## Build with MUSA

-This provides GPU acceleration using Moore Threads GPU. Make sure to have the MUSA toolkit installed.
+This provides GPU acceleration using Moore Threads GPUs. The configuration below follows upstream validation with MUSA SDK 5.2.0 and MTT S5000 (PH1, `mp_31`). Install that toolkit before building. Older SDKs such as 4.2 may lack the CUB headers required by the current backend.

 ```shell
 mkdir build && cd build
-cmake .. -DCMAKE_C_COMPILER=/usr/local/musa/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/musa/bin/clang++ -DSD_MUSA=ON -DCMAKE_BUILD_TYPE=Release
+cmake .. -DCMAKE_C_COMPILER=/usr/local/musa/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/musa/bin/clang++ -DSD_MUSA=ON -DMUSA_ARCHITECTURES=31 -DCMAKE_BUILD_TYPE=Release
 cmake --build . --config Release
 ```

+The MUSA Docker image uses the same SDK 5.2.0 development image as upstream for both build and runtime stages, and targets `mp_31` only. It includes the development tools and PyTorch from that base image, so it is larger than a minimal runtime image. Other GPU architectures require separate validation.
+
 ## Build with Metal

 Using Metal makes the computation run on the GPU. Currently, there are some issues with Metal when performing operations on very large matrices, making it highly inefficient at the moment. Performance improvements are expected in the near future.
@@ -14,6 +14,7 @@ Depending on the architecture, different models handle reference images differen
 | [**Qwen Image Edit**](./qwen_image_edit.md) | `qwen` |
 | **Qwen Image LAYERED** | `qwen_layered` |
 | [**Flux.2 [Dev] / Flux.2 [Klein]**](./flux2.md) | `flux2` |
+| [**FLUX.3 Image Klein / Klein Base**](./flux3.md) | `flux2` |
 | [**Boogu Image Edit**](./boogu_image.md) | `z_image_omni` |
 | **Krea2 (Community Edit LoRAs)** | `krea2_ostris_edit` |
 | [**Mage-Flow-Edit**](./mage_flow.md#image-editing) | `mage_flow` |
@@ -0,0 +1,48 @@
+# How to Use
+
+FLUX.3 Image Klein uses a FLUX.3 diffusion transformer, the FLUX.2 VAE, and Qwen3-VL 4B as the LLM text encoder.
+
+## Download weights
+
+- Download FLUX.3 Image Klein
+    - safetensors: https://huggingface.co/black-forest-labs/flux-3-image-klein/tree/main
+- Download FLUX.3 Image Klein Base
+    - safetensors: https://huggingface.co/black-forest-labs/flux-3-image-klein-base/tree/main
+- Download vae
+    - safetensors: https://huggingface.co/black-forest-labs/FLUX.2-dev/tree/main
+    - The `ae.safetensors` included with FLUX.3 can also be used.
+- Download Qwen3-VL 4B
+    - safetensors: https://huggingface.co/Comfy-Org/Krea-2/tree/main/text_encoders
+    - gguf: https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF/tree/main
+
+Use the root `model.safetensors` for the diffusion model.
+
+## Examples
+
+### FLUX.3 Image Klein
+
+The distilled model uses 16 steps and CFG 1.0.
+
+```
+.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\flux-3-image-klein.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --vae ..\models\vae\flux2_ae.safetensors -p "a lovely cat holding a sign says 'flux3.cpp'" --cfg-scale 1.0 --steps 16 --sampling-method euler -v --offload-to-cpu
+```
+
+<img src="../assets/flux3/flux3-image-klein.png" />
+
+```
+.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\flux-3-image-klein.safetensors --vae ..\models\vae\flux2_ae.safetensors --llm  ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf -r ..\assets\flux\flux1-dev-q8_0.png -p "change 'flux.cpp' to 'flux3-image-klein.cpp'" --cfg-scale 1.0 --sampling-method euler -v  --offload-to-cpu --steps 16
+```
+
+<img src="../assets/flux3/flux3-image-klein-edit.png" />
+
+Repeat `-r` to use multiple reference images, up to 10. See [Image Editing](./edit.md) for reference-image preprocessing options.
+
+### FLUX.3 Image Klein Base
+
+```
+.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\flux-3-image-klein-base.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --vae ..\models\vae\flux2_ae.safetensors -p "a lovely cat holding a sign says 'flux3.cpp'" --cfg-scale 3.5 --steps 20 --sampling-method euler -v --offload-to-cpu
+```
+
+<img src="../assets/flux3/flux3-image-klein-base.png" />
+
+The default discrete scheduler uses the upstream time shift of 6.93 for both models. Use `--flow-shift` to override it.
@@ -0,0 +1,34 @@
+# How to Use
+
+MOSS-Image uses a 5B diffusion transformer, the FLUX.2 VAE, and Qwen3.5 9B as the LLM text encoder. Both Base and SFT are supported. Prompt enhancement is not included.
+
+## Download weights
+
+- Download MOSS-Image Base or SFT
+    - safetensors: https://huggingface.co/Comfy-Org/MOSS-Image/tree/main/diffusion_models
+- Download vae
+    - safetensors: https://huggingface.co/Comfy-Org/MOSS-Image/tree/main/vae
+- Download Qwen3.5 9B
+    - gguf: https://huggingface.co/unsloth/Qwen3.5-9B-GGUF/tree/main
+- Download `tokenizer.json`
+    - https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base/tree/main/tokenizer
+
+Pass the external `tokenizer.json` with `--tokenizer`; Qwen3.5 uses a different vocabulary from the embedded Qwen tokenizer.
+
+## Examples
+
+### MOSS-Image Base
+
+```
+.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\moss_image_base_int8_convrot_attn.safetensors --llm ..\models\text_encoders\Qwen3.5-9B-Q4_K_M.gguf --vae ..\models\vae\flux2_ae.safetensors --tokenizer ..\models\tokenizers\tokenizer_qwen3.5.json --cfg-scale 3.0 -p "a lovely cat holding a sign says 'moss-image.cpp'" -v --offload-to-cpu -H 1024 -W 1024
+```
+
+### MOSS-Image SFT
+
+```
+.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\moss_image_sft_int8_convrot_attn.safetensors --llm ..\models\text_encoders\Qwen3.5-9B-Q4_K_M.gguf --vae ..\models\vae\flux2_ae.safetensors --tokenizer ..\models\tokenizers\tokenizer_qwen3.5.json --cfg-scale 3.0 -p "a lovely cat holding a sign says 'moss-image.cpp'" -v --offload-to-cpu -H 1024 -W 1024
+```
+
+<img src="../assets/moss/sft.png" />
+
+Use dimensions divisible by 16. The default discrete scheduler uses the upstream time shift of 5. Use `--flow-shift` to override it. The negative CFG branch skips DiT layer 22 (zero-based), matching the upstream pipeline.
@@ -4,6 +4,7 @@
 #include <cctype>
 #include <cstdint>
 #include <cstring>
+#include <filesystem>
 #include <fstream>
 #include <iomanip>
 #include <iostream>
@@ -276,7 +277,7 @@ namespace {
    }

    bool read_file(const std::string& path, std::vector<uint8_t>& data, std::string& error) {
-        std::ifstream fin(path, std::ios::binary);
+        std::ifstream fin(std::filesystem::u8path(path), std::ios::binary);
        if (!fin) {
            error = "failed to open file: " + path;
            return false;
@@ -188,7 +188,7 @@ struct SDCliParams {
            if (++index >= argc) {
                return -1;
            }
-            imatrix_in.push_back(argv[index]);
+            imatrix_in.push_back(argv_to_utf8(index, argv));
            return 1;
        };

@@ -323,13 +323,14 @@ void sd_log_cb(enum sd_log_level_t level, const char* log, void* data) {
 bool load_images_from_dir(const std::string dir,
                          std::vector<SDImageOwner>& images,
                          int max_image_num = 0) {
-    if (!fs::exists(dir) || !fs::is_directory(dir)) {
+    const fs::path dir_path = fs::u8path(dir);
+    if (!fs::exists(dir_path) || !fs::is_directory(dir_path)) {
        LOG_ERROR("'%s' is not a valid directory\n", dir.c_str());
        return false;
    }

    std::vector<fs::directory_entry> entries;
-    for (const auto& entry : fs::directory_iterator(dir)) {
+    for (const auto& entry : fs::directory_iterator(dir_path)) {
        if (entry.is_regular_file()) {
            entries.push_back(entry);
        }
@@ -337,12 +338,12 @@ bool load_images_from_dir(const std::string dir,

    std::sort(entries.begin(), entries.end(),
              [](const fs::directory_entry& a, const fs::directory_entry& b) {
-                  return a.path().filename().string() < b.path().filename().string();
+                  return a.path().filename().u8string() < b.path().filename().u8string();
              });

    for (const auto& entry : entries) {
-        std::string path = entry.path().string();
-        std::string ext  = entry.path().extension().string();
+        std::string path = entry.path().u8string();
+        std::string ext  = entry.path().extension().u8string();
        std::transform(ext.begin(), ext.end(), ext.begin(), ::tolower);

        if (ext == ".jpg" || ext == ".jpeg" || ext == ".png" || ext == ".bmp" || ext == ".webp") {
@@ -397,19 +398,19 @@ void step_callback(int step, int frame_count, sd_image_t* image, bool is_noisy,
    // is_noisy is set to true if the preview corresponds to noisy latents, false if it's denoised latents
    // unused in this app, it will either be always noisy or always denoised here
    if (frame_count == 1) {
-        fs::path path = cli_params->preview_path;
-        if (encoded_image_format_from_path(path.string()) == EncodedImageFormat::UNKNOWN)
+        fs::path path = fs::u8path(cli_params->preview_path);
+        if (encoded_image_format_from_path(path.u8string()) == EncodedImageFormat::UNKNOWN)
            path += ".png";
-        if (std::regex_search(path.string(), format_specifier_regex))
-            path = fs::path(format_frame_idx(path.string(), continuous_preview_counter++));
-        if (!write_image_to_file(path.string(),
+        if (std::regex_search(path.u8string(), format_specifier_regex))
+            path = fs::u8path(format_frame_idx(path.u8string(), continuous_preview_counter++));
+        if (!write_image_to_file(path.u8string(),
                                 image->data,
                                 image->width,
                                 image->height,
                                 image->channel,
                                 "",
                                 cli_params->compression_quality)) {
-            LOG_ERROR("save preview image to '%s' failed", path.string().c_str());
+            LOG_ERROR("save preview image to '%s' failed", path.u8string().c_str());
        }
    } else {
        int fps = cli_params->preview_method == PREVIEW_PROJ ? cli_params->preview_fps / 4 : cli_params->preview_fps;
@@ -420,12 +421,12 @@ void step_callback(int step, int frame_count, sd_image_t* image, bool is_noisy,
 }

 static fs::path get_video_audio_sidecar_path(const SDCliParams& cli_params) {
-    fs::path out_path     = cli_params.output_path;
+    fs::path out_path     = fs::u8path(cli_params.output_path);
    fs::path base_path    = out_path;
    fs::path ext          = out_path.has_extension() ? out_path.extension() : fs::path{};
-    std::string ext_lower = ext.string();
+    std::string ext_lower = ext.u8string();
    std::transform(ext_lower.begin(), ext_lower.end(), ext_lower.begin(), ::tolower);
-    const EncodedImageFormat output_format = encoded_image_format_from_path(out_path.string());
+    const EncodedImageFormat output_format = encoded_image_format_from_path(out_path.u8string());
    if (!ext.empty()) {
        if (output_format == EncodedImageFormat::JPEG ||
            output_format == EncodedImageFormat::PNG ||
@@ -450,14 +451,14 @@ bool save_results(const SDCliParams& cli_params,
    }

    namespace fs      = std::filesystem;
-    fs::path out_path = cli_params.output_path;
+    fs::path out_path = fs::u8path(cli_params.output_path);

    if (!out_path.parent_path().empty()) {
        std::error_code ec;
        fs::create_directories(out_path.parent_path(), ec);
        if (ec) {
            LOG_ERROR("failed to create directory '%s': %s",
-                      out_path.parent_path().string().c_str(), ec.message().c_str());
+                      out_path.parent_path().u8string().c_str(), ec.message().c_str());
            return false;
        }
    }
@@ -465,9 +466,9 @@ bool save_results(const SDCliParams& cli_params,
    fs::path base_path = out_path;
    fs::path ext       = out_path.has_extension() ? out_path.extension() : fs::path{};

-    std::string ext_lower = ext.string();
+    std::string ext_lower = ext.u8string();
    std::transform(ext_lower.begin(), ext_lower.end(), ext_lower.begin(), ::tolower);
-    const EncodedImageFormat output_format = encoded_image_format_from_path(out_path.string());
+    const EncodedImageFormat output_format = encoded_image_format_from_path(out_path.u8string());
    if (!ext.empty()) {
        if (output_format == EncodedImageFormat::JPEG ||
            output_format == EncodedImageFormat::PNG ||
@@ -493,8 +494,8 @@ bool save_results(const SDCliParams& cli_params,
        std::string params          = gen_params.embed_image_metadata
                                          ? get_image_params(ctx_params, gen_params, metadata_seed, cli_params.mode)
                                          : "";
-        const bool ok               = write_image_to_file(path.string(), img.data, img.width, img.height, img.channel, params, cli_params.compression_quality);
-        LOG_INFO("save result image %d to '%s' (%s)", idx, path.string().c_str(), ok ? "success" : "failure");
+        const bool ok               = write_image_to_file(path.u8string(), img.data, img.width, img.height, img.channel, params, cli_params.compression_quality);
+        LOG_INFO("save result image %d to '%s' (%s)", idx, path.u8string().c_str(), ok ? "success" : "failure");
        return ok;
    };

@@ -502,14 +503,14 @@ bool save_results(const SDCliParams& cli_params,
        if (generated_audio == nullptr) {
            return;
        }
-        if (write_wav_to_file(wav_path.string(),
+        if (write_wav_to_file(wav_path.u8string(),
                              generated_audio->data,
                              generated_audio->sample_count,
                              generated_audio->channels,
                              generated_audio->sample_rate)) {
-            LOG_INFO("save result audio to '%s'", wav_path.string().c_str());
+            LOG_INFO("save result audio to '%s'", wav_path.u8string().c_str());
        } else {
-            LOG_WARN("failed to save result audio to '%s'", wav_path.string().c_str());
+            LOG_WARN("failed to save result audio to '%s'", wav_path.u8string().c_str());
        }
    };

@@ -522,7 +523,7 @@ bool save_results(const SDCliParams& cli_params,
        pattern += ext;

        for (int i = 0; i < num_results; ++i) {
-            fs::path img_path = format_frame_idx(pattern.string(), output_begin_idx + i);
+            fs::path img_path = fs::u8path(format_frame_idx(pattern.u8string(), output_begin_idx + i));
            if (write_image(img_path, i)) {
                sucessful_reults++;
            }
@@ -540,11 +541,11 @@ bool save_results(const SDCliParams& cli_params,

        fs::path video_path = base_path;
        video_path += ext;
-        std::string final_ext_lower = ext.string();
+        std::string final_ext_lower = ext.u8string();
        std::transform(final_ext_lower.begin(), final_ext_lower.end(), final_ext_lower.begin(), ::tolower);
        const bool mux_audio = generated_audio != nullptr && (final_ext_lower == ".avi" || final_ext_lower == ".webm");
-        if (create_video_from_sd_images(video_path.string().c_str(), results, num_results, gen_params.fps, cli_params.compression_quality, mux_audio ? generated_audio : nullptr, params) == 0) {
-            LOG_INFO("save result video to '%s'", video_path.string().c_str());
+        if (create_video_from_sd_images(video_path.u8string().c_str(), results, num_results, gen_params.fps, cli_params.compression_quality, mux_audio ? generated_audio : nullptr, params) == 0) {
+            LOG_INFO("save result video to '%s'", video_path.u8string().c_str());
            if (generated_audio != nullptr && !mux_audio) {
                fs::path wav_path = video_path;
                wav_path.replace_extension(".wav");
@@ -552,7 +553,7 @@ bool save_results(const SDCliParams& cli_params,
            }
            return true;
        } else {
-            LOG_ERROR("Failed to save result video to '%s'", video_path.string().c_str());
+            LOG_ERROR("Failed to save result video to '%s'", video_path.u8string().c_str());
            return false;
        }
    }
@@ -701,7 +702,7 @@ int main(int argc, const char* argv[]) {
    LOG_DEBUG("%s", gen_params.to_string().c_str());

    if (!cli_params.imatrix_out.empty()) {
-        if (fs::exists(cli_params.imatrix_out) &&
+        if (fs::exists(fs::u8path(cli_params.imatrix_out)) &&
            std::find(cli_params.imatrix_in.begin(), cli_params.imatrix_in.end(), cli_params.imatrix_out) == cli_params.imatrix_in.end()) {
            LOG_WARN("imatrix file '%s' already exists and will be overwritten", cli_params.imatrix_out.c_str());
        }
@@ -75,7 +75,7 @@ static std::string utf16_to_utf8(const std::wstring& wstr) {
    return utf8;
 }

-static std::string argv_to_utf8(int index, const char** argv) {
+std::string argv_to_utf8(int index, const char** argv) {
    (void)argv;
    int argc;
    wchar_t** argv_w = CommandLineToArgvW(GetCommandLineW(), &argc);
@@ -91,7 +91,7 @@ static std::string argv_to_utf8(int index, const char** argv) {
 }

 #else  // Linux / macOS
-static std::string argv_to_utf8(int index, const char** argv) {
+std::string argv_to_utf8(int index, const char** argv) {
    return std::string(argv[index]);
 }

@@ -839,16 +839,17 @@ ArgOptions SDContextParams::get_options() {
 void SDContextParams::build_embedding_map() {
    static const std::vector<std::string> valid_ext = {".gguf", ".safetensors", ".pt", ".ckpt"};

-    if (!fs::exists(embedding_dir) || !fs::is_directory(embedding_dir)) {
+    const fs::path embedding_dir_path = fs::u8path(embedding_dir);
+    if (!fs::exists(embedding_dir_path) || !fs::is_directory(embedding_dir_path)) {
        return;
    }

-    for (auto& p : fs::directory_iterator(embedding_dir)) {
+    for (auto& p : fs::directory_iterator(embedding_dir_path)) {
        if (!p.is_regular_file())
            continue;

        auto path       = p.path();
-        std::string ext = path.extension().string();
+        std::string ext = path.extension().u8string();

        bool valid = false;
        for (auto& e : valid_ext) {
@@ -860,8 +861,8 @@ void SDContextParams::build_embedding_map() {
        if (!valid)
            continue;

-        std::string key   = path.stem().string();
-        std::string value = path.string();
+        std::string key   = path.stem().u8string();
+        std::string value = path.u8string();

        embedding_map[key] = value;
    }
@@ -1572,7 +1573,7 @@ ArgOptions SDGenerationParams::get_options() {
        if (++index >= argc) {
            return -1;
        }
-        ref_image_paths.push_back(argv[index]);
+        ref_image_paths.push_back(argv_to_utf8(index, argv));
        return 1;
    };

@@ -1580,7 +1581,7 @@ ArgOptions SDGenerationParams::get_options() {
        if (++index >= argc) {
            return -1;
        }
-        ref_video_paths.push_back(argv[index]);
+        ref_video_paths.push_back(argv_to_utf8(index, argv));
        return 1;
    };

@@ -1588,7 +1589,7 @@ ArgOptions SDGenerationParams::get_options() {
        if (++index >= argc) {
            return -1;
        }
-        ref_video_audio_paths.push_back(argv[index]);
+        ref_video_audio_paths.push_back(argv_to_utf8(index, argv));
        return 1;
    };

@@ -1596,7 +1597,7 @@ ArgOptions SDGenerationParams::get_options() {
        if (++index >= argc) {
            return -1;
        }
-        ref_audio_paths.push_back(argv[index]);
+        ref_audio_paths.push_back(argv_to_utf8(index, argv));
        return 1;
    };

@@ -1604,7 +1605,7 @@ ArgOptions SDGenerationParams::get_options() {
        if (++index >= argc) {
            return -1;
        }
-        ref_audio_paths.push_back(argv[index]);
+        ref_audio_paths.push_back(argv_to_utf8(index, argv));
        return 1;
    };

@@ -1703,15 +1704,15 @@ ArgOptions SDGenerationParams::get_options() {
        if (++index >= argc) {
            return -1;
        }
-        const char* arg = argv[index];
-        std::ifstream f(arg, std::ios::binary);
+        const std::string arg = argv_to_utf8(index, argv);
+        std::ifstream f(fs::u8path(arg), std::ios::binary);
        try {
            prompt = std::string(std::istreambuf_iterator<char>{f}, {});
        } catch (const std::ios_base::failure&) {
            f.setstate(std::ios_base::failbit);
        }
        if (f.fail()) {
-            LOG_ERROR("error: failed to read prompt file '%s'\n", arg);
+            LOG_ERROR("error: failed to read prompt file '%s'\n", arg.c_str());
            return -1;
        }
        return 1;
@@ -1721,15 +1722,15 @@ ArgOptions SDGenerationParams::get_options() {
        if (++index >= argc) {
            return -1;
        }
-        const char* arg = argv[index];
-        std::ifstream f(arg, std::ios::binary);
+        const std::string arg = argv_to_utf8(index, argv);
+        std::ifstream f(fs::u8path(arg), std::ios::binary);
        try {
            negative_prompt = std::string(std::istreambuf_iterator<char>{f}, {});
        } catch (const std::ios_base::failure&) {
            f.setstate(std::ios_base::failbit);
        }
        if (f.fail()) {
-            LOG_ERROR("error: failed to read negative prompt file '%s'\n", arg);
+            LOG_ERROR("error: failed to read negative prompt file '%s'\n", arg.c_str());
            return -1;
        }
        return 1;
@@ -2050,22 +2051,22 @@ static bool resolve_model_file_from_dir(const std::string& model_name,
    if (model_name.empty() ||
        model_name.find('/') != std::string::npos ||
        model_name.find('\\') != std::string::npos ||
-        fs::path(model_name).has_root_path() ||
+        fs::u8path(model_name).has_root_path() ||
        ends_with_valid_ext()) {
        LOG_ERROR("%s must be a model name without path or extension: %s", label, model_name.c_str());
        return false;
    }

-    fs::path model_dir_path = model_dir;
+    fs::path model_dir_path = fs::u8path(model_dir);
    for (const auto& ext : valid_ext) {
-        fs::path try_path = model_dir_path / (model_name + ext);
+        fs::path try_path = model_dir_path / fs::u8path(model_name + ext);
        if (fs::exists(try_path) && fs::is_regular_file(try_path)) {
-            resolved_path = try_path.lexically_normal().string();
+            resolved_path = try_path.lexically_normal().u8string();
            return true;
        }
    }

-    LOG_ERROR("can not find %s %s in %s", label, model_name.c_str(), model_dir_path.lexically_normal().string().c_str());
+    LOG_ERROR("can not find %s %s in %s", label, model_name.c_str(), model_dir_path.lexically_normal().u8string().c_str());
    return false;
 }

@@ -2359,10 +2360,18 @@ bool SDGenerationParams::extract_and_remove_lora(const std::string& lora_model_d
        }

        fs::path final_path;
-        if (is_absolute_path(raw_path)) {
-            final_path = raw_path;
-        } else {
-            final_path = fs::path(lora_model_dir) / raw_path;
+        try {
+            if (is_absolute_path(raw_path)) {
+                final_path = fs::u8path(raw_path);
+            } else {
+                final_path = fs::u8path(lora_model_dir) / fs::u8path(raw_path);
+            }
+        } catch (const std::exception&) {  // not UTF-8, e.g. a prompt file in another encoding
+            LOG_WARN("invalid lora path %s", raw_path.c_str());
+            tmp     = m.suffix().str();
+            prompt  = std::regex_replace(prompt, re, "", std::regex_constants::format_first_only);
+            missing = true;
+            continue;
        }
        if (!fs::exists(final_path)) {
            bool found = false;
@@ -2376,7 +2385,7 @@ bool SDGenerationParams::extract_and_remove_lora(const std::string& lora_model_d
                }
            }
            if (!found) {
-                LOG_WARN("can not found lora %s", final_path.lexically_normal().string().c_str());
+                LOG_WARN("can not found lora %s", final_path.lexically_normal().u8string().c_str());
                tmp     = m.suffix().str();
                prompt  = std::regex_replace(prompt, re, "", std::regex_constants::format_first_only);
                missing = true;
@@ -2384,7 +2393,7 @@ bool SDGenerationParams::extract_and_remove_lora(const std::string& lora_model_d
            }
        }

-        const std::string key = final_path.lexically_normal().string();
+        const std::string key = final_path.lexically_normal().u8string();

        if (is_high_noise)
            high_noise_lora_map[key] += mul;
@@ -12,6 +12,10 @@
 #include "resource_owners.hpp"
 #include "stable-diffusion.h"

+// The command-line argument at `index` as UTF-8 (on Windows, read again from
+// the UTF-16 command line: argv itself is in the ANSI code page).
+std::string argv_to_utf8(int index, const char** argv);
+
 #define SAFE_STR(s) ((s) ? (s) : "")
 #define BOOL_STR(b) ((b) ? "true" : "false")

@@ -15,10 +15,12 @@

 #define STB_IMAGE_IMPLEMENTATION
 #define STB_IMAGE_STATIC
+#define STBI_WINDOWS_UTF8
 #include "stb_image.h"

 #define STB_IMAGE_WRITE_IMPLEMENTATION
 #define STB_IMAGE_WRITE_STATIC
+#define STBIW_WINDOWS_UTF8
 #include "stb_image_write.h"

 #define STB_IMAGE_RESIZE_IMPLEMENTATION
@@ -147,7 +149,7 @@ private:
 #endif

 bool read_binary_file_bytes(const char* path, std::vector<uint8_t>& data) {
-    std::ifstream fin(fs::path(path), std::ios::binary);
+    std::ifstream fin(fs::u8path(path), std::ios::binary);
    if (!fin) {
        return false;
    }
@@ -170,7 +172,7 @@ bool read_binary_file_bytes(const char* path, std::vector<uint8_t>& data) {
 }

 bool write_binary_file_bytes(const std::string& path, const std::vector<uint8_t>& data) {
-    std::ofstream fout(fs::path(path), std::ios::binary);
+    std::ofstream fout(fs::u8path(path), std::ios::binary);
    if (!fout) {
        return false;
    }
@@ -714,7 +716,7 @@ static std::pair<uint64_t, uint64_t> audio_sample_range_for_video_frame(const sd
 }

 EncodedImageFormat encoded_image_format_from_path(const std::string& path) {
-    std::string ext = fs::path(path).extension().string();
+    std::string ext = fs::u8path(path).extension().u8string();
    std::transform(ext.begin(), ext.end(), ext.begin(), ::tolower);

    if (ext == ".jpg" || ext == ".jpeg" || ext == ".jpe") {
@@ -1413,7 +1415,7 @@ bool write_wav_to_file(const std::string& path,
        return false;
    }

-    std::ofstream file(path, std::ios::binary);
+    std::ofstream file(fs::u8path(path), std::ios::binary);
    if (!file.is_open()) {
        return false;
    }
@@ -1475,7 +1477,7 @@ bool load_wav_from_file(const std::string& path,
    sample_rate = 0;
    channels    = 0;

-    std::ifstream file(path, std::ios::binary);
+    std::ifstream file(fs::u8path(path), std::ios::binary);
    uint8_t riff_header[12];
    if (!file.read(reinterpret_cast<char*>(riff_header), sizeof(riff_header)) ||
        std::memcmp(riff_header, "RIFF", 4) != 0 ||
@@ -1,5 +1,6 @@
 #include "routes.h"

+#include <filesystem>
 #include <fstream>
 #include <iterator>

@@ -7,7 +8,7 @@ void register_index_endpoints(httplib::Server& svr, const SDSvrParams& svr_param
    const std::string serve_html_path = svr_params.serve_html_path;
    svr.Get("/", [serve_html_path, index_html](const httplib::Request&, httplib::Response& res) {
        if (!serve_html_path.empty()) {
-            std::ifstream file(serve_html_path);
+            std::ifstream file(std::filesystem::u8path(serve_html_path));
            if (file) {
                std::string content((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
                res.set_content(content, "text/html");
@@ -28,10 +28,10 @@ static std::string extract_and_remove_sd_cpp_extra_args(std::string& text) {
 static fs::path resolve_display_model_path(const ServerRuntime& runtime) {
    const auto& ctx = *runtime.ctx_params;
    if (!ctx.model_path.empty()) {
-        return fs::path(ctx.model_path);
+        return fs::u8path(ctx.model_path);
    }
    if (!ctx.diffusion_model_path.empty()) {
-        return fs::path(ctx.diffusion_model_path);
+        return fs::u8path(ctx.diffusion_model_path);
    }
    return {};
 }
@@ -583,9 +583,9 @@ void register_sdapi_endpoints(httplib::Server& svr, ServerRuntime& rt) {
    svr.Get("/sdapi/v1/sd-models", [runtime](const httplib::Request&, httplib::Response& res) {
        fs::path model_path = resolve_display_model_path(*runtime);
        json entry;
-        entry["title"]      = model_path.stem();
-        entry["model_name"] = model_path.stem();
-        entry["filename"]   = model_path.filename();
+        entry["title"]      = model_path.stem().u8string();
+        entry["model_name"] = model_path.stem().u8string();
+        entry["filename"]   = model_path.filename().u8string();
        entry["hash"]       = "8888888888";
        entry["sha256"]     = "8888888888888888888888888888888888888888888888888888888888888888";
        entry["config"]     = nullptr;
@@ -598,7 +598,7 @@ void register_sdapi_endpoints(httplib::Server& svr, ServerRuntime& rt) {
        fs::path model_path = resolve_display_model_path(*runtime);
        json r;
        r["samples_format"]      = "png";
-        r["sd_model_checkpoint"] = model_path.stem();
+        r["sd_model_checkpoint"] = model_path.stem().u8string();
        res.set_content(r.dump(), "application/json");
    });

@@ -90,10 +90,10 @@ static json make_vae_tiling_json(const sd_tiling_params_t& params) {
 static fs::path resolve_display_model_path(const ServerRuntime& runtime) {
    const auto& ctx = *runtime.ctx_params;
    if (!ctx.model_path.empty()) {
-        return fs::path(ctx.model_path);
+        return fs::u8path(ctx.model_path);
    }
    if (!ctx.diffusion_model_path.empty()) {
-        return fs::path(ctx.diffusion_model_path);
+        return fs::u8path(ctx.diffusion_model_path);
    }
    return {};
 }
@@ -22,7 +22,7 @@ static std::string lower_ascii(std::string value) {
 }

 static bool is_supported_model_ext(const fs::path& p) {
-    auto ext = lower_ascii(p.extension().string());
+    auto ext = lower_ascii(p.extension().u8string());
    return ext == ".gguf" || ext == ".pt" || ext == ".pth" || ext == ".safetensors";
 }

@@ -226,7 +226,7 @@ bool SDSvrParams::validate() {
        return false;
    }

-    if (!serve_html_path.empty() && !fs::exists(serve_html_path)) {
+    if (!serve_html_path.empty() && !fs::exists(fs::u8path(serve_html_path))) {
        LOG_ERROR("error: serve_html_path file does not exist: %s", serve_html_path.c_str());
        return false;
    }
@@ -260,7 +260,7 @@ void refresh_lora_cache(ServerRuntime& rt) {
        return;
    }

-    fs::path lora_dir = rt.ctx_params->lora_model_dir;
+    fs::path lora_dir = fs::u8path(rt.ctx_params->lora_model_dir);
    std::error_code ec;
    if (fs::exists(lora_dir, ec) && !ec && fs::is_directory(lora_dir, ec) && !ec) {
        try {
@@ -293,13 +293,13 @@ void refresh_lora_cache(ServerRuntime& rt) {
                it.increment(ec);
            }
        } catch (const std::exception& e) {
-            LOG_WARN("error while scanning lora directory '%s': %s", lora_dir.string().c_str(), e.what());
+            LOG_WARN("error while scanning lora directory '%s': %s", lora_dir.u8string().c_str(), e.what());
            return;
        }
    }

    if (ec) {
-        LOG_WARN("error while scanning lora directory '%s': %s", lora_dir.string().c_str(), ec.message().c_str());
+        LOG_WARN("error while scanning lora directory '%s': %s", lora_dir.u8string().c_str(), ec.message().c_str());
        return;
    }

@@ -328,7 +328,7 @@ void refresh_upscaler_cache(ServerRuntime& rt) {
        previous_cache = *rt.upscaler_cache;
    }

-    fs::path upscaler_dir = rt.ctx_params->hires_upscalers_dir;
+    fs::path upscaler_dir = fs::u8path(rt.ctx_params->hires_upscalers_dir);
    std::error_code ec;
    if (fs::exists(upscaler_dir, ec) && !ec && fs::is_directory(upscaler_dir, ec) && !ec) {
        try {
@@ -384,13 +384,13 @@ void refresh_upscaler_cache(ServerRuntime& rt) {
                it.increment(ec);
            }
        } catch (const std::exception& e) {
-            LOG_WARN("error while scanning upscalers directory '%s': %s", upscaler_dir.string().c_str(), e.what());
+            LOG_WARN("error while scanning upscalers directory '%s': %s", upscaler_dir.u8string().c_str(), e.what());
            return;
        }
    }

    if (ec) {
-        LOG_WARN("error while scanning upscalers directory '%s': %s", upscaler_dir.string().c_str(), ec.message().c_str());
+        LOG_WARN("error while scanning upscalers directory '%s': %s", upscaler_dir.u8string().c_str(), ec.message().c_str());
        return;
    }

@@ -1995,6 +1995,9 @@ struct LLMEmbedder : public Conditioner {
            if (sd_version_is_pid(version)) {
                throw std::runtime_error("PiD requires an external Gemma 2 tokenizer.json; pass --tokenizer FILE or set sd_ctx_params_t::tokenizer");
            }
+            if (version == VERSION_MOSS_IMAGE) {
+                throw std::runtime_error("MOSS-Image requires the official Qwen3.5 tokenizer.json; pass --tokenizer FILE or set sd_ctx_params_t::tokenizer");
+            }
        }
        LLM::LLMArch arch = LLM::LLMArch::QWEN2_5_VL;
        if (version == VERSION_FLUX2) {
@@ -2006,6 +2009,7 @@ struct LLMEmbedder : public Conditioner {
        } else if (sd_version_is_pid(version)) {
            arch = LLM::LLMArch::GEMMA2_2B;
        } else if (version == VERSION_QWEN_IMAGE_2_1 ||
+                   sd_version_is_flux3(version) ||
                   sd_version_is_lingbot_video(version) ||
                   sd_version_is_ideogram4(version) ||
                   sd_version_is_boogu_image(version) ||
@@ -2017,6 +2021,8 @@ struct LLMEmbedder : public Conditioner {
            arch = LLM::LLMArch::QWEN3_VL;
        } else if (sd_version_is_z_image(version) || sd_version_is_z_image_l2p(version) || version == VERSION_OVIS_IMAGE || version == VERSION_FLUX2_KLEIN) {
            arch = LLM::LLMArch::QWEN3;
+        } else if (version == VERSION_MOSS_IMAGE) {
+            arch = LLM::LLMArch::QWEN3_5;
        }
        llm        = std::make_shared<LLM::LLMRunner>(arch,
                                               backend,
@@ -2031,6 +2037,8 @@ struct LLMEmbedder : public Conditioner {
            pad_id = 199999;
        } else if (arch == LLM::LLMArch::GEMMA2_2B) {
            pad_id = 0;
+        } else if (arch == LLM::LLMArch::QWEN3_5) {
+            pad_id = 248044;
        }
        tokenizer = tokenizers.create(TokenizerConfig::MAIN, llm->config.vocab_size, pad_id);
        if (!tokenizer) {
@@ -2201,6 +2209,9 @@ struct LLMEmbedder : public Conditioner {
            weights.insert(weights.end(), suffix_tokens.size(), 1.f);
        }

+        if (sd_version_is_flux3(version)) {
+            min_length = std::min(max_length, ((tokens.size() + 79) / 80) * 80);
+        }
        std::vector<float> mask;
        tokenizer->pad_tokens(tokens, &weights, &mask, min_length, max_length);

@@ -2254,6 +2265,15 @@ struct LLMEmbedder : public Conditioner {
            }
        }

+        sd::Tensor<int32_t> position_ids;
+        if (sd_version_is_flux3(version)) {
+            position_ids = sd::Tensor<int32_t>::zeros({static_cast<int64_t>(tokens.size() * 4)});
+            for (size_t axis = 0; axis < 4; ++axis) {
+                for (size_t i = 0; i < tokens.size(); ++i) {
+                    position_ids[axis * tokens.size() + i] = mask[i] == 0.f ? 1 : static_cast<int32_t>(i);
+                }
+            }
+        }
        auto hidden_states = llm->compute(n_threads,
                                          input_ids,
                                          attention_mask,
@@ -2262,7 +2282,8 @@ struct LLMEmbedder : public Conditioner {
                                          false,
                                          false,
                                          deepstack_image_embeds,
-                                          image_grids);
+                                          image_grids,
+                                          position_ids);
        if (hidden_states.empty()) {
            LOG_ERROR("LLM prompt encoding failed");
            return {};
@@ -3025,6 +3046,20 @@ struct LLMEmbedder : public Conditioner {

                prompt += "<|im_end|>\n<|im_start|>assistant\n";
            }
+        } else if (version == VERSION_MOSS_IMAGE) {
+            prompt_template_encode_start_idx = 0;
+            max_length                       = 4096;
+            prompt                           = "<|im_start|>user\n" + conditioner_params.text + "<|im_end|>\n<|im_start|>assistant\n<think>\n";
+            prompt_attn_range                = {0, 0};
+        } else if (sd_version_is_flux3(version)) {
+            prompt_template_encode_start_idx = 0;
+            max_length                       = 4096;
+            out_layers                       = {4, 8, 12, 16, 20, 24, 28, 32};
+            prompt                           = "<|im_start|>user\n";
+            prompt_attn_range.first          = static_cast<int>(prompt.size());
+            prompt += conditioner_params.text;
+            prompt_attn_range.second = static_cast<int>(prompt.size());
+            prompt += "<|im_end|>\n<|im_start|>assistant\n";
        } else if (version == VERSION_FLUX2_KLEIN) {
            prompt_template_encode_start_idx = 0;
            min_length                       = 512;
@@ -3,6 +3,7 @@
 #include <cstdint>
 #include <cstring>
 #include <exception>
+#include <filesystem>
 #include <fstream>
 #include <map>
 #include <memory>
@@ -108,7 +109,7 @@ static bool preallocate_output_file(const std::string& output_path, uint64_t fil
        return true;
    }

-    std::fstream file(output_path, std::ios::binary | std::ios::in | std::ios::out);
+    std::fstream file(std::filesystem::u8path(output_path), std::ios::binary | std::ios::in | std::ios::out);
    if (!file.is_open()) {
        if (error != nullptr) {
            *error = "failed to open output file '" + output_path + "' for preallocation";
@@ -251,7 +252,7 @@ static bool stream_tensor_data(ModelLoader& model_loader,

    for (int worker = 0; worker < n_threads; worker++) {
        workers.emplace_back([&]() {
-            std::fstream output_file(output_path, std::ios::binary | std::ios::in | std::ios::out);
+            std::fstream output_file(std::filesystem::u8path(output_path), std::ios::binary | std::ios::in | std::ios::out);
            if (!output_file.is_open()) {
                fail("failed to open output file '" + output_path + "' for tensor writing");
                return;
@@ -344,22 +344,15 @@ ggml_tensor* ggml_ext_pad(ggml_context* ctx,
 }

 static ggml_tensor* conv_1d(ggml_context* ctx, ggml_tensor* x, ggml_tensor* w, int s0, int p0, int d0, bool force_prec_f32) {
-    ggml_tensor* result;
-    if (force_prec_f32) {
-        ggml_tensor* patches = ggml_im2col(ctx, w, x, s0, 0, p0, 0, d0, 0, false, GGML_TYPE_F32);
-        result               = ggml_mul_mat(ctx,
-                                            ggml_reshape_2d(ctx, patches, patches->ne[0], patches->ne[2] * patches->ne[1]),
-                                            ggml_reshape_2d(ctx, w, w->ne[0] * w->ne[1], w->ne[2]));
-        result               = ggml_reshape_3d(ctx, result, patches->ne[1], w->ne[2], patches->ne[2]);
-    } else {
-        result = ggml_conv_1d(ctx, w, x, s0, p0, d0);
+    if (!force_prec_f32) {
+        return ggml_conv_1d(ctx, w, x, s0, p0, d0);
    }
-    if (x->ne[2] > 1) {
-        // mul_mat packs positions and batches before output channels: [OL, N, OC].
-        result = ggml_reshape_3d(ctx, result, result->ne[0], x->ne[2], w->ne[2]);
-        result = ggml_cont(ctx, ggml_permute(ctx, result, 0, 2, 1, 3));
-    }
-    return result;
+    ggml_tensor* patches = ggml_im2col(ctx, w, x, s0, 0, p0, 0, d0, 0, false, GGML_TYPE_F32);
+    ggml_tensor* result  = ggml_mul_mat(ctx,
+                                        ggml_reshape_2d(ctx, patches, patches->ne[0], patches->ne[2] * patches->ne[1]),
+                                        ggml_reshape_2d(ctx, w, w->ne[0] * w->ne[1], w->ne[2]));
+    result               = ggml_reshape_3d(ctx, result, patches->ne[1], patches->ne[2], w->ne[2]);
+    return ggml_cont(ctx, ggml_permute(ctx, result, 0, 2, 1, 3));
 }

 ggml_tensor* ggml_ext_conv_1d(ggml_context* ctx,
@@ -3,6 +3,7 @@
 #include <algorithm>
 #include <cstdlib>
 #include <cstring>
+#include <filesystem>
 #include <fstream>

 #include "core/ggml_extend_backend.h"
@@ -100,7 +101,7 @@ void ggml_ext_tensor_diff(
 }

 ggml_tensor* load_tensor_from_file(ggml_context* ctx, const std::string& file_path) {
-    std::ifstream file(file_path, std::ios::binary);
+    std::ifstream file(std::filesystem::u8path(file_path), std::ios::binary);
    if (!file.is_open()) {
        LOG_ERROR("failed to open '%s'", file_path.c_str());
        return nullptr;
@@ -3,6 +3,7 @@

 #include <array>
 #include <cstring>
+#include <filesystem>
 #include <fstream>
 #include <stdexcept>
 #include <string>
@@ -105,7 +106,7 @@ namespace sd {

    template <typename T>
    inline Tensor<T> load_tensor_from_file_as_tensor(const std::string& file_path) {
-        std::ifstream file(file_path, std::ios::binary);
+        std::ifstream file(std::filesystem::u8path(file_path), std::ios::binary);
        if (!file.is_open()) {
            throw std::runtime_error("failed to open tensor file: " + file_path);
        }
@@ -7,6 +7,7 @@
 #include <cstdlib>
 #include <cstring>
 #include <exception>
+#include <filesystem>
 #include <fstream>
 #include <locale>
 #include <regex>
@@ -123,12 +124,12 @@ int round_up_to(int value, int base) {
 #include <windows.h>

 bool file_exists(const std::string& filename) {
-    DWORD attributes = GetFileAttributesA(filename.c_str());
+    DWORD attributes = GetFileAttributesW(std::filesystem::u8path(filename).c_str());
    return (attributes != INVALID_FILE_ATTRIBUTES && !(attributes & FILE_ATTRIBUTE_DIRECTORY));
 }

 bool is_directory(const std::string& path) {
-    DWORD attributes = GetFileAttributesA(path.c_str());
+    DWORD attributes = GetFileAttributesW(std::filesystem::u8path(path).c_str());
    return (attributes != INVALID_FILE_ATTRIBUTES && (attributes & FILE_ATTRIBUTE_DIRECTORY));
 }

@@ -152,8 +153,8 @@ std::unique_ptr<MmapWrapper> MmapWrapper::create(const std::string& filename, bo
    void* mapped_data = nullptr;
    size_t file_size  = 0;

-    HANDLE file_handle = CreateFileA(
-        filename.c_str(),
+    HANDLE file_handle = CreateFileW(
+        std::filesystem::u8path(filename).c_str(),
        GENERIC_READ,
        FILE_SHARE_READ,
        nullptr,
@@ -66,6 +66,8 @@ enum SDVersion {
    VERSION_MING_IMAGE,
    VERSION_Z_IMAGE_L2P,
    VERSION_IRIS,
+    VERSION_FLUX3,
+    VERSION_MOSS_IMAGE,
    VERSION_COUNT,
 };

@@ -118,6 +120,10 @@ static inline bool sd_version_is_flux(SDVersion version) {
    return false;
 }

+static inline bool sd_version_is_flux3(SDVersion version) {
+    return version == VERSION_FLUX3;
+}
+
 static inline bool sd_version_is_flux2(SDVersion version) {
    if (version == VERSION_FLUX2 || version == VERSION_FLUX2_KLEIN) {
        return true;
@@ -280,7 +286,14 @@ static inline bool sd_version_uses_flux_vae(SDVersion version) {
 }

 static inline bool sd_version_uses_flux2_vae(SDVersion version) {
-    if (sd_version_is_flux2(version) || sd_version_is_ernie_image(version) || sd_version_is_lens(version) || sd_version_is_ideogram4(version) || sd_version_is_sefi_image(version) || sd_version_is_llada_image(version)) {
+    if (sd_version_is_flux2(version) ||
+        sd_version_is_flux3(version) ||
+        sd_version_is_ernie_image(version) ||
+        sd_version_is_lens(version) ||
+        sd_version_is_ideogram4(version) ||
+        sd_version_is_sefi_image(version) ||
+        sd_version_is_llada_image(version) ||
+        version == VERSION_MOSS_IMAGE) {
        return true;
    }
    return false;
@@ -311,6 +324,7 @@ static inline bool sd_version_is_inpaint(SDVersion version) {
 static inline bool sd_version_is_dit(SDVersion version) {
    if (sd_version_is_flux(version) ||
        sd_version_is_flux2(version) ||
+        sd_version_is_flux3(version) ||
        sd_version_is_ltxav(version) ||
        sd_version_is_minimax_h3(version) ||
        sd_version_is_sd3(version) ||
@@ -336,7 +350,8 @@ static inline bool sd_version_is_dit(SDVersion version) {
        sd_version_is_mage_flow(version) ||
        sd_version_is_sensenova_u1(version) ||
        sd_version_is_pixart(version) ||
-        version == VERSION_IRIS) {
+        version == VERSION_IRIS ||
+        version == VERSION_MOSS_IMAGE) {
        return true;
    }
    return false;
@@ -13,8 +13,8 @@
 #include <vector>

 #include "core/ggml_extend.h"
-#include "core/ggml_runner.h"
 #include "core/ggml_extend_backend.h"
+#include "core/ggml_runner.h"
 #include "model.h"

 class GGMLBlock {
@@ -0,0 +1,256 @@
+#ifndef __SD_MODEL_DIFFUSION_FLUX3_HPP__
+#define __SD_MODEL_DIFFUSION_FLUX3_HPP__
+
+#include "model/diffusion/flux.hpp"
+
+namespace Flux3 {
+
+    struct Flux3Config {
+        int64_t hidden_size = 3072;
+        int64_t context_dim = 20480;
+        int64_t channels    = 128;
+        int64_t mlp_dim     = 9216;
+        int heads           = 24;
+        int depth           = 5;
+        int single_depth    = 28;
+        bool flow_map       = false;
+
+        static Flux3Config detect_from_weights(const String2TensorStorage& tensors, const std::string& prefix) {
+            Flux3Config config;
+            int depth = 0, single_depth = 0;
+            bool detected = false;
+            for (const auto& [name, tensor] : tensors) {
+                if (!starts_with(name, prefix + ".")) {
+                    continue;
+                }
+                std::string key = name.substr(prefix.size() + 1);
+                if (key == "txt_in.weight") {
+                    detected           = true;
+                    config.context_dim = tensor.ne[0];
+                    config.hidden_size = tensor.ne[1];
+                } else if (key == "emb_in.image.weight") {
+                    config.channels = tensor.ne[0];
+                } else if (key == "single_blocks.0.mlp_out.weight") {
+                    config.mlp_dim = tensor.ne[0];
+                } else if (key == "ref_time_in.in_layer.weight") {
+                    config.flow_map = true;
+                }
+                if (starts_with(key, "txt_mode_blocks.")) {
+                    depth = std::max(depth, atoi(key.substr(16).c_str()) + 1);
+                }
+                if (starts_with(key, "single_blocks.")) {
+                    single_depth = std::max(single_depth, atoi(key.substr(14).c_str()) + 1);
+                }
+            }
+            if (depth > 0)
+                config.depth = depth;
+            if (single_depth > 0)
+                config.single_depth = single_depth;
+            config.heads = static_cast<int>(config.hidden_size / 128);
+            if (detected) {
+                LOG_VERBOSE("flux3: hidden_size = %" PRId64 ", context_dim = %" PRId64 ", depth = %d, single_depth = %d, flow_map = %s",
+                            config.hidden_size, config.context_dim, config.depth, config.single_depth, config.flow_map ? "true" : "false");
+            }
+            return config;
+        }
+    };
+
+    static ggml_tensor* token_slice(ggml_context* ctx, ggml_tensor* x, int64_t start, int64_t length) {
+        return ggml_view_3d(ctx, x, x->ne[0], length, x->ne[2], x->nb[1], x->nb[2], start * x->nb[1]);
+    }
+
+    struct Block : public GGMLBlock {
+        int heads;
+
+        Block(const Flux3Config& config)
+            : heads(config.heads) {
+            for (const auto* name : {"q_proj", "k_proj", "v_proj", "attn_out"}) {
+                blocks[name] = std::make_shared<Linear>(config.hidden_size, config.hidden_size, false);
+            }
+            blocks["mlp_in"]  = std::make_shared<Linear>(config.hidden_size, 2 * config.mlp_dim, false);
+            blocks["mlp_out"] = std::make_shared<Linear>(config.mlp_dim, config.hidden_size, false);
+            blocks["norm"]    = std::make_shared<Flux::QKNorm>(config.hidden_size / heads);
+        }
+
+        ggml_tensor* linear(GGMLRunnerContext* ctx, const std::string& name, ggml_tensor* x) {
+            return std::dynamic_pointer_cast<Linear>(blocks[name])->forward(ctx, x);
+        }
+
+        ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x, ggml_tensor* pe, const std::vector<Flux::ModulationOut>& mods, const std::vector<int64_t>& lengths) {
+            auto gc            = ctx->ggml_ctx;
+            ggml_tensor* x_mod = nullptr;
+            int64_t offset     = 0;
+            for (size_t i = 0; i < lengths.size(); ++i) {
+                auto part = token_slice(gc, x, offset, lengths[i]);
+                part      = Flux::modulate(gc, ggml_norm(gc, part, 1e-6f), mods[i].shift, mods[i].scale);
+                x_mod     = x_mod ? ggml_concat(gc, x_mod, part, 1) : part;
+                offset += lengths[i];
+            }
+            auto split = [&](ggml_tensor* v) {
+                return ggml_reshape_4d(gc, v, x->ne[0] / heads, heads, x->ne[1], x->ne[2]);
+            };
+            auto norm        = std::dynamic_pointer_cast<Flux::QKNorm>(blocks["norm"]);
+            auto q           = norm->query_norm(ctx, split(linear(ctx, "q_proj", x_mod)));
+            auto k           = norm->key_norm(ctx, split(linear(ctx, "k_proj", x_mod)));
+            auto v           = split(linear(ctx, "v_proj", x_mod));
+            auto attn        = Rope::attention(ctx, q, k, v, pe, nullptr);
+            auto mlp         = ggml_ext_silu_act(gc, linear(ctx, "mlp_in", x_mod));
+            auto update      = ggml_add(gc, linear(ctx, "attn_out", attn), linear(ctx, "mlp_out", mlp));
+            ggml_tensor* out = nullptr;
+            offset           = 0;
+            for (size_t i = 0; i < lengths.size(); ++i) {
+                auto part = ggml_mul(gc, token_slice(gc, update, offset, lengths[i]), mods[i].gate);
+                part      = ggml_add(gc, token_slice(gc, x, offset, lengths[i]), part);
+                out       = out ? ggml_concat(gc, out, part, 1) : part;
+                offset += lengths[i];
+            }
+            return out;
+        }
+    };
+
+    struct Model : public GGMLBlock {
+        Flux3Config config;
+
+        Model(const Flux3Config& config)
+            : config(config) {
+            blocks["txt_in"]  = std::make_shared<Linear>(config.context_dim, config.hidden_size, false);
+            blocks["time_in"] = std::make_shared<Flux::MLPEmbedder>(256, config.hidden_size, false);
+            if (config.flow_map) {
+                blocks["ref_time_in"] = std::make_shared<Flux::MLPEmbedder>(256, config.hidden_size, false);
+            }
+            for (const std::string mode : {"image", "image_cond", "txt"}) {
+                blocks["early_stream_modulations." + mode]  = std::make_shared<Flux::Modulation>(config.hidden_size, false, false);
+                blocks["single_stream_modulations." + mode] = std::make_shared<Flux::Modulation>(config.hidden_size, false, false);
+                const std::string base                      = mode == "txt" ? "txt_mode_blocks." : "content_mode_blocks." + mode + ".";
+                for (int i = 0; i < config.depth; ++i) {
+                    blocks[base + std::to_string(i)] = std::make_shared<Block>(config);
+                }
+                if (mode != "txt") {
+                    blocks["emb_in." + mode] = std::make_shared<Linear>(config.channels, config.hidden_size, false);
+                }
+            }
+            for (int i = 0; i < config.single_depth; ++i) {
+                blocks["single_blocks." + std::to_string(i)] = std::make_shared<Block>(config);
+            }
+            blocks["final_layer.image"] = std::make_shared<Flux::LastLayer>(config.hidden_size, 1, config.channels, false, false);
+        }
+
+        ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* image, ggml_tensor* text, ggml_tensor* reference, ggml_tensor* timestep, ggml_tensor* zero, ggml_tensor* target, ggml_tensor* pe) {
+            auto gc             = ctx->ggml_ctx;
+            const int64_t width = image->ne[0], height = image->ne[1];
+            auto time_in   = std::dynamic_pointer_cast<Flux::MLPEmbedder>(blocks["time_in"]);
+            auto image_vec = time_in->forward(ctx, ggml_ext_timestep_embedding(gc, timestep, 256, 10000, 1000.f));
+            auto clean_vec = time_in->forward(ctx, ggml_ext_timestep_embedding(gc, zero, 256, 10000, 1000.f));
+            if (config.flow_map) {
+                auto ref_time = std::dynamic_pointer_cast<Flux::MLPEmbedder>(blocks["ref_time_in"]);
+                auto ref_vec  = ref_time->forward(ctx, ggml_ext_timestep_embedding(gc, target, 256, 10000, 1000.f));
+                image_vec     = ggml_add(gc, image_vec, ref_vec);
+                clean_vec     = ggml_add(gc, clean_vec, ref_vec);
+            }
+            std::vector<std::string> modes{"image"};
+            std::vector<ggml_tensor*> inputs{DiT::patchify(gc, image, 1, 1)};
+            if (reference) {
+                modes.push_back("image_cond");
+                inputs.push_back(reference);
+            }
+            modes.push_back("txt");
+            inputs.push_back(text);
+            std::vector<Flux::ModulationOut> single_mods;
+            std::vector<int64_t> lengths;
+            ggml_tensor* joint = nullptr;
+            int64_t offset     = 0;
+            for (size_t m = 0; m < modes.size(); ++m) {
+                const auto& mode = modes[m];
+                auto vec         = m == 0 ? image_vec : clean_vec;
+                auto early       = std::dynamic_pointer_cast<Flux::Modulation>(blocks["early_stream_modulations." + mode])->forward(ctx, vec)[0];
+                single_mods.push_back(std::dynamic_pointer_cast<Flux::Modulation>(blocks["single_stream_modulations." + mode])->forward(ctx, vec)[0]);
+                lengths.push_back(inputs[m]->ne[1]);
+                auto h                 = std::dynamic_pointer_cast<Linear>(blocks[mode == "txt" ? "txt_in" : "emb_in." + mode])->forward(ctx, inputs[m]);
+                auto mode_pe           = ggml_view_4d(gc, pe, 2, 2, 64, lengths.back(), pe->nb[1], pe->nb[2], pe->nb[3], offset * pe->nb[3]);
+                const std::string base = mode == "txt" ? "txt_mode_blocks." : "content_mode_blocks." + mode + ".";
+                for (int i = 0; i < config.depth; ++i) {
+                    h = std::dynamic_pointer_cast<Block>(blocks[base + std::to_string(i)])->forward(ctx, h, mode_pe, {early}, {lengths.back()});
+                }
+                joint = joint ? ggml_concat(gc, joint, h, 1) : h;
+                offset += lengths.back();
+            }
+            for (int i = 0; i < config.single_depth; ++i) {
+                joint = std::dynamic_pointer_cast<Block>(blocks["single_blocks." + std::to_string(i)])->forward(ctx, joint, pe, single_mods, lengths);
+            }
+            auto out = std::dynamic_pointer_cast<Flux::LastLayer>(blocks["final_layer.image"])->forward(ctx, token_slice(gc, joint, 0, lengths[0]), image_vec);
+            return DiT::unpatchify(gc, out, height, width, 1, 1);
+        }
+    };
+
+    struct Flux3Runner : public DiffusionModelRunner {
+        Flux3Config config;
+        Model model;
+        std::vector<float> pe_values;
+        sd::Tensor<float> zero = sd::Tensor<float>::zeros({1});
+
+        Flux3Runner(ggml_backend_t backend, const String2TensorStorage& tensors, const std::string& prefix, std::shared_ptr<RunnerWeightManager> weight_manager = nullptr)
+            : DiffusionModelRunner(backend, prefix, weight_manager),
+              config(Flux3Config::detect_from_weights(tensors, prefix)),
+              model(config) {
+            model.init(params_ctx, tensors, prefix);
+        }
+
+        std::string get_desc() override { return "flux3"; }
+        void get_param_tensors(std::map<std::string, ggml_tensor*>& tensors, const std::string& prefix) override {
+            model.get_param_tensors(tensors, prefix);
+        }
+
+        sd::Tensor<float> compute(int n_threads, const DiffusionParams& params) override {
+            if (params.ref_latents && params.ref_image_params.pass_to_dit && params.ref_latents->size() > 10) {
+                LOG_ERROR("FLUX.3 supports at most 10 reference images");
+                return {};
+            }
+            const auto* extra = diffusion_extra_as<Flux3DiffusionExtra>(params);
+            GGML_ASSERT(params.x && params.context && params.timesteps);
+            GGML_ASSERT(!config.flow_map || extra->target_timestep);
+            auto get_graph = [&]() {
+                auto gf   = new_graph_custom(FLUX_GRAPH_SIZE);
+                auto x    = make_input(*params.x);
+                auto text = make_input(*params.context);
+                GGML_ASSERT(x->ne[3] == 1 && text->ne[2] == 1);
+                Rope::Embedding embedding;
+                auto append_image = [&](int64_t w, int64_t h, float index) {
+                    embedding.positions.append_image(static_cast<int>(h), static_cast<int>(w), 1, 0.5f, 0.5f);
+                    for (int64_t y = 0; y < h; ++y) {
+                        for (int64_t x = 0; x < w; ++x) {
+                            embedding.ids.push_back({0.f, y * 0.5f, x * 0.5f, index});
+                        }
+                    }
+                };
+                append_image(x->ne[0], x->ne[1], 0.f);
+                ggml_tensor* references = nullptr;
+                if (params.ref_latents && params.ref_image_params.pass_to_dit) {
+                    int index = 0;
+                    for (const auto& ref : *params.ref_latents) {
+                        auto r = make_input(ref);
+                        append_image(r->ne[0], r->ne[1], -float(++index));
+                        r          = DiT::patchify(compute_ctx, r, 1, 1);
+                        references = references ? ggml_concat(compute_ctx, references, r, 1) : r;
+                    }
+                }
+                for (int64_t i = 0; i < text->ne[1]; ++i) {
+                    embedding.ids.push_back({0.f, 0.f, 0.f, static_cast<float>(i)});
+                }
+                embedding.positions.append_tokens(text->ne[1]);
+                embedding.values = Rope::embed_nd(embedding.ids, 1, 10000.f, {32, 32, 32, 32}, embedding.layout, &embedding.frequencies);
+                pe_values        = finish_rope_pe(std::move(embedding));
+                auto pe          = ggml_new_tensor_4d(compute_ctx, GGML_TYPE_F32, 2, 2, 64, pe_values.size() / 256);
+                set_backend_tensor_data(pe, pe_values.data());
+                auto runner_ctx = get_context();
+                auto out        = model.forward(&runner_ctx, x, text, references, make_input(*params.timesteps), make_input(zero),
+                                         config.flow_map ? make_input(*extra->target_timestep) : nullptr, pe);
+                ggml_build_forward_expand(gf, out);
+                return gf;
+            };
+            return restore_trailing_singleton_dims(GGMLRunner::compute(get_graph, n_threads, false), params.x->dim());
+        }
+    };
+
+}  // namespace Flux3
+
+#endif  // __SD_MODEL_DIFFUSION_FLUX3_HPP__
@@ -64,6 +64,10 @@ struct FluxDiffusionExtra {
    float pulid_id_weight               = 1.0f;
 };

+struct Flux3DiffusionExtra {
+    const sd::Tensor<float>* target_timestep = nullptr;
+};
+
 struct AnimaDiffusionExtra {
    const sd::Tensor<int32_t>* t5_ids   = nullptr;
    const sd::Tensor<float>* t5_weights = nullptr;
@@ -149,6 +153,7 @@ using DiffusionExtraParams = std::variant<std::monostate,
                                          UNetDiffusionExtra,
                                          SkipLayerDiffusionExtra,
                                          FluxDiffusionExtra,
+                                          Flux3DiffusionExtra,
                                          AnimaDiffusionExtra,
                                          QwenImage21DiffusionExtra,
                                          WanDiffusionExtra,
@@ -0,0 +1,262 @@
+#ifndef __SD_MODEL_DIFFUSION_MOSS_IMAGE_HPP__
+#define __SD_MODEL_DIFFUSION_MOSS_IMAGE_HPP__
+
+#include <algorithm>
+#include <cinttypes>
+
+#include "model/common/ggml_block.hpp"
+#include "model/common/rope.hpp"
+#include "model/diffusion/dit.hpp"
+#include "model/diffusion/model.hpp"
+#include "model_loader.h"
+
+namespace MOSSImage {
+    struct MOSSImageConfig {
+        int64_t dim      = 3072;
+        int64_t ffn_dim  = 8192;
+        int64_t text_dim = 4096;
+        int64_t channels = 128;
+        int64_t head_dim = 128;
+        int64_t depth    = 42;
+        int64_t freq_dim = 256;
+
+        static MOSSImageConfig detect_from_weights(const String2TensorStorage& weights, const std::string& prefix) {
+            MOSSImageConfig config;
+            int64_t depth = 0;
+            for (const auto& [name, tensor] : weights) {
+                if (!starts_with(name, prefix + ".")) {
+                    continue;
+                }
+                auto key = name.substr(prefix.size() + 1);
+                if (key == "video_embedder.weight") {
+                    config.channels = tensor.ne[0];
+                    config.dim      = tensor.ne[1];
+                } else if (key == "text_embedder.proj.weight") {
+                    config.text_dim = tensor.ne[0];
+                } else if (key == "layers.0.proj.ffn_down.weight") {
+                    config.ffn_dim = tensor.ne[0];
+                } else if (key == "layers.0.q_norm.weight") {
+                    config.head_dim = tensor.ne[0];
+                } else if (key == "time_embedding.0.weight") {
+                    config.freq_dim = tensor.ne[0];
+                }
+                if (starts_with(key, "layers.")) {
+                    depth = std::max<int64_t>(depth, std::stoi(key.substr(7)) + 1);
+                }
+            }
+            if (depth) {
+                config.depth = depth;
+                LOG_VERBOSE("moss_image: dim = %" PRId64 ", ffn_dim = %" PRId64 ", depth = %" PRId64 ", head_dim = %" PRId64,
+                            config.dim, config.ffn_dim, config.depth, config.head_dim);
+            }
+            return config;
+        }
+    };
+
+    inline ggml_tensor* situ(ggml_context* c, ggml_tensor* hidden, ggml_tensor* gate) {
+        auto a = ggml_scale(c, ggml_tanh(c, ggml_scale(c, gate, 0.25f)), 4.f);
+        a      = ggml_mul(c, a, ggml_sigmoid(c, gate));
+        auto b = ggml_scale(c, ggml_tanh(c, ggml_scale(c, hidden, 1.f / 25.f)), 25.f);
+        return ggml_mul(c, a, b);
+    }
+
+    struct TextBlock : public GGMLBlock {
+        MOSSImageConfig config;
+
+        TextBlock(const MOSSImageConfig& config)
+            : config(config) {
+            blocks["norm1"]    = std::make_shared<RMSNorm>(config.dim, 1e-6f);
+            blocks["norm2"]    = std::make_shared<RMSNorm>(config.dim, 1e-6f);
+            blocks["q_norm"]   = std::make_shared<RMSNorm>(config.head_dim, 1e-6f);
+            blocks["k_norm"]   = std::make_shared<RMSNorm>(config.head_dim, 1e-6f);
+            blocks["qkv"]      = std::make_shared<Linear>(config.dim, config.dim * 3, true);
+            blocks["proj"]     = std::make_shared<Linear>(config.dim, config.dim, true);
+            blocks["ffn_up"]   = std::make_shared<Linear>(config.dim, config.ffn_dim, true);
+            blocks["ffn_gate"] = std::make_shared<Linear>(config.dim, config.ffn_dim, true);
+            blocks["ffn_down"] = std::make_shared<Linear>(config.ffn_dim, config.dim, true);
+        }
+
+        ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x, ggml_tensor* pe) {
+            auto c      = ctx->ggml_ctx;
+            auto linear = [&](const char* name, ggml_tensor* h) { return std::dynamic_pointer_cast<Linear>(blocks[name])->forward(ctx, h); };
+            auto norm   = [&](const char* name, ggml_tensor* h) { return std::dynamic_pointer_cast<RMSNorm>(blocks[name])->forward(ctx, h); };
+            auto y      = norm("norm1", x);
+            auto qkv    = linear("qkv", y);
+            auto split  = [&](int i) {
+                return ggml_reshape_4d(c, ggml_ext_slice(c, qkv, 0, i * config.dim, (i + 1) * config.dim), config.head_dim, config.dim / config.head_dim, x->ne[1], x->ne[2]);
+            };
+            auto q = norm("q_norm", split(0));
+            auto k = norm("k_norm", split(1));
+            auto v = split(2);
+            y      = Rope::attention(ctx, q, k, v, pe, nullptr, 1.f, false);
+            y      = linear("proj", y);
+            x      = ggml_add(c, x, y);
+            y      = norm("norm2", x);
+            y      = situ(c, linear("ffn_up", y), linear("ffn_gate", y));
+            y      = linear("ffn_down", y);
+            return ggml_add(c, x, y);
+        }
+    };
+
+    struct Block : public GGMLBlock {
+        MOSSImageConfig config;
+
+        Block(const MOSSImageConfig& config)
+            : config(config) {
+            blocks["norm1"]            = std::make_shared<RMSNorm>(config.dim, 1e-6f);
+            blocks["norm2"]            = std::make_shared<RMSNorm>(config.dim, 1e-6f);
+            blocks["q_norm"]           = std::make_shared<RMSNorm>(config.head_dim, 1e-6f);
+            blocks["k_norm"]           = std::make_shared<RMSNorm>(config.head_dim, 1e-6f);
+            blocks["proj.qkv"]         = std::make_shared<Linear>(config.dim, config.dim * 3, false);
+            blocks["proj.o"]           = std::make_shared<Linear>(config.dim, config.dim, false);
+            blocks["proj.ffn_up_gate"] = std::make_shared<Linear>(config.dim, config.ffn_dim * 2, false);
+            blocks["proj.ffn_down"]    = std::make_shared<Linear>(config.ffn_dim, config.dim, false);
+        }
+
+        ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x, ggml_tensor* pe, ggml_tensor* mod) {
+            auto c      = ctx->ggml_ctx;
+            auto linear = [&](const char* name, ggml_tensor* h) { return std::dynamic_pointer_cast<Linear>(blocks[name])->forward(ctx, h); };
+            auto norm   = [&](const char* name, ggml_tensor* h) { return std::dynamic_pointer_cast<RMSNorm>(blocks[name])->forward(ctx, h); };
+            auto part   = [&](int i) { return ggml_ext_slice(c, mod, 0, i * config.dim, (i + 1) * config.dim); };
+            auto y      = norm("norm1", x);
+            y           = ggml_add(c, ggml_add(c, y, ggml_mul(c, y, part(1))), part(0));
+            auto qkv    = linear("proj.qkv", y);
+            auto split  = [&](int i) {
+                return ggml_reshape_4d(c, ggml_ext_slice(c, qkv, 0, i * config.dim, (i + 1) * config.dim), config.head_dim, config.dim / config.head_dim, x->ne[1], x->ne[2]);
+            };
+            auto q       = norm("q_norm", split(0));
+            auto k       = norm("k_norm", split(1));
+            auto v       = split(2);
+            y            = Rope::attention(ctx, q, k, v, pe, nullptr, 1.f, false);
+            y            = linear("proj.o", y);
+            y            = ggml_mul(c, y, part(2));
+            x            = ggml_add(c, x, y);
+            y            = norm("norm2", x);
+            y            = ggml_add(c, ggml_add(c, y, ggml_mul(c, y, part(4))), part(3));
+            auto up_gate = linear("proj.ffn_up_gate", y);
+            y            = situ(c, ggml_ext_slice(c, up_gate, 0, 0, config.ffn_dim), ggml_ext_slice(c, up_gate, 0, config.ffn_dim, 2 * config.ffn_dim));
+            y            = linear("proj.ffn_down", y);
+            y            = ggml_mul(c, y, part(5));
+            return ggml_add(c, x, y);
+        }
+    };
+
+    struct Model : public GGMLBlock {
+        MOSSImageConfig config;
+
+        Model(const MOSSImageConfig& config)
+            : config(config) {
+            blocks["video_embedder"]     = std::make_shared<Linear>(config.channels, config.dim, true);
+            blocks["text_embedder.proj"] = std::make_shared<Linear>(config.text_dim, config.dim, true);
+            for (int i = 0; i < 2; ++i) {
+                blocks["text_embedder.blocks." + std::to_string(i)] = std::make_shared<TextBlock>(config);
+            }
+            blocks["time_embedding.0"]        = std::make_shared<Linear>(config.freq_dim, config.dim, true);
+            blocks["time_embedding.2"]        = std::make_shared<Linear>(config.dim, config.dim, true);
+            blocks["shared_block_modulation"] = std::make_shared<Linear>(config.dim, 6 * config.dim, true);
+            blocks["video_head_modulation"]   = std::make_shared<Linear>(config.dim, 2 * config.dim, true);
+            for (int64_t i = 0; i < config.depth; ++i) {
+                blocks["layers." + std::to_string(i)] = std::make_shared<Block>(config);
+            }
+            blocks["final_norm_video"]   = std::make_shared<RMSNorm>(config.dim, 1e-6f);
+            blocks["final_linear_video"] = std::make_shared<Linear>(config.dim, config.channels, false);
+        }
+
+        ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* image, ggml_tensor* time, ggml_tensor* text, ggml_tensor* pe, const std::vector<int>& skip) {
+            auto c            = ctx->ggml_ctx;
+            auto linear       = [&](const char* name, ggml_tensor* h) { return std::dynamic_pointer_cast<Linear>(blocks[name])->forward(ctx, h); };
+            int64_t image_len = image->ne[0] * image->ne[1];
+            auto x            = linear("video_embedder", DiT::patchify(c, image, 1, 1));
+            text              = linear("text_embedder.proj", text);
+            auto text_pe      = ggml_ext_slice(c, pe, 3, image_len, image_len + text->ne[1]);
+            for (int i = 0; i < 2; ++i) {
+                text = std::dynamic_pointer_cast<TextBlock>(blocks["text_embedder.blocks." + std::to_string(i)])->forward(ctx, text, text_pe);
+            }
+            time      = linear("time_embedding.2", ggml_silu(c, linear("time_embedding.0", time)));
+            auto mod  = linear("shared_block_modulation", time);
+            auto head = linear("video_head_modulation", time);
+            x         = ggml_concat(c, x, text, 1);
+            for (int64_t i = 0; i < config.depth; ++i) {
+                if (std::find(skip.begin(), skip.end(), i) == skip.end()) {
+                    x = std::dynamic_pointer_cast<Block>(blocks["layers." + std::to_string(i)])->forward(ctx, x, pe, mod);
+                    sd::ggml_graph_cut::mark_graph_cut(x, "moss_image.layers." + std::to_string(i), "x");
+                }
+            }
+            x          = ggml_ext_slice(c, x, 1, 0, image_len);
+            x          = std::dynamic_pointer_cast<RMSNorm>(blocks["final_norm_video"])->forward(ctx, x);
+            auto shift = ggml_ext_slice(c, head, 0, 0, config.dim);
+            auto scale = ggml_ext_slice(c, head, 0, config.dim, 2 * config.dim);
+            x          = ggml_add(c, ggml_add(c, x, ggml_mul(c, x, scale)), shift);
+            x          = linear("final_linear_video", x);
+            return DiT::unpatchify(c, x, image->ne[1], image->ne[0], 1, 1);
+        }
+    };
+
+    struct MOSSImageRunner : public DiffusionModelRunner {
+        MOSSImageConfig config;
+        Model model;
+        std::vector<float> pe_data, time_data;
+
+        MOSSImageRunner(ggml_backend_t backend, const String2TensorStorage& weights, const std::string& prefix, std::shared_ptr<RunnerWeightManager> weight_manager = nullptr)
+            : DiffusionModelRunner(backend, prefix, weight_manager), config(MOSSImageConfig::detect_from_weights(weights, prefix)), model(config) {
+            model.init(params_ctx, weights, prefix);
+        }
+
+        std::string get_desc() override { return "MOSS-Image"; }
+        void get_param_tensors(std::map<std::string, ggml_tensor*>& tensors, const std::string& prefix) override {
+            model.get_param_tensors(tensors, prefix);
+        }
+
+        sd::Tensor<float> compute(int n_threads, const DiffusionParams& inputs) override {
+            if (!inputs.x || !inputs.context || !inputs.timesteps || inputs.context->empty()) {
+                LOG_ERROR("MOSS-Image requires latents, timesteps and text features");
+                return {};
+            }
+            std::vector<int> skip;
+            if (auto extra = std::get_if<SkipLayerDiffusionExtra>(&inputs.extra)) {
+                if (extra->skip_layers) {
+                    skip = *extra->skip_layers;
+                }
+            }
+            auto graph = [&]() {
+                auto gf   = new_graph_custom(65536);
+                auto x    = make_input(*inputs.x);
+                auto text = make_input(*inputs.context);
+                GGML_ASSERT(x->ne[3] == 1 && x->ne[2] == config.channels && text->ne[0] == config.text_dim);
+                int64_t h = x->ne[1], w = x->ne[0], nt = text->ne[1];
+                Rope::Embedding embedding;
+                embedding.positions.append_image(static_cast<int>(h), static_cast<int>(w));
+                embedding.positions.images.back().height_axis = 2;
+                embedding.positions.images.back().width_axis  = 3;
+                embedding.positions.append_tokens(nt);
+                for (int64_t token = 0; token < h * w + nt; ++token) {
+                    float pos[4] = {0.f, float(token - h * w), 0.f, 0.f};
+                    if (token < h * w) {
+                        pos[1] = float(nt);
+                        pos[2] = float(token / w) - float(h - 1) / 2.f;
+                        pos[3] = float(token % w) - float(w - 1) / 2.f;
+                    }
+                    embedding.ids.push_back({pos[0], pos[1], pos[2], pos[3]});
+                }
+                embedding.values = Rope::embed_nd(embedding.ids, 1, 10000.f, std::vector<int>(4, static_cast<int>(config.head_dim / 4)), embedding.layout, &embedding.frequencies);
+                pe_data          = finish_rope_pe(std::move(embedding));
+                time_data.resize(config.freq_dim);
+                for (int64_t i = 0; i < config.freq_dim / 2; ++i) {
+                    double angle                       = inputs.timesteps->values()[0] * std::pow(10000.0, -double(i) / double(config.freq_dim / 2));
+                    time_data[i]                       = float(std::cos(angle));
+                    time_data[i + config.freq_dim / 2] = float(std::sin(angle));
+                }
+                auto pe   = ggml_new_tensor_4d(compute_ctx, GGML_TYPE_F32, 2, 2, config.head_dim / 2, h * w + nt);
+                auto time = ggml_new_tensor_3d(compute_ctx, GGML_TYPE_F32, config.freq_dim, 1, 1);
+                set_backend_tensor_data(pe, pe_data.data());
+                set_backend_tensor_data(time, time_data.data());
+                auto ctx = get_context();
+                ggml_build_forward_expand(gf, model.forward(&ctx, x, time, text, pe, skip));
+                return gf;
+            };
+            return restore_trailing_singleton_dims(GGMLRunner::compute(graph, n_threads, false), inputs.x->dim());
+        }
+    };
+}  // namespace MOSSImage
+
+#endif  // __SD_MODEL_DIFFUSION_MOSS_IMAGE_HPP__
@@ -43,6 +43,7 @@ namespace LLM {
        QWEN2_5_VL,
        QWEN3,
        QWEN3_VL,
+        QWEN3_5,
        MISTRAL_SMALL_3_2,
        MINISTRAL_3_3B,
        GEMMA3_12B,
@@ -59,6 +60,7 @@ namespace LLM {
        "qwen2.5vl",
        "qwen3",
        "qwen3vl",
+        "qwen3.5",
        "mistral_small3.2",
        "ministral3.3b",
        "gemma3_12b",
@@ -121,6 +123,7 @@ namespace LLM {
        bool qk_norm                    = false;
        bool final_norm                 = true;
        bool rms_norm_add               = false;
+        bool qwen35_gguf                = false;
        bool normalize_input            = false;
        int64_t vocab_size              = 152064;
        int64_t max_position_embeddings = 128000;
@@ -134,6 +137,11 @@ namespace LLM {
        bool qkv_fused              = false;
        bool bidirectional          = false;
        float partial_rotary        = 1.f;
+        int linear_head_dim         = 128;
+        int linear_key_heads        = 16;
+        int linear_value_heads      = 32;
+        int linear_conv_kernel      = 4;
+        std::set<int> linear_attention_layers;

        // DeepSeek-V3-style grouped-sigmoid MoE routing (LLaDA2)
        int64_t moe_intermediate_size = 0;
@@ -167,6 +175,19 @@ namespace LLM {
                config.num_kv_heads = 8;
                config.qkv_bias     = false;
                config.rms_norm_eps = 1e-5f;
+            } else if (arch == LLMArch::QWEN3_5) {
+                config.hidden_size             = 4096;
+                config.intermediate_size       = 12288;
+                config.head_dim                = 256;
+                config.num_heads               = 16;
+                config.num_kv_heads            = 4;
+                config.qkv_bias                = false;
+                config.qk_norm                 = true;
+                config.rms_norm_add            = true;
+                config.partial_rotary          = 0.25f;
+                config.rope_thetas             = {10000000.f};
+                config.max_position_embeddings = 262144;
+                config.vocab_size              = 248320;
            } else if (arch == LLMArch::QWEN3 || arch == LLMArch::QWEN3_VL) {
                config.head_dim     = 128;
                config.num_heads    = 32;
@@ -365,6 +386,41 @@ namespace LLM {
            if ((arch == LLMArch::QWEN3 || arch == LLMArch::QWEN3_VL) && config.num_layers == 28) {
                config.num_heads = 16;
            }
+            if (arch == LLMArch::QWEN3_5) {
+                config.qwen35_gguf  = tensor_storage_map.count(prefix + ".model.layers.0.linear_attn.A") > 0;
+                config.rms_norm_add = !config.qwen35_gguf;
+                for (const auto& [name, tensor] : tensor_storage_map) {
+                    if (!starts_with(name, prefix + ".")) {
+                        continue;
+                    }
+                    if (ends_with(name, "self_attn.q_norm.weight")) {
+                        config.head_dim = static_cast<int>(tensor.ne[0]);
+                    } else if (ends_with(name, "linear_attn.norm.weight")) {
+                        config.linear_head_dim = static_cast<int>(tensor.ne[0]);
+                    } else if (ends_with(name, "linear_attn.conv1d.weight")) {
+                        config.linear_conv_kernel = static_cast<int>(tensor.ne[0]);
+                        const auto pos            = name.find("layers.");
+                        config.linear_attention_layers.insert(std::stoi(name.substr(pos + 7)));
+                    }
+                }
+                for (const auto& [name, tensor] : tensor_storage_map) {
+                    if (!starts_with(name, prefix + ".")) {
+                        continue;
+                    }
+                    if (ends_with(name, "self_attn.q_proj.weight")) {
+                        config.num_heads = static_cast<int>(tensor.ne[1] / (2 * config.head_dim));
+                    } else if (ends_with(name, "self_attn.k_proj.weight")) {
+                        config.num_kv_heads = static_cast<int>(tensor.ne[1] / config.head_dim);
+                    } else if (ends_with(name, "linear_attn.in_proj_z.weight")) {
+                        config.linear_value_heads = static_cast<int>(tensor.ne[1] / config.linear_head_dim);
+                    }
+                }
+                for (const auto& [name, tensor] : tensor_storage_map) {
+                    if (starts_with(name, prefix + ".") && ends_with(name, "linear_attn.in_proj_qkv.weight")) {
+                        config.linear_key_heads = static_cast<int>((tensor.ne[1] / config.linear_head_dim - config.linear_value_heads) / 2);
+                    }
+                }
+            }
            if (arch == LLMArch::QWEN3_VL &&
                (config.num_layers == 50 || config.num_layers == 64) &&
                config.hidden_size == 5120) {
@@ -1390,6 +1446,81 @@ namespace LLM {
        }
    };

+    struct Qwen35DeltaNet : public UnaryBlock {
+        int head_dim;
+        int key_heads;
+        int value_heads;
+        int conv_dim;
+        int kernel;
+        bool gguf_layout;
+
+        Qwen35DeltaNet(const LLMConfig& config)
+            : head_dim(config.linear_head_dim), key_heads(config.linear_key_heads), value_heads(config.linear_value_heads), conv_dim(head_dim * (key_heads * 2 + value_heads)), kernel(config.linear_conv_kernel), gguf_layout(config.qwen35_gguf) {
+            auto hidden_size      = config.hidden_size;
+            blocks["in_proj_qkv"] = std::make_shared<Linear>(hidden_size, conv_dim, false);
+            blocks["in_proj_z"]   = std::make_shared<Linear>(hidden_size, head_dim * value_heads, false);
+            blocks["in_proj_a"]   = std::make_shared<Linear>(hidden_size, value_heads, false);
+            blocks["in_proj_b"]   = std::make_shared<Linear>(hidden_size, value_heads, false);
+            blocks["out_proj"]    = std::make_shared<Linear>(head_dim * value_heads, hidden_size, false);
+            blocks["norm"]        = std::make_shared<LLMRMSNorm>(head_dim, 1e-6f);
+        }
+
+        void init_params(ggml_context* ctx, const String2TensorStorage& = {}, std::string = "") override {
+            params["conv1d.weight"]             = gguf_layout ? ggml_new_tensor_2d(ctx, GGML_TYPE_F32, kernel, conv_dim) : ggml_new_tensor_3d(ctx, GGML_TYPE_F32, kernel, 1, conv_dim);
+            params[gguf_layout ? "A" : "A_log"] = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, value_heads);
+            params["dt_bias"]                   = ggml_new_tensor_1d(ctx, GGML_TYPE_F32, value_heads);
+        }
+
+        ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x) override {
+            auto c          = ctx->ggml_ctx;
+            const int64_t n = x->ne[1];
+            GGML_ASSERT(x->ne[2] == 1);
+            auto linear        = [&](const char* name) { return std::dynamic_pointer_cast<Linear>(blocks[name])->forward(ctx, x); };
+            auto qkv           = linear("in_proj_qkv");
+            auto padded        = ggml_pad_ext(c, qkv, 0, 0, kernel - 1, 0, 0, 0, 0, 0);
+            ggml_tensor* mixed = nullptr;
+            // Causal depthwise convolution, with zero state for every independent prompt.
+            for (int i = 0; i < kernel; ++i) {
+                auto w    = ggml_cont(c, ggml_view_2d(c, params["conv1d.weight"], 1, conv_dim, kernel * sizeof(float), i * sizeof(float)));
+                w         = ggml_reshape_1d(c, w, conv_dim);
+                auto part = ggml_mul(c, ggml_ext_slice(c, padded, 1, i, i + n), w);
+                mixed     = mixed ? ggml_add(c, mixed, part) : part;
+            }
+            mixed      = ggml_silu(c, mixed);
+            auto heads = [&](int start, int count) {
+                auto h = ggml_ext_slice(c, mixed, 0, start, start + head_dim * count);
+                return ggml_reshape_4d(c, h, head_dim, count, n, 1);
+            };
+            auto repeat_keys = [&](ggml_tensor* h) {
+                h = ggml_scale(c, ggml_rms_norm(c, h, 1e-6f / head_dim), 1.f / std::sqrt(float(head_dim)));
+                // llama.cpp reorders V heads and stores negative exp(A_log) and offset norms.
+                if (gguf_layout) {
+                    return h;
+                }
+                // ggml GDN broadcasts heads cyclically; Qwen uses adjacent repeated heads.
+                h          = ggml_reshape_4d(c, h, head_dim, 1, key_heads, n);
+                auto shape = ggml_new_tensor_4d(c, GGML_TYPE_F32, head_dim, value_heads / key_heads, key_heads, n);
+                return ggml_reshape_4d(c, ggml_repeat(c, h, shape), head_dim, value_heads, n, 1);
+            };
+            auto q     = repeat_keys(heads(0, key_heads));
+            auto k     = repeat_keys(heads(head_dim * key_heads, key_heads));
+            auto v     = heads(2 * head_dim * key_heads, value_heads);
+            auto beta  = ggml_reshape_4d(c, ggml_sigmoid(c, linear("in_proj_b")), 1, value_heads, n, 1);
+            auto decay = ggml_softplus(c, ggml_add(c, linear("in_proj_a"), params["dt_bias"]));
+            auto a     = gguf_layout ? params["A"] : ggml_neg(c, ggml_exp(c, params["A_log"]));
+            decay      = ggml_mul(c, decay, a);
+            decay      = ggml_reshape_4d(c, decay, 1, value_heads, n, 1);
+            auto state = ggml_ext_zeros(c, head_dim, head_dim, value_heads, 1);
+            auto out   = ggml_gated_delta_net(c, q, k, v, decay, beta, state, 1);
+            out        = ggml_view_3d(c, out, head_dim, value_heads, n, head_dim * sizeof(float), head_dim * value_heads * sizeof(float), 0);
+            out        = std::dynamic_pointer_cast<LLMRMSNorm>(blocks["norm"])->forward(ctx, out);
+            auto z     = ggml_reshape_3d(c, linear("in_proj_z"), head_dim, value_heads, n);
+            out        = ggml_mul(c, out, ggml_silu(c, z));
+            out        = ggml_reshape_3d(c, out, head_dim * value_heads, n, 1);
+            return std::dynamic_pointer_cast<Linear>(blocks["out_proj"])->forward(ctx, out);
+        }
+    };
+
    struct Attention : public GGMLBlock {
    protected:
        LLMArch arch;
@@ -1439,7 +1570,7 @@ namespace LLM {
                GGML_ASSERT(!k_eq_v);
                blocks["query_key_value"] = std::make_shared<Linear>(config.hidden_size, (num_heads + num_kv_heads * 2) * head_dim, config.qkv_bias);
            } else {
-                blocks["q_proj"] = std::make_shared<Linear>(config.hidden_size, num_heads * head_dim, config.qkv_bias);
+                blocks["q_proj"] = std::make_shared<Linear>(config.hidden_size, num_heads * head_dim * (arch == LLMArch::QWEN3_5 ? 2 : 1), config.qkv_bias);
                blocks["k_proj"] = std::make_shared<Linear>(config.hidden_size, num_kv_heads * head_dim, config.qkv_bias);
                if (!k_eq_v) {
                    blocks["v_proj"] = std::make_shared<Linear>(config.hidden_size, num_kv_heads * head_dim, config.qkv_bias);
@@ -1502,6 +1633,13 @@ namespace LLM {
                v = k_eq_v ? k : v_proj->forward(ctx, x);  // [N, n_token, num_kv_heads*head_dim]
            }

+            ggml_tensor* output_gate = nullptr;
+            if (arch == LLMArch::QWEN3_5) {
+                q           = ggml_reshape_4d(ctx->ggml_ctx, q, head_dim * 2, num_heads, n_token, N);
+                output_gate = ggml_ext_slice(ctx->ggml_ctx, q, 0, head_dim, head_dim * 2);
+                output_gate = ggml_reshape_3d(ctx->ggml_ctx, output_gate, head_dim * num_heads, n_token, N);
+                q           = ggml_ext_slice(ctx->ggml_ctx, q, 0, 0, head_dim);
+            }
            q = ggml_reshape_4d(ctx->ggml_ctx, q, head_dim, num_heads, n_token, N);     // [N, n_token, num_heads, head_dim]
            k = ggml_reshape_4d(ctx->ggml_ctx, k, head_dim, num_kv_heads, n_token, N);  // [N, n_token, num_kv_heads, head_dim]
            v = ggml_reshape_4d(ctx->ggml_ctx, v, head_dim, num_kv_heads, n_token, N);  // [N, n_token, num_kv_heads, head_dim]
@@ -1658,7 +1796,7 @@ namespace LLM {
                                  1.f,
                                  32.f,
                                  1.f);
-            } else if (arch == LLMArch::LLADA2_MOE) {
+            } else if (arch == LLMArch::LLADA2_MOE || arch == LLMArch::QWEN3_5) {
                // LLaDA2 slices the head (query[..., :rotary_dim]) instead of zero-padding
                // inv_freq like gemma does, so rotate_half pairs i with i + rotary_dim/2 and the
                // frequencies use rotary_dim as the exponent denominator. Passing n_dims =
@@ -1734,6 +1872,9 @@ namespace LLM {
                x = ggml_ext_attention_ext(ctx, q, k, v, num_heads, attention_mask, true, ctx->flash_attn_enabled);  // [N, n_token, hidden_size]
            }

+            if (output_gate) {
+                x = ggml_mul(ctx->ggml_ctx, x, ggml_sigmoid(ctx->ggml_ctx, output_gate));
+            }
            x = out_proj->forward(ctx, x);  // [N, n_token, hidden_size]
            return x;
        }
@@ -1783,7 +1924,11 @@ namespace LLM {
            if (!config.sliding_attention.empty()) {
                sliding_attention = config.sliding_attention[layer_index % config.sliding_attention.size()];
            }
-            blocks["self_attn"] = std::make_shared<Attention>(config, sliding_attention == 0);
+            if (config.arch == LLMArch::QWEN3_5 && config.linear_attention_layers.count(layer_index)) {
+                blocks["linear_attn"] = std::make_shared<Qwen35DeltaNet>(config);
+            } else {
+                blocks["self_attn"] = std::make_shared<Attention>(config, sliding_attention == 0);
+            }
            if (config.arch == LLMArch::GPT_OSS_20B) {
                blocks["mlp"] = std::make_shared<GPTOSSMLP>(config);
            } else if ((config.arch == LLMArch::LLADA2_MOE || config.arch == LLMArch::BAILING_MOE) && layer_index >= config.first_k_dense_replace) {
@@ -1813,7 +1958,7 @@ namespace LLM {
                             ggml_tensor* rope_cos               = nullptr,
                             ggml_tensor* rope_sin               = nullptr) {
            // x: [N, n_token, hidden_size]
-            auto self_attn                                  = std::dynamic_pointer_cast<Attention>(blocks["self_attn"]);
+            auto self_attn                                  = blocks.count("self_attn") ? std::dynamic_pointer_cast<Attention>(blocks["self_attn"]) : nullptr;
            auto input_layernorm                            = std::dynamic_pointer_cast<LLMRMSNorm>(blocks["input_layernorm"]);
            auto pre_ffw_norm                               = std::dynamic_pointer_cast<LLMRMSNorm>(blocks[pre_ffw_norm_name]);
            std::shared_ptr<LLMRMSNorm> post_attention_norm = nullptr;
@@ -1833,7 +1978,11 @@ namespace LLM {

            auto residual = x;
            x             = input_layernorm->forward(ctx, x);
-            x             = self_attn->forward(ctx, x, input_pos, block_attention_mask, rope_index, rope_cos, rope_sin);
+            if (blocks.count("linear_attn")) {
+                x = std::dynamic_pointer_cast<Qwen35DeltaNet>(blocks["linear_attn"])->forward(ctx, x);
+            } else {
+                x = self_attn->forward(ctx, x, input_pos, block_attention_mask, rope_index, rope_cos, rope_sin);
+            }
            if (post_attention_norm != nullptr) {
                x = post_attention_norm->forward(ctx, x);
            }
@@ -2338,7 +2487,8 @@ namespace LLM {
                                 const DeepStackImageEmbeds& deepstack_image_embeds_tensor,
                                 const std::vector<ImageGrid>& image_grids,
                                 std::set<int> out_layers,
-                                 bool return_all_hidden_states = false) {
+                                 bool return_all_hidden_states           = false,
+                                 const sd::Tensor<int32_t>& position_ids = {}) {
            ggml_cgraph* gf        = new_graph_custom(LLM_GRAPH_SIZE);
            ggml_tensor* input_ids = make_input(input_ids_tensor);
            std::vector<std::pair<int, ggml_tensor*>> image_embeds;
@@ -2359,6 +2509,7 @@ namespace LLM {
            if (config.arch == LLMArch::MISTRAL_SMALL_3_2 ||
                config.arch == LLMArch::MINISTRAL_3_3B ||
                config.arch == LLMArch::QWEN3 ||
+                config.arch == LLMArch::QWEN3_5 ||
                config.arch == LLMArch::GEMMA3_12B ||
                config.arch == LLMArch::GEMMA4_12B ||
                config.arch == LLMArch::GEMMA2_2B ||
@@ -2402,6 +2553,10 @@ namespace LLM {
                }
            }

+            if (!position_ids.empty()) {
+                GGML_ASSERT(position_ids.numel() == static_cast<int64_t>(input_pos_vec.size()));
+                input_pos_vec = position_ids.values();
+            }
            auto input_pos = ggml_new_tensor_1d(compute_ctx,
                                                GGML_TYPE_I32,
                                                input_pos_vec.size());
@@ -2475,7 +2630,8 @@ namespace LLM {
                                  bool return_all_hidden_states                      = false,
                                  bool auto_runner_end                               = true,
                                  const DeepStackImageEmbeds& deepstack_image_embeds = {},
-                                  const std::vector<ImageGrid>& image_grids          = {}) {
+                                  const std::vector<ImageGrid>& image_grids          = {},
+                                  const sd::Tensor<int32_t>& position_ids            = {}) {
            auto get_graph = [&]() -> ggml_cgraph* {
                return build_graph(input_ids,
                                   attention_mask,
@@ -2483,7 +2639,8 @@ namespace LLM {
                                   deepstack_image_embeds,
                                   image_grids,
                                   out_layers,
-                                   return_all_hidden_states);
+                                   return_all_hidden_states,
+                                   position_ids);
            };
            return restore_trailing_singleton_dims(GGMLRunner::compute(get_graph, n_threads, auto_runner_end),
                                                   input_ids.dim() + 1);
@@ -363,33 +363,22 @@ namespace MiniMaxH3 {
            auto pre       = std::dynamic_pointer_cast<AudioAttentionProjection>(blocks["pre_block"]);
            auto mean_proj = std::dynamic_pointer_cast<LTXV::Conv1D>(blocks["mean_proj"]);

-            // GGML's batched conv1d storage interleaves the stream dimension
-            // with output channels. Subsequent layers then read stereo samples
-            // as adjacent feature channels. Run each mono stream independently,
-            // matching PyTorch's reshape(B*S, 1, samples), and concatenate only
-            // the completed normalized latents.
            const int64_t streams = waveform->ne[2] * waveform->ne[3];
            waveform              = ggml_reshape_3d(ctx->ggml_ctx,
                                                    waveform,
                                                    waveform->ne[0],
                                                    1,
                                                    streams);
-            ggml_tensor* stereo_z = nullptr;
-            for (int64_t stream = 0; stream < streams; ++stream) {
-                auto mono = ggml_ext_slice(ctx->ggml_ctx, waveform, 2, stream, stream + 1);
-                auto x    = encoder->forward(ctx, mono);
-                x         = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, x, 1, 0, 2, 3));
-                x         = pre->forward(ctx, x);
-                x         = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, x, 1, 0, 2, 3));
-                auto z    = mean_proj->forward(ctx, x);
+            auto x                = encoder->forward(ctx, waveform);
+            x                     = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, x, 1, 0, 2, 3));
+            x                     = pre->forward(ctx, x);
+            x                     = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, x, 1, 0, 2, 3));
+            auto z                = mean_proj->forward(ctx, x);

-                auto mean = ggml_reshape_4d(ctx->ggml_ctx, params["latents_mean"], 1, kLatentChannels, 1, 1);
-                auto std  = ggml_reshape_4d(ctx->ggml_ctx, params["latents_std"], 1, kLatentChannels, 1, 1);
-                z         = ggml_div(ctx->ggml_ctx, ggml_sub(ctx->ggml_ctx, z, mean), std);
-                z         = ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, z, 0, 2, 1, 3));
-                stereo_z  = stereo_z == nullptr ? z : ggml_concat(ctx->ggml_ctx, stereo_z, z, 1);
-            }
-            return stereo_z;
+            auto mean = ggml_reshape_4d(ctx->ggml_ctx, params["latents_mean"], 1, kLatentChannels, 1, 1);
+            auto std  = ggml_reshape_4d(ctx->ggml_ctx, params["latents_std"], 1, kLatentChannels, 1, 1);
+            z         = ggml_div(ctx->ggml_ctx, ggml_sub(ctx->ggml_ctx, z, mean), std);
+            return ggml_cont(ctx->ggml_ctx, ggml_permute(ctx->ggml_ctx, z, 0, 2, 1, 3));
        }

        ggml_tensor* decode(GGMLRunnerContext* ctx, ggml_tensor* latent) {
@@ -3,6 +3,7 @@
 #include <algorithm>
 #include <cstdint>
 #include <cstdio>
+#include <filesystem>
 #include <fstream>
 #include <ostream>
 #include <string>
@@ -19,7 +20,7 @@ static void set_error(std::string* error, const std::string& message) {
 }

 bool is_gguf_file(const std::string& file_path) {
-    std::ifstream file(file_path, std::ios::binary);
+    std::ifstream file(std::filesystem::u8path(file_path), std::ios::binary);
    if (!file.is_open()) {
        return false;
    }
@@ -178,7 +179,7 @@ bool GGUFStreamingWriter::write_metadata(const std::string& file_path,
    }

    LOG_INFO("trying to save tensors to %s", file_path.c_str());
-    FILE* file = fopen(file_path.c_str(), "wb+");
+    FILE* file = ggml_fopen(file_path.c_str(), "wb+");
    if (file == nullptr) {
        set_error(error, "failed to open output file '" + file_path + "'");
        close();
@@ -2,6 +2,7 @@
 #define __SD_MODEL_IO_GGUF_READER_EXT_H__

 #include <cstdint>
+#include <filesystem>
 #include <fstream>
 #include <string>
 #include <vector>
@@ -184,7 +185,7 @@ private:

 public:
    bool load(const std::string& file_path) {
-        std::ifstream fin(file_path, std::ios::binary | std::ios::ate);
+        std::ifstream fin(std::filesystem::u8path(file_path), std::ios::binary | std::ios::ate);
        if (!fin) {
            LOG_ERROR("failed to open '%s'", file_path.c_str());
            return false;
@@ -26,15 +26,15 @@ static void set_error(std::string* error, const std::string& message) {
 }

 static std::string resolve_index_shard_path(const std::string& index_path, const std::string& shard_path) {
-    fs::path shard_fs_path(shard_path);
+    fs::path shard_fs_path = fs::u8path(shard_path);
    if (shard_fs_path.is_absolute()) {
-        return shard_fs_path.lexically_normal().string();
+        return shard_fs_path.lexically_normal().u8string();
    }
-    return (fs::path(index_path).parent_path() / shard_fs_path).lexically_normal().string();
+    return (fs::u8path(index_path).parent_path() / shard_fs_path).lexically_normal().u8string();
 }

 bool is_safetensors_file(const std::string& file_path) {
-    std::ifstream file(file_path, std::ios::binary);
+    std::ifstream file(fs::u8path(file_path), std::ios::binary);
    if (!file.is_open()) {
        return false;
    }
@@ -153,7 +153,7 @@ bool read_safetensors_file(const std::string& file_path,
                           std::string* error,
                           std::map<std::string, std::string>* metadata,
                           std::set<std::string>* tensor_names) {
-    std::ifstream file(file_path, std::ios::binary);
+    std::ifstream file(fs::u8path(file_path), std::ios::binary);
    if (!file.is_open()) {
        set_error(error, "failed to open '" + file_path + "'");
        return false;
@@ -418,7 +418,7 @@ bool read_safetensors_index_file(const std::string& file_path,
                                 std::string* error) {
    shard_paths.clear();

-    std::ifstream file(file_path);
+    std::ifstream file(fs::u8path(file_path));
    if (!file.is_open()) {
        set_error(error, "failed to open '" + file_path + "'");
        return false;
@@ -522,7 +522,7 @@ bool write_safetensors_file(const std::string& file_path,

    const std::string header_str = header.dump();

-    std::ofstream file(file_path, std::ios::binary);
+    std::ofstream file(fs::u8path(file_path), std::ios::binary);
    if (!file.is_open()) {
        set_error(error, "failed to open '" + file_path + "' for writing");
        return false;
@@ -596,7 +596,7 @@ bool SafetensorsStreamingWriter::write_metadata(const std::string& file_path,
    data_start_                  = ST_HEADER_SIZE_LEN + header_str.size();

    LOG_INFO("trying to save tensors to %s", file_path.c_str());
-    std::ofstream file(file_path, std::ios::binary | std::ios::trunc);
+    std::ofstream file(fs::u8path(file_path), std::ios::binary | std::ios::trunc);
    if (!file.is_open()) {
        set_error(error, "failed to open '" + file_path + "' for writing");
        return false;
@@ -2,6 +2,7 @@

 #include <algorithm>
 #include <cstdint>
+#include <filesystem>
 #include <fstream>
 #include <string>
 #include <unordered_map>
@@ -98,7 +99,7 @@ static std::string torch_legacy_diagnostics(const std::string& file_path, const
 bool read_torch_legacy_file(const std::string& file_path,
                            std::vector<TensorStorage>& tensor_storages,
                            std::string* error) {
-    std::ifstream file(file_path, std::ios::binary);
+    std::ifstream file(std::filesystem::u8path(file_path), std::ios::binary);
    if (!file.is_open()) {
        set_error(error, "failed to open '" + file_path + "'");
        return false;
@@ -4,6 +4,7 @@
 #include <cinttypes>
 #include <cstdarg>
 #include <cstdlib>
+#include <filesystem>
 #include <fstream>
 #include <functional>
 #include <mutex>
@@ -333,7 +334,7 @@ bool ModelLoader::init_from_safetensors_index_file(const std::string& file_path,
        }
    }

-    std::ifstream index_file(file_path);
+    std::ifstream index_file(std::filesystem::u8path(file_path));
    const auto index = nlohmann::json::parse(index_file);
    for (const auto& entry : index.at("weight_map").items()) {
        const auto expected = (std::filesystem::u8path(file_path).parent_path() /
@@ -438,6 +439,16 @@ bool ModelLoader::init_from_diffusers_file(const std::string& file_path, const s
 SDVersion ModelLoader::get_sd_version() const {
    TensorStorage token_embedding_weight, input_block_weight, context_ebedding_weight;

+    if (tensor_storage_map.find("model.diffusion_model.emb_in.image.weight") != tensor_storage_map.end() &&
+        tensor_storage_map.find("model.diffusion_model.txt_mode_blocks.0.q_proj.weight") != tensor_storage_map.end()) {
+        return VERSION_FLUX3;
+    }
+
+    if (tensor_storage_map.count("model.diffusion_model.video_embedder.weight") &&
+        tensor_storage_map.count("model.diffusion_model.shared_block_modulation.weight")) {
+        return VERSION_MOSS_IMAGE;
+    }
+
    bool has_multiple_encoders = false;
    bool is_unet               = false;

@@ -1145,7 +1156,7 @@ bool ModelLoader::load_tensors(on_new_tensor_cb_t on_new_tensor_cb,
                        return;
                    }
                } else if (!mmapped) {
-                    file.open(file_path, std::ios::binary);
+                    file.open(std::filesystem::u8path(file_path), std::ios::binary);
                    if (!file.is_open()) {
                        LOG_ERROR("failed to open '%s'", file_path.c_str());
                        failed = true;
@@ -1663,6 +1663,26 @@ std::string convert_tensor_name(std::string name, SDVersion version) {

    replace_with_prefix_map(name, prefix_map);

+    if (version == VERSION_MOSS_IMAGE) {
+        const std::string te_prefix = "text_encoders.llm.";
+        if (starts_with(name, te_prefix + "blk.")) {
+            static const std::vector<std::pair<std::string, std::string>> qwen35_name_map = {
+                {"attn_post_norm.", "post_attention_layernorm."},
+                {"post_attention_norm.", "post_attention_layernorm."},
+                {"attn_qkv.", "linear_attn.in_proj_qkv."},
+                {"attn_gate.", "linear_attn.in_proj_z."},
+                {"ssm_conv1d.", "linear_attn.conv1d."},
+                {"ssm_dt.bias", "linear_attn.dt_bias"},
+                {"ssm_norm.", "linear_attn.norm."},
+                {"ssm_out.", "linear_attn.out_proj."},
+                {"ssm_alpha.", "linear_attn.in_proj_a."},
+                {"ssm_beta.", "linear_attn.in_proj_b."},
+                {"ssm_a", "linear_attn.A"},
+            };
+            replace_with_name_map(name, qwen35_name_map);
+        }
+    }
+
    if (version == VERSION_QWEN_IMAGE_2_1 || sd_version_is_boogu_image(version) || sd_version_is_krea2(version) || sd_version_is_mage_flow(version) || sd_version_is_minimax_h3(version)) {
        const std::string hf_vision_prefix = "text_encoders.llm.model.visual.";
        if (starts_with(name, hf_vision_prefix)) {
@@ -111,6 +111,8 @@ const char* model_version_to_str[] = {
    "Ming-Image",
    "Z-Image L2P",
    "Iris-3B",
+    "Flux.3 Image Klein",
+    "MOSS-Image",
 };

 static_assert(VERSION_COUNT == sizeof(model_version_to_str) / sizeof(model_version_to_str[0]),
@@ -1489,9 +1491,10 @@ bool StableDiffusionGGML::build_denoiser() {
                   sd_version_is_boogu_image(version) ||
                   sd_version_is_pid(version) ||
                   sd_version_is_ideogram4(version) ||
-                   version == VERSION_IRIS) {
+                   version == VERSION_IRIS ||
+                   version == VERSION_MOSS_IMAGE) {
            pred_type = FLOW_PRED;
-            if (sd_version_is_wan(version)) {
+            if (sd_version_is_wan(version) || version == VERSION_MOSS_IMAGE) {
                default_flow_shift = 5.f;
            } else if (sd_version_is_hunyuan_video(version)) {
                default_flow_shift = 7.f;
@@ -1514,6 +1517,9 @@ bool StableDiffusionGGML::build_denoiser() {
            } else {
                default_flow_shift = 3.f;
            }
+        } else if (sd_version_is_flux3(version)) {
+            pred_type          = FLUX_FLOW_PRED;
+            default_flow_shift = 6.93f;
        } else if (sd_version_is_flux(version) ||
                   sd_version_is_flux2(version) ||
                   sd_version_is_longcat(version) ||
@@ -1579,7 +1585,11 @@ bool StableDiffusionGGML::build_denoiser() {
        }
        case FLUX_FLOW_PRED: {
            LOG_INFO("running in Flux FLOW mode");
-            denoiser = std::make_shared<FluxFlowDenoiser>();
+            if (sd_version_is_flux3(version)) {
+                denoiser = std::make_shared<Flux3FlowDenoiser>();
+            } else {
+                denoiser = std::make_shared<FluxFlowDenoiser>();
+            }
            break;
        }
        case SEFI_FLOW_PRED: {
@@ -2558,6 +2568,13 @@ sd::Tensor<float> StableDiffusionGGML::sample(const std::shared_ptr<DiffusionMod
        if (sd_version_is_hunyuan_video(version) && step + 1 < sigmas.size()) {
            hunyuan_timestep_r_tensor = sd::Tensor<float>::from_vector({sigmas[step + 1]});
        }
+        sd::Tensor<float> flux3_target_timestep;
+        if (version == VERSION_FLUX3) {
+            // The sign marks intermediate evaluations; the magnitude is the one-based step.
+            const size_t target_index = static_cast<size_t>(std::abs(step));
+            GGML_ASSERT(target_index > 0 && target_index < sigmas.size());
+            flux3_target_timestep = sd::Tensor<float>::from_vector({sigmas[target_index]});
+        }
        sd::Tensor<float> noised_input = x * c_in;
        if (apply_denoise_mask && (version == VERSION_WAN2_2_TI2V || sd_version_is_ltxav(version) || sd_version_is_lingbot_video(version))) {
            noised_input = noised_input * denoise_mask + sampling_init_latent * (1.0f - denoise_mask);
@@ -2629,11 +2646,13 @@ sd::Tensor<float> StableDiffusionGGML::sample(const std::shared_ptr<DiffusionMod
                    unet_extra.ip_scale   = ip_adapter_strength;
                }
                diffusion_params.extra = unet_extra;
-            } else if (sd_version_is_sd3(version)) {
+            } else if (sd_version_is_sd3(version) || version == VERSION_MOSS_IMAGE) {
                diffusion_params.extra = SkipLayerDiffusionExtra{local_skip_layers};
            } else if (sd_version_is_flux(version) || sd_version_is_flux2(version) || sd_version_is_longcat(version) || sd_version_is_sefi_image(version)) {
                diffusion_params.extra = FluxDiffusionExtra{&guidance_tensor,
                                                            local_skip_layers};
+            } else if (sd_version_is_flux3(version)) {
+                diffusion_params.extra = Flux3DiffusionExtra{flux3_target_timestep.empty() ? nullptr : &flux3_target_timestep};
            } else if (version == VERSION_QWEN_IMAGE_2_1) {
                diffusion_params.extra = QwenImage21DiffusionExtra{&condition.c_token_types};
            } else if (sd_version_is_anima(version)) {
@@ -2856,9 +2875,12 @@ sd::Tensor<float> StableDiffusionGGML::sample(const std::shared_ptr<DiffusionMod
        if (uncond_out.empty() && !uncond.empty()) {
            if (!skip_uncond) {
                const std::vector<int>* uncond_skip_layers = nullptr;
+                const std::vector<int> moss_skip_layers    = {22};
                if (is_skiplayer_step && slg_uncond) {
                    LOG_VERBOSE("Skipping layers at uncond step %d\n", step);
                    uncond_skip_layers = &skip_layer_guidance.layers();
+                } else if (version == VERSION_MOSS_IMAGE) {
+                    uncond_skip_layers = &moss_skip_layers;
                }
                uncond_out = run_condition(uncond,
                                           uncond.c_concat.empty() ? nullptr : &uncond.c_concat,
@@ -3180,6 +3202,8 @@ std::string StableDiffusionGGML::get_default_ref_image_preset(SDVersion version)
        return "longcat";
    } else if (sd_version_is_flux(version)) {
        return "flux_kontext";
+    } else if (sd_version_is_flux3(version)) {
+        return "flux2";
    } else if (sd_version_is_flux2(version) || sd_version_is_sefi_image(version)) {
        return "flux2";
    } else if (version == VERSION_QWEN_IMAGE_LAYERED) {
@@ -15,6 +15,7 @@
 #include "model/diffusion/control.hpp"
 #include "model/diffusion/ernie_image.hpp"
 #include "model/diffusion/flux.hpp"
+#include "model/diffusion/flux3.hpp"
 #include "model/diffusion/hidream_o1.hpp"
 #include "model/diffusion/hunyuan.hpp"
 #include "model/diffusion/ideogram4.hpp"
@@ -30,6 +31,7 @@
 #include "model/diffusion/minit2i.hpp"
 #include "model/diffusion/mmdit.hpp"
 #include "model/diffusion/model.hpp"
+#include "model/diffusion/moss_image.hpp"
 #include "model/diffusion/pid.hpp"
 #include "model/diffusion/pixart.hpp"
 #include "model/diffusion/qwen_image.hpp"
@@ -173,6 +175,11 @@ namespace sd::model_builders {
                                                                  version,
                                                                  weight_manager,
                                                                  sd_ctx_params->model_args);
+        } else if (sd_version_is_flux3(version)) {
+            result.conditioner = std::make_shared<LLMEmbedder>(ctx.backends.runtime_backend(SDBackendModule::TE),
+                                                               tensor_storage_map, version, "", false, weight_manager, tokenizers);
+            result.diffusion   = std::make_shared<Flux3::Flux3Runner>(ctx.backends.runtime_backend(SDBackendModule::DIFFUSION),
+                                                                    tensor_storage_map, "model.diffusion_model", weight_manager);
        } else if (sd_version_is_flux2(version) || sd_version_is_sefi_image(version)) {
            bool is_chroma     = false;
            result.conditioner = std::make_shared<LLMEmbedder>(ctx.backends.runtime_backend(SDBackendModule::TE),
@@ -462,6 +469,11 @@ namespace sd::model_builders {
                                                               tensor_storage_map, version, "", false, weight_manager, tokenizers);
            result.diffusion   = std::make_shared<Iris::IrisRunner>(ctx.backends.runtime_backend(SDBackendModule::DIFFUSION),
                                                                  tensor_storage_map, "model.diffusion_model", weight_manager);
+        } else if (version == VERSION_MOSS_IMAGE) {
+            result.conditioner = std::make_shared<LLMEmbedder>(ctx.backends.runtime_backend(SDBackendModule::TE),
+                                                               tensor_storage_map, version, "", false, weight_manager, tokenizers);
+            result.diffusion   = std::make_shared<MOSSImage::MOSSImageRunner>(ctx.backends.runtime_backend(SDBackendModule::DIFFUSION),
+                                                                            tensor_storage_map, "model.diffusion_model", weight_manager);
        } else {  // SD1.x SD2.x SDXL
            std::map<std::string, std::string> embbeding_map;
            for (uint32_t i = 0; i < sd_ctx_params->embedding_count; i++) {
@@ -1452,6 +1452,29 @@ struct FluxFlowDenoiser : public DiscreteFlowDenoiser {
    }
 };

+struct Flux3FlowDenoiser : public FluxFlowDenoiser {
+    float t_to_sigma(float t) override {
+        float u = (t + 1.f) / TIMESTEPS;
+        return shift * u / (1.f + (shift - 1.f) * u);
+    }
+
+    std::vector<float> get_sigmas(uint32_t n, int image_seq_len, scheduler_t scheduler_type, SDVersion version, const char* extra_sample_args = nullptr) override {
+        if (scheduler_type != DISCRETE_SCHEDULER) {
+            return Denoiser::get_sigmas(n, image_seq_len, scheduler_type, version, extra_sample_args);
+        }
+        if (n == 0) {
+            return {};
+        }
+        std::vector<float> sigmas;
+        sigmas.reserve(n + 1);
+        for (uint32_t i = 0; i <= n; ++i) {
+            float u = 1.f - static_cast<float>(i) / n;
+            sigmas.push_back(shift * u / (1.f + (shift - 1.f) * u));
+        }
+        return sigmas;
+    }
+};
+
 struct SefiFlowDenoiser;

 struct SefiFlowDenoiser : public FluxFlowDenoiser {
@@ -10,6 +10,7 @@
 #include <cmath>
 #include <cstdlib>
 #include <cstring>
+#include <filesystem>

 static IMatrixCollector imatrix_collector;

@@ -227,7 +228,7 @@ void IMatrixCollector::save_imatrix(std::string fname, int ncall) const {
        LOG_WARN("storing only %zu out of %zu entries\n", to_store.size(), stats_.size());
    }

-    std::ofstream out(fname, std::ios::binary);
+    std::ofstream out(std::filesystem::u8path(fname), std::ios::binary);
    out.write((const char*)&n_entries, sizeof(n_entries));
    for (const auto& name : to_store) {
        const auto& stat = stats_.at(name);
@@ -251,7 +252,7 @@ void IMatrixCollector::save_imatrix(std::string fname, int ncall) const {
 }

 bool IMatrixCollector::load_imatrix(const char* fname) {
-    std::ifstream in(fname, std::ios::binary);
+    std::ifstream in(std::filesystem::u8path(fname), std::ios::binary);
    if (!in) {
        LOG_ERROR("failed to open %s\n", fname);
        return false;
@@ -4,6 +4,7 @@
 #include <array>
 #include <climits>
 #include <cstdlib>
+#include <filesystem>
 #include <fstream>
 #include <queue>
 #include <sstream>
@@ -495,7 +496,7 @@ struct HFTokenizer::Impl {

 HFTokenizer::HFTokenizer(const std::string& path)
    : impl_(new Impl) {
-    std::ifstream stream(path, std::ios::binary);
+    std::ifstream stream(std::filesystem::u8path(path), std::ios::binary);
    tokenizer_require(stream.good(), "cannot open " + path);
    TokenizerJSON config;
    stream >> config;
Author	SHA1	Message	Date
leejet	f0aac524d3	ci: align MUSA Docker image with upstream SDK 5.2.0 and mp_31	2026-10-17 14:14:29 +08:00
leejet	68285df319	feat: add moss-image support (#2148 )	2026-10-17 03:38:38 +08:00
leejet	af0fc72c96	chore: sync ggml and remove obsolete conv1d workarounds (#2147 )	2026-10-17 02:39:51 +08:00
therealshulgin	2f016e8e30	fix: handle UTF-8 file paths on Windows (#2145 )	2026-10-17 00:50:42 +08:00
therealshulgin	8c442fb0ad	fix: open model files with UTF-8 paths on Windows (#2144 )	2026-10-17 00:50:09 +08:00
leejet	8cd731e7b4	feat: add FLUX.3 Image Klein and Klein Base support (#2143 )	2026-10-16 02:42:07 +08:00