feat(server): add ADetailer support to image generation (#2164)

docs/adetailer.md

@@ -1,9 +1,10 @@
 # ADetailer
 
 `sd-cli` can run a YOLOv8 object detector on an existing or newly generated
-image and perform a cropped inpaint pass for every detected object. The first
-implementation supports YOLOv8 detection checkpoints. YOLOv8 segmentation and
-MediaPipe models are not supported yet.
+image, and `sd-server` can run it after image generation. Both perform a cropped
+inpaint pass for every detected object. The implementation supports YOLOv8
+detection checkpoints. YOLOv8 segmentation and MediaPipe models are not
+supported yet.
 
 ## Convert a detector
 
@@ -68,6 +69,36 @@ ADetailer can also run automatically after normal image generation:
   --extra-ad-args "confidence=0.3,denoising_strength=0.4,inpaint_width=512,inpaint_height=512"
 ```
 
+The same options can be supplied when starting `sd-server`. When `--ad-model` is set, its detector runs after OpenAI-compatible and native asynchronous image generation. SDAPI requests enable ADetailer through `alwayson_scripts`, as described below. If no object is detected, the original image is returned unchanged. RGBA output keeps its original alpha channel, including when detected regions are repaired. Detector initialization or detail-pass failures are reported as request failures.
+
+Server image-generation requests limit base, hires, and detail-pass dimensions to 4096 pixels per side. ADetailer step overrides are limited to 1–100; `steps=0` in `extra_ad_args` keeps inheriting the main generation steps. These limits also apply to inherited dimensions and `extra_ad_args` overrides. Requests exceeding the limits return a parameter error before generation. CLI limits are unchanged.
+
+### SDAPI requests
+
+`/sdapi/v1/txt2img` and `/sdapi/v1/img2img` accept the WebUI ADetailer argument format:
+
+```json
+{
+  "prompt": "portrait photo",
+  "alwayson_scripts": {
+    "ADetailer": {
+      "args": [true, false, {
+        "ad_model": "face_yolov8n.pt",
+        "ad_prompt": "[PROMPT], detailed face",
+        "ad_use_steps": true,
+        "ad_steps": 20
+      }]
+    }
+  }
+}
+```
+
+The detector must be configured at server startup with `--ad-model`. `ad_model` must match that file's basename; the original `.pt` basename is also accepted for a converted detector. Requests cannot load arbitrary detector paths. Omitting the script, setting its first argument to `false`, selecting `"None"`, or setting `ad_tab_enable` to `false` disables detailing. Dictionary-only argument arrays are also accepted. Multiple enabled dictionaries run in order using the configured detector.
+
+Supported settings include prompts, confidence, mask filtering by area or confidence, mask count and area ratios, dilation/erosion, offsets, mask merging/inversion, blur, denoising strength, masked or full-image inpainting, padding, and optional overrides for inpaint dimensions, steps, CFG, sampler/scheduler, and CLIP skip. The corresponding `ad_use_*` flags control whether overrides apply. Otherwise dimensions, steps, CFG, sampler, scheduler, and CLIP skip inherit the main request. Denoising strength defaults to `0.4`. Pixel mask settings are limited to the larger output dimension, with a minimum limit of 32 and a maximum of 16384 pixels.
+
+Model/VAE switching, non-default noise multipliers, face restoration, ControlNet, and skipping the initial img2img pass are not supported; active requests for these features return an error. Unknown settings and invalid active values also return errors. During detailing, `/sdapi/v1/progress` remains active with an `adetailer` job; its percentage and preview stay fixed until detailing finishes.
+
 An empty ADetailer prompt inherits the main prompt. `[PROMPT]` inserts the main
 prompt, `[SEP]` assigns different prompts to consecutive masks, and `[SKIP]`
 skips the corresponding mask.
@@ -82,6 +113,7 @@ All settings other than the detector path and prompts are passed through
 | `nms` | `0.45` | NMS IoU threshold |
 | `max_detections` | `100` | Maximum detections retained after NMS |
 | `mask_k_largest` | `0` | Keep only the largest K masks; zero keeps all |
+| `mask_filter_method` | `area` | Rank retained masks by `area` or `confidence` |
 | `mask_min_ratio` | `0` | Minimum bbox area relative to the image |
 | `mask_max_ratio` | `1` | Maximum bbox area relative to the image |
 | `dilate_erode` | `4` | Positive values dilate; negative values erode |
@@ -90,6 +122,7 @@ All settings other than the detector path and prompts are passed through
 | `merge_masks`, `invert_mask` | `false` | Boolean alternatives to `mask_mode` |
 | `mask_blur` | `4` | Final composite feather radius |
 | `inpaint_padding` | `32` | Padding around the detected region |
+| `inpaint_only_masked` | `true` | Crop to the detected region; `false` uses the full image with the detection mask |
 | `inpaint_width`, `inpaint_height` | mode-specific | `512x512` after generation; input/output size in `adetailer` mode |
 | `denoising_strength` | mode-specific | `0.4` after generation; inherits `--strength` in `adetailer` mode |
 | `steps` | `0` | Detail steps; zero inherits the main generation |

examples/common/common.cpp

@@ -2219,7 +2219,7 @@ bool SDGenerationParams::from_json_str(
         if (target_custom_sigmas != nullptr &&
             sample_json.contains("custom_sigmas") &&
             sample_json["custom_sigmas"].is_array() &&
-            !sample_json["custom_sigmas"].empty()) { // empty [] = "not specified", keep server/CLI default
+            !sample_json["custom_sigmas"].empty()) {  // empty [] = "not specified", keep server/CLI default
             *target_custom_sigmas = sample_json["custom_sigmas"].get<std::vector<float>>();
         }
         if (sample_json.contains("sample_method") && sample_json["sample_method"].is_string()) {

examples/server/async_jobs.cpp

@@ -244,6 +244,10 @@ bool execute_img_gen_job(ServerRuntime& runtime,
         }
         clear_preview_callback();
         results.adopt(raw_results, num_results);
+        if (!results.empty() && !apply_server_adetailer(runtime, job.img_gen, params, results)) {
+            error_message = "ADetailer failed";
+            return false;
+        }
     }
 
     const int num_results = results.count();

examples/server/routes_openai.cpp

@@ -75,7 +75,7 @@ static bool build_openai_generation_request(const httplib::Request& req,
         error_message = "invalid params";
         return false;
     }
-    return true;
+    return validate_server_img_gen_request(runtime, request, error_message);
 }
 
 static bool build_openai_edit_request(const httplib::Request& req,
@@ -158,22 +158,22 @@ static bool build_openai_edit_request(const httplib::Request& req,
     request.gen_params.height      = height;
     request.gen_params.batch_count = n;
 
-    std::string sd_cpp_extra_args_str          = extract_and_remove_sd_cpp_extra_args(request.gen_params.prompt);
-    const json extra_args                      = sd_cpp_extra_args_str.empty()
-                                                     ? json::object()
-                                                     : json::parse(sd_cpp_extra_args_str, nullptr, false);
-    const bool has_extra_init_image_override   = extra_args.is_object() && extra_args.contains("init_image");
-    const bool has_extra_ref_images_override   = extra_args.is_object() && extra_args.contains("ref_images");
-    const bool has_extra_strength_override     = extra_args.is_object() &&
-                                                 extra_args.contains("strength") &&
-                                                 extra_args["strength"].is_number();
-    const bool has_extra_mask_image            = extra_args.is_object() &&
-                                                 extra_args.contains("mask_image") &&
-                                                 !extra_args["mask_image"].is_null();
+    std::string sd_cpp_extra_args_str        = extract_and_remove_sd_cpp_extra_args(request.gen_params.prompt);
+    const json extra_args                    = sd_cpp_extra_args_str.empty()
+                                                   ? json::object()
+                                                   : json::parse(sd_cpp_extra_args_str, nullptr, false);
+    const bool has_extra_init_image_override = extra_args.is_object() && extra_args.contains("init_image");
+    const bool has_extra_ref_images_override = extra_args.is_object() && extra_args.contains("ref_images");
+    const bool has_extra_strength_override   = extra_args.is_object() &&
+                                             extra_args.contains("strength") &&
+                                             extra_args["strength"].is_number();
+    const bool has_extra_mask_image = extra_args.is_object() &&
+                                      extra_args.contains("mask_image") &&
+                                      !extra_args["mask_image"].is_null();
     const bool supports_ref_image_conditioning = sd_ctx_supports_ref_image_conditioning(runtime.sd_ctx);
     const bool may_need_uploaded_init_image    = !has_extra_init_image_override &&
-                                                 (!supports_ref_image_conditioning || !mask_bytes.empty() ||
-                                                  has_extra_mask_image || has_extra_strength_override || has_extra_ref_images_override);
+                                              (!supports_ref_image_conditioning || !mask_bytes.empty() ||
+                                               has_extra_mask_image || has_extra_strength_override || has_extra_ref_images_override);
     SDImageOwner uploaded_image_for_init;
     bool uploaded_image_was_first_reference = false;
     for (auto& bytes : images_bytes) {
@@ -252,7 +252,7 @@ static bool build_openai_edit_request(const httplib::Request& req,
         return false;
     }
 
-    return true;
+    return validate_server_img_gen_request(runtime, request, error_message);
 }
 
 static bool execute_sync_img_gen_request(ServerRuntime& runtime,
@@ -270,6 +270,10 @@ static bool execute_sync_img_gen_request(ServerRuntime& runtime,
             num_results = 0;
         }
         results.adopt(raw_results, num_results);
+        if (!results.empty() && !apply_server_adetailer(runtime, request, img_gen_params, results)) {
+            error_message = "ADetailer failed";
+            return false;
+        }
     }
 
     if (results.empty()) {

examples/server/routes_sdapi.cpp

@@ -3,9 +3,12 @@
 #include <algorithm>
 #include <cctype>
 #include <cstring>
+#include <limits>
 #include <regex>
+#include <stdexcept>
 #include <string_view>
 #include <unordered_map>
+#include <unordered_set>
 
 #include "common/common.h"
 #include "common/media_io.h"
@@ -81,6 +84,202 @@ static enum sample_method_t get_sdapi_sample_method(std::string name) {
     return it != hardcoded.end() ? it->second : SAMPLE_METHOD_COUNT;
 }
 
+static bool parse_sdapi_adetailer(const json& body,
+                                  ServerRuntime& runtime,
+                                  bool img2img,
+                                  ImgGenJobRequest& request,
+                                  std::string& error_message) {
+    request.adetailer_enabled = false;
+    request.adetailer_passes.clear();
+    if (!body.contains("alwayson_scripts")) {
+        return true;
+    }
+    try {
+        const auto& scripts = body.at("alwayson_scripts");
+        if (!scripts.is_object()) {
+            throw std::invalid_argument("alwayson_scripts must be an object");
+        }
+        if (!scripts.contains("ADetailer")) {
+            return true;
+        }
+        const auto& args = scripts.at("ADetailer").at("args");
+        if (!args.is_array()) {
+            throw std::invalid_argument("ADetailer.args must be an array");
+        }
+        if (args.empty() || (args[0].is_boolean() && !args[0].get<bool>())) {
+            return true;
+        }
+        static const std::unordered_set<std::string> keys = {
+            "ad_model", "ad_model_classes", "ad_tab_enable", "ad_prompt", "ad_negative_prompt",
+            "ad_confidence", "ad_mask_filter_method", "ad_mask_k", "ad_mask_min_ratio", "ad_mask_max_ratio",
+            "ad_dilate_erode", "ad_x_offset", "ad_y_offset", "ad_mask_merge_invert", "ad_mask_blur",
+            "ad_denoising_strength", "ad_inpaint_only_masked", "ad_inpaint_only_masked_padding",
+            "ad_use_inpaint_width_height", "ad_inpaint_width", "ad_inpaint_height", "ad_use_steps", "ad_steps",
+            "ad_use_cfg_scale", "ad_cfg_scale", "ad_use_sampler", "ad_sampler", "ad_scheduler",
+            "ad_use_checkpoint", "ad_checkpoint", "ad_use_vae", "ad_vae", "ad_use_noise_multiplier",
+            "ad_noise_multiplier", "ad_use_clip_skip", "ad_clip_skip", "ad_restore_face", "ad_controlnet_model",
+            "ad_controlnet_module", "ad_controlnet_weight", "ad_controlnet_guidance_start", "ad_controlnet_guidance_end", "is_api"};
+        for (const auto& config : args) {
+            if (!config.is_object()) {
+                continue;
+            }
+            auto boolean = [&](const char* key, bool fallback) {
+                if (!config.contains(key))
+                    return fallback;
+                if (!config.at(key).is_boolean())
+                    throw std::invalid_argument(std::string(key) + " must be boolean");
+                return config.at(key).get<bool>();
+            };
+            auto text = [&](const char* key, const char* fallback) {
+                if (!config.contains(key))
+                    return std::string(fallback);
+                if (!config.at(key).is_string())
+                    throw std::invalid_argument(std::string(key) + " must be a string");
+                return config.at(key).get<std::string>();
+            };
+            auto number = [&](const char* key, double fallback, double minimum, double maximum) {
+                double value = fallback;
+                if (config.contains(key)) {
+                    if (!config.at(key).is_number())
+                        throw std::invalid_argument(std::string(key) + " must be numeric");
+                    value = config.at(key).get<double>();
+                }
+                if (!std::isfinite(value) || value < minimum || value > maximum)
+                    throw std::invalid_argument(std::string(key) + " is out of range");
+                return value;
+            };
+            auto integer = [&](const char* key, int fallback, int minimum, int maximum = std::numeric_limits<int>::max()) {
+                if (config.contains(key) && !config.at(key).is_number_integer())
+                    throw std::invalid_argument(std::string(key) + " must be an integer");
+                return static_cast<int>(number(key, fallback, minimum, maximum));
+            };
+            const std::string model = text("ad_model", "None");
+            if (model == "None" || !boolean("ad_tab_enable", true)) {
+                continue;
+            }
+            for (const auto& item : config.items()) {
+                if (keys.count(item.key()) == 0)
+                    throw std::invalid_argument("unsupported ADetailer setting: " + item.key());
+            }
+            const auto& configured = runtime.default_gen_params->ad_model_path;
+            if (configured.empty())
+                throw std::invalid_argument("ADetailer requires a server-configured --ad-model");
+            const auto path = fs::u8path(configured);
+            if (model.find_first_of("/\\") != std::string::npos ||
+                (lower_ascii(model) != lower_ascii(path.filename().u8string()) && lower_ascii(model) != lower_ascii(path.stem().u8string() + ".pt"))) {
+                throw std::invalid_argument("ADetailer model must match the server-configured detector");
+            }
+            if (!text("ad_model_classes", "").empty())
+                throw std::invalid_argument("ad_model_classes is not supported");
+            if (boolean("ad_use_checkpoint", false)) {
+                const auto checkpoint = config.contains("ad_checkpoint") && config.at("ad_checkpoint").is_null() ? "" : text("ad_checkpoint", "Use same checkpoint");
+                if (!checkpoint.empty() && checkpoint != "None" && checkpoint != "Use same checkpoint")
+                    throw std::invalid_argument("ADetailer checkpoint switching is not supported");
+            }
+            if (boolean("ad_use_vae", false)) {
+                const auto vae = config.contains("ad_vae") && config.at("ad_vae").is_null() ? "" : text("ad_vae", "Use same VAE");
+                if (!vae.empty() && vae != "None" && vae != "Use same VAE")
+                    throw std::invalid_argument("ADetailer VAE switching is not supported");
+            }
+            if (boolean("ad_use_noise_multiplier", false) && number("ad_noise_multiplier", 1, 0.5, 1.5) != 1)
+                throw std::invalid_argument("ADetailer noise multiplier is not supported");
+            if (boolean("ad_restore_face", false))
+                throw std::invalid_argument("ADetailer face restoration is not supported");
+            if (text("ad_controlnet_model", "None") != "None")
+                throw std::invalid_argument("ADetailer ControlNet is not supported");
+
+            ServerADetailerPass pass;
+            pass.prompt          = text("ad_prompt", "");
+            pass.negative_prompt = text("ad_negative_prompt", "");
+            pass.width           = request.gen_params.get_resolved_width();
+            pass.height          = request.gen_params.get_resolved_height();
+            pass.strength        = static_cast<float>(number("ad_denoising_strength", 0.4, 0, 1));
+            auto append          = [&](const char* key, const std::string& value) {
+                if (!pass.extra_args.empty())
+                    pass.extra_args += ",";
+                pass.extra_args += std::string(key) + "=" + value;
+            };
+            auto append_number = [&](const char* target, const char* key, double fallback, double minimum, double maximum) {
+                append(target, json(number(key, fallback, minimum, maximum)).dump());
+            };
+            auto append_integer = [&](const char* target, const char* key, int fallback, int minimum, int maximum = std::numeric_limits<int>::max()) {
+                append(target, std::to_string(integer(key, fallback, minimum, maximum)));
+            };
+            append_number("confidence", "ad_confidence", 0.3, 0, 1);
+            append_integer("mask_k_largest", "ad_mask_k", 0, 0);
+            const auto filter = text("ad_mask_filter_method", "Area");
+            if (filter != "Area" && filter != "Confidence")
+                throw std::invalid_argument("invalid ad_mask_filter_method");
+            append("mask_filter_method", lower_ascii(filter));
+            const auto min_ratio = number("ad_mask_min_ratio", 0, 0, 1);
+            const auto max_ratio = number("ad_mask_max_ratio", 1, 0, 1);
+            if (min_ratio > max_ratio)
+                throw std::invalid_argument("ad_mask_min_ratio exceeds ad_mask_max_ratio");
+            append("mask_min_ratio", json(min_ratio).dump());
+            append("mask_max_ratio", json(max_ratio).dump());
+            // Bound pixel operations before converting them to the native mask parameters.
+            const int mask_limit = std::clamp(std::max(pass.width, pass.height), 32, 16384);
+            append_integer("dilate_erode", "ad_dilate_erode", 4, -mask_limit, mask_limit);
+            append_integer("x_offset", "ad_x_offset", 0, -mask_limit, mask_limit);
+            append_integer("y_offset", "ad_y_offset", 0, -mask_limit, mask_limit);
+            append_integer("mask_blur", "ad_mask_blur", 4, 0, mask_limit);
+            append_integer("inpaint_padding", "ad_inpaint_only_masked_padding", 32, 0, mask_limit);
+            append("inpaint_only_masked", boolean("ad_inpaint_only_masked", true) ? "true" : "false");
+            const auto merge = text("ad_mask_merge_invert", "None");
+            if (merge != "None" && merge != "Merge" && merge != "Merge and Invert")
+                throw std::invalid_argument("invalid ad_mask_merge_invert");
+            append("mask_mode", merge == "None" ? "none" : merge == "Merge" ? "merge"
+                                                                            : "merge_invert");
+            if (boolean("ad_use_inpaint_width_height", false)) {
+                pass.width  = integer("ad_inpaint_width", 512, 1, k_max_generation_dimension);
+                pass.height = integer("ad_inpaint_height", 512, 1, k_max_generation_dimension);
+            }
+            if (boolean("ad_use_steps", false))
+                append_integer("steps", "ad_steps", 28, 1, k_max_adetailer_steps);
+            if (boolean("ad_use_cfg_scale", false))
+                append_number("cfg_scale", "ad_cfg_scale", 7, 0, std::numeric_limits<float>::max());
+            if (boolean("ad_use_clip_skip", false)) {
+                pass.clip_skip = integer("ad_clip_skip", 1, 1);
+                if (pass.clip_skip > 12)
+                    throw std::invalid_argument("ad_clip_skip is out of range");
+            }
+            if (boolean("ad_use_sampler", false)) {
+                std::string sampler = text("ad_sampler", "DPM++ 2M Karras");
+                if (sampler != "Use same sampler") {
+                    const std::string suffix = " karras";
+                    const bool legacy_karras = sampler.size() > suffix.size() && lower_ascii(sampler.substr(sampler.size() - suffix.size())) == suffix;
+                    if (legacy_karras)
+                        sampler.resize(sampler.size() - suffix.size());
+                    const auto method = get_sdapi_sample_method(sampler);
+                    if (method == SAMPLE_METHOD_COUNT)
+                        throw std::invalid_argument("unsupported ADetailer sampler");
+                    append("sample_method", sd_sample_method_name(method));
+                    if (legacy_karras && text("ad_scheduler", "Use same scheduler") == "Use same scheduler")
+                        append("scheduler", sd_scheduler_name(KARRAS_SCHEDULER));
+                }
+                const auto scheduler = text("ad_scheduler", "Use same scheduler");
+                if (scheduler == "Automatic") {
+                    pass.automatic_scheduler = true;
+                } else if (scheduler != "Use same scheduler") {
+                    const auto value = str_to_scheduler(scheduler.c_str());
+                    if (value == SCHEDULER_COUNT)
+                        throw std::invalid_argument("unsupported ADetailer scheduler");
+                    append("scheduler", sd_scheduler_name(value));
+                }
+            }
+            request.adetailer_passes.push_back(std::move(pass));
+        }
+        if (!request.adetailer_passes.empty() && img2img && args.size() > 1 && args[1].is_boolean() && args[1].get<bool>()) {
+            throw std::invalid_argument("ADetailer skip img2img is not supported");
+        }
+        request.adetailer_enabled = !request.adetailer_passes.empty();
+        return true;
+    } catch (const std::exception& error) {
+        error_message = std::string("invalid ADetailer args: ") + error.what();
+        return false;
+    }
+}
+
 static bool build_sdapi_img_gen_request(const json& j,
                                         ServerRuntime& runtime,
                                         bool img2img,
@@ -259,7 +458,8 @@ static bool build_sdapi_img_gen_request(const json& j,
         return false;
     }
 
-    return true;
+    return parse_sdapi_adetailer(j, runtime, img2img, request, error_message) &&
+           validate_server_img_gen_request(runtime, request, error_message);
 }
 
 static nlohmann::json prepare_info_field(const SDContextParams& ctx_params,
@@ -349,7 +549,7 @@ static void preview_callback(int step, int frame_count, sd_image_t* frames, bool
 }
 
 struct SDAPIPreviewGuard {
-    SDAPIPreviewGuard(const std::string& id_task, const sd_img_gen_params_t& params, const std::string& job_type) {
+    SDAPIPreviewGuard(const std::string& id_task, const sd_img_gen_params_t& params, const std::string& job_type, bool detailing = false) {
         std::lock_guard<std::mutex> lock(g_preview.mutex);
         g_preview.id_task  = id_task;
         g_preview.total    = 0;
@@ -358,11 +558,20 @@ struct SDAPIPreviewGuard {
         g_preview.step   = 0;
         g_preview.active = true;
         // Each batch item has a base pass and, when enabled, a hires pass.
-        g_preview.job_count = static_cast<int64_t>(std::max(1, params.batch_count)) * (params.hires.enabled ? 2 : 1);
+        g_preview.job_count = static_cast<int64_t>(std::max(1, params.batch_count)) * (params.hires.enabled ? 2 : 1) + (detailing ? 1 : 0);
         g_preview.job_no    = 0;
         sd_set_preview_callback(preview_callback, PREVIEW_PROJ, 1, true, false, nullptr);
     }
 
+    void begin_detailing() {
+        sd_set_preview_callback(nullptr, PREVIEW_NONE, 1, false, false, nullptr);
+        std::lock_guard<std::mutex> lock(g_preview.mutex);
+        g_preview.job_type = "adetailer";
+        g_preview.job_no   = static_cast<int>(g_preview.job_count - 1);
+        g_preview.step     = 0;
+        g_preview.total    = 0;
+    }
+
     ~SDAPIPreviewGuard() {
         sd_set_preview_callback(nullptr, PREVIEW_NONE, 1, false, false, nullptr);
         std::lock_guard<std::mutex> lock(g_preview.mutex);
@@ -410,13 +619,21 @@ void register_sdapi_endpoints(httplib::Server& svr, ServerRuntime& rt) {
 
             {
                 std::lock_guard<std::mutex> lock(*runtime->sd_ctx_mutex);
-                SDAPIPreviewGuard preview_guard(id_task, img_gen_params, img2img ? "img2img" : "txt2img");
+                const bool detailing = request.adetailer_enabled && !runtime->default_gen_params->ad_model_path.empty();
+                SDAPIPreviewGuard preview_guard(id_task, img_gen_params, img2img ? "img2img" : "txt2img", detailing);
                 sd_image_t* raw_results = nullptr;
                 if (!generate_image(runtime->sd_ctx, &img_gen_params, &raw_results, &num_results)) {
                     raw_results = nullptr;
                     num_results = 0;
                 }
                 results.adopt(raw_results, num_results);
+                if (detailing && !results.empty())
+                    preview_guard.begin_detailing();
+                if (!results.empty() && !apply_server_adetailer(*runtime, request, img_gen_params, results)) {
+                    res.status = 500;
+                    res.set_content(R"({"error":"ADetailer failed"})", "application/json");
+                    return;
+                }
             }
 
             if (results.empty()) {

examples/server/routes_sdcpp.cpp

@@ -410,9 +410,9 @@ static json make_capabilities_json(ServerRuntime& runtime) {
     result["defaults_by_mode"]    = defaults_by_mode;
     result["limits"]              = {
                      {"min_width", 64},
-                     {"max_width", 4096},
+                     {"max_width", k_max_generation_dimension},
                      {"min_height", 64},
-                     {"max_height", 4096},
+                     {"max_height", k_max_generation_dimension},
                      {"max_batch_count", 8},
                      {"max_queue_size", manager.max_pending_jobs},
                      {"max_upscale_width", k_max_upscale_dimension},
@@ -457,7 +457,7 @@ static bool parse_img_gen_request(const json& body,
         error_message = "invalid generation parameters";
         return false;
     }
-    return true;
+    return validate_server_img_gen_request(runtime, request, error_message);
 }
 
 static bool parse_vid_gen_request(const json& body,

examples/server/runtime.cpp

@@ -3,6 +3,7 @@
 #include <algorithm>
 #include <cctype>
 #include <chrono>
+#include <cmath>
 #include <cstdlib>
 #include <filesystem>
 #include <mutex>
@@ -18,6 +19,157 @@
 
 namespace fs = std::filesystem;
 
+bool validate_server_img_gen_request(const ServerRuntime& runtime,
+                                     const ImgGenJobRequest& request,
+                                     std::string& error_message) {
+    auto dimensions_valid = [&](double width, double height) {
+        if (!std::isfinite(width) || !std::isfinite(height) ||
+            width < 1 || height < 1 || width > k_max_generation_dimension || height > k_max_generation_dimension) {
+            error_message = "generation width and height must be between 1 and " + std::to_string(k_max_generation_dimension);
+            return false;
+        }
+        return true;
+    };
+    const auto& params = request.gen_params;
+    const int width    = params.get_resolved_width();
+    const int height   = params.get_resolved_height();
+    if (!dimensions_valid(width, height)) {
+        return false;
+    }
+    if (params.hires_enabled) {
+        const double hires_width  = params.hires_width > 0 ? params.hires_width : params.hires_height > 0 ? params.hires_height
+                                                                                                          : std::round(width * params.hires_scale);
+        const double hires_height = params.hires_height > 0 ? params.hires_height : params.hires_width > 0 ? params.hires_width
+                                                                                                           : std::round(height * params.hires_scale);
+        if (!dimensions_valid(hires_width, hires_height)) {
+            return false;
+        }
+    }
+    if (!request.adetailer_enabled || runtime.default_gen_params->ad_model_path.empty()) {
+        return true;
+    }
+    auto pass_valid = [&](const ServerADetailerPass& pass) {
+        if (!dimensions_valid(pass.width, pass.height)) {
+            return false;
+        }
+        // Match the native key normalization, but leave full option parsing to ADetailer.
+        auto trim = [](const std::string& text) {
+            const auto first = text.find_first_not_of(" \t\r\n");
+            return first == std::string::npos ? std::string() : text.substr(first, text.find_last_not_of(" \t\r\n") - first + 1);
+        };
+        std::stringstream stream(pass.extra_args);
+        std::string item;
+        while (std::getline(stream, item, ',')) {
+            const auto equals = item.find('=');
+            if (equals == std::string::npos) {
+                continue;
+            }
+            auto key = trim(item.substr(0, equals));
+            std::transform(key.begin(), key.end(), key.begin(), [](unsigned char c) { return static_cast<char>(std::tolower(c)); });
+            if (key != "inpaint_width" && key != "inpaint_height" && key != "steps") {
+                continue;
+            }
+            const auto value  = trim(item.substr(equals + 1));
+            const int minimum = key == "steps" ? 0 : 1;
+            const int maximum = key == "steps" ? k_max_adetailer_steps : k_max_generation_dimension;
+            try {
+                size_t used      = 0;
+                const int parsed = std::stoi(value, &used);
+                if (used == value.size() && parsed >= minimum && parsed <= maximum) {
+                    continue;
+                }
+            } catch (const std::exception&) {
+            }
+            error_message = "ADetailer " + key + " must be between " + std::to_string(minimum) + " and " + std::to_string(maximum);
+            return false;
+        }
+        return true;
+    };
+    if (request.adetailer_passes.empty()) {
+        ServerADetailerPass pass;
+        pass.extra_args = params.extra_ad_args;
+        return pass_valid(pass);
+    }
+    return std::all_of(request.adetailer_passes.begin(), request.adetailer_passes.end(), pass_valid);
+}
+
+bool apply_server_adetailer(ServerRuntime& runtime,
+                            const ImgGenJobRequest& request,
+                            const sd_img_gen_params_t& params,
+                            SDImageVec& results) {
+    const auto& gen_params    = request.gen_params;
+    const auto& ad_model_path = runtime.default_gen_params->ad_model_path;
+    if (!request.adetailer_enabled || ad_model_path.empty()) {
+        return true;
+    }
+    std::string error_message;
+    if (!validate_server_img_gen_request(runtime, request, error_message)) {
+        LOG_ERROR("%s", error_message.c_str());
+        return false;
+    }
+
+    std::vector<ServerADetailerPass> passes = request.adetailer_passes;
+    if (passes.empty()) {
+        ServerADetailerPass pass;
+        pass.prompt          = gen_params.ad_prompt;
+        pass.negative_prompt = gen_params.ad_negative_prompt;
+        pass.extra_args      = gen_params.extra_ad_args;
+        passes.push_back(std::move(pass));
+    }
+
+    ADetailerCtxPtr ad_ctx(new_adetailer_ctx(ad_model_path.c_str(),
+                                             runtime.ctx_params->n_threads,
+                                             runtime.ctx_params->effective_backend.c_str(),
+                                             runtime.ctx_params->effective_params_backend.c_str()));
+    if (ad_ctx == nullptr) {
+        LOG_ERROR("new_adetailer_ctx failed");
+        return false;
+    }
+
+    for (const auto& pass : passes) {
+        sd_adetailer_params_t ad_params{};
+        ad_params.prompt          = pass.prompt.empty() ? nullptr : pass.prompt.c_str();
+        ad_params.negative_prompt = pass.negative_prompt.empty() ? nullptr : pass.negative_prompt.c_str();
+        ad_params.extra_ad_args   = pass.extra_args.c_str();
+        for (int i = 0; i < results.count(); ++i) {
+            if (results[i].data == nullptr) {
+                continue;
+            }
+            sd_img_gen_params_t ad_generation_params = params;
+            ad_generation_params.seed                = params.seed + i;
+            ad_generation_params.width               = pass.width;
+            ad_generation_params.height              = pass.height;
+            ad_generation_params.strength            = pass.strength;
+            if (pass.clip_skip > 0) {
+                ad_generation_params.clip_skip = pass.clip_skip;
+            }
+            if (pass.automatic_scheduler) {
+                ad_generation_params.sample_params.scheduler = SCHEDULER_COUNT;
+            }
+
+            sd_image_t* detailed_images = nullptr;
+            int detailed_count          = 0;
+            if (!adetail_image(ad_ctx.get(),
+                               runtime.sd_ctx,
+                               results[i],
+                               &ad_params,
+                               &ad_generation_params,
+                               &detailed_images,
+                               &detailed_count) ||
+                detailed_count <= 0 || detailed_images == nullptr || detailed_images[0].data == nullptr) {
+                free_sd_images(detailed_images, detailed_count);
+                LOG_ERROR("ADetailer failed for image %d", i + 1);
+                return false;
+            }
+            free(results[i].data);
+            results[i]         = detailed_images[0];
+            detailed_images[0] = {0, 0, 0, nullptr};
+            free_sd_images(detailed_images, detailed_count);
+        }
+    }
+    return true;
+}
+
 static std::string lower_ascii(std::string value) {
     std::transform(value.begin(), value.end(), value.begin(), [](unsigned char c) {
         return static_cast<char>(std::tolower(c));

examples/server/runtime.h

@@ -67,18 +67,38 @@ struct ServerRuntime {
     ServerModels* models;
 };
 
+struct ServerADetailerPass {
+    std::string prompt;
+    std::string negative_prompt;
+    std::string extra_args;
+    int width                = 512;
+    int height               = 512;
+    float strength           = 0.4f;
+    int clip_skip            = -1;
+    bool automatic_scheduler = false;
+};
+
 struct ImgGenJobRequest {
     SDGenerationParams gen_params;
     std::string output_format = "png";
     int output_compression    = 100;
     std::string preview_mode  = "none";
     int preview_interval      = 1;
+    bool adetailer_enabled    = true;
+    std::vector<ServerADetailerPass> adetailer_passes;
 
     sd_img_gen_params_t to_sd_img_gen_params_t() {
         return gen_params.to_sd_img_gen_params_t();
     }
 };
 
+constexpr int k_max_generation_dimension = 4096;
+constexpr int k_max_adetailer_steps      = 100;
+
+bool validate_server_img_gen_request(const ServerRuntime& runtime,
+                                     const ImgGenJobRequest& request,
+                                     std::string& error_message);
+
 struct VidGenJobRequest {
     SDGenerationParams gen_params;
     std::string output_format = "webm";
@@ -111,3 +131,7 @@ void refresh_lora_cache(ServerRuntime& rt);
 std::string get_lora_full_path(ServerRuntime& rt, const std::string& path);
 void refresh_upscaler_cache(ServerRuntime& rt);
 int64_t unix_timestamp_now_ms();
+bool apply_server_adetailer(ServerRuntime& runtime,
+                            const ImgGenJobRequest& request,
+                            const sd_img_gen_params_t& params,
+                            SDImageVec& results);

src/detailer.cpp

@@ -291,7 +291,9 @@ static void filter_and_sort_detections(std::vector<ADetailerDetection>* detectio
         std::partial_sort(detections->begin(),
                           detections->begin() + params.mask_k_largest,
                           detections->end(),
-                          [](const auto& a, const auto& b) { return detection_area(a) > detection_area(b); });
+                          [&](const auto& a, const auto& b) {
+                              return params.mask_filter_by_confidence ? a.confidence > b.confidence : detection_area(a) > detection_area(b);
+                          });
         detections->resize(params.mask_k_largest);
     }
 
@@ -545,7 +547,16 @@ static sd_image_t resize_crop_mask(const Mask& source, const CropRegion& crop, i
     return output;
 }
 
-static sd_image_t copy_as_rgb(const sd_image_t& source) {
+static sd_image_t copy_for_detailing(const sd_image_t& source) {
+    if (source.channel == 4) {
+        sd_image_t output = source;
+        size_t bytes      = static_cast<size_t>(source.width) * source.height * source.channel;
+        output.data       = static_cast<uint8_t*>(malloc(bytes));
+        if (output.data != nullptr) {
+            memcpy(output.data, source.data, bytes);
+        }
+        return output;
+    }
     CropRegion full{0, 0, static_cast<int>(source.width), static_cast<int>(source.height)};
     return resize_crop_image(source, full, source.width, source.height);
 }
@@ -566,7 +577,7 @@ static void composite_crop(sd_image_t* destination,
                 continue;
             }
             float source_x           = (x - crop.x1 + 0.5f) * scale_x - 0.5f;
-            size_t destination_index = (static_cast<size_t>(y) * destination->width + x) * 3;
+            size_t destination_index = (static_cast<size_t>(y) * destination->width + x) * destination->channel;
             for (int c = 0; c < 3; ++c) {
                 float generated_value                    = bilinear_channel(generated, source_x, source_y, c) * 255.f;
                 float original_value                     = destination->data[destination_index + c];
@@ -787,6 +798,10 @@ static bool parse_adetailer_params(const sd_adetailer_params_t& public_params,
             ok = parse_int(value, &params->max_detections);
         } else if (key == "mask_k_largest") {
             ok = parse_int(value, &params->mask_k_largest);
+        } else if (key == "mask_filter_method") {
+            const std::string method          = lower_copy(value);
+            ok                                = method == "area" || method == "confidence";
+            params->mask_filter_by_confidence = method == "confidence";
         } else if (key == "mask_min_ratio") {
             ok = parse_float(value, &params->mask_min_ratio);
         } else if (key == "mask_max_ratio") {
@@ -805,6 +820,8 @@ static bool parse_adetailer_params(const sd_adetailer_params_t& public_params,
             ok = parse_int(value, &params->mask_blur);
         } else if (key == "inpaint_padding") {
             ok = parse_int(value, &params->inpaint_padding);
+        } else if (key == "inpaint_only_masked") {
+            ok = parse_bool(value, &params->inpaint_only_masked);
         } else if (key == "inpaint_width") {
             ok = parse_int(value, &params->inpaint_width);
         } else if (key == "inpaint_height") {
@@ -920,7 +937,7 @@ bool adetail_image(adetailer_ctx_t* context,
     filter_and_sort_detections(&detections, input_image.width, input_image.height, params);
     auto masks = make_masks(detections, input_image.width, input_image.height, params);
 
-    sd_image_t current = copy_as_rgb(input_image);
+    sd_image_t current = copy_for_detailing(input_image);
     if (current.data == nullptr) {
         return false;
     }
@@ -939,12 +956,13 @@ bool adetail_image(adetailer_ctx_t* context,
         if (!mask_bbox(masks[i], &crop)) {
             continue;
         }
-        crop                   = expand_crop(crop,
-                                             current.width,
-                                             current.height,
-                                             params.inpaint_padding,
-                                             params.inpaint_width,
-                                             params.inpaint_height);
+        crop                   = params.inpaint_only_masked ? expand_crop(crop,
+                                                                          current.width,
+                                                                          current.height,
+                                                                          params.inpaint_padding,
+                                                                          params.inpaint_width,
+                                                                          params.inpaint_height)
+                                                            : CropRegion{0, 0, static_cast<int>(current.width), static_cast<int>(current.height)};
         sd_image_t local_image = resize_crop_image(current, crop, params.inpaint_width, params.inpaint_height);
         sd_image_t local_mask  = resize_crop_mask(masks[i], crop, params.inpaint_width, params.inpaint_height);
         if (local_image.data == nullptr || local_mask.data == nullptr) {

src/detailer.h

@@ -27,30 +27,32 @@ enum ADetailerSort {
 };
 
 struct ADetailerParams {
-    const char* prompt            = nullptr;
-    const char* negative_prompt   = nullptr;
-    int input_size                = 640;
-    float confidence              = 0.3f;
-    float nms_threshold           = 0.45f;
-    int max_detections            = 100;
-    int mask_k_largest            = 0;
-    float mask_min_ratio          = 0.f;
-    float mask_max_ratio          = 1.f;
-    int dilate_erode              = 4;
-    int x_offset                  = 0;
-    int y_offset                  = 0;
-    bool merge_masks              = false;
-    bool invert_mask              = false;
-    int mask_blur                 = 4;
-    int inpaint_padding           = 32;
-    int inpaint_width             = 512;
-    int inpaint_height            = 512;
-    float denoising_strength      = 0.4f;
-    int steps                     = 0;
-    float cfg_scale               = -1.f;
-    sample_method_t sample_method = SAMPLE_METHOD_COUNT;
-    scheduler_t scheduler         = SCHEDULER_COUNT;
-    ADetailerSort sort_by         = ADETAILER_SORT_NONE;
+    const char* prompt             = nullptr;
+    const char* negative_prompt    = nullptr;
+    int input_size                 = 640;
+    float confidence               = 0.3f;
+    float nms_threshold            = 0.45f;
+    int max_detections             = 100;
+    int mask_k_largest             = 0;
+    bool mask_filter_by_confidence = false;
+    float mask_min_ratio           = 0.f;
+    float mask_max_ratio           = 1.f;
+    int dilate_erode               = 4;
+    int x_offset                   = 0;
+    int y_offset                   = 0;
+    bool merge_masks               = false;
+    bool invert_mask               = false;
+    int mask_blur                  = 4;
+    int inpaint_padding            = 32;
+    bool inpaint_only_masked       = true;
+    int inpaint_width              = 512;
+    int inpaint_height             = 512;
+    float denoising_strength       = 0.4f;
+    int steps                      = 0;
+    float cfg_scale                = -1.f;
+    sample_method_t sample_method  = SAMPLE_METHOD_COUNT;
+    scheduler_t scheduler          = SCHEDULER_COUNT;
+    ADetailerSort sort_by          = ADETAILER_SORT_NONE;
 };
 
 struct ADetailerGGML {