stable-diffusion.cpp/docs/backend_probe.md

Adding backend performance probes

src/core/probe/backend_probe.h provides the shared measurement machinery. Operator policy stays separate: src/core/probe/flash_attn.cpp, for example, defines attention shapes and its CPU/CUDA/HIP/Vulkan shortcuts, then calls sd_run_backend_probe for unknown configurations. A future convolution probe can use the same runner without changing the attention implementation.

For each representative shape, create a BackendProbeCase:

• build(graph, candidate) creates inputs and either the reference or candidate graph in graph.ctx, registers input tensors in graph.inputs, and assigns graph.output. Leave the output null if the requested implementation is unavailable; do not silently benchmark its fallback as the candidate.
• By default, inputs receive deterministic F32 data, with the same seed for both variants. Supply initialize for F16 convolution weights, integer inputs, or operator-specific values. It runs after allocation, outside timed execution. Both variants must receive equivalent logical inputs.
• Default output validation requires matching shapes and contiguous F32 outputs, checks finiteness, and applies absolute and relative tolerances. Supply compare for other output types or validation rules.

Graphs must be safe to execute repeatedly without modifying their inputs. BackendProbeGraph owns its GGML context and buffer, so all return paths release probe resources. Constructors allocate metadata only; supported() can also be used before a known-architecture shortcut, without allocating device tensors.

BackendProbeOptions controls graph metadata capacity, warmup count, measurement count, measured-time budget, minimum free device memory, comparison tolerances, and minimum improvement. Warmup and allocation are excluded from timing. The runner alternates the two variants, synchronizes the backend around execution, and compares sums of per-case medians. The time budget is checked before each measurement pair; it cannot interrupt an in-flight backend operation.

Only use candidate_faster when completed is true. On incomplete probes, error reports why comparison failed and the timing totals may be partial. The caller handles logging, known architecture rules, fallback policy, and context-lifetime caching. Keep caches separate by operation (or key a shared cache by both operation and backend); an attention result must not select a convolution implementation. If shapes or settings vary within one context, include them in the corresponding cache key as well.

No convolution policy or CLI option is added by this infrastructure.

Module assignment parsing remains in src/core/ggml_extend_backend.cpp, next to backend assignment parsing. The probe directory does not parse CLI/API options.