Files
Xiaoze Fan 609191d81f feat(gguf): take image input on Qwen and Gemma-4 GGUF checkpoints from an mmproj (#803)
* feat(gguf): load the vision tower from a llama.cpp mmproj

Qwen3.5 / Qwen3.6 / Qwen3.8 (qwen3vl_merger) and Gemma-4 26B-A4B / 31B (gemma4v) / 12B (gemma4uv) take image input from a separate mmproj file: --mmproj, else one in the model's own folder. The tower runs in bf16 (quantized mmproj files are dequantized) and its config comes from the GGUF metadata plus per-arch defaults.

* docs(gguf): describe mmproj image input without a model list

* test(gguf): drop the mmproj tests

* fix(gguf): escape the model folder when globbing for mmproj files

* fix(gguf): skip auto-discovered mmproj files the text model cannot use

* fix(gguf): serve text from a GGUF that carries vision tensors itself
2026-10-22 14:56:19 -07:00
..
2026-08-11 22:53:25 +00:00