Files
Xiaoze Fanandzihaomu e5d3bd3faf feat(gguf): serve GGUF checkpoints on vendored ggml-cuda kernels (#690)
* build(kernel): vendor ggml-cuda from llama.cpp 8c2134a8 behind a tvm-ffi binding

* build(deps): vendor the gguf-py reader

* feat(quant): add the GGUF quant kind and an embedding quant method

* feat(gguf): serve Flash-Next and Gemma4 GGUF on the GGUF quant kind and drop the old Q4_0 path

* fix(kernel): build ggml-cuda for plain sm_120 by turning off the native fp4 MMA

* perf(gguf): write ggml results in the activation dtype and keep MoE intermediates in bf16

* build(kernel): compress the ggml fatbin to keep the multi-arch kernel cache small

* fix(gguf): read F32, F16 and BF16 PLE tables without ggml

* fix(moe): pad the offload slot cache for ggml MMQ tail reads

* fix(gguf): support hip builds and runtime devices

* fix(gguf): copy same-type dequantize and check for missing ggml converters

* fix(gguf): load quantized qwen4exp ple_key and ple_value

* feat(gguf): run float parts of a fused GGUF linear as bf16 segments

* fix(gguf): load quantized qwen4exp hyper-connection and gdn ba weights

* fix(moe): zero the prefill staging area after the last prefill layer

---------

Co-authored-by: zihaomu <zihaomu@outlook.com>
2026-10-15 16:33:16 -07:00

15 lines
628 B
Python

from .base import BankSpec, ExpertView, MoEConfig, MoEKernel, MoEMethod
from . import fp8_block, gguf, mxfp4, mxfp8, nvfp4, unquantized
from .fp8_block import Fp8BlockMoEMethod
from .mxfp4 import Mxfp4MoEMethod
from .mxfp8 import Mxfp8MoEMethod
from .nvfp4 import Nvfp4MoEMethod
from .unquantized import UnquantizedMoEMethod
from .gguf import GGUFMoEMethod
__all__ = [
"BankSpec", "ExpertView", "MoEConfig", "MoEKernel", "MoEMethod",
"UnquantizedMoEMethod", "Fp8BlockMoEMethod", "Nvfp4MoEMethod", "Mxfp4MoEMethod", "Mxfp8MoEMethod", "GGUFMoEMethod",
"fp8_block", "gguf", "mxfp4", "mxfp8", "nvfp4", "unquantized",
]