Files
Xiaoze Fan 27740fec66 feat(kvcache): add fp8 and int8 per-token-head KV cache formats (#894)
* feat(kvcache): add fp8 and int8 per-token-head KV cache formats

* fix(kernel): keep the bf16 m3 sparse k/v loads inline

* fix(attention): refuse a backend that cannot read the kv format

- filter the m3 page-size fallback by kv format
- release the scale buffer when a qsa or bsa rebuild fails
- keep the bf16 qsa and m3 attend code identical to main
- drop docstrings that claimed no register spills
- drop guard tests

* fix(kernel): widen dsa latent row offsets past int32

* feat(spec): store the mtp draft kv in the target's format
2026-11-11 15:47:50 -08:00
..
2026-08-11 22:53:25 +00:00
2026-08-18 12:26:05 -07:00