mirror of
https://github.com/FlashML-org/FreeToken.git
synced 2026-11-21 01:52:30 +08:00
The qsa_sparse_paged_attention tiles are tuned for the GB300's 228KB shared memory; RDNA gfx11/gfx12 have ~64KB/CU, so k+v staging overflows LDS. Query the device's actual shared-memory limit and shrink the KV tile only when it doesn't fit, ROCm-gated so NVIDIA tiles stay byte-identical. The gguf ROCm-compile fixes this branch also carried are now covered upstream by the ggml-cuda vendoring in #690 (separate DEFAULT_HIP_CFLAGS + vendors/hip.h), so this branch reduces to the QSA clamp only.