mirror of
https://github.com/FlashML-org/FreeToken.git
synced 2026-11-22 05:49:20 +08:00
The pointer path of the Triton prefill GEMM loads the e4m3 scales one byte per thread, too small for cp.async, so every K step waits on that load and a CTA barrier. Read them as int32 words (4 scales each) on GeForce sm_86 / sm_89 so they join the pipeline. - SCALE_WORDS constexpr in `_prefill_nvfp4_moe_kernel` (off by default) - Triton prefill tuning as one record per (GPU class, cc); GeForce sm_86 / sm_89 turn scale words on, with 8 warps when K >= 2048, and sm_86 moves to BLOCK_SIZE_M 64 from 100 tokens per expert