Rohit Ghumare 8bab0b754a feat(phase-07/03): add rust multi-head + grouped-query attention ...

Hand-rolled MHA with split_heads/combine_heads operating on row-major
Vec<f32> tiles, plus a GQA variant that shares KV heads across Q groups.
Prints per-head weight matrices, MHA-vs-GQA KV cache ratio, and a 5K
forward microbench. Stdlib only.

2026-05-25 20:08:39 +01:00

..

main.py

feat(phase-07/03): multi-head attention

2026-04-22 23:53:35 +01:00

main.rs

feat(phase-07/03): add rust multi-head + grouped-query attention

2026-05-25 20:08:39 +01:00