* feat(spec): add the speculative verify infrastructure
* feat(scheduler): run decode steps as speculative verify steps
* chore(engine): mark the speculative decoding config fields with one comment
* feat(server): add short aliases for the speculative decoding flags
Hybrid 45-layer model (34 KDA + 11 DSA) with NVFP4 MoE (288 experts):
- KDA linear-attention kernels and op, hybrid-linear pool dispatch
generalized to any paged family
- NoPE-MLA attention with a kpool DSA indexer backend: 64-token pages,
1/ratio shadow index slab, per-request tail rings, CUDA-graph-safe
decode, prefix-cache snapshot contract
- mHC (x4 residual streams) layer with a fused split-K triton kernel;
torch reference kept as the semantic pin
- clamped-SwiGLU activation across the MoE stack
- weight loading for NVFP4 exports in the multimodal-wrapper layout:
ModelOpt and compressed-tensors (RedHatAI) tensor kinds;
Co-authored-by: Shuo Yang <73746844+andy-yang-1@users.noreply.github.com>
* feat(server)!: add --gpu to choose the GPU on multi-GPU machines
--gpu takes a GPU UUID (as nvidia-smi -L prints) or an nvidia-smi index.
It is applied as CUDA_VISIBLE_DEVICES in the parent before the workers spawn, so the engine still binds cuda:<rank>.
/v1/stats reports the engine's GPU under "gpus".
ft bench bw takes the same --gpu and writes one profile per GPU (benchbw/<gpu-uuid>.json); the legacy benchbw.json is still read by GPU name.
The daemon's /bench/profile returns the running serve's GPU profile.
BREAKING CHANGE: ft checkpoint --device is removed; use --gpu.
* refactor!: resolve --gpu via NVML and bind by UUID, not CUDA_VISIBLE_DEVICES
BREAKING CHANGE: ft bench bw --device is removed; use --gpu.
* refactor: name the id namespaces and drop the multi-device e4m3 scan
Splits the published id into _assigned_physical (UUID) and
_assigned_visible (CUDA ordinal). One process runs on one GPU, so
e4m3_native() judges that card instead of scanning every visible device.