Files
tokenspeed/python
Nicolas Perrin fcf48aea6c refactor(models): share the DeepSeek-layout NextN draft loader (#3648)
## Summary

The DeepSeek NextN draft (`deepseek_nextn.py`) and the GLM-MoE-DSA NextN
draft (`glm_moe_dsa_nextn.py`, which `DeepseekV32ForCausalLMNextN`
reuses) each carried a copy of the same code:

- the NextN model's `__init__`;
- the checkpoint-name mapping for the NextN layer;
- the `load_weights` loop: stacked MLP, MoE experts, the fused q/kv
a-projection pair, and plain tensors;
- the q/kv a-projection pairing state and its
`begin`/`abort`/`end_weight_update` hooks.

They now share it:

- `DeepseekModelNextN` builds its decoder from a `decoder_cls` class
attribute. `GlmMoeDsaModelNextN` subclasses it, sets `decoder_cls =
GlmMoeDsaDecoderLayer`, and keeps its own `forward`.
- A new `DeepseekNextNWeights` mixin in `deepseek_nextn.py` holds the
name mapping, the loader loop and the three update hooks. Both draft
classes list it first.
- Per-model behavior stays in each class:
- **DeepSeek** fails on a parameter name it doesn't know. **GLM** looks
it up with `get_param` and skips it if missing. A small `_nextn_param`
hook keeps both behaviors.
- **GLM's `load_weights`** loads its indexer weights as they stream by
and passes every other tensor on to the shared loop, in the same order
as before.
- Each model keeps its own `forward`, LM head,
`checkpoint_weight_name_filter` and `post_load_weights`.

**One behavior change:** a checkpoint name with a non-numeric layer
index, such as `model.layers.x.mlp.gate.weight`, used to make the
DeepSeek draft raise on `int()`. It is now skipped, as the GLM draft
already did. Well-formed names map the same way as before.

No model forward code changes. 172 fewer lines.

## Test Plan

On GB200, main (588408510) against this branch.

- [x] **Same weights after loading.** Tiny real DeepSeek-V3, GLM-MoE-DSA
and DeepSeek-V3.2 NextN drafts were built on one GPU, and each loaded a
synthetic checkpoint in a shuffled order. The checkpoint covered:
- the NextN layer's attention, with q_a/kv_a pairs, norms, the MoE gate
and correction bias, each expert, and the shared experts;
  - the GLM indexer weights;
- names the loader skips: target layers, the LM head, embeddings, rotary
`inv_freq`, an extra bias, and an unknown name for GLM.

  The drafts then went through three more steps:
- a weight-update session whose q_a and kv_a arrive in separate
`load_weights` calls;
  - a session that never delivers kv_a;
  - two malformed names.

After each step, every parameter, buffer and tensor attribute was
hashed, along with the errors and the pending pairs: 365 entries in all.
All match main except the two malformed-name results above. Two runs of
main give identical results.
- [x] **Same structure.** The three drafts have the same parameter and
buffer names, shapes and dtypes, and the same module types, as on main.
- [x] **Same test results.** 34 test files covering these models, from
DeepSeek V3/V4.1, GLM and dots3 to the NextN shard filter and
weight-update sessions, give identical per-file results: 1146 passed and
51 skipped on both sides. 47 tests fail on both sides, in
`test_kimi_k31`, `test_qwen35_nextn_quantization` and
`test_mla_lora_scale`, the same as on main.
- [x] **Every module imports.** All modules under
`tokenspeed.runtime.layers` and `tokenspeed.runtime.models` import: 149
of 149.
- [x] **New test.** `test/runtime/test_nextn_draft_weights.py` builds
real DeepSeek and GLM drafts and checks that:
- the NextN layer lands on the draft and its decoder, and a target
layer's tensor is skipped;
- GLM's indexer `wk` and `weights_proj` fill the fused projection when
they arrive mid-stream;
  - a GLM q_a/kv_a pair split across update chunks is fused.

  It passes on main too.
- [x] pre-commit on the changed files.

Signed-off-by: Nicolas <nperrin.ai@gmail.com>
2026-11-19 14:26:12 -05:00
..
2026-05-06 13:27:02 +00:00