* feat(spec): add the speculative verify infrastructure
* feat(scheduler): run decode steps as speculative verify steps
* chore(engine): mark the speculative decoding config fields with one comment
* feat(server): add short aliases for the speculative decoding flags
* test(deepseek-v41): add independent text and vision reference
* feat(quant): support block-32 and unrounded inference outputs
* feat(attention): add csa2 packed cache and sparse backend
* feat(deepseek-v41): serve text and images through shared runtime
* perf(mhc): tune batch-stable reduction tiles
* perf(moe): autotune fp4 decode launch geometry
* perf(moe): admit recurring misses against actual lru victims
* revert: perf(moe): admit recurring misses against actual lru victims
This reverts commit 1652d200f0ebb825379cf44c28baf52971e3a461.
* perf(mhc): derive the reduction tile from the width, not a timing
* perf(moe): replace the ds_fp4 decode autotune with a fixed two-warp launch
* fix(deepseek-v41): fence engram staging against in-flight lookups
* fix(moe): return the cpu routed sum in fp32 and round it in python
* fix(deepseek-v41): keep the vision tower and aligner unquantized
* fix(scheduler): admit bounded-replay hits by the history their resume reads
* refactor(deepseek-v41): name the attention family after the model, not csa2
* refactor(kvcache): import the dsv41 pool only for dsv41 models
* refactor(deepseek-v41): name the replay switch --swa-decoder-replay
* fix(server): keep reasoning_effort a string on the wire
* refactor(models): keep the prefill replay hooks off BaseLLMModel
* refactor(deepseek-v41): drop the fp32 routed sum and batch-invariant prefill
* fix(moe): weight ds_fp4 routes in the down epilogue again
* docs(models): anchor the v4.1 replay paragraph on the tech report
* refactor(deepseek-v41): run the decoder source's narrowed prefill on Block
* refactor(kvcache): let the dsv41 pool own the history a resume keeps live
* refactor(deepseek-v41): resume bounded-replay hits by capping the prefix match
* refactor(deepseek-v41): describe the attention group as a dsv4 variant and derive its geometry from the args
* refactor(deepseek-v41): keep the engram table locator in the model package
* refactor(deepseek-v41): drop the fp32 lm head output
- return the head's bf16 logits, as DeepSeek-V4 does
- revert ParallelLMHead(out_dtype=) and LinearKernel.apply_out_dtype, which only this call used
- remove test_head_preserves_fp32_logit_margin
- cast test_vision's prefill logits to fp32 before comparing, as the harness does
* refactor(quant): carry the w4a8 activation block on MoEConfig
- drop ActDesc and QuantScheme.act; the scheme only describes stored tensors
- Fp8BlockConfig sets MoEConfig.act_block from weight_block_size for its fp4 experts via a layer_config hook
- the ds_fp4 GPU kernel and CPU executor read the block from the layer config and require it
* refactor(deepseek-v41): size the kv pool from the args, not a geometry copy
- delete kvcache/dsv41_geometry.py; the pool and cost model read dsv41_args by attribute, as the DSV4 pool reads dsv4_args
- source / ratio aliasing reads args.roles; the duplicated source and ratio validation is gone
- the cost model owns the tier row formats, row bytes and the private window layers
- the pool keeps the format and window/ratio checks, and exposes the formats and is_private_window to the attention backend
* refactor(kvcache): move the dsv4 pool into kvcache/dsv4
* refactor(kvcache): move the dsv41 pool into kvcache/dsv4
* chore(benchmarks): drop the dsv41 selection and fp8 linear benchmarks
* test(deepseek-v41): drop tests that repeat other coverage
* test(deepseek-v41): load the tiny model's resident experts as banks
* fix(server): parse deepseek-v4.1's space-led dsml tool calls
---------
Co-authored-by: andy-yang-1 <andy_yang@berkeley.edu>
Serve Qwen3.8-Flash-Next (HF model_type qwen4_exp) text-only: 36 GDN +
12 QSA compressed-sparse attention layers on 4 hyper-connection residual
streams, a PLE n-gram embedding layer backed by a 47.7 GiB pinned-host
table with UVA gather, and 512 NVFP4 / block-fp8 routed experts (top-10)
plus a gated shared expert.
- attention: qsa_sparse backend (AttnType.QSA) over QSAKVCache -- paged
GQA K/V, a 1/ratio compressed index-key slab shadowing the KV pages,
and a per-request pending ring sized from index_ratio
- kvcache: declarative slot-sibling states (ModelConfig.slot_states) on
LinearStatePool carry the PLE conv history and n-gram context through
the hybrid-radix snapshot/COW lifecycle
- scheduler: hybrid prefill chunks align to the page size so snapshots
land on donatable boundaries
- kernels: triton kernels adapted from vLLM/SGLang (hc, qsa, ple gather,
moe router / shared gate) plus an original radix block top-k; int64
row addressing throughout
- moe: non-power-of-2 top-k router, deep-K marlin decode config, one
fp8 scale-bank padding rule shared with the AOT row table
- engine: the PLE table load reserves its pinned bytes from the pin
budget before the expert banks plan their residency