04b61f31a1
Co-authored-by: Zhichen Zeng <zczeng@uw.edu> Co-authored-by: Shi Dong <shi.dong@radixark.ai> Co-authored-by: Yueming Yuan <yym022502@gmail.com> Co-authored-by: Shi-Dong <Shi-Dong@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Haoguang Cai <kai.cai@radixark.ai> Co-authored-by: Mingyi Lu <wisclmy0611@gmail.com> Co-authored-by: Jiajun Li <guapisolo@gmail.com>
3.1 KiB
title, description
| title | description |
|---|---|
| Experimental Features | Backends and features that exist in tree but are not production-ready — opt-in at your own risk. |
These features live in the Miles tree but are not production-ready. They typically have rough edges, missing parallelism, or known bugs against current dependency versions. Use them when you want to iterate quickly or co-develop a feature, not for the long-running training jobs you'd publish results from.
FSDP backend
A PyTorch FSDP2 training backend lives at miles/backends/fsdp_utils/.
It trades maximum throughput for zero conversion overhead: there is no
torch_dist step, Miles reads architecture information from the HuggingFace
config.json, and weights load directly via AutoModelForCausalLM.from_pretrained().
The distributed optimizer is built into FSDP, and mixed precision falls out of standard
PyTorch.
Status. Experimental. Known bug after SGLang v0.5.10. No TP / PP / CP / EP — runs as plain FSDP data parallel only. Suitable for fast iteration on small-to-mid dense models, not for production runs.
When to reach for it
- Iterating on a new model architecture and you don't want to write a Megatron spec yet.
- Small-to-mid dense workloads where the parallelism story doesn't matter.
- You want a HuggingFace-native checkpoint at every step with no conversion.
For large MoE models, multi-rack jobs, or anything where TP / PP / CP / EP matters, use the production Megatron-LM backend instead.
Enabling it
--train-backend fsdp
Flag mapping vs. Megatron
Most RL-level flags carry over unchanged. Backend-specific differences:
| Concern | Megatron | FSDP |
|---|---|---|
| Model load | --load + architecture args |
--hf-checkpoint (single flag, required) |
| Tensor parallel | --tensor-model-parallel-size |
Not supported yet |
| Pipeline parallel | --pipeline-model-parallel-size |
Not supported yet |
| Expert parallel | --expert-model-parallel-size |
Not supported yet |
| Context parallel | --context-parallel-size |
Not supported yet |
| Optimizer | --use-distributed-optimizer (forced on by Miles) |
Built-in |
| Gradient checkpoint | --recompute-granularity / method / num-layers |
--gradient-checkpointing (boolean) |
| CPU offload | Distributed optimizer | --fsdp-cpu-offload |
| CPU backend | (in distributed optimizer) | --fsdp-cpu-backend |
| Attention backend | Decided by Megatron Core | --attn-implementation flash_attention_2 / sdpa / eager |
| Mixed precision | --fp16 / --bf16 |
--fp16 (bf16 inferred) |
| Extra backend config | — | --config <yaml> |
Quick start
export WANDB_API_KEY=<key>
# Model + data
hf download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B
hf download --repo-type dataset BytedTsinghua-SIA/DAPO-Math-17K --local-dir /root/dapo-math-17k
hf download --repo-type dataset zhuzilin/aime-2024 --local-dir /root/aime-2024
# Code
git clone https://github.com/radixark/miles.git && cd miles
pip install -e . --no-deps
# Launch — no conversion step
bash scripts/run-qwen3-4B-fsdp.sh