Yueming Yuan
6d0cd93f10
DeepSeek-V4 support-v2: native megatron implementation as option ( #2706 )
2026-08-31 13:21:05 -07:00
Yuzhen Zhou and Jiajun Li
cc92260df1
fix: align MLA RoPE types with model configs ( #2716 )
...
Co-authored-by: Jiajun Li <guapisolo@gmail.com >
2026-08-23 04:14:30 -07:00
Zhichen Zeng
bb6f7dbd31
feat: add the Qwen3.8-27B recipe ( #2575 )
2026-08-16 14:50:54 -07:00
Yueming Yuan
221dbc7eaa
Delete the glm4-9B, mimo-7B, moonlight-16B and deepseek-r1 launch scripts ( #2354 )
2026-08-11 18:22:26 -07:00
Jiajun Li and Yuzhen Zhou
26c82b599c
fix(kimi): align YaRN parameters with checkpoints ( #2377 )
...
Co-authored-by: Yuzhen Zhou <zyzshishui@gmail.com >
2026-08-11 14:03:51 -07:00
fzyzcjy
fe30593dd6
Replace the model config shell scripts with python ( #1910 )
2026-08-09 18:51:21 +08:00
Zhichen Zeng and yueming-yuan
98eec31762
Nemotron RL support ( #1284 )
...
Co-authored-by: yueming-yuan <yym022502@gmail.com >
2026-08-04 22:49:18 -07:00
Zhichen Zeng
5c517599fb
[tml] Inkling model support ( #1683 )
2026-08-03 13:59:01 -07:00
Ethan (Yusheng) Su
bf536263c4
[Lora] Add GDN (Qwen3.5/Qwen3-Next) LoRA target-module mapping ( #1597 )
...
Signed-off-by: Yusheng Su <yushengsu.thu@gmail.com >
2026-07-15 14:15:35 -07:00
5bdd942329
Add GLM-5/5.1/5.2 (744B MoE) LoRA RL support ( #1559 )
...
Signed-off-by: Yusheng Su <yushengsu.thu@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: yueming-yuan <yym022502@gmail.com >
2026-07-07 15:46:00 -07:00
Zhichen Zeng and Yueming Yuan
6ccc2cab6d
[feat] Gemma-4 model support ( #1232 )
...
Co-authored-by: Yueming Yuan <yym022502@gmail.com >
2026-06-24 12:44:14 -07:00
Yueming Yuan
4dfe338576
Support GLM-5.2 744B-A40B ( #1376 )
2026-06-22 13:26:51 -07:00
Ziang Li and yueming-yuan
3d56ef6a9b
Support jdopensource/JoyAI-LLM-Flash ( #838 )
...
Co-authored-by: yueming-yuan <yym022502@gmail.com >
2026-06-18 14:07:31 -07:00
Zhichen Zeng
0cec1eff84
CI: fix kimi-2.5 oom ci ( #1346 )
2026-06-15 22:17:46 -07:00
Zhichen Zeng and Yueming Yuan
e719999efe
CI: Kimi-K2.5 python launch and 4-layer CI test ( #1280 )
...
Co-authored-by: Yueming Yuan <yueming.yuan@radixark.ai >
2026-06-12 17:26:44 -07:00
39bf9b0018
deepseek v4 model support ( #1181 )
...
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com >
Co-authored-by: Zhichenzzz <zhichen.zeng@radixark.ai >
Co-authored-by: Zhichenzzz <zczeng@uw.edu >
2026-06-04 20:12:04 -07:00
Ziang Li and yueming-yuan
5a19382403
Support DeepSeek v3.2 ( #963 )
...
Co-authored-by: yueming-yuan <yym022502@gmail.com >
2026-05-28 14:20:16 -07:00
Zhichen Zeng
14fafadaa4
fix: expert_overlap and pp issues in nemotron-3-super ( #1150 )
2026-05-19 23:18:19 -07:00
Lawrence Wu
95e10bcbe3
fix: quote $MOE_LAYER_FREQ ( #1094 )
2026-05-10 15:17:47 -07:00
Zhichen Zeng
372846f1b8
Feat: NVIDIA Nemotron-3 RL support ( #1033 )
2026-04-30 17:57:24 -07:00
Zhichen Zeng
1dd7dec7e3
Feat: Qwen3.6 RL support ( #1022 )
2026-04-30 17:57:03 -07:00
3e0b5a44e5
[rollout] (4.5/N) Update swe-agent-v2 example scripts using current TITO design. ( #747 )
...
Co-authored-by: maocheng23 <maocheng@berkeley.edu >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-29 17:36:38 -07:00
2354ce836a
[model] Add Qwen3.5 model support (4B, 9B, 27B and 35B-A3B) ( #740 )
...
Co-authored-by: Jiajun Li <guapisolo@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-23 16:16:27 -07:00
c9b0939d01
[model] Support qwen3 next mtp training ( #655 )
...
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Yueming Yuan <yym022502@gmail.com >
2026-03-22 14:49:56 -07:00
fzyzcjy
4d45205c09
Parameterize num-layers in qwen3-30B-A3B model script to support 5 layer model ( #765 )
2026-03-20 15:08:29 +08:00
maocheng23
15f4d41724
Fix/glm 4.7 flash script ( #759 )
2026-03-19 20:02:31 -07:00
Jiajun Li and Wenjie Ma
46556aeb94
[model] Add gpt-oss-20b-bf16 script ( #590 )
...
Co-authored-by: Wenjie Ma <55629401+euclidgame@users.noreply.github.com >
2026-03-12 15:45:19 -07:00
Yueming Yuan and gemini-code-assist[bot]
600ac74529
[model] support GLM-5 ( #626 )
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-27 22:16:34 -08:00
Jiajun Li and Yueming Yuan
88187f9b40
[feat] glm-4.7-flash support with r3+mtp ci ( #623 )
...
Co-authored-by: Yueming Yuan <yym022502@gmail.com >
2026-02-23 13:53:52 -08:00
miles-code-angel
8a988dc3d2
[auto-sync] update code ( #383 )
2026-01-02 18:42:42 -08:00
miles.pr.bot
810eaa4274
update code
2025-11-24 14:55:30 +08:00
fzyzcjy
256dc3949c
Super tiny fix duplicated and unused args ( #240 )
2025-11-24 14:44:29 +08:00
fzyzcjy
947dd6d955
Tiny fix model config error when sourcing files ( #177 )
2025-11-15 18:15:21 +08:00
miles.pr.bot
3a1692e22a
update code
2025-11-15 10:25:45 +08:00
fzyzcjy
adfe3ce448
Super tiny add truncated-layers deepseek model config ( #162 )
2025-11-14 21:26:30 +08:00
fzyzcjy
3338791269
Tiny unify qwen3 instruct model variant scripts ( #149 )
2025-11-14 15:47:13 +08:00
fzyzcjy
fda225312e
Super tiny add Qwen3-4B-Instruct-2507 model config ( #128 )
2025-11-01 17:26:38 +08:00
miles.pr.bot
a9ad745a9a
update code
2025-11-01 17:06:56 +08:00
fzyzcjy
950e5518b1
Super tiny add Qwen3 1.7B config and generalize test script ( #35 )
2025-10-13 11:53:03 +08:00
Ying Sheng
10c8163382
a new start
2025-10-08 23:09:54 -07:00
Zilin Zhu
ab13a4aed8
[model] Support Qwen3-Next-80B-A3B ( #417 )
...
* [model] Support Qwen3-Next-80B-A3B
* bugfix
* use fla's ShortConvolution and support data packing
2025-10-04 22:12:07 +08:00
Gao016 and gaochang
a29386384f
fix bug for convert Qwen3-235B-A22B HF model weight to Megatorn torch_dist format ( #386 )
...
* for qwen3-32B task
* fix bug for convert Qwen3-235B-A22B HF model weight to Megatorn torch_dist
---------
Co-authored-by: gaochang <gaochang@U-19PX2WQ1-0350.local >
2025-09-26 22:07:49 +08:00
Zilin Zhu
aa3375040b
[model] fix qwen3-14B.sh
2025-09-12 02:50:19 -07:00
Gao016 and gaochang
b806b5ca50
for qwen3-32B task ( #253 )
...
Co-authored-by: gaochang <gaochang@U-19PX2WQ1-0350.local >
2025-09-03 23:16:59 +08:00
Yuchen Fan
9697b8f4bf
fix: modify the rotary-base to 1000000 for consistency ( #252 )
2025-09-03 15:22:56 +08:00
zlHuang
1fc0fae8a8
Create qwen3-32B.sh ( #208 )
2025-08-30 20:24:21 +08:00
zlHuang
ba3ea21262
Create qwen3-14B.sh ( #207 )
2025-08-30 20:24:02 +08:00
Jiajun Li
176b2eb792
Add XiaomiMiMo/MiMo-7B-RL with MTP support ( #128 )
...
* merge commits
* fix setting in sh
* use fa3 instead of flashInfer to avoid bug
* update params and comments for mimo-rl
2025-08-26 10:51:16 +08:00
Ethan (Yusheng) Su
36711aa668
[Hardware] AMD Dockerfile update and test ( #158 )
...
* update the dockerfile
* update the doc
* revise aiter and sglang version
* update the dockerfile
* update
* update the training script
* update docker/Dockerfile.rocm
* update
* support llama3
* update
* update
* del model_convert.sh
* add the converted model resource
* update dockerfile
* update doc
* update
* formate
* add the docker/dockerfile supportted commit ID - Need to fix commit ID:c22f55b arised issue on AMD GPU in future
2025-08-23 09:09:55 +08:00
Zilin Zhu
9d9ab78ba1
fix qwen2.5-1.5B model args
2025-08-13 08:11:19 -07:00
Yueming Yuan