310 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Ethan (Yusheng) Su	36711aa668	[Hardware] AMD Dockerfile update and test (#158) ... * update the dockerfile * update the doc * revise aiter and sglang version * update the dockerfile * update * update the training script * update docker/Dockerfile.rocm * update * support llama3 * update * update * del model_convert.sh * add the converted model resource * update dockerfile * update doc * update * formate * add the docker/dockerfile supportted commit ID - Need to fix commit ID:c22f55b arised issue on AMD GPU in future	2025-08-23 09:09:55 +08:00
fy1214	68f040a86e	[fix] fix the inplace div backward (#214)	2025-08-23 09:09:35 +08:00
Chengxing Xie	bfdb1fffc7	fix wandb error (#213)	2025-08-22 19:04:25 +08:00
Chengxing Xie	a4dccc6604	add rollout actor (#210)	2025-08-21 22:27:21 +08:00
Zilin Zhu	e943681211	[feat] support training with sglang draft model without mtp (#204) ... * [feat] support training with sglang draft model without mtp * fix lint * fix lint	2025-08-20 21:07:42 +08:00
Zilin Zhu	d3306394d2	bugfix for gspo with long prompt	2025-08-20 03:25:36 -07:00
Zilin Zhu	8e994cecba	always cpu actor weights after training	2025-08-17 19:57:43 -07:00
Chengxing Xie	3bbcbcc704	fix actor udpate bug (#198)	2025-08-18 10:47:34 +08:00
Xiang Long	7f3fd93e48	[feat] Add wandb offline support (#194) ... * Add wandb offline and dir support * Fix offline logic * Remove redundant getattr	2025-08-18 10:32:40 +08:00
Chengxing Xie	d7fe7e46c2	fix wandb log fail error (#192)	2025-08-15 12:41:47 +08:00
Zilin Zhu	968f6deb10	use token-in token-out as default (#191)	2025-08-15 11:47:15 +08:00
Zilin Zhu	2275717282	[script] increase --sglang-mem-fraction-static to 0.7	2025-08-14 20:28:57 -07:00
Zilin Zhu	6ae99c3352	Revert "Minor extract generate one sample (#168)" (#190) ... This reverts commit fe5e4e95bf.	2025-08-14 19:23:01 +08:00
Zilin Zhu	9da9d55e81	bugfix	2025-08-14 01:53:10 -07:00
Zilin Zhu	c36a0142f7	fix README	2025-08-14 01:11:02 -07:00
Zilin Zhu	3b4eeabf3f	[refactor] rename buffer to rollout controller and extract buffer to data source (#189)	2025-08-14 16:06:26 +08:00
Zilin Zhu	dfd91a58ef	Revert "Add new API for rollout_fn (#169)" (#188) ... This reverts commit 4c253edb9d.	2025-08-14 15:08:16 +08:00
Zilin Zhu	70564d9f5c	Revert "Extract buffer logic to legacy adapter rollout fn (#171)" (#187) ... This reverts commit 118b6a02b4.	2025-08-14 14:56:53 +08:00
Zilin Zhu	d0712a50bb	Revert "Implement PartialRolloutFn abstraction for clean partial rollout (#172)" (#186) ... This reverts commit 2a95a217c2.	2025-08-14 14:56:24 +08:00
Zilin Zhu	a96a02e9d1	Revert "Remove singleton GenerateState (#173)" (#185) ... This reverts commit 27ef0076f3.	2025-08-14 14:55:39 +08:00
Zilin Zhu	fdb203b917	Revert "Remove several fields from global rollout state (#174)" (#184) ... This reverts commit 51dd193d0b.	2025-08-14 14:54:33 +08:00
Zilin Zhu	0197de44f4	Revert "Minor cleanup sglang_rollout (#175)" (#183) ... This reverts commit 2dbeb3a44c.	2025-08-14 14:53:35 +08:00
Zilin Zhu	25964b6d84	Revert "Refactor sglang rollout main loop logic (#176)" (#182) ... This reverts commit 9d0697db56.	2025-08-14 14:52:37 +08:00
Zilin Zhu	9d9ab78ba1	fix qwen2.5-1.5B model args	2025-08-13 08:11:19 -07:00
zlHuang	252e891a6b	Update actor.py (#162) ... In self.compute_log_prob, the model weights on the GPU are updated according to the tag. This step seems to be repeated.	2025-08-13 22:47:31 +08:00
Zilin Zhu	f2d60714ba	don't set LD_PRELOAD when not offloading	2025-08-13 07:45:54 -07:00
fzyzcjyandZilin Zhu	9d0697db56	Refactor sglang rollout main loop logic (#176) ... * more * more * more * more * more * fmt * mv * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * rm * Revert "rm" This reverts commit 53184f608f501af4757709cd396ae94516f00dd6. * cp * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * fmt * more * more * more * more * fmt * more * fmt * more * more * more * remaining_batch_size * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * cherry pick --------- Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>	2025-08-13 21:41:32 +08:00
fzyzcjyandZilin Zhu	2dbeb3a44c	Minor cleanup sglang_rollout (#175) ... * more * more * more * more * more * fmt * mv * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * rm * Revert "rm" This reverts commit 53184f608f501af4757709cd396ae94516f00dd6. * cp * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * fmt * more * more * more * more * fmt * more * fmt * more * more * more * cherry pick --------- Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>	2025-08-13 21:28:05 +08:00
fzyzcjyandZilin Zhu	51dd193d0b	Remove several fields from global rollout state (#174) ... * more * more * more * more * more * fmt * mv * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * rm * Revert "rm" This reverts commit 53184f608f501af4757709cd396ae94516f00dd6. * cp * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * fmt * more * more * more * more * fmt * cherry pick --------- Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>	2025-08-13 21:24:13 +08:00
fzyzcjyandZilin Zhu	27ef0076f3	Remove singleton GenerateState (#173) ... * more * more * more * more * more * fmt * mv * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * rm * Revert "rm" This reverts commit 53184f608f501af4757709cd396ae94516f00dd6. * cp * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * fmt * more --------- Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>	2025-08-13 21:21:51 +08:00
Haoboandzhanghaobo	074e861c61	add mtp layer for convert hf->distcp (#177) ... Co-authored-by: zhanghaobo <haobo.zhang@aminer.cn>	2025-08-13 19:05:56 +08:00
fzyzcjyandZilin Zhu	2a95a217c2	Implement PartialRolloutFn abstraction for clean partial rollout (#172) ... * more * more * more * more * more * fmt * mv * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * rm * Revert "rm" This reverts commit 53184f608f501af4757709cd396ae94516f00dd6. * cp * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * fmt --------- Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>	2025-08-13 19:04:41 +08:00
fzyzcjyandZilin Zhu	118b6a02b4	Extract buffer logic to legacy adapter rollout fn (#171) ... * more * more * more * more * more * fmt * mv * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * rm * fmt --------- Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>	2025-08-13 18:55:18 +08:00
fzyzcjy	4c253edb9d	Add new API for rollout_fn (#169) ... * more * more * more * more * more * fmt * mv * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * more * fmt	2025-08-13 17:38:17 +08:00
fzyzcjy	fe5e4e95bf	Minor extract generate one sample (#168) ... * more * more * more * more * more * fmt * mv	2025-08-13 17:17:07 +08:00
fzyzcjy	92b7530b99	Minor refactor noset visible devices (#167) ... * more * mv * more * fmt	2025-08-13 17:16:04 +08:00
Zilin Zhu	4bd54da81e	[bugfix] don't destroy pg when not offloading	2025-08-12 22:53:29 -07:00
Zilin Zhu	fb2234c156	[docker] update docker for torch_memory_saver	2025-08-12 08:27:45 -07:00
Zilin Zhu	336db0e4f6	[memory] use torch_memory_saver and remove --experimental-offload (#165)	2025-08-12 23:21:31 +08:00
Zilin Zhu	264618c384	[docker] update patch and relevant bugfix	2025-08-11 23:37:46 -07:00
Zilin Zhu	2d84cc5a9e	fix typo	2025-08-11 01:37:04 -07:00
Zilin Zhu	8db583b745	backward compatible for flatten tensor	2025-08-11 01:04:54 -07:00
Stefan He	3c96ed2ab2	Support Flatten Tensor to accelerate Qwen3-30B-A3B from ~10s to ~6s (#155) ... * Flatten Tensor * fix precommit * add patch * fix git ignore * fix	2025-08-11 15:31:22 +08:00
Jiajun Li	c22f55ba6f	[bug fix] CUDA error: invalid device ordinal when CUDA_VISIBLE_DEVICES start id is not equal to 0 (#116) ... * fix on train side * fix on rollout side * revert noset cvd on rollout side * add notice in --rollout-num-gpus-per-node	2025-08-11 14:17:57 +08:00
kang sheng	c7951ed8da	fix typo (#152) ... * fix typo * fix typo	2025-08-11 13:35:53 +08:00
Zilin Zhu	b8cb97da03	[refactor] separate generate and eval in Buffer to make the code cleaner (#161)	2025-08-11 12:04:50 +08:00
Stefan He	079dbcfc3b	fix pre-commit (#160)	2025-08-11 11:58:55 +08:00
Zilin Zhu	f727894376	[refactor] move log_eval_data to Buffer (#159)	2025-08-11 11:42:02 +08:00
Yuzhen Zhou	9a48ba04d1	add llama config (#157) ... * fix * precommit * add llama config	2025-08-11 09:39:02 +08:00
Yuzhen Zhou	33ab6b76f8	fix (#156) ... * fix * precommit	2025-08-11 08:49:36 +08:00