Commit Graph
2154 Commits
Author SHA1 Message Date
Zhiyao Jiang 031fe3d727 ci: pin the DSv4 bshd/thd parity test to the miles impl (#3162) 2026-09-09 15:59:02 -07:00
Shi-DongandZhichenzzz f01f9a3166 fix: use host weights when syncing an offloaded actor (#3128)
Signed-off-by: Shi Dong <shi.dong@radixark.ai>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
2026-09-09 15:29:59 -07:00
Yueming Yuan fa7939f899 ci(lark): report every failing test in a job (#3168) 2026-09-09 15:17:32 -07:00
Yueming Yuan b6dc4301b5 fix(ci): keep the Nemotron-H plugin importable without megatron.bridge (#3169) 2026-09-09 15:17:16 -07:00
Shi-DongandZhichenzzz f1eac9bad4 Fix missing Nemotron-H Mamba convolution mappings (#3163)
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
2026-09-09 14:43:08 -07:00
Shi-Dong 021d311e89 Fix Nemotron-H forward compatibility when MTP is disabled (#3164) 2026-09-09 14:42:53 -07:00
Shi-Dong d8dd0dec8d Scan full responses for repetition metrics (#3115) 2026-09-09 12:04:50 -07:00
Yueming YuanandAlan Livshin 327eefa0fb ci(lark): explain each failed job on the CI status card (#3152)
Co-authored-by: Alan Livshin <alan.livshin@alivshin-mbp.tail134ba0.ts.net>
2026-09-09 11:31:52 -07:00
Yueming Yuan f207d590b0 ci: repeat a failed test's last output in the suite summary (#3149) 2026-09-09 11:28:40 -07:00
Yueming Yuan aa46eada5b [CI] disable the two FT determinism tests #3148 missed (#3160) 2026-09-09 10:57:44 -07:00
Xinyu JiangandZhiyao Jiang 8d9826eacc [AMD] docker: retire the rocm700 variants, add a rocm10-mi35x variant (#2854)
Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com>
2026-09-08 22:37:39 -07:00
7bf05390d7 [feat] Enable THD packed-sequence training for DeepSeek-V4-Flash, with context parallelism (#2038)
Co-authored-by: Xinyu Jiang <xinyuj2@andrew.cmu.edu>
Co-authored-by: guapisolo <guapisolo@gmail.com>
Co-authored-by: Yueming Yuan <yym022502@gmail.com>
2026-09-08 22:04:54 -07:00
Tao LinandClaude Fable 5 4f8e0b86f0 examples: launcher for the in-process Harbor path (#2836)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-08 21:52:41 -04:00
Yueming Yuan bf482051a8 Bound the trainer's GPU memory during the LoRA weight sync (#3147) 2026-09-08 18:11:40 -07:00
Tao LinandClaude Fable 5 4501b2655d smoke: manual golden episodes on real sandbox APIs (#2868)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-08 20:00:13 -04:00
Yueming Yuan 7d31a5ff1c [CI] temporarily disable failing FT tests (#3148) 2026-09-08 16:30:30 -07:00
Yueming Yuan aaab50cde1 Support --colocate-memory-peak-device gpu under LoRA (#3100) 2026-09-08 16:11:06 -07:00
Tao LinandClaude Fable 5 4dff6f2625 examples: in-process Harbor agent function (#2806)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-08 18:14:35 -04:00
Yueming Yuan 3de96596f1 Dump debug train data once per DP/CP shard instead of on every rank (#3141) 2026-09-07 23:10:32 -07:00
Zhichen Zeng db3d6b56c8 Bump sglang to v0.5.19 (#3124) 2026-09-07 22:04:14 -07:00
91e40b5423 Decide the pipeline stage from the parallel state in compute_advantages_and_returns (#3125)
Co-authored-by: Qijia Shen <qijia.shen@eigent.ai>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
2026-09-07 21:22:21 -07:00
Jiajun Li 78747f6912 refactor(docs): nest CI pages under developer (#3134) 2026-09-06 23:25:38 -07:00
Jiajun LiandDavid 4a58e4e101 fix(rollout): drop groups with missing rewards (#2814)
Co-authored-by: David <12414531+DavidBellamy@users.noreply.github.com>
2026-09-06 22:56:07 -07:00
Jiajun Li 5185f133e2 refactor(rollout): centralize common group filters (#3094) 2026-09-06 22:56:06 -07:00
Jiajun Li 51853a165a refactor(rollout): rename shared filter output (#3095) 2026-09-06 22:56:06 -07:00
Shi-Dong d2fc97ce58 Tolerate missing optional SGLang Anthropic helpers (#3114) 2026-09-04 14:48:36 -07:00
Zhichen ZengandVedant Gupta 7741150a29 feat(megatron): reset optimizer state by optimizer class (#3091)
Co-authored-by: Vedant Gupta <vedant.gupta@harmonic.fun>
2026-09-04 13:32:36 -07:00
fzyzcjy dc2726907b Fail CI runs on crashed async eval points or abnormal eval results (#2823) 2026-09-04 21:33:40 +08:00
fzyzcjy a6d743f8bf Await dynamic port probes instead of blocking the worker manager loop (#2898) 2026-09-04 15:03:33 +08:00
fzyzcjy 3408099cfc Make the FT api server bind host a flag defaulting to loopback (#2897) 2026-09-04 15:03:07 +08:00
fzyzcjy 704009f41f Reacquire the context lock safely when a released section is cancelled (#2896) 2026-09-04 15:02:39 +08:00
fzyzcjy 2495e4f817 Decide implicit healing after the config file overrides the args (#2895) 2026-09-04 15:02:12 +08:00
fzyzcjy bed46c206f Rename the trainer group and cell to TrainerController and TrainerCell (#2176) 2026-09-04 15:01:44 +08:00
fzyzcjy db698299c5 Serialize cell suspension and fault injection against weight updates (#2894) 2026-09-04 15:01:16 +08:00
fzyzcjy 4db21c40ac Make the train actor threaded only under fault tolerance (#2893) 2026-09-04 15:00:48 +08:00
fzyzcjy d60cee6dd7 Temporarily remove vanilla fault tolerance for the eval fleet (#2497) 2026-09-04 15:00:21 +08:00
fzyzcjy fb910cf3f4 Let workers deregister from the miles router (#2475) 2026-09-04 14:59:52 +08:00
fzyzcjy 1211351f1c Report a rollout cell stuck initializing as unhealthy (#2474) 2026-09-04 14:59:23 +08:00
fzyzcjy 36b5cc6c90 Bound and wrap the free-port scan (#2473) 2026-09-04 14:58:56 +08:00
fzyzcjy 307c9c75f4 Run the weight equality check before leaving StateInitializing (#2472) 2026-09-04 14:58:29 +08:00
fzyzcjy 4f6be0661a Fail the run when the FT api server cannot start (#2471) 2026-09-04 14:58:01 +08:00
fzyzcjy 3f74a8d8eb Assert that the rollout executor keeps receiving weight versions (#2470) 2026-09-04 14:57:33 +08:00
fzyzcjy 84b41252cb Publish the new weight version to the rollout executor (#2502) 2026-09-04 14:57:07 +08:00
fzyzcjy 38943d491c Stop the fast suite from starting its own ray clusters and leaking accelerators (#2311) 2026-09-04 14:56:41 +08:00
fzyzcjy 7108fa998a Route all trainer worker calls through the worker handle instead of Ray handles (#2175) 2026-09-04 14:56:11 +08:00
fzyzcjy c4ee942164 Add a ray worker handle with death confirmation (#2174) 2026-09-04 14:55:42 +08:00
fzyzcjy d034c9256e Kill failed trainer workers through their own RPC channel instead of the manager (#2173) 2026-09-04 14:55:17 +08:00
fzyzcjy e9cbd07400 Merge the now duplicated actor and rollout cell handlers (#2172) 2026-09-04 14:54:49 +08:00
fzyzcjy dfdb532b9a Remove the pending state in trainer cells (#2171) 2026-09-04 14:54:21 +08:00
fzyzcjy 4c324937d1 Remove the now-unused stopped trainer cell state (#2170) 2026-09-04 14:53:55 +08:00