Michael
4250360dfc
[AMD] ci: run ROCm CI on the ROCm 10 MI35X image ( #4514 )
2026-10-28 22:40:50 -07:00
HEJIAN SANG and Claude Opus 5.5
8855b832e4
Pass packed-document boundaries to HF attention so FSDP runs on FlashAttention-4 ( #4316 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-28 11:29:44 -07:00
HEJIAN SANG and Claude Opus 5.5
47c0899eae
[2/9] Add docker/Dockerfile.tpu: miles on TPU v7x from sglang-torchtpu's release image ( #4268 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-27 10:17:43 -07:00
Zhichen Zeng
843938ba14
[3/3] ci: track training MFU in the metric history ( #4098 )
2026-10-27 00:36:28 -07:00
Jiajun Li
b5f051f862
ci: rebalance eight-GPU tests from H200 to H100 ( #4332 )
2026-10-23 12:23:12 -07:00
Jiajun Li
37325a3d95
Install SGLang's radix_transfer_plugins in the CUDA image ( #4329 )
2026-10-23 11:04:15 -07:00
Xinyu Jiang and Zhiyao Jiang
1e4ad549a1
[AMD] ci: bump ROCm SGLang to v0.5.22 and expand MI350 nightly coverage ( #4202 )
...
Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com >
2026-10-19 18:23:06 -07:00
Jiajun Li
0309247cea
fix(ci): remove the default raw reward upper gate ( #4160 )
2026-10-17 20:55:01 -07:00
Jiajun Li
1647bd3b25
Bump sglang to v0.5.22 ( #4125 )
2026-10-16 17:43:49 -07:00
Jiajun Li
ff9853c0b1
ci: gate performance benchmarks on est_time with a perf label ( #4033 )
2026-10-14 16:12:38 -07:00
Jiajun Li
64842375d6
fix(ci): make the default grad_norm gate a 4x sanity band ( #4094 )
2026-10-14 14:51:58 -07:00
Jiajun Li
aa5dbf0b22
feat(ci): key metric history by metric and step ( #4089 )
2026-10-14 13:16:41 -07:00
Jiajun Li
ddb6113545
ci: keep NVSHMEM on NVLink on every RoCE-only runner ( #4088 )
2026-10-14 12:56:34 -07:00
Jiajun Li
3f5c754c58
feat(ci): enforce metric history automatically after ten goldens ( #4023 )
2026-10-14 12:50:15 -07:00
Jiajun Li
2766ab0ac1
ci: move ten 4x H200 tests that fit on two GPUs to 2x H200 ( #4053 )
2026-10-13 23:31:46 -07:00
Jiajun Li
77309a60d9
ci: move slow CPU tests to tagged Stage B registrations ( #4008 )
2026-10-10 15:55:38 -07:00
Jiajun Li
9bdc3daaca
ci: queue Hopper and ROCm shards without matrix throttling ( #4006 )
2026-10-10 14:26:22 -07:00
Jiajun Li
ccc8009ade
ci: split Hopper jobs to reduce failed-job rerun cost ( #4005 )
2026-10-10 02:41:27 -07:00
Jiajun Li
3a05ce560e
fix(ci): gate slow Kubernetes and rollout CPU tests by domain ( #3993 )
2026-10-09 02:24:55 -07:00
Yueming Yuan
dba9001cc6
ci: schedule B200 tests by minimum GPU count ( #3983 )
...
Schedule B200 tests with their minimum GPU count and dynamically lease free devices. Preserve test coverage, isolate concurrent jobs, and protect GPUs held by surviving job containers.
Bound admission waits with per-batch job budgets while preserving per-file execution timeouts.
2026-10-08 22:33:45 -07:00
Jiajun Li
6f94a01372
fix(ci): match H200 shards to runner capacity ( #3990 )
2026-10-08 19:25:54 -07:00
Zhichen Zeng
49b57dffea
[1/8] refactor(kernels): move quant, MoE, activation, norm, HC and RoPE kernels to miles/kernels ( #3605 )
2026-10-08 12:57:20 -07:00
Kaixi Hou
f81d52a40e
Add support for Vera Rubin ( #3700 )
2026-10-08 11:07:31 -07:00
Jiajun Li and Claude Opus 5.5
49464b3cd1
ci: offload the 8x H200 suite to H100, B200 and 4x H200 ( #3956 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 00:33:50 -07:00
Jiajun Li and Claude Opus 5.5
233a0cbe41
ci: write the rules for adding a CI test in .claude/rules ( #3953 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 00:33:50 -07:00
Jiajun Li
a0b2171b02
fix(ci): keep 4-GPU H200 DeepEP on NVLink for RoCE runners ( #3955 )
2026-10-07 22:47:11 -07:00
Jiajun Li
9350dda7ca
feat(ci): rerun PR workflows when labels are removed ( #3978 )
2026-10-07 22:46:44 -07:00
Jiajun Li
1e1966293e
fix(ci): cancel file tests and ROCm after PR merge ( #3927 )
2026-10-06 14:14:07 -07:00
Jiajun Li
ae1d5fa60c
fix(ci): skip GPU runners for empty test selections ( #3902 )
2026-10-06 13:36:48 -07:00
Michael and Xinyu Jiang
3d388872ac
[AMD] docker: add rocm10-mi30x (gfx942) image variant ( #3858 )
...
Co-authored-by: Xinyu Jiang <xinyuj2@andrew.cmu.edu >
2026-10-05 15:49:56 -07:00
Yueming Yuan
a095490625
ci: allow Megatron Bridge version overrides ( #3897 )
2026-10-02 19:37:28 -07:00
Jiajun Li and Yueming Yuan
53f5d94224
build: retire CUDA 12 Docker builds ( #3895 )
...
Co-authored-by: Yueming Yuan <yym022502@gmail.com >
2026-10-02 11:26:28 -07:00
Jiajun Li
64772f50b2
build: bump CUDA 13 images to SGLang v0.5.21 ( #3884 )
2026-10-02 00:00:16 -07:00
Jiajun Li
919a6502fe
ci(wheels): notify Lark when builds or publishing fail ( #3881 )
2026-10-01 15:32:18 -07:00
Jiajun Li and Claude Opus 5.5
1f92079cfc
ci(wheels): add a workflow that rebuilds the x86_64 TE wheels from the fork ( #3871 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 15:23:36 -07:00
Jiajun Li
957e11642f
fix(ci): preserve image cuDNN during dependency resolution ( #3880 )
2026-10-01 15:22:28 -07:00
Jiajun Li
55ec3d870f
test(e2e): rework e2e CI coverage on top of the H200 migration ( #3699 )
2026-09-30 15:54:37 -07:00
Xinyu Jiang and Zhiyao Jiang
874b3e2756
[AMD] docker: build the ROCm images on the sglang v0.5.20 release images ( #3873 )
...
Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com >
2026-09-30 00:10:03 -07:00
Jiajun Li
9e4260de04
Use the base image's Mooncake instead of reinstalling a dev wheel ( #3738 )
2026-09-27 21:55:14 -07:00
Jiajun Li and Claude Opus 5.5
f84b1496e1
test(e2e): move 8x H100 e2e tests to H200 stages and widen parallelism coverage ( #3660 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 20:41:43 -07:00
fzyzcjy
cee2e4b143
Cover the ray x rpc worker comm path in CI ( #3048 )
2026-09-26 20:32:29 +08:00
fzyzcjy
2ff226b1da
Install kubectl and helm into the training image ( #2191 )
2026-09-26 17:42:02 +08:00
Yueming Yuan
3d01116aee
release: gate on CPU and CUDA CI only ( #3705 )
2026-09-25 18:40:18 -06:00
Yueming Yuan
9e5cfd143a
release: publish CUDA 13 only and fingerprint configured wheels ( #3703 )
2026-09-25 01:37:23 -07:00
Xinyu Jiang and Zhiyao Jiang
e30b8b6dd6
[AMD] replace the ROCm 7.2.0 image with ROCm 7.2.4 ( #3326 )
...
Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com >
2026-09-24 03:59:24 -04:00
Jiajun Li
84ebad0569
ci(docker): notify Lark when an automatic image build fails ( #3639 )
2026-09-23 15:33:32 -07:00
Jiajun Li
d2b66cab68
ci(docker): poll upstream every 10 minutes and rebuild at least every 12h ( #3630 )
2026-09-23 15:29:41 -07:00
Jiajun Li
8b4d79bf50
ci: run PR Test on non-main-base PRs only with a run-ci label ( #3636 )
2026-09-23 15:27:23 -07:00
Yueming Yuan
a66064f190
fix(ci): install test dependencies before training pins ( #3580 )
2026-09-21 17:08:17 -07:00
Zhiyao Jiang and Xinyu Jiang
8d22a2c345
docs(AMD): Add an AMD ROCm page and fix the AMD image and hardware notes ( #3209 )
...
Co-authored-by: Xinyu Jiang <xinyuj2@andrew.cmu.edu >
2026-09-17 18:13:19 -07:00