deployment skill: add SGLang cookbook cross-check (analog of recipes.vllm.ai) (#1685)

### What does this PR do?

Type of change: documentation (agent skill)

The `deployment` and `evaluation` skills cross-check vLLM launch
commands against `recipes.vllm.ai`, but the SGLang reference had **no**
equivalent authoritative recipe source. This adds the **SGLang
cookbook** (`docs.sglang.io/cookbook`) — the direct SGLang analog of
`recipes.vllm.ai`, using the same `(hw, variant, quant, strategy,
nodes)` variant-tuple command generator — as the cross-check source for
SGLang deployments.

Changes (canonical `.agents/` source; `.claude/` are symlinks):
- **`SKILL.md`**: cookbook cross-check note under the SGLang
quick-start, mirroring the existing vLLM cu130/sm_103 block — covers the
JS-rendered (Mintlify) page, the raw-markdown fallback in
`sgl-project/sglang` `docs_new/`, the URL-fragment variant selection,
and the SM120 (RTX PRO 6000) nightly-image gotcha.
- **`references/sglang.md`**: a new authoritative-recipe section, a
MoE/FP4 backend flag matrix (`flashinfer_mxfp4` vs `marlin`, `deepep` vs
`megamoe`), and per-hardware notes (Blackwell / Hopper / RTX PRO 6000).

### Usage

N/A — documentation only. Agents now cross-check SGLang launch flags
against `docs.sglang.io/cookbook/<category>/<org>/<model>`, fetching the
raw markdown at
`github.com/sgl-project/sglang/blob/main/docs_new/cookbook/<category>/<org>/<model>.mdx`
when the page is JS-rendered.

### Testing

Markdown-only change to skill docs; no code paths affected.

### Before your PR is "*Ready for review*"

- Is this change backward compatible?: ✅
- If you copied code from any other sources or added a new PIP
dependency, did you follow guidance in `CONTRIBUTING.md`: N/A
- Did you write any new necessary tests?: N/A (docs only)
- Did you update Changelog?: N/A (agent skill docs, not a library
feature/API change)
- Did you get Claude approval on this PR?: ❌ (pending)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Added authoritative cookbook cross-check guidance for SGLang
deployments and how to select model variants via recipe parameters.
* Explained how to fetch raw cookbook content when rendered pages omit
commands and which flag areas the cookbook governs.
* Expanded backend/strategy guidance for MoE, dispatch modes, and
runtime tuning (batching, request sizing, CUDA-graph).
* Added hardware-specific notes including RTX PRO 6000 nightly-image
requirement and processor-specific recommendations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
This commit is contained in:
Zhiyu
2026-06-12 07:57:21 +00:00
committed by GitHub
parent 8a973ce90d
commit cc17f2c459
2 changed files with 64 additions and 0 deletions
+12
View File
@@ -148,6 +148,18 @@ python -m sglang.launch_server \
--host 0.0.0.0 --port 8000
```
For NVFP4 checkpoints, use `--quantization modelopt_fp4`.
> **Cross-check SGLang launch flags via the SGLang cookbook** (the SGLang analog
> of `recipes.vllm.ai`): `docs.sglang.io/cookbook/<category>/<org>/<model>` (e.g.
> `.../autoregressive/DeepSeek/DeepSeek-V4`) — authoritative for parallelism, MoE
> backends, strategy flags, Docker image, and min version. Select the variant via
> the URL fragment `#hw=...&variant=...&quant=...&strategy=...&nodes=...`. The
> page is **JS-rendered** — fetch the raw markdown at
> `raw.githubusercontent.com/sgl-project/sglang/main/docs_new/cookbook/<category>/<org>/<model>.mdx`.
> SM120 (RTX PRO 6000) needs the `lmsysorg/sglang:dev` nightly (`:latest` lacks
> SM120). See `references/sglang.md` for the full backend/flag matrix.
#### TRT-LLM (direct)
```python
@@ -1,5 +1,28 @@
# SGLang Deployment Reference
## Authoritative recipe source — the SGLang cookbook
For any non-trivial model (large MoE, multi-node, Blackwell FP4), **cross-check
the launch command against the SGLang cookbook** before hand-rolling flags. It
is the SGLang analog of `recipes.vllm.ai`: a verified command generator keyed on
a `(hw, variant, quant, strategy, nodes)` tuple.
- **URL:** `docs.sglang.io/cookbook/<category>/<org>/<model>` — e.g.
`docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4`. Select the
variant via the URL fragment, e.g.
`#hw=b200&variant=flash&quant=fp4&strategy=low-latency&nodes=single`.
- **JS-rendered (Mintlify).** A plain fetch usually misses the commands. Fetch
the **raw markdown** instead:
`raw.githubusercontent.com/sgl-project/sglang/main/docs_new/cookbook/<category>/<org>/<model>.mdx`
(the URL path maps directly onto the directory tree). The old
`sgl-project/sgl-cookbook` repo is archived — the live source is `docs_new/`
in the main `sgl-project/sglang` repo.
- **Authoritative for:** parallelism layout (`--tp` / `--ep`, plus
multi-node/data-parallel settings), MoE backends,
strategy-driven flags (MTP, CUDA-graph batch sizing), Docker image tag, and the
minimum SGLang version for the chosen variant. Pull the layout/flags from the
cookbook, then adapt to the GPUs you actually have.
## Requirements
- SGLang >= 0.4.10
@@ -72,6 +95,35 @@ Reference: `examples/specdec_bench/specdec_bench/models/sglang.py`
| `--cuda-graph-max-bs` | Max batch size for CUDA graphs |
| `--attention-backend` | `flashinfer` (default) or `triton` |
## MoE / FP4 backend flags (Blackwell vs Hopper)
For large MoE models the cookbook recipe is authoritative — these are the flags
it selects per hardware. Quantized DeepSeek-style checkpoints are FP4 experts +
FP8 attention/dense.
| Flag | Use |
|------|-----|
| `--moe-runner-backend flashinfer_mxfp4` | Default FP4 MoE runner on Blackwell (SM100/SM103) |
| `--moe-runner-backend marlin` | Hopper W4A16 FP4 MoE runner |
| `--moe-a2a-backend deepep` | Default expert all-to-all backend |
| `--moe-a2a-backend megamoe` | Blackwell-only, high-throughput strategy |
| `--deepep-mode auto\|normal\|low_latency` | DeepEP dispatch mode |
Strategy (low-latency / balanced / high-throughput) tunes
`--cuda-graph-max-bs`, `--max-running-requests`, and MTP (Multi-Token
Prediction) draft steps/tokens — take these from the cookbook variant rather
than guessing.
## Hardware notes
- **Blackwell B200/B300/GB200/GB300 (SM100/SM103):** use the
`flashinfer_mxfp4` MoE runner; `megamoe` only for the high-throughput
strategy.
- **Hopper (H100/H200):** FP4 runs via Marlin W4A16 kernels. Converted FP8
checkpoints (`sgl-project/DeepSeek-V4-*-FP8`) exist for richer parallelism.
- **RTX PRO 6000 (SM120):** `:latest` does **not** support SM120 — use the
`lmsysorg/sglang:dev` nightly image.
## Common Issues
| Issue | Fix |