trending/Model-Optimizer
1
0
Fork 0
mirror of https://github.com/NVIDIA/Model-Optimizer.git synced 2026-10-02 03:14:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

launcher: Nemotron-120B specdec_bench — kill stale _cells/ reference (post-PR-#1564) (#1741)

Browse Source 
## What

Comment-only change to the Nemotron-3-Super-120B-A12B-BF16 DFlash parent
YAML doc-comment header. Rewrites the example invocation from the
pre-PR-#1564 pattern (with `--runtime_params
common/specdec_bench/_cells/<sweep_name>.yaml`) to the current
post-#1564 pattern (CLI-flag-only overrides, no per-cell file).

No yaml content / config change — only the prose example in the header.

## Why

PR #1564 removed the `tools/launcher/common/specdec_bench/_cells/` and
`_runtime_params/` directories. Cell-specific knobs are now CLI
overrides at slurm-invoke time, not committed files. The cell SPEC in
pensieve-intern (`specdec_bench/specs/cell.md`) is explicit about this —
five times in prose.

But this parent YAML's doc-comment kept advertising the OLD pattern.
When pensieve-intern's agent runner scans `tools/launcher/examples/` for
a reference invocation (good practice — agents should imitate working
examples), it lands on this Nemotron parent and copies the stale
pattern. **Five recent agent dispatches on the gemma-4 Epic OMNIML-5022
(cells OMNIML-5024 / 5025 / 5026 / 5027) authored new
`_cells/<sweep>.yaml` files for this reason, despite the SPEC telling
them not to.**

Prose loses to a concrete checked-in counter-example. The Qwen3.5-4B
reference template that cell.md officially points at
(`tools/launcher/examples/Qwen/Qwen3.5-4B/specdec_bench_mtp_vllm.yaml`)
is clean and shows only the bare `uv run slurm.py --yaml ...` form. This
PR makes Nemotron-120B consistent with that template.

## How surfaced

Diagnosed 2026-06-15 on OMNIML-5025 cell_t0_d7 ([intern-agent job
341631795](https://gitlab-master.nvidia.com/omniml/integration/nmm-sandbox/-/jobs/341631795)).
The cell agent authored `_cells/gemma-4-E4B-it_mtp_vllm_t0_d7.yaml` —
the engine's diff-shape classifier should have rejected it, but didn't
(tracked separately as OMNIML-5170). Root-causing the agent's behavior
surfaced this stale doc-comment as the source of the pattern.

## Verification

`grep -rn '_cells\|runtime_params common'` across the entire launcher
tree returned only this file. After this PR, the launcher tree carries
zero stale references.

Pairs with NVIDIA/Model-Optimizer#1738 (gemma-4-E4B-it container fix)
and OMNIML-5170 (engine-side classifier hard-reject for `_cells/` paths
— defense in depth).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated example documentation to clarify how to override per-cell
parameters using CLI flags in Slurm configuration runs.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Chenhan Yu <chenhany@nvidia.com>
This commit is contained in:
Chenhan D. Yu
2026-06-15 12:58:14 -07:00
committed by GitHub
parent 0f61c984a2
commit c661366352
1 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16/specdec_bench_dflash_vllm.yaml
+8 -3
View File
@@ -10,10 +10,15 @@
# H100/A100. Surfaced on OMNIML-4969: tp_size=2 OOMed at model load
# (~80 GB needed per GPU at tp=2, only 79.11 GiB available).
#
# Slurm run on cw_dfw — cells override per-cell runtime_params,
# --save_dir, --block_size, --num_requests via pipeline.task_N.args+=[...]:
# Slurm run on cw_dfw — cells override per-cell knobs via
# pipeline.task_N.args+=[...] CLI flags (post-PR-#1564 convention; no
# per-cell runtime_params file, no _cells/<sweep>.yaml):
#
# uv run slurm.py # --yaml modules/Model-Optimizer/tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16/specdec_bench_dflash_vllm.yaml # --yes detach=true # pipeline.task_0.args+=["--runtime_params common/specdec_bench/_cells/<sweep_name>.yaml","--save_dir /scratchspace/<sweep>/qualitative","--block_size 4"] # pipeline.task_1.args+=["--runtime_params common/specdec_bench/_cells/<sweep_name>.yaml","--save_dir /scratchspace/<sweep>/throughput_32k","--num_requests 80","--block_size 4"]
# uv run slurm.py \
# --yaml modules/Model-Optimizer/tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16/specdec_bench_dflash_vllm.yaml \
# --yes detach=true \
# pipeline.task_0.args+=["--save_dir /scratchspace/<sweep>/qualitative","--block_size 4"] \
# pipeline.task_1.args+=["--save_dir /scratchspace/<sweep>/throughput_32k","--num_requests 80","--block_size 4"]
#
# Reference run: cicd_1781024226 (cw_dfw) produced
# qualitative Average_AL = 2.7316 (11-cat breakdown)