mirror of
https://github.com/NVIDIA/Model-Optimizer.git
synced 2026-11-03 08:14:29 +08:00
gcunhaseandClaude Opus 4.7
0e4227812c
Adds ``modelopt.onnx.quantization.sensitivity``, a per-op-type or per-node
accuracy sensitivity ranking primitive for ONNX PTQ, plus a coverage / threshold
based exclusion picker that turns the ranking into an actionable
``--nodes_to_exclude`` or ``--op_types_to_exclude`` list.
Primitive (sensitivity.score):
- For each quantizable target (op type or individual node), invokes the existing
``modelopt.onnx.quantization.quantize`` entry point to insert calibrated Q/DQ
on just that target, runs the reference and quantized ONNXs through
ONNXRuntime on the same calibration inputs, and computes a proxy metric
between the two graph-output activation sets. Higher score means the target
adds more accuracy loss if quantized -- so callers keep high-scoring targets
at higher precision.
- Op-type granularity via ``--op_types_to_quantize`` (fast; ~10-15 probes on a
typical graph); per-node granularity via ``--nodes_to_quantize`` regex (deep
dive; N_nodes probes).
- Three proxy metrics via ``metrics.py``: kl_div (default, softmax-normalized),
mse (raw), cos_dist (1 - cosine_similarity).
- Real calibration data via .npy / .npz / directory path, or synthetic random
fallback (directional-only; warned in the CLI and marked in the output JSON's
calibration_source field).
- Default op_types_scope excludes layout / copy ops via
``modelopt.onnx.op_types.is_copy_op`` -- Transpose / Reshape / Concat and
friends show up in ORT's default quantizable set, but their sensitivity
signal reflects Q/DQ insertion at data-movement boundaries rather than any
INT8-kernel trade-off, so ranking them clutters the output with
"don't do this anyway" entries.
Picker (sensitivity.suggest_exclusion, sensitivity.summarize_exclusion):
- Coverage mode (default): return the largest target set whose cumulative
sensitivity score stays at or below ``coverage * total_mass``. The actual
coverage is always less than or equal to the requested value ("at most X%"),
so the operator never gets more exclusion than they asked for.
Architecture-portable because the target is a fraction, not an absolute
number.
- Threshold mode: return every target whose individual sensitivity score
exceeds ``threshold``. Simpler and more predictable when the operator
already knows what per-target sensitivity magnitude they consider
"too sensitive to quantize" for a specific model.
- ``near_tie_ratio`` (default 0.99) emits a logger.warning when the cut-off
between included and excluded targets is a near-tie -- flags potential
intra-group precision fragmentation. Set to ``None`` to disable.
- Companion ``summarize_exclusion`` reports the effect of an exclusion set:
coverage_pct, num_excluded, num_previously_quantized, num_remaining_quantized,
excluded_mass, total_mass.
Public surface:
- Python: ``from modelopt.onnx.quantization.sensitivity import score,
suggest_exclusion, summarize_exclusion``.
- CLI: ``python -m modelopt.onnx.quantization.sensitivity --onnx_path=...
--calibration_data_path=... --granularity=op_type --metric=kl_div``.
- Output JSON schema includes scores, calibration_source,
num_calibration_samples, metric, granularity, target_precision.
Tests:
- ``tests/gpu/onnx/quantization/test_sensitivity.py``: synthetic-graph tier
(real deterministic inputs -> LayerNormalization scores above Conv) plus a
synthetic-random regression tier (calibration_data=None still preserves the
directional invariant), plus CoAtNet-0 op-type and per-node integration
stubs marked @pytest.mark.manual (gated by --run-manual and a
MODELOPT_SENSITIVITY_FIXTURES env var).
- ``tests/unit/onnx/quantization/test_sensitivity_picker.py``: coverage /
threshold / min_score_floor / max_nodes / near-tie warning tests.
- ``tests/unit/onnx/quantization/test_nodes_to_quantize.py``: validates the
existing ``--nodes_to_quantize`` include-only flag that per-node granularity
relies on.
Documentation:
- ``docs/source/guides/_onnx_quantization.rst``: new "Quantization Sensitivity
Scan" chapter plus a "Turning scores into an exclusion list" subsection with
both policy modes, ``:ref:`` cross-reference to the metric options, and a
note that the picker documentation assumes per-node granularity for
simplicity but the same logic applies to per-op-type.
Signed-off-by: gcunhase <4861122+gcunhase@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>