### What does this PR do?
**Type of change**: documentation
**Overview**: This PR updates the documentation and does some folder
re-structuring and file re-naming related to
https://github.com/NVIDIA/Model-Optimizer/pull/951.
### Usage
Documentation
### Testing
Documentation
### Before your PR is "*Ready for review*"
- Is this change backward compatible?: ✅
- If you copied code from any other sources or added a new PIP
dependency, did you follow guidance in `CONTRIBUTING.md`: ✅
- Did you write any new necessary tests?: N/A <!--- Mandatory for new
features or examples. -->
- Did you update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?:
✅ (renamed `AutoQDQ` to `Autotune`)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Renamed AutoQDQ to Autotune across guides and changelog.
* Updated Autotune guide descriptions and wording.
* Added a new section on optimizing Q/DQ node placement with Autotune,
including CLI usage and API links (appears twice in one README).
* Applied minor grammar and capitalization corrections.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: gcunhase <4861122+gcunhase@users.noreply.github.com>
## What does this PR do?
This PR upload user guide of Automated QDQ placement tool. This tool
automatically search QDQ insertion points with better performance.
**Overview:** ?
## Usage
<!-- You can potentially add a usage example below. -->
```python
# Add a code snippet demonstrating how to use this
```
## Testing
<!-- Mention how have you tested your change if applicable. -->
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes/No <!--- If No, explain
why. -->
- **Did you write any new necessary tests?**: Yes/No
- **Did you add or update any necessary documentation?**: Yes/No
- **Did you update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes/No <!--- Only for new features, API changes, critical bug fixes or
bw breaking changes. -->
## Additional Information
<!-- E.g. related issue. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Added comprehensive guide for Automated Q/DQ Placement Optimization
workflow, including quick start instructions, advanced usage patterns,
configuration options, best practices, and troubleshooting.
* **New Features**
* Exposed public API for CLI parser programmatic access.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Will Guo <willg@nvidia.com>
Signed-off-by: Gwena Cunha <4861122+gcunhase@users.noreply.github.com>
Co-authored-by: Gwena Cunha <4861122+gcunhase@users.noreply.github.com>
Co-authored-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
### What does this PR do?
- Add Security related coding practices in `SECURITY.md` and merge with
`2_security.rst`
- Update `CONTRIBUTING.md` for instructions to follow if copying code
from other repositories
- Update PR template
- Cleanup dependency files
- New API `mto.load_modelopt_state` doing the insecure `torch.load(f,
weights_only=False)` instead of doing it separately everywhere. This
also allows us to later improve the input validation for
`modelopt_state_path` or use safer alternatives to `torch.load`
### Testing
<!-- Mention how have you tested your change if applicable. -->
N/A
### Before your PR is "*Ready for review*"
Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md)
and your commits are signed (`git commit -s -S`).
Make sure you read and follow the [Security Best
Practices](https://github.com/NVIDIA/Model-Optimizer/blob/main/SECURITY.md#security-coding-practices-for-contributors)
(e.g. hardcoded `trust_remote_code=True`, `torch.load(...,
weights_only=True)`, `pickle`, etc.).
- Is this change backward compatible?: ✅ <!--- If ❌, explain why. -->
- If you copied code from any other source, did you follow IP policy in
[CONTRIBUTING.md](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md#-copying-code-from-other-sources)?:
✅ <!--- Mandatory -->
- Did you write any new necessary tests?: NA <!--- Mandatory for new
features or examples. -->
- Did you add or update any necessary documentation and update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?:
NA <!--- Only for new features, API changes, critical bug fixes or
backward incompatible changes. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Expanded and reorganized security guidance and contributor procedures;
updated PR template and several READMEs with clearer security,
submission, and installation instructions
* Replaced an older security document with an enhanced, centralized
security guidance
* **Chores**
* Adjusted example dependency lists and optional extras (adds, removals,
and version constraints)
* Enabled automated incremental reviews, added pre-merge security
checks, and introduced a knowledge-base of coding/security guidelines
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
## What does this PR do?
Documentation
**Overview:**
- Update support matrix, changelog, deployment page, example readmes as
per recent feature and model support on Windows side.
## Testing
- No testing, its just documentation change
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes/No <!--- If No, explain
why. -->
- **Did you write any new necessary tests?**: Yes/No
- **Did you add or update any necessary documentation?**: Yes/No
- **Did you update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes/No <!--- Only for new features, API changes, critical bug fixes or
bw breaking changes. -->
## Additional Information
<!-- E.g. related issue. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added ONNX Mixed Precision Weight-only quantization (INT4/INT8)
support.
* Introduced diffusion-model quantization on Windows.
* Added new accuracy benchmarks (Perplexity and KL-Divergence).
* Expanded deployment with multiple ONNX Runtime Execution Providers
(CUDA, DirectML, TensorRT-RTX).
* **Bug Fixes**
* Fixed ONNX 1.19 compatibility issue with CuPy during INT4 AWQ
quantization.
* **Documentation**
* Updated installation guides with system requirements and multiple
backend options.
* Reorganized deployment documentation with comprehensive execution
provider guidance.
* Expanded example workflows with improved setup instructions and
support matrices.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: vipandya <vipandya@nvidia.com>
## What does this PR do?
**Type of change:** New feature
**Overview:**
AutoCast runs full type inference to get the new types after adding
casts. ONNX doesn't have a separate function for type inference, and it
is done as part of shape inference. Shape inference is a much more
complex task than type inference, especially when dynamic shapes are
involved. We're seeing some shape inference related bugs in AutoCast.
Typically we can WAR, but it's cumbersome. A local implementation might
allow users to WAR shape inference related issues. This is opt-in and
marked as experimental.
## Usage
python -m modelopt.onnx.autocast --onnx_path /path/to/input.onnx
[options] --use_standalone_type_inference
## Testing
Added use_standalone_type_inference=True to all existing
PrecisionConverter tests.
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes
- **Did you write any new necessary tests?**: Yes
- **Did you add or update any necessary documentation?**: Yes
- **Did you update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes <!--- Only for new features, API changes, critical bug fixes or bw
breaking changes. -->
## Additional Information
A more permanent fix would be to decouple type and shape inference in
ONNX, we should invest in that when we have the resources - see
https://github.com/onnx/onnx/issues/7100
. This is a quick fix, which is also why it is opt-in and not the
default mode.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added `--use_standalone_type_inference` flag to ONNX AutoCast,
enabling type-only inference as an alternative to standard shape
inference. Useful as a workaround when shape inference fails or to
reduce computational overhead.
* **Documentation**
* Added "Type Inference Control" section with usage examples and caveats
for the new standalone type inference option.
* **Tests**
* Extended test coverage to validate both standard and standalone type
inference paths.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Gal Hubara Agam <96368689+galagam@users.noreply.github.com>
- [x] Address feedback on Threat and Vuln Analysis (TAVA) doc by ProdSec
team
- [x] Add note on safe usage of pickle deserialization of
modelopt-generated state files
**TODO: [Separate PR]** Replace pickle usage in
`modelopt/torch/opt/plugins/megatron.py` - Needs fix on
TransformerEngine first as we copy from
https://github.com/NVIDIA/TransformerEngine/blob/3ff0b8d4/transformer_engine/pytorch/module/base.py#L863
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added `--trust_calibration_data` CLI flag for secure ONNX quantization
with pickle data files.
* **Improvements**
* Enhanced security validation for generated quantization code.
* Simplified data loading by removing pickle-based caching—data is now
always loaded fresh.
* Added security guidance throughout model state loading operations.
* **Documentation**
* Updated guides with security best practices for model state handling.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
## What does this PR do?
- Deprecate `num_query_groups` parameter pruning support in
`mcore_minitron` mode. This parameter pruning was not part of the paper
but added hacky support in the past which makes refactoring current
implementation a bit challenging hence removed its support.
- Besides, `num_query_groups` should be multiple of TP so its less
likely to be pruned.
- Users can still prune it if they want to by using ModelOpt 0.40.0 or
earlier
- Going forward, `num_query_groups` if passed in pruning `export_config`
should be same as original else raises an Exception.
- As a follow-up PR, I will de-couple DynamicModule with forward hooks /
hparam importance logic so its easy for users to change to different
scoring logic
## Testing
<!-- Mention how have you tested your change if applicable. -->
- [x] CI/CD Tests pass
- [x] Compare MMLU for pruning Qwen3-8B (nmm-sanbox) with previous and
current implementation
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: No. See reasoning above <!---
If No, explain why. -->
- **Did you write any new necessary tests?**: No
- **Did you add or update any necessary documentation?**: Yes
- **Did you update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes <!--- Only for new features, API changes, critical bug fixes or bw
breaking changes. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Deprecations**
* num_query_groups parameter in Minitron pruning is deprecated. Use
ModelOpt 0.40.0 or earlier if pruning with this parameter is needed.
* **Documentation**
* Updated NAS search parameters documentation
* Refined pruning scope documentation for Transformer, MoE, and Mamba
models
* Updated example configurations and support matrix reflecting current
supported pruning parameters
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
## What does this PR do?
**Type of change:** Bug fix
**Overview:** Autocast crashes if the input batch size in the ONNX model
is different to the calibration data input batch size. For example:
calibration data has shape `[10, 6, 3, 480, 800]` and ONNX model has
shape `[1, 6, 3, 480, 800]`. The quantization workflow interprets this
as 10 calibration samples, so ideally, Autocast would also interpret
them similarly. This PR just allows Autocast to exit gracefully with a
custom message.
## Usage
```python
$ python -m modelopt.onnx.autocast --onnx_path=$MODEL_NAME.onnx --calibration_data=calib_data_10.npz
```
## Testing
See bug 5676209.
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes
- **Did you write any new necessary tests?**: No
- **Did you add or update any necessary documentation?**: Yes
- **Did you update
[Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**:
No
## Additional Information
Original error:
```sh
polygraphy.exception.exception.PolygraphyException: Input tensor: image | Received incompatible shape: (10, 6, 3, 480, 800).
Note: Expected a shape compatible with: BoundedShape([1, 6, 3, 480, 800], min=None, max=None)
```
Autocast error:
```sh
ValueError: Input shape from 'image' does not match provided input shape: [1, 6, 3, 480, 800] vs [10, 6, 3, 480, 800]. Please make sure that your calibration data matches the ONNX input shapes.
```
---------
Signed-off-by: gcunhase <4861122+gcunhase@users.noreply.github.com>
- [x] Product Rename: TensorRT Model Optimizer to Model Optimizer
(OMNIML-3033)
- [x] Mention in Latest News section with date on the date of merging
this PR (12/08)
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
## What does this PR do?
**Type of change:** New feature <!-- Use one of the following: Bug fix,
new feature, new example, new tests, documentation. -->
- Support pruning `num_moe_experts`, `moe_ffn_hidden_size`, and
`moe_shared_expert_intermediate_size` in `mcore_minitron` pruning
## Testing
<!-- Mention how have you tested your change if applicable. -->
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes <!--- If No, explain why.
-->
- **Did you write any new necessary tests?**: Yes
- **Did you add or update any necessary documentation?**: Yes
- **Did you update
[Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes <!--- Only for new features, API changes, critical bug fixes or bw
breaking changes. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added Mixture of Experts (MoE) pruning support with new configurable
dimensions for expert count and intermediate sizes
* Extended NAS architecture search capabilities to include MoE model
parameters
* **Documentation**
* Updated support matrix and pruning documentation for MoE-compatible
models
* Clarified available pruning dimensions and parameters for MoE
architectures
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Co-authored-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
## What does this PR do?
new feature
**Overview:**
AutoCast: Allow configuring target opset from CLI and main API.
## Usage
<!-- You can potentially add a usage example below. -->
python -m modelopt.onnx.autocast --onnx_path model.onnx --opset 22
```python
# Add a code snippet demonstrating how to use this
```
## Testing
<!-- Mention how have you tested your change if applicable. -->
pytest tests/unit/onnx/autocast/test_autocast.py -k test_opset
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes
- **Did you write any new necessary tests?**: Yes
- **Did you add or update any necessary documentation?**: Yes
- **Did you update
[Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**:
No - minor API addition <!--- Only for new features, API changes,
critical bug fixes or bw breaking changes. -->
## Additional Information
<!-- E.g. related issue. -->
Signed-off-by: Gal Hubara Agam <96368689+galagam@users.noreply.github.com>
Gwena Cunha