mirror of
https://github.com/NVIDIA/Model-Optimizer.git
synced 2026-10-21 06:10:05 +08:00
Ajinkya Rasane
51c3614db0
## What does this PR do? **Type of change:** New Feature **Overview:** - Enable ONNX export for auto quantized models - Update documentation and changelog ## Usage <!-- You can potentially add a usage example below. --> ``` python torch_quant_to_onnx.py --quantize_mode=auto \ --onnx_save_path=./vit_base_patch16_224.nvfp4_fp8.onnx \ --calibration_data_size 64 \ --auto_quantization_formats NVFP4_AWQ_LITE_CFG FP8_DEFAULT_CFG \ --batch_size 128 ``` ## Testing <!-- Mention how have you tested your change if applicable. --> ``` python evaluate.py --onnx_path=vit_base_patch16_224.nvfp4_fp8.onnx \ --model_name=vit_base_patch16_224 \ --results_path=./results.txt \ --batch_size 128 ``` Accuracy results ``` The top1 accuracy of the model is 84.15% The top5 accuracy of the model is 97.396% ``` Reference accuracy for fp16 ``` The top1 accuracy of the model is 85.102% The top5 accuracy of the model is 97.526% ``` ## Before your PR is "*Ready for review*" <!-- If you haven't finished some of the above items you can still open `Draft` PR. --> - **Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)** and your commits are signed. - **Is this change backward compatible?**: Yes - **Did you write any new necessary tests?**: No - **Did you add or update any necessary documentation?**: Yes - **Did you update [Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**: Yes <!--- Only for new features, API changes, critical bug fixes or bw breaking changes. --> --------- Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>
55 lines
1.5 KiB
ReStructuredText
55 lines
1.5 KiB
ReStructuredText
Welcome to Model Optimizer (ModelOpt) documentation!
|
|
####################################################
|
|
|
|
.. toctree::
|
|
:glob:
|
|
:maxdepth: 1
|
|
:caption: Getting Started
|
|
|
|
getting_started/[0-9]*
|
|
Quick Start: PTQ - PyTorch <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/llm_ptq>
|
|
Quick Start: PTQ - ONNX <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/onnx_ptq>
|
|
Quick Start: PTQ - PyTorch to ONNX <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/torch_onnx>
|
|
Quick Start: PTQ - Windows <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/windows>
|
|
Quick Start: QAT <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/llm_qat>
|
|
Quick Start: Pruning <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/pruning>
|
|
Quick Start: Distillation <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/llm_distill>
|
|
Quick Start: Speculative Decoding <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/speculative_decoding>
|
|
Quick Start: Sparsity <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/llm_sparsity>
|
|
|
|
.. toctree::
|
|
:glob:
|
|
:maxdepth: 1
|
|
:caption: Guides
|
|
|
|
guides/[0-9]*
|
|
|
|
.. toctree::
|
|
:glob:
|
|
:maxdepth: 1
|
|
:caption: Deployment
|
|
|
|
deployment/[0-9]*
|
|
|
|
.. toctree::
|
|
:glob:
|
|
:maxdepth: 1
|
|
:caption: Examples
|
|
|
|
examples/[0-9]*
|
|
|
|
|
|
.. toctree::
|
|
:glob:
|
|
:maxdepth: 1
|
|
:caption: Reference
|
|
|
|
reference/[0-9]*
|
|
|
|
.. toctree::
|
|
:glob:
|
|
:maxdepth: 1
|
|
:caption: Support
|
|
|
|
support/[0-9]*
|