## What does this PR do?
**Type of change:** ?
Bug fix
**Overview:** ?
1. We are passing incorrect extra args to pipeline inference.
2. Need a default empty list for the list argument component-dtype
## Testing
Passed SDXL pipeline without component dtype
---------
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
## What does this PR do?
**Type of change:**
new feature
**Overview:**
Support Wan2.2 t2v diffusers quantization
1. fix torch2.9 support
2. add Wan2.2 t2v diffusers pipeline quantization
Main difference of the Wan2.2 pipeline comparing to exisiting
pipelines is that there are 2 backbone models for denoising. For the
quantization therefore we need to quantize both of them.
However, it turns out our base library does not well support
quantization of multiple models in the same time. Therefore, the
change here just stick to quantize a single model each time, and then
run the quantization multiple times.
So, we need to allow users to pick which backbone to quantize,
therefore adding a new argment for it
3. add a workaround for the exporting ONNX issue when we upgrade
diffusers to >= 0.35.0. The issue lies is the exporting of the
torch.nn.RMSNorm. Some pipelines in the diffusers > 0.35.0 use the
torch version RMSNorm while before that they use the diffusers' own
version of RMSNorm. It turns out they are directly replacable so the
workaround is to simply replace the torch RMSNorm usages with
diffusers RMSNorm. But we need to fix it properly soon by porting our
ONNX export to be based on torch dynamo instead of torchscript. Issue
reported from external user:
https://github.com/NVIDIA/TensorRT-Model-Optimizer/issues/262
4. allow use of a prompts file, which is simply a text file with a list
of prompts, one prompt each line
5. allow each component of a pipeline to have different dtype accuracy.
added a new list stype command line arg --component-dtype for this.
example: --component-dtype vae:Float
6. print the summary of the quantized model so users can capture issues
from
log
## Usage
python quantize.py \
--model wan2.2-t2v-14b \
--format fp8 \
--batch-size 4 \
--calib-size 64 \
--n-steps 20 \
--backbone transformer \
--model-dtype BFloat16 \
--component-dtype vae:Float \
--trt-high-precision-dtype BFloat16 \
--quantized-torch-ckpt-save-path ./wan_transformer.pt \
--onnx-dir wan-transformer-onnx \
--prompts-file wan-prompts.txt
## Testing
Tested SDXL_BASE, LTX_VIDEO_DEV, WAN22_T2V
## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->
- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes/No <!--- If No, explain
why. -->
- **Did you write any new necessary tests?**: Yes/No
- **Did you add or update any necessary documentation?**: Yes/No
- **Did you update
[Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes/No <!--- Only for new features, API changes, critical bug fixes or
bw breaking changes. -->
## Additional Information
https://github.com/NVIDIA/TensorRT-Model-Optimizer/issues/262
---------
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
Shengliang Xu