mirror of
https://github.com/NVIDIA/Model-Optimizer.git
synced 2026-11-04 08:27:53 +08:00
kinjalpatel27
263b2b72ff
## What does this PR do? **Type of change:** : Bug fix **Overview:** moved vLLM fakequant checkpoint export code to separate files: 1. for HF export -> modelopt.torch.export.plugins.vllm_fq_hf 2. for megatron export -> modelopt.torch.export.plugins.vllm_fq_megatron ## Usage Refer to [README.md](https://github.com/NVIDIA/TensorRT-Model-Optimizer/tree/096ee13ea62bbb0ce0a4e4128c439651374d6235/examples/vllm_serve#load-qatptq-model-and-serve-in-vllm-wip) ## Testing - Tested HF approach by exporting bf16 model using QAT script and running vllm server, verified amax values match - Tested MCore approach by quantizing and exporting bf16 model using quantize.sh and export.sh script and running vllm server, verified amax values match - Tested using unit tests in `tests/gpu/torch/export/test_vllm_fq_hf_export.py` and `tests/gpu/torch/export/test_vllm_fq_megatron_export.py` ## Before your PR is "*Ready for review*" <!-- If you haven't finished some of the above items you can still open `Draft` PR. --> - **Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)** and your commits are signed. - **Is this change backward compatible?**: Yes - **Did you write any new necessary tests?**: NA - **Did you add or update any necessary documentation?**: Yes - **Did you update [Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**: NA ## Additional Information <!-- E.g. related issue. --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added dedicated export functions for vLLM fakequant checkpoint format, supporting both HuggingFace and Megatron Core models. * **Refactor** * Simplified export API by removing conditional export flags for cleaner, more predictable behavior. * Reorganized export functionality into focused plugin modules for improved maintainability. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Kinjal Patel <kinjalpravin@nvidia.com>