PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution

LiBo Zhu, Jianze Li, Haotong Qin, Wenbo Li, Yulun Zhang, Yong Guo and Xiaokang Yang, "PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution", arXiv, 2024

[arXiv] [supplementary material] [visual results]

🔥🔥🔥 News

• 2024-11-25: This repo is released.

Abstract: Diffusion-based image super-resolution (SR) models have shown superior performance at the cost of multiple denoising steps. However, even though the denoising step has been reduced to one, they require high computational costs and storage requirements, making it difficult for deployment on hardware devices. To address these issues, we propose a novel post-training quantization approach with adaptive scale in one-step diffusion (OSD) image SR, PassionSR. First, we simplify OSD model to two core components, UNet and Variational Autoencoder (VAE) by removing the CLIPEncoder. Secondly, we propose Learnable Boundary Quantizer (LBQ) and Learnable Equivalent Transformation (LET) to optimize the quantization process and manipulate activation distributions for better quantization. Finally, we design a Distributed Quantization Calibration (DQC) strategy that stabilizes the training of quantized parameters for rapid convergence. Comprehensive experiments demonstrate that PassionSR with 8-bit and 6-bit obtains comparable visual results with full-precision model. Moreover, our PassionSR achieves significant advantages over recent leading low-bit quantization methods for image SR.

---

HR	LR	OSEDiff(32-bit)	EfficientDM(8-bit)	PassionSR(8-bit)

⚒️ TODO

• Release code and pretrained models

🔗 Contents

1. Datasets
2. Calibration
3. Results
4. Citation

🔎 Results

PassionSR significantly out-performs previous methods at the setting of W8A8 and W6A6.

Evaluation on Synthetic Datasets

quantitative comparisons in Table 2 of the main paper (click to expand)

visual comparison in Figure 6 of the main paper (click to expand)

📎 Citation

If you find the code helpful in your research or work, please cite the following paper(s).

@article{zhu2024passionsr,
  title={PassionSR: Post-Training Quantization  with Adaptive Scale in One-Step Diffusion based Image Super-Resolution},
  author={Libo Zhu, Jianze Li, Haotong Qin, Wenbo Li, Yulun Zhang, Yong Guo and Xiaokang Yang},
  journal={arXiv preprint arXiv:2411.17106},
  year={2024}
}