RSCD (BS-RSCD & JCD)

[CVPR2021] Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes (CVF Link)

by Zhihang Zhong, Yinqiang Zheng, Imari Sato

We contributed the first real-world dataset (BS-RSCD) and end-to-end model (JCD) for joint rolling shutter correction and deblurring tasks.

We collected the data samples using the proposed beam-splitter acquisition system as below:

In the near future, we will add more data samples with larger distortion to the dataset ...

If you are interested in real-world datasets for pure deblurring tasks, please refer to ESTRNN & BSD.

Prerequisites

Install the dependent packages:

conda create -n rscd python=3.8
conda activate rscd
sh install.sh

Download lmdb files of BS-RSCD (or Fastec-RS for RSC tasks).

(PS, for how to create lmdb file, you can refer to ./data/create_rscd_lmdb.ipynb)

Training

Please specify the <path> (e.g. "./dataset/ ") where you put the dataset file or change the default value in " ./para/paramter.py".

Train JCD on BS-RSCD:

python main.py --data_root <path> --model JCD --dataset rscd_lmdb --video

Train JCD on Fastec-RS:

python main.py --data_root <path> --model JCD --dataset fastec_rs_lmdb --video

Testing

Please download checkpoints and unzip it under the main directory.

Run the pre-trained model on BS-RSCD:

python main.py --test_only --dataset rscd_lmdb --test_checkpoint ./checkpoints/JCD_BS-RSCD.tar --video

Inference for video file:

python video_inference.py --src <input_path> --dst <output_path> --checkpoint ./checkpoints/JCD_BS-RSCD.tar

Citing

If BS-RSCD and JCD are useful for your research, please consider citing:

@inproceedings{zhong2021towards,
  title={Towards rolling shutter correction and deblurring in dynamic scenes},
  author={Zhong, Zhihang and Zheng, Yinqiang and Sato, Imari},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={9219--9228},
  year={2021}
}