Official Pytorch Implementation of "Blur2Blur: Blur Conversion for Unsupervised Image Deblurring on Unknown Domains" (CVPR'24)

Bang-Dang Pham   ·   Phong Tran   ·   Anh Tran   ·   Cuong Pham   ·   Rang Nguyen   ·   Minh Hoai  

VinAI Research, Vietnam

TL;DR: Blur2Blur converts images from unknown blur into a known blur. This version retains the original content while applying a different blur kernel that has been effectively trained and captured by supervision deblurring models.

Abstract: This paper presents an innovative framework designed to train an image deblurring algorithm tailored to a specific camera device. This algorithm works by transforming a blurry input image, which is challenging to deblur, into another blurry image that is more amenable to deblurring. The transformation process, from one blurry state to another, leverages unpaired data consisting of sharp and blurry images captured by the target camera device. Learning this blur-to-blur transformation is inherently simpler than direct blur-to-sharp conversion, as it primarily involves modifying blur patterns rather than the intricate task of reconstructing fine image details. The efficacy of the proposed approach has been demonstrated through comprehensive experiments on various benchmarks, where it significantly outperforms state-of-the-art methods both quantitatively and qualitatively.

Details of the model architecture and experimental results can be found in our paper:

@inproceedings{pham2024blur2blur,
 author={Pham, Bang-Dang and Tran, Phong and Tran, Anh and Pham, Cuong and Nguyen, Rang and Hoai, Minh},
 booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
 title={Blur2Blur: Blur Conversion for Unsupervised Image Deblurring on Unknown Domains},
 year={2024}
}

Please CITE our paper whenever this repository is used to help produce published results or incorporated into other software.

Table of contents

1. Getting Started
2. Datasets
3. Blur2Blur
4. Results
5. Acknowledgments
6. Contacts

Getting Started ✨

Prerequisites

• Python >= 3.7
• Pytorch >= 1.9.0
• CUDA >= 10.0

Installation

Install dependencies:

git clone https://github.com/VinAIResearch/Blur2Blur
cd Blur2Blur

conda create -n blur2blur python=3.9  
conda activate blur2blur  
pip install -r requirements.txt  

Datasets 💾

Data Preperation

You can download our proposed RB2V dataset by following this script:

chmod +x ./dataset/download_RB2V.sh
bash ./dataset/download_RB2V.sh

Download datasets REDS, GoPro and RSBlur then unzip to folder ./dataset and organize following this format:

dataset
├── Name of Unknown-Known dataset e.g. RB2V-GoPro
    ├── trainA
    ├──── (Train) Blurry set of Unknown Blur
    ├──── ...
    ├── trainB
    ├──── (Train) Sharp set of Unknown Blur
    ├──── ...
    ├── trainC
    ├──── (Train) Blurry set of Known Blur
    ├──── ...
    ├── trainD
    ├──── (Train) Sharp set of Known Blur
    ├──── ...
    ├── testA
    ├──── (Test) Blurry set of Unknown Blur
    ├──── ...

where:

• trainA, trainB: Blur/Sharp images from the Unknown Blur dataset for blur kernel conversion.
• trainC, trainD: Blur/Sharp images from the Known Blur dataset for the target blur domain.
• testA: Blurry images from the test-set of the Unknown Blur dataset.

Blur2Blur 🚀

Training

To train the model:

python train.py --dataroot path/to/dataset \
                --name exp_name \
                --model blur2blur --netG mimounet \
                --batch_size 1 \
                --dataset_mode unaligned \
                --norm instance --pool_size 0 \
                --display_id -1

or

bash ./scripts/train.sh

Using Blur2Blur

To evaluate the model:

python test.py --dataroot datasets/GoPro/b2b_exp/RB2V_GOPRO_filter \
                --name exp_name \
                --eval \
                --model blur2blur --netG mimounet \
                --checkpoints_dir ckpts/ \
                --dataset_mode unaligned \
                --norm instance \

or

bash ./scripts/test.sh

Results 🏆

For more interactive results, you can take a look at my project page: https://zero1778.github.io/blur2blur/

Acknowledgments

We would like to extend our gratitude to the following implementations for their contributions to the development of Blur2Blur:

• pix2pix for providing the foundational code model.
• uvcgan2 for helping to enhance the baseline code.
• MIMO-UNet and Blur Kernel Extractor for creating the backbone and submodules integral to our project.

Contacts 📬

If you have any questions or suggestions about this repo, please feel free to contact me ([email protected]).