Burst Photography for Learning to Enhance Extremely Dark Images

This is a Tensorflow implementation of "Burst Photography for Learning to Enhance Extremely Dark Images", Ahmet Serdar Karadeniz, Erkut Erdem, Aykut Erdem, IEEE Trans. Image Processing, November 2021.

Project Website

Paper

Dataset

Getting Started

Dependencies

Prepare the environment (optional).

conda create -n dark-burst-photography python=3.6
conda activate dark-burst-photography

Clone this repository and install the required libraries.

git clone https://github.com/hucvl/dark-burst-photography
cd src
pip install -r requirements.txt

git clone https://github.com/alexlee-gk/lpips-tensorflow
cd lpips-tensorflow
python setup.py install

Testing

1. Download the pretrained models and put them to the folders checkpoint/Sony/burst_l1_res_se_motion_cx and checkpoint/Fuji/burst_fuji.
2. Download the SID dataset or just use the sample images in this repository.
3. Run python test.py

Training

1. For the perceptual and contextual losses, download the pre-trained VGG-19 model:

   python download_vgg_models.py
   

2. For multiscale training, set the following variables inside train.py:

   Coarse network

   train_coarse = True
   finetune = False

   Fine network

   train_coarse = False
   finetune = False

   Set-based burst network

   train_coarse = False
   finetune = True
   n_burst = 8

3. Train the model

   python train.py
   

Training/testing Video Model

1. Download the pretrained model and put it to the folder src/seeing-motion/checkpoints/burst_l1_drv_full.
2. Download the DRV dataset.
3. Run python test_image_dbp.py (static videos) or python test_video_dbp.py (dynamic videos) for testing and python train_dbp.py for training.

License

MIT License.

Citation

If you use this code for your research, please consider citing our paper:

@ARTICLE{dark-burst-photography,
author={Ahmet Serdar Karadeniz and Erkut Erdem and Aykut Erdem},
journal={IEEE Transactions on Image Processing},
title={Burst Photography for Learning to Enhance Extremely Dark Images},
year={2021},
volume={30},
pages={9372-9385}
}