|
| 1 | +# Learning Enriched Features for Fast Image Restoration and Enhancement (TPAMI 2022) |
| 2 | + |
| 3 | +[Syed Waqas Zamir](https://scholar.google.es/citations?user=WNGPkVQAAAAJ&hl=en), [Aditya Arora](https://adityac8.github.io/), [Salman Khan](https://salman-h-khan.github.io/), [Munawar Hayat](https://scholar.google.com/citations?user=Mx8MbWYAAAAJ&hl=en), [Fahad Shahbaz Khan](https://scholar.google.es/citations?user=zvaeYnUAAAAJ&hl=en), [Ming-Hsuan Yang](https://scholar.google.com/citations?user=p9-ohHsAAAAJ&hl=en), and [Ling Shao](https://scholar.google.com/citations?user=z84rLjoAAAAJ&hl=en) |
| 4 | + |
| 5 | +[](https://arxiv.org/abs/2111.09881) |
| 6 | + |
| 7 | + |
| 8 | + |
| 9 | +# Code will be released before April 20. |
| 10 | + |
| 11 | +<hr /> |
| 12 | + |
| 13 | +> **Abstract:** *Given a degraded input image, image restoration aims to recover the missing high-quality image content. Numerous |
| 14 | +applications demand effective image restoration, e.g., computational photography, surveillance, autonomous vehicles, and remote |
| 15 | +sensing. Significant advances in image restoration have been made in recent years, dominated by convolutional neural networks |
| 16 | +(CNNs). The widely-used CNN-based methods typically operate either on full-resolution or on progressively low-resolution |
| 17 | +representations. In the former case, spatial details are preserved but the contextual information cannot be precisely encoded. In the |
| 18 | +latter case, generated outputs are semantically reliable but spatially less accurate. This paper presents a new architecture with a |
| 19 | +holistic goal of maintaining spatially-precise high-resolution representations through the entire network, and receiving complementary |
| 20 | +contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing the |
| 21 | +following key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange |
| 22 | +across the multi-resolution streams, (c) non-local attention mechanism for capturing contextual information, and (d) attention based |
| 23 | +multi-scale feature aggregation. Our approach learns an enriched set of features that combines contextual information from multiple |
| 24 | +scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on six real image benchmark |
| 25 | +datasets demonstrate that our method, named as MIRNet-v2 , achieves state-of-the-art results for a variety of image processing tasks, |
| 26 | +including defocus deblurring, image denoising, super-resolution, and image enhancement.* |
| 27 | +<hr /> |
| 28 | + |
| 29 | + |
| 30 | +## Citation |
| 31 | +If you use MIRNet_v2, please consider citing: |
| 32 | + |
| 33 | + @article{Zamir2022MIRNetv2, |
| 34 | + title={Restormer: Efficient Transformer for High-Resolution Image Restoration}, |
| 35 | + author={Syed Waqas Zamir and Aditya Arora and Salman Khan and Munawar Hayat |
| 36 | + and Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao}, |
| 37 | + journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)}, |
| 38 | + year={2022} |
| 39 | + } |
| 40 | + |
| 41 | + |
| 42 | +## Contact |
| 43 | +Should you have any question, please contact waqas.zamir@inceptioniai.org |
| 44 | + |
| 45 | + |
| 46 | +**Acknowledgment:** This code is based on the [BasicSR](https://github.com/xinntao/BasicSR) toolbox. |
| 47 | + |
| 48 | +## Our Related Works |
| 49 | +- Restormer: Efficient Transformer for High-Resolution Image Restoration, CVPR 2022. [Paper](https://arxiv.org/abs/2111.09881) | [Code](https://github.com/swz30/Restormer) |
| 50 | +- Multi-Stage Progressive Image Restoration, CVPR 2021. [Paper](https://arxiv.org/abs/2102.02808) | [Code](https://github.com/swz30/MPRNet) |
| 51 | +- Learning Enriched Features for Real Image Restoration and Enhancement, ECCV 2020. [Paper](https://arxiv.org/abs/2003.06792) | [Code](https://github.com/swz30/MIRNet) |
| 52 | +- CycleISP: Real Image Restoration via Improved Data Synthesis, CVPR 2020. [Paper](https://arxiv.org/abs/2003.07761) | [Code](https://github.com/swz30/CycleISP) |
0 commit comments