Attentive Recurrent Network

Attentive Recurrent Generative Adversarial Network for Shadow detection and removal

---

📝Abstract

Shadows presenting in most images are an obstacle to various deep learning based image processing tasks. Shadows in image are generated by the complex interaction of various light sources and various objects, and performance of various Computer Vision tasks can be improved by removing them. In this paper, we implement a network that detects shadows from image, removes shadows via attention mechanism, and handle complex shadows gradually by performing tasks in recurrent manner. Experiment was conducted on efficiently delivering data from previous step to next step.

🇰🇷

대부분의 영상에 존재하는 그림자는 다양한 딥러닝 기반 영상처리 작업을 수행함에 방해가 되는 요소이다. 영상 내 그림자는 다양한 광원과 다양한 물체들의 상호작용에 의해 복잡하게 생성되며 이를 제거하는 것을 통해 다양한 Computer Vision task의 성능을 향상시킬 수 있다. 이 논문에서는 영상 내 그림자를 감지하여 Attention mechanism을 통해 그림자를 제거하고 Recurrent 하게 작업을 수행하며 복잡한 그림자를 단계적으로 제거하는 네트워크를 구현하였으며, Recurrent 한 네트워크에서 이전 단계의 데이터를 다음 단계에 효율적으로 전달하는 방식에 대한 실험을 수행하였다.

---

🗺️ Network Structure

Shadows in images are generated by various interactions between light sources and objects. It is hard for network to handle shadows in complex scenes at once. So we adopted Attention Mechanism to efficiently handle shadow and Recurrent Network to gradually handle complex shadow.

Overall Structure

Shadow Detector

Implemented both shadow detector (LSTM(left) and Concat.(right)), and compared performance of two network

Removal Encoder

---

🔎 Experiments

• compared to LSTM detector Network, Concat. detector Network shows slightly imporved performance at both BER and RMSE metric
• also can see that Concat. detector Network is removing shadow Gradually

---

📓 Conclusion

• LSTM detector Network cannot gradually work due to high dependency to previous step

  

• Concat. detector Network can reduce dependency to previous step, thus can gradually handle shadows in image.

  

---

📆 Project Schedule

2021.01 ~ 2021.08

📝 Contributor

김민우 / MINWOO KIM

🛠️ Tools & Dataset

Dataset : ISTD Dataset