Video Deraining Mechanism by Preserving the Temporal Consistency and Intrinsic Properties of Rain Streaks
Article Sidebar
Main Article Content
Abstract
The problem of video deraining is one of the most focused research areas where several techniques are introduced to achieve higher visual quality in the results. The process of video deraining tries to eliminate the presence of rain streaks in the videos so that the overall quality of the video can be enhanced. The existing frameworks tried to accurately eliminate the rain streaks in videos, but there is still room for improvement to preserve the temporal consistencies and intrinsic properties of rain streaks. This work does the job with the combination of handcrafted and deep priors that deeply extract and identify the nature of rain streaks in different dimensions. The proposed work included three main steps: Prior extraction, Derain modelling and optimization. Four major priors are extracted from the frames where the gradient prior (GP) and sparse prior (SP) are extracted from the rain streaks, and the smooth temporal prior (STP) and deep prior (DP) are extracted from the clean video. The unidirectional total variation (UTV) is applied to extract the GP, and the L1 normalization method is followed to extract the SP and STP. The DP is then extracted from the clean frames using the residual gated recurrent deraining network (Res-GRRN) model based on deep learning. Derain modelling is carried out based on the extracted priors, and the stochastic alternating direction multiplier method (SADMM) algorithm is utilized to solve the optimization problem. The proposed approach is then implemented in python and evaluated using the real-world dataset. The overall PSNR achieved by the proposed approach is 39.193dB, which is more optimal than the existing methods.
Article Details
References
H. Wang, Y. Wu, M. Li, Q. Zhao, D. Meng, “A survey on rain removal from video and single image,” arXiv preprint arXiv: 1909.08326, 2019.
P. Hettiarachchi, R. Nawaratne, D. Alahakoon, D. D. Silva, N. Chilamkurti, “Rain streak removal for single images using conditional generative adversarial networks,” Applied Sciences vol. 11, no. 5, pp. 2214, 2021.
S. Wang, L. Zhu, H. Fu, J. Qin, C.B. Schönlieb, W. Feng, S, Wang, “Rethinking Video Rain Streak Removal: A New Synthesis Model and a Deraining Network with Video Rain Prior,” In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIX, Cham: Springer Nature Switzerland pp. 565-582, 2022.
L. Zhu, Z. Deng, X. Hu, H. Xie, X. Xu, J. Qin, P.A. Heng, “Learning gated non-local residual for single-image rain streak removal,” IEEE Transactions on Circuits and Systems for Video Technology vol. 31, no. 6, pp. 2147-2159, 2020.
K. H. Lee, E. Ryu, J. K. Kim, “Progressive rain removal via a recurrent convolutional network for real rain videos,” IEEE Access vol. 8, pp. 203134-203145, 2020.
M. R. Islam, M. Paul, “Video Deraining Using the Visual Properties of Rain Streaks,” IEEE Access vol. 10, pp. 202-212, 2021.
P. E. Darney, I. J. Jacob, “Rain Streaks Removal in digital images by Dictionary based sparsity process with MCA Estimation,” Journal of Innovative Image Processing vol. 3, no. 3, pp. 174-189, 2021,
V. Frants, S. Agaian, K. Panetta, “QSAM-Net: Rain streak removal by quaternion neural network with self-attention module,” arXiv preprint arXiv: 2208.04346 2022.
Y. Ding, M. Li, T. Yan, F. Zhang, Y. Liu, R.W. H. Lau, “Rain streak removal from light field images,” IEEE Transactions on Circuits and Systems for Video Technology vol. 32, no. 2, pp. 467-482, 2021.
Y. Wang, T.Z. Huang, “A tensor-based low-rank model for single-image rain streaks removal,” IEEE Access vol. 7, pp. 83437-83448, 2019.
R. Quan, X. Yu, Y. Liang, Y. Yang, “Removing raindrops and rain streaks in one go,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9147-9156, 2021.
M. R. Islam, M, Paul, “Rain streak removal in a video to improve visibility by TAWL algorithm,” arXiv preprint arXiv: 2007.05167 2020.
T. Wang, X. Yang, K. Xu, S. Chen, Q. Zhang, R. W. H. Lau, “Spatial attentive single-image deraining with a high quality real rain dataset,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12270-12279, 2019.
M. Li, X. Cao, Q. Zhao, L. Zhang, D. Meng, “Online rain/snow removal from surveillance videos,” IEEE Transactions on Image Processing vol. 30, pp. 2029-2044, 2021.
E. Khatab, A. Onsy, M. Varley, A. Abouelfarag, “A Lightweight Network for Real-Time Rain Streaks and Rain Accumulation Removal from Single Images Captured by AVs,” Applied Sciences vol. 13, no. 1, pp. 219, 2023.
H. Wang, Q. Xie, Q. Zhao, D. Meng, “A model-driven deep neural network for single image rain removal,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 3103-3112, 2020.
Z. Sun, S. Xiong, R. W. Liu, “Directional regularized tensor modeling for video rain streaks removal,” arXiv preprint arXiv: 1902.07090, 2019.
W. Yan, R. T. Tan, W. Yang, D. Dai, “Self-aligned video deraining with transmission-depth consistency,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11966-11976, 2021.
Ahmed Abdelaziz, Machine Learning Approaches for Predicting Stock Market Volatility , Machine Learning Applications Conference Proceedings, Vol 3 2023.
L. Yi, Q. Zhao, W. Wei, Z. Xu, “Robust online rain removal for surveillance videos with dynamic rains.” Knowledge-Based Systems vol. 222, pp. 107006, 2021.
ARECHE, F. O. ., & Palsetty , K. . (2021). Ubiquitous Counter Propagation Network in Analysis of Diabetic Data Using Extended Self-Organizing Map. Research Journal of Computer Systems and Engineering, 2(2), 51:57. Retrieved from https://technicaljournals.org/RJCSE/index.php/journal/article/view/33
H. Wang, Y. Wu, M. Li, Q. Zhao, D. Meng, “Survey on rain removal from videos or a single image,” Science China Information Sciences vol. 65, no. 1, pp. 111101, 2022.
Y.T. Wang, X.L. Zhao, T.X. Jiang, L.J. Deng, T.H. Ma, Y.T. Zhang, T.Z. Huang, “A total variation and group sparsity based tensor optimization model for video rain streak removal,” Signal Processing: Image Communication vol. 73, pp. 96-108, 2019.
J.H. Zhuang, Y.S. Luo, X.L. Zhao, T.X. Jiang, “Reconciling handcrafted and self-supervised deep priors for video directional rain streaks removal,” IEEE Signal Processing Letters vol. 28, pp. 2147-2151, 2021.
P. K. Sharma, S. Ghosh, A. Sur, “High-quality frame recurrent video de-raining with multi-contextual adversarial network,” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) vol. 17, no. 2, pp. 1-24, 2021.
M. R. Islam, M. Paul, “Video rain-streaks removal by combining data-driven and feature-based models,” Sensors vol. 21, no. 20, pp. 6856, 2021.
A. Kulkarni, P. W. Patil, S. Murala, “Progressive subtractive recurrent lightweight network for video deraining,” IEEE Signal Processing Letters vol. 29, pp. 229-233, 2021.