An Improved Hodrick-Prescott Decomposition Based Near-Infrared Adaptive Denoising Method
XIE De-hong1, LI Jun-feng2, LIU Di3, WAN Xiao-xia4, YE Yi1
1. School of Light Industry and Food, Nanjing Forestry University, Nanjing 210037, China
2. School of Packaging and Printing Engineering, Henan University of Animal Husbandry and Economy, Zhengzhou 450046, China
3. Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University, Beijing 100048, China
4. Hubei Province Engineering Technical Center for Digitization and Virtual Reproduction of Color Information of Culture Relics, Wuhan University, Wuhan 430079, China
Abstract：During the rapid detection of pesticide residue in fruits and vegetables by near-infrared (NIR) spectroscopy, NIR spectroscopy is often contaminated by noises. Meanwhile, the peaks in NIR spectroscopy of chemical components of pesticides and fruits and vegetables are weak and highly overlapped, so denoising the NIR spectroscopy has risks of smoothing weak peaks of pesticide components or generating peaks of non- chemical components. In the subsequent classification or chemical composition analysis, the above problems deteriorate the accuracy of classification of the NIR spectroscopy and influence analysis of chemical components of pesticide residue. In order to solve the conflict between noise suppression and peak maintenance of the NIR spectroscopy, an adaptive denoising method is proposed based on an improved Hodrick-Prescott decomposition model. In the model, L2 norm of the residual between the noisy near-infrared spectroscopy and its restored spectroscopy is used as the residual term to describe the Gaussian noise structure, and L2 norm of the second-order difference of the restored spectroscopy is used as the regularization term to penalize the restored spectroscopy. The penalty can force the restored spectroscopy to reduce its gradient, resulting in smoothing noises and keeping the original peaks. In order to acquire the regularization parameter in the optimization equation of the Hodrick-Prescott decomposition model adaptively, an L-curve method is combined into the method. So in the method, the optimal regularization parameters are obtained by solving the parameters corresponding to the maximum curvature point of the L-curve, which can balance the regularization term and the residual term in the Hodrick-Prescott decomposition model and finally obtain ideal restored spectroscopy. In order to compare wavelet decomposition methods with bior6.8 basis and sym8 basis and complete ensemble empirical mode decomposition method, signal-to-noise ratio (SNR) is computed, and a support vector machine (SVM) classification model is established by NIR spectroscopy of Shanghai Qing with pesticide and without pesticide. The experimental results show that the SNR value of the denoised NIR spectroscopy can reach to 33.35 dB when using the proposed method to deal with the noisy spectroscopy with 18.79 dB. Then the method is applied to denoising the NIR spectroscopy in the training set and the testing set, the recognition rate of vector machine classification model trained by the training set and testing set are 93.58% and 71.18% respectively. This recognition rate is significantly higher than the results using the above three denoising methods and is close to the results of the original uncontaminated spectroscopy. The results validate that the proposed method has better denoising effect than other methods mentioned, which can improve the stability of NIR classification model to pesticide residue detection.
谢德红，李俊锋，刘 菂，万晓霞，叶 艺. 基于改进Hodrick-Prescott分解模型的近红外自适应降噪方法[J]. 光谱学与光谱分析, 2020, 40(05): 1650-1655.
XIE De-hong, LI Jun-feng, LIU Di, WAN Xiao-xia, YE Yi. An Improved Hodrick-Prescott Decomposition Based Near-Infrared Adaptive Denoising Method. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2020, 40(05): 1650-1655.
[1] JIANG Meng-yun, GONG Wen-wen, LIU Qing-ju, et al(蒋梦云, 巩文雯, 刘庆菊，等). Guangdong Agricultural Sciences（广东农业科学）, 2018, 45(11): 60.
[2] ZHU Wen-bin，LEI Bing-shan, LEI Zhi-yong(朱文斌, 雷秉山, 雷志勇). Infrared Technology(红外技术), 2018, 40(11): 1047.
[3] Guo Q, Zhang C. Science Asia, 2012, 38: 207.
[4] Santhosh M, Venkaiah C, Vinod Kumar D M. Energy Conversion and Management, 2018, 168: 482.
[5] Elouaham S, Dliou A, Latif R, et al. International Journal of Computer Applications, 2016, 149(7): 39.
[6] Xia Y X, Ni Y Q. International Journal of Distributed Sensor Networks, 2018, 14(12): 1.
[7] Ouahilal M, Mohajir M E, Chahhou M, et al. Journal of Big Data, 2017, 4(1): 31.
[8] Chen M, Su H, Zhou Y, et al. Biomedical Optics Express, 2016, 7(12): 5021.
[9] Stuart B H. Infrared Spectroscopy: Fundamentals and Applications: John Wiley & Sons, Ltd, 2005.
[10] Ullah R, Khan S, Javaid S, et al. Biomedical Optics Express, 2018, 9(2): 844.
