|
|
|
|
|
|
Spectral Pre-Processing Based on Convolutional Neural Network |
JIAO Qing-liang1, LIU Ming1*, YU Kun2, LIU Zi-long2, 3, KONG Ling-qin1, HUI Mei1, DONG Li-quan1, ZHAO Yue-jin1 |
1. Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
2. Henan Key Laboratory of Infrared Materials and Spectrum Measures and Applications, School of Physics, Henan Normal University, Xinxiang 453007, China
3. Institute of Optics & Laser Metrology,National Institute of Metrology, Beijing 100013,China |
|
|
Abstract The techniques of Spectral Pre-Processingis important in qualitative spectral analysis. They were aiming at the problems of poor preservation of weak peaks in denoising algorithms, over-deduction of real spectral in baseline correction algorithms, inaccurate location of spectral peaks method, and error accumulation caused by serial use of various preprocessing algorithms. In this paper, we design a convolutional neural network (CNN). This CNN consists of two modules: baseline correction and denoising module and peak detection module, which are connected and output independently. In the ideal conditions, according to the spectrumlinear function, the ideal spectrum can be obtained by fitting the spectral peak location. Therefore, connecting the output of the spectral peak detection module to the baseline correction and denoising module can improve the accuracy of baseline correction and denoising effectively; and high-quality spectrum is the basis ofspectral peak detection. Hence the interconnection of two modules can improve the quality of the reconstructed spectrum effectively.The spectral baseline correctionand denoising module is a feed forward network consisting of several convolution layers, activation functions and batch normalization layers.The characteristic peak detection module is a multi-scale feature fusion network, that uses different-sized convolution kernels to divide the feature spectrum into different scales. The module fuses feature of different sizes to estimate the location of the characteristic spectral peak. During the training of the proposed CNN, the spectrum obtained by spectrophotometers with different temperatures, humidity and preheating times were used as input samples. The spectrum obtained from standard instruments of the National Institute of Metrology was used as output samples.In the experiment, noise with different SNR and Gaussian baselines with different peak values were added to the synthesized spectrum applied to evaluate the superiority of proposed CNN in denoising, baseline correction and spectral characteristic peak detection. Then the near-infrared spectrum of the corn, which was added noise and baseline, were taken as samples. Moreover, they, which were preprocessed using preprocessing algorithms, were applied to estimate the concentration of water and oilin corn by partial least squares. The estimated concentration compares with the real concentration, which is measured with a standard instrument, to prove the advantage of the proposed CNN. Experiments show that theproposed CNN can achieve good results in both single-task and multi-task preprocessing. Moreover, the spectrum preprocessed by the proposed CNN can get the most accurate results in quantitative analysis, which demonstrates that the proposed CNN has strong practical value.
|
Received: 2020-12-29
Accepted: 2021-04-11
|
|
Corresponding Authors:
LIU Ming
E-mail: bit411liu@bit.edu.cn
|
|
[1] Bai Yanru, Liu Quan. Biomedical Optics Express, 2020, 11(1): 200.
[2] Delwiche S R, Reeves J B. Applied Spectroscopy, 2010, 64(1): 73.
[3] Chen Hao, Xu Weilian, Broderick Neil, et al. Journal of Raman Spectroscopy, 2018, 49(9): 1529.
[4] QIU Xuan-bing, SUN Dong-yuan, LI Chuan-liang, et al(邱选兵, 孙冬远, 李传亮, 等). Spectroscopy and Spectral Analysis(光谱学与光谱分析), 2019, 39(2): 628.
[5] Bertinetto Carlo G,Vuorinen Tapani. Applied Spectroscopy, 2014, 68(2): 155.
[6] Zhang Zhimin, Chen Shan, Liang Yizeng. Analyst, 2010, 135(5): 1138.
[7] Baek Sung-June, Park Aaron, Ahn Young-Jin, et al. Analyst, 2015, 140(1): 250.
[8] Zhang Zhimin, Tong Xia, Ying Peng, et al. Analyst, 2015, 140(23): 7955.
[9] YU Kun, JIAO Qing-liang, LIU Zi-long, et al(于 坤, 焦青亮, 刘子龙, 等). Acta Optica Sinica(光学学报), 2019, 39(9): 0930008.
[10] Joel Wahl, Mikael Sjdahl, Kerstin Ramser. Applied Spectroscopy, 2020, 74(4): 427.
[11] Roberto C Carvajal, Luis E Arias, Hugo O. Applied Spectroscopy, 2016, 70(4): 604.
[12] Zhang Xiaolei, Xu Jinfan, Yang Jie, et al. Analytica Chimica Acta, 2020, 1119: 41. |
[1] |
LI Xin-ting, ZHANG Feng, FENG Jie*. Convolutional Neural Network Combined With Improved Spectral
Processing Method for Potato Disease Detection[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 215-224. |
[2] |
LAN Yan1,WANG Wu1,XU Wen2,CHAI Qin-qin1*,LI Yu-rong1,ZHANG Xun2. Discrimination of Planting and Tissue-Cultured Anoectochilus Roxburghii Based on SMOTE and Inception-CNN[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 158-163. |
[3] |
LIU Shu-hong1, 2, WANG Lu-si3*, WANG Li-sheng3, KANG Zhi-juan1, 2,WANG Lei1, 2,XU Lin1, 2,LIU Ai-qin1, 2. A Spectroscopic Study of Secondary Minerals on the Epidermis of Hetian Jade Pebbles From Xinjiang, China[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 169-175. |
[4] |
WANG Qi-biao1, HE Yu-kai1, LUO Yu-shi1, WANG Shu-jun1, XIE Bo2, DENG Chao2*, LIU Yong3, TUO Xian-guo3. Study on Analysis Method of Distiller's Grains Acidity Based on
Convolutional Neural Network and Near Infrared Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3726-3731. |
[5] |
HUANG Meng-qiang1, KUANG Wen-jian2, 3*, LIU Xiang1, HE Liang4. Quantitative Analysis of Cotton/Polyester/Wool Blended Fiber Content by Near-Infrared Spectroscopy Based on 1D-CNN[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(11): 3565-3570. |
[6] |
CAI Jian-rong1, 2, HUANG Chu-jun1, MA Li-xin1, ZHAI Li-xiang1, GUO Zhi-ming1, 3*. Hand-Held Visible/Near Infrared Nondestructive Detection System for Soluble Solid Content in Mandarin by 1D-CNN Model[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(09): 2792-2798. |
[7] |
PU Shan-shan, ZHENG En-rang*, CHEN Bei. Research on A Classification Algorithm of Near-Infrared Spectroscopy Based on 1D-CNN[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(08): 2446-2451. |
[8] |
TANG Ting, PAN Xin*, LUO Xiao-ling, GAO Xiao-jing. Fusion of ConvLSTM and Multi-Attention Mechanism Network for
Hyperspectral Image Classification[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(08): 2608-2616. |
[9] |
LI Wen-xia1, DU Yu-jun2, WANG Yue1, LIU Zheng-dong3*, ZHENG Jia-hui1, DU Wen-qian1, WANG Hua-ping4. Research on On-Line Efficient Near-Infrared Spectral Recognition and Automatic Sorting Technology of Waste Textiles Based on Convolutional Neural Network[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(07): 2139-2145. |
[10] |
LIANG Wan-jie1, FENG Hui2, JIANG Dong3, ZHANG Wen-yu1, 4, CAO Jing1, CAO Hong-xin1*. Early Recognition of Sclerotinia Stem Rot on Oilseed Rape by Hyperspectral Imaging Combined With Deep Learning[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(07): 2220-2225. |
[11] |
JIANG Xia*, QIU Bo, WANG Lin-qian, GUO Xiao-yu. Automatic Classification Method of Star Spectra Based on
Semi-Supervised Mode[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(06): 1875-1880. |
[12] |
ZHENG Zhi-jie1, LIN Zhen-heng1, 2*, XIE Hai-he2, NIE Yong-zhong3. The Method of Terahertz Spectral Classification and Identification for Engineering Plastics Based on Convolutional Neural Network[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(05): 1387-1393. |
[13] |
CHEN Dong-ying1, 2, ZHANG Hao1, 2*, ZHANG Zi-long1, YU Mu-xin1, CHEN Lu3. Research on the Origin Traceability of Honeysuckle Based on Improved 1D-VD-CNN and Near-Infrared Spectral Data[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(05): 1471-1477. |
[14] |
LI Kai-yu1, ZHANG Hui2, MA Jun-cheng3, ZHANG Ling-xian1*. Segmentation Method for Crop Leaf Spot Based on Semantic Segmentation and Visible Spectral Images[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(04): 1248-1253. |
[15] |
CHEN Qing1, TANG Bin1, 2*, LONG Zou-rong1, 2, MIAO Jun-feng1, HUANG Zi-heng1, DAI Ruo-chen1, SHI Sheng-hui1, ZHAO Ming-fu1, ZHONG Nian-bing1. Water Quality Classification Using Convolution Neural Network Based on UV-Vis Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(03): 731-736. |
|
|
|
|