Spectral Pre-Processing Based on Convolutional Neural Network
JIAO Qing-liang1, LIU Ming1*, YU Kun2, LIU Zi-long2, 3, KONG Ling-qin1, HUI Mei1, DONG Li-quan1, ZHAO Yue-jin1
1. Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
2. Henan Key Laboratory of Infrared Materials and Spectrum Measures and Applications, School of Physics, Henan Normal University, Xinxiang 453007, China
3. Institute of Optics & Laser Metrology，National Institute of Metrology, Beijing 100013，China
Abstract：The techniques of Spectral Pre-Processingis important in qualitative spectral analysis. They were aiming at the problems of poor preservation of weak peaks in denoising algorithms, over-deduction of real spectral in baseline correction algorithms, inaccurate location of spectral peaks method, and error accumulation caused by serial use of various preprocessing algorithms. In this paper, we design a convolutional neural network (CNN). This CNN consists of two modules: baseline correction and denoising module and peak detection module, which are connected and output independently. In the ideal conditions, according to the spectrumlinear function, the ideal spectrum can be obtained by fitting the spectral peak location. Therefore, connecting the output of the spectral peak detection module to the baseline correction and denoising module can improve the accuracy of baseline correction and denoising effectively; and high-quality spectrum is the basis ofspectral peak detection. Hence the interconnection of two modules can improve the quality of the reconstructed spectrum effectively.The spectral baseline correctionand denoising module is a feed forward network consisting of several convolution layers, activation functions and batch normalization layers.The characteristic peak detection module is a multi-scale feature fusion network, that uses different-sized convolution kernels to divide the feature spectrum into different scales. The module fuses feature of different sizes to estimate the location of the characteristic spectral peak. During the training of the proposed CNN, the spectrum obtained by spectrophotometers with different temperatures, humidity and preheating times were used as input samples. The spectrum obtained from standard instruments of the National Institute of Metrology was used as output samples.In the experiment, noise with different SNR and Gaussian baselines with different peak values were added to the synthesized spectrum applied to evaluate the superiority of proposed CNN in denoising, baseline correction and spectral characteristic peak detection. Then the near-infrared spectrum of the corn, which was added noise and baseline, were taken as samples. Moreover, they, which were preprocessed using preprocessing algorithms, were applied to estimate the concentration of water and oilin corn by partial least squares. The estimated concentration compares with the real concentration, which is measured with a standard instrument, to prove the advantage of the proposed CNN. Experiments show that theproposed CNN can achieve good results in both single-task and multi-task preprocessing. Moreover, the spectrum preprocessed by the proposed CNN can get the most accurate results in quantitative analysis, which demonstrates that the proposed CNN has strong practical value.
