|
|
|
|
|
|
| A Spike Removal Algorithm Based on Median Filter and Statistic for
Raman Spectra |
| YE Rui-qian1, HE Hao1, ZHENG Peng1, XU Meng-xi2, WANG Lei1* |
1. School of Aerospace Engineering, Xiamen University, Xiamen 361101, China
2. State Key Laboratory of Physical Chemistry of Solid Surfaces, Collaborative Innovation Center of Chemistry for Energy Materials (iChEM), College of Chemistry and Chemical Engineering, Xiamen University, Xiamen 361101, China
|
|
|
|
|
Abstract Raman spectroscopy is a promising technique widely used in chemistry, biology, and physics. However, as the key part of the Raman spectrometer, the charge-coupled device is vulnerable to cosmic rays, resulting in a random narrow bandwidth and a high-intensity spikes. It will cause a significant reduction in signal contrast. In this paper, we propose a practical spike removal algorithm. Firstly, the algorithm obtains deviation data by separating the median filtered data from the original data. Then, deviation data is sorted from small to large by quantile method, and the intermediate 99% data are selected for Gaussian distribution fitting. Considering the characteristics of high-intensity and sparsity of the spike, the occurrence probability of high intensity data in the spectra is used as the threshold standard to remove spike. Finally, the spikes are replaced by new data using median filtered at corresponding positions. This algorithm restores the original sample information without any debugging parameters. Different intensities of spikes are added in Raman spectra to verify the algorithm, and the experimental results show that this algorithm’s sensitivity can reach 99.5%. Besides, this algorithm is applicable for one-dimensional Raman spectra, two-dimensional Raman images and three-dimensional Raman data cubes, and the performance improves with the increase of dimensionality. Specifically, the one-dimensional spike removal algorithm can detect spikes exceeding 40% of the maximum peak intensity. The Raman data cubes can be detected exceeding 20% of the peak value. The algorithm is used to process 40 000 real Raman spectra and can effectively remove spikes without distorting the real signal, proving the algorithm’s practicability.
|
|
Received: 2021-08-19
Accepted: 2022-02-25
|
|
|
|
Corresponding Authors:
WANG Lei
E-mail: wanglei33@xmu.edu.cn
|
|
[1] YAN Jing-wen, LI Ying, CHEN Liang, et al（闫静文, 李 颖, 陈 靓, 等）. Journal of Atmospheric and Environmental Optics(大气与环境光学学报). 2017, 12(2): 128.
[2] REN Bin(任 斌). Optics & Optoelectronic Technology(光学与光电技术)，2021, 19(4): 1.
[3] LIANG Ying-fang, ZHOU Hua-lan, WANG Yan, et al(梁营芳, 周化岚, 王 燕, 等). Physical Testing and Chemical Analysis Part B：Chemical Analysis(理化检验：化学分册)，2020, 56(4): 487.
[4] TIAN Hui-yan, LIU Yu, HUANG Jiao-qi, et al(田晖艳, 刘 羽, 黄姣祺, 等). Spectroscopy and Spectral Analysis(光谱学与光谱分析)，2020, 40(10): 3021.
[5] LIU Xia, HUO Ya-peng, KANG Wei-jun, et al(刘 厦, 霍亚鹏, 康维钧, 等). Chinese Science Bulletin(科学通报)，2020, 65(15): 1448.
[6] Schulze H G, Turner R F. Applied Spectroscopy，2013，67(4): 457.
[7] Tian Y, Burch K S. Applied Spectroscopy，2016，70(11): 1861.
[8] Zhao J. Applied Spectroscopy，2003，57(11): 1368.
[9] Huang Z, Zeng H, Hamzavi I, et al. Optics Letters，2001，26(22): 1782.
[10] Lee J S, Cox D D. Computational Statistics & Data Analysis，2010，54(12): 3131.
[11] Katsumoto Y, Ozaki Y. Applied Spectroscopy，2003，57(3): 317.
[12] Silveira Jr L, Bodanese B, Zangaro R A, et al. Instrumentation Science Technology，2010，38(4): 268.
[13] Veneri G, Federighi P, Rosini F, et al. Journal of Neuroscience Methods，2011，196(2): 318.
[14] Zhang D, Ben-Amotz D. Applied Spectroscopy，2002，56(1): 91.
[15] Takeuchi H, Hashimoto S, Harada I. Applied Spectroscopy，1993，47(1): 129.
[16] Uckert K, Bhartia R, Michel J. Applied Spectroscopy，2019，73(9): 1019.
[17] Sharan T S, Sharma S, Sharma N. Applied Spectroscopy，2021，88(1): 117.
[18] LI Sheng, DAI Lian-kui（李 晟, 戴连奎）. The Journal of Light Scattering（光散射学报），2011, 23(3): 188.
[19] Ryabchykov O, Bocklitz T, Ramoji A, et al. Chemomtrics and Intelligent Laboratory Systems，2016，155: 1.
|
| [1] |
LIAO Yi-min1, YAN Yin-zhou1, WANG Qiang2*, YANG Li-xue3, PAN Yong-man1, XING Cheng1, JIANG Yi-jian1, 2. Laser-Induced Growth Device and Optical Properties of ZnO
Microcrystals[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(10): 3000-3005. |
| [2] |
QI Dong-li, CHENG Jia, SUN Hui, ZHANG Rui-xin, SONG Jian-yu, QIN Yan-li, LI Hong-da, SHEN Long-hai*. Research on Spectral Characteristics and Photocatalytic Properties of Ball Milled TiO2[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(10): 3063-3067. |
| [3] |
ZHANG Qian, DONG Xiang-hui, YAO Wei-rong, YU Hang, XIE Yun-fei*. Surface-Enhanced Raman Spectroscopy for Rapid Detection of Flunixin Meglumine Residues in Pork[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(10): 3155-3160. |
| [4] |
FAN Yuan-chao, CHEN Xiao-jing*, HUANG Guang-zao, YUAN Lei-ming, SHI Wen, CHEN Xi. Evaluation of Aging State of Wire Insulation Materials Based on
Raman Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(10): 3161-3167. |
| [5] |
LIU Qiang1, LIU Shao-bo1, 2, LU Xue-song1, 2*, FAN Jun-jia1, 2, TIAN Hua1, 2, MA Xing-zhi1, 2, GUI Li-li1, 2. Research Progress in the Application of Raman Spectroscopy in Petroleum Geology[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(09): 2679-2688. |
| [6] |
NI Shuang, WEN Jia-xing, ZHOU Min-jie, HUANG Jing-lin, LE Wei, CHEN Guo, HE Zhi-bing, LI Bo, ZHAO Song-nan, ZHAO Zong-qing, DU Kai*. Theoretical Study on Raman Characteristic Peaks of Coronavirus Spike Protein Based on Deep Learning[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(09): 2757-2762. |
| [7] |
LIU Shi-jie1, ZHU Yao-di1, 2, LI Miao-yun1, 2*, ZHAO Gai-ming1, 2, ZHAO Li-jun1, 2, MA Yang-yang1, 2, WANG Na1. Raman Spectroscopic Characteristic Structure Analysis and Rapid Identification of Food-Borne Pathogen Spores Based on SERS Technology[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(09): 2774-2780. |
| [8] |
LI Jun-meng1, ZHAI Xue-dong1, YANG Zi-han1, ZHAO Yan-ru1, 2, 3, YU Ke-qiang1, 2, 3*. Microscopic Raman Spectroscopy for Diagnosing Roots in Apple
Rootstock Under Heavy Metal Copper Stress[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(09): 2890-2895. |
| [9] |
PENG Jiao-yu1, 2*, YANG Ke-li1, 2, BIAN Shao-ju1, 3, 4, CUI Rui-zhi1, 3, DONG Ya-ping1, 2, LI Wu1, 3. Quantitative Analysis of Monoborates (H3BO3 and B(OH)-4) in Aqueous Solution by Raman Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(08): 2456-2462. |
| [10] |
LI Huan-tong1, 2, CAO Dai-yong3, ZOU Xiao-yan3, ZHU Zhi-rong1, ZHANG Wei-guo1, XIA Yan4. Raman Spectroscopic Characterization and Surface Graphitization Degree of Coal-Based Graphite With the Number of Aromatic Layers[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(08): 2616-2623. |
| [11] |
CHEN Wei-na1, GUO Zhong-zheng1, LI Kai-kai1, YANG Yu-zhu1, YANG Xu2*. Micro Confocal Raman Spectroscopy Combined With Chemometrical Method for Forensic Differentiation of Electrostatic Copy Paper[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(07): 2033-2038. |
| [12] |
GE Deng-yun, XU Min-min, YUAN Ya-xian*, YAO Jian-lin*. Surface-Enhanced Raman Spectroscopic Investigation on the Effect of
Solution pH on Dehydroxylation of Hydroxythiophenol Isomers[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(07): 2076-2081. |
| [13] |
ZHU Xiang1, 2*, YUAN Chao-sheng1, CHENG Xue-rui1, LI Tao1, ZHOU Song1, ZHANG Xin1, DONG Xing-bang1, LIANG Yong-fu2, WANG Zheng2. Study on Performances of Transmitting Pressure and Measuring Pressure of [C4mim][BF4] by Using Spectroscopic Techniques[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(06): 1674-1678. |
| [14] |
HUANG Bin, DU Gong-zhi, HOU Hua-yi*, HUANG Wen-juan, CHEN Xiang-bai*. Raman Spectroscopy Study of Reduced Nicotinamide Adenine Dinucleotide[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(06): 1679-1683. |
| [15] |
WANG Ming-xuan, WANG Qiao-yun*, PIAN Fei-fei, SHAN Peng, LI Zhi-gang, MA Zhen-he. Quantitative Analysis of Diabetic Blood Raman Spectroscopy Based on XGBoost[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(06): 1721-1727. |
|
|
|
|
