|
|
|
|
|
|
| A Spike Removal Algorithm Based on Median Filter and Statistic for
Raman Spectra |
| YE Rui-qian1, HE Hao1, ZHENG Peng1, XU Meng-xi2, WANG Lei1* |
1. School of Aerospace Engineering, Xiamen University, Xiamen 361101, China
2. State Key Laboratory of Physical Chemistry of Solid Surfaces, Collaborative Innovation Center of Chemistry for Energy Materials (iChEM), College of Chemistry and Chemical Engineering, Xiamen University, Xiamen 361101, China
|
|
|
|
|
Abstract Raman spectroscopy is a promising technique widely used in chemistry, biology, and physics. However, as the key part of the Raman spectrometer, the charge-coupled device is vulnerable to cosmic rays, resulting in a random narrow bandwidth and a high-intensity spikes. It will cause a significant reduction in signal contrast. In this paper, we propose a practical spike removal algorithm. Firstly, the algorithm obtains deviation data by separating the median filtered data from the original data. Then, deviation data is sorted from small to large by quantile method, and the intermediate 99% data are selected for Gaussian distribution fitting. Considering the characteristics of high-intensity and sparsity of the spike, the occurrence probability of high intensity data in the spectra is used as the threshold standard to remove spike. Finally, the spikes are replaced by new data using median filtered at corresponding positions. This algorithm restores the original sample information without any debugging parameters. Different intensities of spikes are added in Raman spectra to verify the algorithm, and the experimental results show that this algorithm’s sensitivity can reach 99.5%. Besides, this algorithm is applicable for one-dimensional Raman spectra, two-dimensional Raman images and three-dimensional Raman data cubes, and the performance improves with the increase of dimensionality. Specifically, the one-dimensional spike removal algorithm can detect spikes exceeding 40% of the maximum peak intensity. The Raman data cubes can be detected exceeding 20% of the peak value. The algorithm is used to process 40 000 real Raman spectra and can effectively remove spikes without distorting the real signal, proving the algorithm’s practicability.
|
|
Received: 2021-08-19
Accepted: 2022-02-25
|
|
|
|
Corresponding Authors:
WANG Lei
E-mail: wanglei33@xmu.edu.cn
|
|
[1] YAN Jing-wen, LI Ying, CHEN Liang, et al（闫静文, 李 颖, 陈 靓, 等）. Journal of Atmospheric and Environmental Optics(大气与环境光学学报). 2017, 12(2): 128.
[2] REN Bin(任 斌). Optics & Optoelectronic Technology(光学与光电技术)，2021, 19(4): 1.
[3] LIANG Ying-fang, ZHOU Hua-lan, WANG Yan, et al(梁营芳, 周化岚, 王 燕, 等). Physical Testing and Chemical Analysis Part B：Chemical Analysis(理化检验：化学分册)，2020, 56(4): 487.
[4] TIAN Hui-yan, LIU Yu, HUANG Jiao-qi, et al(田晖艳, 刘 羽, 黄姣祺, 等). Spectroscopy and Spectral Analysis(光谱学与光谱分析)，2020, 40(10): 3021.
[5] LIU Xia, HUO Ya-peng, KANG Wei-jun, et al(刘 厦, 霍亚鹏, 康维钧, 等). Chinese Science Bulletin(科学通报)，2020, 65(15): 1448.
[6] Schulze H G, Turner R F. Applied Spectroscopy，2013，67(4): 457.
[7] Tian Y, Burch K S. Applied Spectroscopy，2016，70(11): 1861.
[8] Zhao J. Applied Spectroscopy，2003，57(11): 1368.
[9] Huang Z, Zeng H, Hamzavi I, et al. Optics Letters，2001，26(22): 1782.
[10] Lee J S, Cox D D. Computational Statistics & Data Analysis，2010，54(12): 3131.
[11] Katsumoto Y, Ozaki Y. Applied Spectroscopy，2003，57(3): 317.
[12] Silveira Jr L, Bodanese B, Zangaro R A, et al. Instrumentation Science Technology，2010，38(4): 268.
[13] Veneri G, Federighi P, Rosini F, et al. Journal of Neuroscience Methods，2011，196(2): 318.
[14] Zhang D, Ben-Amotz D. Applied Spectroscopy，2002，56(1): 91.
[15] Takeuchi H, Hashimoto S, Harada I. Applied Spectroscopy，1993，47(1): 129.
[16] Uckert K, Bhartia R, Michel J. Applied Spectroscopy，2019，73(9): 1019.
[17] Sharan T S, Sharma S, Sharma N. Applied Spectroscopy，2021，88(1): 117.
[18] LI Sheng, DAI Lian-kui（李 晟, 戴连奎）. The Journal of Light Scattering（光散射学报），2011, 23(3): 188.
[19] Ryabchykov O, Bocklitz T, Ramoji A, et al. Chemomtrics and Intelligent Laboratory Systems，2016，155: 1.
|
| [1] |
LI Jie, ZHOU Qu*, JIA Lu-fen, CUI Xiao-sen. Comparative Study on Detection Methods of Furfural in Transformer Oil Based on IR and Raman Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 125-133. |
| [2] |
WANG Fang-yuan1, 2, HAN Sen1, 2, YE Song1, 2, YIN Shan1, 2, LI Shu1, 2, WANG Xin-qiang1, 2*. A DFT Method to Study the Structure and Raman Spectra of Lignin
Monomer and Dimer[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 76-81. |
| [3] |
XING Hai-bo1, ZHENG Bo-wen1, LI Xin-yue1, HUANG Bo-tao2, XIANG Xiao2, HU Xiao-jun1*. Colorimetric and SERS Dual-Channel Sensing Detection of Pyrene in
Water[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 95-102. |
| [4] |
WANG Xin-qiang1, 3, CHU Pei-zhu1, 3, XIONG Wei2, 4, YE Song1, 3, GAN Yong-ying1, 3, ZHANG Wen-tao1, 3, LI Shu1, 3, WANG Fang-yuan1, 3*. Study on Monomer Simulation of Cellulose Raman Spectrum[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 164-168. |
| [5] |
WANG Lan-hua1, 2, CHEN Yi-lin1*, FU Xue-hai1, JIAN Kuo3, YANG Tian-yu1, 2, ZHANG Bo1, 4, HONG Yong1, WANG Wen-feng1. Comparative Study on Maceral Composition and Raman Spectroscopy of Jet From Fushun City, Liaoning Province and Jimsar County, Xinjiang Province[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 292-300. |
| [6] |
LI Wei1, TAN Feng2*, ZHANG Wei1, GAO Lu-si3, LI Jin-shan4. Application of Improved Random Frog Algorithm in Fast Identification of Soybean Varieties[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3763-3769. |
| [7] |
WANG Zhi-qiang1, CHENG Yan-xin1, ZHANG Rui-ting1, MA Lin1, GAO Peng1, LIN Ke1, 2*. Rapid Detection and Analysis of Chinese Liquor Quality by Raman
Spectroscopy Combined With Fluorescence Background[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3770-3774. |
| [8] |
LIU Hao-dong1, 2, JIANG Xi-quan1, 2, NIU Hao1, 2, LIU Yu-bo1, LI Hui2, LIU Yuan2, Wei Zhang2, LI Lu-yan1, CHEN Ting1，ZHAO Yan-jie1*，NI Jia-sheng2*. Quantitative Analysis of Ethanol Based on Laser Raman Spectroscopy Normalization Method[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3820-3825. |
| [9] |
LU Wen-jing, FANG Ya-ping, LIN Tai-feng, WANG Hui-qin, ZHENG Da-wei, ZHANG Ping*. Rapid Identification of the Raman Phenotypes of Breast Cancer Cell
Derived Exosomes and the Relationship With Maternal Cells[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3840-3846. |
| [10] |
LI Qi-chen1, 2, LI Min-zan1, 2*, YANG Wei2, 3, SUN Hong2, 3, ZHANG Yao1, 3. Quantitative Analysis of Water-Soluble Phosphorous Based on Raman
Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3871-3876. |
| [11] |
GUO He-yuanxi1, LI Li-jun1*, FENG Jun1, 2*, LIN Xin1, LI Rui1. A SERS-Aptsensor for Detection of Chloramphenicol Based on DNA Hybridization Indicator and Silver Nanorod Array Chip[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(11): 3445-3451. |
| [12] |
ZHU Hua-dong1, 2, 3, ZHANG Si-qi1, 2, 3, TANG Chun-jie1, 2, 3. Research and Application of On-Line Analysis of CO2 and H2S in Natural Gas Feed Gas by Laser Raman Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(11): 3551-3558. |
| [13] |
LIU Jia-ru1, SHEN Gui-yun2, HE Jian-bin2, GUO Hong1*. Research on Materials and Technology of Pingyuan Princess Tomb of Liao Dynasty[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(11): 3469-3474. |
| [14] |
LI Wen-wen1, 2, LONG Chang-jiang1, 2, 4*, LI Shan-jun1, 2, 3, 4, CHEN Hong1, 2, 4. Detection of Mixed Pesticide Residues of Prochloraz and Imazalil in
Citrus Epidermis by Surface Enhanced Raman Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(10): 3052-3058. |
| [15] |
ZHAO Ling-yi1, 2, YANG Xi3, WEI Yi4, YANG Rui-qin1, 2*, ZHAO Qian4, ZHANG Hong-wen4, CAI Wei-ping4. SERS Detection and Efficient Identification of Heroin and Its Metabolites Based on Au/SiO2 Composite Nanosphere Array[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(10): 3150-3157. |
|
|
|
|
