Rapid Detection of Total Organic Carbon in Oil Shale Based on Near
Infrared Spectroscopy
LI Quan-lun1, CHEN Zheng-guang1*, SUN Xian-da2
1. College of Information and Electrical Engineering, Heilongjiang Bayi Agricultural University, Daqing 163319, China
2. Key Laboratory of Continental Shale Hydrocarbon Accumulation and Efficient Development, Ministry of Education, Northeast Petroleum University, Daqing 163318, China
Abstract：To quickly detect the total organic carbon (TOC) content of oil shale, the TOC content and near-infrared spectrum data of 230 rock samples were measured in a certain block of the Songliao basin. The Monte Carlo method eliminates 14 abnormal samples, and the remaining 216 samples are pretreated by the method of detrended and baseline correction. The feature wavelength is selected by successive projections algorithm (SPA), uninformative variable elimination(UVE) algorithm and competitive adaptive reweighted sampling(CARS) method. The SPXY method divides the sample set into calibration set(144 samples) and validation set(72 samples) according to the ratio of 2∶1. Then linear partial least squares (PLS) model, nonlinear support vector machine (SVM) model and random forest (RF) model are adopted to predict the TOC content of oil shale. The determination coefficient (R2) and root mean square error (RMSE) was used as the evaluation indexes of the model to explore the influence of different characteristic wavelength selection methods on TOC modeling of oil shale and to compare the accuracy of different modeling methods on TOC content prediction of oil shale. The results show that the feature wavelength extraction can optimize the model. SPA, UVE and CARS extract 16, 253 and 65 wavelength points respectively. After the feature wavelength extraction, the model determination coefficient is improved, and the root means square error is decreased. This shows that the feature wavelength extraction plays an important role in simplifying the model and improving model efficiency. In addition, The performance of the nonlinear RF and SVM model is better than that of the linear PLS model. The reason is that the carbon in oil shale exists in all kinds of hydrocarbons, and the absorption peaks of different hydrocarbon groups interact with each other, which makes the complex nonlinear relationship between the TOC content of oil shale and the near-infrared spectroscopy data. Therefore, the nonlinear SVM and RF model can show better performance. Compared with other models, the coefficient of determination (R2v) and root mean square error (RMSEV) of the CARS-SVM model invalidation set show better results, reaching 0.906 6 and 0.222 0 respectively. This model can be used to rapidly detect TOC content in oil shale. The results of this study show that the application of near-infrared spectroscopy in the rapid detection of TOC content in oil shale is feasible, and the CARS-SVM model can show good prediction performance, which provides a new method and idea for the rapid detection of TOC content in oil shale in China.
Key words：Near-infrared; Total organic carbon in oil shale; Characteristic wavelength; Support vector machine; Random forest
李泉伦，陈争光，孙先达. 近红外光谱的油页岩总有机碳快速检测[J]. 光谱学与光谱分析, 2022, 42(06): 1691-1697.
LI Quan-lun, CHEN Zheng-guang, SUN Xian-da. Rapid Detection of Total Organic Carbon in Oil Shale Based on Near
Infrared Spectroscopy. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(06): 1691-1697.
[1] ZHANG Xiao-ming，PAN Yi，YANG Shuang-chun，et al（章小明，潘 一，杨双春，等）. Contemporary Chemical Industry（当代化工），2012，41(4)：377.
[2] LI Yao-xiang，WANG Hong-tao，GENG Zhi-wei，et al（李耀翔，汪洪涛，耿志伟，等）. Journal of West China Forestry Science（西部林业科学），2014，43(3)：1.
[3] Bensaidance M R, Turgeon A F, Lauzier F，et al. BMJ Open，2020，10(11)：e043300.
[4] CHEN Hua-zhou，CHEN Fu，SHI Kai，et al（陈华舟，陈 福，石 凯，等）. Transactions of the Chinese Society for Agricultural Machinery（农业机械学报），2015，46(5)：233.
[5] ZHAO Zhen-ying，LIN Jun，ZHANG Fu-dong，et al（赵振英，林 君，张福东，等）. Spectroscopy and Spectral Analysis（光谱学与光谱分析），2014，34(11)：2948.
[6] SHEN Yan，ZHANG Xiao-ping，LIANG Ai-zhen，et al（申 艳，张晓平，梁爱珍，等）. Chinese Journal of Applied Ecology（应用生态学报），2010，21(1)：109.
[7] WANG Sai-ya，WANG Shi-bo，GE Shi-rong，et al（王赛亚，王世博，葛世荣，等）. Journal of China Coal Society（煤炭学报），2020，45(8)：3024.
[8] Romeo M，Adams M，Hind A，et al. Journal of Near Infrared Spectroscopy，2002，10(3)：223.
[9] WANG Zhi-hong，ZHANG Fu-dong，TENG Fei，et al（王智宏，张福东，滕 飞，等）. Optics and Precision Engineering（光学精密工程)，2015, 23(2): 371.
[10] YIN Bao-quan，SHI Yin-xue，SUN Rui-zhi，et al（尹宝全，史银雪，孙瑞志，等）. Transactions of the Chinese Society for Agricultural Machinery（农业机械学报），2015，46(S1)：122.
[11] Yun Y H，Li H D，Deng B C，et al. Trends in Analytical Chemistry，2019，113：102.
[12] Li H，Liang Y，Xu Q，et al. Analytica Chimica Acta，2009，648(1)：77.
[13] Suykens J A K，Vandewalle J. Neural Processing Letters，1999，9(3)：293.
[14] Breiman L. Machine Learning，2001，45(1)：5.
[15] ZHOU Yu-qing，QIN Meng-zhi，MA Zhi-hong（周宇晴，秦梦芝，马志宏）. Journal of Tianjin Agricultural University（天津农学院学报），2016，23(2)：49.
[16] QIU Lan-lan，LI Ming-mei（裘兰兰，李明梅）. China Pharmacy（中国药房），2016，27(27)：3850.
