Abstract：In the present paper, the outlier detection methods for determination of oil yield in oil shale using near-infrared (NIR) diffuse reflection spectroscopy was studied. During the quantitative analysis with near-infrared spectroscopy, environmental change and operator error will both produce outliers. The presence of outliers will affect the overall distribution trend of samples and lead to the decrease in predictive capability. Thus, the detection of outliers are important for the construction of high-quality calibration models. The methods including principal component analysis-Mahalanobis distance (PCA-MD) and resampling by half-means (RHM) were applied to the discrimination and elimination of outliers in this work. The thresholds and confidences for MD and RHM were optimized using the performance of partial least squares (PLS) models constructed after the elimination of outliers, respectively. Compared with the model constructed with the data of full spectrum, the values of RMSEP of the models constructed with the application of PCA-MD with a threshold of a value equal to the sum of average and standard deviation of MD, RHM with the confidence level of 85%, and the combination of PCA-MD and RHM, were reduced by 48.3%, 27.5% and 44.8%, respectively. The predictive ability of the calibration model has been improved effectively.
赵振英，林 君，张怀柱* . 近红外光谱法分析油页岩含油率中异常样品识别和剔除方法的研究 [J]. 光谱学与光谱分析, 2014, 34(06): 1707-1710.
ZHAO Zhen-ying, LIN Jun, ZHANG Huai-zhu* . Research on Outlier Detection Methods for Determination of Oil Yield in Oil Shales Using Near-Infrared Spectroscopy . SPECTROSCOPY AND SPECTRAL ANALYSIS, 2014, 34(06): 1707-1710.
[1] Zhang Xueqing, Zhang Lanying, Ren Hejun. Applied Mechanics and Materials, 2013, 295-298: 146. [2] Ille Johannes, Hans Luik, Jorgen A Bojesen-Koefoed, et al. Oil Shale, 2012, 29(3): 206. [3] Wang Sha, Jiang Xiumin, Han Xiangxin, et al. Energy, 2012, 42: 224. [4] The Measuration of Oil Yield form Oil Shale-Low Temperature Carbonization(油页岩含油率测定法(低温干馏法)) , SH/T 0508—92. [5] WANG Zhi-hong, LIU Jie, WANG Jing-ru, et al(王智宏，刘 杰，王静茹，等). Journal of Jinlin University·Engineering and Technology Edition(吉林大学学报·工学版)，2013，43(4): 1018. [6] LU Wan-zhen(陆婉珍). The Modern Analysis Technique of Near-Infrared Spectrum(现代近红外光谱分析技术). 2rd Ed.(第2版). Beijing: China Petrochemical Press(北京：中国石化出版社)，2006. [7] Liu Zhichao, Cai Wensheng, Shao Xueguang. Science in China Series B: Chemistry, 2008, 51(8): 751. [8] Egan W J, Morgan S L. Anal. Chem.，1998, 70: 2375. [9] CHEN Bin, ZOU Xian-yong, ZHU Wen-jing(陈 斌，邹贤勇，朱文静). Journal of Jiangsu University·Natural Science Edition(江苏大学学报·自然科学版), 2008. 7. [10] CHU Xiao-li(褚小立). Molecular Spectroscopy Analytical Technology Combined with Chemometrics and its Applications(化学计量学方法与分子光谱分析技术). Beijing: Chemical Industry Press (北京：化学工业出版社), 2011.
