基于Wasserstein散度的t-SNE相似性度量方法研究

doi:10.3964/j.issn.1000-0593(2023)12-3806-07

Abstract
References
Related Citation (15)

Download: PDF (2936 KB)
Export: BibTeX | EndNote (RIS)

Abstract Near-infrared spectroscopy has the characteristics of high dimension, high redundancy, and nonlinearity, which seriously affects the similarity measurement results between samples. This paper proposes a t-distributed stochastic nearest neighbor embedding algorithm (Wt-SNE) based on Wasserstein divergence. Based on the idea of manifold learning algorithm, Gaussian distribution is used to convert the distance of high-dimensional data into a probability distribution, and t-distribution is used to represent the probability distribution of corresponding data points in low-dimensional space, which is more inclined to long-tailed distribution. The probability distribution embedding of high-dimensional data is mapped to the low-dimensional space. The low-dimensional manifold structure is reconstructed, the Wasserstein divergence is introduced to measure the difference between the probability distributions in the two spaces, and the similarity of the two distributions is improved by reducing the divergence value. In this way, the dimensionality reduction processing of high-dimensional data is realized. In order to verify the effectiveness of the Wt-SNE algorithm, this paper first performs dimensionality reduction projection on tobacco NIR spectral data and compares it with PCA, LPP, and t-SNE algorithms. The results show that the sample category boundaries in the low-dimensional space are more obvious after the dimensionality reduction of the Wt-SNE algorithm. Secondly, the KNN, SVM, and PLS-DA classifiers were used to predict the tobacco origin of the reduced-dimensional data, and the accuracy rates were 93.8%, 91.5%, and 92.7% respectively, indicating that the reduced-dimensional data not only reconstructed the spatial structure of the original spectrum but also retained the similarity relationship between samples. Finally, tobacco from a particular cigarette formula was selected for single material target tobacco replacement, and the replacement samples were selected based on the Marginal distance between the candidate samples and the target samples. The experiments showed that the replacement tobacco selected by Wt-SNE had the highest similarity to the target tobacco, the chemical composition contents such as nicotine and total sugar were less different from those of the target tobacco, and the aroma, smoke, and taste scores showed high consistency. The method can effectively measure the similarity between the NIR spectra of the tobacco and provide a strong basis for the maintenance of the cigarette formula.

Key words：Near-infrared spectrum; Data dimension reduction; t-SNE algorithm; Wasserstein divergence; Similarity measurement

Received: 2022-10-04 Accepted: 2023-04-14

ZTFLH:

O657.33

Corresponding Authors: QIN Yu-hua E-mail: yuu71@163.com

Cite this article:

LIU Xin-peng,SUN Xiang-hong,QIN Yu-hua, et al. Research on t-SNE Similarity Measurement Method Based on Wasserstein Divergence[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3806-3812.

URL:

https://www.gpxygpfx.com/EN/10.3964/j.issn.1000-0593(2023)12-3806-07 OR https://www.gpxygpfx.com/EN/Y2023/V43/I12/3806

[1] CHEN Yuan-zhe, WANG Qiao-hua, GAO Sheng, et al(陈远哲，王巧华，高升，等). Laser & Optoelectronics Progress(激光与光电子学进展), 2021, 58(12): 1230001.
[2] GAO Yun-fei, FU Lin-yu, QU Jun, et al(高云飞，付霖宇，瞿军，等). Electronics Optics & Control(电光与控制), 2019, 26(6): 18.
[3] LIU Zhe, MENG Hui, ZHANG Yong-bin, et al(刘喆，孟辉，张永彬，等). Acta Optica Sinica(光学学报), 2022, 42(4): 0430001.
[4] LI Hong-da, LI De-cheng, ZENG Rong(李宏达，李德成，曾荣). Acta Pedologica Sinica(土壤学报), 2021, 58(5): 1224.
[5] ZANG Zhuo, LIN Hui, YANG Min-hua(臧卓，林辉，杨敏华). Science of Surveying and Mapping(测绘科学), 2014, 39(2): 146.
[6] XU Bao-ding, DING Xiang-qian, QIN Yu-hua, et al(徐宝鼎，丁香乾，秦玉华，等). Laser & Optoelectronics Progress(激光与光电子学进展), 2019, 56(3): 251.
[7] JIANG Bin, ZHAO Zi-liang, WANG Shu-ting, et al(姜斌，赵梓良，王淑婷，等). Spectroscopy and Spectral Analysis(光谱学与光谱分析), 2020, 40(9): 2913.
[8] MA Yan-jun, LI Xue-ying, MA Li, et al(马雁军，李雪莹，马莉，等). Acta Tabacaria Sinica(中国烟草学报), 2017, 23(3): 38.
[9] MA Meng-hao, WANG Zhe(马幪浩，王喆). Computer Engineering and Applications(计算机工程与应用), 2022, 58(5): 193.
[10] BIAN Rong-zheng, ZHANG Jian, ZHOU Liang, et al(边荣正，张鉴，周亮，等). Journal of Computer-Aided Design & Computer Graphics(计算机辅助设计与图形学学报), 2021, 33(11): 1746.
[11] YAN Cai-rong, ZHANG Qing-long, ZHAO Xue, et al(燕彩蓉，张青龙，赵雪，等). Journal of Computer Research and Development(计算机研究与发展), 2016, 52(12): 2793.
[12] LI Jian-guo, ZHAO Hai-tao, SUN Shao-yuan(李建国，赵海涛，孙韶媛). Computer Science(计算机科学), 2019, 46(6): 212.
[13] WANG Xiao-shun, CHEN Dan, LIN Lei-cheng(王孝顺，陈丹，林垒城). Application Research of Computers(计算机应用研究), 2020, 37(10): 3164.
[14] Rubner Y, Tomasi C, Guibas L J. International Journal of Computer Vision, 2000, 40(2): 99.
[15] Zhou Wending，Bao Shijian，Xu Fangmin, et al. The Journal of China Universities of Posts and Telecommunications, 2020, 27(1): 1.
[16] MENG Xiao-chen, WANG Yue, ZHU Lian-qing(孟晓辰，王玥，祝连庆). Journal of Biomedical Engineering(生物医学工程学杂志), 2018, 35(5): 697.
[17] XU Hui-di, LIN Lu-lu, LI Zheng, et al(徐荟迪，林露璐，李征，等). Acta Optica Sinica(光学学报), 2019, 39(3): 0330001.