Classification of Hybrid Convolution Hyperspectral Images Based on
Attention Mechanism
LIU Yu-juan1, 2, 3, LIU Yan-da1, 2, 3, YAN Zhen1, 4, ZHANG Zhi-yong1, 2, 3, CAO Yi-ming1, 2, 3, SONG Ying1, 2, 3*
1. College of Instrumentation & Electrical Engineering，Jilin University, Changchun 130061, China
2. National Geophysical Exploration Equipment Engineering Research Center，Jilin University, Changchun 130061, China
3. Key Laboratory of Geophysical Exploration Equipment Ministry of Education，Jilin University, Changchun 130061, China
4. School of Instrumentation Science and Engineering，Harbin Institute of Technology，Harbin 150006，China
摘要： 高光谱图像以其高分辨率的空间和光谱信息在军事、航空航天及民用等遥感领域均有重要应用，具有重要的研究意义。深度学习具有学习能力强、覆盖范围广及可移植性强的优势，成为目前高精度高光谱图像分类技术研究的热点。其中卷积神经网络(CNN)因强大的特征提取能力广泛应用于高光谱图像分类方法研究中，取得了有效的研究成果，但该类方法通常单独基于2D-CNN或3D-CNN进行，针对高光谱图像的单一特征，一是不能充分利用高光谱数据本身完整的特征信息；二是虽然相应提取网络局部特征优化性好，但是整体泛化能力不足，在深度挖掘HSI的空间和光谱信息方面存在局限性。鉴于此，提出了基于注意力机制的混合卷积神经网络模型(HybridSN_AM )，使用主成分分析法对高光谱图像进行降维，采用卷积神经网络作为分类模型的主体，通过注意力机制筛选出更有区分度的特征，使模型能够提取到更精确、更核心的空间-光谱信息，实现高光谱图像的高精度分类。对Indian Pines(IP)、University of Pavia (UP)和Salinas (SA)三个数据集进行了应用实验，结果表明，基于该模型的目标图像总体分类精度、平均分类精度和Kappa系数均高于98.14%、97.17%、97.87%。与常规HybridSN模型对比表明，HybridSN_AM模型在三个数据集上的分类精度分别提升了0.89%、0.07%和0.73%。有效解决了高光谱图像空间-光谱特征提取与融合的难题，提高HSI分类的精度，具有较强的泛化能力，充分验证了注意力机制结合混合卷积神经网络在高光谱图像分类中的有效性和可行性，对高光谱图像分类技术的发展及应用具有重要的科学价值。
关键词：高光谱图像分类；注意力机制；卷积神经网络；多特征融合；主成分分析
Abstract：Hyperspectral Imagery (HSI), based on its high-resolution spatial and spectral information, has important applications in military, aerospace, civil, and other remote sensing fields, which has great research significance. Deep learning has the advantages of strong learning ability, wide coverage, and strong portability, which has become a hot spot in the research of high-precision hyperspectral image classification. Convolutional Neural Networks (CNN) are widely used in the research of hyperspectral image classification because of their powerful feature extraction ability and have achieved effective research results. Still, such methods are usually based on 2D-CNN or 3D-CNN alone. For the single feature of hyperspectral image, the complete feature information of hyperspectral data cannot be fully utilized. Secondly, the local feature optimization of the corresponding extraction network is good, but the overall generalization ability is insufficient. There are limitations in the deep mining of spatial and spectral information of HSI. Because of this, this paper proposes a Hybrid Spectral Convolutional Neural Network Attention Mechanism (HybridSN_AM) based on attention mechanism. The principal component analysis method is used to reduce the dimension of hyperspectral images, and the convolutional neural network is used as the main body of the classification model to screen out more distinguishable features through the attention mechanism so that the model can extract more accurate and more core joint space-spectral information, and realize high-precision classification of hyperspectral images. The proposed method was applied to three datasets: Indian Pines(IP), the University of Pavia (UP), and Salinas (SA). The experimental results show that the overall classification accuracy, average classification accuracy, and kappa coefficient of target images based on this model are higher than 98.14%, 97.17%，and 97.87%. Compared with the conventional HybridSN model, the classification accuracy of the HybridSN_AM model on the three data sets increased by 0.89%, 0.07%，and 0.73%, respectively. It effectively solves the problem of hyperspectral image joint space-spectral feature extraction and fusion, improves the accuracy of HSI classification, and has strong generalization ability. It fully verifies the effectiveness and feasibility of the attention mechanism combined with a hybrid convolutional neural network in hyperspectral image classification, which has important theoretical value for developing and applying hyperspectral image classification technology.
刘玉娟，刘颜达，闫 振，张智勇，曹益铭，宋 莹. 注意力机制的混合卷积高光谱图像分类方法[J]. 光谱学与光谱分析, 2024, 44(10): 2916-2922.
LIU Yu-juan, LIU Yan-da, YAN Zhen, ZHANG Zhi-yong, CAO Yi-ming, SONG Ying. Classification of Hybrid Convolution Hyperspectral Images Based on
Attention Mechanism. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(10): 2916-2922.
[1] TONG Qing-xi, ZHANG Bing, ZHANG Li-fu(童庆禧, 张 兵, 张立福). National of Remote Sensing Bulletin(遥感学报)，2016, 20(5): 689.
[2] YE Zhen, BAI Lin, HE Ming-yi(叶 珍, 白 璘, 何明一). Journal of Image and Graphics(中国图象图形学报), 2021, 26(8): 1737.
[3] YE Cheng-ming, LI Yao, CUI Peng, et al(叶成名, 李 尧, 崔 鹏, 等). The Chinese Journal of Geological Hazard and Control(中国地质灾害与防治学报), 2018, 29(5): 89.
[4] Hecker C A, Riley D N, van Meijde M, et al. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3): 1545.
[5] WU Jian-feng, HUANG Shu-cai, KANG Hong-xia, et al(吴建峰, 黄树彩, 康红霞, 等). Aerospace Technology(空天技术), 2015, 362(2): 68.
[6] Kendler S, Ron I, Cohen S, et al. IEEE Sensors Journal, 2019, 19(7): 2657.
[7] Melgani F, Bruzzone L. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(8): 1778.
[8] SU Hong-jun(苏红军). National of Remote Sensing Bulltin, 2022, 26(8): 1504.
[9] Licciardi G, Marpu P R, Chanussot J, et al. IEEE Geoscience and Remote Sensing Letters, 2012, 9(3): 447.
[10] DUAN Xiao-chuan, WANG Guang-jun, LIANG Si-hai, et al(段小川, 王广军, 梁四海, 等). Remote Sensing Information(遥感信息), 2021, 36(3): 76.
[11] Haut J M, Paoletti M E, Plaza J, et al. IEEE Transactions on Geoscience & Remote Sensing, 2018, 56(11): 6440.
[12] Chen Y, Jiang H, Li C, et al, IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10): 6232.
[13] GAO Hong-min, ZHU Min, CAO Xue-ying, et al(高红民, 朱 敏, 曹雪莹, 等). Journal of Image and Graphics(中国图象图形学报), 2023, 28(4): 1173.
[14] SUN Wen-bin, WANG Rong, GAO Rong-hua, et al(孙文斌, 王 荣, 高荣华, 等). Spectroscopy and Spectral Analysis(光谱学与光谱分析), 2022, 42(5): 1572.
[15] Mourya Sharan, Amuru Sai Dhiraj, Kuchi Kiran Kumar. IEEE Wireless Communications Letters, 2023, 12(1): 40.
