基于HE-CSR的红外与可见光图像改进融合方法

doi:10.3964/j.issn.1000-0593(2025)02-0558-11

摘要
参考文献
相关文章 (14)

全文: PDF (86248 KB)
输出: BibTeX | EndNote (RIS)

摘要：红外与可见光图像由于二者之间存在互补特性而成为图像融合研究领域重要的源图像。目前红外与可见光图像融合方法存在的一个问题是图像中纹理信息的细节保存能力有限。为解决此问题，首先采用基于直方图均衡化（HE）的方法分别对配准后红外和可见光图像灰度值的范围进行动态扩展，实现图像增强，使得图像中的纹理信息更加突出，同时图像的背景与纹理细节之间的对比度也得以提高。其次，采用L₀梯度最小化滤波器分别对增强后的图像进行平滑处理，得到图像的背景层，然后通过源图像与背景层进行差运算得到细节层，实现红外和可见光图像的分解。再次，将卷积稀疏表示（CSR）与特征相似性分析结合应用于红外与可见光图像融合：对两个包含丰富纹理信息的细节层采用基于卷积稀疏表示的融合策略进行融合，该过程中为了降低卷积稀疏表示的误配敏感度，采用基于窗口的平均策略对图像活动水平图进行处理，使卷积稀疏表示对误配不敏感；针对背景图像中存在大量冗余信息的问题，对两个背景层进行特征相似性分析，并以此作为确定二者在的融合过程中的重要程度的依据。最后，通过L₀梯度最小化图像分解的逆变换重构初步融合后的细节层和背景层，得到红外与可见光图像的融合结果。采用TNO数据集中21个场景的场景1（建筑）和场景2（树林）两组图像进行主观视觉分析，观测结果表明基于HE-CSR的融合方法较现有CVT、DTCWT、FPDE、GTF、IFEVIP、LP、RP和CSR共8种图像融合典型方法，在视觉上更好地保留了图像中的纹理细节；同时，进一步对TNO数据集所有场景图像融合效果进行客观指标评价，结果显示基于HE-CSR的融合结果的SF、SD、SCD、AG、EN、CC的6个评价指标值分别为7.316 6、37.350 5、1.704 1、5.571 4、6.756 3和0.744 6，分别提高了19.54%、21.87%、13.11%、31.31%、2.17%和8.23%。实验结果表明，所提出的HE-CSR融合方法在主观视觉分析和客观指标评价上都优于其他典型方法，为红外与可见光图像融合提供一种新的更有效的模型及方法。

关键词：图像融合；直方图均衡化；特征相似性分析；卷积稀疏表示

Abstract：Infrared and visible light images have become important source images in the field of image fusion research due to their complementary characteristics, and the current infrared and visible light image fusion methods have the problem of limited ability to preserve the details of texture information in the image. In this paper, firstly, the histogram equalization (HE) method is used to dynamically expand the range of gray values of infrared and visible images after alignment to achieve image enhancement, which makes the texture information in the image more prominent, and at the same time, the contrast between the background of the image and the details of the texture is also improved. Secondly, the gradient minimization filter is used to smooth the enhanced image to obtain the background layer of the image, and then the source image and the background layer are used to obtain the detail layer by difference operation to realize the decomposition of the infrared and visible light images. Again, the convolutional sparse representation (CSR) is combined with feature similarity analysis for infrared and visible image fusion: the two detail layers containing rich texture information are fused using the fusion strategy based on the convolutional sparse representation, and in this process to reduce the mismatch sensitivity of the convolutional sparse representation method, a window-based averaging strategy is adopted to process the activity level map of the image, to make the convolutional sparse representation insensitive to mismatches; For the problem of large amount of redundant information in the background image, the feature similarity analysis of the two background layers is carried out, and this is used as the basis for determining the degree of importance of the two in the fusion process. Finally, the preliminary fused detail and background layers are reconstructed by the inverse transform of gradient minimization image decomposition, and the fusion results of infrared and visible light images are finally obtained. Two sets of images, scenes 1 (buildings) and 2 (woods), from 21 scenes in the TNO dataset are used for subjective visual analysis. The observation results show that the HE-CSR-based fusion method visually retains the image's texture details better than the existing eight typical image fusion methods, including CVT, DTCWT, FPDE, GTF, IFEVIP, LP, RP, and CSR. At the same time, the objective index evaluation of the image fusion effect of all scenes in the TNO dataset is further conducted. The results show that the six evaluation index values of SF, SD, SCD, AG, EN, and CC for the HE-CSR-based fusion results are 7.316 6, 37.350 5, 1.704 1, 5.571 4, 6.756 3, and 0.744 6, which are respectively improved by 19.54%, 21.87%, 13.11%, 31.31%, 2.17%, and 8.23%. The experimental results show that the HE-CSR fusion method proposed in this paper outperforms other typical methods in subjective visual analysis and objective index evaluation and provides a new and more effective model and method for infrared and visible image fusion.

Key words：Image fusion; Histogram equalization; Feature similarity analysis; Convolutional sparse representation

收稿日期: 2023-10-30 修订日期: 2024-05-30

中图分类号:

TP391

基金资助: 国家自然科学基金项目（62162036，62172062），云南省“兴滇英才支持计划”青年人才项目（YNQR-QNRC-2019-085）资助

通讯作者: 郑万波 E-mail: zwanbo2001@163.com

作者简介: 朱榕，女，1998年生，昆明理工大学理学院硕士研究生 e-mail: zhurongdyx@163.com

引用本文:

朱榕，郑万波，王耀，谭春琳. 基于HE-CSR的红外与可见光图像改进融合方法[J]. 光谱学与光谱分析, 2025, 45(02): 558-568.
ZHU Rong, ZHENG Wan-bo, WANG Yao, TAN Chun-lin. Improved Fusion Algorithm for Infrared and Visible Images Based on Image Enhancement and Convolutional Sparse Representation. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2025, 45(02): 558-568.

链接本文:

https://www.gpxygpfx.com/CN/10.3964/j.issn.1000-0593(2025)02-0558-11 或 https://www.gpxygpfx.com/CN/Y2025/V45/I02/558

[1] ZHOU Pei-pei, HOU Xing-lin(周培培, 侯幸林). Journal of System Simulation(系统仿真学报), 2022, 34(6): 1267.
[2] LI Zi-tong, ZHAO Jian-kang, XU Jing-ran, et al(李紫桐, 赵健康, 徐静冉, 等). Acta Photonica Sinica(光子学报), 2023, 52(11): 1.
[3] YANG Chang-chun, YE Zan-ting, LIU Ban-teng, et al(杨长春, 叶赞挺, 刘半藤, 等). Journal of Zhejiang University (Engineering Science)[浙江大学学报（工学版）], 2023, 57(2): 226.
[4] LIU Gang, XU Lin-feng(刘刚，许林峰). Control and Decision(控制与决策), 2010, 25(4): 623.
[5] Liu J, Fan X, Jiang J, et al. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(1): 105.
[6] Zhou Z, Wang B, Li S, et al. Information Fusion, 2016, 30: 15.
[7] Ma J, Zhou Z, Wang B, et al. Infrared Physics & Technology, 2017, 82: 8.
[8] Li J, Huo H, Li C, et al. IEEE Transactions on Multimedia, 2021, 23: 1383.
[9] Ma J, Zhang H, Shao Z, et al. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 1.
[10] ZHANG Zhou-yu, CAO Yun-feng, DING Meng, et al(张洲宇, 曹云峰, 丁萌, 等). Journal of Harbin Institute of Technology(哈尔滨工业大学学报), 2021, 53(12): 51.
[11] YANG Yan-chun, LI Xiao-miao, DANG Jian-wu, et al(杨艳春, 李小苗, 党建武, 等). Journal of Beijing University of Aeronautics and Astronautics(北京航空航天大学学报), 2023, 49(9): 2317.
[12] XU Shao-ping, CHEN Xiao-jun, LUO Jie, et al(徐少平, 陈晓军, 罗洁, 等). Pattern Recognition and Artificial Intelligence(模式识别与人工智能), 2022, 35(12): 1089.
[13] CHEN Yong, ZHANG Jiao-jiao, WANG Zhen(陈永, 张娇娇, 王镇). Optics and Precision Engineering(光学精密工程), 2022, 30(18): 2253.
[14] Li G, Lin Y, Qu X. Information Fusion, 2021, 71: 109.
[15] MIN Li, TIAN Lin-lin, ZHAO Huai-ci, et al(闵莉, 田林林, 赵怀慈, 等). Control and Decision(控制与决策), 2024, 39(1): 227.
[16] FENG Xin, FANG Chao, GONG Hai-feng, et al(冯鑫, 方超, 龚海峰, 等). Spectroscopy and Spectral Analysis(光谱学与光谱分析), 2023, 43(2): 590.
[17] LI Yan-feng, LIU Ming-yang, HU Jia-ming, et al(李延风, 刘名扬, 胡嘉明, 等). Journal of Jilin University (Engineering and Technology Edition)[吉林大学学报(工学版)], 2024, 54(6): 1777.
[18] YANG Pei, GAO Lei-fu, ZI Ling-ling(杨培, 高雷阜, 訾玲玲). Journal of Image and Graphics(中国图象图形学报), 2021, 26(10): 2433.
[19] PANG Zhong-xiang, LIU Gui-hua, CHEN Chun-mei, et al(庞忠祥, 刘桂华, 陈春梅, 等). Control and Decision(控制与决策), 2024, 39(3): 910.
[20] Nencini F, Garzelli A, Baronti S, et al. Information Fusion, 2007, 8(2): 143.
[21] Lewis J J, O'Callaghan R J, Nikolov S G, et al. Information Fusion, 2007, 8(2): 119.
[22] GAO Xue-qin, LIU Gang, XIAO Gang, et al(高雪琴, 刘刚, 肖刚, 等). Acta Automatica Sinica(自动化学报), 2020, 46(4): 796.
[23] Yu R, Chen W, Zhou D. IEEE Access, 2020, 8: 50091.
[24] Zhang Y, Zhang L, Bai X, et al. Infrared Physics & Technology, 2017, 83: 227.
[25] Ma J, Ma Y, Li C. Information Fusion, 2018, 45: 153.
[26] Toet A. Pattern Recognition Letters, 1989, 9(4): 245.
[27] Liu Y, Chen X, Ward R K, et al. IEEE Signal Processing Letters, 2016, 23(12): 1882.
[28] HE Zhi-bo, ZENG Xiang-jin, DENG Chen, et al(何智博, 曾祥进, 邓晨, 等). Infrared Technology(红外技术), 2023, 45(6): 598.
[29] Ren Y, Li Z, Xu C. Mathematics, 2023, 11(17): 3689.
[30] Toet A. Data in Brief, 2017, 15: 249.
[31] ZHU Wen-qing, TANG Xin-yi, ZHANG Rui, et al(朱雯青, 汤心溢, 张瑞, 等). Journal of Infrared and Millimeter Waves(红外与毫米波学报), 2021, 40(5): 696.
[32] TANG Lin-feng, ZHANG Hao, XU Han, et al(唐霖峰, 张浩, 徐涵, 等). Journal of Image and Graphics(中国图象图形学报), 2023, 28(1): 3.