|
|
|
|
|
|
| Three-Scale Deconstruction and Sparse Representation of Infrared and Visible Image Fusion |
| JI Jing-yu, ZHANG Yu-hua, XING Na, WANG Chang-long, LIN Zhi-long*, YAO Jiang-yi |
Army Engineering University Shijiazhuang Campus, Shijiazhuang 050003，China
|
|
|
|
|
Abstract To improve the processing performance of source images with noise and improve the contrast and structural information of the fused images, an infrared and visible image fusion algorithm based on three-scale decomposition and sparse representation is proposed. First, to enhance the noise removal effect and maintain the structure and edge characteristics of the source image, the rolling guidance filter is used to decompose the source image, and the source image is decomposed into the base layer and the detail layer. Secondly, in order to make full use of the details and energy in the base component and reduce the complexity of the model, a structure-texture decomposition model is constructed. The base layer is decomposed into the base structure layer and base texture layer again, Then by analyzing the different characteristics of the three components, different fusion rules are utilized to fuse the three components respectively. The detail component contains the main noise components, but the noise level is different. Therefore, the sparse fusion denoising parameter is adaptively determined according to the noise level of the image to realize the fusion and denoising of the detail component at the same time and can effectively improve computational efficiency; for the base structure components, which contain fewer detail features, the weighted average technology based on the visual saliency map is directly used for pre-fusion; for the base texture components, because they contain visually important information or image features, such as edges, lines and contours and other activity information, which can reflect the main details of the original base image, the principal component analysis method is used for pre-fusion. Finally, the fused image is obtained by reconstructing the detail, base structure and texture layers. In order to verify the effectiveness of the proposed method, several groups of infrared and visible images were selected for experiments and compared with five recent methods, including CNN, FPDE, ResNet, IFEVIP and TIF, and the results were analyzed in a subjective and objective form. The experimental results show that, compared with other image fusion algorithms, this method can consider the fusion of noise-perturbed and noise-free images and can retain the details, brightness and structure of the source image in the fusion image with or without noise. Moreover, can effectively eliminate noise.
|
|
Received: 2022-01-12
Accepted: 2023-08-28
|
|
|
|
Corresponding Authors:
LIN Zhi-long
E-mail: 601386022@qq.com
|
|
[1] SHEN Yu, YUAN Yu-bin, PENG Jing, et al(沈 瑜, 苑玉彬, 彭 静，等). Spectroscopy and Spectral Analysis(光谱学与光谱分析), 2021, 41(7): 2023.
[2] Angel Y, Turner D, Parkes D, Parkes S, et al. Remote Sensing, 2020, 12(1): 34.
[3] Li S, Kwok J T, Wang Y. Information Fusion, 2001, 2(3): 169.
[4] Kong Weiwei. Infrared Physics & Technology, 2014, 63: 110.
[5] Long J, Shelhamer E, Darrell T. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640.
[6] Li S, Yin H, Fang L. IEEE Transactions on Bio-Medical Engineering, 2012, 59(12): 3450.
[7] Elad M, Aharon M. Image Denoising Via Learned Dictionaries and Sparse Representation. IEEE Computer Society Conference on Computer Vision & Pattern Recognition. New York, USA: IEEE, 2006: 3736.
[8] Yang B, Li S. IEEE Transactions on Instrumentation and Measurement, 2010, 59(4): 884.
[9] Liu Y, Wang Z. IET Image Processing, 2015, 9(5): 347.
[10] Kim M, Han D K, Ko H. Information Fusion, 2016, 27: 198.
[11] Li H, Wang Y, Yang Z, et al. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1082.
[12] Zhang Qi, Shen X Y, Xu L, et al. Rolling Guidance Filter. European Conference on Computer Vision，2014.
[13] He K, Sun J, Tang X. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 35: 1397
[14] Lee H, Jeon J, Kim J, et al. Computer Graphics Forum，2017, 36(6): 262.
[15] Zhu B, Liu J Z, Cauley S F, et al. Nature, 2017, 555(7697): 487.
[16] Aharon M, Elad M, Bruckstein A. IEEE Transactions on Signal Processing, 2006, 54(11): 4311.
[17] Donoho D L, Tsaig Y, Drori I, et al. IEEE Transactions on Information Theory, 2012, 58(2): 1094.
[18] Cheng M N, Mitra N J, Huang X, et al. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(3): 569.
[19] Gonzalez R C, Woods R E, Masters B R. Journal of Biomedical Optics, 2009, 14: 029901.
[20] Liu Y, Chen X, Cheng J, et al. International Journal of Wavelets, Multiresolution and Information Processing, 2018, 16(3): 1850018.
[21] Bavirisetti D P, Xiao G, Liu G. Multi-Sensor Image Fusion Based on Fourth Order Partial Differential Equations. 20th International Conference on Information Fusion (Fusion), 2017.
[22] Li H, Wu X J, Durrani T S. Infrared Physics & Technology, 2019, 102: 103039.
[23] Zhang Y, Zhang L J, Bai X Z. Infrared Physics & Technology, 2017, 83: 227.
[24] Bavirisetti D P, Dhuli R. Infrared Physics & Technology, 2016, 76: 52.
[25] Chibani Y. Isprs Journal of Photogrammetry & Remote Sensing, 2006, 60(5): 306.
[26] Xydeas C S, Petrovic V. Objective Image Fusion Performance Measure, IEEE Xplore, 2000, doi: 10.1049/el:20000267.
[27] Chen Yin, Blum R S. Image & Vision Computing, 2009, 27(10): 1421.
[28] Qu G, Zhang D, Yan P. Electronics Letters, 2002, 38(7): 313.
[29] Wang Qiang, Shen Y, Jin J. Image Fusion, 2008, 19: 469.
[30] Yang C, Zhang J Q, Wang X R, et al. Information Fusion, 2008, 9(2): 156. |
| [1] |
LI Si-yuan, JIAO Jian-nan, WANG Chi*. Specular Reflection Removal Method Based on Polarization Spectrum
Fusion and Its Application in Vegetation Health Monitoring[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(11): 3607-3614. |
| [2] |
ZHU Wen-qing1, 2, 3, ZHANG Ning1, 2, 3, LI Zheng1, 2, 3*, LIU Peng1, 3, TANG Xin-yi1, 3. A Multi-Task Convolutional Neural Network for Infrared and Visible Multi-Resolution Image Fusion[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(01): 289-296. |
| [3] |
XU Xue-bin1, 2, XING Xiao-min1, 2*, AN Mei-juan1, 2, CAO Shu-xin1, 2, MENG Kan1, 2, LU Long-bin1, 2. Palmprint Recognition Method Based on Multispectral Image Fusion[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42(11): 3615-3625. |
| [4] |
ZHOU Bing, LI Bing-xuan*, HE Xuan, LIU He-xiong，WANG Fa-zhen. Classification of Camouflages Using Hyperspectral Images Combined With Fusing Adaptive Sparse Representation and Correlation Coefficient[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2021, 41(12): 3851-3856. |
| [5] |
KONG Hui-hua1, 2, LIAN Xiang-yuan1, CHEN Ping2, PAN Jin-xiao1, 2. Research on Color Characterization of Material Components Based on Spectral CT[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2021, 41(11): 3612-3617. |
| [6] |
CUI Xiao-rong, SHEN Tao*, HUANG Jian-lu, SUN Bin-bin. Infrared Mid-Wave and Long-Wave Image Fusion Based on FABEMD and Improved Local Energy Window[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2021, 41(07): 2043-2049. |
| [7] |
SHEN Yu, YUAN Yu-bin*, PENG Jing. Research on Near Infrared and Color Visible Fusion Based on PCNN in Transform Domain[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2021, 41(07): 2023-2027. |
| [8] |
ZHANG Gen-wei1, PENG Si-long2, 3, GUO Teng-xiao1，YANG Jie1, YANG Jun-chao1, ZHANG Xu1, CAO Shu-ya1*, HUANG Qi-bin1*. Ion Mobility Spectrometry Spectrum Reconstruction and Characteristic Peaks Extraction Algorithm Research[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2020, 40(09): 2681-2685. |
| [9] |
LIANG Kun1, 2, ZHANG Xia-xia1, 2, DING Jing1, 2, XU Jian-hong3, HAN Dong-shen1, 2, SHEN Ming-xia1, 2*. Discrimination of Wheat Scab Infection Level by Fourier Mid-Infrared Technology Combined with Sparse Representation Based Classification Method[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2019, 39(10): 3251-3255. |
| [10] |
ZHANG Jin1, WANG Jie1, SHEN Yan3, ZHANG Jin-bo4, CUI Hong-liang1,2*, SHI Chang-cheng2*. Wavelet-Based Image Fusion Method Applied in the Terahertz Nondestructive Evaluation[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2017, 37(12): 3683-3688. |
| [11] |
LIU Feng1, SHEN Tong-sheng2, GUO Shao-jun1，ZHANG Jian3. Multi-Spectral Ship Target Recognition Based on Feature Level Fusion[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2017, 37(06): 1934-1940. |
| [12] |
LIU Jia-ni, JIN Wei-qi*, LI Li, WANG Xia . Visible and Infrared Thermal Image Fusion Algorithm Based on Self-Adaptive Reference Image [J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2016, 36(12): 3907-3914. |
| [13] |
WANG Cai-ling1, 2, WANG Hong-wei3, HU Bing-liang1, WEN Jia4, XU Jun5, LI Xiang-juan2 . A Novel Spatial-Spectral Sparse Representation for Hyperspectral Image Classification Based on Neighborhood Segmentation [J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2016, 36(09): 2919-2924. |
| [14] |
LIN Su-zhen, WANG Dong-juan, WANG Xiao-xia, ZHU Xiao-hong. Multi-Band Texture Image Fusion Based on the Embedded Multi-Scale Decomposition and Possibility Theory[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2016, 36(07): 2337-2343. |
| [15] |
WEN Jia1, 2, ZHAO Jun-suo2, WANG Cai-ling3, XIA Yu-li2 . Decomposition of Interference Hyperspectral Images Using Improved Morphological Component Analysis [J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2016, 36(01): 254-258. |
|
|
|
|
