Multi-Band Remote Sensing Image Sharpening: A Survey
TAO Jing-zhe1, 3, SONG De-rui1, 3, SONG Chuan-ming2, WANG Xiang-hai1, 2*
1. School of Geography, Liaoning Normal University, Dalian 116029, China
2. School of Computer and Information Technology, Liaoning Normal University, Dalian 116081, China
3. National Marine Environmental Monitoring Center, Dalian 116023, China
Abstract：Due to the limitations of imaging mechanisms, the current technical conditions of remote sensing hardware are not yet able to acquire multi-band remote sensing images with high spatial and high spectral resolution simultaneously. Multi-band remote sensing image is a three-dimensional image collection that reflects the information of different narrow-band intervals. It contains two-dimensional spatial information and one-dimensional spectral information. The spatial information reflects the geometric characteristics of the scene, and the spectral information corresponds to the electromagnetic wave characteristics of the ground objects in different bands. To compensate for the deficiency of spatial information acquisition in multi-band remote sensing images, sharpening of the images, which enhances their spatial resolution by using auxiliary images, has been emphasized. The sharpening of multi-band remote sensing images can not only improve the visual effect of the images, but also lay the foundation for subsequent qualitative and quantitative remote sensing applications such as ground object classification, change detection and parameter inversion, and thus has been a very significant and continuously active research direction in the field of remote sensing image processing. This paper reviews the research progress of multi-band remote sensing image sharpening methods. Firstly, the connotation of multi-band remote sensing image sharpening is expressed. Secondly, from the perspective of panchromatic sharpening of multispectral (MS) images and in the context of algorithm implementation techniques, the research progress and problems of MS image sharpening methods based on Component Substitution (CS), Multi-resolution Analysis (MRA), Optimization Model (OM) and Deep Learning (DL) are investigated and discussed respectively. Thirdly, the sharpening characteristics of HS images are analyzed in light of the characteristics of Hyperspectral (HS) images that are different from those of MS, and some specific HS sharpening methods that are different from those of MS are discussed and summarized. Finally, the future development of multi-band remote sensing image sharpening methods is prospected. The reasons why the CS and MRA methods are currently more recognized by the mainstream and the future sharpening field will show a relevant convergence of multiple methods are discussed, respectively.
陶兢喆，宋德瑞，宋传鸣，王相海. 多波段遥感影像锐化方法研究进展[J]. 光谱学与光谱分析, 2023, 43(10): 2999-3008.
TAO Jing-zhe, SONG De-rui, SONG Chuan-ming, WANG Xiang-hai. Multi-Band Remote Sensing Image Sharpening: A Survey. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(10): 2999-3008.
[1] Yue L W, Shen H F, Li J, et al. Signal Processing, 2016, 128(11): 389.
[2] Alparone L, Aiazzi B, Baronti S, et al. Remote Sensing Image Fusion. CRC Press, 2015.
[3] Garzelli A. Remote Sensing, 2016, 8(10): 797.
[4] Meng X, Shen H, Li H, et al. Information Fusion, 2019, 46(3): 102.
[5] Vivone G, Dalla Mura M, Garzelli A, et al. IEEE Geoscience and Remote Sensing Magazine, 2020, 9(1): 53.
[6] Paris C, Bioucas-Dias J, Bruzzone L. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(3): 1545.
[7] Yokoya N, Iwasaki A. Proc IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2013, 4086.
[8] Pignatti S, Palombo A, Pascucci S, et al. Proc IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2013, 4558.
[9] Natale V G, Kafri A, Tidhar G A, et al. Proc 5th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2013, doi: 10.1109/WHISPERS.2013.8080667.
[10] Tu T M, Su S C, Shyu H C, et al. Information Fusion, 2001, 2(3): 177.
[11] Aiazzi B, Baronti S, Selva M. IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(10): 3230.
[12] Choi J, Yu K, Kim Y. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(1): 295.
[13] Garzelli A, Nencini F, Capobianco L. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(1): 228.
[14] Ghadjati M, Moussaoui A, Boukharouba A. Remote Sensing Letters, 2019, 10(3): 264.
[15] Ghahremani M, Ghassemian H. IEEE Geoscience and Remote Sensing Letters, 2016, 13(11): 1606.
[16] Restaino R, Dalla Mura M, Vivone G, et al. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2): 753.
[17] Wang W, Jiao L, Yang S. IEEE Geoscience and Remote Sensing Letters, 2015, 12(4): 781.
[18] Vivone G. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6421.
[19] Schowengerdt R A. Remote Sensing: Models and Methods for Image Processing. Amsterdam: Elsevier, 2006.
[20] Ranchin T, Wald L. Photogrammetric Engineering and Remote Sensing, 2000, 66(1): 49.
[21] Vivone G, Marano S, Chanussot J. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(9): 6152.
[22] Lu X, Zhang J, Li T, et al. IEEE Geoscience and Remote Sensing Letters, 2016, 13(7): 892.
[23] Delleji T, Kallel A, Ben Hamida A. International Journal of Remote Sensing, 2016, 37(24): 6041.
[24] Choi J, Kim G, Park N, et al. Remote Sensing, 2017, 9(10): 976.
[25] Vivone G, Restaino R, Chanussot J. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(2): 984.
[26] Sulaiman A G, Elashmawi W H, El-Tawel G S. Sensing and Imaging, 2020, 21(1): 3.
[27] Choi J, Park H, Seo D. Remote Sensing, 2019, 11(6): 633.
[28] Hallabia H, Kallel A, Hamida A B. Proc 2nd International Conference on Advanced Technologies for Signal and Image Processing (ATSIP). IEEE, 2016: 505(doi: 10.1109/ATSIP.2016.7523137).
[29] Vivone G, Addesso P, Restaino R, et al. IEEE Transactions on Geoscience and Remote Sensing, 2018, 57(1): 540.
[30] Otazu X, Gonzalez-Audicana M, Fors O, et al. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(10): 2376.
[31] Shah V P, Younan N H, King R L. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(5): 1323.
[32] Jameel A, Riaz M M, Ghafoor A. IEEE Sensors Journal, 2016, 16(1): 192.
[33] Kim Y, Kim M, Choi J, et al. IEEE Geoscience and Remote Sensing Letters, 2017, 14(12): 2295.
[34] Nunez J, Otazu X, Fors O, et al. IEEE Transactions on Geoscience and Remote Sensing, 1999, 37(3): 1204.
[35] Vivone G, Alparone L, Garzelli A, et al. Remote Sensing, 2019, 11(19): 2315.
[36] Ballester C, Caselles V, Igual L, et al. International Journal of Computer Vision, 2006, 69(1): 43.
[37] Tony F C, SHEN Jian-hong（陈繁昌, 沈建红）. Image Processing and Analysis: Variational, PDE, Wavelet and Stochastic Methods（图像处理与分析: 变分, PDE, 小波及随机方法）. Beijing: China Science Publishing & Media Ltd, Science Press（北京: 中国科技出版传媒有限公司科学出版社）, 2009.
[38] Boyd S, Parikh N, Chu E. Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers. Norwell: Now Publishers Inc, 2011.
[39] Moeller M, Wittman T, Bertozzi A L. Proceedings of SPIE-The International Society for Optical Engineering, 2009, 7334: 7334/E (doi: 10.1117/12.818243).
[40] Zhang Y, De Backer S, Scheunders P. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(11): 3834.
[41] Joshi M, Jalobeanu A. IEEE Transactions on Geoscience and Remote Sensing, 2010, 48(3): 1245.
[42] Palsson F, Sveinsson J R, Ulfarsson M O. IEEE Geoscience and Remote Sensing Letters, 2014, 11(1): 318.
[43] Jiang Y, Ding X, Zeng D, et al. Proceedings of the IEEE International Conference on Computer Vision, 2015: 540 (doi: 10.1109/ICCV.2015.69).
[44] Chen C, Li Y, Liu W, et al. IEEE Transactions on Image Processing, 2015, 24(11): 4213.
[45] Takeyama S, Ono S, Kumazawa I. Proc 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, 1603.
[46] Pan Z W, Shen H L. IEEE Trans Image Process, 2019, 28(4): 1783.
[47] Khademi G, Ghassemian H. IEEE Geoscience and Remote Sensing Letters, 2016, 13(11): 1606.
[48] Wang W, Liu H, Liang L, et al. International Journal of Remote Sensing, 2019, 40(8): 3029.
[49] Palsson F, Ulfarsson M O, Sveinsson J R. IEEE Geoscience and Remote Sensing Letters, 2020, 17(4): 656.
[50] Yin H. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7): 4938.
[51] Gogineni R, Chaturvedi A. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(10): 4024.
[52] Romano Y, Elad M, Milanfar P. SIAM Journal on Imaging Sciences, 2017, 10(4): 1804.
[53] Mäkinen Y, Azzari L, Foi A. IEEE Transactions on Image Processing, 2020, 29(8): 8339.
[54] Teodoro A M, Bioucas-Dias J M, Figueiredo M A T. IEEE Transactions on Image Processing, 2018, 28(1): 451.
[55] Shao Z, Wang L, Wang Z, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(8): 2663.
[56] Scarpa G, Vitale S, Cozzolino D. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(9): 5443.
[57] Liu X, Wang Y, Liu Q. Proc 25th IEEE International Conference on Image Processing (ICIP), 2018, 873.
[58] Xu B, Wang N, Chen T, et al. Computer Science, 2015, arXiv.1505.00853.
[59] Huang W, Xiao L, Wei Z, et al. IEEE Geoscience and Remote Sensing Letters, 2015, 12(5): 1037.
[60] Zhong J, Yang B, Huang G, et al. Sensing and Imaging, 2016, 17(1): 10 (doi: 10.1007/s11220-016-0135-6).
[61] Masi G, Cozzolino D, Verdoliva L, et al. Remote Sensing, 2016, 8(7): 594.
[62] Wang Y, Deng L J, Zhang T J, et al. Proceedings of the 29th ACM International Conference on Multimedia, 2021, 4472.
[63] Wang D, Li Y, Ma L, et al. Remote Sensing, 2019, 11(22): 2608.
[64] Benzenati T, Kessentini Y, Kallel A. Expert Systems with Applications, 2022, 188: 115996.
[65] Zhang Y, Liu C, Sun M, et al. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(8): 5549.
[66] Qu Y, Baghbaderani R K, Qi H, et al. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(4): 3192.
[67] Wu X, Huang T Z, Deng L J, et al. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, 14687.
[68] Palsson F, Sveinsson J R, Ulfarsson M O. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5): 639.
[69] Xie W, Lei J, Cui Y, et al. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(5): 1529.
[70] Cao X, Fu X, Hong D, et al. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5404713(doi: 10.1109/TGRS.2021.3115501).
[71] Xiong Z, Guo Q, Liu M, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14(11): 897.
[72] Eismann M T, Hardie R C. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(9): 1924.
[73] He L, Zhu J, Li J, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(8): 3092.
[74] He L, Zhu J, Li J, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13(9): 5898.
[75] He G, Zhong J, Lei J, et al. Remote Sensing, 2019, 11(22): 2691.
[76] Veganzones M A, Simoes M, Licciardi G, et al. IEEE Transactions on Image Processing, 2015, 25(1): 274.
[77] Yokoya N, Yairi T, Iwasaki A. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(2): 528.
[78] Selva M, Aiazzi B, Butera F, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6): 3008.
[79] Chen Z, Pu H, Wang B, et al. IEEE Geoscience and Remote Sensing Letters, 2014, 11(8): 1418.
[80] Picone D, Restaino R, Vivone G, et al. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5): 739.
[81] Xu Y, Wu Z, Chanussot J, et al. IEEE Transactions on Image Processing, 2019, 28(6): 3034.
[82] Wang X, Mu Z, Song R, et al. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(7): 4803.
[83] Lee D D, Seung H S. Nature, 1999, 401(10): 788.
[84] Nascimento J M P, Dias J M B. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(4): 898.
[85] Robinson G D, Gross H N, Schott J R. Remote Sensing of Environment, 2000, 71(3): 272.
[86] Bendoumi M A, He M, Mei S. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(10): 6574.
[87] Li X, Zhang L, You J. Remote Sensing, 2019, 11(6): 694.
[88] Kim Y, Choi J, Han D, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6): 2997.
[89] Loncan L, Almeida L B D, Bioucas-Dias J M, et al. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3): 27.
[90] Yokoya N, Grohnfeldt C, Chanussot J. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(2): 29.
[91] Xie B, Zhang H K, Huang B. Remote Sensing, 2017, 9(5): 443.
