|
|
|
|
|
|
| Multi-Band Remote Sensing Image Sharpening: A Survey |
| TAO Jing-zhe1, 3, SONG De-rui1, 3, SONG Chuan-ming2, WANG Xiang-hai1, 2* |
1. School of Geography, Liaoning Normal University, Dalian 116029, China
2. School of Computer and Information Technology, Liaoning Normal University, Dalian 116081, China
3. National Marine Environmental Monitoring Center, Dalian 116023, China
|
|
|
|
|
Abstract Due to the limitations of imaging mechanisms, the current technical conditions of remote sensing hardware are not yet able to acquire multi-band remote sensing images with high spatial and high spectral resolution simultaneously. Multi-band remote sensing image is a three-dimensional image collection that reflects the information of different narrow-band intervals. It contains two-dimensional spatial information and one-dimensional spectral information. The spatial information reflects the geometric characteristics of the scene, and the spectral information corresponds to the electromagnetic wave characteristics of the ground objects in different bands. To compensate for the deficiency of spatial information acquisition in multi-band remote sensing images, sharpening of the images, which enhances their spatial resolution by using auxiliary images, has been emphasized. The sharpening of multi-band remote sensing images can not only improve the visual effect of the images, but also lay the foundation for subsequent qualitative and quantitative remote sensing applications such as ground object classification, change detection and parameter inversion, and thus has been a very significant and continuously active research direction in the field of remote sensing image processing. This paper reviews the research progress of multi-band remote sensing image sharpening methods. Firstly, the connotation of multi-band remote sensing image sharpening is expressed. Secondly, from the perspective of panchromatic sharpening of multispectral (MS) images and in the context of algorithm implementation techniques, the research progress and problems of MS image sharpening methods based on Component Substitution (CS), Multi-resolution Analysis (MRA), Optimization Model (OM) and Deep Learning (DL) are investigated and discussed respectively. Thirdly, the sharpening characteristics of HS images are analyzed in light of the characteristics of Hyperspectral (HS) images that are different from those of MS, and some specific HS sharpening methods that are different from those of MS are discussed and summarized. Finally, the future development of multi-band remote sensing image sharpening methods is prospected. The reasons why the CS and MRA methods are currently more recognized by the mainstream and the future sharpening field will show a relevant convergence of multiple methods are discussed, respectively.
|
|
Received: 2022-03-21
Accepted: 2022-11-03
|
|
|
|
Corresponding Authors:
WANG Xiang-hai
E-mail: xhwang@lnnu.edu.cn
|
|
[1] Yue L W, Shen H F, Li J, et al. Signal Processing, 2016, 128(11): 389.
[2] Alparone L, Aiazzi B, Baronti S, et al. Remote Sensing Image Fusion. CRC Press, 2015.
[3] Garzelli A. Remote Sensing, 2016, 8(10): 797.
[4] Meng X, Shen H, Li H, et al. Information Fusion, 2019, 46(3): 102.
[5] Vivone G, Dalla Mura M, Garzelli A, et al. IEEE Geoscience and Remote Sensing Magazine, 2020, 9(1): 53.
[6] Paris C, Bioucas-Dias J, Bruzzone L. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(3): 1545.
[7] Yokoya N, Iwasaki A. Proc IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2013, 4086.
[8] Pignatti S, Palombo A, Pascucci S, et al. Proc IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2013, 4558.
[9] Natale V G, Kafri A, Tidhar G A, et al. Proc 5th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2013, doi: 10.1109/WHISPERS.2013.8080667.
[10] Tu T M, Su S C, Shyu H C, et al. Information Fusion, 2001, 2(3): 177.
[11] Aiazzi B, Baronti S, Selva M. IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(10): 3230.
[12] Choi J, Yu K, Kim Y. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(1): 295.
[13] Garzelli A, Nencini F, Capobianco L. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(1): 228.
[14] Ghadjati M, Moussaoui A, Boukharouba A. Remote Sensing Letters, 2019, 10(3): 264.
[15] Ghahremani M, Ghassemian H. IEEE Geoscience and Remote Sensing Letters, 2016, 13(11): 1606.
[16] Restaino R, Dalla Mura M, Vivone G, et al. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2): 753.
[17] Wang W, Jiao L, Yang S. IEEE Geoscience and Remote Sensing Letters, 2015, 12(4): 781.
[18] Vivone G. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6421.
[19] Schowengerdt R A. Remote Sensing: Models and Methods for Image Processing. Amsterdam: Elsevier, 2006.
[20] Ranchin T, Wald L. Photogrammetric Engineering and Remote Sensing, 2000, 66(1): 49.
[21] Vivone G, Marano S, Chanussot J. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(9): 6152.
[22] Lu X, Zhang J, Li T, et al. IEEE Geoscience and Remote Sensing Letters, 2016, 13(7): 892.
[23] Delleji T, Kallel A, Ben Hamida A. International Journal of Remote Sensing, 2016, 37(24): 6041.
[24] Choi J, Kim G, Park N, et al. Remote Sensing, 2017, 9(10): 976.
[25] Vivone G, Restaino R, Chanussot J. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(2): 984.
[26] Sulaiman A G, Elashmawi W H, El-Tawel G S. Sensing and Imaging, 2020, 21(1): 3.
[27] Choi J, Park H, Seo D. Remote Sensing, 2019, 11(6): 633.
[28] Hallabia H, Kallel A, Hamida A B. Proc 2nd International Conference on Advanced Technologies for Signal and Image Processing (ATSIP). IEEE, 2016: 505(doi: 10.1109/ATSIP.2016.7523137).
[29] Vivone G, Addesso P, Restaino R, et al. IEEE Transactions on Geoscience and Remote Sensing, 2018, 57(1): 540.
[30] Otazu X, Gonzalez-Audicana M, Fors O, et al. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(10): 2376.
[31] Shah V P, Younan N H, King R L. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(5): 1323.
[32] Jameel A, Riaz M M, Ghafoor A. IEEE Sensors Journal, 2016, 16(1): 192.
[33] Kim Y, Kim M, Choi J, et al. IEEE Geoscience and Remote Sensing Letters, 2017, 14(12): 2295.
[34] Nunez J, Otazu X, Fors O, et al. IEEE Transactions on Geoscience and Remote Sensing, 1999, 37(3): 1204.
[35] Vivone G, Alparone L, Garzelli A, et al. Remote Sensing, 2019, 11(19): 2315.
[36] Ballester C, Caselles V, Igual L, et al. International Journal of Computer Vision, 2006, 69(1): 43.
[37] Tony F C, SHEN Jian-hong（陈繁昌, 沈建红）. Image Processing and Analysis: Variational, PDE, Wavelet and Stochastic Methods（图像处理与分析: 变分, PDE, 小波及随机方法）. Beijing: China Science Publishing & Media Ltd, Science Press（北京: 中国科技出版传媒有限公司科学出版社）, 2009.
[38] Boyd S, Parikh N, Chu E. Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers. Norwell: Now Publishers Inc, 2011.
[39] Moeller M, Wittman T, Bertozzi A L. Proceedings of SPIE-The International Society for Optical Engineering, 2009, 7334: 7334/E (doi: 10.1117/12.818243).
[40] Zhang Y, De Backer S, Scheunders P. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(11): 3834.
[41] Joshi M, Jalobeanu A. IEEE Transactions on Geoscience and Remote Sensing, 2010, 48(3): 1245.
[42] Palsson F, Sveinsson J R, Ulfarsson M O. IEEE Geoscience and Remote Sensing Letters, 2014, 11(1): 318.
[43] Jiang Y, Ding X, Zeng D, et al. Proceedings of the IEEE International Conference on Computer Vision, 2015: 540 (doi: 10.1109/ICCV.2015.69).
[44] Chen C, Li Y, Liu W, et al. IEEE Transactions on Image Processing, 2015, 24(11): 4213.
[45] Takeyama S, Ono S, Kumazawa I. Proc 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, 1603.
[46] Pan Z W, Shen H L. IEEE Trans Image Process, 2019, 28(4): 1783.
[47] Khademi G, Ghassemian H. IEEE Geoscience and Remote Sensing Letters, 2016, 13(11): 1606.
[48] Wang W, Liu H, Liang L, et al. International Journal of Remote Sensing, 2019, 40(8): 3029.
[49] Palsson F, Ulfarsson M O, Sveinsson J R. IEEE Geoscience and Remote Sensing Letters, 2020, 17(4): 656.
[50] Yin H. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7): 4938.
[51] Gogineni R, Chaturvedi A. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(10): 4024.
[52] Romano Y, Elad M, Milanfar P. SIAM Journal on Imaging Sciences, 2017, 10(4): 1804.
[53] Mäkinen Y, Azzari L, Foi A. IEEE Transactions on Image Processing, 2020, 29(8): 8339.
[54] Teodoro A M, Bioucas-Dias J M, Figueiredo M A T. IEEE Transactions on Image Processing, 2018, 28(1): 451.
[55] Shao Z, Wang L, Wang Z, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(8): 2663.
[56] Scarpa G, Vitale S, Cozzolino D. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(9): 5443.
[57] Liu X, Wang Y, Liu Q. Proc 25th IEEE International Conference on Image Processing (ICIP), 2018, 873.
[58] Xu B, Wang N, Chen T, et al. Computer Science, 2015, arXiv.1505.00853.
[59] Huang W, Xiao L, Wei Z, et al. IEEE Geoscience and Remote Sensing Letters, 2015, 12(5): 1037.
[60] Zhong J, Yang B, Huang G, et al. Sensing and Imaging, 2016, 17(1): 10 (doi: 10.1007/s11220-016-0135-6).
[61] Masi G, Cozzolino D, Verdoliva L, et al. Remote Sensing, 2016, 8(7): 594.
[62] Wang Y, Deng L J, Zhang T J, et al. Proceedings of the 29th ACM International Conference on Multimedia, 2021, 4472.
[63] Wang D, Li Y, Ma L, et al. Remote Sensing, 2019, 11(22): 2608.
[64] Benzenati T, Kessentini Y, Kallel A. Expert Systems with Applications, 2022, 188: 115996.
[65] Zhang Y, Liu C, Sun M, et al. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(8): 5549.
[66] Qu Y, Baghbaderani R K, Qi H, et al. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(4): 3192.
[67] Wu X, Huang T Z, Deng L J, et al. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, 14687.
[68] Palsson F, Sveinsson J R, Ulfarsson M O. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5): 639.
[69] Xie W, Lei J, Cui Y, et al. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(5): 1529.
[70] Cao X, Fu X, Hong D, et al. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5404713(doi: 10.1109/TGRS.2021.3115501).
[71] Xiong Z, Guo Q, Liu M, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14(11): 897.
[72] Eismann M T, Hardie R C. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(9): 1924.
[73] He L, Zhu J, Li J, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(8): 3092.
[74] He L, Zhu J, Li J, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13(9): 5898.
[75] He G, Zhong J, Lei J, et al. Remote Sensing, 2019, 11(22): 2691.
[76] Veganzones M A, Simoes M, Licciardi G, et al. IEEE Transactions on Image Processing, 2015, 25(1): 274.
[77] Yokoya N, Yairi T, Iwasaki A. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(2): 528.
[78] Selva M, Aiazzi B, Butera F, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6): 3008.
[79] Chen Z, Pu H, Wang B, et al. IEEE Geoscience and Remote Sensing Letters, 2014, 11(8): 1418.
[80] Picone D, Restaino R, Vivone G, et al. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5): 739.
[81] Xu Y, Wu Z, Chanussot J, et al. IEEE Transactions on Image Processing, 2019, 28(6): 3034.
[82] Wang X, Mu Z, Song R, et al. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(7): 4803.
[83] Lee D D, Seung H S. Nature, 1999, 401(10): 788.
[84] Nascimento J M P, Dias J M B. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(4): 898.
[85] Robinson G D, Gross H N, Schott J R. Remote Sensing of Environment, 2000, 71(3): 272.
[86] Bendoumi M A, He M, Mei S. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(10): 6574.
[87] Li X, Zhang L, You J. Remote Sensing, 2019, 11(6): 694.
[88] Kim Y, Choi J, Han D, et al. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6): 2997.
[89] Loncan L, Almeida L B D, Bioucas-Dias J M, et al. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3): 27.
[90] Yokoya N, Grohnfeldt C, Chanussot J. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(2): 29.
[91] Xie B, Zhang H K, Huang B. Remote Sensing, 2017, 9(5): 443.
|
| [1] |
LI Hu1, ZHONG Yun1, 2, FENG Ya-ting1, LIN Zhen1, ZHU Shi-jiang1, 2*. Multi-Vegetation Index Soil Moisture Inversion Model Based on UAV
Remote Sensing[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 207-214. |
| [2] |
ZHU Wen-jing1, 2，FENG Zhan-kang1, 2，DAI Shi-yuan1, 2，ZHANG Ping-ping3，JI Wen4，WANG Ai-chen1, 2，WEI Xin-hua1, 2*. Multi-Feature Fusion Detection of Wheat Lodging Information Based on UAV Multispectral Images[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 197-206. |
| [3] |
WANG Cai-ling1，ZHANG Jing1，WANG Hong-wei2*, SONG Xiao-nan1, JI Tong3. A Hyperspectral Image Classification Model Based on Band Clustering and Multi-Scale Structure Feature Fusion[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 258-265. |
| [4] |
ZHANG Nan-nan1, 3, CHEN Xi-ya1，CHANG Xin-fang1, XING Jian1, GUO Jia-bo1, CUI Shuang-long1*, LIU Yi-tong2*, LIU Zhi-jun1. Distributed Design of Optical System for Multi-Spectral Temperature
Pyrometer[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 230-233. |
| [5] |
GAO Hong-sheng1, GUO Zhi-qiang1*, ZENG Yun-liu2, DING Gang2, WANG Xiao-yao2, LI Li3. Early Classification and Detection of Kiwifruit Soft Rot Based on
Hyperspectral Image Band Fusion[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 241-249. |
| [6] |
WU Hu-lin1, DENG Xian-ming1*, ZHANG Tian-cai1, LI Zhong-sheng1, CEN Yi2, WANG Jia-hui1, XIONG Jie1, CHEN Zhi-hua1, LIN Mu-chun1. A Revised Target Detection Algorithm Based on Feature Separation Model of Target and Background for Hyperspectral Imagery[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 283-291. |
| [7] |
GAO Wei-ling, ZHANG Kai-hua*, XU Yan-fen, LIU Yu-fang*. Data Processing Method for Multi-Spectral Radiometric Thermometry Based on the Improved HPSOGA[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3659-3665. |
| [8] |
ZHANG Ning-chao1, YE Xin1, LI Duo1, XIE Meng-qi1, WANG Peng1, LIU Fu-sheng2, CHAO Hong-xiao3*. Application of Combinatorial Optimization in Shock Temperature
Inversion[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3666-3673. |
| [9] |
CHU Bing-quan1, 2, LI Cheng-feng1, DING Li3, GUO Zheng-yan1, WANG Shi-yu1, SUN Wei-jie1, JIN Wei-yi1, HE Yong2*. Nondestructive and Rapid Determination of Carbohydrate and Protein in T. obliquus Based on Hyperspectral Imaging Technology[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3732-3741. |
| [10] |
SHEN Si-cong, ZHANG Jing-xue, CHEN Ming-hui, LI Zhi-wei, SUN Sheng-nan, YAN Xue-bing*. Estimation of Above-Ground Biomass and Chlorophyll Content of
Different Alfalfa Varieties Based on UAV Multi-Spectrum[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3847-3852. |
| [11] |
HAO Zi-yuan1, YANG Wei1*, LI Hao1, YU Hao1, LI Min-zan1, 2. Study on Prediction Models for Leaf Area Index of Multiple Crops Based on Multi-Source Information and Deep Learning[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3862-3870. |
| [12] |
WANG Zhen-tao1, DAI Jing-min1*, YANG Sen2. Research on Multi-Spectral Thermal Imager Explosion Flame True
Temperature Field Measurment[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3885-3890. |
| [13] |
HUANG You-ju1, TIAN Yi-chao2, 3*, ZHANG Qiang2, TAO Jin2, ZHANG Ya-li2, YANG Yong-wei2, LIN Jun-liang2. Estimation of Aboveground Biomass of Mangroves in Maowei Sea of Beibu Gulf Based on ZY-1-02D Satellite Hyperspectral Data[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3906-3915. |
| [14] |
ZHOU Bei-bei1, LI Heng-kai1*, LONG Bei-ping2. Variation Analysis of Spectral Characteristics of Reclaimed Vegetation in an Ionic Rare Earth Mining Area[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3946-3954. |
| [15] |
LIANG Jin-xing1, 2, 3, XIN Lei1, CHENG Jing-yao1, ZHOU Jing1, LUO Hang1, 3*. Adaptive Weighted Spectral Reconstruction Method Against
Exposure Variation[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(11): 3330-3338. |
|
|
|
|
