Hyperspectral Reconstruction From RGB Images for Tissue Oxygen
Saturation Assessment
WANG Yu-chen1, 2, KONG Ling-qin1, 2, 3*, ZHAO Yue-jin1, 2, 3, DONG Li-quan1, 2, 3*, LIU Ming1, 2, 3, HUI Mei1, 2
1. School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
2. Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, Beijing Institute of Technology, Beijing 100081, China
3. Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing 314019, China
Abstract：Tissue Oxygenation is an important indicator of blood perfusion and oxygenation in tissues and blood. It is of great significance in clinical diagnosis and daily monitoring. Hyperspectral imaging is a new method to evaluate StO2 because of its non-contact and abundant spectral information. However, hyperspectral imaging equipment is expensive and complex to operate. These disadvantages limit its use environment and development. Traditional industrial cameras have a high spatial resolution of RGB images of skin tissue, but their spectral resolution is low. If the spectral resolution can be improved, it is possible to measure physiological parameters with high precision. This paper proposes a new method for estimating StO2 based on hyperspectral reconstruction of RGB images. Based on the depth learning method, the reconstruction model from RGB image to the hyperspectral image of skin tissue is constructed, and the hyperspectral image of skin tissue with high reliability is obtained. Then, the spatial two-dimensional distribution of StO2 is obtained using the improved Beer Lambert law formulations. This paper collected RGB images and hyperspectral images of hands of 49 subjects under different blood perfusion conditions using a common visible light camera and a hyperspectral camera as data sets. Based on the dimensionality reduction and denoising of hyperspectral images, the 450~600 nm (including 31 spectral channels) bands were selected as the reconstructed spectral bands according to the characteristic spectra of oxyhemoglobin and deoxyhemoglobin. The convolutional neural networks for spectral reconstruction of skin tissue based on depth learning is constructed. The experimental results show that the skin reflectance spectra obtained by the reconstructed model are in good agreement with those obtained by the hyperspectral camera, and the mean absolute error (MAE) between the two in the test set is 0.009 38, the root mean square error (RMSE) was 0.014 81. Then the similarity between the StO2 measurements from the reconstructed model and those from the hyperspectral camera was quantitatively evaluated. We used the samples in the test set to generate the StO2 spatial distribution maps by two methods respectively, and the two-dimensional correlation coefficients between them were in the reliable range (greater than 94%). These results show that the proposed method based on hyperspectral reconstruction of visible light images is reliable. This study provided a simple and low-cost method of StO2 monitoring for clinical diagnosis and monitoring of various diseases.
[1] Lu G, Fei B. Journal of Biomedical Optics, 2014, 19(1): 010901.
[2] ZHAO Jing, MA Bei, LIU Ming, et al（赵 静，马 贝，刘 明，等）. Spectroscopy and Spectral Analysis（光谱学与光谱分析）, 2022, 42(2): 512.
[3] Zuzak K J, Schaeberle M D, Gladwin M T, et al. Circulation, 2001, 104(24): 2905.
[4] Jakovels D, Spigulis J, Saknite I. Biophotonics: Photonic Solutions for Better Health Care Ⅱ. Proc SPIE, 2010, 7715: 77152Z.
[5] Kim T, Visbal-Onufrak M A, Konger R L, et al. Biomedical Optics Express, 2017, 8(11): 5282.
[6] Park S M, Visbal-Onufrak M A, Haque M M, et al. Optica, 2020, 7(6): 563.
[7] Alvarez-Gila A, Van De Weijer J, Garrote E. Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB. IEEE International Conference on Computer Vision Workshops, 2017.
[8] Shi Z, Chen C, Xiong Z, et al. HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images. IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018.
[9] Li J, Wu C, Song R, et al. Adaptive Weighted Attention Network with Camera Spectral Sensitivity Prior for Spectral Reconstruction from RGB Images. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.
[10] Sharma N, Hefeeda M. Hyperspectral Reconstruction from RGB Images for Vein Visualization. 11th ACM Multimedia Systems Conference, 2020.
[11] Jolivot R, Vabres P, Marzani F. Computerized Medical Imaging and Graphics, 2011, 35(2): 85.
[12] Zhao Y, Po L M, Yan Q, et al. Hierarchical Regression Network for Spectral Reconstruction from RGB Images. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.
[13] Cooper C E, Elwell C E, Meek J H, et al. Pediatric Research, 1996, 39(1): 32.
[14] Mansfield J R, Sowa M G, Payette J R, et al. IEEE Transactions on Medical Imaging, 1998, 17(6): 1011.
[15] Miclos S, Parasca S V, Calin M A, et al. Biomedical Optics Express, 2015, 6(9): 3420.
[16] Chen T, Yuen P, Richardson M, et al. The Imaging Science Journal, 2015, 63(5): 290.
[17] Prahl S. Optical Absorption of Hemoglobin. Oregon Medical Laser Center, http://omlc.ogi.edu/spectra/hemoglobin, 1999.
[18] Jacques S. Optical Absorption of Melanin. Oregon Medical Laser Center, https://omlc.org/spectra/melanin/index.html, 2001.
