|
|
|
|
|
|
Parallel Extraction and Analysis of Abnormal Features of QSO Spectra Based on Sparse Subspace |
MA Yang, ZHANG Ji-fu, CAI Jiang-hui, YANG Hai-feng, ZHAO Xu-jun* |
School of Computer Science and Technology,Taiyuan University of Science and Technology, Taiyuan 030024, China |
|
|
Abstract Quasi-Stellar Object (QSO), the most distant celestial body observed by humans, has important scientific value for the universe evolution.Quasars are far away from the earth, and their redshift values are large, which results in few features appearing in the optical observation window. Hence, constructing a QSO template is a difficult task, and then making the automatic identification of QSO become an urgent problem. The abnormal characteristics extraction and analysis of QSO spectra are helpful to solve the above problems, there by further providing strong evidence for exploring the mysteries of the universe. The outlier detection method, one of the main research contents in the data mining field, can detect rare data objects and anomalous characteristics from massive size data. Therefore, outlier detection can facilitate novel schemes for identifying rare QSOs and achieving validation. As a new generation of big data distributed processing framework, Spark provides an efficient, easy-to-implement and reliable parallel programming platform for analyzing and processing massive celestial spectra. The overarching goal of this paper is to investigate parallel detection methods based on sparse-subspace for QSO anomalous characteristics. We aim to optimize the performance of parallel abnormal detection through the virtue of the high-performance data processing capacity of the Spark programming model on clusters. This research embraces the following three modules, namely, QSO spectral feature reduction, sparse-subspace construction and search of QSO spectral data, and parallel algorithm design and analysis of QSO abnormal characteristics extraction. The QSO spectral feature reduction module exhibits superb performance in speeding up abnormal characteristic’s detection efficiency by the attribute correlation analysis. Specifically, some spectral feature lines with clustering structure are identified, which are usually concentrated in dense regions and are meaningless for detecting anomalous spectral features. The module aims to prune the redundant feature lines so as to narrow the search range of abnormal quasars. The second module is the sparse-subspace construction and search module, which extends the particle swarm optimization method to search sparse subspaces so as to obtain the anomalous features quickly. At the heart of this module is the determination of the sparse-subspace that contains QSO spectra anomalous features, where the subspace density of QSO spectra is measured by a threshold of sparse coefficients. In the third module, a parallel detection algorithm for abnormal spectral data under the MapReduce framework is proposed. The algorithm consists of two MapReduce: parallel data reduction strategy and sparse-subspace parallel search technique. Finally, the detectedanomalous features of some QSOs are analyzed, measured and verified by human eyes, which fully demonstrates that the sparse-subspace can provide effective support and strong evidence for identifying candidate sources of special and unknown QSOs.
|
Received: 2020-11-02
Accepted: 2021-02-10
|
|
Corresponding Authors:
HAO Xu-jun
E-mail: zxj0226@126.com
|
|
[1] Luo A L, Zhao Y H, Zhao G, et al. Research in Astronomy and Astrophysics, 2015, 15(8): 1095.
[2] Liu X W, Zhao G, Hou J L. Research in Astronomy and Astrophysics, 2015, 15(8): 1089.
[3] Logan C H A, Fotopoulou S. Astronomy & Astrophysics, 2020, 633: A154.
[4] Lawrence, Andy. Nature Astronomy, 2018, 2(2): 102.
[5] Makhija S, Saha S, Basak S, et al. Astronomy and Computing, 2019, 29: 100313.
[6] Rubinur K, Das M, Kharb P, et al. Monthly Notices of the Royal Astronomical Society, 2017, 465(4): 4772.
[7] Yang Y, Cai J, Yang H, et al. Expert Systems with Applications, 2020, 139: 112846.
[8] Zhao Xujun, Rao Yuanqi, Cai Jianghui, et al. IEEE Access, 2020, 8: 29987.
[9] QU Cai-xia, YANG Hai-feng, CAI Jiang-hui, et al(屈彩霞, 杨海峰, 蔡江辉, 等). Spectroscopy and Spectral Analysis (光谱学与光谱分析), 2020, 40(4): 1304.
[10] Cheng L T, Zhang F H. Research in Astronomy and Astrophysics, 2020, 20(9): 148.
[11] Li L J, Qian S B, Zhang J, et al. Research in Astronomy and Astrophysics, 2020, 20(6): 94.
[12] Sun S P, Liao S H, Guo Q, et al. Research in Astronomy and Astrophysics, 2020, 20(4): 21.
[13] Frew D J, Parker Q A, Bojičić I S. Monthly Notices of the Royal Astronomical Society, 2016, 455(2): 1459. |
[1] |
FAN Ping-ping,LI Xue-ying,QIU Hui-min,HOU Guang-li,LIU Yan*. Spectral Analysis of Organic Carbon in Sediments of the Yellow Sea and Bohai Sea by Different Spectrometers[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 52-55. |
[2] |
YANG Chao-pu1, 2, FANG Wen-qing3*, WU Qing-feng3, LI Chun1, LI Xiao-long1. Study on Changes of Blue Light Hazard and Circadian Effect of AMOLED With Age Based on Spectral Analysis[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 36-43. |
[3] |
BAO Hao1, 2,ZHANG Yan1, 2*. Research on Spectral Feature Band Selection Model Based on Improved Harris Hawk Optimization Algorithm[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2024, 44(01): 148-157. |
[4] |
LI Qi-chen1, 2, LI Min-zan1, 2*, YANG Wei2, 3, SUN Hong2, 3, ZHANG Yao1, 3. Quantitative Analysis of Water-Soluble Phosphorous Based on Raman
Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(12): 3871-3876. |
[5] |
LIANG Jin-xing1, 2, 3, XIN Lei1, CHENG Jing-yao1, ZHOU Jing1, LUO Hang1, 3*. Adaptive Weighted Spectral Reconstruction Method Against
Exposure Variation[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(11): 3330-3338. |
[6] |
MA Qian1, 2, YANG Wan-qi1, 2, LI Fu-sheng1, 2*, CHENG Hui-zhu1, 2, ZHAO Yan-chun1, 2. Research on Classification of Heavy Metal Pb in Honeysuckle Based on XRF and Transfer Learning[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(09): 2729-2733. |
[7] |
HUANG Chao1, 2, ZHAO Yu-hong1, ZHANG Hong-ming2*, LÜ Bo2, 3, YIN Xiang-hui1, SHEN Yong-cai4, 5, FU Jia2, LI Jian-kang2, 6. Development and Test of On-Line Spectroscopic System Based on Thermostatic Control Using STM32 Single-Chip Microcomputer[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(09): 2734-2739. |
[8] |
ZHENG Yi-xuan1, PAN Xiao-xuan2, GUO Hong1*, CHEN Kun-long1, LUO Ao-te-gen3. Application of Spectroscopic Techniques in Investigation of the Mural in Lam Rim Hall of Wudang Lamasery, China[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(09): 2849-2854. |
[9] |
WANG Jun-jie1, YUAN Xi-ping2, 3, GAN Shu1, 2*, HU Lin1, ZHAO Hai-long1. Hyperspectral Identification Method of Typical Sedimentary Rocks in Lufeng Dinosaur Valley[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(09): 2855-2861. |
[10] |
WANG Jing-yong1, XIE Sa-sa2, 3, GAI Jing-yao1*, WANG Zi-ting2, 3*. Hyperspectral Prediction Model of Chlorophyll Content in Sugarcane Leaves Under Stress of Mosaic[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(09): 2885-2893. |
[11] |
WANG Yu-qi, LI Bin, ZHU Ming-wang, LIU Yan-de*. Optimizations of Sample and Wavelength for Apple Brix Prediction Model Based on LASSOLars Algorithm[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(05): 1419-1425. |
[12] |
LI Shuai-wei1, WEI Qi1, QIU Xuan-bing1*, LI Chuan-liang1, LI Jie2, CHEN Ting-ting2. Research on Low-Cost Multi-Spectral Quantum Dots SARS-Cov-2 IgM and IgG Antibody Quantitative Device[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(04): 1012-1016. |
[13] |
JIN Cui1, 4, GUO Hong1*, YU Hai-kuan2, LI Bo3, YANG Jian-du3, ZHANG Yao1. Spectral Analysis of the Techniques and Materials Used to Make Murals
——a Case Study of the Murals in Huapen Guandi Temple in Yanqing District, Beijing[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(04): 1147-1154. |
[14] |
DING Kun-yan1, HE Chang-tao2, LIU Zhi-gang2*, XIAO Jing1, FENG Guo-ying1, ZHOU Kai-nan3, XIE Na3, HAN Jing-hua1. Research on Particulate Contamination Induced Laser Damage of Optical Material Based on Integrated Spectroscopy[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(04): 1234-1241. |
[15] |
ZHANG Bao-ping1, NING Tian1, ZHANG Fu-rong1, CHEN Yi-shen1, ZHANG Zhan-qin2, WANG Shuang1*. Study on Raman Spectral Characteristics of Breast Cancer Based on
Multivariable Spectral Data Analysis Methods[J]. SPECTROSCOPY AND SPECTRAL ANALYSIS, 2023, 43(02): 426-434. |
|
|
|
|