A ‎robust ‎‎unsupervised ‎‎feature ‎s‎election based on ‎‎subspace ‎l‎earning and ‎‎adaptive ‎‎graph ‎‎structure

Document Type : Research Article

Authors

1 Department of Applied Mathematics‎, ‎University of Kurdistan‎, ‎Sanandaj‎, ‎Iran

2 Department of Applied Mathematics, University of Kurdistan, Sanandaj, Iran

3 School of Engineering‎, ‎RMIT University‎, ‎Melbourne‎, ‎Australia

Abstract

Feature selection is vital for improving high-dimensional data analysis by identifying a subset of representative and uncorrelated features. This paper presents an unsupervised feature selection algorithm based on subspace learning and adaptive graph structure (UFSAG). The UFSAG uses matrix factorization to preserve global data structure and incorporates local correlations into its objective function. It also integrates sample similarity graph learning to maintain data geometry. Unlike prior methods, UFSAG employs adaptive local structure learning to reduce noise and enhance feature selection. By inducing row sparsity in the feature coefficient matrix using the $\ell_{2,1}$-norm, UFSAG identifies representative features. Comparative experiments on six datasets show UFSAG's superior clustering performance over twelve state-of-the-art methods.

Keywords

Main Subjects

References

[1] V. Amarnadh, N.R. Moparthi, Range control-based class imbalance and optimized granular elastic net regression feature selection for credit risk assessment, Knowl. Inf. Syst. 66 (2024) 5281–5310.
[2] M.C. Barbieri, B.I. Grisci, M. Dorn, Analysis and comparison of feature selection methods towards performance and stability, Expert Syst. Appl. 16 (2024) 123667.
[3] R. Bellman, A mathematical formulation of variational processes of adaptive type, Proceedings of the fourth Berkeley symposium on mathematical statistics and probability (1961) 37–49.
[4] D.P. Bertsekas, Nonlinear Programming, Athena Scientific, Massachusetts, 2016.
[5] D. Cai, X. He, J. Han, T.S. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. 33 (2010) 1548–1560.
[6] D. Cai, C. Zhang, X. He, Unsupervised feature selection for multi-cluster data, Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining (2010) 333–342.
[7] S. Chen, S. Ma, A. Man-Cho So, T. Zhang, Nonsmooth optimization over the Stiefel manifold and beyond: Proximal gradient method and recent variants, SIAM Rev. 66 (2024) 319–352.
[8] J.G. Dy, C.E. Brodley, Feature selection for unsupervised learning, J. Mach. Learn. Res. 5 (2004) 845–889.
[9] M.X. Hou, J.X. Li, G.M. Lu, A supervised non-negative matrix factorization model for speech emotion recognition, Speech Commun. 124 (2020) 13–20.
[10] A.J. Izenman, Introduction to manifold learning, Wiley Interdisciplinary Reviews: Comput. Statist. 4 (2012) 439–446.
[11] R. Kumar, R.S. Anand, Bearing fault diagnosis using multiple feature selection algorithms with SVM, Prog. Artif. Intell. 13 (2024) 119–133.
[12] J. Lee, W. Se , D.-W. Kim, Efficient information-theoretic unsupervised feature selection, Electron. Lett. 54 (2018) 76–77.
[13] D. Lee, H.S. Seung, Algorithms for non-negative matrix factorization, Proceedings of the 13th international conference on neural information processing systems (2000) 535–541.
[14] C. Li, H. Che, MF. Leung, C. Liu, Z. Yan, Robust multi-view non-negative matrix factorization with adaptive graph and diversity constraints, Inf. Sci. 634 (2023) 587–607.
[15] Z. Li, Y. Yang, J. Liu, X. Zhou, H. Lu, Unsupervised feature selection using nonnegative spectral analysis, Proceedings of the AAAI Conference on Artificial Intelligence (2021) 1026–1032.
[16] Y. Liu, K. Liu, C. Zhang, J. Wang, X. Wang, Unsupervised feature selection via diversity-induced self-representation, Neurocomputing 219 (2017) 350–363.
[17] H.R. Li, J.S. Zhang, G. Shi, J.M. Liu, Graph-based discriminative nonnegative matrix factorization with label information, Neurocomputing 266 (2017) 91–100.
[18] H. Lim, D.-W. Kim, Pairwise dependence-based unsupervised feature selection, Pattern Recognit. 111 (2021) 107663.
[19] H. Liu, H. Motoda, R. Setiono, Z. Zhao, Feature selection: An ever evolving frontier in data mining, [Feature Selection in Data Mining Logo] Proceedings of machine learning research (2010) 4–13.
[20] M. Meila, Comparing clusterings–an information based distance, J. Multivar. Anal. 98 (2007) 873–895.
[21] A. Moslemi, A. Ahmadian, Subspace learning for feature selection via rank revealing QR factorization: Fast feature selection, Expert Syst. Appl. 256 (2024) 124919.
[22] F. Nie, H. Huang, X. Cai, C. Ding, Efficient and robust feature selection via joint ℓ2,1-norms minimization, Adv. Neur. In. 23 (2010) 1–9.
[23] F. Nie, X. Wang, H. Huang, Clustering and projected clustering with adaptive neighbors, Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining (2014) 977–986.
[24] M. Noroozi, M. Salahi, S. Eskandari, Feature selection via mixed-integer program and supervised infinite feature selection method, J. Math. Model. 13(2) (2025) 341–356.
[25] O.N. Oyelade, E.F. Aminu, H. Wang, K. Rafferty, An adaptation of hybrid binary optimization algorithms for medical image feature selection in neural network for classification of breast cancer, Neurocomputing 617 (2025) 129018.
[26] M.G. Parsa, H. Zare, M. Ghatee, Low-rank dictionary learning for unsupervised feature selection, Expert Syst. Appl. 202 (2022) 117149.
[27] M. Qi, T. Wang, F. Liu, B. Zhang, J. Wang, Y. Yi, Unsupervised feature selection by regularized matrix factorization, Neurocomputing 273 (2018) 593–610.
[28] M. Samareh-Jahani, G. Aghamollaei, M. Eftekhari, F. Saberi-Movahed, Unsupervised feature selection guided by orthogonal representation of feature space, Neurocomputing 516 (2023) 61–76.
[29] M. Samareh-Jahani, F. Saberi-Movahed, M. Eftekhari, G. Aghamollaei, P. Tiwari, Low-Redundant unsupervised feature selection based on data structure learning and feature orthogonalization, Expert Syst. Appl. 240 (2024) 122556.
[30] H. Sohrabi, S. Esmaeili, P. Moradi, Feature representation via graph regularized entropy weighted nonnegative matrix factorization, AUT J. Math. Comput., 5 (2024) 289–304.
[31] J. Tang, H. Feng, Robust local-coordinate non-negative matrix factorization with adaptive graph for robust clustering, Inf. Sci. 610 (2022) 1058–1077.
[32] H. Wang, H. Huang, C. Ding, Simultaneous clustering of multi-type relational data via symmetric nonnegative matrix tri-factorization, Proceedings of the 20th ACM international conference on information and knowledge management (2011) 279–284.
[33] S. Wang, W. Pedrycz, Q. Zhu, W. Zhu, Subspace learning for unsupervised feature selection via matrix factorization, Pattern Recognit. 48 (2015) 10–19.
[34] S. Wang, J. Tang, H. Liu, Embedded unsupervised feature selection, Proceedings of the AAAI conference on artificial intelligence (2015) 470–476.
[35] Y. Wu, Y. Wang, L. Hu, J. Hu, DCGNN: Adaptive deep graph convolution for heterophily graphs, Inf. Sci. 666 (2024) 120–427.
[36] W.H. Wu, S. Kwong, J.H. Hou, Y.H. Jia, H.H.S. Ip, Simultaneous dimensionality reduction and classification via dual embedding regularized nonnegative matrix factorization, IEEE Trans. Image Process. 28 (2019) 3836–3847.
[37] W. Xu, Y. Gong, Document clustering by concept factorization, Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval, Sheffield, UK (2004) 202–209.
[38] Y. Yang, H.T. Shen, Z. Ma, Z. Huang, X. Zhou, ℓ2,1-norm regularized discriminative feature selection for unsupervised learning, Proceedings of international joint conference on artificial intelligence (2011) 1589–1594.
[39] Y. Yi, S. Lai, S. Li, J. Dai, W. Wang, J. Wang, RRNMF-MAGL: Robust regularization non-negative matrix factorization with multi-constraint adaptive graph learning for dimensionality reduction. Inf. Sci. 640 (2023) 119029.
[40] Z. Zhou, G. Si, H. Sun, K. Qu, W. Hou, A robust clustering algorithm based on the identification of core points and k-NN kernel density estimation, Expert Syst. Appl. 195 (2022) 116573.
[41] S. Zhou, P. Song, Y. Yu, W. Zheng, Structural regularization based discriminative multi-view unsupervised feature selection, Knowl. Based Syst. 272 (2023) 110601.
[42] Y. Zhao, S. Zheng, J. Pei, X. Yang, Multiple discriminant preserving support subspace RBFNNs with graph similarity learning, Inf. Sci. 619 (2023) 421–438.
Journal of Mathematical Modeling
Volume 14, Issue 1
March 2026
Pages 207-230

Files

Share

How to cite

Statistics