Softmax self representation learning for unsupervised feature selection

Nasserassadi, Hossein; Kyanfar, Faranges; Saberi Movahed, Farid; Salemi Parizi, Abbas

doi:10.22124/jmm.2026.32592.2964

Softmax self representation learning for unsupervised feature selection

Articles in Press

Document Type : Research Article

Authors

¹ Department of Applied Mathematics, Shahid Bahonar University of Kerman, Kerman, Iran.

² Department of Applied Mathematics, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman

³ Department of Computer Science, Faculty of Sciences and Modern Technologies, Graduate University of Advanced Technology, Kerman, Iran

10.22124/jmm.2026.32592.2964

Abstract

Self-Representation (SR) models play a fundamental role in numerous unsupervised learning tasks, particularly in feature selection and clustering, by capturing intrinsic relational structures. However, learning reliable weight matrices in SR models remains challenging, as conventional nonnegativity constraints are often insufficient to control coefficient magnitudes. This limitation can lead to dense, unstable, and weakly interpretable solutions. To address these issues, we propose a softmax-based reparameterization for both sample and feature SR weight matrices. This probabilistic normalization enforces nonnegativity and unit-sum constraints, suppresses coefficient explosion, and induces competitive and interpretable affinity structures. Moreover, the proposed reparameterization transforms the original constrained optimization into an unconstrained problem, enabling efficient and stable gradient-based optimization. Building on this framework, we develop three SR variants, termed Softmax SR, Softmax Mixture SR, and Softmax Bilinear SR, each equipped with an efficient iterative optimization scheme. Extensive experiments on four benchmark datasets demonstrate that the proposed methods consistently outperform state-of-the-art approaches, highlighting the effectiveness of our approach in addressing key challenges in SR model optimization and its practical potential in real-world applications. Moreover,
they achieve consistent performance gains, delivering overall improvements of around 2--7 percentage points in clustering accuracy (ACC), normalized mutual information (NMI), and Purity compared to competing non-Softmax SR and related methods across four benchmark datasets.
The code is also available at \url{https://github.com/FaridSabM/SSR}.}

Keywords