Enhancing support vector machine performance through boundary-focused covariance matrix learning

Document Type : Research Article

Authors

Department of Applied Mathematics, University of Mazandaran, Babolsar, Iran

Abstract

Support Vector Machine (SVM) has proven effective in various classification tasks by focusing on maximizing the margin between classes. However, standard SVM often fails to consider the underlying geometric structure of data near the decision boundary, particularly in scenarios where class overlap or noise occurs. This paper proposes a new method that focuses on boundary-critical samples those near the decision boundary by integrating their covariance matrix into the learning process. The principal eigenvector of this covariance matrix is then utilized to guide the classifier towards the most informative regions of the data, enabling it to capture the local geometry better. Importantly, this modification is confined to the objective function, ensuring the convexity of the optimization problem is preserved. Experimental results across various datasets, including linearly and non-linearly separable ones, demonstrate that the proposed method provides competitive or slightly improved classification performance.

Keywords

Main Subjects

References

[1] M.A. Nanda, K.B. Seminar, D. Nandika, A. Maddu, A comparison study of kernel functions in the support vector machine and its application for termite detection, Inf. 9 (2018) 5.
[2] A. Alhudhaif, A non-linear optimization based robust attribute weighting model for the two-class classification problems, PeerJ Comput. Sci. 9 (2023) e1598.
[3] N. Amaya-Tejera, M. Gamarra, J.I. Velez, E. Zurek, A distance-based kernel for classification via support vector machines , Front. Artif. Intell. 7 (2024) 1287875.
[4] P. Arabie, L. Hubert, G. De Soete, Clustering and Classification, World Science, 1996.
[5] L. Auria, R.A. Moro, Support vector machines (SVM) as a technique for solvency analysis, (2008). [Unpublished/working paper].
[6] C.J. Burges, A tutorial on support vector machine for pattern recognition, Data Min. Knowl. Discov. 2 (1998) 955–974.
[7] L.J. Cao, K.S. Chua, W.K. Chong, H.P. Lee, Q.M. Gu, A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine, Neurocomputing 55 (2003) 321–336.
[8] J. Cervantes, F. Garcia-Lamont, L. Rodriguez-Mazahua, A. Lopez, A comprehensive survey on support vector machine classification: Applications, challenges and trends, Neurocomputing 408 (2020) 189–215.
[9] C.C. Chang, C.J. Lin, LIBSVM: a library for support vector machines, ACM Trans. Intell. Syst. Technol. 2 (2011) 27.
[10] I. Cinar, M. Koklu, Classification of rice varieties using artificial intelligence methods, Int. J. Intell. Syst. Appl. Eng. 7 (2019) 188–194.
[11] C. Cortes, V. Vapnik, Support-vector networks, Mach. Learn. 20 (1995) 273–297.
[12] N. Cristianini, J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press, 2000.
[13] A. Garg, D. Roth, Margin distribution and learning, Proc. 20th Int. Conf. Mach. Learn. (2003) 210–217.
[14] G.H. Golub, C.F. Van Loan, Matrix Computations, Johns Hopkins University Press, Baltimore, 1989.
[15] M. Gonen, E. Alpaydin, Localized multiple kernel learning, Proc. 25th Int. Conf. Mach. Learn. (2008) 352–359.
[16] G. Guo, S.Z. Li, K. Chan, Face recognition by support vector machines, Proc. 4th IEEE Int. Conf. Automat. Face Gesture Recogn. (2000) 196–201.
[17] E. Hafezieh, A. Tavakoli, M. Matinfar, Strategies for disease diagnosis by machine learning techniques, J. Math. Model. 11(3) (2023) 441–450.
[18] M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, B. Scholkopf, Support vector machines, IEEE Intell. Syst. 13 (1998) 18–28.
[19] R. Hu, X. Zhu, Y. Zhu, J. Gan, Robust SVM with adaptive graph learning, World Wide Web 23 (2020) 1945–1968.
[20] I.T. Jolliffe, Principal component analysis for special types of data, Springer, New York, 2002.
[21] S.A. Kasnavi, M. Aminafshar, M.M. Shariati, N.E.J. Kashan, M. Honarvar, The effect of kernel selection on genome wide prediction of discrete traits by support vector machine, Gene Rep. 11 (2018) 279–282.
[22] G. Khyathi, K.P. Indumathi, J. Hasin, S. Siluvai, G. Krishnaprakash, Support vector machines: a literature review on their application in analyzing mass data for public health, Cureus 17 (2025) e77169.
[23] D.P. Kroese, Z. Botev, T. Taimre, R. Vaisman, Data Science and Machine Learning: Mathematical and Statistical Methods, Chapman and Hall/CRC, 2019.
[24] C.F. Lin, S.D. Wang, Fuzzy support vector machines, IEEE Trans. Neural Netw. 13 (2002) 464–471.
[25] A. Mackiewicz, W. Ratajczak, Principal components analysis (PCA), Comput. Geosci. 19 (1993) 303–342.
[26] D.A. Pisner, D.M. Schnyer, Support Vector Machine, in: Machine Learning, Academic Press, 2020.
[27] A. Pourmoezi, M. Eslami, A. Tavakoli, A new insight on the model of support vector machine, Comput. Sci. Eng. 4 (2025) 297–308.
[28] Rice (Cammeo and Osmancik) [Dataset]. (2019). UCI Machine Learning Repository. https://doi.org/10.24432/C5MW4Z.
[29] B. Scholkopf, A. Smola, K.R. Muller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput. 10 (1998) 1299–1319.
[30] B. Scholkopf, A.J. Smola, R.C. Williamson, P.L. Bartlett, New support vector algorithms, Neural Comput. 12 (2000) 1207–1245.
[31] J.A. Sidey-Gibbons, C.J. Sidey-Gibbons, Machine learning in medicine: a practical introduction, BMC Med. Res. Methodol. 19 (2019) 64.
[32] M.H.N. Solhdar, M.J. Solahdar, S. Eskandarix, An intrusion detection system with a parallel multi-layer neural network, J. Math. Model. 9(3) (2021) 437–450.
[33] L. Zhou, S. Pan, J. Wang, A.V. Vasilakos, Machine learning on big data: Opportunities and challenges, Neurocomputing 237 (2017) 350–361.
[34] F. Zhu, N. Ye, W. Yu, S. Xu, G. Li, Boundary detection and sample reduction for one-class support vector machines, Neurocomputing 123 (2014) 166–173.
Journal of Mathematical Modeling
Volume 14, Issue 1
March 2026
Pages 285-304

Files

Share

How to cite

Statistics