Researchers have long regarded model accuracy as the primary metric for evaluating the performance of classification algorithms. The current evaluation approach, which relies solely on model accuracy, often leads to inappropriate evaluation of classifiers, regardless of the dataset's separability and complexity. This limitation underscores the need for a new and more comprehensive method. We argue that accuracy-based evaluation can be misleading, even when considering measures of data separability and complexity. We compare the error rates of well-known classifiers on Gaussian-generated datasets and show that, paradoxically, many algorithms' observed errors are lower than that of the theoretical optimal classifier, leading to an overestimation of their performance. We consider a model invalid if its error rate is lower than the optimal classifier error, known as the Bayes error rate. To identify such invalid models, we introduce a procedure and propose an algorithm for model validation based on the Bayes error rate.
Kazemzadeh Gharechopogh, H. and Mohammadpour, A. (2025). Is evaluation based on accuracy of classification algorithms misleading? An approach to model validation using Bayes error rate. Journal of Mathematical Modeling, 13(3), 1-11. doi: 10.22124/jmm.2025.28746.2649
MLA
Kazemzadeh Gharechopogh, H. , and Mohammadpour, A. . "Is evaluation based on accuracy of classification algorithms misleading? An approach to model validation using Bayes error rate", Journal of Mathematical Modeling, 13, 3, 2025, 1-11. doi: 10.22124/jmm.2025.28746.2649
HARVARD
Kazemzadeh Gharechopogh, H., Mohammadpour, A. (2025). 'Is evaluation based on accuracy of classification algorithms misleading? An approach to model validation using Bayes error rate', Journal of Mathematical Modeling, 13(3), pp. 1-11. doi: 10.22124/jmm.2025.28746.2649
CHICAGO
H. Kazemzadeh Gharechopogh and A. Mohammadpour, "Is evaluation based on accuracy of classification algorithms misleading? An approach to model validation using Bayes error rate," Journal of Mathematical Modeling, 13 3 (2025): 1-11, doi: 10.22124/jmm.2025.28746.2649
VANCOUVER
Kazemzadeh Gharechopogh, H., Mohammadpour, A. Is evaluation based on accuracy of classification algorithms misleading? An approach to model validation using Bayes error rate. Journal of Mathematical Modeling, 2025; 13(3): 1-11. doi: 10.22124/jmm.2025.28746.2649