Classification of heart disease with machine learning: a comparison of grid search, random search, and Bayesian Optimization

Andi, Tri and Ismail, Amelia Ritahani and Pranolo, Andri and Kusuma, Candra Juni Cahyo (2026) Classification of heart disease with machine learning: a comparison of grid search, random search, and Bayesian Optimization. International Journal on Informatics Visualization, 10 (1). pp. 262-270. ISSN 2549-9610 E-ISSN 2549-9904

Preview

PDF - Published Version Download (3MB) | Preview

Abstract

This study systematically evaluates the effect of hyperparameter optimization on the performance of predictive models by comparing three main techniques: Grid Search, Random Search, and Bayesian Optimization. Four commonly used machine learning algorithms: Logistic Regression (LR), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Gradient Boosting were tested on benchmark datasets from the UC Machine Learning Repository. The results of the study show that hyperparameter optimization significantly improves prediction accuracy compared to baseline models, with the optimal method varying across algorithms. Specifically, Random Search achieved the highest accuracy of 0.883333 for Logistic Regression and 0.833333 for Gradient Boosting, while Bayesian Optimization demonstrated superior performance on SVM with an accuracy of 0.883333. Grid Search proved most effective for KNN, achieving an accuracy of 0.866667. A comprehensive analysis using additional metrics such as precision and recall reinforces these findings, showing that accuracy improvements do not come at the expense of other performance metrics. Bayesian Optimization stands out for computational efficiency, especially for complex models, though Grid Search remains relevant for limited hyperparameter spaces. The main contribution of this research is to provide practical guidance for machine learning practitioners on selecting optimization techniques that align with algorithm characteristics. This study also analyses the trade-off between search thoroughness and computational resources required, and provides recommendations for balancing accuracy and efficiency in model development. Although the research results are promising, several limitations need to be acknowledged, including reliance on a single dataset and the need for further validation across multiple domains. Future research could explore hybrid optimization approaches that combine the strengths of various methods, develop adaptive strategies for ensemble models, or conduct more comprehensive evaluations across diverse datasets. By linking theoretical findings to practical applications, this research provides a valuable framework for efficiently optimizing machine learning models

Item Type:	Article (Journal)
Uncontrolled Keywords:	Hyperparameter optimization, logistic regression, support vector machine, K-Nearest Neighbors, gradient boosting, grid search, random search, bayesian optimization, machine learning, model performance
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science
Depositing User:	Amelia Ritahani Ismail
Date Deposited:	20 Feb 2026 15:36
Last Modified:	20 Feb 2026 15:36
Queue Number:	2026-02-Q2208
URI:	http://irep.iium.edu.my/id/eprint/127509

Actions (login required)

View Item