Supervised Learning for Classification in Data Science: A Comparative Perspective

Abstract

The processing of big data has become a top priority, especially with the emergence of machine learning algorithms. Standard models suffer from several limitations and lack robustness. This paper studies four machine learning models: SVM, XGBoost, Random Forest, and logistic regression, applied to the Titanic dataset for binary classification. Model performance is evaluated using the following metrics: precision, recall, accuracy, F1-score, and area under the ROC curve (AUC), as well as through confusion matrix analysis. We applied a 5-fold cross-validation method, and statistical significance was assessed using the Wilcoxon test. Based on the analysis of the results, logistic regression achieved the best performance on the test set in terms of precision and AUC of the ROC curve, while Random Forest and XGBoost provided competitive and stable results across the validation folds. SVM also performed well, but it was slightly less effective. The Wilcoxon test shows no statistically significant difference between the models (p > 0.05). Examination of the confusion matrix reveals a tendency for ensemble methods to minimize false negatives. Overall, logistic regression demonstrates strong and good generalization ability, while ensemble methods constitute robust alternatives for binary classification on structured data.