Performance Enhancement on Classification of Imbalanced Data using eXtreme Gradient Boosting (XGBoost)

Abstract

In real world applications, data are generated with uneven distribution called imbalance data which consists of majority and minority classes. For imbalanced data most of the classifier is biased towards the majority class. This means the classifier provides good accuracy for the majority class but very poor accuracy for the minority class. There have been many attempts at dealing with classification of imbalanced datasets. The common practice to handle the problem is to re-balance the data by random under-sampling and random over-sampling. Many papers have applied the Synthetic Minority Over-sampling Technique (SMOTE) to avoid overfitting or over-sampling. A cost-sensitive learning approach provides the solution by adjusting the costs of various classes. The algorithmic approach includes different ensemble algorithms. Among them, XGBoost is the powerful gradient boosting method which is designed for speed and performance. This thesis focus on enhancing the performance of XGBoost for the classification of imbalanced data by performing k-fold cross validation and hyper- parameters tuning. Hyperparameters tuning is the optimization of the classifier that maximizes the performance of the model. XGBoost consists of many hyperparameters. For the tuning, the best hyperparameters will be selected by grid search technique. The performance of the XGBoost was evaluated by calculating the evaluation metrics: accuracy, recall, precision and f-measure with the help of a confusion matrix. Before the hyperparameter tuning the performance evaluation of accuracy, recall, precision and f-measure was obtained as 86.98%, 45.49%, 65.85% and 53.81% respectively. After the hyperparameter tuning the performance was enhanced as 89.46%, 55.47%, 74.75% and 63.69% respectively. Further the 10-fold cross validation enhanced the accuracy to 90.308%.