Clustering improves support vector machines classification performance even without the kernel trick

Abstract

With suitable kernels and parameters, support vector machines (SVMs) usually outperform other classification methods in terms of accuracy and generalization ability. However, finding appropriate kernels and hyperparameters is time-consuming, especially for large-scale classification tasks. In this paper, we demonstrate that clustering—when designed with classification in mind—can effectively replace kernelization. We propose a novel class-aware clustering method, SEKmeans, which jointly optimizes intra-cluster compactness, cluster purity, and intra-cluster linear separability within a unified objective. This design explicitly facilitates downstream linear classification and enables linear classifiers to capture nonlinear patterns without using any kernel function.We conduct extensive comparisons with linear SVM, linear Twin-SVM (TWSVM), kernel SVM, kernel Twin-SVM, the clustering-based CSVM, and two K-means-based counterparts of SVM and TWSVM, namely K_SVM and K_TWSVM. Experimental results show that our SEKmeans-based variants of SVM and TWSVM(SEK_SVM and SEK_TWSVM) achieve competitive accuracy and F1-scores comparable to those of kernel-based methods, while requiring substantially less computational time.