Support Vector Machine accuracy improvement with k-means clustering

Abstract

Support Vector Machine (SVM) is a classifier tool which, originally, uses a hyperplane as a border for separating two classes of data in hyperspace. However, if data from each class are not clustered together, the two classes might not be linearly separable. Typically, researchers attempted to resolve this issue by replacing the hyperplane with a complex border via kernel tricks. However, these kernel tricks could result in a longer training time or only a minute accuracy improvement (or both). On the other hand, if data from one class are separated into subclasses according to their proximity, then all the subclasses should be easily separated by hyperplanes. Therefore, this paper proposes a method to improve the accuracy of linear SVM by first applying k means clustering to each class of input data. Then, after clustered, a multi-classes linear SVM is trained using each subclass as a separate class. Thus, the trained SVM can identify any new input into a subclass which can be easily mapped to the correct class. To evaluate, the proposed method is experimentally used to classify images of Thai character where multiple fonts of characters can be taken as hidden clusters within classes. Empirically, the proposed method could achieve over 6% improvement from a linear SVM or SVMs with RBF or polynomial kernel. © 2013 IEEE.