A comparison between shallow and deep architecture classifiers on small dataset

Many machine learning algorithms have been introduced to solve different types of problem. Recently, many of these algorithms have been applied to deep architecture model and showed very impressive performance. In general, deep architecture model suffers from over-fitting problem when there is a small number of training data. In this paper, we attempted to remedy this problem in deep architecture with regularization techniques including overlap pooling, flipped-image augmentation and dropout, and we also compared a deep structure model (convolutional neural network (CNN)) with shallow structure models (support vector machine and artificial neural network with one hidden layer) on a small dataset. It was statistically confirmed that the shallow models achieved better performance than the deep model that did not use a regularization technique. However, a deep model augmented with a regularization technique-CNN with dropout technique-was competitive to the shallow models.