Image recognition using multi-layer sparse feature extraction with ADMM

Abstract

Being motivated by the Saak (Subspace approximation with augmented kernels) transform, we propose an image recognition scheme using multi-layer sparse feature extraction with a convex solver ADMM (Alternating Direction Method of Multipliers). The Saak transform consists of a multi-layer PCA (Principal Component Analysis) and S/P (Sign-to-Position) conversion to avoid sign confusion. This paper adopts sparse representation instead of PCA and also compares the S/P conversion with the activation function ReLU (Rectified Linear Unit), which is realized by involving the projection mapping onto the non-negative set in convex formulas. The Saak transform uses PCA not only for feature extraction but also for dimension compression of feature vectors. We expect that our method does not need the dimension compression since sparse representation compresses features more than PCA. Experimental results on the MNIST and Fashion-MNIST dataset show that the proposed method is equivalent to the Saak transform in recognition accuracy, and that our method can make features more sparse and extract features that have high discriminant power locally.