Loading...
An extreme learning machine based pretraining method for multi-layer neural networks
Author(s)
Noinongyao, Pavit
Watchareeruetai, Ukrit
Date Issued
July 2, 2018
Type
Conference Paper
Abstract
One approach in training a deep neural network to perform effectively is to do unsupervised pretraining on each layer, followed by fine-tuning the whole network. A common way is to train an unsupervised model of neural network such as restricted Boltzmann machines or autoencoders and stack them on top of another. Although these unsupervised pretraining approaches yield good performance, relying on back-propagation, due to iterative learning process, they still suffer from a long pretraining time. Extreme learning machine (ELM) is an analytical training approach which is extremely fast and gives a solution with a good generalization performance. In this paper, we apply a new ELM based unsupervised learning, named backward ELM based autoencoder (BELM-AE), to pretrain each layer of a neural network before using a back-propagation based learning algorithm to fine-tune the whole network. Experimental results show that the new pretraining method requires significantly shorter training time and also yields better testing performance on various datasets.
Citation
Proceedings 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems and 19th International Symposium on Advanced Intelligent Systems Scis Isis 2018, 608-613, 2018