Jearanaitanakij, Kietikul
Loading...
Preferred name
Jearanaitanakij, Kietikul
Main Affiliation
Email
kietikul.je@kmitl.ac.th
2 results
Now showing 1 - 2 of 2
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Spartan simplicity: A pruning algorithm for neural nets(2008-08-01); Pinngern, OuenHaving more hidden units than necessary can produce a neural network that has a poor generalization. This paper proposes a new algorithm for pruning unnecessary hidden units away from the single-hidden layer feedforward neural networks, resulting in a Spartan network. Our approach is simple and easy to implement, yet produces a very good result. The idea is to train the network until it begins to lose its generalization. Then the algorithm measures the sensitivity and automatically prunes away the most irrelevant unit. We define this sensitivity as the absolute difference between the desirable output and the output of the pruned network. Unlike other pruning methods, our algorithm is distinct in calculating the sensitivity from the validation set, instead of the training set, without increasing the asymptotic time complexity of the back-propagation algorithm. In addition, for a classification problem, we raise a point that the sensitivities of some well-known pruning algorithms may still underestimate the irrelevance of hidden unit even though the validation set is used in measuring the sensitivity. We resolve this problem by considering the number of misclassified patterns as the main concern. The Spartan simplicity algorithm is applied to three artificial and seven standard benchmarks. In most problems, the algorithm can produce a compact-sized network with high generalization ability in comparison with other pruning algorithms. © 2008 World Scientific Publishing Company. - Some of the metrics are blocked by yourconsent settings
Item type:Publication, An information gain technique for acceleration of convergence of artificial neural networks(2005-12-01); Pinngern, OuenThis paper presents an application of information gain to accelerate the convergence time of Artificial Neural Networks (ANNs). We improve Hagiwara's convergence acceleration algorithm by applying information gain to it. The first step of our proposed technique is to calculate information gains of all features (or attributes) in training data and pass those gains through all hidden units in the next layer. During the training process, the algorithm monitors sum-squared error at the output layer. When the variation of sum-squared error becomes small, the worst hidden unit is detected. Next, all the weights connected to the worst hidden unit are reset to random values within the appropriate ranges. These ranges are determined by the propagated information gain of the worst hidden unit. Then, the network is retrained. When the number of weight resetting trials reaches a certain number, a new hidden unit is added to the network and the whole training process is repeated. Our experimental results on standard benchmarks show remarkable outputs in terms of convergence time. © 2005 IEEE.