Loading...
Determine the irrelevance of hidden unit from the validation set
Author(s)
Pinngern, Ouen
Date Issued
December 1, 2006
Type
Conference Paper
Abstract
This paper proposes a method to determine the irrelevance of the hidden unit in the artificial neural network. Unlike other approaches, we calculate the sensitivity of the hidden unit from the validation set, instead of the training set. The advantage of using the validation set to calculate the sensitivity is that we never overestimate the relevance of hidden unit. In other words, we always remove the unit that has the least effect on the validation set error. As a result, the pruned neural network has the highest generalization when compared with other choices of removals. Our sensitivity is based on the activation difference of the output unit. This activation difference is the gap between the activation of output units when a particular hidden unit is present and when it is removed. We have applied our technique to two standard benchmark problems. The experimental results show that the proposed technique can correctly determine the least irrelevant hidden unit. © 2006 IEEE.
Citation
2006 International Symposium on Communications and Information Technologies Iscit, 831-834, 2006