The reduction of gradient noise in gradient-based algorithm by using variable step-size technique

Punchalard, Rachu; Chumchu, Prawit; Benjangkaprasert, Chawalit; Anantrasirichai, Noppin; Sangaroon, Ornlarp; Jenjerapongvej, Kanok

The reduction of gradient noise in gradient-based algorithm by using variable step-size technique

Author(s)

Punchalard, Rachu

Chumchu, Prawit

Benjangkaprasert, Chawalit

Anantrasirichai, Noppin

Sangaroon, Ornlarp

Jenjerapongvej, Kanok

Date Issued

December 1, 2000

Type

Conference Paper

Abstract

This paper proposes a different approach to control step-size adaptation. The objective is to ensure large step-size when the algorithm is far from the optimum with step-size decreasing as we approach the optimum. The proposed algorithm achieves this objective by employing the output signal energy to control step-size update. In the early stage of adaptation the estimate of the output signal energy is large, resulting in a large step-size. As we approach the optimal, the output signal energy approaches zero, resulting in a small step-size. This provides the fast convergence due to large initial step-size while ensuring low gradient noise due to the small final step-size, resulting in low variance of the filter coefficient.

Citation

IEEE Asia Pacific Conference on Circuits and Systems Proceedings, 415-418, 2000