Masa-aki SATO and Shin ISHII
On-line EM Algorithm for the Normalized Gaussian Network
Abstract:Normalized Gaussian Network (NGnet) (Moody and Darken 1989) is a network of local linear regression units. The model softly partitions the input space by normalized Gaussian functions and each local unit linearly approximates the output within the partition.
In this article, we propose a new on-line EM algorithm for the NGnet, which is derived from the batch EM algorithm (Xu, Jordan and Hinton 1995) by introducing a discount factor. We show that the on-line EM algorithm is equivalent to the batch EM algorithm if a specific scheduling of the discount factor is employed. In addition, we show that the on-line EM algorithm can be considered as a stochastic approximation method to find the maximum likelihood estimator. A new regularization method is proposed in order to deal with a singular input distribution. In order to manage dynamic environments, where the input-output distribution of data changes with time, unit manipulation mechanisms such as unit production, unit deletion, and unit division are also introduced based on the probabilistic interpretation.
Experimental results show that our approach is suitable for function approximation problems in dynamical environments. We also applied our on-line EM algorithm to a reinforcement learning problem. It is shown that the NGnet, when using the on-line EM algorithm, learns the value function much faster than the method based on the gradient descent algorithm.