TR-I-0052 :1988. 11

中村雅己,鹿野清宏

ニューラルネットによる英文単語列予測モデルの検討

Abstract:Using traditional statistical approaches, it is difficult to make an N-gram word prediction model to construct an accurate word recognition system because of the increased demand for sample data and parameters to memorize probabilities. To solve this problem, NETgrams, which are neural networks for N-gram word category prediction in text, are proposed. NETgrams are constructed by a trained Bigram network with two hidden layers. Each hidden layer learns the coarse-coded Micro Features (MF1 or MF2) of the input or output word category. NETgrams can easily be expanded from Bigram to N-gram networks without explosively increasing the number of free parameters. NETgrams are tested by training experiments with a Brown Corpus English Text Database. The training method is the Back-Propagation algorithm. After training, the Trigram word category prediction rates for test data show that the NETgrams are comparable to the statistical model and compress information more than 130 times. Results of analyzing the hidden layer (Micro Features) show that the word categories are classified into some linguistically significant groups. We are now training the 4-gram networks and obtaining good results. In addition, this paper proposes a new method to speed up the Back-Propagation algorithm, which dynamically controls the training parameters, updating step size and momentum. This new method can automatically determine better parameters and achieve a shorter training time.