ATR Interpreting Telecommunications Research Labs

Natural Speech Recognition




Yoshinori Sagisaka


This report gives an overview of research on robust speech recognition designed to build a system that can handle fluent or colloquial pronunciations, different acoustic characteristics among various speakers, flexibility in word order, and the use of interjections observed in natural speech. Constituent technologies needed to build such a system have been developed, including feature extraction and its parameterization, acoustic model training methods, speaker adaptation/speaker-independent models, and statistical modeling for linguistic constraints and effective search of word candidates. These technologies have been integrated into a speech recognition toolkit called ATR SPREC and trained with large-scale speech corpora.


1. Feature Parameter Representation for Speech Recognition

Acoustic parameters for speech coding have traditionally been used as feature parameters for speech recognition. However, they are not necessarily the best parameters to use for discriminating each phoneme in speech recognition. ATR-ITL members, in cooperation with ATR Human Information Processing Research Laboratories, have devised a method to design acoustic parameters that directly minimize discrimination errors as well as parameters that reflect perceptual masking characteristics. Moreover, a subband-spectrum centroid parameter has been proposed as a robust recognition feature representation.


2. Acoustic Modeling and Pronunciation Modeling

An efficient training algorithm for Hidden Markov Models (HMM) based on the maximum-likelihood criterion (ML-SSS) is proposed. To feed back recognition error characteristics to model building, pronunciation variations of HMM models are derived from errors over phoneme sequences, and the distributions of HMM model parameters are retrained to avoid local errors. As a pronunciation model, a statistical training method was proposed to absorb acoustic variations at the phonetic symbol level.


3. Speaker Adaptation/Speaker-Independent Model

One of the major challenges in speech recognition as a form of pattern recognition is how to deal with diverse speech features due to differences among speakers in age, sex, and birthplace. To reduce pattern-matching errors, input speech is used to calibrate HMM models. For this purpose, we integrated Vector fields Smoothing (VFS) for the interpolation and smoothing of small quantities of training data, and Maximum A Priori (MAP) for the efficient use of large-capacity speaker-independent data into the MAP-VFS speaker adaptation method based on production models employing very small amounts of data as well as a dynamic speaker adaptation model that can be progressively trained to the speaker and gradually brought closer to a model trained by using all data at one time. In addition to the speaker adaptation method, a speaker weight training method that enables training with a small number of samples, and speaker-independent model using a tree-structured speaker clustering, are proposed.
　 All of these acoustic models are run simultaneously in the speech recognition process so that the speaker model with the closest acoustic characteristics can be selected. This has enabled the automatic recognition of voice quality, including the distinction of male and female voices for speech output.


4. Effective Search of Word Candidates Using Statistical Language Constraints

Speech recognition is a search problem, and so the use of language information constraints is essential for identifying speech content. The word N-gram is used as a linguistic constraint that permits flexible expressions in words. A wide variety of devices are used in N-gram modeling: variable order N-grams for memory reduction, multi-class composite N-grams based on the connection direction for an inflectional language like Japanese, MAP interpolation between word and part-of-speech characteristics for data sparseness, MAP adaptation for porting to different tasks, and statistical phonotactics modeling for unregistered personal names. For efficient word candidate searches, multi-pass time-synchronous beam search was proposed to output word graphs and a new stack decoder was designed to reduce memory requirements. To output recognition results that are acceptable for the succeeding translation process, a finite-state transducer was built to cope with insertion and deletion errors. Moreover, speech understanding capabilities have been realized by using HMM models for topic detection, and decision trees have been made to identify the intermediate linguistic expressions that are designed for semantic interpretation.


5. Systematization of Speech Recognition, Natural Speech Data Collection, and Performance Evaluation

A modular-type system architecture was designed to replace constituent modules for quick response to advances in research, on-demand use of arbitrary data (e.g. prosody parameters), and efficient data transmission. Under this design, software modules of constituent technologies produced by the research were integrated into the ATR SPREC toolkit for speech recognition research. For a quantitative evaluation of the recognition performance, enormous databases were constructed from spontaneous conversations related to travel arrangement and from Japanese speaker-independent speech data provided by many speakers of all ages from across the nation. A series of evaluative tests using these databases, conducted about once every three or four months for three years, showed that recognition errors were reduced by 70 percent from the initial level. The final word recognition accuracy for the 27.000-word travel-specific dialogue database reached 87.7 percent.