Interview with Dr. Makoto Nagao




ATR: You have been involved as an advisory member in the establishment of ATR Interpreting Telephony Research Laboratories and later of ATR Interpreting Telecommunication Research Laboratories. Could you talk briefly about the history of research on machine translation and speech recognition, and about how the research projects at ATR were born?

Dr. Nagao: Research into machine translation and speech recognition has a long history, but both fields were being developed independently until recently. Looking back at the history of each field briefly, I believe that Japanese researchers have contributed a great deal to the development of machine translation. In the early 1980s, Japan was leading the world in machine translation and it still remains a driving force. Translation from Japanese to English and vice versa is very challenging, but the widespread international use of English has given Japan no choice but to grapple with the job of improving its translation technologies. This necessity has caused quick progress in both theory and technology in machine translation in Japan.
In speech recognition, the U.S. has clearly been the leader in developing world-leading technologies. Research in Japan started as early as 1957 or 1958. Kyoto University has been conducting a broad array of studies in the fields of speech and language processing. These and other Japanese research have made great strides in the fields. By the middle of the 1980s both machine translation and speech recognition had reached fairly mature stages. Many researchers began recognizing the benefits of studying both fields together with a view of integrating them into a single system.
During the preparation stage of ATR's establishment around 1984 and 1985, we, the members of the ATR Basic Concept Examination Committee, designed a new research approach, which initiated the integration of the two separate fields into one in order to create a comprehensive speech translation system. In 1986, ATR Interpreting Telephony Research Laboratories was founded in line with this new research approach and offered a very attractive work environment for many researchers. During the 14 years since its establishment, ATR has produced a steady stream of impressive achievements, which convinces me that our original decision was correct.

ATR: Linguistic and cultural barriers are going to be the next challenges that need to be overcome before creating a really effective system integrating speech recognition and machine translation. What are your views on the prospects of solving these problems?

Dr. Nagao: Those are exactly the challenges we have to tackle now. Research so far has been limited to purely literal translation with little attention paid to cultural and other kinds of background knowledge. But that is not enough for good translation of naturally spoken language, which requires reference to the related cultural background and context. We need to focus our effort more on this aspect. Attention needs to be paid to how long sentences are structured and to how the subtleties of human relations are reflected in the words and phrases. Therefore, researchers need to understand the cross-cultural differences in translating languages, such as differences in the situation of Japanese translation and English translation. One major challenge is how to enable a system to factor in the relations between the speakers and the situation when it translates a conversation. The quality of translation is greatly improved if the relationship between the speakers | whether they are teacher and student, friends, man and woman, and so on | is effectively taken into account when doing the translation task. Also important is the choice of the best expressions on a case-by-case basis, which is a subject now eagerly explored at ATR. There are translations that are understandable but quite clumsy or not contextually correct as Japanese or English. Producing good translations demands the delicate task of choosing suitable words and phrases. This Involves the job of amassing a mountain of examples that allow the system to select the best words and phrases according to the context in which the expressions are made. This is venturing into the complexity of semantics and word usage like "pragmatics" far beyond the world of grammar. It is, however, extremely difficult to create general rules and formulas for handling these issues. All too often they must be treated on a case-by-case basis. Consequently, it is necessary to experientially accumulate translation examples in actual tasks, use them for choosing suitable translation, and then continue this process. This machine learning process is identical to human language acquisition in a sense. This kind of research is gradually getting under way at various places, including ATR.

ATR: What is a realistic time frame that we can expect the advanced translation technology to be available for our practical use that can handle cultural differences and context?

Dr. Nagao: It will take several years to compile a good dictionary of translation examples, and it will be about 10 years before the technology becomes commercially practical. It is a good bet, I believe, that a translation system with a new structure that can produce higher-quality translations will be available around 2010.
However, a relatively high level of machine translation technology is already available. Most personal computers today come with a translation system as standard equipment. The quality of translation is not bad as long as you input grammatically correct sentences. In addition, studies on speech recognition are making great headway, bringing a variety of speech technologies into actual use. These developments enable systems to integrate speech and machine translation technologies. Many of the technologies and products are still for professionals, but 10 to 15 years from now, will be widely accessible to anybody.

ATR: You are well aware of the current global position of Japan's basic research in the field of machine translation and speech recognition, including those at ATR. What areas are most in need of improvement?

Dr. Nagao: Japanese researchers are doing cutting-edge research on machine translation and speech recognition. However, if I may say, taking international leadership in the field and contributing to the world with its original ideas are the most important areas where Japan can improve. We need to show more "breakthrough thinking". The overall level of research and technologies are high, but more effort is needed to find new concepts that set trends in research at a global level. Another important factor is to continuously gather basic data. This is a task you must carry regardless of any difficulties such as opposition, or financial and time limitations. An effort should be designed to build up a solid foundation for this kind of research, like the dictionary of translation examples I mentioned earlier.
In terms of taking international leadership, I think ATR's decision to initiate a joint international research project such as C-STAR, deserves high praise. Speaking for myself, I perceived the importance of international cooperation in machine translation more than 10 years ago, so I was involved in organizing machine translation researchers around the world and creating the International Association of Machine Translation. The association has been serving as a forum for discussions and exchanges of information related to the field. We have been holding international conferences every two years, and the seventh meeting was held last year in Singapore. In general, however, Japan has seldom taken world leadership in organizing research efforts in basic research fields. ATR's international initiative and efforts to raise the level of research around the world in the fields of speech recognition and speech translation get a lot of kudos.

ATR: Can you explain more about the need of "breakthrough thinking" and concepts to set research trends throughout the world? What are the requirements for producing such innovative concepts?

Dr. Nagao: Above all, individual researchers need to think in more original and creative ways. One factor hindering this is the traditional tendency toward groupism in Japan. When a Japanese researcher is working in a team, he tends to restrain his individuality, thinking he should put the team's goal ahead of his own and not act too independently. But I strongly believe that it is important to encourage researchers to produce audacious and ambitious ideas, the ones that may sound far-fetched but should come to something in five or 10 years, and then pursue them with a passion. Many Japanese researchers have various brilliant ideas but restrain themselves in a group and away from expressing their ideas, fearing they would be regarded as unrealistic and ridiculed. Under a bushel, they are losing a lot. If this mindset were changed, a lot more unique ideas would emerge.
Besides the importance of individual originality, I believe that Japan needs to establish a more culturally unique research approach as well. Research in the fields of machine translation and speech translation is research into language and speech. A language depends on culture in some respects. The structure of English, for instance, is quite different from that of Japanese.
So simply learning about the achievements of Western researchers and applying their approach to research in Japan often does not work. I believe research into Japanese language processing demands efforts to find the best research method based on the language's unique content and structure and the features of the Japanese linguistic culture. Fortunately, research in Japan is moving in that direction.
Reevaluating current trends in research is also a way to contribute to originality. I noticed that the international trend is toward placing more importance on the number of publications. Under pressure to publish more, many researchers try to produce these as frequently as possible, writing one even about the slightest progress. Some researchers put more emphasis on the quality of papers than on the number. Many western researchers in the language translation fields are more likely to be theory-oriented, showing high interest in concentrating on formulating theories. Many Japanese researchers, in contrast, devote their energy to amassing actual linguistic data and analyzing them I believe such original approaches contribute to advancements in the field.

ATR: So original thinking and meticulous data collection is very important. Let us now turn to the broader picture. What are your views on the needs for international accumulation and sharing of human knowledge?

Dr. Nagao: That is absolutely necessary. There should be more international cooperation on accumulating encyclopedic and other types of knowledge. While each country should keep studying its own cultural and ethnic aspects on its own, international sharing of such knowledge is extremely important. It is, for instance, impossible to really understand the differences between the Japanese language and the English language without studying the two languages together. To develop shared insight into these problems, it is important for countries to compare what they have learned with each other through international cooperation and to accumulate common knowledge by holding forums for joint discussions. This requires various types of joint international research programs.
Accumulating international knowledge of language is a challenge we should continuously strive for. Another and greater challenge, however, is successfully promoting the international cooperation necessary to share knowledge. The chief challenge here is to elucidate the mental process that occurs in the interaction of human communications. This is the knowledge, which is needed to share knowledge. For instance, while I am talking with you now, I am trying to guess how well you are following what I am saying and at the same time proceeding from one thing to another. This is the mental process. There are differences in this process between Japanese and Americans, for instance. Communication between Japanese is based on a large amount of shared information that does not have to be expressed. In this way, they understand each other by adding in their mind the omitted information to what is actually being said. They leave all the shared information | the things they already know | unsaid. More ethnically diverse people, such as Americans, on the other hand, don't think they can omit what they can guess, probably because of their historical and cultural backgrounds. They tend to not take too much for granted the things they can guess, but clarify by saying it in their conversation. Understanding such cultural differences, even subtle ones, becomes a very helpful tool to facilitate the international cooperation for sharing knowledge.
For a computer system for dialogues with a person or for translation, we found that more information needs to be added to what is actually said when translating Japanese speech than in translating American speech. What needs to be added has much to do with person's mentality and mental process. If you listen carefully to an interpreter changing Japanese speech into English, you notice that the interpreter supplements the words with a considerable amount of additional information so that they are really understood. The question is how well a machine can do this. This is a very challenging task. I think collaboration with researchers in the fields of psychology and cognitive science would lead to very interesting research. This is a kind of knowledge which ATR research projects have been searching for to enhance human communications.

ATR: At ATR, brilliant researchers from around the world are currently involved in studies on the speech translation system for translating from Japanese to Chinese, English, Korean, and German. It is working very well. Could you articulate how else we can enhance international cooperation in a concrete manner?

Dr. Nagao: I have a good example. At Kyoto University, an interesting experiment on remote lectures is under way. As professor at the university delivers a lecture in English to students there, the images and sounds are transmitted to UCLA so that students at UCLA can also take the lecture course of Kyoto University. As part of the experiment, pairs of a Kyoto University student and a UCLA student are required to jointly produce reports. They soon start exchanging opinions via the Internet every day. I believe that a small group of people required to work together on the same research subject often builds up ideal cooperation.

ATR: So do you think that practical cooperation on specific tasks works better than an organization-oriented approach?

Dr. Nagao: Both approaches are equally important. A large framework approach to organize international joint research projects and mutual cooperation is, of course, necessary as well. But current Internet technology enables various types of effective cooperation for specific tasks. A research manager's job is to select research topics, decide on what kind of theme should be assigned to each topic, and then encourage individual joint research projects under organized leadership.

ATR: You have also said that collaboration with researchers in such fields as psychology or cognitive sciences would lead to interesting developments. More and more science technologies are approaching natural human communication, including the understanding of mental processes and cultural differences. What kind of Ideal relationship do you think people should develop with science and technology, or machines in the coming years? Also some people argue that machine translation or speech translation systems will never reach human levels, or that the technology to help people improve their skills and abilities is more important than the development of machine translation technology. How would you respond to these?

Dr. Nagao: There are certainly people who argue that communications through a machine are not good. I think this type of argument comes from the mind of "all or nothing". It is impossible to make very delicate, emotionally charged communications through a machine. But such delicate communications constitute just a fraction of our daily communications.
In most cases, simpler communications are sufficient. For business communications in particular, machine translation technology would be adequate with a little more sophistication. There is no doubt about the usefulness of machine translation and speech translation systems in such situations. Less that 10 percent of Japanese people can communicate in English or another foreign language without the help of a translation system. An effective translation system would enable 50 percent of Japanese to manage communication in a foreign language, even though it may not be very sophisticated. Also there are a number of situations where such a system could achieve 80 or 90 percent of the communication objectives. A human interpreter would be needed only if we shoot for a higher level of translation in a certain situation. I never say that we do not need to learn languages, but just stress that better use of machines could help deepen people's general understanding of the world.

ATR: For my last question, could you tell us your views and philosophy on how we should live in this rapidly developing information-based society?

Dr. Nagao: One thing I want to stress is the need for us to come up with ways to avoid getting submerged in an overwhelming sea of information. I feel the strong need to educate people to make selective use of information in this society with its excessive information supply. Man should live by building up an independent and steadfast character, which can not be easily affected by other people's opinions. An excess of information is not a bad thing in itself. The problem is how you make use of it. The value of information heavily depends on the persons who access it. We all must think how to make wise use of information and how to send out our own. We, educators and researchers, should bear this in mind and continue educating young people to acquire such skills and abilities.

ATR: Thank you very much.
(Interviewer: Mayumi Mori-Hubert)