Toward the Creation of New Communications Incorporating Multimedia
Technologies
NAKATSU Ryohei
ATR Media Integration & Communications Research Laboratories (ATR-MIC) was established
in 1995 as a seven-year project to develop technologies, systems, and services
for use in future communications. During this period, ATR-MIC has succeeded in
developing a variety of basic technologies for future communications as well as
numerous advanced communications systems utilizing these technologies. This article
provides a summary of its aims and scope as well as an evaluation of the project.
This project has two basic objectives. The first is to realize the same level
of natural and realistic communication between persons in distant locations as
if they were talking face-to-face. This objective has long been a goal of telecommunications
research. The second objective is to strive to develop innovative communication
methods and methodologies that go beyond the concept of realistic communication.
Conventional media, such as novels and movies, can create in our minds "cyberspaces"
that seem surprisingly realistic even though they do not exist in the physical
sense. We believe that innovative technologies could be developed to create "hyper-realistic"
spaces that appear more realistic than is possible with conventional media.
To achieve the above objectives, we adopted two basic approaches. The first is
to focus on the treatment of nonverbal information in communication. Although
it is well known that nonverbal information plays a very important role in our
communication, it has been neglected in the engineering arena, mainly because
of the difficulty of manipulating it. In our project we conducted systematic research
on nonverbal communication. The second approach is to carry out interdisciplinary
research to address the issue of future communications. Although the importance
of interdisciplinary research has recently become well recognized, our approach
is somewhat special in the sense that we carried out collaborative research between
engineers and artists. We call this approach "Art & Technology," and we adopted
it for several reasons, including the lack of a methodology for handling nonverbal
information. This is especially true for the various forms of nonverbal self-expression
such as facial expressions, gestures, the painting of pictures and the playing
of music. We do not have the proper methodology for such expression, so we must
rely on intuition. In this regard, artists have the skills to express themselves
nonverbally. In addition, although we want to transcend realistic communication
in its physical sense, few engineering methodologies can transcend physical reality.
Here too, artists excel at creating "cyber worlds" that are not real but appear
very realistic to our minds.
We have identified the following main topics as research themes.
1. Technology for Creating a Communications Environment
Technology for creating a communications environment seeks methods of producing
a communications space with the illusion of presence, or creating a communications
space that transcends reality. Because visual information plays the most important
role in our interactions with our environment, the study of various technologies
is necessary to allow us to recognize and generate a visual environment. Another
key element in the creation of the illusion of a communications presence is the
ability to share tactile sensations. We therefore studied the generation of tactile
sensations such as force feedback and walking.
2. Agent Interface Technology
Agent interface technology aims to promote mutual understanding in human communication
through the medium of the computer. The computer produces a virtual human being
(an "agent") to act as an intermediary in human communication. Toward this end,
agents must be developed that appear human, understand human conversations, and
enliven human communication.
3. Image Representation Technology
Image representation technology aims to represent one's thoughts through video
and sound and convey them directly to others without the medium of language. In
this endeavor, it is first necessary to compile a database of various images and
sounds that represent thoughts. A method must then be developed to represent the
mind's images by transforming and synthesizing the video images and sounds included
in the database.
4. Human Communication Science
Before proceeding with the research explained above, one must understand how people
communicate. Observations of person-to-person communication and human-computer
interaction are contributing to the development of a model of basic human communication
behavior that could be applicable to the above research.
To assist with visualization of the above concept, we established three conceptual
systems as research targets. The first is an advanced telecommunications system
with the ability to create the illusion of presence by creating both realistic
and hyper-realistic environments. The second is an agent meeting system in which
the agent supports a teleconferencing system such as that described above. The
third is an image communications system that enables people to express their thoughts
through images and sounds rather than language, thus supporting nonverbal communication
among participants. On this basis, we encouraged researchers to develop a variety
of demonstration systems, exhibit them on numerous occasions, note the reactions
they elicited, and use these reactions as feedback for suggesting the next stage
of development.
Details of the technologies and systems used in each of the above areas will be
described later. The following is a general evaluation of our research project,
especially our basic approaches.
Regarding the manipulation of nonverbal information, while varied research has
been carried out in this area, most of it has been conducted in isolation rather
than systematically. Consequently, our research has attracted great interest from
researchers worldwide, as seen by the number of keynote speeches and talks we
have been invited to give. Moreover, kansei, or sensitivity, has become a very
common aspect of nonverbal information processing even in international conferences.
Kansei has been adopted as a technical term and is often used in the communications
and human interface research communities.
The "Art & Technology" approach is another major issue. Although the tendency
to introduce technology in the realm of art is growing, the engineering side has
seldom initiated collaborations with artists. From such collaborations between
artists and engineers we have been able to develop a variety of innovative communications
systems that were exhibited at various technology exhibits, attracting much interest.
Because these systems can also be interpreted as new interactive art forms, many
have been exhibited and awarded at world-renowned art exhibitions. Our innovations
have attracted great interest from both the art and engineering worlds. Inspired
by our activities, several new movements have recently materialized around the
world. For example, new university departments and new academic societies focused
on the interdisciplinary margins of art and technology are being established.
In the art world, museums have been established that focus on new forms of art
that utilize technology. As a pioneer in this field, ATR-MIC has become well known
globally in the twin domains of art and engineering.
We are clearly proud that our project has initiated numerous innovative research
trials that fortunately have been successful and have become known worldwide.
We hope that these positive evaluations will now launch new projects that continue
these efforts in ATR and elsewhere.
