Research on Media Integration for the Generation of Communication
Environments
MIYASATO Tsutomu
When people attempt to understand their partners' intentions and emotions during
human-to-human communication, relying solely on the words and visual information
communicated by their partners is insufficient. In long-distance communication,
something that supersedes mere words is obviously required. Just as it is important
to understand the context in order to correctly learn the meanings of words, it
is necessary to have information on the environments surrounding one's partners
and the environments that each has experienced. We carried out research in order
to enable communication through somatic sensations. This effort intended to use
the cumulative experience of the entire body in establishing common ground for
the communication of shared experience.
VR Component Technologies for Audio and Visual Sensations
We proposed and prototyped a 3-D Head Mounted Display with motion parallax as
a natural 3-D display method that is comfortable for the eyes. We performed physiological
and psychological evaluations that confirmed a reduction in fatigue as compared
with the use of conventional head mounted displays.
We compared the relative importance of facial image information and sound information
in the recognition of different emotions. We clarified that all emotions are influenced
significantly by one of these two factors.
We also developed a virtual object with facial expressions. This object expresses
its emotions through facial images in response to various user actions. As a result,
the object can be used to teach a user to master ways of behaving in various situations.
To create a stronger sense of the presence of an audience within a virtual space,
we developed a system in which the applause sound generated by a user is classified
and the same type of applause is created by a generated virtual audience.
VR Component Technologies for the Sensation of Locomotion
The sensation of walking plays a very important role together with the sense of
sight in helping us to intuitively sense the physical dimensions of a space. We
established ground surface generation technologies that mechanically alter the
smoothness and roughness of a walking surface experienced by a user walking on
a virtual road. With an image analysis and a walking speed estimation technique,
we prototyped a treadmill-type walking road simulator with a display that moves
synchronously with the free walking speed of the user. To mimic the floor profile
of a remote location, we also developed a device capable of generating, recreating,
and freely changing the conditions of a walking surface in a remote location.
A VR System Emphasizing Physical Sensations
The walking system was employed to create a remote sports telecasting
system that allows the user to feel the same sensations as an athlete. The
system uses a movie recorded from the viewpoint of a marathon runner and seeks
to recreate the runner's mental and physical sensations. It adjusts the treadmill
speed according to the pre-measured physical strength of the user. We take users
out of their armchairs and place them on the field. By presenting the breathing
sounds and foot sounds of pursuing runners, the system creates a strong sense
of presence. The results of our experiments with subjects indicate that we succeeded
in recreating the marathon experience with a strong sensation of immersion not
previously available from television entertainment.
Reference
