Trends Towards a Ubiquitous Network Society and Expectations for Device Research at ATR

1. Introduction

We conduct a research project to develop new media that supports communication for attaining greater mutual understanding in sharing experiences, in order to bring about a new global and diverse society. Our goal for the project is to invent novel media to record and share with others the experiences that carry valuable information, not limited to experiences of special events such as overseas travel and sports events, but activities from everyday life, from the moment you get up until the moment you turn out the lights. We also pursue media for the automatic recording of a wide range of information sources that goes beyond visual and linguistic information (Fig. 1).

For example, when recording the experience of riding a roller coaster, in addition to the actual images and sounds, we want to record data such as tension, heart rate, perspiration, and brain waves with biometric sensors. If we can communicate to other people this excitement or state of tension by utilizing these data, then there will be a greater potential for communication in which experiences are shared and new experiences are created. Even more, discoveries in learning and the tips that you pick up in your daily life, or even a landscape that calms your heart, will become important components of experience-sharing communication with family and others.
In order to effectively share experiences with others, recorded experiences must include a broad range of knowledge annotated with emotional attributes, as well as extensive information regarding the situated contexts. To achieve this, the first requirement is to develop media that can efficiently record the individual’s actions from various perspectives. We conduct research using a “stuffed animal robot” as media for recording the data of human experience (Fig. 2).

2. Stuffed animals: media for sharing experiences

Diaries and journals are typical examples of a traditional method of experience recording. They were recorded using notebooks, sketchbooks, computers, or other tools that have no direct relationship with the experience itself. In our research, the recording media is a stuffed animal, which can become a virtual partner that shares experiences at the time and location that the events take place.
Our stuffed animal contains a PC module, a video camera, a microphone, accelerometer sensors, pressure sensors, bend sensors, infrared proximity sensors, temperature sensors, and other sensors built into its various body parts. With the system for recording and storing a wide range of events, we are now conducting experiments in which specified events called “episodes” are automatically labeled and extracted during the course of everyday activities.
We believe that the stuffed animal will become one medium that can be easily accepted in human society, because they have a head, hands, and feet, and are soft to touch, thus have been popular among human beings for centuries. Unlike furniture, stationery, or computers, they are kept and placed close to people, so there is greater potential for them to record a wide range of information about human experience, such as body temperature and tense, that could not be obtained by other tools. Furthermore, it has been shown that humans tend to behave toward stuffed animals and other anthropomorphic objects almost as though the objects were actually human. With this type of human behavior, we think the stuffed animal can be used as media to record a person’s contacts with the stuffed animal, such as touching, talking, pulling body parts, and carrying it around. We can also record the surrounding conditions when the stuffed animal sits idle by itself. Two or more stuffed animals via a network are able to communicate various aspects of their respective situations. When one person’s stuffed animal is touched, a second stuffed animal in a remote location would move or talk, indicating the owner’s situation.
If we think of the video camera as one example of more traditional recording media, even though people carry cameras all the time, many of them often end up being frustrated at having missed opportunities for ideal shots. Many think that it is also not interesting that the person taking the films him/herself is rarely in the shots. If a stuffed animal is used to record data, the users do not need to look for such opportunities themselves, but rather can record all the elements of their life in natural and easy ways. Retrieving meaningful information from the video footage is another difficult and frustrating job if the video camera is just operated aimlessly. With our new media, it is possible to retrieve necessary information at any time. We label video data with different categories such as the biological data of the heart rate, as well as the data relating to human interactions with the stuffed animal (squeezing, touching, etc.).
In order to make the best use of the human experience recorded through the stuffed animal, an important research theme will be to find ways of expressing the experiences data. Currently, we test the output methods in which another person can read the recorded data as a kind of “picture diary.”

3. Media in a ubiquitous environment society

The essential issue in a ubiquitous society is to establish new media and mechanisms for safe and easy information exchange.
An example of new forms of media that could accomplish this is sensor-equipped clothing “wearable computers,” as sensors in clothing are effective in recording the movement of the actual individual. Another example is the methods in which sensors are placed throughout a room to record the individual’s actions. Sensors installed in a room are useful in taking records from a distant perspective, because of the physical distance from the person. In comparison, in the case of recording methods that use robots, these robots can act as a kind of partner, sometimes sympathizing with the user’s perspective, and at other times recording experiences objectively from the perspective of a third person, depending on the preference of the user. We believe that this kind of intermediate media is essential to a ubiquitous society. We have focused our attention on robots that can share activities with humans, and are conducting research and development targeting other kinds of robots that can talk of their own accord, like “Robovie” and “Muu.” These robots actively speak to and interact dynamically with humans, while stuffed animals, on the other hand, are characterized by a presence that draws out active behaviors of humans (Fig. 3).
As we have shown here, media for recording and expressing actions that generate sympathy in human beings are essential to research in the shared communication of experiences, and likely to become one of the new important applications that enable richer communications in ubiquitous information environments.
　


Novel Media in a Ubiquitous Network Society Media Information Science Laboratories


	Communications have focused mainly on transmitting the primitive factors of information, such as “what, when, and where,” using telephones, e-mail, and other information technologies. Our vision for the coming ubiquitous information environment society, however, is that it will be possible not only to transmit facts represented with text, images, and sound, but also to record personal experiences along with situations and emotional status, plus share all these with other people. In order to achieve this, novel media will be required to take the place of existing conventional communication media; that is, new media that can record and express feelings and situations that relate to human personal experience.

1. Introduction We conduct a research project to develop new media that supports communication for attaining greater mutual understanding in sharing experiences, in order to bring about a new global and diverse society. Our goal for the project is to invent novel media to record and share with others the experiences that carry valuable information, not limited to experiences of special events such as overseas travel and sports events, but activities from everyday life, from the moment you get up until the moment you turn out the lights. We also pursue media for the automatic recording of a wide range of information sources that goes beyond visual and linguistic information (Fig. 1). For example, when recording the experience of riding a roller coaster, in addition to the actual images and sounds, we want to record data such as tension, heart rate, perspiration, and brain waves with biometric sensors. If we can communicate to other people this excitement or state of tension by utilizing these data, then there will be a greater potential for communication in which experiences are shared and new experiences are created. Even more, discoveries in learning and the tips that you pick up in your daily life, or even a landscape that calms your heart, will become important components of experience-sharing communication with family and others. In order to effectively share experiences with others, recorded experiences must include a broad range of knowledge annotated with emotional attributes, as well as extensive information regarding the situated contexts. To achieve this, the first requirement is to develop media that can efficiently record the individual’s actions from various perspectives. We conduct research using a “stuffed animal robot” as media for recording the data of human experience (Fig. 2). 2. Stuffed animals: media for sharing experiences Diaries and journals are typical examples of a traditional method of experience recording. They were recorded using notebooks, sketchbooks, computers, or other tools that have no direct relationship with the experience itself. In our research, the recording media is a stuffed animal, which can become a virtual partner that shares experiences at the time and location that the events take place. Our stuffed animal contains a PC module, a video camera, a microphone, accelerometer sensors, pressure sensors, bend sensors, infrared proximity sensors, temperature sensors, and other sensors built into its various body parts. With the system for recording and storing a wide range of events, we are now conducting experiments in which specified events called “episodes” are automatically labeled and extracted during the course of everyday activities. We believe that the stuffed animal will become one medium that can be easily accepted in human society, because they have a head, hands, and feet, and are soft to touch, thus have been popular among human beings for centuries. Unlike furniture, stationery, or computers, they are kept and placed close to people, so there is greater potential for them to record a wide range of information about human experience, such as body temperature and tense, that could not be obtained by other tools. Furthermore, it has been shown that humans tend to behave toward stuffed animals and other anthropomorphic objects almost as though the objects were actually human. With this type of human behavior, we think the stuffed animal can be used as media to record a person’s contacts with the stuffed animal, such as touching, talking, pulling body parts, and carrying it around. We can also record the surrounding conditions when the stuffed animal sits idle by itself. Two or more stuffed animals via a network are able to communicate various aspects of their respective situations. When one person’s stuffed animal is touched, a second stuffed animal in a remote location would move or talk, indicating the owner’s situation. If we think of the video camera as one example of more traditional recording media, even though people carry cameras all the time, many of them often end up being frustrated at having missed opportunities for ideal shots. Many think that it is also not interesting that the person taking the films him/herself is rarely in the shots. If a stuffed animal is used to record data, the users do not need to look for such opportunities themselves, but rather can record all the elements of their life in natural and easy ways. Retrieving meaningful information from the video footage is another difficult and frustrating job if the video camera is just operated aimlessly. With our new media, it is possible to retrieve necessary information at any time. We label video data with different categories such as the biological data of the heart rate, as well as the data relating to human interactions with the stuffed animal (squeezing, touching, etc.). In order to make the best use of the human experience recorded through the stuffed animal, an important research theme will be to find ways of expressing the experiences data. Currently, we test the output methods in which another person can read the recorded data as a kind of “picture diary.” 3. Media in a ubiquitous environment society The essential issue in a ubiquitous society is to establish new media and mechanisms for safe and easy information exchange. An example of new forms of media that could accomplish this is sensor-equipped clothing “wearable computers,” as sensors in clothing are effective in recording the movement of the actual individual. Another example is the methods in which sensors are placed throughout a room to record the individual’s actions. Sensors installed in a room are useful in taking records from a distant perspective, because of the physical distance from the person. In comparison, in the case of recording methods that use robots, these robots can act as a kind of partner, sometimes sympathizing with the user’s perspective, and at other times recording experiences objectively from the perspective of a third person, depending on the preference of the user. We believe that this kind of intermediate media is essential to a ubiquitous society. We have focused our attention on robots that can share activities with humans, and are conducting research and development targeting other kinds of robots that can talk of their own accord, like “Robovie” and “Muu.” These robots actively speak to and interact dynamically with humans, while stuffed animals, on the other hand, are characterized by a presence that draws out active behaviors of humans (Fig. 3). As we have shown here, media for recording and expressing actions that generate sympathy in human beings are essential to research in the shared communication of experiences, and likely to become one of the new important applications that enable richer communications in ubiquitous information environments.