Human Communication Science:
A Step Toward Social Agents
KATAGIRI Yasuhiro
What is a "Social Agent"?
Advances in media technology have opened up the rich possibilities of applying
nonverbal methods in human-computer interactions. Computers provided with their
own faces, bodies and voices can change the way people deal with them. Not only
do such computers function to obtain information from and provide it to humans,
they can also be treated as partners both in the office and at home. People may
even attribute pseudo-sentience to such computers and afford them peer status
in society. We have proposed the concept of "social agents" to capture and explore
the new relationship between humans and computers.
Accuracy and efficiency of information transmission have been the primary objectives
in the design of language-based human-computer interactions. Emotional satisfaction,
social affinity and trust become no less important with social agents. We have
pursued the following three research topics in order to elucidate conceptual elements
of, and explore supporting technologies for, social agents: construction of social
communication with artificial agents; analysis of multi-modal information exchange
in human face-to-face natural communication; and socio-psychological factors in
human-computer interactions.
Construction of Social Communication
People often talk to each other for the sheer enjoyment of conversation. How can
we reproduce the joy of casual chatting in interactions with computers? We have
developed a series of artificial agents to investigate this question. "Talking
Eye" is an interactive computer graphics agent developed on top of the Ecological
Architecture1
. It can engage in artificial conversations with people and among others of its
type. "MiMIC" responds by imitating the prosodic contours of its partner's speech.
It was developed to demonstrate the power of mimicry in social communication.
"Muu" is an interactive and sociable toy robot with its own visual, auditory and
tactile sensors and actuators to orient itself toward particular objects2
. "Robovie" is a human-size robot that interacts with people by means of gestures,
eye contacts and speech3
. Through the development of these artificial communication agents, we discovered
that the exchange of nonverbal information plays an essential role in producing
enjoyable communication.
Analysis of Multi-modal Information Exchange
To ensure a natural and smooth flow of conversation in daily face-to-face communication,
people exchange a vast amount of nonverbal information, including tone of voice,
direction of eye gaze, and gestures. To elucidate the structures underlying this
multitude of information exchange in conversation, we conducted a fine-grained
analysis of naturally occurring conversations, focusing on the interplay between
speech prosody and dialogue expressions. We have found that the coordination of
temporal features such as speech rate and response timing plays an important role
in the natural flow of conversation4
. Thanks to advanced speech technologies, we have also developed a combination
of corpus-based statistical data analysis and experimental analysis with the controlled
manipulation of speech, which provided us with a novel method of obtaining precise
findings without losing the naturalness of conversation.
Pictures and drawings are often much more effective at communicating than language.
Why and how is it that a visual representation often surpasses a linguistic representation?
How is it possible to effectively support visual communication in a network environment?
We analyzed spontaneous drawings produced on a shared electronic whiteboard in
collaborative problem-solving situations. We found that the immediate feedback
provided by instantly produced drawings contributes to smooth graphical exchange.
We have also identified a sequential style of integration of linguistic and visual
information in graphical communication.
Socio-psychological Factors
Management of social emotions is an essential component of social agents. We have
demonstrated, through a series of psychological experiments, that people do experience
social emotions, such as reciprocity, politeness and affinity, when using computers.
People treat computers, although unconsciously, as though they were human. We
also conducted a cross-cultural study in the U.S.A. and Japan that revealed evidence
of cultural differences in the style of human-like responses to computers5
. A real-world study on the effects of social emotions among those using computers
was performed at the ATR open house in 1999. We developed an agent-based exhibit
recommendation system and demonstrated that establishment of an affinitive relationship
with agents can lead to social persuasion and can influence the behavior of visitors6
.
Whither from Here?
We have successfully developed and established a conception of social communication.
An array of nonverbal cues exchanged in communication, be they between people
or between people and computers, function essentially as social signals. They
help to coordinate the smooth flow of communication, and hence to establish and
manage social relationships, which in turn leads to social persuasion.
A vast research effort needs to be expended to elucidate individual social signals,
their classifications, signaling mechanisms and multimodal integration processes.
The outcome of this research should contribute to the development of technologies
for social agents that maintain a good balance between attribution and design
of human-like qualities for computers. The recognition and elucidation of the
risks of persuasion technologies that appeal to social emotions are also important
research areas that can help to prevent the exploitation of these potentially
powerful and beneficial technologies.
Reference
