Research to Support Image Expression:

Toward a Technical Understanding and Utilization of Human Kansei



TADENUMA Makoto


What kind of nonverbal communication has been most effective since ancient times? Ancient mural paintings are powerful enough to evoke images of hunting. When we cannot communicate effectively with people whose language we do not speak, we use gestures. Similarly, some people express their emotions through rhythmical sounds and melodies. Our research focuses on nonverbal expressions of images applying a kansei (sensitivity) processing technique. Using this approach, we have been working to establish a support system for enhancing the efficiency of nonverbal image expression.


Using Paintings or Designs

We sought ways of allowing untrained persons to better express themselves through painting and photography.
First, we analyzed many master paintings for three major factors: composition, coloration, and brushwork. From a database of this information, we developed a painting assistant called "Cyber Atelier1 ." The system is a successful integration of three functions: a composition guide for setting an object in an appropriate location, a color selector for using representative colors with coloration in proper proportions, and a texture feature for adding the brushwork of particular master painters to a painting.

We also created "Cyber Photographer," which applies the composition guide. With this process, a panoramic photograph is segmented into regions according to their visual appeal. A scene in which the main object is most appropriately located is cut from it. Next, we developed an "Image Re-Composer2 ," which allows extracted objects to be relocated against a different background while ensuring good composition.

Another trial conveyed messages by using an iconic language with its own rules to connect icons and communicate effectively. This iconic language was called "CAILS3 ." We tested how well the system conveyed meaning when used in different languages, thereby proving its effectiveness in cross-linguistic communication. Some encouraging and heartwarming results were also found in trials with children with disabilities.


Using Body Movement

We selected dance as a method of image expression that uses the body, and we studied how technology might support it.

We began our experiment by categorizing the impression of dance images into seven representative categories. After statistically analyzing the relationship between these impressions and the physical parameters of the dance image, we devised an equation that approximates the relationship quite accurately. With this technique, we computerized a system that presumes the kind of image the dancer intends to express, generates a suitable video and music in real time, and combines those with the dancer's real image. This interactive dance system, called "MIDAS4 ," can emphasize the image that a dancer intends to express (Figure 1). We also experimented with an advanced image support system5 that adds 3-dimensional information to human motion.


Synchronizing Music and Moving Images

We studied the enhanced effect of image expression by synchronizing the rhythm of music with the rhythm in a moving image. Rhythm can be extracted by analyzing both the spatial frequency of a visual texture and the optical flow of pixels across the screen, and by comparing how they vary over time. We established a method of overlapping multiple images using a script description. This enabled us to develop the "Image Wave6 " system with which we can synthesize the rhythm in a constructed image in synchronization with each rhythmic component in music. Experiments show that the system effectively highlights image-based expression.


Future of the Image Expression Support System

Recently, international academic societies have recognized the importance of kansei information processing as an essential factor in the development of future communication systems. One prominent embodiment of this is an image expression support system. We are confident that this field will advance with deeper study of human kansei. In that sense, we believe that our research is quite significant in paving the way for this growing field of study.


Reference