Young-Duk Park, Kyung-Ho Loken-Kim,
Larel Fais and Suguru Mizunashi
Analysis of Gesture Behavior in a
Multimedia/multimodal Interpreting Experiment
;Human vs Wizard of Oz Interpretation Method
Abstract:To investigate the user's linguistic and paralinguistic behaviors when communicating with
a multimedia/multimodal interpreting system, we performed two experiments based on the
human and the Wizard of Oz interpretation method. In this report, we mainly describe the
results of subjects' gesture behaviors in the experiments. We analyzed the collected
gestures from the usage ratio, the shapes, and their intention. The experimental results
shows that the gestures are very useful to the not only to the users but also to the machine
in the interpreting telecommunication. Therefore, we believe that the future machine
interpreting cannot be completed without analyzing the characteristics of the
multimedia/multimodal data and researching the recognition methodology which contains
other modalities as well as speech. To take the research more concrete the more data
collection for gestures in various interpreting situations will be necessary.