1. Sentence: In this paper, we investigate human sensitivity to audio mismatches (i.e., when individuals’ voices are not matched to their gestures)
and visual desynchronization (i.e., when the body motions of the individuals in a group are mis-aligned in time) in virtual human conversers. (3rd sentence, Abstract)
2. Problem: Very little is known about how gestures of virtual conversing agents interacting with each other is perceived by the user.
3. Key Idea: To use a set of experiments to find how people react when presented with a virtual conversation with visual and aural information – Do they rely on one sense more than another?
4a. What the paper does: Provides guidelines to enhance the believability of synthetic conversations in real-time applications depicting groups – Identifies that people attend more to the body motions of characters than to the audio; and if body gestures of the group seem plausible, it is not always necessary to ensure that these gestures and audio beats match. However, it does help to match the audio to the motion of at least one talker.
4b. What it could be used for: Generate more realistic and believable conversing groups or crowds.
5. Resources: Video on paper’s web-page