liu.seSearch for publications in DiVA
Change search
ReferencesLink to record
Permanent link

Direct link
On creating multimodal virtual humans-real time speech driven facial gesturing
Department of Telecommunications, Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia.
Linköping University, Department of Electrical Engineering. Linköping University, The Institute of Technology.
Department of Telecommunications, Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia.
2011 (English)In: MULTIMEDIA TOOLS AND APPLICATIONS, ISSN 1380-7501, Vol. 54, no 1, 165-179 p.Article in journal (Refereed) Published
Abstract [en]

Because of extensive use of different computer devices, human-computer interaction design nowadays moves towards creating user centric interfaces. It assumes incorporating different modalities that humans use in everyday communication. Virtual humans, who look and behave believably, fit perfectly in the concept of designing interfaces in more natural, effective, as well as social oriented way. In this paper we present a novel method for automatic speech driven facial gesturing for virtual humans capable of real time performance. Facial gestures included are various nods and head movements, blinks, eyebrow gestures and gaze. A mapping from speech to facial gestures is based on the prosodic information obtained from the speech signal. It is realized using a hybrid approach-Hidden Markov Models, rules and global statistics. Further, we test the method using an application prototype-a system for speech driven facial gesturing suitable for virtual presenters. Subjective evaluation of the system confirmed that the synthesized facial movements are consistent and time aligned with the underlying speech, and thus provide natural behavior of the whole face.

Place, publisher, year, edition, pages
Springer Science Business Media , 2011. Vol. 54, no 1, 165-179 p.
Keyword [en]
Facial gestures, Visual prosody, Multimodal interfaces, Facial animation, Speech processing, Human-computer interaction
National Category
Engineering and Technology
URN: urn:nbn:se:liu:diva-68903DOI: 10.1007/s11042-010-0526-yISI: 000291061100009OAI: diva2:422022
Available from: 2011-06-10 Created: 2011-06-10 Last updated: 2011-06-13

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Forchheimer, Robert
By organisation
Department of Electrical EngineeringThe Institute of Technology
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 61 hits
ReferencesLink to record
Permanent link

Direct link