liu.seSearch for publications in DiVA
Change search
ReferencesLink to record
Permanent link

Direct link
Eyes on multimodal interaction
Linköping University, Department of Computer and Information Science. Linköping University, The Institute of Technology.
2004 (English)Doctoral thesis, monograph (Other academic)
Abstract [en]

Advances in technology are making it possible for users to interact with computers by various modalities, often through speech and gesture. Such multimodal interaction is attractive because it mimics the patterns and skills in natural human-human communication. To date, research in this area has primarily focused on giving commands to computers. The focus of this thesis shifts from commands to dialogue interaction. The work presented here is divided into two parts. The first part looks at the impact of the characteristics of the spoken feedback on users' experience of a multimodal dialogue system. The second part investigates whether and how eye-gaze can be utilized in multimodal dialogue systems.

Although multimodal interaction has attracted many researchers, little attention has been paid to how users experience such systems. The first part of this thesis investigates what makes multimodal dialogue systems either human-like or tool-like, and what qualities are most important to users. In a simulated multimodal timetable information system users were exposed to different levels of spoken feedback. The results showed that the users preferred the system to be either clearly tool-like, with no spoken words, or clearly human-like, with complete and natural utterances. Furthermore, the users' preference for a human-like multimodal system tended to be much higher after they had actual experience than beforehand based on imagination.

Eye-gaze plays a powerful role in human communication. In a computer-mediated collaborative task involving a tourist and a tourist consultant, the second part of this thesis starts with examining the users' eye-gaze patterns and their functions in deictic referencing, interest detection, topic switching, ambiguity reduction, and establishing common ground in a dialogue. Based on the results of this study, an interactive tourist advisor system that encapsulates some of the identified patterns and regularities was developed. In a "stress test" experiment based on eye-gaze patterns only, the developed system conversed with users to help them plan their conference trips. Results demonstrated thateye-gaze can play an assistive role in managing future multimodal human-computer dialogues.

Place, publisher, year, edition, pages
Linköping: Linköping University , 2004. , 226 p.
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 893
National Category
Computer Science
URN: urn:nbn:se:liu:diva-24226Local ID: 3823ISBN: 91-85295-30-2OAI: diva2:244543
Public defence
2004-11-19, Visionen, Hus B, Linköpings Universitet, Linköping, 13:15 (Swedish)
Available from: 2009-10-07 Created: 2009-10-07 Last updated: 2013-01-30

Open Access in DiVA

No full text

Search in DiVA

By author/editor
Qvarfordt, Pernilla
By organisation
Department of Computer and Information ScienceThe Institute of Technology
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 103 hits
ReferencesLink to record
Permanent link

Direct link