liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Traces Of Modal Synergy: Studying Interactive Musical Sonificationof Images in General-Audience Use
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. (Information Visualization)
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. (Media Design and Image Reproduction)ORCID iD: 0000-0001-5678-6565
2019 (English)In: The 25th International Conference on Auditory Display (ICAD 2019) / [ed] Vickers, Paul and Gröhn, Matti and Stockman, Tony, Newcastle upon Tyne, UK, 2019, p. 199-206Conference paper, Published paper (Refereed)
Abstract [en]

Photone is an interactive installation combining color images with musical sonification. The musical expression is generated based on the syntactic (as opposed to semantic) features of an image as it is explored by the userメs pointing device, intending to catalyze a holistic user experience we refer to as modal synergy where visual and auditory modalities multiply rather than add. We collected and analyzed two months' worth of data from visitorsメ interactions with Photone in a public exhibition at a science center. Our results show that a small proportion of visitors engaged in sustained interaction with Photone, as indicated by session times. Among the most deeply engaged visitors, a majority of the interaction was devoted to visually salient objects, i.e., semantic features of the images. However, the data also contains instances of interactive behavior that are best explained by exploration of the syntactic features of an image, and thus may suggest the emergence of modal synergy.

Place, publisher, year, edition, pages
Newcastle upon Tyne, UK, 2019. p. 199-206
Keywords [en]
Sonification
National Category
Other Engineering and Technologies
Identifiers
URN: urn:nbn:se:liu:diva-159112DOI: 10.21785/icad2019.010ISBN: 0967090466 (electronic)ISBN: 9780967090467 (electronic)OAI: oai:DiVA.org:liu-159112DiVA, id: diva2:1339252
Conference
The 25th International Conference on Auditory Display (ICAD 2019), Newcastle upon Tyne, England, 23–27 June 2019
Available from: 2019-07-26 Created: 2019-07-26 Last updated: 2024-01-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records

Rönnberg, NiklasLöwgren, Jonas

Search in DiVA

By author/editor
Rönnberg, NiklasLöwgren, Jonas
By organisation
Media and Information TechnologyFaculty of Science & Engineering
Other Engineering and Technologies

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 70 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf