liu.seSearch for publications in DiVA
Change search
Refine search result
12 1 - 50 of 51
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Rind, Alexander
    et al.
    St. Pölten University of Applied Sciences, St. Pölten, Austria.
    Enge, Kajetan
    St. Pölten University of Applied Sciences, St. Pölten, Austria.
    Iber, Michael
    St. Pölten University of Applied Sciences, St. Pölten, Austria.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Lenzi, Sara
    Ikerbasque Basque Foundation for Science, Bilbao, Spain.
    Elmquist, Elias
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Caiola, Valentina
    City University of Hong Kong, Kowloon, Hong Kong.
    Lan, Fei
    University of Utah, Salt Lake City, USA.
    Höldrich, Robert
    University of Music and Performing Arts Graz, Graz, Austria.
    Aigner, Wolfgang
    St. Pölten University of Applied Sciences, St. Pölten, Austria.
    Integrating Sonification and Visualization – But Why?2024Conference paper (Other academic)
    Abstract [en]

    The research communities studying visualization and sonification share exceptionally similar goals, essentially aiming to make data interpretable to humans. One community uses visual representations, while the other employs auditory (nonspeech) repre- sentations of data. Although the two communities have much in common, they developed mostly in parallel, with only compara- tively few examples of integrated audiovisual data analysis idioms presented so far. This panel brings together researchers from both the fields of visualization and sonification to collectively discuss the question: ‘Integrating Sonification and Visualization – but why?’ 

    In the panel discussion, we will tackle this question along two main hypotheses: Combining the modalities to (1) increase the “bandwidth from data to brain,” or (2) to increase a user’s personal engagement during the data analysis process. On the one hand, designers might aim to communicate more data in less time or gain more and more complex insights from the data by using a multi-modal display. This argument follows an understanding that two senses should be capable of processing more information than “just” one. On the other hand, sometimes, a more engaged analysis of the represented data is desirable. Engagement with data visualization stands as a crucial topic in numerous contexts within our field, encouraging “deeper” thinking by expert analysts, readers of data journalism articles, and students in educational settings. We hypothesize that integrating visualization with sonification holds the potential to enhance user engagement during analysis. Through the panel discussion, we want to delve into the spectrum between aiming for bandwidth and engagement, seeking to understand the opportunities and challenges of integrating sonification and visualization. 

  • 2.
    Enge, Kajetan
    et al.
    St Poelten Univ Appl Sci, Austria; Univ Mus & Performing Arts Graz, Austria.
    Elmquist, Elias
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Caiola, Valentina
    City Univ Hong Kong, Peoples R China.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rind, Alexander
    St Poelten Univ Appl Sci, Austria.
    Iber, Michael
    St Poelten Univ Appl Sci, Austria.
    Lenzi, Sara
    Ikerbasque, Spain; Univ Deusto, Spain.
    Lan, Fangfei
    Univ Utah, UT USA.
    Höldrich, Robert
    Univ Mus & Performing Arts Graz, Austria.
    Aigner, W.
    St Poelten Univ Appl Sci, Austria.
    Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization2024In: Computer graphics forum (Print), ISSN 0167-7055, E-ISSN 1467-8659, Vol. 43, no 3, article id e15114Article in journal (Refereed)
    Abstract [en]

    The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual representations of data, and the other community employs auditory (non-speech) representations of data. While the two communities have a lot in common, they developed mostly in parallel over the course of the last few decades. With this STAR, we discuss a collection of work that bridges the borders of the two communities, hence a collection of work that aims to integrate the two techniques into one form of audiovisual display, which we argue to be “more than the sum of the two.” We introduce and motivate a classification system applicable to such audiovisual displays and categorize a corpus of 57 academic publications that appeared between 2011 and 2023 in categories such as reading level, dataset type, or evaluation system, to mention a few. The corpus also enables a meta-analysis of the field, including regularly occurring design patterns such as type of visualization and sonification techniques, or the use of visual and auditory channels, showing an overall diverse field with different designs. An analysis of a co-author network of the field shows individual teams without many interconnections. The body of work covered in this STAR also relates to three adjacent topics: audiovisual monitoring, accessibility, and audiovisual data art. These three topics are discussed individually in addition to the systematically conducted part of this research. The findings of this report may be used by researchers from both fields to understand the potentials and challenges of such integrated designs while hopefully inspiring them to collaborate with experts from the respective other field.

  • 3.
    Elmquist, Elias
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Enge, Kajetan
    St. Pölten University of Applied Sciences, St. Pölten, Austria; University of Music and Performing Arts Graz, Graz, Austria.
    Rind, Alexander
    t. Pölten University of Applied Sciences, St. Pölten, Austria.
    Navarra, Carlo
    Linköping University, Department of Thematic Studies, Tema Environmental Change. Linköping University, Faculty of Arts and Sciences.
    Höldrich, Robert
    University of Music and Performing Arts Graz, Graz, Austria.
    Iber, Michael
    St. Pölten University of Applied Sciences, St. Pölten, Austria.
    Bock, Alexander
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Ynnerman, Anders
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Aigner, Wolfgang
    St. Pölten University of Applied Sciences, St. Pölten, Austria.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Parallel Chords: an audio-visual analytics design for parallel coordinates2024In: Personal and Ubiquitous Computing, ISSN 1617-4909, E-ISSN 1617-4917Article in journal (Refereed)
    Abstract [en]

    One of the commonly used visualization techniques for multivariate data is the parallel coordinates plot. It provides users with a visual overview of multivariate data and the possibility to interactively explore it. While pattern recognition is a strength of the human visual system, it is also a strength of the auditory system. Inspired by the integration of the visual and auditory perception in everyday life, we introduce an audio-visual analytics design named Parallel Chords combining both visual and auditory displays. Parallel Chords lets users explore multivariate data using both visualization and sonification through the interaction with the axes of a parallel coordinates plot. To illustrate the potential of the design, we present (1) prototypical data patterns where the sonification helps with the identification of correlations, clusters, and outliers, (2) a usage scenario showing the sonification of data from non-adjacent axes, and (3) a controlled experiment on the sensitivity thresholds of participants when distinguishing the strength of correlations. During this controlled experiment, 35 participants used three different display types, the visualization, the sonification, and the combination of these, to identify the strongest out of three correlations. The results show that all three display types enabled the participants to identify the strongest correlation — with visualization resulting in the best sensitivity. The sonification resulted in sensitivities that were independent from the type of displayed correlation, and the combination resulted in increased enjoyability during usage.

    Download full text (pdf)
    fulltext
  • 4.
    Gorenko, Ivar
    et al.
    Linköping University, Department of Science and Technology. Linköping University, Faculty of Science & Engineering.
    Besançon, Lonni
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Forsell, Camilla
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Supporting Astrophysical Visualization with Sonification2024Conference paper (Refereed)
    Abstract [en]

    This poster presents initial design steps exploring how sonification can be used to support visualization for comprehension of space and time in astronomical data. Radio signals travel at the speed of light. With a visualization of the universe, it is possible to travel faster than light and pass the radio waves leaving earth. We can then travel back in time. We propose to use sonification consisting of songs representing each year as a musical journey through space and time to create an engaging experience.

  • 5.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Where Visualization Fails, Sonification Speaks2024Conference paper (Refereed)
    Abstract [en]

    Traveling by public transport can be challenging for a visually impaired traveler. However, visual information can be supported by sonification, the use of non-speech sound to convey information about data. This research project aims to explore how sonification can be used to provide information to a traveler at a bus stop. Three situations are described together with different sonification design approaches that will later be further developed and evaluated. 

  • 6.
    Ziemer, Tim
    et al.
    University of Hamburg, Germany.
    Lenzi, Sara
    Delft University of Technology, Netherlands.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Hermann, Thomas
    Bielefeld University, Germany.
    Bresin, Roberto
    KTH Royal Institute of Technology, Sweden.
    Introduction to the special issue on design and perception of interactive sonification2023In: Journal on Multimodal User Interfaces, ISSN 1783-7677, E-ISSN 1783-8738, Vol. 17, p. 213-214Article in journal (Other academic)
    Download full text (pdf)
    fulltext
  • 7.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Ringdahl, Rasmus
    Linköping University, Department of Science and Technology, Communications and Transport Systems. Linköping University, Faculty of Science & Engineering.
    Fredriksson, Anna
    Linköping University, Department of Science and Technology, Communications and Transport Systems. Linköping University, Faculty of Science & Engineering.
    Measurement and sonification of construction site noise and particle pollution data2023In: Smart and Sustainable Built Environment, ISSN 2046-6099, E-ISSN 2046-6102, Vol. 12, no 4, p. 742-764Article in journal (Refereed)
    Abstract [en]

    Purpose – The noise and dust particles caused by the construction transport are by most stakeholders experienced as disturbing. The purpose of this study is to explore how sonification can support visualization in construction planning to decrease construction transport disturbances.

    Design/methodology/approach – This paper presents an interdisciplinary research project, combining research on construction logistics, internet of things and sonification. First, a data recording device, including sound, particle, temperature and humidity sensors, was implemented and deployed in a development project. Second, the collected data were used in a sonification design, which was, third, evaluated with potential users.

    Findings – The results showed that the low-cost sensors used could capture “good enough” data, and that the use of sonification for representing these data is interesting and a possible useful tool in urban and construction transport planning.

    Research limitations/implications – There is a need to further evolve the sonification design and better communicate the aim of the sounds used to potential users. Further testing is also needed.

    Practical implications – This study introduces new ideas of how to support visualization with sonification planning the construction work and its impact on the vicinity of the site. Currently, urban planning and construction planning focus on visualizing the final result, with little focus on how to handle disturbances during the construction process.

    Originality/value – Showing the potentials of using low-cost sensor data in sonification, and using sonification together with visualization, is the result of a novel interdisciplinary research area combination.

    Download full text (pdf)
    fulltext
  • 8.
    Elmquist, Elias
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Bock, Alexander
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Lundberg, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Ynnerman, Anders
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Center for Medical Image Science and Visualization (CMIV).
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    SonAir: the design of a sonification of radar data for air traffic control2023In: Journal on Multimodal User Interfaces, ISSN 1783-7677, E-ISSN 1783-8738, Vol. 17, no 3, p. 137-149Article in journal (Refereed)
    Abstract [en]

    Along with the increase of digitalization and automation, a new kind of working environment is emerging in the field of air traffic control. Instead of situating the control tower at the airport, it is now possible to remotely control the airport at any given location, i.e. in a remote tower center (RTC). However, by controlling the airport remotely, the situational awareness and sense of presence might be compromised. By using directional sound, a higher situational awareness could potentially be achieved while also offloading the visual perception which is heavily used in air traffic control. Suitable use cases for sonification in air traffic control were found through workshops with air traffic controllers. A sonification design named SonAir was developed based on the outcome of the workshops, and was integrated with an RTC simulator for evaluating to what degree SonAir could support air traffic controllers in their work. The results suggest that certain aspects of SonAir could be useful for air traffic controllers. A continuous sonification where the spatial positioning of aircraft were conveyed was experienced to be partially useful, but the intrusiveness of SonAir should be further considered to fit the air traffic controllers’ needs. An earcon that conveyed when an aircraft enters the airspace and from which direction was considered useful to support situational awareness.

    Download full text (pdf)
    fulltext
  • 9.
    Svoronos-Kanavas, Iason
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Department of Sound and Musical Instrument Technology, Stylianou Typladou, Kefalonia, Greece.
    Agiomyrgianakis, Vasilis
    Faculty of Music & Audiovisual Arts, Ionian University, Department of Audiovisual Arts, Ionian University, Corfu, Greece.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    An exploratory use of audiovisual displays on oceanographic data2022Conference paper (Refereed)
    Abstract [en]

    The present study is an interdisciplinary endeavour that transmutes science, technology, and aesthetics into an audiovisual experience. The objective is to highlight the potential of combining sonification with visualisation in order to enhance the comprehension of extensive and complex sets of data. Moreover, this paper describes contemporary tools and methods for the implementation of the practice and suggests effective ways to monitor environmental changes. It can be regarded as an exploratory study for familiarisation with the potential of sonification and visualisation in the exploration of environmental data.

  • 10.
    Svensson, Åsa
    et al.
    LFV Air Navigation Services of Sweden.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Expected teamwork attributes between human operator and automation in air traffic control2022Conference paper (Refereed)
    Abstract [en]

    To meet the increasing demands within the air traffic control (ATC) industry, automation is seen as one solution to increase efficiency, safety, and capacity. In ATC, teamwork between air traffic controllers (ATCO) is a key component to safely direct the aircraft. With increasing automation, the human-automation teamwork will become even more important to keep the human operator in the loop and contribute to the operator’s involvement in decision-making. However, operators can be cautious to new automation due to experience has taught them to be cautious and the willingness to accept automation can relate to the purpose of the automation. Therefore, this study presents how ATCOs want to work with an automated tool in the future, what qualities the automation should have, and what the human should be able to do when working with an automated system. Through an online questionnaire, 113 operational ATCOs answered questions about ATCOs’ experiences and expectations of current and future ATM systems, as well as teamwork between the ATCO and automated systems. For this paper, the teamwork aspects of the questionnaire are presented and discussed. The ATCOs believed that they should be able to trust the automation, know the reasoning behind the automation’s suggestions and decisions, and be able to always take control of the situation from the automation. The results will be used in early development processes of automation, in both the ATM domain but also in other control domains, that faces similar challenges. 

  • 11.
    Fredriksson Häägg, Anna
    et al.
    Linköping University.
    Weil, Charlotte
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology.
    Insight-based Evaluation of a Map-based Dashboard2022Conference paper (Refereed)
    Abstract [en]

    Map-based dashboards are used for data exploration every day. The present study used an insight-based methodology for evaluating a map-based dashboard that presents research findings of water management and ecosystem services in the Amazon. In addition to analyzing the insights gained from using the dashboard, the evaluation method was compared to standardized questionnaires and task-based evaluations. The result suggests that the dashboard enabled the participants to gain domain-relevant, complex insights regarding the topic presented. Furthermore, the insight-based analysis highlighted unexpected insights and hypotheses regarding causes and potential adaptation strategies for remediation. Although time- and resource-consuming, the insight-based methodology was shown to have the potential of thoroughly analyzing how end users can utilize map-based dashboards for data exploration and decision making. Finally, the insight-based methodology is argued to evaluate tools in scenarios more similar to real-life usage, compared to task-based evaluation methods.

  • 12.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Forsell, Camilla
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Questionnaires assessing usability of audio-visual representations2022Conference paper (Refereed)
    Abstract [en]

    In this position paper we argue for the need of using standardized questionnaires for assessing usability in audio-visual representations. By using standardized measures of usability, comparability and validity of research studies in this field will be improved. However, it is not clear which questionnaire that is most suitable for assessing usability in audio-visual representations, neither when assessing the modalities individually or the combination. We present a variety of different questionnaires available, and argue for the need of combining different measures as well as developing new.

  • 13.
    Aigner, Wolfgang
    et al.
    St. Pölten University of Applied Sciences, Austria.
    Enge, Kajetan
    St. Pölten University of Applied Sciences, Austria and University of Music and Performing Arts Graz, Austria.
    Iber, Michael
    St. Pölten University of Applied Sciences, Austria.
    Rind, Alexander
    St. Pölten University of Applied Sciences, Austria.
    Elmqvist, Niklas
    University of Maryland, College Park, United States.
    Höldrich, Robert
    University of Music and Performing Arts Graz, Austria.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Walker, Bruce N.
    Georgia Institute of Technology, United States.
    Workshop on Audio-Visual Analytics2022In: AVI 2022: Proceedings of the 2022 International Conference on Advanced Visual Interfaces, 2022Conference paper (Other academic)
    Abstract [en]

    In their daily lives, people use more than one sense to perceive and interpret their environment. Likewise, audio-visual interfaces can support human data analysts better than interfaces relying on just one sense. While the research communities of sonification and visualization have both carried out extensive research on the auditory and visual representation of data, comparatively little is known about their systematic and complementary combination for data analysis. After two workshops at Audio Mostly 2021 and IEEE VIS, this 3rd workshop on audio-visual analytics continues building a community of researchers interested in combining visualization and sonification.

  • 14.
    Aigner, Wolfgang
    et al.
    St Polten Univ Appl Sci, Austria.
    Enge, Kajetan
    St Polten Univ Appl Sci, Austria; Univ Mus & Performing Arts Graz, Austria.
    Iber, Michael
    St Polten Univ Appl Sci, Austria.
    Rind, Alexander
    St Polten Univ Appl Sci, Austria.
    Elmqvist, Niklas
    University of Maryland, College Park, United States.
    Hoeldrich, Robert
    Univ Mus & Performing Arts Graz, Austria.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Univ Maryland, DC USA.
    Walker, Bruce N.
    Georgia Inst Technol, GA USA.
    Workshop on Audio-Visual Analytics2022In: PROCEEDINGS OF THE WORKING CONFERENCE ON ADVANCED VISUAL INTERFACES AVI 2022, ASSOC COMPUTING MACHINERY , 2022Conference paper (Refereed)
    Abstract [en]

    In their daily lives, people use more than one sense to perceive and interpret their environment. Likewise, audio-visual interfaces can support human data analysts better than interfaces relying on just one sense. While the research communities of sonification and visualization have both carried out extensive research on the auditory and visual representation of data, comparatively little is known about their systematic and complementary combination for data analysis. After two workshops at Audio Mostly 2021 and IEEE VIS, this 3rd workshop on audio-visual analytics continues building a community of researchers interested in combining visualization and sonification.

  • 15.
    Svensson, Åsa
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Forsell, Camilla
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Lundberg, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Automation, teamwork, and the feared loss of safety: Air traffic controllers’ experiences and expectations on current and future ATM systems2021In: 32nd European Conference on Cognitive Ergonomics: Designing Virtual and Physical Interactive Systems, ECCE 2021, ACM Digital Library, 2021, p. -8, article id 10Conference paper (Refereed)
    Abstract [en]

    In various control systems, automation is implemented to increase efficiency and safety. With increased automation, it becomes increasingly relevant to view the automation as a team member, rather than as a tool. In best cases, human-automation teamwork keeps workload within acceptable limits, increase situation awareness, and keeps the operator in the control loop. However, human-automation teamwork will only flourish if the automation is developed with the human operator in mind. Therefore, investigations of the current experiences and expectations regarding automation and teamwork are important for the development of automation. Through a questionnaire among Air Traffic Controllers (ATCOs), the present study aims to investigate how ATCOs perceive automation and safety in current and future air traffic control systems and the importance of different teamwork factors for human-human and human-automation collaboration. The results indicate that the ATCOs believe that safety will increase in the future along with increasing automation as long as the automation is working as expected. The ATCOs expressed a fear of deskilling and losing situation awareness with automation, a fear associated with a new ATCO role of monitoring the system and take over when the automation fails. The results suggest that design for human-automation teamwork aspects that ATCOs value, such as adaptability or mutual performance monitoring, could be a way forward.

  • 16.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Löwgren, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Designing the user experience of musical sonification in public and semi-public spaces2021In: SoundEffects, E-ISSN 1904-500X, Vol. 10, no 1, p. 125-141Article in journal (Refereed)
    Abstract [en]

    Sonification refers to sonic expression of data or information. It is often thought of as an auditory complement, providing additional information about data which can reveal patterns and facilitate interpretation and understanding of the data. Hence, the listening space created by a sonifi cation is always a hybrid where auditory augmentation complements other information modalities and, in some cases, also spatial qualities. In this work, we focus on sonifi cation in public and semi-public spaces, and specifi cally on musical sonifi cation – the use of musical sounds to create a sonic environment, augmenting or complementing a physical shared space. We draw upon established approaches in interaction design to focus our work on the user experience of musical sonifi cation in public and semi-public spaces. Specifi cally, we fi rst identify the experiential qualities of sonic atmosphere and performativity as important aspects of sonifi cation in public and semi-public spaces, then use those experiential qualities generatively in the speculative design of a musical sonifi cation sketch. The design sketch comprises a dynamic musical sonifi cation of air quality data, intending to give citizens an awareness and an enhanced individual and interpersonal understanding of air quality in their city.

    Download full text (pdf)
    fulltext
  • 17.
    Elmquist, Elias
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Ejdbo, Malin
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Bock, Alexander
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    OpenSpace Sonification: Complementing Visualization of the Solar System with Sound2021In: Proceedings of the 26th International Conference on Auditory Display (ICAD 2021) / [ed] Areti Andreopoulou, Milena Droumeva, Joseph W. Newbold, Kyla McMullen and Paul Vickers, The International Community for Auditory Display , 2021, p. 135-142Conference paper (Refereed)
    Abstract [en]

    Data visualization software is commonly used to explore outer space in a planetarium environment, where the visuals of the software is typically accompanied with a narrator and supplementary background music. By letting sound take a bigger role in these kinds of presentations, a more informative and immersive experience can be achieved. The aim of the present study was to explore how sonification can be used as a complement to the visualization software OpenSpace to convey information about the Solar System, as well as increasing the perceived immersiveness for the audience in a planetarium environment. This was investigated by implementing a sonification that conveyed planetary properties, such as the size and orbital period of a planet, by mapping this data to sonification parameters. With a user-centered approach, the sonification was designed iteratively and evaluated in both an online and planetarium environment. The results of the evaluations show that the participants found the sonification informative and interesting, which suggest that sonification can be beneficially used as a complement to visualization in a planetarium environment.

    Download full text (pdf)
    fulltext
  • 18.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Sonification for Conveying Data and Emotion2021In: AM '21: Audio Mostly 2021, 2021, p. 56-63Conference paper (Refereed)
    Abstract [en]

    In the present study a sonification of running data was evaluated. The aim of the sonification was to both convey information about the data and convey a specific emotion. The sonification was evaluated in three parts, firstly as an auditory graph, secondly together with additional text information, and thirdly together with an animated visualization, with a total of 150 responses. The results suggest that the sonification could convey an emotion similar to that intended, but at the cost of less good representation of the data. The addition of visual information supported understanding of the sonification, and the auditory representation of data. The results thus suggest that it is possible to design sonification that is perceived as both interesting and fun, and convey an emotional impression, but that there may be a trade off between musical experience and clarity in sonification.

  • 19.
    Besançon, Lonni
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Löwgren, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Tennant, Jonathan P.
    Southern Denmark University Library, Odense, Denmark; Universite de Paris, Rue Charles V, Paris, France; Institute for Globally Distributed Open Research and Education, Ubud, Indonesia.
    Cooper, Matthew
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Open up: a survey on open and non-anonymized peer reviewing2020In: BMC Research Integrity and Peer Review, ISSN 2058-8615, Vol. 5, no 8, p. 1-11Article in journal (Refereed)
    Abstract [en]

    Background

    Our aim is to highlight the benefits and limitations of open and non-anonymized peer review. Our argument is based on the literature and on responses to a survey on the reviewing process of alt.chi, a more or less open review track within the so-called Computer Human Interaction (CHI) conference, the predominant conference in the field of human-computer interaction. This track currently is the only implementation of an open peer review process in the field of human-computer interaction while, with the recent increase in interest in open scientific practices, open review is now being considered and used in other fields.

    Methods

    We ran an online survey with 30 responses from alt.chi authors and reviewers, collecting quantitative data using multiple-choice questions and Likert scales. Qualitative data were collected using open questions.

    Results

    Our main quantitative result is that respondents are more positive to open and non-anonymous reviewing for alt.chi than for other parts of the CHI conference. The qualitative data specifically highlight the benefits of open and transparent academic discussions. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

    Conclusion

    While the benefits are quite clear and the system is generally well-liked by alt.chi participants, they remain reluctant to see it used in other venues. This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation.

    Download full text (pdf)
    fulltext
  • 20.
    Hassan, Kahin Akram
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Forsell, Camilla
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Cooper, Matthew
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Johansson, Jimmy
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Centre for Climate Science and Policy Research, CSPR.
    A Study on 2D and 3D Parallel Coordinates for Pattern Identification in Temporal Multivariate Data2019Conference paper (Refereed)
    Abstract [en]

    Parallel coordinates are commonly used for non-temporal multivariate data, but there is little support for their usability for displaying temporal multivariate data. In this paper, we introduce a study evaluating the usability of 2D and 3D parallel coordinates for pattern identification in temporal multivariate data. The results indicate that 3D parallel coordinates have higher usability, as measured with higher accuracy and faster response time as well as subjective ratings, compared to 2D.

  • 21.
    Hassan, Kahin Akram
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Liu, Yu
    Linköping University, Department of Science and Technology, Physics, Electronics and Mathematics. Linköping University, Faculty of Science & Engineering.
    Besançon, Lonni
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Johansson, Jimmy
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Centre for Climate Science and Policy Research, CSPR.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    A Study on Visual Representations for Active Plant Wall Data Analysis2019In: DATA, E-ISSN 2306-5729, Vol. 4, no 2, article id 74Article in journal (Refereed)
    Abstract [en]

    The indoor climate is closely related to human health, well-being, and comfort. Thus, an understanding of the indoor climate is vital. One way to improve the indoor climates is to place an aesthetically pleasing active plant wall in the environment. By collecting data using sensors placed in and around the plant wall both the indoor climate and the status of the plant wall can be monitored and analyzed. This manuscript presents a user study with domain experts in this field with a focus on the representation of such data. The experts explored this data with a Line graph, a Horizon graph, and a Stacked area graph to better understand the status of the active plant wall and the indoor climate. Qualitative measures were collected with Think-aloud protocol and semi-structured interviews. The study resulted in four categories of analysis tasks: Overview, Detail, Perception, and Complexity. The Line graph was found to be preferred for use in providing an overview, and the Horizon graph for detailed analysis, revealing patterns and showing discernible trends, while the Stacked area graph was generally not preferred. Based on these findings, directions for future research are discussed and formulated. The results and future directions of this research can facilitate the analysis of multivariate temporal data, both for domain users and visualization researchers.

    Download full text (pdf)
    A Study on Visual Representations for Active Plant Wall Data Analysis
  • 22.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Musical Elements in Sonification Support Visual Perception2019In: Proceedings of the 31st European Conference on Cognitive Ergonomics (ECCE 2019), New York, NY, USA: Association for Computing Machinery (ACM), 2019, p. 114-117Conference paper (Refereed)
    Abstract [en]

    Visual representations of data are commonly used to communicate research results. However, such representations might introduce several possible challenges for the human visual perception system, for example in perceiving brightness levels. Sonification, adding sound to the visual representation, might be used to overcome these challenges. As sonification provides additional information, sonification could be useful in supporting interpretations of a visual perception. In the present study, usefulness in terms of accuracy of sonification was investigated with an interactive sonification test. In the experiment, participants were asked to identify the highest brightness level in a monochrome visual representation. The task was performed in four conditions, one with no sonification and three with different sonification settings. The results show that sonification is useful, as measured by higher task accuracy.

  • 23.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Musical sonification supports visual discrimination of color intensity2019In: Behaviour & Information Technology, ISSN 0144-929X, Vol. 38, no 10, p. 1028-1037Article in journal (Refereed)
    Abstract [en]

    Visual representations of data introduce several possible challenges for the human visual perception system in perceiving brightness levels. Overcoming these challenges might be simplified by adding sound to the representation. This is called sonification. As sonification provides additional information to the visual information, sonification could be useful in supporting the visual perception. In the present study, usefulness (in terms of accuracy and response time) of sonification was investigated with an interactive sonification test. In the test, participants were asked to identify the highest brightness level in a monochrome visual representation. The task was performed in four conditions, one with no sonification and three with different sonification settings. The results show that sonification is useful, as measured by higher task accuracy, and that the participant's musicality facilitates the use of sonification with better performance when sonification was used. The results were also supported by subjective measurements, where participants reported an experienced benefit of sonification.

    Download full text (pdf)
    fulltext
  • 24.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Sonification supports perception of brightness contrast2019In: Journal on Multimodal User Interfaces, ISSN 1783-7677, E-ISSN 1783-8738, Vol. 13, no 13, p. 373-381, article id 4Article in journal (Refereed)
    Abstract [en]

    In complex visual representations, there are several possible challenges for the visual perception that might be eased by adding sound as a second modality (i.e. sonification). It was hypothesized that sonification would support visual perception when facing challenges such as simultaneous brightness contrast or the Mach band phenomena. This hypothesis was investigated with an interactive sonification test, yielding objective measures (accuracy and response time) as well as subjective measures of sonification benefit. In the test, the participant’s task was to mark the vertical pixel line having the highest intensity level. This was done in a condition without sonification and in three conditions where the intensity level was mapped to different musical elements. The results showed that there was a benefit of sonification, with higher accuracy when sonification was used compared to no sonification. This result was also supported by the subjective measurement. The results also showed longer response times when sonification was used. This suggests that the use and processing of the additional information took more time, leading to longer response times but also higher accuracy. There were no differences between the three sonification conditions.

    Download full text (pdf)
    fulltext
  • 25.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology.
    Towards Interactive Sonification in Monitoring of Dynamic Processes2019In: Proceedings of the Nordic Sound and Music Computing Conference 2019 (NSMC2019) and the Interactive Sonification Workshop 2019 (ISON2019) / [ed] Andre Holzapfel, Stockholm, 2019, p. 92-99Conference paper (Refereed)
    Abstract [en]

    The modern control room is predominantly made up of visual displays, which might make monitoring different processes a cumbersome and visually cognitively demanding task. Sonification could be used to support the monitoring task. However, it is not clear how the most beneficial sonification should be designed. In this pilot study an experimental setup was developed to explore perception of different sonification approaches. A user experiment was performed to assess perception of the sonification when and where simulated data deviated most from the normal level. It was found that all sonification conditions were generally useful, regardless of the participant’s musical experience, shown both in terms of objective and subjective measurements. Stereo panning of the sound was also generally experienced as helpful, but the use of different pitch might not have been experienced to contribute as much for solving the task. The findings in this pilot study will be further used to create new research ideas about sonification for monitoring of dynamic processes.

  • 26.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Löwgren, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Traces Of Modal Synergy: Studying Interactive Musical Sonificationof Images in General-Audience Use2019In: The 25th International Conference on Auditory Display (ICAD 2019) / [ed] Vickers, Paul and Gröhn, Matti and Stockman, Tony, Newcastle upon Tyne, UK, 2019, p. 199-206Conference paper (Refereed)
    Abstract [en]

    Photone is an interactive installation combining color images with musical sonification. The musical expression is generated based on the syntactic (as opposed to semantic) features of an image as it is explored by the userメs pointing device, intending to catalyze a holistic user experience we refer to as modal synergy where visual and auditory modalities multiply rather than add. We collected and analyzed two months' worth of data from visitorsメ interactions with Photone in a public exhibition at a science center. Our results show that a small proportion of visitors engaged in sustained interaction with Photone, as indicated by session times. Among the most deeply engaged visitors, a majority of the interaction was devoted to visually salient objects, i.e., semantic features of the images. However, the data also contains instances of interactive behavior that are best explained by exploration of the syntactic features of an image, and thus may suggest the emergence of modal synergy.

  • 27.
    Niklas, Rönnberg
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Löwgren, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Photone: Exploring modal synergy in photographic images and music2018In: International Conference on Auditory Display, 2018 [30], Georgia Institute of Technology , 2018, p. 73-79Conference paper (Refereed)
    Abstract [en]

    We present Photone, an interactive installation combining photographic images and musical sonification. An image is displayed, and a dynamic musical score is generated based on the overall color properties of the image and the color value of the pixel under the cursor. Hence, the music changes as the user moves the cursor. This simple approach turns out to have interesting experiential qualities in use. The composition of images and music invites the user to explore the combination of hues and textures, and musical sounds. We characterize the resulting experience in Photone as one of modal synergy where visual and auditory output combine holistically with the chosen interaction technique. This tentative finding is potentially relevant to further research in auditory displays and multimodal interaction.

    Download full text (pdf)
    Photone: Exploring modal synergy in photographic images and music
  • 28.
    Hassan, Kahin Akram
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Johansson, Jimmy
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Forsell, Camilla
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Cooper, Matthew
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    On the Use of Parallel Coordinates for Temporal Multivariate Data2017Conference paper (Other academic)
    Abstract [en]

    This work presents the results from a user centered evaluation of visual representations of temporal multivariate data using 2D and3D parallel coordinates. The objective of the evaluation was to investigate whether 2D or 3D representations increase user performance when the data consists of temporal multivariate data and the visual representation contains interactive user tools. The results show that the 3D parallel coordinates representation outperforms 2D parallel coordinates with regards to both accuracy and response time.This result is of interest to the information visualization community, since it shows the usefulness of visual representations of temporal multivariate data.

    Download full text (pdf)
    On the Use of Parallel Coordinates for Temporal Multivariate Data
  • 29.
    Niklas, Rönnberg
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Sonification Enhances Perception of Color Intensity2017Conference paper (Other academic)
    Abstract [en]

    This poster presents an interactive sonification experiment, designed to evaluate possible benefits of sonification in information visual- ization. The aim of the present study was to explore the use of composed and deliberately designed musical sounds to enhance perception of color intensity in visual representations. It was hypoth- esized, that by using musical sounds for sonification perception of color intensity would be improved. In this evaluation, sonification was mapped to color intensity in visual representations, and the par- ticipants had to identify and mark the highest color intensity, as well as answer a questionnaire about their experience. Both quantitative and qualitative preliminary results suggest a benefit of sonification, and indicate that sonification is useful in data exploration. 

    Download full text (pdf)
    fulltext
  • 30.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Jimmy, Johansson
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Interactive Sonification for Visual Dense Data Displays2016In: Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016, CITEC, Bielefeld University , 2016, p. 63-67Conference paper (Refereed)
    Abstract [en]

    This paper presents an experiment designed to evaluate the possible benefits of sonification in information visualization to give rise to further research challenges. It is hypothesized, that by using musical sounds for sonification when visualizing complex data, interpretation and comprehension of the visual representation could be increased by interactive sonification.

    This hypothesis is evaluated by testing sonification in parallel coordinates and scatter plots. The participants had to identify and mark different density areas in the representations, where amplitude of the sonification was mapped to the density in the data sets. Both quantitative and qualitative results suggest a benefit of sonification. These results indicate that sonification might be useful for data exploration, and give rise to new research questions and challenges. 

    Download full text (pdf)
    Interactive Sonification for Visual Dense Data Displays
  • 31.
    Akram Hassan, Kahin
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Rönnberg, Niklas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Forsell, Camilla
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Johansson, Jimmy
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Centre for Climate Science and Policy Research, CSPR.
    On the Performance of Stereoscopic Versus Monoscopic 3D Parallel Coordinates2016Conference paper (Refereed)
    Abstract [en]

    This work presents the results from an evaluation of stereoscopic versus monoscopic 3D parallel coordinates. The objective of the evaluation was to investigate if stereopsis increases user performance. The results show that stereoscopy has no effect at all on user performance compared to monoscopy. This result is important when it comes to the potential use of stereopsis within the information visualization community.

    Download full text (pdf)
    fulltext
  • 32.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Johansson, Jimmy
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Sonification Support for Information Visualization Dense Data Displays2016In: InfoVis Papers 2016, 2016Conference paper (Refereed)
    Abstract [en]

    This poster presents an experiment designed to evaluate the possible benefits of sonification in information visualization. It is hypothesized, that by using musical sounds for sonification when visualizing complex data, interpretation and comprehension of the visual representation could be increased. In this evaluation of sonification in parallel coordinates and scatter plots, participants had to identify and mark different density areas in the representations. Both quantitative and qualitative results suggest a benefit of sonification. These results indicate that sonification might be useful for data exploration, and give rise to new research questions and challenges.

  • 33.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Lundberg, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Löwgren, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Sonifying the periphery: Supporting the formation of gestalt in air traffic control2016In: Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016, CITEC, Bielefeld University , 2016, p. 23-27Conference paper (Refereed)
    Abstract [en]

    We report a design-led exploration of sonification to provide peripheral awareness in air traffic control centers. Our assumption is that by using musical sounds for sonification of peripheral events, it is possible to create a dynamic soundscape that complements the visual information to support the formation and maintenance of an airspace Gestalt throughout the air traffic controller’s interaction. An interactive sonification concept was designed, focusing on one controlled sector of airspace with inbound and outbound aircraft. A formative assessment of the sonification concept suggests that our approach might facilitate the air traffic controller’s work by providing complementary auditory information about inbound and outbound aircraft, particularly in situations where the traffic volume is moderate to low. 

    Download full text (pdf)
    Sonifying the Periphery: Supporting the Formation of Gestalt in Air Traffic Control
  • 34.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    Löwgren, Jonas
    Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
    The Sound Challenge to Visualization Design Research2016In: Proceedings of EmoVis 2016, ACM IUI 2016 Workshop on Emotion and Visualization, Sonoma, CA, USA, March 10, 2016 / [ed] Andreas Kerren, Daniel Cernea, Margit Pohl, Linköping: Linköping University Electronic Press, 2016, p. 31-34Conference paper (Refereed)
    Abstract [en]

    This paper is an introduction to the emotional qualities of sound and music, and we suggest that the visual and the aural modalities should be combined in the design of visualizations involving emotional expressions. We therefore propose that visualization design should incorporate sonic interaction design drawing on musicology, cognitive neuroscience of music, and psychology of music, and identify what we see as key research challenges for such an approach.

    Download full text (pdf)
    fulltext
  • 35. Order onlineBuy this publication >>
    Rönnberg, Niklas
    Linköping University, Department of Clinical and Experimental Medicine, Division of Neuroscience. Linköping University, Faculty of Health Sciences.
    Assessing cognitive spare capacity as a measure of listening effort using the Auditory Inference Span Test2014Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Hearing loss has a negative effect on the daily life of 10-15% of the world’s population. One of the most common ways to treat a hearing loss is to fit hearing aids which increases audibility by providing amplification. Hearing aids thus improve speech reception in quiet, but listening in noise is nevertheless often difficult and stressful. Individual differences in cognitive capacity have been shown to be linked to differences in speech recognition performance in noise. An individual’s cognitive capacity is limited and is gradually consumed by increasing demands when listening in noise. Thus, fewer cognitive resources are left to interpret and process the information conveyed by the speech. Listening effort can therefore be explained by the amount of cognitive resources occupied with speech recognition. A well fitted hearing aid improves speech reception and leads to less listening effort, therefore an objective measure of listening effort would be a useful tool in the hearing aid fitting process.

    In this thesis the Auditory Inference Span Test (AIST) was developed to assess listening effort by measuring an individual’s cognitive spare capacity, the remaining cognitive resources available to interpret and encode linguistic content of incoming speech input while speech understanding takes place. The AIST is a dual-task hearing-innoise test, combining auditory and memory processing, and requires executive processing of speech at different memory load levels. The AIST was administered to young adults with normal hearing and older adults with hearing impairment. The aims were 1) to develop the AIST; 2) to investigate how different signal-to-noise ratios (SNRs) affect memory performance for perceived speech; 3) to explore if this performance would interact with cognitive capacity; 4) to test if different background noise types would interact differently with memory performance for young adults with normal hearing; and 5) to examine if these relationships would generalize to older adults with hearing impairment.

    The AIST is a new test of cognitive spare capacity which uses existing speech material that is available in several countries, and manipulates simultaneously cognitive load and SNR. Thus, the design of AIST pinpoints potential interactions between auditory and cognitive factors. The main finding of this thesis was the interaction between noise type and SNR showing that decreased SNR reduced cognitive spare capacity more in speech-like noise compared to speech-shaped noise, even though speech intelligibility levels were similar between noise types. This finding applied to young adults with normal hearing but there was a similar effect for older adults with hearing impairment with the addition of background noise compared to no background noise. Task demands, MLLs, interacted with cognitive capacity, thus, individuals with less cognitive capacity were more sensitive to increased cognitive load. However, MLLs did not interact with noise type or with SNR, which shows that different memory load levels were not affected differently in different noise types or in different SNRs. This suggests that different cognitive mechanisms come into play for storage and processing of speech information in AIST and for listening to speech in noise. Thus, the results suggested that a test of cognitive spare capacity seems to be a useful way to assess listening effort, even though the AIST, in the design used in this thesis, might be too cognitively demanding to provide reliable results for all individuals.

    List of papers
    1. Testing listening effort for speech comprehension using the individuals’ cognitive spare capacity
    Open this publication in new window or tab >>Testing listening effort for speech comprehension using the individuals’ cognitive spare capacity
    2011 (English)In: Audiology Research, ISSN 2039-4330, E-ISSN 2039-4349, Vol. 1, no 1, p. 82-85Article in journal (Refereed) Published
    Abstract [en]

    Most hearing aid fittings today are almost solely based on the patient’s audiogram. Although the loss of gain in the cochlea is important, for a more optimal fitting, more individual parameters of the patient’s cochlear loss together with the patient's cognitive abilities to process the auditory signal are required (Stenfelt & Rönnberg, 2009; Edwards, 2007). Moreover, the evaluation of the fitting is often based on a speech in noise task and the aim is to improve the individual patient’s signal to noise ratio (SNR) thresholds. As a consequence, hearing aid fitting may be seen as a process aimed to improve the patient’s SNR threshold rather than to improve communication ability. However, subsequent to a hearing aid fitting, there can be great differences in SNR improvement between patients that have identical hearing impairment in terms of threshold data (the audiogram). The reasons are certainly complex but one contributing factor may be the individual differences in cognitive capacity and associated listening effort. Another way to think about amplified hearing is to ease a subject’s listening effort (Sarampalis, et al., 2009). When the speech signal is degraded by noise or by a hearing impairment, more high-order cognitive or top-down processes are required to perceive and understand the signal, and listening is therefore more effortful. It is assumed that a hearing aid would ease the listening effort for a hearing impaired person. However, it is not clear how to measure the listening effort. We here present a test that will tap into the different cognitive aspects of listening effort, the Auditory Inference Span Test (AIST). The AIST is a dual task hearing in noise test, that combines auditory and memory processing and is well suited as a clinical test for listening effort.

    Place, publisher, year, edition, pages
    Pavia, Italy: PAGEPress, 2011
    National Category
    Otorhinolaryngology
    Identifiers
    urn:nbn:se:liu:diva-68318 (URN)10.4081/audiores.2011.e22 (DOI)
    Available from: 2011-05-18 Created: 2011-05-18 Last updated: 2017-12-11Bibliographically approved
    2. Assessing listening effort by measuring short-term memory storage and processing of speech in noise
    Open this publication in new window or tab >>Assessing listening effort by measuring short-term memory storage and processing of speech in noise
    2014 (English)In: Speech, Language and Hearing, ISSN 2050-5728, Vol. 17, no 3, p. 123-132Article in journal (Refereed) Published
    Abstract [en]

    Objective

    The aim of this study was to investigate the effect of working memory capacity (WMC) and updating ability (UA) on listening effort measured using a new test, the Auditory Inference Span Test (AIST), as an objective measure of listening effort.

    Design

    The AIST is based on Swedish five-word sentences and taps into three memory load levels (MLLs). It was administered in stationary speech-shaped noise at −2, −4, and −6 dB signal-to-noise ratio (SNR). WMC was assessed using the reading span test, and UA was assessed using the letter memory test. We also collected data on speech-in-noise performance and subjectively rated listening effort at the three SNRs.

    Study sample

    Thirty-nine participants with normal hearing thresholds (≤20 dB HL for 250 to 4000 Hz) took part in the study.

    Results

    AIST performance decreased with increasing MLL and was related to WMC and UA. Participants with high WMC performed better than those with low WMC at more favorable SNRs. Participants with high UA performed better than participants with low UA at the intermediate MLL, which made particular demands on the UA. Neither speech recognition scores nor subjectively rated listening effort was associated with AIST performance or either of the cognitive variables.

    Conclusion

    AIST taps into cognitive functions necessary for understanding speech in noise. However, in its current form AIST may be too cognitively taxing to successfully measure graded listening effort in participants with lower cognitive capacity.

    Place, publisher, year, edition, pages
    W Yorks, United Kingdom: Maney Publishing, 2014
    Keywords
    Listening effort, Speech-in-noise, Cognition, Working memory, Updating
    National Category
    Social Sciences Otorhinolaryngology
    Identifiers
    urn:nbn:se:liu:diva-108763 (URN)10.1179/2050572813Y.0000000033 (DOI)
    Available from: 2014-07-03 Created: 2014-07-03 Last updated: 2017-11-06Bibliographically approved
    3.
    The record could not be found. The reason may be that the record is no longer available or you may have typed in a wrong id in the address field.
    4. Adverse listening conditions affect short-term memory storage and processing of speech for older adults with hearing impairment
    Open this publication in new window or tab >>Adverse listening conditions affect short-term memory storage and processing of speech for older adults with hearing impairment
    (English)Manuscript (preprint) (Other academic)
    Abstract [en]

    Purpose: Previous work has shown an effect of noise type on memory for intelligible speech. The aim ofthis study was to investigate the effect of background noise on memory performance of intelligible speech for older adults with hearing impairment using the Auditory Inference Span Test (AIST).

    Method: Twenty participants with ages between 67 and 80 years with symmetrical hearing loss (29 to 47dB HL) performed the AIST, which requires processing of five-word sentences at three memoryload levels (MLLs) in three listening conditions: Quiet, steady-state noise (SSN), and backgroundvoices (ISTS). Individualized SNRs targeted 90% speech intelligibility. AIST performance reflects the amount of cognitive capacity occupied in listening, and consequently indicates the amount of listening effort. Working memory capacity (WMC) was assessed using the reading span test, and updating ability (UA) was assessed using the letter memory test.

    Results: AIST performance decreased in background noise and with increasing MLL. It was related to UA and age but not to WMC. Response times on questions designed to probe sentence recognition increased with the addition of background noise.

    Conclusions: The results demonstrate that the addition of background noise requires more cognitive resourcesto maintain speech recognition performance, leading to higher demands on the cognitive capacity,higher listening effort as measured by poorer memory performance, and longer AIST responsetimes. However, the type of background noise, SSN or ISTS, affected memory performance similarly.

    Keywords
    Speech-in-noise, Cognition, Working memory, Updating, Hearing impairment, Listening effort, Cognitive spare capacity
    National Category
    Clinical Medicine Health Sciences
    Identifiers
    urn:nbn:se:liu:diva-109680 (URN)
    Available from: 2014-08-25 Created: 2014-08-25 Last updated: 2017-11-06Bibliographically approved
    Download full text (pdf)
    Assessing cognitive spare capacity as a measure of listening effort using the Auditory Inference Span Test
    Download (pdf)
    omslag
  • 36.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Division of Neuroscience. Linköping University, Faculty of Health Sciences. Linköping University, The Swedish Institute for Disability Research.
    Rudner, Mary
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Lunner, Thomas
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, Department of Clinical and Experimental Medicine, Division of Neuroscience. Linköping University, Faculty of Health Sciences. Eriksholm Research Centre, Oticon A/S, Snekkersten.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Division of Neuroscience. Linköping University, Faculty of Health Sciences. Linköping University, The Swedish Institute for Disability Research.
    Assessing listening effort by measuring short-term memory storage and processing of speech in noise2014In: Speech, Language and Hearing, ISSN 2050-5728, Vol. 17, no 3, p. 123-132Article in journal (Refereed)
    Abstract [en]

    Objective

    The aim of this study was to investigate the effect of working memory capacity (WMC) and updating ability (UA) on listening effort measured using a new test, the Auditory Inference Span Test (AIST), as an objective measure of listening effort.

    Design

    The AIST is based on Swedish five-word sentences and taps into three memory load levels (MLLs). It was administered in stationary speech-shaped noise at −2, −4, and −6 dB signal-to-noise ratio (SNR). WMC was assessed using the reading span test, and UA was assessed using the letter memory test. We also collected data on speech-in-noise performance and subjectively rated listening effort at the three SNRs.

    Study sample

    Thirty-nine participants with normal hearing thresholds (≤20 dB HL for 250 to 4000 Hz) took part in the study.

    Results

    AIST performance decreased with increasing MLL and was related to WMC and UA. Participants with high WMC performed better than those with low WMC at more favorable SNRs. Participants with high UA performed better than participants with low UA at the intermediate MLL, which made particular demands on the UA. Neither speech recognition scores nor subjectively rated listening effort was associated with AIST performance or either of the cognitive variables.

    Conclusion

    AIST taps into cognitive functions necessary for understanding speech in noise. However, in its current form AIST may be too cognitively taxing to successfully measure graded listening effort in participants with lower cognitive capacity.

  • 37.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Division of Neuroscience. Linköping University, Faculty of Health Sciences. Linköping University, The Swedish Institute for Disability Research.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research.
    Lunner, Thomas
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Oticon Research Centre Eriksholm, Denmark.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Division of Neuroscience. Linköping University, Faculty of Health Sciences.
    Memory performance on the Auditory Inference Span Test is independent of background noise type for young adults with normal hearing at high speech intelligibility2014In: Frontiers in Psychology, E-ISSN 1664-1078, Vol. 5, no 1490Article in journal (Refereed)
    Abstract [en]

    Listening in noise is often perceived to be effortful. This is partly because cognitive resources are engaged in separating the target signal from background noise, leaving fewer resources for storage and processing of the content of the message in working memory. The Auditory Inference Span Test (AIST) is designed to assess listening effort by measuring the ability to maintain and process heard information. The aim of this study was to use AIST to investigate the effect of background noise types and signal-to-noise ratio (SNR) on listening effort, as a function of working memory capacity (WMC) and updating ability (UA). The AIST was administered in three types of background noise: steady-state speech-shaped noise, amplitude modulated speech-shaped noise, and unintelligible speech. Three SNRs targeting 90% speech intelligibility or better were used in each of the three noise types, giving nine different conditions. The reading span test assessed VVMC, while UA was assessed with the letter memory test. Twenty young adults with normal hearing participated in the study. Results showed that AIST performance was not influenced by noise type at the same intelligibility level, but became worse with worse SNR when background noise was speech-like. Performance on AIST also decreased with increasing memory load level. Correlations between AIST performance and the cognitive measurements suggested that WMC is of more importance for listening when SNRs are worse, while UA is of more importance for listening in easier SNRs. The results indicated that in young adults with normal hearing, the effort involved in listening in noise at high intelligibility levels is independent of the noise type. However, when noise is speech-like and intelligibility decreases, listening effort increases, probably due to extra demands on cognitive resources added by the informational masking created by the speech fragments and vocal sounds in the background noise.

  • 38.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research.
    Lunner, Thomas
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark.
    Frequency discrimination and human communication2013Conference paper (Other academic)
    Abstract [en]

    The voice is the most common means of communication and tends to change, sliding up and down the pitch scale when forming phonemes and words, as different emotions and thoughts are expressed. Therefore the ability to discriminate frequencies is of importance for speech intelligibility in a communication situation. Furthermore, this ability is also of importance in speech recognition in noise, by separating the target and noise by spectral and temporal differences of the sources. The choice of rehabilitation is crucial for the frequency discrimination ability. Cochlear implants, for example, lack the ability to pass the temporal fine structure of acoustic waves to the auditory nerve, which in turn lead to reduced precision of phase locking, inferior frequency discrimination ability, and a relatively poor ability to understand speech when background sounds are present.The aim of the study is to investigate how frequency discrimination and temporal resolution abilities interact with performance in speech recognition in noise using a psychoacoustic, speech, and cognitive test battery. These tests will give insight to interactions between performance and hearing status, type of rehabilitation(hearing aid, cochlear implant, and electro-acoustic stimulation), cognitive capacity, and language ability. It is hypothesized that normal hearing participants have a better frequency discrimination ability than hearing impaired participants and by that, better understanding of speech. It is also hypothesized that type of rehabilitation effects performance on frequency discrimination, and that this performance correlates with speech recognition in noise. Finally, it is hypothesized that cognitive capacity and language ability can, to some extent, compensate for loss of frequency resolution in the peripheral auditory system. Preliminary results from the study will be presented and discussed.

  • 39.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Lunner, Thomas
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark.
    Testing listening effort for speech comprehension2012In: Speech Perception and Auditory Disorders, Danavox Jubilee Foundation , 2012, p. 73-80Chapter in book (Other academic)
  • 40.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research.
    AIST - Ett test av lyssningsansträngning2011Conference paper (Other academic)
    Abstract [sv]

    Hörapparatsanpassning kan ses som en process för att minska en persons lyssningsansträngning, men det är oklart hur man bäst mäter lyssningsansträngning på ett objektivt sätt. Auditory Inference Span Test (AIST) syftar därför till att utvecklas till ett kliniskt instrument att använda vid hörapparatsanpassning för att mäta en patients ansträngning att uppfatta tal.

    AIST är ett kombinerat hörsel-, minnes- och bearbetningstest. Testet bygger på idén att ju mer kognitiva resurser som går åt för att bearbeta och förstå tal, desto mindre kognitiva resurser finns kvar för att minnas och lagra talinformation. Testet använder Hagermans meningar i brus, och försökspersonen behöver minnas och bearbeta informationen i talmaterialet för att kunna besvara frågor om innehållet. Poäng på frågorna samt reaktionstid mäts som mått på lyssningsansträngning. Data från pilottester visar på att AIST kan bli ett väl anpassat test för kliniskt bruk för att mäta lyssningsansträngning.

     

  • 41.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research.
    Lunner, Thomas
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences. Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research.
    An objective measure of listening effort: The Auditory Inference Span Test2011Conference paper (Refereed)
    Abstract [en]

    One aim of hearing aid fitting is to ease the patient’s effort in understanding speech, i.e. the listening effort needed to perceive speech in different sound environments. To obtain a good hearing aid fitting, knowledge about the patient’s auditory as well as cognitive abilities seems to be important. However, listening effort is usually not included as a fitting criterion, partly as it is not clear how to measure listening effort objectively.

    The Auditory Inference Span Test (AIST) is a dual-task hearing-in-noise test, that combines auditory and memory processing. The basis for the test is that when more cognitive resources are required for understanding speech, less cognitive resources are available for storage and processing of the speech information. In AIST, Hagerman sentences are presented in noise and the subject is required to recall and process the sentence information. Recall ability is tested with different cognitive loads. Button-press responses are recorded and used as an estimate of listening effort. In a pilot study, listeners showed decreasing accuracy with increasing cognitive load and longer reaction time at maximum cognitive load, suggesting that the test may be suited as a clinical test for listening effort.

    In an ongoing study, the AIST is being evaluated in relation to other auditory and cognitive measures: baseline audiometry (audiogram) and speech in noise test (Hagerman sentences) as well as text based dual processing and storage test (reading span) and updating (letter memory test), as well as subjective rating of listening effort. Data from this study will be presented.

  • 42.
    Rudner, Mary
    et al.
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research.
    Ng, EH
    Rönnberg, Niklas
    Linköping University, Department of Clinical and Experimental Medicine, Division of Neuroscience. Linköping University, Faculty of Health Sciences.
    Mishra, Sushmit
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research.
    Rönnberg, Jerker
    Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research.
    Lunner, Thomas
    Linköping University, Faculty of Arts and Sciences. Linköping University, Department of Clinical and Experimental Medicine. Linköping University, Faculty of Health Sciences. Eriksholm Research Centre, Snekkersten, Denmark.
    Stenfelt, Stefan
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Clinical and Experimental Medicine. Linköping University, Faculty of Health Sciences.
    Cognitive spare capacity as a measure of listening effort2011In: Journal of Hearing Science, ISSN 2083-389X, Vol. 1, no 2, p. EA47-49Article in journal (Refereed)
    Abstract [en]

    There has been a recent interest in listening effort as a factor to be taken into account in the audiological clinic. However, the term “listening effort” is poorly determined and needs to be defined before it can be used as a clinical or research tool. One way of understanding listening effort is in terms of the cognitive resources expended during listening. Cognitive capacity is finite and thus if cognitive capacity is used up during the act of listening to speech there will be fewer cognitive resources left to process the content of the message conveyed. We have introduced the term Cognitive Spare Capacity (CSC) to refer to residual cognitive capacity once successful listening has taken place. This extended abstract described the work we have carried out to date on measures of CSC for research and clinical use. In the course of this work we have developed tests to assess the role of memory load, executive function and audiovisual integration in CSC under challenging conditions. When these tests are fully developed, our aim is that they should allow objective individual assessment of listening effort in cognitive terms. Results to date indicate that under challenging conditions, CSC is an arena for executive processing of temporarily stored information; it is related to individual working memory capacity and can be enhanced by hearing aid signal processing.

  • 43.
    Rudner, Mary
    et al.
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Ng, Hoi Ning, Elaine
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Rönnberg, Niklas
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Mishra, Sushmit
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Rönnberg, Jerker
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Lunner, Thomas
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Stenfelt, Stefan
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Cognitive spare capacity as a measure of listening effort2011In: Journal of Hearing Science, ISSN 2083-389X, Vol. 1, no 2, p. 47-49Article in journal (Refereed)
    Abstract [en]

    There has been a recent interest in listening effort as a factor to be taken into account in the audiological clinic. However, the term “listening effort” is poorly determined and needs to be defined before it can be used as a clinical or research tool. One way of understanding listening effort is in terms of the cognitive resources expended during listening. Cognitive capacity is finite and thus if cognitive capacity is used up during the act of listening to speech there will be fewer cognitive resources left to process the content of the message conveyed. We have introduced the term Cognitive Spare Capacity (CSC) to refer to residual cognitive capacity once successful listening has taken place. This extended abstract described the work we have carried out to date on measures of CSC for research and clinical use. In the course of this work we have developed tests to assess the role of memory load, executive function and audiovisual integration in CSC under challenging conditions. When these tests are fully developed, our aim is that they should allow objective individual assessment of listening effort in cognitive terms. Results to date indicate that under challenging conditions, CSC is an arena for executive processing of temporarily stored information; it is related to individual working memory capacity and can be enhanced by hearing aid signal processing.

  • 44.
    Larsby, Birgitta
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rönnberg, Niklas
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    STAF-dagar i Uppsala: Mycket om kopplingen audiologi och otokirurgi2011In: Audionytt, ISSN 0347-6308, Vol. 38, no 3, p. 31-33Article in journal (Other (popular science, discussion, etc.))
  • 45.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Lunner, Thomas
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Health Sciences.
    Testing listening effort for speech comprehension2011Conference paper (Other academic)
    Abstract [en]

    One aim of hearing aid fitting is to reduce the effort of understanding speech, especially in noisy environments. For a good hearing aid fitting, knowledge about the patient’s auditory abilities is necessary, but knowledge about cognitive abilities may also be important.

     

    The Auditory Inference Span Test (AIST) is a dual-task hearing-in-noise test, that combines auditory and memory processing. In AIST, Hagerman sentences are presented in steady state speech-shaped noise at -2dB, -4dB or -6dB SNR and the subject is required to recall and process the information from the sentences by giving button-press responses to multiple-choice questions thereby assessing what the subject could infer from what was heard.

     

    AIST will be administered to 40 normal hearing subjects (29 to date) and performance related to speech reception threshold, working memory capacity and updating ability, as well as subjective rating of listening effort. Preliminary results show a greater SNR-related improvement in AIST scores at low SNRs than can be explained by improved audibility alone, consistent with release of memory resources due to reduced listening effort. There is also a trend towards a positive relationship between AIST scores and individual working memory capacity and updating ability.

  • 46.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Testing listening effort for speech comprehension using the individuals’ cognitive spare capacity2011In: Audiology Research, ISSN 2039-4330, E-ISSN 2039-4349, Vol. 1, no 1, p. 82-85Article in journal (Refereed)
    Abstract [en]

    Most hearing aid fittings today are almost solely based on the patient’s audiogram. Although the loss of gain in the cochlea is important, for a more optimal fitting, more individual parameters of the patient’s cochlear loss together with the patient's cognitive abilities to process the auditory signal are required (Stenfelt & Rönnberg, 2009; Edwards, 2007). Moreover, the evaluation of the fitting is often based on a speech in noise task and the aim is to improve the individual patient’s signal to noise ratio (SNR) thresholds. As a consequence, hearing aid fitting may be seen as a process aimed to improve the patient’s SNR threshold rather than to improve communication ability. However, subsequent to a hearing aid fitting, there can be great differences in SNR improvement between patients that have identical hearing impairment in terms of threshold data (the audiogram). The reasons are certainly complex but one contributing factor may be the individual differences in cognitive capacity and associated listening effort. Another way to think about amplified hearing is to ease a subject’s listening effort (Sarampalis, et al., 2009). When the speech signal is degraded by noise or by a hearing impairment, more high-order cognitive or top-down processes are required to perceive and understand the signal, and listening is therefore more effortful. It is assumed that a hearing aid would ease the listening effort for a hearing impaired person. However, it is not clear how to measure the listening effort. We here present a test that will tap into the different cognitive aspects of listening effort, the Auditory Inference Span Test (AIST). The AIST is a dual task hearing in noise test, that combines auditory and memory processing and is well suited as a clinical test for listening effort.

  • 47.
    Rudner, Mary
    et al.
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Ng, Hoi Ning, Elaine
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Rönnberg, Niklas
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Mishra, Sushmit
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Rönnberg, Jerker
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Lunner, Thomas
    Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research. Linköping University, Faculty of Arts and Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Understanding auditory effort by measuring cognitive spare capacity2011Conference paper (Other academic)
  • 48.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research.
    Testing effort for speech comprehension using the individuals’ cognitive spare capacity - the Auditory Inference Span test2010Conference paper (Other academic)
    Abstract [en]

    Modern hearing aids use a multitude of parameters to give the user an optimal speech signal. Fitting of the hearing aid becomes a handiwork due to the limited data of the patients hearing status (primarily an audiogram). A hearing in noise test (SNR threshold) is often used to evaluate the fitting. However, testing the SNR threshold as done in clinical use today is not ecological valid. Another way to think about hearing aid fitting is to ease the listening effort.

    Therefore, we propose the Auditory Inference Span Test (AIST) as a clinical tool during hearing aid fitting to assess the patient’s effort to understand speech. AIST is a combined auditory, memory, and processing test. It relies on the idea that the more cognitive resources that are required to process and understand speech, less cognitive resources are available for storage of the speech information. In AIST, sentences are presented in noise and afterwards the patient is required to recall and process the information from the sentences. Correctness and answering speed is measured and scores correlate to the effort required to understand the speech.

    Data from piloting tests indicate that the AIST is well suited as a clinical test for listening effort.

  • 49.
    Niklas, Rönnberg
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology . Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology . Linköping University, Faculty of Health Sciences.
    Testing effort for speech comprehension using the individuals’ cognitive spare capacity – the Auditory Inference Span Test2010In: Adult Hearing Screening 2010, 2010Conference paper (Refereed)
  • 50.
    Rönnberg, Niklas
    et al.
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Stenfelt, Stefan
    Linköping University, Department of Clinical and Experimental Medicine, Technical Audiology. Linköping University, Faculty of Health Sciences.
    Rudner, Mary
    Linköping University, Department of Behavioural Sciences and Learning, Cognition, Development and Disability. Linköping University, Faculty of Arts and Sciences. Linköping University, The Swedish Institute for Disability Research. Linköping University, Department of Behavioural Sciences and Learning, Disability Research.
    The Auditory Inference Span Test – Developing a test for cognitive aspects of listening effort for speech comprehension2010Conference paper (Other academic)
    Abstract [en]

    Most hearing aid fittings today are almost solely based on the patient’s audiogram. However, more individual parameters of the patient’s hearing thresholds together with the patient’s cognitive abilities to process the auditory signal are required. Hearing aid fitting may be seen as a process aimed to improve the patient’s hearing thresholds rather than to improve communication ability. Another way to think about hearing aid fitting is to ease the patient’s listening effort. However, it is not clear how to measure the listening effort.

    Therefore, we propose the Auditory Inference Span Test (AIST) as a clinical tool during hearing aid fitting to assess the patient’s effort to understand speech. AIST is a combined auditory, memory, and processing test. It relies on the idea that the more cognitive resources that are required to process and understand speech, less cognitive resources are available for storage of the speech information. In AIST, sentences are presented in noise and afterwards the patient is required to recall and process the information from the sentences. Correctness and reaction time is recorded as measurements of perceived listening effort.

    Data from piloting tests indicate that the AIST is well suited as a clinical test for listening effort. In a future study to verify that the AIST is sensitive to cognitive capacity, the test will be evaluated with measurements of the subject's cognitive capacity as well as the subject's hearing thresholds. For a clinical test the requirement is that it is fast and easily facilitated. The AIST takes no more than fifteen minutes to complete, and the aim is to further shorten the time and adapt the test for clinical use. This ensures the AIST to be a useable instrument for testing listening effort using the individuals' cognitive spare capacity.

     

12 1 - 50 of 51
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf