Investigating Listening Effort with an Eye to Real-World Benefit

Phonak has a legacy of focusing on purposeful innovation. We’re continually researching how to best support our users with technology that delivers real-world benefits. In this interview, we discuss listening effort and the impact of research-backed hearing technology with Dr. Elizabeth Stewart, Senior Research Audiologist in the Phonak Audiology Research Center (Sonova US).

 

AudiologyOnline: We’re hearing more about listening effort and hearing aids. Why is that?

Elizabeth Stewart, AuD, PhD: We know that listening effort is a primary complaint among individuals with hearing loss – it takes effort to understand speech. When high levels of listening effort must be sustained throughout the day, it can accelerate mental fatigue, which can have consequences for quality of life, general health, and wellbeing.^1-3 Listeners may avoid or withdraw from social situations because it is too hard to understand speech, causing them to become isolated—the opposite of what we want for our patients. When hearing technology reduces listening effort, it frees up mental resources that can be devoted to other things besides just listening, making it less likely that they will become overwhelmed and give up on trying to engage. So reduced listening effort can really make a difference to patient experience.

AudiologyOnline: That makes sense! So how do you test listening effort?

Elizabeth Stewart, AuD, PhD: Actually, listening effort is an indirect outcome, which means that we cannot directly measure it, but we can measure responses in several domains that have been found to be sensitive to changes in listening effort. We could use a cognitive-behavioral measure such as a dual-task paradigm. Brain-related measures including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) can be used to make inferences about listening effort. Pupillometry, skin conductance, and heart rate variability can tell us about autonomic nervous system activity reflecting changes in listening effort. Finally, self-report measures of listening effort can be collected in the clinic or lab via rating scales and questionnaires, or even in the field using Ecological Momentary Assessment (EMA). No measure is perfect, but by using several measures that show consistent outcomes, researchers can generate credible and reliable evidence.

AudiologyOnline: Can you give me an example?

Elizabeth Stewart, AuD, PhD: In a clinical setting, self-reports are happening daily, often spontaneously, when patients report how exhausted they are because listening is so effortful. In a lab setting, these anecdotal reports can be captured in a more organized manner using visual-analog scales asking patients how effortful it is to understand speech presented through loudspeakers. One method of this is the Adaptive Categorical Listening Effort Scaling,⁴ (ACALES; see Figure 1). This task is a measure of subjective listening effort that yields a signal-to-noise ratio (SNR), so it offers a way of quantifying perceived listening effort and comparing across conditions. We might combine this with simultaneous EEG measures to collect additional, objective data about expended effort within the same group of study participants. Or, we could perform this test in the lab and compare the results with reports from participants during a field trial, collected using EMA.

Figure 1. 

AudiologyOnline: I imagine that all of these studies have to do with speech in noise, right?

Elizabeth Stewart, AuD, PhD: While noisy environments are the clearest example of where listening is effortful—and we’ll talk about them next—even quiet situations may be challenging enough to require additional cognitive resources. Think of soft speech or speech at a distance where you have to strain to hear the talker, even without competing noise. Speech Enhancer was designed to address these situations by giving a boost to soft voices in quiet.

We’ve tested Speech Enhancer with conditions of distant speech and speech from an adjacent room, showing subjective listening effort is reduced up to 45% when Speech Enhancer is active.⁵ And then we took it a step further investigating the effect of Speech Enhancer on listening-related fatigue by simulating a typical auditory day and condensing it into a 2.5 hour session in the lab (Time-Compressed Auditory Day, or TCAD). Results showed that Speech Enhancer reduces mental fatigue by up to 21% compared to not using the feature (Figure 2).⁶ When we think of how the consequences of sustained listening effort affect quality of life, these are exciting results.

Figure 2.

AudiologyOnline: You said we would talk about noise. What example do you have for that?

Elizabeth Stewart, AuD, PhD: Our newest feature, Spheric Speech Clarity (SSC), is a deep-neural-network (DNN) denoising system that extracts the speech, enhances it, and integrates it back into the signal, delivering clear speech from any direction. When assessing the benefits of SSC for subjective listening effort, results of the ACALES test showed that SNRs were, on average, 2.9 dB more challenging with SSC than without it. This suggests that SSC allows listeners to tolerate less favorable listening conditions without an increase in their perceived listening effort.⁷

And what’s even more encouraging is that these results align with patient reports. One newly fit Infinio Sphere user shared that she commented to her family as they were leaving a restaurant that she was disappointed it wasn’t noisier so she could really try out her new hearing aids. When her family assured her that the restaurant was, in fact, very noisy, she was stunned because she didn’t feel she had to work any harder to hear.

This is the difference that features shown to reduce listening effort can make in patients’ lives and the reason we at Phonak will continue to focus on research-backed innovation with real world benefits.

References

1. Pichora-Fuller, M. K., Kramer, S. E., Eckert, M. A., Edwards, B., Hornsby, B. W., Humes, L. E., Lemke, U., Lunner, T., Matthen, M., Mackersie, C. L., Naylor, G., Phillips, N. A., Richter, M., Rudner, M., Sommers, M. S., Tremblay, K. L., & Wingfield, A. (2016). Hearing impairment and cognitive energy: the Framework for Understanding Effortful Listening (FUEL). Ear and Hearing, 37(Suppl 1), 5S-27S. https://doi.org/10.1097/AUD.0000000000000312
2. McGarrigle, R., Munro, K. J., Dawes, P., Stewart, A. J., Moore, D. R., Barry, J. G., & Amitay, S. (2014). Listening effort and fatigue: what exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group 'white paper'. International Journal of Audiology, 53(7), 433-440. https://doi.org/10.3109/14992027.2014.890296
3. Winn, M. B., & Teece, K. H. (2021). Listening effort is not the same as speech intelligibility score. Trends in Hearing, 25, 23312165211027688. https://doi.org/10.1177/23312165211027688
4. Krueger, M., Schulte, M., Brand, T., & Holube, I. (2017). Development of an adaptive scaling method for subjective listening effort. The Journal of the Acoustical Society of America, 141(6), 4680. https://doi.org/10.1121/1.4986938
5. Habicht, J., & Schuepbach-Wolf, M. (2024). Speech Enhancer reduces subjective listening effort of speech by up to 45%. Phonak Field Study News retrieved from www.phonak.com/evidence
6. Latzel, M., Heeren, J., & Lesimple, C. (2024). Speech Enhancer reduces listening effort and fatigue. Phonak Field Study News retrieved from www.phonak.com/evidence
7. Wright, A., Kuehnel, V., Keller, M., Seitz-Paquette, K., & Latzel, M. (2024). Spheric Speech Clarity applies DNN signal processing to significantly improve speech understanding from any direction and reduce the listening effort. Phonak Field Study News retrieved from  www.phonak.com/evidence