From Frustrated and Discouraged to Confident and Content: Smartphone Captioned Call Experiences with the InnoCaption App

Learner Outcomes

• After this course, participants will be able to describe the efficacy of smartphone call captioning apps in reducing communication barriers for individuals with self-rated significant hearing/communication challenges.
• After this course, participants will be able to identify the psychological and social benefits associated with the use of real-time captioning technology in everyday communication using their smartphone.
• After this course, participants will be able to describe how to obtain and use this important form of hearing assistive technology when speaking on their smartphones in a range of situations.

Smartphone call captioning is the latest in assistive technology available to persons with hearing loss who may or may not use hearing aids or cochlear implants. This course reviews InnoCaption as a solution for individuals with hearing loss, including a study that demonstrates it can improve the experience of using a smartphone to the point that quality of life and well-being have been positively impacted.

InnoCaption is a mobile application that provides real-time captioning of phone calls. The captioning is unique because it is provided by best-in-class automated speech recognition (ASR) technology and/or live stenographers (Communication Access Realtime Translation, or CART providers). Individuals in the United States with a self-described hearing loss can use the app at no cost because InnoCaption is certified and funded by the Federal Communications Commission (FCC).

InnoCaption was developed by two Co-CEOs whose partnership led to the launch of the first mobile-focused captioned phone service. This passionate, purpose-driven team is on a mission to provide an empowering accessibility solution for the deaf and hard-of-hearing community. InnoCaption initially focused solely on smartphone technology at a time when the industry mainly offered landline phone solutions. While the app utilizes live stenographers for faster and more accurate captioning despite the higher cost, it is the only captioned phone service provider to offer users a choice between automatic speech recognition technology or live CART.

The InnoCaption app offers several key features to enhance the user experience.

• Keep Existing Phone Number: The patented One Number System links the user's existing number to the InnoCaption service. Users can also set up call forwarding to have calls to their existing smartphone number captioned by the app.
• Dual Caption Modes: Real-time captioning is provided through both live stenographers and automated speech recognition software. A unique feature is that InnoCaption puts the user in control, with the ability to switch between modes before or even during calls.
• FCC Certified and HIPAA Compliant: The app is FCC certified and committed to HIPAA compliance to ensure the protection of health information. Although the app does not store electronic Protected Health Information (ePHI), the compliance efforts, which apply HIPAA standards to security measures, are designed to provide peace of mind for individuals in health-related fields and are encouraged for use when speaking with physicians.
• Two-Sided Captions: Users can choose whether they would like captions of only the other party's speech or their own speech as well. Users can switch between one- or two-sided captions at any time, even during a call, by tapping the three-dot menu button.
• Review Past Call Transcripts: Users can view past call transcripts, helping them retain important call details. Transcripts can easily be saved or emailed to the user.
• Captioned Voicemail: The app provides full preview transcripts of voicemail messages generated by the automated speech recognition engine. Users can also have their voicemail messages re-captioned by live captioners if needed.

Adoption of smartphone technology has skyrocketed over the past decade, with the gap between use by young adults and older adults narrowing. A recent Pew Report (2022), covering 2011 to 2021, revealed that smartphone ownership by older adults (aged 50 and above) has risen from 10% to 61% in just 10 years.

Despite this surge in usage, a recent MarkeTrak report (Picou, 2022) highlighted that more than half (53%) of people with hearing loss who did not use hearing aids were dissatisfied with communicating over a cell phone. Even with efforts to ensure hearing aids or implants are streaming to the phone, individuals with hearing aids still report experiencing communication challenges. This increase in smartphone use among older adults, paired with high levels of dissatisfaction among people with hearing loss, signals a significant opportunity for more effective communication, especially distance communication with friends and family.

The Survey Had Four Main Goals

1. Assess overall InnoCaption user experience and frequency of usage.
2. Explore personal feelings associated with smartphone use with and without InnoCaption, including feelings before starting and after using the app.
3. Identify situations in which smartphone use is a priority.
4. Examine if the use of InnoCaption impacts the listening experience, specifically focusing on the psychosocial experiences people reported with the call captioning app.

A baseline survey was sent to approximately 3,606 new InnoCaption users starting in September 2022. Two reminder messages followed the initial invitation to participate. A follow-up survey was sent three months after the baseline survey. The final data analysis was based on 131 respondents who completed both surveys, which represented a 10% response rate - considered good according to Qualtrics.

The question of why patients would want to use a call captioning app with their smartphones has a clear answer: Visual input with captioning compensates for and supplements distortions associated with hearing loss.

Research has shown that when watching television, individuals benefit more from the combined use of closed captioning and hearing aids than from hearing aids alone. It is projected that by combining both sensory functions (auditory and visual), listening effort and listening fatigue would likely be reduced. Moreover, smartphone and tablet use is critical for engagement, especially for people living alone, and is important in the workplace and when communicating with healthcare professionals.

Survey Demographics

The survey respondents exhibited a nice, even split in terms of gender. Women represented 53% of respondents, and men 44%.

The majority of respondents were over 60 years of age. Specifically, 64% of respondents were 61 years of age and older. The age distribution showed a good representation of older adults: 48% were 61–79 years old, and 16% were 80 and above.

Regarding work status, the majority of individuals (61%) were not working or retired. However, 23% were full-time workers and 16% were part-time workers.

Hearing Technology Use

At baseline, a large proportion of respondents were using hearing aids (72%). Additionally, 6% had a cochlear implant (CI) in one or both ears, and 7% were bimodal users (CI + hearing aid). Notably, 13% of respondents certified as having a hearing loss but did not use hearing aids. The follow-up survey saw a slight shift, with 69% using hearing aids and an increase in CI users (10%) and bimodal users (7%), as some people reported getting cochlear implants during the three-month period.

Figure 1: Hearing Technology Use at Baseline and Follow-up Survey. Hearing aids were the most common technology, with 72% at baseline and 69% at follow-up, while 13% at baseline reported having a hearing loss but using no technology.

Survey Results

When asked to self-rate their degree of difficulty hearing or understanding without any hearing technology assistance (e.g., without hearing aids or cochlear implants) on a scale of 1–10, the majority of the sample reported significant difficulty. A considerable number of people rated their difficulty as an 8, 9, or 10, indicating extreme difficulty. Overall, 76% of respondents had a self-reported hearing difficulty of 8 to 10.

Figure 2: The majority of respondents reported extreme difficulty (ratings 8–10), which accounted for 76% of the sample.

Listening Effort

Listening effort was a key measure of the psychosocial impact evaluated in the study, as it is key to psychosocial function. While some improvement was expected, the dramatic reported decrease in effort was pleasantly surprising.

Respondents were asked to rate their level of effort when communicating on their smartphone without using InnoCaption. Even with streaming from hearing aids or implants, the average listening effort rating was 7.1 on a scale of 0 (No Effort) to 10 (Maximum Effort).

In the follow-up survey, when rating their level of effort when making calls with the InnoCaption app, the average score showed a dramatic shift, decreasing to 3.6. The shift was overwhelmingly downward, indicating that respondents used much less effort when communicating with the call captioning app. This was considered a very dramatic and reassuring finding.

Figure 3: Comparison of Self-Rated Level of Effort When Making Calls Without and With InnoCaption. The average effort rating dropped dramatically from 7.1 (without InnoCaption) to 3.6 (with InnoCaption).

Listening Fatigue

Listening fatigue and listening effort are related. Without using the InnoCaption app at baseline, individuals reported experiencing listening fatigue when communicating on their smartphone. The average fatigue rating was 6.7 on a scale of 0 (No Fatigue) to 10 (Extreme Fatigue).

After using the InnoCaption app, the average level of fatigue dropped to 3.0. This was another very dramatic shift, with a high percentage of people reporting no fatigue in the follow-up survey.

The reduction in listening effort during phone calls led to a similarly dramatic decrease in fatigue. Less cognitive effort is required when combining visual input with auditory input. The data support that people with more severe hearing difficulty experience listening fatigue and effort. Prior to using captioning with smartphones, 62% of respondents experienced considerable listening fatigue, whereas only 14% indicated they were experiencing fatigue while using captions. Listening fatigue is linked to self-rated hearing difficulty and well-being. It takes effort to remain focused while communicating, and listening fatigue can contribute to social withdrawal.

Shift in Stress-Related Emotions Associated with Making Smartphone Calls

The study looked for a shift in stress-related emotions by asking individuals to rate their psychosocial experiences when making calls without the app and then when using the app.

In the baseline survey, a majority of respondents felt frustrated, anxious/stressed, and discouraged when making calls without InnoCaption.

After three months of using InnoCaption, respondents showed a significant decline in these negative emotions and an overwhelmingly positive shift to emotions such as confident, happy, and content. The decline in negative emotions was dramatic. Some of the most dramatic shifts in emotional response came from people with the highest self-reported levels of hearing difficulty. This strongly suggests the app is making a difference in the well-being of users.

Figure 4: Shift in Emotions When Making Phone Calls: Comparison of Baseline (without InnoCaption) and Follow-up (with InnoCaption). Negative emotions like frustration and anxiety/stress declined significantly, while positive emotions like confidence, happiness, and contentment saw a dramatic increase after three months of using the app.

Situational Uses

Given the psychosocial benefits observed, a greater use of the app across various situations was expected. Survey respondents indicated using their smartphones to make calls in a variety of settings. The top three situations were conversations with family and friends, making medical appointments, and making other personal appointments or reservations.

While users always used it to converse with family and friends, a greater percentage used the call captioning app and their smartphones when making medical appointments with physicians. There was also an increase in use for calling for Medicare/certain benefits and in the use of telehealth. Since the app is HIPAA compliant, healthcare professionals should recommend this to older adults, especially as use of telehealth and Medicare increases, to ensure they can hear and understand their physicians.

After three months of using InnoCaption, more respondents indicated making calls to healthcare and benefits providers. There was a +25% increase in calls for medical appointments with physicians, a +8% increase in calls for Medicare/certain benefits, and a +14% increase for telehealth appointments.

The study also looked at whether the use of the smartphone call captioning app impacted the overall frequency of telephone use and, in turn, engagement with others. Again, a dramatic change was observed. Before using the smartphone call captioning app, a significant number of people reported that they rarely ever spoke on the phone.

After using InnoCaption for three months (follow-up survey), there was a significant increase in the overall frequency of making phone calls and engagement with others. The percentage of people who reported using their smartphone for phone calls frequently increased by +18%, and those reporting sometimes increased by +11%. Correspondingly, the number of people who said they rarely ever used their smartphone for calls saw a substantial decrease of -65%.

Key Takeaways

The goal of assistive technology is to improve communication and connections, which are essential to human existence, and this is particularly important among older adults, given the loneliness epidemic.

Smartphone captions should be considered a form of assistive technology and recommended as an adjunct to hearing aid users. Professionals should emphasize the importance of asking patients how they are managing with their smartphones and the telephone, as people are having difficulty even with streaming hearing aids, and this technology can make a big difference.

The study objectives to explore the psychosocial benefits of improved communication via captioning and to understand the impact of captioning technology on emotional well-being and connection were achieved. The reduction in listening fatigue and listening effort directly communicates that emotional well-being is positively impacted. The captioning app was found to be efficacious in a number of different situations, given the variety of settings in which people reported using the smartphone and the call captioning.

The InnoCaption Difference

InnoCaption is the only captioned phone service provider that empowers users to choose between best-in-class automated speech recognition technology and live stenographers (CART providers). Users can switch between captioning technologies even during a call at the touch of a button.

Frequently Asked Questions

Who can use the service?

Anyone in the United States with hearing loss who needs captions to understand their phone calls.

Is an audiogram or a signed statement from a healthcare professional required?

No, users registering for the service are only required to self-certify that they have a hearing loss and need captions to understand phone calls.

Is there any cost to using the app?

No, there is no cost for eligible users because InnoCaption is certified by the Federal Communications Commission (FCC) and receives compensation from the Telecommunications Relay Service (TRS) fund.

Audiologists can download a demo of the app, and it is highly recommended to demonstrate it to their patients to ensure they understand how to use it.

References

Gordon-Salant, S., & Callahan, J. S. (2009). The benefits of hearing aids and closed captioning for television viewing by older adults with hearing loss. Ear and Hearing, 30(4), 458–465. doi: 10.1097/AUD.0b013e3181a26ef4.

Holman, J., Hornsby, B., Bess, F., & Naylor, G. (2021). Can listening-related fatigue influence well-being? Examining associations between hearing loss, fatigue, activity levels and well-being. International Journal of Audiology, 60(sup2), 47–59. doi: 10.1080/14992027.2020.1853261.

Pew Research Center. (2022). Share of those 65 and older who are tech users has grown in the past decade. Pew Research Center Short Reads. https://www.pewresearch.org/short-reads/2022/01/13/share-of-those-65-and-older-who-are-tech-users-has-grown-in-the-past-decade/

Picou, E. (2022). Hearing aid benefit and satisfaction results from the MarkeTrak 2022 Survey: Importance of features and hearing care professionals. Thieme Open Access. Article published online: 2022-12-01.