Lab to Real World: Improving an Algorithm to Boost Hearing
Boosted user acceptability by 15% over baseline while preserving intelligibility gains, moving the feature from R&D to ship.
Baseline: the previous algorithm.
The new algorithm improved intelligibility in the lab, but its launch depended on whether users accepted how it sounded in the real world.
The research team had developed a hearing assistance algorithm that improved an in-lab objective metric, intelligibility, over the base algorithm. To be ship-ready, its real-world performance had to be checked and balanced against a subjective experience metric: perceived sound quality.
My ownership
Designed the study, collected and analyzed the data, and presented the results
Iterated the study across four rounds in close collaboration with the research scientist
Turned study insights into algorithm refinement recommendations
To ship, it had to beat the older algorithm on acceptability, not just intelligibility. Where it fell short, we had to know why.
Quantitative: quantify the user acceptability of the new algorithm over other algorithms.
Qualitative: identify areas of improvement that could raise acceptability.
Acceptability
Does the new algorithm get higher acceptability ratings than the older algorithm?
Soundscape effect
Do acceptability ratings change as a function of soundscape?
Sound quality issues
What sound-quality issues do participants bring up?
Where issues occur
In what soundscapes do those issues come up?
In-the-moment surveys, followed by interviews and surveys in a simulated real-world setting, across four iterations.
Quick ratings during listening showed whether each algorithm was acceptable; interviews afterwards explained why.
Participant profile
- 30 participants per round
- Clinically determined mild to moderate hearing loss
- Ages 18–76
- No eyesight problems; non-glasses wearers
- Pre-session audiometry for fitting
Methodology
Yes/no, several per noise level, without breaking the conversation
Standardized surveys after each algorithm and at exit
After each algorithm and at exit, on why it felt better or worse
Analysis
prop.test, Bayesian modeling → stacked yes/no bars
Friedman, Cochran’s Q, effect sizes → boxplots, proportion bars
Lexicon → content analysis → triangulated with surveys
Core session flow outline for each round
Within-subjects: every participant experienced every algorithm
Vendor-owned simulated spaces · spatial positioning varied (near/far or static/moving)
Capped sessions at 120 minutes, with breaks between blocks
In-moment surveys instead of end-of-session recall
Changed how responses worked: from touch input on the smartwatch to more tactile input
Changed the lab setup to keep data collection reliable
Four algorithm iterations to clear the acceptability bar without sacrificing intelligibility.
The bar: beat the older algorithm on acceptability while preserving its intelligibility gains.
Clear isn’t comfortable
The fix made voices muffled
Complaints eased, bar not met
Bar cleared in every noise level
My role: Sound-quality lexicon → content analysis for sound-quality issues → triangulated with Likert surveys → tuning recommendations
- Tuned 3 parameters under NDA
- Sound-quality measurement
- In-lab objective measurement with each round’s algorithm change
- Personalized outfitting with REM to reduce sound-quality issues
- Advanced amplification stakeholder request
The algorithm moved from lab to productization, opening the next wave of research.
- Lab to shipAfter the fourth round, found the optimal balance of algorithm parameters and recommended technology transfer to productization.
- Real-world studyThe study’s success led to a fully real-world study alongside productization, co-designed with research scientists: 50 participants, with 6+ per session.
- Next-generation researchProposed and co-designed, with research scientists, a study on adapting to environment sound type instead of noise level.
My sound-quality lexicon is now used team-wide; next time, I’d bring the statistician in from day one.
What worked
- The lexicon for describing sound-quality issues is now used team-wide.
- Successful hand-off and smooth switching of survey instruments between studies.
What I’d do differently
- Recruit the statistician first. Frequentist tests weren’t sensitive enough to the differences, so we relied on Bayesian modeling of in-the-moment (EMA) responses.
- Avoid an excessive deep dive into one parameter that became a spin-off study; earlier statistical input would have saved it.
*Illustrative visuals. Study data under NDA.