People with a hearing loss often struggle to hear speech at larger gatherings even when wearing hearing aids. To improve hearing aids, more information is needed to understand the processes underlying the ability to perceptually segregate sounds from different sources.
Earlier research suggested that the ease of segregating sounds from competing sound sources is related to the salience of the perceptual differences between the sounds. However, only little evidence has previously been provided to support this idea. In this study, we explored the effect of pitch differences and the effect of pitch salience (strength) on sound segregation abilities. Pitch is the perceptual attribute that allow us to order sounds on a scale from high to low and when varied can produce recognisable melodies. We found that segregation increased with increasing difference in pitch of different sounds and with increasing pitch salience. Moreover, we found a relationship between sound segregation abilities of the individual listeners and their ability to discriminate small differences in pitch. Both findings provide evidence that that sound segregation is facilitated by the salience of perceptual differences. Further evidence was provided by a PhD project, co-supervised by the grant recipient investigating sound segregation in CI users.
A common complaint of hearing-impaired (HI) people – despite wearing hearing instruments – is that they find it very difficult to understand speech at social gatherings and in other noisy situations and that their enjoyment of music is strongly reduced.
The cocktail party problem
In Denmark, more than 30% of people aged 65 years and about 50% of people aged 70 years have a hearing loss. A common complaint of hearing-impaired (HI) people – despite wearing hearing instruments – is that they find it very difficult to understand speech at social gatherings and in other noisy situations and that their enjoyment of music is strongly reduced. Such difficulties can lead to social isolation, which partly accounts for the fact that depression is more common among HI than among normal-hearing (NH) people. The core reasons underlying HI people’s problems with speech communication and music perception are currently not understood. There is a great need for basic research on identifying the deficits in the processing and perception of sound, which are the root cause of these problems.
Earlier research suggested that the ease of segregating sounds from competing sound sources is related to the salience of the perceptual differences between the sounds. However, only little evidence has previously been provided to support this idea.
Subjective sequential stream segregation
Both speech and music are complex sounds, which makes it difficult to analyse the relation between a specific hearing deficit and its consequences for spectral, temporal and spatial perception. Research on sound segregation therefore often uses simpler stimuli that can more easily be controlled and varied systematically. The present study, that was supported by a Carlsberg Foundation Postdoctoral Fellowship in Denmark, used a paradigm that has been employed in many earlier sound segregation studies1. As illustrated in Fig. 1A, the stimuli consisted of ABA_ABA…. sequences where A and B are different sounds (e.g. tones with different fundamental frequency, F0) and “_” indicates a brief pause. These sequences are perceived as having either a galloping rhythm if all tones are perceived as belonging to one stream (integrated), or as a stream of A tones and a stream of B tones (segregated) - the A stream having double the tempo of the B stream. The listeners were asked to indicate by pressing a key on the keyboard when their perception changed between one and two streams. The experiments were carried out in a sound-proof booth and the sounds were presented through headphones.
Figure 1: A) Illustration of the ABA_ABA.. sequences used for the sound segregation experiment. B) Schematic Illustration of the spectrum of the complex tones used. The two examples illustrate a tone with high pitch salience (low harmonic rank) and one with low pitch salience (high harmonic rank). Figures was previously presented in a published paper2.
Pitch salience and its effect on sound segregation in normal-hearing listeners
Pitch is the perceptual attribute of sound that can be ordered from high to low and when varied can create recognisable melodies. The perceived pitch of a harmonic tone is equal to the fundamental frequency, F0, even when the F0 is missing. However, tones containing components close to the F0 in frequency, i.e. with low harmonic rank, elicit a stronger sensation of pitch (salience) than tones with only frequency components far from the F0 (Fig. 1b)3. Our study investigated the effect of difference in F0 between the A and B tones, ΔF0, and pitch salience on sound segregation. Earlier studies have suggested that ease of segregation is related to perceptual salience4. We therefore hypothesised that segregation would be better for conditions with high pitch salience, i.e. low harmonic rank.
Figure 2 shows the results from this study. It can be seen that segregation was more likely to occur for larger differences in F0 and with increasing pitch salience (decreasing harmonic rank) confirming our hypothesis. Moreover, we correlated the segregation scores with results from an experiment measuring the smallest detectable difference in F0 for tones equal to the A-tones in the sound segregation experiment. As shown in Fig. 3, there was a significant correlation between the performance in the two experiments indicating that good segregation is associated with good discrimination. This provide further evidence that stream segregation is related to perceptual salience.
Figure 2: Mean segregation across listeners. The right panel shows results obtained with complex tones and the left panel shows results obtained with pure tones. The x-axis indicates the F0 or frequency of the A-tones. The figure was previously presented in a published paper2.
The relation between the salience of perceptual differences and sound segregation abilities of cochlear-implant users
During and after my Carlsberg Foundation fellowship, I had the pleasure of co-supervising a PhD student who investigated stream segregation in cochlear-implant (CI) users. CIs are medical devices that are given to profoundly deaf individuals. They consist of an internal and an external part; a sound processor that is placed behind the ear and electrodes that are inserted into the cochlear. The sound processor transmits signals to the electrodes that in turn provide electrical stimulation to the auditory nerve. This generate a perception of sound.
Figure 3: Left panel: Relationship between mean percentage segregation for each A-tone F0 and the mean F0 discrimination threshold obtained for the same F0. Right panel: Relationship between the mean percentage segregation score for each listener and the mean F0DL for that listener. The figure was previously presented in a published paper2.
In two of our studies, similar stimuli were used and the difference between the A and B sounds were varied by either varying the distance between electrodes (place)5 or the stimulation rate6. The variation of both parameters elicits perceptual differences thought to be related to pitch. Results from earlier studies were inconclusive in terms of whether CI-users were able to use such cues for sound segregation. Our studies found clear evidence of segregation both when using place and when using rate cues. Moreover, using data from another PhD project allowed us to compare the results from the two studies in terms of the salience of perceptual differences elicited by the difference in rate or place. This comparison (see Fig. 4) showed that the amount of segregation was similar for conditions eliciting the same perceptual difference. Thus, the amount of segregation did not depend on the specific parameter used to elicit a difference between sounds but on the perceptual difference elicited. This provided further evidence of the importance of perceptual difference for sound segregation.
Figure 4: Stream segregation performance when varying the electrode distance or rate as a function of ΔPitch, a measure of perceptual distance. ΔPitch was extracted from data from a verbal attribute magnitude estimate experiment. Figures was previously presented in a published paper6
Future Perspectives
The funding from the Carlsberg Foundation made it possible to steer the direction of my research and to develop as a researcher. I was in this period able to gain experience, establish a professional network, and produce papers that will help in my future career. Moreover, I also got experience co-supervising a PhD student who recently defended his PhD.
The findings from this project that there seem to be little or no relation between hearing loss and the ability to use pitch cues for segregation might be useful for development of future hearing aids. Also, the information that strong pitch salience is also important for segregation for HI listeners emphasize the importance of preserving strong pitch salience when optimising hearing aid algorithms.
Papers describing work funded by this grant:
1. Madsen, S.M.K., Dau, Torsten & Moore, B.C.J. Effect of harmonic rank on sequential sound segregation. Hear. Res. 367, 161–168 (2018).
2. Paredes-Gallardo, A., Madsen, S. M. K., Dau, T. & Marozeau, J. The Role of Place Cues in Voluntary Stream Segregation for Cochlear Implant Users. Trends Hear. 22, (2018).
3. Paredes-Gallardo, A., Madsen, S. M. K., Dau, T. & Marozeau, J. The role of temporal cues in voluntary stream segregation for cochlear implant users. Trends Hear. 22, (2018).
Other relevant papers.
4. Bregman, A. S. Auditory Scene Analysis: The Perceptual Organization of Sound. in 455–528 (Bradford Books, MIT Press, 1990).
5. Bernstein, J. G. W. & Oxenham, A. J. The relationship between frequency selectivity and pitch discrimination: Effects of stimulus level. J. Acoust. Soc. Am. 120, 3916–3928 (2006).
6. Moore, B. C. J. & Gockel, H. Factors influencing sequential stream segregation. Acta Acust. United Acust. 88, 320–333 (2002).
Sara Miay Kim Madsen - Institution page