Neural Basis of the Phonemic Restoration Effect: Using Top-Down and Bottom-Up Processing (excerpt)

by Shweta

Research conducted at Columbia University, Department of Electrical Engineering

ABSTRACT

Our ability to listen to speech in noisy environments is a complex process involving the

auditory system, in which many phenomena remain unexplained. Phonemic restoration helps

us to communicate with others even if we cannot hear every word because we subconsciously

fill in missing phonemes—speech sounds— in speech when there is background noise.

Previous studies have explained the perceptual processes involved in the phonemic restoration

effect, clarifying the role of language and acoustic grouping cues. However, the relationship

between top-down and bottom-up processing in a continuous sound remains largely unknown.

Particularly sensitive to the delineation of brain areas, these processes provide an insightful look

at human behavior. In this study, we focus on this interaction by investigating the breakdown

of the phonemic restoration effect, where subjects will have to surpass noise with different

frequencies in order to correctly identify pseudowords. We played recordings of a series of

mispronounced phonemes, which were interrupted by static-sounding noise, and asked subjects

to report the word they heard while varying the intensity of noise in each trial. In the future, we

will also measure neural activity with an electroencephalogram (EEG) cap while the participants

perform this task to further understand the underlying brain mechanisms behind the processing

required for this task. By clarifying the relationship between what we hear and what we actually

perceive, this study investigates where and how the brain subconsciously interprets unclear

speech, which affects our communication abilities in noisy environments.

INTRODUCTION

In my study, I focus on the interaction between these processes and signal to noise

ratios (SNRs) in order to observe how an interruption of a phoneme by a noise signal affects the

way our brains process sounds. SNR is the ratio of signal strength—in this case, our phonemic

words—to the noise level disrupting the signal. Hence, a ratio higher than 1:1 indicates more

signal than noise. The experiment we conducted created a conflict by saying the phonemes

incorrectly with different SNR levels to see where the two processes in our brain are able to

identify the error in each word.

The potential benefits of this perceptual study include a better understanding of the

phonemic restoration effect and shed light on the processes of the auditory system involved in

selectively listening to speech in a noisy environment. This better understanding in turn enables

novel therapeutic approaches for hearing impaired listeners for whom perception of speech in

noise remains very challenging.

RESULTS

Most of our participants were unable to hear the erroneous phoneme within the

words when the SNR was below 0.1, but the majority of subjects correctly identified a

mispronunciation of the phoneme by SNR 0.8. Moreover, while many subjects heard the error

in “identi-bication,” no one was correctly identifying the phoneme; most subjects were hearing a

similar plosive: “identi-pication.”

There were four cases which we saw most often from our participants: I. Correctly

identified pseudoword (acknowledgesent, apprekiation, etc.); II. Partially correct because

participant said a pseudoword within the same phoneme class (saying “identipication” instead

of “identibication” since they are both plosives); III. Phonemic restoration totally effective

(“miscommunication” instead of “miscommunimation”); and IV. Phonemic restoration partially

effective (“disappointshment” instead of “disappointshent” because the “m” was still heard).

Rogue data that was eliminated from our graphs included words that were mispronounced in

areas where the words weren’t even changed (itentification).

DISCUSSION

The reason for having subjects listen to single pseudowords is in preparation for our

main projects in the upcoming year which will help us identify where in the brain the phonemic

restoration effect takes place. Our data in the future will include epilepsy patients’ responses

to masked phonemes within a contextual sentence. We predict that subjects will have more

difficulty identifying the pseudoword within a sentence than identifying the pseudoword in

our current experiment, where each mispronounced word is said separately and out of context.

Also, our future results will include a visual component in which we will be showing a picture

of a word and will observe if the visually stimulating image affects the phonemic restoration.

Collecting this data involves plotting the power spectrum of each image, which can possibly

indicate more on how top-down and bottom-up processing function together in the brain.

Shweta is a senior.  She says, “I interned at Columbia University, Department of Electrical Engineering, but a main goal in our lab was to bridge the gap between electrical engineering and neuroscience. We took an interdisciplinary approach to study auditory neuroscience by implementing electroencephalogram (EEG) technology and designing our experiment through Praat software.  I did this internship through NYU’s Girls’ STEM Program, where I was one of the top 5 presenters who spoke about our research experience at the Global Center, across from Washington Square Park. I was also top 10 for this research paper on phonemic restoration.”

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s