What was the main question in your dissertation?
I wanted to shed a little bit more light on a really fundamental question in language research: How does the brain generate meaning? I looked at spoken language, in particular. My main goal was to find out more about how listeners get from sound to comprehending language, and how they combine different sources of information during spoken language comprehension.
Can you explain the (theoretical) background a bit more?
In essence, spoken language is an acoustic signal – the speaker produces a series of air pressure fluctuations which the listener can perceive and comprehend. However, there are no straight-forward physical markers for meaning in the acoustic signal; language comprehension can only happen if the listener combines the acoustic information with their knowledge about a specific language. A really straight-forward example for this is listening to a language that we don’t know: In that case, we can still perceive the acoustic signal, but we can’t understand it because we don’t have the required linguistic knowledge.
In my dissertation, I investigated spoken language comprehension through the lens of “perceptual inference”, asking how the brain combines different pieces of information (both from the acoustic signal and our prior linguistic knowledge) to infer meaning. I wanted to know more about the specific types of information that listeners draw on and how they can “adjust” their comprehension strategies depending on the context.
Why is it important to answer this question?
To me, language is one of the most awe-inspiring human behaviours; through physical signals, we’re able to communicate complex thoughts, ideas and feelings to each other. Isn’t that fascinating? Investigating in more detail how our brains generate meaning from these physical signals can hopefully help us better understand human communication, in general.
Can you tell us about one particular project? (question, method, outcome)
In one of my chapters, I wanted to find out more about how the brain combines acoustic information and different types of linguistic knowledge. I conducted an EEG experiment, measuring participants’ brain responses while they listened to different types of linguistic stimuli.
One very interesting finding from previous research is that the brain is able to “track” the acoustic speech signal; that is, groups of neurons in the brain “synchronise” their firing patterns with the rhythm of the acoustic signal. It has previously been suggested that this “tracking” is relevant for language comprehension, but we still don’t quite understand which types of information actually influence this.
My EEG study investigated whether the amount of tracking is influenced by the acoustic information in the speech signal alone, or also by the linguistic information available in the speech. I created an experiment with three different conditions. In one condition, listeners heard “normal” sentences containing structure, meaning, and sentence-level prosody (this refers to the intonation, stress and rhythm in a sentence). In a second condition, listeners heard scrambled word list versions of those sentences, containing meaningful words, but no sentence-level structure or prosody. By comparing participants’ responses to these scrambled word lists compared to the real sentences, I could see to what extent the participants used their knowledge of linguistic structures when tracking the speech signal. In a third condition, listeners heard so-called “jabberwocky” stimuli, which are “sentences” made up of non-existing words. (The name “jabberwocky” comes from Lewis Carroll’s famous nonsense poem about a monster called “the Jabberwock” – check it out here for an example and here for “translations” in different languages) These “jabberwocky” stimuli contain sentence-like structure and prosody, but no straightforward meaning. By comparing participants’ brain responses to these jabberwocky sentences and “normal” sentences, I could see whether participants “tracked” the speech signal when the sentence sounded like a real sentence, but had no straight-forward meaning.
This study showed that the extent to which listeners “track” the speech signal is not just driven by acoustics alone, it is also influenced by the type of linguistic information that is available: “tracking” was stronger when participants were listening to normal sentences than word lists and jabberwocky stimuli. The brain response appears to align more closely to the speech signal if structure and meaning can be inferred.
What was your most interesting/ important finding?
One of the most intriguing findings to me was that listeners are very flexible in how they combine different sources of information for language comprehension. During another one of my experiments, I investigated this in detail and found that listeners actually seem to have different “preferences” with regard to how they use acoustic and linguistic information: Some listeners appeared to “favour” acoustic information that is carried in the speech signal, while others seemed to rely more on their linguistic knowledge. I think this flexibility is really interesting, because it provides such a robust way of handling uncertainties during language comprehension.
What are the consequences/ implications of this finding? How does this push science or society forward?
Although we use language seemingly effortlessly, there are still many things about it that we don’t quite understand, especially when it comes to what happens in the brain. The work in my dissertation provides a small new piece in this enormous puzzle.
What do you want to do next?
I’m currently a lab associate at Disney Research Studios in Zurich. I get to apply lots of the things that I learned during my PhD, which I find very rewarding.
Interviewer: Merel Wolf
Editing: Julia Egger
Dutch translation: Caitlin Decuyper
German translation: Barbara Molz
Final editing: Merel Wolf
