Study Finds the Brain Briefly Tracks Two Conversations in Noise

A recent study reports that when the brain shifts attention from one person speaking to another in a noisy setting, it temporarily follows two conversations simultaneously.

For around one to two seconds, neural responses to the incoming voice start to increase before responses to the former voice have declined. This creates a brief period in which the cortex represents both speech streams.

The findings may help account for why certain people cope better than others in busy restaurants or packed parties. They also indicate that the brain does not completely let go of one conversation before focusing on another, instead retaining the earlier speaker briefly in case the listener wishes to return to them.

Inside the experiment

Dr Sara Carta at Trinity College Dublin (TCD) led the research. Her group placed 24 young adults with normal hearing at the centre of a circular arrangement of six loudspeakers.

Two loudspeakers in front of each participant, positioned 60 degrees apart, simultaneously broadcast separate TED talks. The remaining four speakers, located behind participants, produced a babble of 16 blended voices: the acoustic equivalent of a busy room.

A small on-screen arrow indicated which of the two front speakers each person should attend to, switching sides every 15 to 30 seconds. Brain responses were recorded throughout with a 64-electrode EEG cap.

Participants judged the switching task to be roughly three out of five in difficulty. Nevertheless, they answered questions about the content of the attended talk with 86% accuracy.

The team assessed attention using a technique called a temporal response function. This mathematical filter estimates the extent to which EEG activity follows the fluctuations of a particular speech signal; a strong correlation therefore indicates that the brain is tracking that voice.

Switching attention between competing voices

Selective listening amid noise has been investigated since Colin Cherry described the “cocktail party problem” in a 1953 paper. Subsequent studies found that the brain suppresses much of an ignored speaker’s speech, although it does not block all of it.

However, most previous research asked listeners to remain focused on a single target. Comparatively little work has examined people changing between voices in real time, a situation that more closely reflects ordinary conversation.

Carta’s researchers divided every shift in attention into two distinct processes. Disengagement referred to reduced neural tracking of the former speaker, whereas engagement described increased tracking of the new speaker. Had these processes occurred together, the switch would have been a straightforward exchange.

The findings did not support that pattern. Engagement began before disengagement, and it also finished sooner. The brain response to the new voice intensified while the response to the old voice remained active. For a short time, both streams were represented together in the cortex.

The brain briefly tracks both

This overlap does not automatically indicate that a listener comprehends both speakers at the same time.

When Earth.com asked whether someone partly listening to a conversation across a room genuinely absorbs what they hear, Professor Giovanni Di Liberto of Trinity College Dublin, a senior author of the study, suggested one explanation.

“One possibility is that, as we switch to speaker B, we are still monitoring speaker A for a brief period of time without really understanding. But that understanding may actually happen if needed,” he said.

Earlier research had already established that unattended voices are not completely muted. Even speech a listener is meant to ignore produces measurable acoustic, and even linguistic, traces in the cortex.

The latest finding introduces an important timing detail. This dual encoding during a change of speaker is not merely background surveillance. Rather, it seems to form part of the brain’s process for transferring attention between voices.

Switching takes effort

Alpha waves are brain oscillations between eight and 12 hertz. During listening, they indicate mental effort. When someone concentrates on a target speaker in noise, alpha power at the back of the head generally decreases.

The same pattern emerged in this study, with revealing timing. Alpha power reached its lowest level approximately 4.5 seconds after the arrow changed direction – substantially later than the point at which speech tracking in the brain had already shifted to the new voice.

After the brain had connected with the new speaker, it continued to expend considerable effort while completing its release of the previous speaker. This delay implies that changing attention demands more mental work than maintaining focus on the same voice.

A related investigation used pupil dilation and alpha activity to demonstrate that listeners who switch work harder than those who stay focused on one speaker. The new study locates this added cost towards the end of the shift, while disengagement is still catching up.

Di Liberto next plans to examine whether this gap becomes wider in more demanding listening environments. In comments to Earth.com, he predicted, “I would expect harder tasks, which would involve more cognitive resources, to reduce the length of time where both speakers can be processed.”

Starting each conversation fresh

Comprehending speech involves more than following sounds. Listeners draw on words already heard to anticipate what will follow, and the brain’s language centres display clear signs of this prediction.

The researchers sought to establish what becomes of these predictions when a listener abruptly changes speakers.

To investigate, they modelled four potential strategies with the large language model Mistral-7B. In one, the brain used all earlier context from the new speaker during the trial. A second incorporated everything the listener had attended to across both speakers.

The third approach relied solely on the stream currently being followed. The fourth discarded every piece of earlier context and began again at each switch: a “reset” approach.

The neural data were better explained by the reset strategy than by the alternatives. Compared with lexical entropy – a continuing estimate of the brain’s uncertainty about the next word – it predicted EEG activity more accurately than strategies that retained context.

This outcome differed from what the researchers had anticipated. It is consistent with memory research suggesting that the brain regards sudden scene changes as event boundaries, clearing prior context for whatever follows.

When attention moves between speakers, the brain may therefore abandon the earlier context instead of bringing it along.

Why noisy conversations get harder

Struggling to follow conversation in a noisy environment is a frequent concern for people with hearing loss. It is also among the earliest cognitive changes associated with ageing.

Conventional hearing aids make sounds louder. Yet they offer limited help to listeners wishing to move their attention between people around a table.

The degree of this short-lived dual tracking could differ between individuals.

“The objective metric we derived could be used to study a variety of dimensions, including early development, hearing deficits, and aging, among other things,” said Di Liberto.

Research into devices that infer a wearer’s attention through EEG or comparable signals has been active. A system that identifies only the new target could leave the former speaker unprotected for one or two seconds as the brain completes the process of disengaging from them.

The brain starts over

The findings also point to a boundary on the amount of information the brain retains when moving between conversations.

Instead of connecting what one person said moments earlier with the words of the next speaker, the brain seems to reset.

This may explain why recollections of partly overheard conversations often seem fragmented. The context needed to link the words was never transferred across the attentional switch.