Imagine listening to your favorite song and having a machine read your mind and play it back to you. Sounds like science fiction, right? Well, not anymore. A team of neuroscientists from UC Berkeley, UCSF, UT Austin, and NYU have achieved this feat by decoding and reconstructing the Pink Floyd song “Another Brick in the Wall” from the brain activity of 29 people who listened to it while undergoing epilepsy surgery.
The study, published in the journal Science Advances, is the first to demonstrate that music can be decoded from brain activity, and it could have implications for developing devices that help people who have lost their ability to speak or sing.
The researchers used electrodes, computer models, and brain scans to measure how different brain regions responded to different musical elements, such as tone, rhythm, harmony, and lyrics. They then used machine learning to produce sounds that resembled the song from the brain data.
“We were able to decode different aspects of music from different parts of the brain. We could see how the brain represents pitch, timbre, rhythm, harmony and lyrics,” said Adeen Flinker, lead author of the study and assistant professor of neurology at UCSF.
The reconstructed sounds were not perfect replicas of the original song, but they captured some of its essence. “The goal is not to generate a perfect re-creation of a song. The goal is to understand how the brain represents complex auditory information,” said Robert Knight, senior author of the study and professor of psychology and neuroscience at UC Berkeley.
The study also revealed that the musicality of speech is encoded in the brain, and that speech and music share some common neural pathways. “It’s like we’re reading their minds. We’re seeing what they’re hearing in their heads,” said Edward Chang, co-author of the study and professor of neurological surgery at UCSF.
“This is a proof-of-concept demonstration that we can use these kinds of models to understand how music is represented in the brain. It opens up a lot of possibilities for future work,” said Alexander Huth, co-author of the study and assistant professor of neuroscience at UT Austin.
“Music is a universal language that we all share. It’s amazing that we can use modern technology to reveal how our brains process it,” said David Poeppel, co-author of the study and professor of psychology and neuroscience at NYU.
The researchers hope that their findings will inspire more research on how the brain processes music and other complex sounds, and how this knowledge can be applied to help people with communication disorders.
Relevant articles:
– Scientists reconstruct Pink Floyd song by listening to people’s brainwaves by The Guardian, published on August 15, 2023.
– Scientists recorded a Pink Floyd song from patients’ brain waves. The tech could eventually allow for communication without words by Fortune, published on August 15, 2023.
– Mind-Melting Magic: Scientists Reconstruct Pink Floyd Song Straight From Brain Waves of Listener by Compsmag, published on August 15, 2023.