Recreating Pink Floyd's Song from the Brain; Dr. Vahid Behrovan: We Are Close to Complete Speech Reconstruction for Patients

Scientists have managed to use artificial intelligence to extract and replay the famous Pink Floyd song "Another Brick in the Wall - Part 1" from the brains of individuals. To decode this song, neuroscientists at the University of California, Berkeley analyzed brain activity recorded by implanted electrodes in the brains of 29 individuals with epilepsy while listening to this 1979 rock song. The team chose the Pink Floyd song primarily because it contains 41 seconds of lyrics and two and a half minutes of instrumental, a combination that was useful for understanding how words and melodies are processed in the brain. In this research, the researchers trained the AI on the brainwaves of individuals recorded while listening to the song, except for fifteen seconds, and wanted to see if the device could reconstruct that fifteen seconds, which it largely could. In fact, the AI was able to recognize the rhythm of the music, expression, and emotion. In a statement to Voice of America, Dr. Vahid Behrovan, a neurologist involved in the research, said: "Currently, there are devices that can extract sentences from patients with brain injuries, but the tone, speed, and all the features that convey the full meaning of speech cannot yet be detected using electrodes placed on the scalp, and sentences are mostly delivered in a robotic tone." He added: "If technology advances to the point where we can record waves with the same precision from the surface of the cortex as from the scalp, then the device can reconstruct words with tone and speed and all the characteristics that convey the essence of a sentence, and this is a significant step that brings us closer to complete reconstruction of normal speech for someone who has lost this ability." This technique is used for patients who are capable of constructing sentences in their minds but cannot express them. Previously, scientists had been able to decode and reconstruct words and thoughts from brain activity using electrodes, computer models, and brain scans, but now music and the essence of sentences have also been added to this combination. Dr. Vahid Behrovan states that music is even more complex than speech, and in addition to notes, speed, rhythm, and the progression of music convey the unique feeling of that song. In response to the question of whether such technology could help Persian-speaking patients, he said that Persian, like other languages, has its own weight, rhythm, and unique characteristics that convey a specific feeling to the listener. He added: "If we can teach the device not only based on simple words but also based on sentences with different speeds and feelings, and even different accents, and if the AI has a vast database of Persian words and their tones, later we could accurately capture waves from the scalp, and a Persian-speaking patient who has lost the power of speech could express what they want through this method." Dr. Behrovan emphasized that technology has not yet reached this stage. The research by Berkeley scientists also showed which parts of the brain respond to different elements of music. For example, activity in a region of the superior temporal gyrus is heightened with the onset of specific sounds like guitar notes. This research also indicated that this area in the right hemisphere of the brain, rather than the left, is crucial for decoding music, as the accuracy of song reconstruction decreased when information from that area was removed from the computer model. One limitation of this study was considering whether listeners liked the Pink Floyd song or had previously heard it, as all these factors, especially regarding music, can affect brain performance and thus the accuracy of the decoding model.