Meta's Advanced AI Capabilities for Understanding and Decoding Human Thoughts

Over the past decade, Meta's Fundamental AI Research Lab (FAIR) in Paris has been at the forefront of scientific research. This lab has made significant advancements in the fields of medicine, climate science, and environmental protection while maintaining its commitment to open and reproducible science. Now, looking ahead to the next decade, Meta has focused on achieving Advanced Machine Intelligence (AMI) and utilizing it to develop products and innovations for the benefit of all. In collaboration with the Basque Center for Cognition, Brain and Language (BCBL) in San Sebastián, Spain, Meta has introduced two significant scientific advancements demonstrating how AI can aid in better understanding human intelligence and bring us one step closer to AMI. Decoding sentence production from brain signals: Researchers have succeeded in decoding sentence production from brain activity using non-invasive methods. The results show that AI can reconstruct up to 80% of typed characters from brain signals, often leading to complete sentence reconstruction. Understanding how thoughts are converted into language in the brain: In this study, Meta has shown that AI can help us understand how the brain transforms thoughts into a sequence of words. This research provides a closer look at the language processing mechanism in the brain. Every year, millions lose their ability to communicate due to brain injuries. Currently, one of the available solutions for restoring communication ability is the use of neural prosthetics that receive brain signals and convert them into speech or text with the help of AI. However, these methods are often invasive and require complex surgeries like electrocorticography (ECoG), which have low scalability. In this research, the Meta team used magnetoencephalography (MEG) and electroencephalography (EEG) methods to record brain signals from 35 healthy volunteers at the BCBL while typing. A new AI model was then trained to reconstruct typed sentences solely from these signals. The results showed that this model could decode up to 80% of typed characters, which is twice the accuracy of the classic EEG method. Despite this advancement, there are still significant challenges to using this method in therapeutic environments: Limited accuracy: The performance of AI models is still not flawless and may reconstruct incorrect information in some cases. Need for a specific environment: Using MEG requires individuals to be in specially designed rooms with a protected magnetic field and to remain still during the experiment, which reduces its effectiveness in real-world conditions. Need for testing on real patients: This research was conducted on healthy individuals, and it remains to be seen whether this technology can be equally effective for those with brain injuries. Another fundamental challenge in neuroscience is understanding the neural mechanisms of language production in the brain. Previous research in this area has been difficult because mouth and tongue movements disrupt the neural signals recorded by brain imaging methods. To investigate this topic more closely, the Meta research team used AI to analyze MEG signals while typing. By recording 1000 images of the brain per second, researchers were able to identify the moment when thoughts are converted into words, syllables, and even individual letters. Studies showed that the brain creates a sequence of dynamic neural representations that allows it to process and store words and sequential movements simultaneously. These findings provide deep insights into how the brain functions in language production and could be crucial in the development of AMI. As a leading company in developing open-source technologies, Meta is in a position to address medical challenges using AI. For instance, the French company BrightHeart uses Meta's DINOv2 model to assist doctors in diagnosing congenital heart defects through ultrasound. This company recently received FDA 510(k) approval for its software, one of the key factors in its success being the use of Meta's open-source models. Additionally, the American company Virgo uses DINOv2 for analyzing endoscopic videos. This model has been able to perform at global standards across a wide range of AI assessment metrics for endoscopy, such as identifying anatomical landmarks, assessing the severity of ulcerative colitis, and identifying functional intestinal polyps. AI is rapidly changing our understanding of the brain and cognitive processes. Recent research has shown that AI can decode language directly from the brain, understand cognitive processes related to language production, and help develop therapeutic solutions for individuals with communication disabilities. Looking to the next decade, Meta intends to continue its research in Advanced Machine Intelligence (AMI) and leverage this technology to address some of the biggest scientific and social challenges. Continued collaboration with leading research institutions will help us utilize these advancements to improve the lives of individuals worldwide.