TECHNOLOGY
By examining a person’s brain activity, artificial intelligence (AI) can produce a song that matches the genre, rhythm, mood and instrumentation of music the individual recently heard. Scientists have previously ‘reconstructed’ other sounds from brain activity, such as human speech, bird song and horse whinnies. However, few studies have attempted to recreate music from brain signals. Now, researchers have built an AI-based pipeline called Brain2Music that harnesses brain imaging data to generate music that resembles short snippets of songs a person was listening to when their brain was scanned.
The scientists used brain scans that had previously been collected via a technique called functional magnetic resonance imaging (fMRI), which tracks the flow of oxygen-rich blood to the brain to see which regions are most active. The scans were collected from five participants as they listened to 15-second music clips spanning a range of genres, including blues, classical, country, disco, hip-hop, jazz and pop.
Using a portion of the brain imaging data and song clips, the researchers first trained an AI program to find links between features of the music, including the instruments used and its genre, rhythm and mood, and participants’ brain signals. The music’s mood was defined by researchers using labels such as happy, sad, tender, exciting, angry or scary. The AI was customised for each person, drawing links between their unique brain activity patterns and various musical elements.
After being trained on a selection of data, the AI could convert the remaining, previously unseen brain imaging data into a form that represented musical elements of the original song clips. The researchers then fed this information into another AI model previously developed by Google, called MusicLM. MusicLM used the information to generate musical clips that can be listened to online and fairly accurately resemble
