Ai reads brain activity to tell which bit of a movie you are watching

2 min read

Neuroscience

Topics
Topics
Study participants viewed the film (500) Days of Summer
COLLECTION CHRISTOPHEL/ALAMY

MOVIE characters and locations can be deciphered by monitoring the activity of people’s neurons while they watch a film, a discovery that could help us understand how the brain stores memories as they are being made.

A deep brain structure called the hippocampus, which is critical for memory, contains neurons that encode the identity of objects, such as tools or buildings. The cells involved then respond to stimuli depicting those objects. But a lot of our knowledge about this comes from studying people while they view isolated images, rather than responding to changing scenes.

To learn more, Franziska Gerken at the Technical University of Munich in Germany and her colleagues turned to 29 people with epilepsy who had electrodes implanted into their brains to monitor for seizures before they underwent surgery for their condition.

The researchers used this opportunity to record the activity of about 2300 neurons in and around each individual’s hippocampus while they watched the film (500) Days of Summer.

Gerken and her colleagues then used an artificial intelligence model to analyse how these neural responses differed according to specific features of the film, such as characters and locations, and visual categories, like scene transitions.

~2300 hippocampus neurons were scanned as people watched a film

When the participants rewatched the film, the researchers could tell which content people were viewing by looking at the neuronal responses.

The neurons in all the regions in and around the hippocampus responded to the film’s features and visual categories, but activity in some regions reflected these more than others. For example, the activity of visually responsive neurons in a region known as the parahippocampal gyrus was linked to scene tran