How ‘Home Alone’ Helped Unlock Brain Scan Data

Using Hit Films, UMD Neuroscientists Uncovered Latent Information to Illuminate Brain Activity Over Time

Clinicians or researchers studying brain scan analyses from functional magnetic resonance imaging (fMRI) typically see a static distribution of brain signals captured in a single instant. Now, University of Maryland researchers seeking to develop a new method to understand mental activity over periods of time have turned to an unexpected source: Hollywood blockbusters.

In a paper published last month in PLOS Computational Biology, Luiz Pessoa, professor of psychology and director of the Maryland Neuroimaging Center, and Joseph JaJa, professor and interim chair of electrical and computer engineering with a joint appointment at the University of Maryland Institute for Advanced Computer Studies, presented a novel machine learning-based approach to uncover this temporal information otherwise hidden in fMRI brain scans.

The neural network algorithms they developed, together with novel statistical approaches, were applied to a large public data set of fMRI scans of subjects watching clips of humorous or suspenseful films like “Home Alone,” “Star Wars” and “Ocean’s 11.”

Their hypothesis was that after a neural network—an artificial intelligence system inspired by the human brain—has digested the data in these brain scans from watching hit movies, it would recognize similar information in new fMRI scans that would otherwise remain latent. Instead of just a snapshot, Pessoa and JaJa showed how recurrent neural networks (which focus on temporal sequences) can extract information about the brain’s responses over longer timescales in contexts similar to those we encounter in everyday life.

“Neuroscience is incorporating extremely sophisticated data analysis, including machine learning, neural networks, and advanced statistical methods,” said Pessoa. “The collaboration with Joseph JaJa allows neuroscientists to work together with engineers to advance our understanding of the brain.”

The paper and collaboration were born out of a 2017 seed grant award from UMD’s Brain and Behavior Institute (BBI).

“The BBI is proud to support exciting new research partnerships across the life and physical sciences such as the collaboration between Pessoa and JaJa,” said Elizabeth Quinlan, professor of biology, Clark Leadership Chair in Neuroscience, and director of the BBI. “The development and application of machine learning techniques and artificial intelligence approaches are driving the vital growth and expansion of collaborative neuroscience on our campus.”

With recent funding to Pessoa from the National Institute of Mental Health, the researchers plan to continue studies into the brain mechanisms of emotion and motivation.

“We need to break down silos and promote integration across disciplines,” Pessoa noted. “This cross-disciplinarity is the only way that we can make serious progress when trying to understand one of the most challenging problems in all of science—the brain.”

Story by Nathaniel Underland originally appeared in Maryland Today. 

Photo collage by Emma J. Howells; Mark Hamill and George Clooney images courtesy of AA Film Archive/Alamy Stock Photo; McCauley Culkin image courtesy of Allstar Picture Library/Alamy Stock Photo

Published on Tue, Nov 9, 2021 - 11:13AM