From Brain Science to AI and back: using deep neural networks to represent and decode brain activity linked to behavior

A positive synergy between artificial intelligence (AI) and neuroscience holds the potential to advance both fields. Knowledge about biological neural networks may inspire artificial neural networks for AI. Artificial neural networks may provide computational models and tools to study the brain. In line with this notion, this thesis first reports a bi-directional and recurrent neural network based on the predictive coding theory in neuroscience for visual learning and recognition. The network can achieve competitive performance despite having notably fewer layers and parameters. The thesis then focuses on a neural network model of Bayesian brain for encoding and decoding brain activity during dynamic natural vision. The third part of the thesis describes a generalizable, modular and explainable framework for individualized representation learning of resting-state fMRI (rs-fMRI). This framework is generalizable across multiple behavior prediction tasks through a shared “base” model optimized with self-supervised learning on rs-fMRI, and is explainable to identify brain structures responsible for individualized behavioral prediction. Taken together, the thesis presents methods for improving artificial neural networks using biologically plausible rules and using AI to understand brain activity. The findings provide more insights into the integration of basic AI and neuroscience research and have potential clinical applications.

Chair: Professor Zhongming Liu