Researchers from ATR Computational Neuroscience Laboratories and Kyoto University in Japan developed a deep learning-based algorithm that can generate images from brain activity.
“The reconstruction algorithm starts from a random image and iteratively optimize the pixel values so that the DNN (deep neural network) features of the input image become similar to those decoded from brain activity across multiple DNN layers,” mentioned the researchers in their paper ‘Deep image reconstruction from human brain activity’. “The resulting optimized image is taken as the reconstruction from the brain activity.”
Using a GTX 1080Ti and Tesla K80 GPU with the cuDNN-accelerated Caffe deep learning framework, the researchers trained their decoders on fMRI data measured while human subjects looked at images over a ten-month period. The images included geometric shapes, natural phenomena and letters of the alphabet.
Below are the results of the algorithm.
“Our brain processes visual information by hierarchically extracting different levels of features or components of different complexities,” Kyoto University Graduate School of Informatics Professor Yukiyasu Kamitani shared about the impact of their work, “These neural networks or AI models can be used as a proxy for the hierarchical structure of the human brain.”
Read more >
Related resources
- GTC session: Unlocking AI to Build the Metaverse (Spring 2023)
- GTC session: Fireside Chat with Ilya Sutskever and Jensen Huang: AI Today and Vision of the Future (Spring 2023)
- GTC session: Detecting Skin Diseases using AI (Spring 2023)
- NGC Containers: MATLAB
- Webinar: Inception Workshop 101 - Getting Started with Conversational AI
- Webinar: Simplify and Accelerate AI Model Development with PyTorch Lightning, NVIDIA NGC, and AWS
About the Authors
