Researchers from ATR Computational Neuroscience Laboratories and Kyoto University in Japan developed a deep learning-based algorithm that can generate images from brain activity.
“The reconstruction algorithm starts from a random image and iteratively optimize the pixel values so that the DNN (deep neural network) features of the input image become similar to those decoded from brain activity across multiple DNN layers,” mentioned the researchers in their paper ‘Deep image reconstruction from human brain activity’. “The resulting optimized image is taken as the reconstruction from the brain activity.”
Using a GTX 1080Ti and Tesla K80 GPU with the cuDNN-accelerated Caffe deep learning framework, the researchers trained their decoders on fMRI data measured while human subjects looked at images over a ten-month period. The images included geometric shapes, natural phenomena and letters of the alphabet.
Below are the results of the algorithm.
“Our brain processes visual information by hierarchically extracting different levels of features or components of different complexities,” Kyoto University Graduate School of Informatics Professor Yukiyasu Kamitani shared about the impact of their work, “These neural networks or AI models can be used as a proxy for the hierarchical structure of the human brain.”
Read more >
