Sound Identification System from Auditory Cortex by Using fMRI and Deep Learning: Study on Experimental Design for Capturing Brain Images

Abstract

This study establishes a technique estimating sounds using deep learning based on brain images when humans hear sounds captured by fMRI. Humans are hearing complex sounds with mixed frequencies, so we develop estimating complex sound system to establish this technique. So far, we developed a system identifying single sound based on brain images when humans hear single sound by using CNN one of the deep learning, but it doesn’t support complex sound. Therefore, we focus on complex sound and aim to develop a system identifying complex sounds based on brain images when humans hear complex sound. Since identification results generally depend on the brain image used for identification, this report considers block design and event-related design which fMRI experimental designs for capturing brain images to understand effect of stability for brain activity on brain image. As a result, the identification rates of two types complex sounds were almost the same for both designs, and the effect of the stability of brain activity didn’t appear in the identification rates so we decide use event-related design.