GTC-DC 2019: Deep Multimodal Data Fusion for Pathology Applications

We’ll present a variety of different computational paradigms to fuse information from microscopic images of tissue biopsies, corresponding genomic data, and patient and familial histories. Subjective clinical diagnosis is often based on multimodal information from microscopic and molecular information as well as data from patient and familial histories. But most recent work in objective pathology image analysis doesn’t take into account additional information that can influence diagnosis or prognosis. We’ll demonstrate that fusing multimodal information significantly improves survival prediction, characterization, and prognostication. This paradigm can also be used to identify new biomarkers and morphological features that can lead to the development of new grading schemes.