Note: This video may require joining the NVIDIA Developer Program or login

GTC Silicon Valley-2019 ID:S9959:Context-Aware Synthesis of Object Instance in 2D and 3D Scenes

Xueting Li(University of California, Merced),Sifei Liu(NVIDIA)
We'll discuss learning to synthesize object instances such as a person or car in both 2D and 3D scenes. We will introduce our work we presented at NeurIPS 2018 on context-aware synthesis and placement of object instances. We propose a generative model that learns to generate and insert an object instance into an image in a semantically coherent manner. In particular, we represent object instances using masks and learn to insert them into semantic label maps of images. Our talk will also cover our recent work around putting humans in a scene and learning affordance in 3D indoor environments. This extends the learning of context from 2D to 3D scenes in which the synthesized objects are semantically coherent and geometrically correct. We'll show that both projects add technical insights and have potential applications in content creation.

View the slides (pdf)