Simulate Elastic Objects in Any Representation with NVIDIA Kaolin Library

Recent advancements in generative AI and multi-view reconstruction have introduced new ways to rapidly generate 3D content. However, to be useful for downstream applications like robotics, design, AR/VR, and games, it must be possible to manipulate these 3D models in a physically plausible way.

This poses a major challenge to traditional physics simulation algorithms, which were designed to work on well-conditioned, manually curated geometries, such as tetrahedralized volumetric meshes.

To open up physics simulation to a wide array of input geometries, the NVIDIA Kaolin Library latest release integrates a new state-of-the-art technique called Simplicits, a unified representation that enables simulation not only on messy meshes and point clouds but also learned representations like Gaussian Splats and NeRFs.

The Kaolin physics API exposes two levels of abstraction for working with Simplicits: low-level functionality for physics experts and a high-level API for generative AI developers.

The following video demonstrates elastic simulations for a variety of meshes. You can set up and visualize an interactive simulation in a Jupyter notebook with just a few lines of code, shown by a chair model being interacted with in real time. This enables the rapid prototyping of new interactive applications for the plethora of geometric representations emerging from AI research. For more information, see the Kaolin visualizer.

You can also pass splat segments into Simplicits to simulate the captured object, as shown by simulating the Gaussian splat of an apple.

The video also demonstrates muscle simulation using Simplicits with simple, volumetric motion of bones and anisotropic muscle fibers. Simplicits can handle material heterogeneity between bones and muscles. For more information, see the muscle simulation tutorial.