Recent advancements in generative AI and multi-view reconstruction have introduced new ways to rapidly generate 3D content. However, to be useful for downstream applications like robotics, design, AR/VR, and games, it must be possible to manipulate these 3D models in a physically plausible way.
This poses a major challenge to traditional physics simulation algorithms, which were designed to work on well-conditioned, manually curated geometries, such as tetrahedralized volumetric meshes.
To open up physics simulation to a wide array of input geometries, the NVIDIA Kaolin Library latest release integrates a new state-of-the-art technique called Simplicits, a unified representation that enables simulation not only on messy meshes and point clouds but also learned representations like Gaussian Splats and NeRFs.
The Kaolin physics API exposes two levels of abstraction for working with Simplicits: low-level functionality for physics experts and a high-level API for generative AI developers.
The following video demonstrates elastic simulations for a variety of meshes. You can set up and visualize an interactive simulation in a Jupyter notebook with just a few lines of code, shown by a chair model being interacted with in real time. This enables the rapid prototyping of new interactive applications for the plethora of geometric representations emerging from AI research. For more information, see the Kaolin visualizer.
You can also pass splat segments into Simplicits to simulate the captured object, as shown by simulating the Gaussian splat of an apple.
The video also demonstrates muscle simulation using Simplicits with simple, volumetric motion of bones and anisotropic muscle fibers. Simplicits can handle material heterogeneity between bones and muscles. For more information, see the muscle simulation tutorial.
Summary
Representation agnostic physics simulation is available now in NVIDIA Kaolin Library.
Join us at SIGGRAPH 2024 to learn more about Simplicits and the new 3D deep learning technologies added to Kaolin Library:
- NVIDIA Research Presents AI and the Next Frontier of Graphics
- Dive into 3D Deep Learning, Physics Simulation and Interactive 3D Prototyping with NVIDIA Kaolin Library
For more information, see the following resources:
- Simplicits: Mesh-Free, Geometry-Agnostic, Elastic Simulation (research paper)
- Kaolin Library:
- /NVIDIAGameWorks/kaolin GitHub repo
- Installation Guide
- kaolin.physics
Ask questions and engage with the 3D deep learning community on the NVIDIA Kaolin forum.
