Kaolin is a suite of tools for accelerating 3D Deep Learning research.
NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints and more.
NVIDIA Kaolin Wisp is a faster-paced library for research in neural fields, a sub-class of 3D representations that includes NERFs and Neural SDFs. Built on the foundations of the core Kaolin Library, Wisp provides modular functions, training scripts, mix-and-match components, and extensible interactive neural field visualizer to help researchers stay on the bleeding edge of this quickly evolving research area.
NVIDIA Omniverse Kaolin App is an interactive application for 3D deep learning researchers that allows inspecting 3D datasets, interactive visualization of 3D outputs of a model during training, and synthetic dataset rendering. Built on Omniverse Kit, the research application benefits from high-fidelity RTX rendering and will gain new functionality periodically from new extensions.
In combination, these tools can massively reduce the time needed to develop AI research for a wide range of 3D applications.
Continuous Additions from NVIDIA Research
Follow library releases for new research components from NVIDIA Toronto AI Lab and across NVIDIA. Latest releases included Deep Marching Tetrahedra, differentiable mesh subdivision, and Structured Point Clouds (SPC) acceleration data structure supporting efficient volumetric rendering.
Modular Differentiable Renderer
Develop cutting-edge inverse graphics applications using modular and optimized mesh differentiable renderer.
3D Data Loading
Easily load large 3D datasets to train your machine learning models. Make use of import and export utilities for OBJ and USD formats.
GPU Optimized 3D Operations
Convert between 3D representations using fast and reliable conversion operations. Use GPU-optimized implementations of 3D loss functions and a growing collection of other operations on 3D data.
Export 3D checkpoints for meshes, point clouds and voxel grids in USD format, allowing interactive visualization of model training.
Fast-paced library for Neural Fields research built over Kaolin Library and PyTorch
Recipes for Popular Neural Field Approaches
Complete and easy-to-modify recipes for training NERFs and Neural SDFs using a variety of approaches.
Building Blocks for Neural Fields Research
Mix-and-match components to develop new neural fields research. Available building blocks include a variety of feature representations, including octrees, hash grids, codebooks and triplanes.
Extensible Neural Fields Visualizer
Take your research to a new level with an interactive renderer that supports flexible rendering of neural primitives pipelines shipped with the library, interactive visualization during training, and integration with OpenGL style primitives for additional layers of information.
Omniverse Kaolin App
Leveraging the NVIDIA Omniverse Platform, Omniverse Kaolin App allows high fidelity rendering and interactive visualization of 3D data and training results.
Visualize Model Training
Scrub through iterations of 3D checkpoints exported using the Kaolin python API to understand how your model is training. Debug models in development by inspecting 3D outputs and produce beautiful renderings of final results.
Render Synthetic Data
Render massive training datasets with RTX ray and path tracing and export useful ground truth labels, such as segmentation maps, bounding boxes, depth maps and normals.
Inspect 3D Datasets
Effortlessly sample, render and inspect 3D datasets to gain intuition about your training data and identify inconsistencies.