NVIDIA Kaolin
Kaolin is a suite of tools for accelerating 3D Deep Learning research.
Get StartedNVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations. It includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, camera classes, volumetric acceleration data structures, 3D checkpoints, and more.
NVIDIA Kaolin Wisp is a faster-paced library for research in neural fields, a subclass of 3D representations that includes NeRFs and Neural SDFs. Built on the foundations of the core Kaolin Library, Wisp provides modular functions, training scripts, mix-and-match components, and an extensible interactive neural field visualizer to help researchers stay on the bleeding edge of this quickly evolving research area.
NVIDIA Omniverse Kaolin App is an interactive application that allows 3D deep learning researchers to inspect 3D datasets, interact with visualizations of 3D outputs of a model during training, and render synthetic datasets. Built on Omniverse Kit, the application benefits from high-fidelity RTX™ rendering and will periodically gain new functionality from additional extensions.
Kaolin Library
With core utilities and advanced features for 3D deep learning research, Kaolin Library includes a modular Python API built on PyTorch.
Continuous Additions from NVIDIA Research
Follow library releases for new research components from the NVIDIA Toronto AI Lab and across NVIDIA. Latest releases included Deep Marching Tetrahedra, differentiable mesh subdivision, and structured point clouds (SPCs) acceleration data structure supporting efficient volumetric rendering.
Modular Differentiable Rendering
Develop cutting-edge inverse graphics applications using modular and optimized implementations of differentiable rendering. This includes a mesh differentiable renderer with two rasterization backends, DefTet tetrahedral meshes volumetric rendering, and ray-tracing features for SPCs, allowing both surface and volumetric differentiable rendering.
3D Data Loading and Modular Cameras
Easily load large 3D datasets to train machine learning models. Make use of import and export utilities for OBJ and USD formats. Make use of a consistent and modular camera API, available in release 0.12.0.
GPU-Optimized 3D Operations
Convert between 3D representations using fast and reliable conversion operations, including marching cube and marching tetrahedra, point cloud sampling from mesh and various conversion to SPCs. Use GPU-optimized implementations of 3D loss functions such as point-to-mesh distance, nearest point distance, chamfer distance, AMIPS loss, and a collection of other operations on 3D data, such as topology processing on mesh, extraction and projection of orthographic depth maps, and sparse convolution on SPCs.
3D Checkpoints
Export 3D checkpoints for meshes, point clouds, and voxel grids in USD format, allowing interactive visualization of model training.
Kaolin Wisp
Fast-paced library for Neural Fields research built over Kaolin Library and PyTorch
Recipes for Popular Neural Field Approaches
Complete and easy-to-modify recipes for training NERFs and Neural SDFs using a variety of approaches.
Building Blocks for Neural Fields Research
Mix-and-match components to develop new neural fields research. Available building blocks include a variety of feature representations, including octrees, hash grids, codebooks and triplanes.
Extensible Neural Fields Visualizer
Take your research to a new level with an interactive renderer that supports flexible rendering of neural primitives pipelines shipped with the library, interactive visualization during training, and integration with OpenGL style primitives for additional layers of information.
Omniverse Kaolin App
Leveraging the NVIDIA Omniverse Platform, Omniverse Kaolin App allows high fidelity rendering and interactive visualization of 3D data and training results.
Visualize Model Training
Scrub through iterations of 3D checkpoints exported using the Kaolin python API to understand how your model is training. Debug models in development by inspecting 3D outputs and produce beautiful renderings of final results.
Render Synthetic Data
Render massive training datasets with RTX ray and path tracing and export useful ground truth labels, such as segmentation maps, bounding boxes, depth maps and normals.
Inspect 3D Datasets
Effortlessly sample, render and inspect 3D datasets to gain intuition about your training data and identify inconsistencies.
