NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints and more.

NVIDIA Kaolin Wisp is a faster-paced library for research in neural fields, a sub-class of 3D representations that includes NERFs and Neural SDFs. Built on the foundations of the core Kaolin Library, Wisp provides modular functions, training scripts, mix-and-match components, and extensible interactive neural field visualizer to help researchers stay on the bleeding edge of this quickly evolving research area.

NVIDIA Omniverse Kaolin App is an interactive application for 3D deep learning researchers that allows inspecting 3D datasets, interactive visualization of 3D outputs of a model during training, and synthetic dataset rendering. Built on Omniverse Kit, the research application benefits from high-fidelity RTX rendering and will gain new functionality periodically from new extensions.

In combination, these tools can massively reduce the time needed to develop AI research for a wide range of 3D applications.

Continuous Additions from NVIDIA Research

Follow library releases for new research components from NVIDIA Toronto AI Lab and across NVIDIA. Latest releases included Deep Marching Tetrahedra, differentiable mesh subdivision, and Structured Point Clouds (SPC) acceleration data structure supporting efficient volumetric rendering.

Modular Differentiable Renderer

Develop cutting-edge inverse graphics applications using modular and optimized mesh differentiable renderer.

3D Data Loading

Easily load large 3D datasets to train your machine learning models. Make use of import and export utilities for OBJ and USD formats.

GPU Optimized 3D Operations

Convert between 3D representations using fast and reliable conversion operations. Use GPU-optimized implementations of 3D loss functions and a growing collection of other operations on 3D data.

3D Checkpoints

Export 3D checkpoints for meshes, point clouds and voxel grids in USD format, allowing interactive visualization of model training.

Recipes for Popular Neural Field Approaches

Complete and easy-to-modify recipes for training NERFs and Neural SDFs using a variety of approaches.

Building Blocks for Neural Fields Research

Mix-and-match components to develop new neural fields research. Available building blocks include a variety of feature representations, including octrees, hash grids, codebooks and triplanes.

Extensible Neural Fields Visualizer

Take your research to a new level with an interactive renderer that supports flexible rendering of neural primitives pipelines shipped with the library, interactive visualization during training, and integration with OpenGL style primitives for additional layers of information.

Leveraging the NVIDIA Omniverse Platform, Omniverse Kaolin App allows high fidelity rendering and interactive visualization of 3D data and training results.

Visualize Model Training

Scrub through iterations of 3D checkpoints exported using the Kaolin python API to understand how your model is training. Debug models in development by inspecting 3D outputs and produce beautiful renderings of final results.

Render Synthetic Data

Render massive training datasets with RTX ray and path tracing and export useful ground truth labels, such as segmentation maps, bounding boxes, depth maps and normals.

Inspect 3D Datasets

Effortlessly sample, render and inspect 3D datasets to gain intuition about your training data and identify inconsistencies.