Kaolin is a suite of tools for accelerating 3D Deep Learning research.

Get Started

NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations. It includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, camera classes, volumetric acceleration data structures, 3D checkpoints, and more.

NVIDIA Kaolin Wisp is a faster-paced library for research in neural fields, a subclass of 3D representations that includes NeRFs and Neural SDFs. Built on the foundations of the core Kaolin Library, Wisp provides modular functions, training scripts, mix-and-match components, and an extensible interactive neural field visualizer to help researchers stay on the bleeding edge of this quickly evolving research area.

NVIDIA Omniverse Kaolin App is an interactive application that allows 3D deep learning researchers to inspect 3D datasets, interact with visualizations of 3D outputs of a model during training, and render synthetic datasets. Built on Omniverse Kit, the application benefits from high-fidelity RTX™ rendering and will periodically gain new functionality from additional extensions.

Kaolin Library

With core utilities and advanced features for 3D deep learning research, Kaolin Library includes a modular Python API built on PyTorch.

Continuous Additions from NVIDIA Research

Follow library releases for new research components from the NVIDIA Toronto AI Lab and across NVIDIA. Latest releases included Deep Marching Tetrahedra, differentiable mesh subdivision, and structured point clouds (SPCs) acceleration data structure supporting efficient volumetric rendering.

Kaolin supports 3D structure and 2D mesh rendering of images

Develop inverse graphics apps with modular differentiable rendering

Modular Differentiable Rendering

Develop cutting-edge inverse graphics applications using modular and optimized implementations of differentiable rendering. This includes a mesh differentiable renderer with two rasterization backends, DefTet tetrahedral meshes volumetric rendering, and ray-tracing features for SPCs, allowing both surface and volumetric differentiable rendering.

3D Data Loading and Modular Cameras

Easily load large 3D datasets to train machine learning models. Make use of import and export utilities for OBJ and USD formats. Make use of a consistent and modular camera API, available in release 0.12.0.

Load large 3D datasets to train machine learning models

Kaolin library provides GPU-Optimized 3D conversion operations

GPU-Optimized 3D Operations

Convert between 3D representations using fast and reliable conversion operations, including marching cube and marching tetrahedra, point cloud sampling from mesh and various conversion to SPCs. Use GPU-optimized implementations of 3D loss functions such as point-to-mesh distance, nearest point distance, chamfer distance, AMIPS loss, and a collection of other operations on 3D data, such as topology processing on mesh, extraction and projection of orthographic depth maps, and sparse convolution on SPCs.

3D Checkpoints

Export 3D checkpoints for meshes, point clouds, and voxel grids in USD format, allowing interactive visualization of model training.

Exporting 3D checkpoints allows interactive visualization of model training

Kaolin Wisp

Fast-paced library for Neural Fields research built over Kaolin Library and PyTorch

Recipes for Popular Neural Field Approaches

Complete and easy-to-modify recipes for training NERFs and Neural SDFs using a variety of approaches.

Building Blocks for Neural Fields Research

Mix-and-match components to develop new neural fields research. Available building blocks include a variety of feature representations, including octrees, hash grids, codebooks and triplanes.

Extensible Neural Fields Visualizer

Take your research to a new level with an interactive renderer that supports flexible rendering of neural primitives pipelines shipped with the library, interactive visualization during training, and integration with OpenGL style primitives for additional layers of information.

Download Now

Omniverse Kaolin App

Leveraging the NVIDIA Omniverse Platform, Omniverse Kaolin App allows high fidelity rendering and interactive visualization of 3D data and training results.

Visualize Model Training

Scrub through iterations of 3D checkpoints exported using the Kaolin python API to understand how your model is training. Debug models in development by inspecting 3D outputs and produce beautiful renderings of final results.

Render Synthetic Data

Render massive training datasets with RTX ray and path tracing and export useful ground truth labels, such as segmentation maps, bounding boxes, depth maps and normals.

Inspect 3D Datasets

Effortlessly sample, render and inspect 3D datasets to gain intuition about your training data and identify inconsistencies.

Omniverse Kaolin App now available. Read more about it today!