At GTC Fall 2020, NVIDIA announced an upcoming developer SDK, RTX Direct Illumination (RTXDI), which enables and accelerates the rendering of dynamic direct lighting and shadows from many light sources. In this post, I highlight the RTXDI capabilities, ahead of the initial public release planned for spring 2021. You can request early access to RTXDI now.
RTXDI uses the power of ray tracing to remove artistic constraints common in traditional rasterization pipelines. With RTXDI, the artist’s imagination need not be gated on light complexity or shadow map costs. All lights in a scene can dynamically move, turn on and off, change color, and cast correct shadows and lighting. To enable a light, add emissivity to a surface’s material properties. Performance is largely decoupled from lighting complexity, so RTXDI seamlessly scales from one light to millions.
Features
The key features of RTXDI include the following:
- Moves all lights dynamically and interacts with dynamic geometry
- Provides lighting from various types of primitives, including mesh lights made of triangles, point lights, rectangles, spheres, cylinders, and environment maps
- Enables emissive textures and IES profiles where applicable
- Adds, removes, merges, or splits light geometry dynamically.
- Scales excellently on performance from one light to millions on RTX GPUs
- Does not require precomputation or baking
- Integrates with RTXGI
- Works with our ReLAX denoiser, especially designed for RTXDI; compatible with general purpose real-time denoisers like SVGF
- Provides full source code, for easy integration and customization
- Runs on any GPU with hardware accelerated ray tracing capability
Long-term, RTXDI is a collection of the best research and development efforts enabling this flexible and powerful application of ray tracing. Initial releases build on recently published research on reservoir-based spatiotemporal importance resampling (ReSTIR), including a variety of improvements dramatically increasing robustness and performance.
Asset implications
RTXDI should integrate easily with existing assets and pipelines. RTXDI only assumes the following:
- Emissive surfaces have material properties including an easily evaluated emissive power.
- Emissive meshes and primitive lights (spheres, rectangles, and so on) are enumerated and provided to RTXDI.
- An existing G-buffer containing data needed for evaluation of materials and lighting for any combination of lights and screen pixels.
Any emissive surface provided to RTXDI automatically illuminates its surroundings. You can easily add light primitives not supported by RTXDI by providing an appropriate sampling function for the new primitive.
Technical background
Realistic rendering requires solving the rendering equation one time per pixel, integrating the incident light to determine how much reflects toward your camera. For direct illumination, the integral devolves into a sum over all lights. Computing a pixel’s direct lighting thus requires querying each light and testing its visibility. Unfortunately, this scales linearly with light count.
Modern engines use tiled light culling, potentially visible sets, and similar techniques to reduce costs by culling lights per-pixel. However, dynamic visibility relies on re-computing shadow maps each frame. This is a substantial cost in modern engines and does not scale to many lights. Beyond static shadows in baked light maps and a few hero lights, additional emitters in today’s games typically provide localized bounce lighting without visibility.
Consider photos and renderings with hundreds of complex area lights, such as the one shown in Figure 2. How many shadows are visible in any pixel? In the circled region, the intuitive answer is “none,” despite the complex visibility to scene lights.
Over the larger image, generalize this question to “what is the maximum number of shadows perceptually visible in a pixel?” The answer is quite small, suggesting that you could test visibility only for just a few perceptually important lights rather than querying visibility to all lights. An important caveat, however, is this set of important lights changes per pixel. All lights may cast important shadows somewhere, meaning with rasterization we still need shadow maps for all lights.
However, a ray tracer can trace unique shadow rays for each pixel. In that case, the question becomes, which shadow queries are needed and how do you identify them? For most numerical integration, the answer to “which samples” is generally “use good importance sampling.”
Importance resampling
To achieve good importance sampling, we built on a technique called resampled importance sampling (RIS), introduced by Talbot et al. RIS amplifies sample quality, like how a denoiser amplifies image quality. The spatiotemporal reuse introduced by ReSTIR accentuates this benefit, enabling neighboring pixels and prior frames to efficiently guide the selection of better-quality shadow rays. This enables the use of cheap, easy-to-define sampling distributions suitable for fast GPU execution as inputs, letting RIS and ReSTIR amplify the quality.
Filtering has long been built on the observation that nearby pixels have similar colors. Temporal antialiasing, image denoisers, and upscalers all use this to help fill missing details. Importance resampling techniques extend this observation, for example, to nearby pixels should use similar light samples. Essentially, ReSTIR filters the sampling distributions (before lighting) to select high-quality shadow rays. This significantly reduces the required ray budget.
From a mathematical perspective, resampling techniques refactor the lighting equation in an arbitrary, developer-controlled way. Using this control, you can reorder computations so that expensive operations, like ray tracing, occur only infrequently.
You can use this control in other ways, like reordering computation to reduce memory incoherence and code divergence throughout the lighting process. An appealing property of ReSTIR-like resampling is its use of constant numbers of light samples across scenes of widely varying complexity, giving outstanding scalability. However, code and memory divergence can also increase costs as complexity grows, for example, varying with lights residing in L1, L2, or global memory. RTXDI introduces additional resampling to reorder computations, keeping lighting costs near constant across scenes with widely varying light types and counts.
Integration considerations
Because RTXDI acts as a quality amplifier, improving the selected shadow rays, it can be integrated into an engine in various ways.
In its simplest form, RTXDI replaces an engine’s direct lighting system, including all costs associated with culling and identifying important lights, shadow mapping, and ambient occlusion. By design, it naturally localizes samples to visible lights and provides contact-hardening soft shadows from area lights, hard shadows from point and directional lights, and ambient occlusion from large hemispherical sources.
Of course, it can build on existing engine capabilities to increase scalability. For instance, given a large scene and a potentially visible set of lights, RTXDI can select shadow rays from only known-visible lights. This often improves quality over standard RTXDI but inherits the limitations of any underlying data structures.
You also have control over which computations are used to improve sampling. For instance, our prototypes use our full material model for best reuse from nearby pixels. However, where full material evaluations are costly, for example, due to multi-layered or composite models, simpler material approximations can drive the resampling.
Conclusion
RTXDI encapsulates a simple, robust, and powerful implementation of spatiotemporal importance resampling that powers complex, dynamic lighting from a massive number of lights. Its assumptions about your engine are minimal, either replacing existing lighting solutions or building on them.
This statistical approach to light sampling removes many limitations in traditional raster engines. Artistic decisions should no longer revolve around a limited set of a dozen or fewer lights. All lights should be treated equally!
For more information, see the following resources:
- Early access to RTXDI
- Rendering Games with Millions of Ray-Traced Lights (GTC Fall 2020)
- Rendering Millions of Dynamic Lights in Real-Time
- New NVIDIA Path Tracing Breakthrough Interactively Renders 7,000 Dynamic Lights
- Spatiotemporal Reservoir Resampling for Real-Time Ray Tracing with Dynamic Direct Lighting
- Importance Resampling for Global Illumination
