With the recent introduction of DirectX Raytracing and NVIDIA RTX, interoperability between the rasterization pipeline and the ray tracing pipeline can be applied in real-time applications.
The new technology does a lot of heavy lifting, but developers can’t just throw rays everywhere. They still have a limited ray budget to work with. They need to know how and when to place the rays to get meaningful, additional contributions from ray traced samples.
In Chapter 22 of Ray Tracing Gems (free ebook available here), NVIDIA engineers Josef Spjut and Adam Marrs describe how to take advantage of the benefits that temporal anti-aliasing provides, while addressing its failures. They also explain how to work within the constraints of a real game engine.
At SIGGRAPH 2019, Spjut presented a talk sharing his team’s research on improving temporal anti-aliasing with adaptive ray tracing. The full video can be viewed here. Below is a five minute excerpt, which introduces their approach.
“As an overview of our approach: first, we just run temporal anti-aliasing. This ensures that the results will not be worse than temporal anti-aliasing as a baseline, if we choose to use those results. Second, we compute the segmentation mask, which hopefully will help us identify which parts of the image temporal anti-aliasing is failing to solve for,” explains Spjut. “Then, we replace the typical heuristic that is used in temporal anti-aliasing, to try to overcome the blurring and ghosting with adaptive ray tracing. We choose the rays and we put them where they need to be. Then, we’ll run FXAA on some other regions, and composite all of those results into a final image”.
The image below visualizes the technique. It’s an image of a segmentation mask. In the blue region, results from typical TAA are used. In the yellow region, rays have been shot per-pixel to anti-alias. And for the red region – which is a region of the screen recently revealed due to camera motion – one would run FXAA, then use normal TAA on subsequent frames.
The full talk, entitled SIGGRAPH 2019: Ray Tracing Gems 1.1 – Improving Temporal Antialiasing with Adaptive Ray Tracing, can be found here.
