Simulation / Modeling / Design

Coffee Break: Ray Plus Raster Era Begins

After decades of research, NVIDIA has unearthed the holy grail of video game graphics: real-time ray tracing! This series of videos will explain why you need to add ray tracing to your pipeline now. The idea isn’t to use ray tracing as the only rendering technique, but to combine it with traditional rasterization to generate the best possible blend of performance and image quality.

It can be hard to find an hour in your busy work schedule to view a long presentation.  We want to help you with that by re-working our best long-form talks into a series of coffee break friendly clips. We’ll cover the following topics in this video set:

  1. Graphics in Video Games: 1978 – 2018 (5:33 min)
  2. Why You Should Embrace Rays NOW (5:12 min)
  3. Shadowed Area Lights and Ambient Occlusion (4:19 min)
  4. Dynamic Global Illumination (11:56 min)
  5. Beyond Triangles (3:25 min)
  6. Design First Around Rays (3:53 min)
  7. Falcor and Abstraction (9:07 min)
  8. Falcor Rendering Library (8:13 min)

If you can put aside just ten minutes a day for one week, you’ll know more about implementing ray tracing than most of your peers! Alternatively, you can skip straight to the bits that that offer insights particular to your interests.

Graphics in Video Games: 1978 – 2018 (5:33 min)

How did we get to this point in the ray tracing technology curve? And where are we compared to the film industry?

Five Key Points from Part 1:

  • 1978 – 1999: the “Antiquity” era. Anything before programmable shading (Half-Life).
  • 2000 – 2018: the “Raster Heroes” Era.  Runtime GPU Rasterization and Offline CPU Ray Tracing (Uncharted 4, The Legend of Zelda: Breath of the Wild).
  • Starting now: the “Ray + Raster Coexistence” era. GPU accelerated baking, dynamic probes and light maps, hard shadows, refraction transparency, mirror reflection, and callable shaders.
  • The game industry is now where the film industry was 20 years ago: peak rasterization. You can see this if you view Pixar’s A Bug’s Life. Games look great through massive artist effort, but quality is about to explode through new ray tracing technology enabling new workflows.
  • Film has an advantage, because it’s not interactive! We can’t just “do what film does” even though we share goals and some underlying techniques.

Part 2: Why You Should Embrace Rays NOW (5:12 min)

It’s critical to get a head start with ray tracing, as it’s about to redefine game graphics.
In this video, we make the case for this, emphasizing the key point that run-time rendering technology is ultimately valuable because it speeds production.

Five Key Points from Part 2:

  • The goal: combine the best of today’s rasterization VFX and new real-time ray tracing to produce simpler, faster pipelines that are easier to understand. The new pipelines enable artists to be 4-5x more productive than today for similar or better cinematic effects.
  • It’s production workflow that we are really concerned about. Improved artist throughput is worth two milliseconds at runtime.
  • Pixar’s Coco is pure path tracing. No shadow maps, no separate lighting passes, all done in camera. Incredible work flow, incredible visuals. That’s our guiding star.
  • Start redesigning now. Embrace first-class rays today to harness the curve of the coming years. Schedule removal of pain points from runtime VFX complexity via ray tracing.
  • The next few years will be a golden age of rasterization + ray tracing hybrid methods that are fast, robust, and easy to develop for compared to all-raster engines.
  • Full-blown path tracing will be possible in a few years in real time because of denoising technologies. Start learning about denoising while we’re in the ray + raster era.

Part 3: Shadowed Area Lights and Ambient Occlusion (4:19 min)

This video covers hybrid ray/raster algorithms, which can augment or improve the quality of existing raster-based solutions.

Five Key Points from Part 3:

  • Stochastic shading is noisy. So use LTC for perfect unshadowed area light shading.
  • We cannot blur area shadow samples directly due to correlation problems as it makes dim highlights instead of cutouts. Instead, modulate the image by the ratio of denoised ray traced shading to unshadowed analytic shading.
  • Without our special new denoiser, no blurring of lighting or materials, and only one pass per frame for any number of lights.
  • You can augment an existing algorithm and produce much nicer results even though you’re limited to a small number of rays per pixe.
  • Hybrid work in which you take samples from screen space and then augment those samples with ray tracing can produce a very nice result that is close to ground truth.

Part 4: Dynamic Global Illumination (11:56 min)

This video discusses what gets us really excited as developers – global illumination, which solves the biggest lighting problems.

Five Key Points from Part 4:

  • What do you do about ghosting and all other problems with temporal accumulation? These problems mostly occur due to working in screen space (screen boundaries, disocclusion when objects move, etc.). The solution: don’t accumulate in screen space at all. Instead, accumulate in light map space.
  • Use light map space as a cache for illumination. Falloff is guided by object motion at the hit point and at the distance to the closest occluder.
  • When refining dynamic light maps, use lit texels to light the next frame.
  • You’ll be using a light map denoising filter with a with a kernel based on distance to occluders, motion, etc.
  • “Light map” can be replaced by your favorite data structure (probes, voxels, hash, etc.)

Part 5: Beyond Triangles (3:25 min)

You see a lot of focus on using ray tracing for intersecting triangles, but it’s not the only thing you can do with ray tracing. In this video, we explore what can be done beyond triangles, and the benefits of using NVIDIA RTX Acceleration.

Five Key Points from Part 5:

  • DXR supports custom intersection shaders.
  • The traversal quickly finds which shader(s) to run.
  • Use custom intersections to avoid artifacts and compute analytic coverage.
  • Rays can be be faster and alias less than tri raster.
  • The fundamental problem with tessellation and ray tracing is normally you would have to tessellate your mesh, then build an acceleration structure over the tessellation, and then trace against that. This can be fairly expensive. You instead gain great efficiency by doing the subdivision inside your shader instead. For instance, you can intersect over 1M hairs in real time.

Part 6: Design First Around Rays (3:53 min)

When teams design around rays first, ray tracing brings efficiency back to art pipelines.This makes for happier, more creatively fulfilled artists, who can rapidly create more content which is unique and higher quality.

Five Key Points from Part 6:

  • Design VFX around rays first for robustness and leverage screen space for acceleration.
  • Keep high-level, good tradeoff algorithms: they work with rays too. Tiled deferred, instancing, material & geometry LOD, TAA, occlusion culling, geometry streaming.
  • Games already look great, so start looking at where it’s costing too much for things to look great, then apply ray tracing solutions in those cases to accelerate overall rendering performance.
  • Ray tracing is more than just shiny reflections and shadows. When ray tracing is fully integrated, you’re going to get true cinematic lighting.
  • Ray tracing brings us to the point where each studio can use the same set of engine tools, but have a really unique look. You’re going to get back to the point where your VFX teams and artists have headroom again, allowing artists time to explore new looks, and new mechanics inspired by those looks. Happy artists is the biggest payoff of ray tracing!

Part 7: Falcor and Abstraction (9:07 min)

In this video, we explain what Falcor is, make clear why it is valuable to developers, describe how DirectX Ray Tracing has been integrated, and show off what Falcor can do.

Five Key Points from Part 7:

  • Falcor is NVIDIA’s real-time rendering R&D infrastructure, which has increased idea throughput and improved collaboration.
  • Currently, Falcor can render images at 70% of a AAA-engine image quality. It uses a physically-based shading system, and has a modular rendering techniques library.
  • Falcor is intuitive and simple to use, with API-abstraction and clear interfaces and it’s just 50,000 lines of code.
  • Falcor supports the latest technologies and APIs, including DirectX Raytracing.
  • Falcor allows you to experiment with shading techniques – including DXR – before integrating them into your own engine.

Part 8: Falcor Rendering Library (8:13 min)

Coding time! We’re going to walk you through how to write a basic ray tracer that shoots a shadow ray and does some simple glossy reflections with Falcor. We’ll also run down what is available in the Falcor 3.0 Effects Library.

Three Key Points from Part 8:

  • The only thing you need to know about DXR to get started is the ray tracing pipeline.
  • The Falcor 3.0 Effects library includes:
    • Improved physically-based shading system
    • Point- spot-, directional- and area-lights
    • Light-maps
    • Light-probes
    • Modern shadow-map techniques
    • Skinned animations
    • Cameras, meshes and lights motion-paths
    • Post-processing techniques
    • Temporal Anti-Aliasing
    • Screen Space Ambient Occlusion
    • Integration with NVIDIA’s VXGI, Shadowlib and HBAO+
  • The Open Research Content Archive can be found at, with great content from NVIDIA, Amazon, and Epic Games.

We hope you’ve gotten value from the this installment in NVIDIA’s Coffee Break series!

To learn more about NVIDIA RTX Technology, check out the developer page. You can also find out more about Falcor here. The full GDC presentation can be found on NVIDIA’s Developer YouTube page.


Discuss (0)