Developer Blog

Data Science | Graphics / Simulation |

Generating Ray-Traced Caustic Effects in Unreal Engine 4, Part 1

Caustics are common optical phenomenon in the real world. From the sloshing sparkles by water surfaces to the curved highlights in the backlight of clear glass, they are everywhere. However, simulating accurate caustic in 3D graphics is not an easy job. For those who have been involved in creating ray tracers, you might know that the difficulty is twofold: following light rays bounced off high polished surfaces or going through transparent objects multiple times makes the computation time-consuming and tuning the counter-intuitive settings requires the artists to put in extra effort.

With today’s game engines built upon the rasterization pipeline, developers are usually choosing to ignore caustics or go with some simplistic fakes, making them visually implausible and hard to interact with. Fortunately, the arrival of GPU ray tracing in recent years has brought the possibility of accelerating photon mapping, the most popular technique for rendering accurate caustics, into real-time territory.

Since then, we have developed a few new methods for using fast GPU photon mapping to generate caustics. Our work is implemented on Unreal Engine 4 so that most end-users can access it with the quality of a commercial product as well as benefiting through open-source repositories. The effects focus on two areas:

  • The delicate caustics around metallic and transparent meshes, called mesh caustics (Figure 1)
  • The caustics above and under water surfaces, emphasizing fluid motion and being able to cover large area, called water caustics (Figure 2)

Both methods are low cost for high framerate usages, fully interactive with dynamic surroundings, and ready-to-use with no data preprocessing. In addition, the whole sets of editing UI, command lines, and performance stats are also included as parts of the integration. The beta version of the source code and sample assets have been released on UE4 NVRTX_Caustics depository under the NVIDIA GameWorks license. For more information, see the Release Information section at the end of this post.

Screenshots of mesh caustics in the test scenes of "POV-Ray Glasses" and "Prisms"
Figure 1. Screenshots of mesh caustics in UE4 editor. Top and middle rows: A collection of glass caustics from the classic “POV-Ray glasses” by courtesy of Gilles Tran, in which the caustics and glass meshes are ray traced with 12-bounce refraction and reflection. Bottom: A simple test scene showing the dispersion effect through multiple prisms.
Screenshots of water caustics in the test scenes of "Swimming Pool" and "Seaside Town"
Figure 2. Screenshots of water caustics. Top row: A testing UE4 scene of swimming pool. Bottom row: The seaside town scene covered by large sea surface. Water caustics can be seen above water (reflected) and underwater (refracted) in both scenes.

We introduce the beta release of mesh caustics and water caustics in two posts. In this first part, we introduce mesh caustics: the basic workflow of the algorithm, several improvements on image quality and efficiency (adaptive sampling, photon differentials, and so on), and profiling results. Besides the caustics, we also realized that multi-bounce refraction and reflection are essential to produce the high-quality images in such scene setups. Thus, we present a few enhancements over UE4’s ray tracing pipeline for a better outcome for multi-bounced rays.

Part two of this series introduces water caustics: Generating Ray-Traced Caustic Effects in Unreal Engine 4, Part 2.

Mesh caustics

We started simulating mesh caustics by restructuring techniques from a category of light propagation methods called photon splatting, a variation of the classic photon mapping. Originally, photon mapping was proposed as a two-stage method:

  • Tracing – The forward tracing of light particles, that is, photons, from light sources through the scene.
  • Density estimation – Searching around the given surface points to calculate the density of nearby photons for illumination reconstruction.

 In our approach, the second stage is altered with photon splatting. Instead of finding out the local photon density, it draws photons against the scene depth using additive blending after the photons hit opaque surfaces. The splatting has several advantages over the classic method. It uses GPU rasterization to speed up, only simple data structures are needed for storing photons, and most importantly, it allows coupling with photon differentials, an essential technique to boost image quality, at low computational cost. The algorithm of photon differentials tracks the small perturbations of each ray and uses them to estimate both the size and the shape of the photon. It creates an elliptical footprint when it hits a non-specular surface. With such an anisotropic footprint setup, the algorithm allows reconstructing caustic patterns at a low sampling rate.

To achieve real-time rendering, we revamped the processes of photon tracing, splatting, and differential calculating for making the most of GPU ray tracing and rasterization capacities. We introduced a temporal feedback to control the photon density in screen space so that the quality-performance tradeoff can be easily balanced. We call this approach adaptive anisotropic photon scattering (AAPS).

Basic workflow

AAPS maintains the following buffers:

  • Task buffer – A structured buffer recording light rays to trace in the current frame.
  • Photon buffer – A structured buffer recording photon data: hit position, photon footprint, and intensity.
  • Feedback buffers – Two textures in light-view space tracking the accumulated screen-space area of photon footprints and intensity variance over the last several frames.
  • Caustics buffer – A render target for splatting photons in screen space.

The workflow is executed in four steps (Figure 3):

  • Emit photons according to the task buffer and trace through the scene. For any photon hitting an opaque surface, create a record in the photon buffer and add its footprint area to the feedback buffer.
  • Perform photon splatting on the caustics buffer: each photon from the photon buffer is drawn as an elliptical footprint. The shape and intensity of the footprint are adjusted by the photon differentials and surface normal, respectively.
  • Apply the caustics buffer to the scene by using the deferred lighting functions provided by the engine.
  • Combine the feedback buffers of the previous frame and the current frame to generate the task buffer for the next frame.
Basic workflow of AAPS
Figure 3. The AAPS workflow.

Using mesh caustics in the UE4 editor

Beta release notice: The parameters, editing UI, and command lines discussed in this post are subject to change in the final release version. Before using the caustic features, make sure that the ray tracing settings in UE4 are configured properly.

To enable mesh caustics in UE4 editor, start by configuring the following settings:

  • Enable mesh caustics in the current post-process volume by checking Ray Tracing Mesh Caustics, – Enabled
  • Pick a light source for casting caustics by checking Cast Mesh Caustics in the light source’s property sheet.
  • Select the objects to produce caustics by selecting a mesh and navigating to its materials. Open the material editor to the Details tab and check Cast Ray Traced Reflection Caustics and Cast Ray Traced Refraction Caustics, if the material is transparent.

After configuring those settings, Figure 4 shows that the default caustic effects can then be seen around the glass and metal objects. Most light properties, including intensity, color, texture mask and falloff, have direct effects on the appearance of the caustics. Four types of lights are supported so far: directional, point, spot, and rect lights. Up to eight lights can cast caustics in the same view.

Steps to enable mesh caustics in UE4
Figure 4. Enable caustic effects: First row: A simple setup containing a directional light, an opaque box, a glass ball, and a metal ball. Second row: Contains post-process volume, light source and materials configured with the caustic flags, and then the refracted and reflected caustics can be seen around glass and metal spheres, respectively. Third row: Adds a point light to create an additional set of caustics.

Besides the light sources, the shape, intensity, and color of the caustics are also affected by the base color, opacity, normal map, index of refraction, roughness, and metallic values of the meshes’ materials. Opaque objects with higher roughness or lower metallic values cast weaker caustics. Transparent objects can cast both reflection and refraction caustics, which can be enabled by checking both Cast Ray Traced Reflection Caustics and Cast Ray Traced Refraction Caustics options in the material editor’s Details tab. The index of refraction (IOR) is linked to specular node in compliance to Disney’s PBR model, varying between 0 to 1.

In the post-process volume, a set of parameters are added for tuning mesh caustics:

  • Intensity scales the intensity of caustics patterns.
  • Max Trace Depth limits maximum bounces for each ray, which may affect tracing performance.
  • Enable Temporal Filtering toggles temporal filtering, which reduces flickering effect and improves performance.

Photon differentials

Visualizing the photon footprints
Figure 5. Left: Photon footprint can be seen at low configuration, revealing its anisotropic nature. Right: The reconstructed caustics pattern.

AAPS employs the technique of photon differentials during the photon tracing stage. It’s an effective technique to reconstruct clear optical patterns while keeping the noise at a low level. By treating each photon as a beam traveling through the scene and ended as an elliptical footprint on a surface, it manages to cover the caustic region by a few tightly packed photons, which in turn can produce sharp details without random sampling. In Figure 5, the left image displays the anisotropic nature of photons by setting the photon size to a small value; and the right image shows the result pattern when each photon is blended at right size with no help of denoising.

Adaptive sampling

We used an adaptive sampling scheme to cast more rays in visually significant regions, such as caustics close to the camera or perceived flickering areas. The scheme is implemented through the feedback buffers, which serve as a heatmap of photon density in screen space. Figure 6 shows that one feedback buffer is oriented to a light source. When a photon is stored in the photon buffer, its screen space info is added to the feedback buffer. The data in the buffer are then used for allocating photon budget for the next frame. High density and closeup regions receive more photons with smaller size, while low density and distant regions get fewer photons with larger size. The feedback buffer from the previous frame is also used in this process to improve the temporal stability.

Illustration of the feedback buffers
Figure 6. Feedback buffer is placed in the light-view space to track screen space photon density. Top: the illustration of feedback buffer for capturing the photon distribution in a certain range. Bottom: the debug view of the feedback buffer of the test scene.

Within the parameters to control adaptive sampling, the resolution of feedback buffer, photon size and variance gain are most important as they combine to decide the overall quality and cost of AAPS:

  • Mesh Caustics Precision in per light properties—The parameter decides the resolution of the feedback buffer, which is 128×128 by default. Higher resolutions produce finer adaptive photon distribution, low frame latency during fast camera moves, but with higher cost. Lower resolutions are better on performance, but frame lagging may be observed for small caustic patterns when the camera changes quickly. Different lights can choose different resolutions.
  • Adaptive Photon Size in post-process volume properties—The parameter controls the target photon density and can be used for tuning smoothness. Lower value indicates higher density and more caustic details, but higher cost. Higher value gives blurry result but fast rendering time. For comparison, see Figure 7.
  • Adaptive Variance Gain in post-process volume properties—The parameter focuses on adjusting temporal stableness, that is, flickering. Higher value results in placing more photons in high variance regions to reduce the flickering artifact, but which costs more time to render.

To check actual photon density, in the viewport menu, choose View, Ray Tracing Debug, Mesh Caustics Debug Data, and then set the Debug Screen Data Type option in the post-process volume to Photon (Figure 7).

Comparing the two different values of Adaptive Photon Size
Figure 7. Comparison of Adaptive Photon Size set to 20 (upper left) and 80 (upper right). The second row shows the debug view of photon density.

The ray-tracing debug menu also provided an overlay live chart to show the total photon count over past frames (Figure 8). It can be turned on by setting Debug Light Data Type option to Photon Count in the post-process volume. The chart is handy when you are tuning the adaptive parameters for better quality and performance.

Live chart of total photon number
Figure 8. Debug overlay chart of total photon count over past frames. The maximum photon count in the chart is set to 1,000,000 by default.

Additional performance optimizations

There are a few additional tweaks for optimizing the performance. The culling parameters in post-process volume set thresholds to discard photons if the energy they carried drop below a certain low level (Figure 9).

  • Mid Cull threshold—During the process of photon tracing, discard the photon if its energy is below this threshold. Higher value helps more on performance but may result in noise, darker caustics, and losing details.
  • Final Cull threshold—Discard the photon at the time it is stored to the photon buffer, if its energy is below this threshold after interacting with the surface; that is, darker surfaces have less photons stored. Higher value gives better performance but may lose details.
Illustration of the two photon culling methods
Figure 9. Culling photons during the tracing and after hitting opaque surfaces.

Dispersion

For transparent objects with high IOR-like diamond, or optical devices like prisms, AAPS can produce a dispersion effect based on the RGB triplet of the light source color. To enable dispersion, in the material editor’s Details tab, set Ray Traced Caustics Dispersion Amount to a value greater than 0. To avoid separated color bands, check the Ray Traced Caustics Jittered Dispersion flag (Figure 10).  

Simulating the light dispersion through prisms
Figure 10. Simulating light dispersion. Left: Dispersion amount set to 0. Middle: Dispersion amount set to 0.5. Right: Jitter enabled to achieve continuous spectrum.

Enhancements for ray-traced transparency in UE4

Comparing the three different values of maximum refraction bounces
Figure 11. Comparison of maximum refraction bounces (command line r.RayTracing.Translucency.MaxRefractionRays) set to 3, 5 and 9 from left to right. Notice the incorrect black regions on glass meshes due to lack of bounces. For this scene, the parameter of maximum bounces is set to 12 to produce images shown in Figure 1.

Because the caustic effects are commonly featured in sceneries with multiple glass objects, we found that it is necessary to improve the quality and efficiency for the current ray-traced transparency code in UE4 to make the overall image appealing (Figure 11). However, rendering multi-layered glass is a challenging job for both rasterization and ray-tracing pipelines. We introduced the following enhancements for better rendering of transparent-rich scenes:

  • Implement light absorption for solid transparent objects. With the calculation for absorption, the light transmitted through volumes varies with the thickness of object parts. Such appearances are crucial for tinted glass and liquid.
  • Use throughput cull to drop insignificant ray path segments during the tracing.

To enable the enhancements in UE4 editor, in the post-process volume, set the Type option under the Translucency category to Ray Tracing, and check Enable Absorption. In the material editor, check the Ray Traced Translucency Absorption option. Figure 12 shows the visual difference before and after enabling absorption materials.

Comparing before and after enabling absorption
Figure 12. Comparison of before (left) and after (right) enabling the absorption. The materials’ opacity values have been adjusted to achieve similar results.

The absorption material is reusing the parameters Base Color from the existing model, which determines the tinted color after absorption, and Opacity, which determines the amount of light being absorbed (Figure 13).

Absorptions with different opacity values
Figure 13. Absorptions with different opacity values.

To reduce the ray-tracing cost of multiple bounces, we employed a throughput cull method to drop the reflection and refraction paths whose contribution to the final image are below certain criteria (Figure 14). The parameters Min Reflection Throughput and Min Refraction Throughput in the post-process volume set the ratios of the final primary ray intensity. A sub-tree of secondary rays is pruned if its contribution is estimated less than the thresholds.

Optimizing multi-bounce refraction by throughput culling
Figure 14. The throughput cull method tracks the portion of light that contributes to pixel color and stop tracing when the portion falls below the given thresholds.

Limitations and extended usages

The mesh caustics integration in UE4 has the following limitations:

  • Reflected and refracted caustics are generated in two separate passes, which means that having both types of caustics for one object nearly doubles the cost. We suggest turning off reflected caustics for transparent objects if possible.
  • Caustic effects seen through reflection and refraction are limited to the screen space. For example, if you place a mirror near a surface that receives caustics, you can only observe the caustics that fall in the current viewport through the mirror.
  • The support for rect lights is not optically accurate. For performance consideration, the soft caustics cast by rect lights are done by modifying the photon differentials on both ray origins and directions, which means it may not produce the correct contact hardening look with regard to ray-traced soft shadows.
  • The current implementation doesn’t consider roughness value in a physically accurate way. Caustics are still sharp for rough surfaces.
  • The spectrum separation after dispersion is not physically accurate because the RGB color triplet lacks continuous spectrum information.

For extended usages other than regular caustic effects from glass or metal surfaces, the mesh caustics integration also works for rendering transparent shadows and light spots through textured windows. Figure 15 shows a scene with stained glass. The light cast through the windows can be efficiently simulated as a one-bounce refracted caustic. The cost is light even if the effect covers a large portion of the screen.

Using mesh caustics to simulate transparent shadows
Figure 15. Using refracted mesh caustics to simulate transparent shadows cast by the stained-glass windows.

Performance

We picked two scenes for a performance test. The first scene is the classic POV-ray glasses scene shown in Figure 1, in which all transparent objects have both reflected and refracted caustics enabled, and the number of ray bounces is up to 12. The other scene is from the game Bright Memory: Infinite containing a shattered glass bottle (Figure 16), where the caustic photons bounce up to eight times before hitting the ground. Both scenes are tested against 1920×1080 and 2560×1440 resolutions on several GPUs. The frame-time breakdowns are obtained from the UE4 command stat gpu and shown in Figure 17.

Mesh caustics cast from a shattered glass bottle in the game Bright Memory
Figure 16. A cutscene from the game Bright Memory: Infinite featuring the mesh caustics cast from a shattered glass bottle. The shot is used for the performance test.
Rendering time breakdown of the scenes of POV-ray glasses and Bright Memory demo
Figure 17. Rendering time breakdown for the scenes of POV-ray glasses (Figure 1) and Bright Memory demo (Figure 16) at resolution 1920×1080 and 2560×1440.

The performance charts show that the cost of caustics is fractional in comparison to other ray-tracing regimes—for example, the multi-bounce reflection and refraction on the glass objects spanning the 60%+ frame time but which are essential for visual quality in a caustic-rich scene. That said, for better performance, you may have to put more effort toward finding the acceptable appearance of translucent materials other than tweaking caustic parameters. We also suggest enabling the enhancements for the ray-traced transparency described in previous sections.

Further investigation based on profiling tools shows more details of mesh caustics overhead:

  • In the test of POV-ray glasses, the caustic rendering takes 0.8~2.9ms under 1080p across various GPUs, and 1.0~3.6ms under 1440p.
  • In the test of the Bright Memory demo, it takes 1.0~3.4ms under 1080p across various GPUs, and 1.2~4.3ms under 1440p.
  • The cost of rendering mesh caustics is mainly affected by the number of photons, maximum tracing depth, and the screen coverage. The first two factors relate to photon tracing, which has roughly 60% of the caustic rendering time; the third decides how many screen pixels are blended during photon scattering, which takes about 20% of the caustics rendering time.

To keep the cost of photon tracing under control, the key is to reduce the unnecessary photons:

  • Disable reflected caustics for transparent objects if possible. For instance, if a light is behind a transparent object most of the time, the reflected caustics are insignificant and should be turned off.
  • Always tune the options of adaptive photon size, adaptive variance gain, and the two cull thresholds mentioned in the previous sections to find the minimum acceptable values.
  • Use the debug overlay chart of photon count to track the total photon number. For typical usages, 50–100K photons per light source is good enough for outputting high-quality results.
  • Under high resolutions like 4K and 8K, consider using an upscaling technique, for example, DLSS, to reach a high framerate if available. Otherwise, try the half-resolution option to save blending bandwidth during photon scattering (Caustics Buffer Scale under the Ray Tracing Mesh Caustics – Advanced group in the post-process volume).

Release information

The UE4 integration of caustic effects are available for early access at the UE4 NVRTX_Caustics depository. To obtain the access to the branch, follow the Unreal Engine instructions.

Besides ray traced caustics, this branch contains a series of new NVIDIA RTX features and improvements over existing techniques, including enhanced hybrid translucency, multi-bounce refraction optimization, and a new SVGF-based RTGI denoiser. For more information about the sample assets, see the README file in the repository.

The current integration is based on UE4.25.3. All features and optimizations included in the standard UE4 NVRTX branch, for example, DLSS 2.1, hybrid translucency, ray-tracing debug, and visualization, are also included in the caustics branch. For more information about using these NVRTX features and installation steps, see the NvRTX/UnrealEngine GitHub repo.