HDR for UE4
As you may have read in our last post, there are several things that you may want to do to provide a great HDR experience from your game. To help demonstrate these things, we went and plumbed them into the Unreal Engine. This is all available on GitHub right now for those with UE4 access. (https://github.com/ehartNV/UnrealEngine_HDR) This was designed to integrate with UE 4.12 with little hassle, and it handles just about everything that you need to make an HDR game in UE4. The only truly missing bits are the way in which you wish to expose things to the user and tuning that might be specific to your content. We’ve included an integration guide in the branch to help explain the changes, but I’ll provide an overview here as well.
The core addition to UE4 to support HDR displays is a pipeline to get the high-precision data to the framebuffer. First, this means changing the allocation of the swap chain to being fp16. Next, it means modifying stages that ordinarily would damage the range or precision. This includes adding an enhanced ACES tone mapping path (UE4 actually has an ACES-inspired one in place already), and changing the formats on some intermediate stages. The final step is to make other pieces compatible. This includes adjusting the UI to deal with overlaying an sRGB UI on top of an HDR scene, and recording screenshots in EXR format to support the extended range. Here is a more complete list of the features you can expect:
- High-precision output for HDR displays
- Metadata signalling for displays
- HDR screenshots as EXR files (data stored as scRGB in file)
- Extended LUT encodings for HDR
- Enhanced UI controls for HDR
- ACES tone mapping support
- Full ALU tone map implementation
- Visualization of dynamic range
On the mechanics of how things work, the RHI was extended in two ways. First, the viewport resize functions now take an optional format parameter. This allows the engine to dynamically transition in and out of HDR without taxing the SDR path or requiring a full engine restart. Second, the RHI adds a new global variable to identify whether an HDR compatible display is detected. The rest of the changes are mostly clustered in the Slate RHI renderer and the PostProcessing portion of the Renderer. A few changes have to be sprinkled through the engine to handle ripples tied to interface enhancements.
It is worth mentioning some caveats here. First, the support is presently Windows only. Linux and OSX need to develop an infrastructure to support the enhanced range. We are really excited about the Android space, especially with products like SHIELD, so stay tuned for updates in this realm. Second, the support is presently restricted to D3D11. When I integrated the support, the D3D12 RHI was explicitly disabling fullscreen exclusive. (r.fullscreenmode 0 is ignored by the engine if the RHI is D3D12) This prevents the display driver from getting the full-precision data necessary. Once this gets resolved, the D3D12 support is a simple extension on what is there for D3D11. Additionally, the OpenGL RHI is presently not supported either. The challenge there is working through potential issues with SetPixelFormat. Stay tuned for possible updates. Finally, the LUT-based color grading is disabled in the HDR mode. The primary reason is that these grades are defined in an SDR color space and therefore destroy the range and precision. UE4 offers non-LUT color grading - that is still completely compatible with the HDR pipeline.
Finally, the changes here aren’t just about support to display the scene on HDR displays, they are also about visualization and debugging. As HDR displays aren’t a pervasive item yet, it has modes to allow artists to visualize the luminance in the scene without an HDR display. It also allows the export of HDR data without tone mapping for analysis or development in an external tool. These can all help your artists make the content the best it can be.
To visualize scene luminance:
This will produce a color ramp in log2 space with cyan placed at middle gray. White will represent strong highlights that should max out the luminance on a 1000 nit display.
To capture untonemapped HDR:
r.hdrOutput 1 hdr.TonemapMode 2
This places the engine in HDR mode (you can do it even on SDR displays, bright stuff will just appear clipped) to enable the full precision output. It then tells the tone mapper to operate in pass-through mode. Once that is done, every screenshot you take will dump an EXR with no tonemapping.
So, there you have it. HDR support that should be ready to use. If you have a display and want to try it out right now, you can sync the branch, build the code, and grab content straight out of the launcher. I personally recommend Infiltrator as content that looks quite good in HDR. The SunTemple demo also does well. Just run these demos in fullscreen with r.fullscreenmode set to 0 to enable fullscreen exclusive mode, and set r.HdrOutput to 1 to turn on the HDR display support. It is important to remember that this content was developed with no HDR display support in mind. (Other than theories around general HDR rendering) They still look great, and it is amazing to think of what can happen when an artist has the tools and motivation to double-check how their work holds up in HDR.