Welcome to the VkRay tutorials. In these tutorials we are going to look at the basics of the ray tracing using Vulkan API. Tutorials will lead you from the beginning doing the initialization step-by-step moving to the complete example of the API usage. Each tutorial has the list of files which were created during it and may be used in the later tutorials. It is recommended to open tutorial files linked at the start of the each tutorial and read the code following along the tutorial text.
Here is the list of the tutorials.
Tutorial 2: Acceleration Structure
Tutorial 10: Instance Resources
Tutorial 11: Different Vertex Formats
Tutorial files:
Application.h
Application.cpp
01_InitRayTracing.cpp
We begin by defining application basis for the other tutorials. Application.h/Application.cpp files contain code for the Application class which takes care of the window creation, Vulkan initialization and main loop. It also includes several helper classes for dealing with resources, i.e. images, shaders, buffers, which encapsulate common Vulkan code. Their purpose is to reduce code clutter in the following tutorials.
TutorialApplication class inherits from the Application and shows how to properly create device with ray tracing extension enabled, resolve ray tracing specific function pointers and ray tracing properties which will be used throughout rest of the tutorials.
Tutorial files:
RayTracingApplication.h
RayTracingApplication.cpp
02_AccelerationStructure.cpp
The tutorial introduces RayTracingApplication class with the code developed in the Tutorial 1 to keep all the basic ray tracing functionality in one place. Upcoming tutorial classes will be inherited from this one.
Main goal of this tutorial is to show how to create and fill acceleration structures (VkAccelerationStructureNV). To use ray tracing functionality users have to convert their geometry data into API specific form. Take a brief look at CreateAccelerationStructures() function. It shows the process step by step. We begin with creating regular vertex and index buffers which contain scene geometry to be traced. To simplify the code, we will use only single triangle. The API does not use buffers directly, so we must create and fill one or more VkGeometryNV structs. These are used in the following steps.
The geometry is organized into two levels: bottom and top. For each of the levels we must create acceleration structures (referred further as AS). Bottom level AS wraps actual geometry data, while top level manages bottom level ones. For example, imagine the scene that contains the table and many chairs. In this case we will have only 2 bottom level AS (one for the table and one for the chair) and one top level AS linking to the bottom level ones. AS creation process consists of two steps: creation and building. At first, we create both AS using vkCreateAccelerationStructureNV(), allocating and binding memory to the handle (the process is similar to creating regular Vulkan buffer). As the result we have VkAccelerationStructureNV and VkDeviceMemory per AS.
Next, we create instance buffers. You can have single geometry object, however, it’s possible to have many instances of the same geometry with various transforms and properties. This is where instance buffers come into play. Regular Vulkan buffer is created and filled with VkGeometryInstance structs (one per instance). In our case it’s single instance, i.e. only one triangle will be shown. Using the scene example above – every chair should have its own instance object.
Finally, AS instances must be built. The process uses scratch buffer (it’s regular Vulkan buffer used internally during the calculations) and all the data we have prepared so far. Build process is done on the command buffer using memory barriers via vkCmdBuildAccelerationStructureNV() method. Building will transform the provided geometry into highly efficient format used during the ray tracing process.
Do not forget to destroy the allocated resources. Class destructor will take care of all created objects (we will not mention resource clean-up in the following tutorials, however you can always look at the destructor).
Tutorial files:
03_Pipeline.cpp
rt_basic.rgen
Continuing from where Tutorial 2 left off, we will now create VkPipeline object which references ray tracing objects.
Here we begin by creating VkDescriptorSetLayout/VkPipelineLayout objects. We will use two bindings here - one for the top level acceleration structure and one for the storage image. This image will contain the result of the tracing in the following tutorials as we won’t trace anything in this tutorial yet.
Ray tracing pipeline creation process differs from the regular Vulkan code. VkRayTracingPipelineCreateInfoNV struct must be filled and passed to vkCreateRaytracingPipelinesNV() method. Notably, we have to fill in shader stages and groups before doing that. Ray tracing introduces several new shader types, including ray generation, any hit, closest hit, miss and others. They have corresponding enum entries in VkShaderStageFlagBits, i.e.:
· VK_SHADER_STAGE_RAYGEN_BIT_NV
· VK_SHADER_STAGE_ANY_HIT_BIT_NV
· VK_SHADER_STAGE_CLOSEST_HIT_BIT_NV
· VK_SHADER_STAGE_MISS_BIT_NV
All tutorial shaders are contained in the Source/Shaders folder alongside the compiler and batch file which launches the compilation. At this point we only load single ray generation shader rt_basic.rgen. It is minimalistic shader which does not do any ray tracing yet, only stores UV coordinates calculated from the current pixel (in the same manner as compute shaders do) in the provided image. Notice new variable semantic rayPayloadNV which is not currently used but will be used during the tracing in the future. Of course, to compile this (and all other ray tracing shaders) we must request special shader extension GL_NV_ray_tracing.
To create the pipeline, we also need to describe shader groups. A group is represented by the VkRayTracingShaderGroupCreateInfoNV struct. Groups setup the ray tracing process, i.e. which shaders are called in which order, linking to the actual shaders loaded previously. We only have single shader in this tutorial thus only single group is created. However, there will be more groups in the further tutorials.
Now we have all required data structures and can finally create the pipeline. Notice that while we use VkRayTracingPipelineCreateInfoNV/ vkCreateRayTracingPipelinesNV() to create the pipeline, the result is represented by the standard VkPipeline handle. Binding and destroying done as usual in Vulkan. Pipeline binding is done in the RecordCommandBufferForFrame() function. We will add more code to this function eventually.
Tutorial files:
04_DescriptorSet.cpp
This tutorial adds two new functions: CreateShaderBindingTable() and CreateDescriptorSet().
Shader binding table (SBT for short) is the regular host visible Vulkan buffer filled with data about the shader groups (mentioned in the previous tutorial). It can also contain additional user data available during the shader execution (this will be demonstrated in the separate tutorial). SBT is created and then mapped as usual buffer. After that vkGetRayTracingShaderGroupHandlesNV() must be called to fill the SBT. We only have single group so far, so we start at firstGroup 0 and groupCount is 1. Notice that SBT is not used in this tutorial, however will be used in the following tutorial as the parameter to the trace function.
CreateDescriptorSet() contains mostly regular Vulkan code. Recall from the Tutorial 3 that we have two bindings in the layout, so we will need two descriptors. The only unusual code you will find here is VkWriteDescriptorSetAccelerationStructureNV struct that is referenced by the pNext field of the AS VkWriteDescriptorSet that is easy to overlook. Image write descriptor is done regularly.
In addition to the pipeline binding we also bind the descriptor set in the RecordCommandBufferForFrame() function now.
Tutorial files:
05_RayGen.cpp
The tutorial will finally show something on the screen. The only difference with Tutorial 4 is in the RecordCommandBufferForFrame() function. To invoke the ray tracing we call vkCmdTraceRaysNV() method. This call sets things in motion and execute raytracing shaders. On the input it has several parameters using SBT and the tracing area size (width, height, depth – similar to compute shaders dispatch).
Recall that currently bound ray generation shader (rt_basic.rgen) does not do actual tracing, but only outputs UV coordinates calculated from the pixel position. It’s calculated using gl_LaunchIDNV and gl_LaunchSizeNV built-in variables representing index of the current pixel and the dimensions passed to the vkCmdTraceRaysNV() method accordingly.
Tutorial files:
06_Shaders.cpp
rt_06_shaders.rgen
rt_06_shaders.rchit
rt_06_shaders.rmiss
In this tutorial we will do some real ray tracing! The tutorial uses new set of shaders which do simplistic tracing which results in the colored triangle drawn on the screen. The only change on the application side comprises of the new shaders loading and groups (there will be 3 groups now, one per generation, hit and miss routines).
rt_06_shaders.rgen shader uses new traceNV() function which does actual tracing and use ray origin, direction, payload and other parameters as the input. rt_06_shaders.rmiss shader will be called when no intersection has happened (i.e. ray has missed all the triangles) and will return the background color. rt_06_shaders.rchit shader will be called on the closest triangle hit and will output barycentric coordinates of the hit point (interpreted as the color) using hitAttributeNV semantic. In any case rayPayloadInNV variable semantic is used to write the resulting value (referenced as rayPayloadNV in the ray generation shader and is linked by the location index).
Tutorial files:
07_InstanceBuffer.cpp
As was noted in previous tutorials we can have many instances of the same geometry in the scene. This tutorial shows slightly more advanced usage of the instance structs than simply drawing the triangle.
The changes are in the CreateAccelerationStructures() function. First, vertex/index buffers now contain data for the triangle, the quad and the icosahedron (geometry array contains 3 elements as well). Second, we create 3 bottom level AS instead of 1 (one per primitive). Third, we create 5 instances in total – 3 for the triangle, 1 for the quad and 1 for the icosahedron. Notice that we change transform member of the VkGeometryInstance struct to change instance placement.
Tutorial files:
08_AnimateAndRefit.cpp
To support non-static geometry, we should update acceleration structures every frame so that we always have up to date data for the ray tracing routines. However, building AS from the scratch every frame is not very performant solution. To address this problem we can update AS using the call to vkCmdBuildAccelerationStructureNV() instead of doing full rebuild.
The gist of the tutorial is in the RecordCommandBufferForFrame() function. In addition to the regular binding and tracing functions we also call vkCmdBuildAccelerationStructureNV() to update bottom and top level acceleration structures (notice VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_UPDATE_BIT_NV flag passed to the VkAccelerationStructureInfoNV struct and VK_TRUE passed to the update parameter). The animation is done in the UpdateDataForFrame() function. It is possible to change vertex data, instance data or both before rebuilding acceleration structures. As a result, we get the animated triangle being ray traced.
Tutorial files:
09_SecondaryRays.cpp
rt_09_first.rgen
rt_09_first.rchit
rt_09_first.rmiss
rt_09_secondary.rchit
rt_09_secondary.rmiss
So far, we have only used single ray per pixel, terminating the tracing at the point of the first triangle hit. This tutorial features casting another set of rays upon hit. Notice new set of shaders, which contains shaders for the secondary hits and miss.
Ray generation shader calls traceNV() as before. First miss shader returns background color as expected. First closest hit shader, however, calls traceNV() once again specifying secondary hit and miss shaders in its parameters. Origin of the second trace is the hit point of the first trace. The direction is set to the imaginary light source (in this case it’s the opposite direction of the directional light). Notice that the sphere is also self-shadowed due to rays hit the sphere from the inside. Result of the second tracing is used to determine whether the point is in shadow or not (as secondary hit and miss return gl_HitTNV and gl_RayTmaxNV accordingly). We multiply this by the triangle color to get the final color. There is also one interesting detail contained in the rt_09_first.rchit shader – usage of the gl_RayFlagsTerminateOnFirstHitNV flag. It means that whenever we got any intersection (not necessary the closest one) closest hit shader will be called and the process will terminate. This makes sense due to it does not really matter which object blocked the light source thus casting a shadow.
On the application side we have 5 shader groups now: generation, first hit, second hit, first miss, second miss. Don’t forget that vkCmdTraceRaysNV() function should properly get offsets to the shaders as shown in the RecordCommandBufferForFrame(). Pay attention to the parameters to traceNV() method in the rt_09_first.rchit shader. sbtRecordOffset and missIndex parameters are set to 1 now to indicate secondary hit and miss shaders. This correlates with the parameters passed to vkCmdTraceRaysNV() function. Index of 1 means the offset relative to the according hit/miss shader binding offset parameter to that function.
Tutorial files:
10_InstanceResources.cpp
rt_10_shaders.rgen
rt_10_shaders.rchit
rt_10_shaders.rmiss
This tutorial features fetching external resources via geometry instance id and resources inlined into SBT itself.
First change is in the VkGeometryInstance struct filling. We now specify instanceId and instanceOffset for each of the instances. Both are set to index of the instance, i.e. 0, 1, or 2 (overall, we got 3 instances in this tutorial). instanceId represents custom index of the geometry available to the shaders. instanceOffset, however, is not used explicitly and specifies offset to the SBT hit shader group and custom data associated with the group. Look into CreateShaderBindingTable() function, it contains more code comparing to the previous tutorials. In addition to writing generation and miss shader group handles we create 3 (instead of 1) hit shader groups. Each group paired with the additional 4 floats written right after the hit shader group handle. Notice that, _hitShaderAndDataSize variable is set to contain the proper hit group stride and is later used during the call to vkCmdTraceRaysNV() method. Furthermore, CreateUniformBuffers() method creates regular Vulkan uniform buffer containing 3 color vectors. To understand how much data can be put into SBT we can use VkPhysicalDeviceRayTracingPropertiesNV::maxShaderGroupStride field (this is queried in the InitRayTracing() method of the RayTracingApplication class).
In the GLSL (rt_10_shaders.rchit) we declare regular uniform buffer (which contains 3 color vector filled above). gl_InstanceCustomIndexNV which corresponds to the instanceId is used to fetch the data from this buffer. To access SBT inlined data we have to define following declaration layout(shaderRecordNV) buffer InlineData. It contains vec4 inlineData which we have written to the SBT earlier (yet another color vector). We sum up these two vectors to get the final color. Since each of the components contain value of 0.5 (in R, G or B channels) for both uniform buffer and SBT data, the final values equal 1.0 for each of the color channels.
Notice that to allow resource indexing using instanceId we use GL_EXT_nonuniform_qualifier shader extension via nonuniformEXT() function. On the application side this requires VK_EXT_descriptor_indexing extension which allows us to specify an unbound resource array.
Tutorial files:
11_DifferentVertexFormats.cpp
rt_11_shaders.rgen
rt_11_box.rchit
rt_11_icosahedron.rchit
rt_11_shaders.rmiss
In this tutorial we add object tracing with different vertex formats and materials. Basically, it’s leveraging the technique introduced in the previous tutorial (although there’s no SBT data this time). Normals, texture coordinates, textures are accessed through unbound descriptor arrays in hit shaders (box and icosahedron each have their own hit shader specified using instanceOffset). As previously we use gl_InstanceCustomIndex to sample the data from the buffers. This allows us to use different vertex formats and different materials with variable number of textures. Closest hit shaders only differ in main() function (specifically in vertex attributes fetching and albedo calculation).
