Simulation / Modeling / Design

Pro Tip: Improved GLSL Syntax for Vulkan DescriptorSet Indexing

Sometimes the evolution of programming languages creates situations where “simple” tasks take a bit more complexity to express. Syntax annoyance slows down development or can negatively affect readability of code during maintenance. With this in mind, we recently released an open-source sample of a GLSL header generator for DescriptorSet-indexed types in Vulkan.

For example, look at ray tracing in Vulkan via `VK_NV_ray_tracing`.  We sometimes need to access all textures in the scene or want to avoid high frequency bindings.  The functionality of `VK_EXT_descriptor_indexing` and `GL_EXT_nonuniform_qualifier` proves very useful in these cases. Resources like textures and samplers can be stored in large unsized tables, allowing us to use indices to represent them. However, the GLSL syntax to perform the texturing operations in our shaders can suffer a bit.

Here is an example for regular GLSL syntax:

``` glsl

// we use a big resource table for descriptorset indexing
layout(set = 1, binding=0) uniform sampler2D  res_sampler2Ds[];

// let's make use of GL_EXT_buffer_reference2
layout(buffer_reference, scalar) buffer Material 
  // We want to be cache efficient so we use small datatypes for
  // the indices that represent our resources.
  // Also notice the use of "scalar" layout for better packing.
  uint16_t albedoTex;
  uint16_t normalTex;

layout(set=0, binding=0, scalar) uniform inputData
  // pass the entire array of material data as refernece
  Material materials;


  // Within our shading code we may operate on arbitrary objects or 
  // parts of the scene. Especially in raytracing the "materialIdx"
  // could vary depending on the object we hit.

  Material  mat = materials[materialIdx];
  // When this index is non-uniform (e.g. raytracing)
  // our texture fetches can start to look pretty ugly.
  // We have to cast back to "uint" for resource table lookup
  // and also have to flag the type of access as non-uniform.
  vec4 albedo = texture(res_sampler2Ds[nonuniformEXT(uint(mat.albedoTex))], uv);

<p>After including the generated file, new types are available that wrap the index. Operator overloading (thanks to Jeff Bolz for the idea) is used for the texture functions. This makes the code easier to read and write.</p>
<pre class="prettyprint">``` glsl

// we use a define to tell the included file 
// where descriptorsets start that are used for indexing

#define DIT_DSET_IDX  1

// this file is generated by the lua script
#include "sampler_indexed_types.glsl"

layout(buffer_reference, scalar) buffer Material {
  // Through the include new types are available that
  // wrap the index. For example:
  //    struct sampler2D_u16 { 
  //      uint16_t idx;
  //    };
  sampler2D_u16 albedoTex;
  sampler2D_u16 normalTex;

  // Rest as before, but now with a  
  // much nicer looking texture access!
  vec4 albedo = texture(mat.albedoTex, uv);

  // The included file provides the generated function overloads
  // that take care of the indexed lookup, for example:
  //    vec4 texture(sampler2D_u16, vec2 uv) {
  //      return texture(dit_sampler2D[nonuniformEXT(uint(mat.albedoTex))], uv)
  //    }


While the sample above had only one sampler type and one texture function. You can imagine that overloading for all texture functions and all sampler and texture types can be quite tedious. This is where an automated solution becomes very useful and what we released serves as a proof of concept.

However, no easy-to-parse complete set of texturing functions exists. Furthermore we need some information regarding the association of types and functions with extensions. Therefore we recently contributed the ability to dump a symbol table from the GLSL reference compiler (glslang).

`glslangValidator --dump-builtin-symbols --target-env vulkan1.1 dummy.frag > vk_fragment.txt`

The script takes such a symbol table file as input and allows a bit of customization to control the output through a config file, example below:

``` Lua
return {
  -- descriptorsets are generated as layout(set = P_DSET_IDX + offset) 
  -- extension usage is wrapped into: #if P_EXT..extension 
  P_EXT     	= "DIT_",
  -- descriptorsets are generated as: uniform TYPE P_TABLE..TYPE[];
  P_TABLE   	= "dit_",
  -- constructors are generated as:   P_CONSTRUCTOR..TYPE( ... )
  -- all table accesses are made as: table[P_ACCESS(uint(idx))]
  P_ACCESS  	= "nonuniformEXT",
  -- created by output using glslangValidator --dump-builtin-symbols
  symbolFile	= "vk_fragment.txt",
  -- optional check the results of the internal "simplifySymbols"
  symbolFileOut = nil,
  -- generated output file name
  outputFile	= "sampler_indexed_types.glsl",
  -- datatype to be used for the index within the struct 
  indexType 	= "uint16_t",
  -- Define what types should be generated.
  -- The number is the set offset applied to P_DSET_IDX.
  -- Leave table nil and all samplers get exported to same dset
  output = {
	sampler2D     	= 0,
	texture2D     	= 1,
	usamplerBuffer	= 2,


Please visit the project for further details.

Discuss (1)