NvDisplay

NvDisplay is the display architecture supported on Linux starting in NVIDIA DRIVE® OS 6.0.

Display State Manager

Drive Setmode

When driver setmode is enabled, display setup + modeset will happen during the resmgr init phase instead of deferring this to when the first display client comes up. So, enabling driver setmode can improve "power-on to first pixel visible on screen" latency. On QNX safety, driver setmode is enabled by default. On QNX Standard, driver setmode can be enabled using Device Tree. Driver setmode can be enabled only for the configuration which uses DP Serializer, on other configurations enabling it will fail to load display driver.

An example Device Tree fragment is shown below. In this example, driver setmode is enabled.
\ { 
    display@13800000 { 
        nvidia,driver-setmode; 
    }; 
}; 

OpenWFD Modeset

OpenWFD's modesetting happens in the following manner:
  • When a OpenWFD client first initializes the WFD library, OpenWFD queries NvKms for the current mode using NVKMS_IOCTL_GET_CURRENT_MODE
    • If current mode is not NULL:
      • OpenWFD exposes the sole active mode to clients and skips invoking the NvKms APIs for mode query and verification

        • Any client requests for modesets (via wfdSetPortMode) will be ignored by the OpenWFD driver (with a message logged in slog2info. This is not an error message)

          • This enables existing client apps to work as-is without any changes.
    • If current mode is NULL:
      • OpenWFD uses NvKms' mode query and validation IOCTLs to build a list of available modes and set one out of these modes as the preferred one (this is usually reported by NvKms)

      • At runtime OpenWFD performs a modeset iff the WFD client invokes wfdSetPortMode followed by a wfdDeviceCommit. This is the legacy way of doing modesets and all current WFD apps are already following this approach.

Components

The following are NvDisplay components and their libraries:

  • OpenWFD (libtegrawfd.so)
  • NvKms (nvidia-modeset.ko)

OpenWFD

OpenWFD (Open Windowing Foundation Display) is a Khronos API that provides a low-level hardware abstraction interface for windowing systems and applications to make use of display hardware.

Applications that need to interact with NvDisplay should do so via the OpenWFD API.

The specification of the OpenWFD API is available at Khronos' registry at https://www.khronos.org/registry/OpenWF/specs/OpenWF_Display_1_0_Specification.pdf (hereby referred to as the spec).

The NvDisplay architecture on NVIDIA DRIVE® OS includes an OpenWFD driver (libtegrawfd.so – hereby referred to as the WFD driver) that supports a subset of the core OpenWFD APIs listed in the spec along with additional NVIDIA specific extensions.

Usage guidelines for the core OpenWFD APIs can be obtained from the spec and any deviations from the spec or additional details will be described in this document.

Supported OpenWFD APIs

The following table lists all the core OpenWFD APIs and the available support for them on the WFD driver.

API

Support in libtegrawfd.so

wfdGetStrings Supported on Linux build
wfdIsExtensionSupported Supported on Linux build
wfdGetError Supported on Linux build
wfdEnumerateDevices Supported on Linux build
wfdCreateDevice Supported on Linux build
wfdDestroyDevice Supported on Linux build
wfdDeviceCommit Supported on Linux build
wfdGetDeviceAttribi Supported on Linux build
wfdSetDeviceAttribi Supported on Linux build
wfdCreateEvent Not supported
wfdDestroyEvent Not supported
wfdGetEventAttribi Not supported
wfdDeviceEventAsync Not supported
wfdDeviceEventWait Not supported
wfdDeviceEventFilter Not supported
wfdEnumeratePorts Supported on Linux build
wfdCreatePort Supported on Linux build
wfdDestroyPort Supported on Linux build
wfdGetPortModes Supported on Linux build
wfdGetPortModeAttrib{i/f} Supported on Linux build
wfdSetPortMode Supported on Linux build
wfdGetCurrentPortMode Supported on Linux build
wfdGetPortAttribi Supported on Linux build
wfdGetPortAttribf Supported on Linux build
wfdGetPortAttribiv Supported on Linux build
wfdGetPortAttribfv Supported on Linux build
wfdSetPortAttribi Supported on Linux build
wfdSetPortAttribf Supported on Linux build
wfdSetPortAttribiv Supported on Linux build
wfdSetPortAttribfv Supported on Linux build
wfdBindPipelineToPort Supported on Linux build
wfdGetDisplayDataFormats Not supported
wfdGetDisplayData Not supported
wfdEnumeratePipelines Supported on Linux build
wfdCreatePipeline Supported on Linux build
wfdDestroyPipeline Supported on Linux build
wfdCreateSourceFromImage Supported on Linux build
wfdCreateSourceFromStream Not supported
wfdDestroySource Supported on Linux build
wfdCreateMaskFromImage Not supported
wfdCreateMaskFromStream Not supported
wfdDestroyMask Not supported
wfdBindSourceToPipeline Supported on Linux build
wfdBindMaskToPipeline Not supported
wfdGetPipelineAttribi Supported on Linux build
wfdGetPipelineAttribf Supported on Linux build
wfdGetPipelineAttribiv Supported on Linux build
wfdGetPipelineAttribfv Supported on Linux build
wfdSetPipelineAttribi Supported on Linux build
wfdSetPipelineAttribf Supported on Linux build
wfdSetPipelineAttribiv Supported on Linux build
wfdSetPipelineAttribfv Supported on Linux build
wfdGetPipelineTransparency Supported on Linux build
wfdSetPipelineTSColor Not supported
wfdGetPipelineLayerOrder Supported on Linux build

The following OpenWFD attributes have limited support for querying:

  • WFD_PORT_BACKGROUND_COLOR – Cannot be queried via wfdGetPortAttribi API. wfdGetPortAttribiv and wfdGetPortAttribfv APIs can still be used for querying WFD_PORT_BACKGROUND_COLOR.

Supported OpenWFD Extensions

The following OpenWFD extensions are supported by the OpenWFD Driver.

WFD_NVX_create_source_from_nvscibuf
WFD_NVX_commit_non_blocking
WFD_NVX_nvscisync
WFD_NVX_port_mode_timings

These extensions are described in the NVIDIA DRIVE® OS 6.0 SDK API Reference.

OpenWFD Usage Guidelines

  • The macro WFD_NVX_create_source_from_nvscibuf needs to be defined before including the headers wfd.h and wfdext.h for enabling the WFD_NVX_create_source_from_nvscibuf extension.
  • The macro WFD_WFDEXT_PROTOTYPES must be defined before including the wfd.h and wfdext.h headers.
  • To clear the contents on a WFDPipeline (performing a null flip), wfdBindSourceToPipeline must be called with 0 as the argument for WFDSource before calling wfdDeviceCommit on the WFDPipeline. This is demonstrated in the openwfd_nvsci_sample application present in the SDK.
  • During the deinitialization of OpenWFD applications, you must follow these steps:
    1. Perform a null flip on all WFDPorts and WFDPipelines that were previously bound to a WFDSource.
      • If nonblocking commits are being used and WFD_PIPELINE_POSTFENCE_SCANOUT_BEGIN_NVX is set to WFD_TRUE, the post-flip fence of the null flip commit should be waited on.
      • If nonblocking commits are being used and WFD_PIPELINE_POSTFENCE_SCANOUT_BEGIN_NVX is set to WFD_FALSE, the post-flip fence of the wfdDeviceCommit call before the null flip commit should be waited on.
    2. Destroy all WFDSource objects by invoking wfdDestroySource on them only after all null flip commits are complete. Blocking wfdDeviceCommit calls ensures that the null flip is complete when the wfdDeviceCommit call returns. For nonblocking wfdDeviceCommit calls, synchronization must be handled by the OpenWFD client as described previously.
    3. Destroy all WFDPipeline and WFDPort objects by using wfdDestroyPipeline and wfdDestroyPort, respectively, before finally destroying the WFDDevice object using wfdDestroyDevice.
  • For committing to multiple WFDPipeline handles bound to the same WFDPort, it is recommended to call wfdDeviceCommit* with WFDCommitType set to WFD_COMMIT_ENTIRE_PORT.
  • When you use WFD_COMMIT_ENTIRE_PORT in a wfdDeviceCommit call, all the WFDPipeline objects part of the commit must either have a surface bound(normal flip) or have no surfaces bound (null flip). A mix of normal and a null flip in one WFD_COMMIT_ENTIRE_PORT wfdDeviceCommit call is not allowed due to a limitation on the NvDisplay resource manager. To ensure that any errors are caught, the OpenWFD API wfdGetError must be invoked after calls to the following OpenWFD APIs that take in a WFDDevice handle as one of the inputs and return void:
    • wfdDeviceCommit
    • wfdDeviceCommitWithNvSciSyncFenceNVX
    • wfdDestroyPort
    • wfdSetPortMode
    • wfdGetPortAttrib{iv/fv}
    • wfdSetPortAttrib{i/v/iv/fv}
    • wfdBindPipelineToPort
    • wfdDestroyPipeline
    • wfdDestroySource
    • wfdBindSourceToPipeline
    • wfdGetPipelineAttrib{iv/fv}
    • wfdSetPipelineAttrib{i/v/iv/fv}
  • When flipping to display, you should use a minimum of two WFDSource handles (double buffering) to prevent tearing or corruption.

    For more details, see section 8.1 in the OpenWF Display Specification.

  • The following prerequisites must be fulfilled for use cases to suspend to RAM (SC7 mode):

    • Before entering suspend-to-RAM (SC7) mode, it is recommended to perform a null flip to display.
    • To exit SC7 mode, a modeset must be performed before flipping to display.

Display Serializer

The following sections describe the display serializer.

Configuring Video Timings

For both SST and MST mode, the mode timings that are used for each stream must be configured in Device Tree. Only one mode timing can be specified at a time for each video stream. The timings that are exposed in the EDIDs of the serializer and the panels connected to the downstream deserializer are completely ignored.

An example Device Tree fragment is shown below. In this example, a standard 1920x1080 at 60 Hz timing is specified for the first video stream, and a 1280x720 at 60 Hz timing is specified for the second video stream:

\ {
    display@13800000 {
        display-timings {
            display-connector-0 {
                dcb-index = <0>;

                stream-0 {
                    timings-phandle = <&mode0>;
                };

                stream-1 {
                    timings-phandle = <&mode1>;
                };
            };
        };

        mode0: 1920-1080-60Hz {
            clock-frequency-khz = <148500>;
            hactive = <1920>;
            vactive = <1080>;
            hfront-porch = <88>;
            hback-porch = <148>;
            hsync-len = <44>;
            vfront-porch = <4>;
            vback-porch = <36>;
            vsync-len = <5>;
            rrx1k = <60000>;
            pps-data = [
                11 00 00 89 30 80 04 38
                07 80 04 38 03 c0 03 c0
                02 00 03 58 00 20 73 3e
                00 0d 00 0f 00 1d 00 0e
                18 00 10 f0 03 0c 20 00
                06 0b 0b 33 0e 1c 2a 38
                46 54 62 69 70 77 79 7b
                7d 7e 01 02 01 00 09 40
                09 be 19 fc 19 fa 19 f8
                1a 38 1a 78 22 b6 2a b6
                2a f6 2a f4 43 34 63 74
                00 00 00 00 00 00 00 00
                00 00 00 00 00 00 00 00
                00 00 00 00 00 00 00 00
                00 00 00 00 00 00 00 00
                00 00 00 00 00 00 00 00 ];
        };

        mode1: 1280-720-60Hz {
            clock-frequency-khz = <74250>;
            hactive = <1280>;
            vactive = <720>;
            hfront-porch = <110>;
            hback-porch = <220>;
            hsync-len = <40>;
            vfront-porch = <5>;
            vback-porch = <20>;
            vsync-len = <5>;
            rrx1k = <60000>;
        };
    };
};
In the preceding example:
  • "display@13800000" is the overall parent node for the entire display device. This node already exists today.
    • "display-timings" is used to specify which timings are used for each stream.
      • "display-connector-0" specifies the timing information for the first display connector. If there are multiple display connectors present on the board that require fixed timings, then a new "display-connector" node must be created for each connector.
        • "dcb-index" specifies the logical index X of the DCB -> Display Devices -> Display Device X entry in the display DCB blob that this connector entry applies to. If there is only one display connector on the board, then "dcb-index" defaults to 0.
        • The "stream" nodes specify the phandle of the mode timing node that applies to the given video stream.
    • Each "mode" node contains the actual mode timing parameters that will be used for a given video stream.
      • "clock-frequency-khz": Pixel clock frequency in KHz
      • "hactive": Horizontal active
      • "vactive": Vertical active
      • "hfront-porch": Horizontal front porch
      • "hback-porch": Horizontal back porch
      • "hsync-len": Horizontal sync width
      • "vfront-porch": Vertical front porch
      • "vback-porch": Vertical back porch
      • "vsync-len": Vertical sync width
      • "rrx1k": Refresh rate in units of 0.001Hz
      • "pps-data": All 128B of the DSC PPS

        This property should be specified if DSC will be enabled for the given timing.

Each "display-connector" can only have up to two (2) "stream" nodes. Note that it is fine to specify two (2) "stream" nodes even if the display serializer operates in SST mode because only the first "stream" node is consumed by the display driver. The extra node is ignored.

Configuring the Serializer Driver

If you are using the NVIDIA reference Maxim serializer driver, there are various ways to configure the driver by Device Tree. An example Device Tree fragment is shown below that configures the Maxim serializer chip in MST mode:

i2c@31e0000 {
    status = "okay";
    maxim_ser: max_gmsl_dp_ser@40 {
        compatible = "maxim,max_gmsl_dp_ser";
        reg = <0x40>;
        status = "okay";
        max_gmsl_dp_ser-pwrdn = <&tegra_main_gpio TEGRA234_MAIN_GPIO(G, 3) GPIO_ACTIVE_HIGH>;
        ser-errb = <&tegra_main_gpio TEGRA234_MAIN_GPIO(G, 7) 0>;
        dprx-link-rate = <0x1e>;
        dprx-lane-count = <0x4>;
        enable-mst;
        mst-payload-ids  = <0x1 0x3 0x2 0x4>;
        gmsl-stream-ids  = <0x0 0x1 0x2 0x3>;
        gmsl-link-select = <0x0 0x0 0x1 0x1>;        enable-dp-fec;
        enable-dsc = <1 0>;
        enable-gmsl-fec = <1 0>;
    };
};

A description of each of the preceding properties:

  • Required properties:
    • compatible: Must be "maxim,max_gmsl_dp_ser".
    • reg: I2C address of the Maxim display serializer.
    • max_gmsl_dp_ser-pwrdn: GPIO pin number of the PWRDN pin. This pin is used to power up the Maxim display serializer chip.
    • gmsl-link-select: This property is an array of four unsigned 8-bit values that determines the GMSL output link to enable for each video pipe X, Y, Z, and U. The possible values for each pipe are:
      • 0x0 (Link A)
      • 0x1 (Link B)
      • 0x2 (Link A + B)
  • Optional properties:
    • dprx-link-rate: Configures the DP link rate of the serializer chip.

      The default value is 0x1E (HBR3). The possible values are:
      • 0xA (HBR)
      • 0x14 (HBR2)
      • 0x1E (HBR3)
    • dprx-lane-count: Configures the DP lane count of the serializer chip.

      The default value is 0x4. The possible values are:
      • 0x1
      • 0x2
      • 0x4
    • ser-errb: GPIO pin number of the ERRB pin. This pin is used for error and fault reporting by the serializer chip.

    • enable-mst: This is a Boolean property. If this property is present, the driver will enable MST mode.

    • mst-payload-ids: This property is an array of four unsigned 8-bit values, which represent MST payload IDs of pipe X, Y, Z, U. This property is mandatory if enable-mst property is mentioned in dt.

    • gmsl-stream-ids: This property is an array of four unsigned 8-bit values, which represent GMSL stream IDs of pipe X, Y, Z, U. This property is mandatory if enable-mst property is mentioned in dt.

    • enable-dp-fec: This is a boolean property. When this property is present, the driver will enable FEC on the DP link if the serializer chip supports it.
    • enable-dsc: This property is an array of two 32-bit values, where each value indicates whether DSC is enabled or not. The first entry corresponds to video pipe X, and the second entry corresponds to video pipe Y. DSC is only supported on pipe X currently.
    • enable-gmsl-fec: This property is an array of two 32-bit values, where each value indicates whether FEC is enabled on the GMSL link. The first entry corresponds to GMSL Link A, and the second entry corresponds to GMSL Link B.

Modeset Limitations

The Maxim display serializers do not support dynamic mode changes in MST mode without requiring a reset of the serializer chip in-between. In this context, a mode change refers to changing the number of video streams and/or the display timings that are used for each stream. As such, if the Maxim display serializers are configured in MST mode, it is recommended to always enable all video streams that will be used at once.

Head to Window Assignment

The following sections describe head to window assignment.

Configuring Head to Window Assignment

Configure head to window assignment in the Device Tree. If an assignment is not specified in the DT, the driver assigns windows (2N) and (2N + 1) to HEAD N. In DT, specify the assignment using 64 bit mask, which is interpreted as:

Head-Bitmask Window-Number
BITMASK(0-7) 0
BITMASK(8-15) 1
BITMASK(16-23) 2
BITMASK(24-31) 3
BITMASK(32-39) 4
BITMASK(40-47) 5
BITMASK(48-55) 6
BITMASK(56-63) 7

The display driver fails to load if an invalid assignment is specified in the DT. The specified assignment must adhere to the conditions below:

  1. The specified window number must be supported by hardware.
  2. The specified head number must be supported by hardware.
  3. The same window must not be assigned simultaneously for multiple heads.
  4. At least one window must be assigned to at least one head (that is, specified window-head mask should not be 0).

The display driver culls the head with no windows assigned, and all the heads above it. For example, if hardware supports three heads and the user uses the assignment mask to assign valid windows to head-0 and head-2 but no windows to head-1, then head-1 and head-2 is culled.

An example Device Tree fragment is shown below. In this example, window-0, window-1, and window-2 is assigned to head-0 and window-3 is assigned to head-1.

\ {
    display@13800000 {
        nvidia,window-head-mask = <0x00000000 0x02010101>;
    };
};

In the preceding example:

  • display@13800000 is the overall parent node for the entire display device. This node already exists today.
    • nvidia,window-head-mask is used to specify the 64-bit window head assignment mask.

Restrictions

NvDisplay architecture has the following restrictions:

  • Only one NvDisplay client process can be active at a time on the NVIDIA DRIVE Orin platforms.
    • On the NVIDIA DRIVE® OS Linux NVIDIA Orin platforms, nvidia-drm should not be installed when you run non-DRM applications.
  • NVIDIA DRIVE® OS does not support passive DP-to-HDMI (DP++) cables, dongles, or adapters.