Detailed Description

Defines Deep Neural Network (DNN) module for performing inference using NVIDIA^® TensorRT^™ models.

Data Structures
struct	dwDNNMetaData
	Specifies TensorRT model header. More...

Modules
	Clusterer Interface
	Defines the Clusterer module for performing DBSCAN clusterer on bounding boxes.

	DNNTensor Interface
	Defines DNNTensor module for managing tensor content.

	DataConditioner Interface
	Defines the DataConditioner module for performing common transformations on input images for DNN.

Typedefs
typedef struct dwDNNObject *	dwDNNHandle_t
	Handles representing Deep Neural Network interface. More...

typedef struct dwDNNObject const *	dwConstDNNHandle_t

Functions
DW_API_PUBLIC dwStatus	dwDNN_initializeTensorRTFromFile (dwDNNHandle_t const network, const char8_t const modelFilename, const dwDNNPluginConfiguration *const pluginConfiguration, dwProcessorType const processorType, dwContextHandle_t const context)
	Creates and initializes a TensorRT Network from file. More...

DW_API_PUBLIC dwStatus	dwDNN_initializeTensorRTFromFileWithEngineId (dwDNNHandle_t const network, const char8_t const modelFilename, const dwDNNPluginConfiguration *const pluginConfiguration, dwProcessorType const processorType, uint32_t engineId, uint32_t handleId, dwContextHandle_t const context)
	Creates and initializes a TensorRT Network from file with DLA Engine ID. More...

DW_API_PUBLIC dwStatus	dwDNN_initializeTensorRTFromMemory (dwDNNHandle_t const network, const char8_t const modelContent, uint32_t const modelContentSize, const dwDNNPluginConfiguration *const pluginConfiguration, dwProcessorType const processorType, dwContextHandle_t const context)
	Creates and initializes a TensorRT Network from memory. More...

DW_API_PUBLIC dwStatus	dwDNN_initializeTensorRTFromMemoryWithEngineId (dwDNNHandle_t const network, const char8_t const modelContent, uint32_t const modelContentSize, const dwDNNPluginConfiguration *const pluginConfiguration, dwProcessorType const processorType, uint32_t engineId, uint32_t handleId, dwContextHandle_t const context)
	Creates and initializes a TensorRT Network from memory with DLA Engine ID. More...

DW_API_PUBLIC dwStatus	dwDNN_reset (dwDNNHandle_t const network)
	Resets a given network. More...

DW_API_PUBLIC dwStatus	dwDNN_release (dwDNNHandle_t const network)
	Releases a given network. More...

DW_API_PUBLIC dwStatus	dwDNN_inferSIO (float32_t const dOutput, const float32_t const dInput, uint32_t const batchsize, dwDNNHandle_t const network)
	Forwards pass from the first input blob to the first output blob (a shortcut for a single input - single output network). More...

DW_API_PUBLIC dwStatus	dwDNN_inferRaw (float32_t const const dOutput, const float32_t const const dInput, uint32_t const batchsize, dwDNNHandle_t const network)
	Forwards pass from all input blobs to all output blobs. More...

DW_API_PUBLIC dwStatus	dwDNN_setCUDAStream (cudaStream_t const stream, dwDNNHandle_t const network)
	Sets the CUDA stream for infer operations. More...

DW_API_PUBLIC dwStatus	dwDNN_getCUDAStream (cudaStream_t *const stream, dwDNNHandle_t const network)
	Gets the CUDA stream used by the feature list. More...

DW_API_PUBLIC dwStatus	dwDNN_getInputSize (dwBlobSize *const blobSize, uint32_t const blobIndex, dwDNNHandle_t const network)
	Gets the input blob size at blobIndex. More...

DW_API_PUBLIC dwStatus	dwDNN_getOutputSize (dwBlobSize *const blobSize, uint32_t const blobIndex, dwDNNHandle_t const network)
	Gets the output blob size at blobIndex. More...

DW_API_PUBLIC dwStatus	dwDNN_getInputTensorProperties (dwDNNTensorProperties *const tensorProps, uint32_t const blobIndex, dwDNNHandle_t const network)
	Gets the input tensor properties at blobIndex. More...

DW_API_PUBLIC dwStatus	dwDNN_getOutputTensorProperties (dwDNNTensorProperties *const tensorProps, uint32_t const blobIndex, dwDNNHandle_t const network)
	Gets the output tensor properties at blobIndex. More...

DW_API_PUBLIC dwStatus	dwDNN_getInputBlobCount (uint32_t *const count, dwDNNHandle_t const network)
	Gets the input blob count. More...

DW_API_PUBLIC dwStatus	dwDNN_getOutputBlobCount (uint32_t *const count, dwDNNHandle_t const network)
	Gets the output blob count. More...

DW_API_PUBLIC dwStatus	dwDNN_getInputIndex (uint32_t const blobIndex, const char8_t const blobName, dwDNNHandle_t const network)
	Gets the index of an input blob with a given blob name. More...

DW_API_PUBLIC dwStatus	dwDNN_getOutputIndex (uint32_t const blobIndex, const char8_t const blobName, dwDNNHandle_t const network)
	Gets the index of an output blob with a given blob name. More...

DW_API_PUBLIC dwStatus	dwDNN_getMetaData (dwDNNMetaData *const metaData, dwDNNHandle_t const network)
	Returns the metadata for the associated network model. More...

DW_API_PUBLIC dwStatus	dwDNN_infer (dwDNNTensorHandle_t const outputTensors, uint32_t const outputTensorCount, dwConstDNNTensorHandle_t const inputTensors, uint32_t const inputTensorCount, dwDNNHandle_t const network)
	Runs inference pipeline on the given input. More...

Typedef Documentation

◆ dwConstDNNHandle_t

typedef struct dwDNNObject const* dwConstDNNHandle_t

Definition at line 47 of file DNN.h.

◆ dwDNNHandle_t

typedef struct dwDNNObject* dwDNNHandle_t

Handles representing Deep Neural Network interface.

Definition at line 46 of file DNN.h.

Function Documentation

◆ dwDNN_getCUDAStream()

DW_API_PUBLIC dwStatus dwDNN_getCUDAStream	(	cudaStream_t *const	stream,
		dwDNNHandle_t const	network
	)

Gets the CUDA stream used by the feature list.

Parameters

[out]	stream	The CUDA stream currently used.
[in]	network	A handle to the DNN module.

Returns: DW_INVALID_HANDLE if the given network handle or the stream are NULL or of wrong type.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getInputBlobCount()

DW_API_PUBLIC dwStatus dwDNN_getInputBlobCount	(	uint32_t *const	count,
		dwDNNHandle_t const	network
	)

Gets the input blob count.

Parameters

[out]	count	A pointer to the number of input blobs.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle or count are NULL.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getInputIndex()

DW_API_PUBLIC dwStatus dwDNN_getInputIndex	(	uint32_t *const	blobIndex,
		const char8_t *const	blobName,
		dwDNNHandle_t const	network
	)

Gets the index of an input blob with a given blob name.

Parameters

[out]	blobIndex	A pointer to the index of the blob with the given name.
[in]	blobName	A pointer to the name of an input blob.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle, blobIndex or blobName are null or an input blob with the given name is not found.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getInputSize()

DW_API_PUBLIC dwStatus dwDNN_getInputSize	(	dwBlobSize *const	blobSize,
		uint32_t const	blobIndex,
		dwDNNHandle_t const	network
	)

Gets the input blob size at blobIndex.

Parameters

[out]	blobSize	A pointer to the where the input blob size is returned.
[in]	blobIndex	Specifies the blob index; must be in the range [0 dwDNN_getInputBlobCount()-1].
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle or the blob size are NULL or blobIndex is not in range [0 dwDNN_getInputBlobCount()-1].
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getInputTensorProperties()

DW_API_PUBLIC dwStatus dwDNN_getInputTensorProperties	(	dwDNNTensorProperties *const	tensorProps,
		uint32_t const	blobIndex,
		dwDNNHandle_t const	network
	)

Gets the input tensor properties at blobIndex.

Parameters

[out]	tensorProps	Tensor properties.
[in]	blobIndex	Specifies the blob index; must be in the range [0 dwDNN_getInputBlobCount()-1].
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle or the blob size are NULL or blobIndex is not in range [0 dwDNN_getInputBlobCount()-1].
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getMetaData()

DW_API_PUBLIC dwStatus dwDNN_getMetaData	(	dwDNNMetaData *const	metaData,
		dwDNNHandle_t const	network
	)

Returns the metadata for the associated network model.

Parameters

[out]	metaData	A pointer to a metadata structure.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle or metaData are NULL.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getOutputBlobCount()

DW_API_PUBLIC dwStatus dwDNN_getOutputBlobCount	(	uint32_t *const	count,
		dwDNNHandle_t const	network
	)

Gets the output blob count.

Parameters

[out]	count	A pointer to the number of output blobs.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle or count are NULL.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getOutputIndex()

DW_API_PUBLIC dwStatus dwDNN_getOutputIndex	(	uint32_t *const	blobIndex,
		const char8_t *const	blobName,
		dwDNNHandle_t const	network
	)

Gets the index of an output blob with a given blob name.

Parameters

[out]	blobIndex	A pointer to the index of the blob with the given name.
[in]	blobName	A pointer to the name of an output blob.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_HANDLE - if provided network handle, blobIndex or blobName are NULL or an output blob with the given name is not found or the network handle is of wrong type
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getOutputSize()

DW_API_PUBLIC dwStatus dwDNN_getOutputSize	(	dwBlobSize *const	blobSize,
		uint32_t const	blobIndex,
		dwDNNHandle_t const	network
	)

Gets the output blob size at blobIndex.

Parameters

[out]	blobSize	A pointer to the where the output blob size is returned.
[in]	blobIndex	Specifies the blob index; must be in the range [0 dwDNN_getOutputBlobCount()-1].
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle is or blobSize are NULL or blobIndex is not in range [0 dwDNN_getOutputBlobCount()-1].
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_getOutputTensorProperties()

DW_API_PUBLIC dwStatus dwDNN_getOutputTensorProperties	(	dwDNNTensorProperties *const	tensorProps,
		uint32_t const	blobIndex,
		dwDNNHandle_t const	network
	)

Gets the output tensor properties at blobIndex.

Parameters

[out]	tensorProps	Tensor properties.
[in]	blobIndex	Specifies the blob index; must be in the range [0 dwDNN_getOutputBlobCount()-1].
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle is or blobSize are NULL or blobIndex is not in range [0 dwDNN_getOutputBlobCount()-1].
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_infer()

DW_API_PUBLIC dwStatus dwDNN_infer	(	dwDNNTensorHandle_t *const	outputTensors,
		uint32_t const	outputTensorCount,
		dwConstDNNTensorHandle_t *const	inputTensors,
		uint32_t const	inputTensorCount,
		dwDNNHandle_t const	network
	)

Runs inference pipeline on the given input.

Parameters

[out]	outputTensors	Output tensors.
[in]	outputTensorCount	Number of output tensors.
[in]	inputTensors	Input tensors.
[in]	inputTensorCount	Number of input tensors.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle, input or output are NULL.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_inferRaw()

DW_API_PUBLIC dwStatus dwDNN_inferRaw	(	float32_t const const	dOutput,
		const float32_t const const	dInput,
		uint32_t const	batchsize,
		dwDNNHandle_t const	network
	)

Forwards pass from all input blobs to all output blobs.

Note: : The size of the output blob and input blob must match the respective sizes returned by dwDNN_getOutputSize() and dwDNN_getInputSize().; : The umber of blobs must match the respective number of blobs returned by dwDNN_getInputBlobCount() and dwDNN_getOutputBlobCount().

Parameters

[out]	dOutput	A pointer to an array of pointers to the input blobs in GPU Memory.
[in]	dInput	A pointer to an array of pointers to the output blobs in GPU Memory.
[in]	batchsize	Batch size for inference. Batch size must be equal or less than the one that was given at the time of model generation.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle , output or input are NULL.
DW_INTERNAL_ERROR - if DNN engine cannot execute inference on the given network.
DW_SUCCESS otherwise.

Note: Inference performance might be suboptimal if the network has been generated for a larger batch size.; If the processor type is DLA, the inference will always perform on maximum batch size. The only advantage of giving a lower batchsize as input is that the output to be copied will be smaller in size.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_inferSIO()

DW_API_PUBLIC dwStatus dwDNN_inferSIO	(	float32_t *const	dOutput,
		const float32_t *const	dInput,
		uint32_t const	batchsize,
		dwDNNHandle_t const	network
	)

Forwards pass from the first input blob to the first output blob (a shortcut for a single input - single output network).

Note: This method requires the network to be single input - single output.; The size of the output blob and input blob must match the respective sizes returned by dwDNN_getOutputSize() and dwDNN_getInputSize().

Parameters

[out]	dOutput	A pointer to the output blob in GPU memory.
[in]	dInput	A pointer to the input blob in GPU memory.
[in]	batchsize	Batch size for inference. Batch size must be equal or less than the one that was given at the time of model generation.
[in]	network	Network handle created with dwDNN_initialize().

Returns: DW_INVALID_ARGUMENT - if provided network handle, output or input are NULL or if provided network is not a single input - single output network.
DW_INTERNAL_ERROR - if DNN engine cannot execute inference on the given network.
DW_SUCCESS otherwise.

Note: Inference performance might be suboptimal if the network has been generated for a larger batch size.; If the processor type is DLA, the inference will always perform on maximum batch size. The only advantage of giving a lower batchsize as input is that the output to be copied will be smaller in size.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_initializeTensorRTFromFile()

DW_API_PUBLIC dwStatus dwDNN_initializeTensorRTFromFile	(	dwDNNHandle_t *const	network,
		const char8_t *const	modelFilename,
		const dwDNNPluginConfiguration *const	pluginConfiguration,
		dwProcessorType const	processorType,
		dwContextHandle_t const	context
	)

Creates and initializes a TensorRT Network from file.

Parameters

[out]	network	A pointer to network handle that will be initialized from parameters.
[in]	modelFilename	A pointer to the name of the TensorRT model file.
[in]	pluginConfiguration	An optional pointer to plugin configuration for custom layers.
[in]	processorType	Processor that the inference should run on. Note that the model must be generated for this processor type.
[in]	context	Specifies the handle to the context under which the DNN module is created.

Returns: DW_INVALID_ARGUMENT - if pointer to the network handle or the model filename are NULL.
DW_DNN_INVALID_MODEL - if the provided model is invalid.
DW_CUDA_ERROR - if compute capability does not met the network type's requirements.
DW_FILE_NOT_FOUND - if given model file does not exist.
DW_SUCCESS otherwise.

Note

The network file must be created by the TensorRT_optimization tool.

DNN module will look for metadata file named <modelFilename>.json in the same folder. If it is present, metadata will be loaded from that file. Otherwise, it will be filled with default values. Example metadata:

{
    "dataConditionerParams" : {
        "meanValue" : [0.0, 0.0, 0.0],
        "splitPlanes" : true,
        "pixelScaleCoefficient": 1.0,
        "ignoreAspectRatio" : false,
        "doPerPlaneMeanNormalization" : false
    }
}

\

API Group

Init: Yes
Runtime: No
De-Init: No

◆ dwDNN_initializeTensorRTFromFileWithEngineId()

DW_API_PUBLIC dwStatus dwDNN_initializeTensorRTFromFileWithEngineId	(	dwDNNHandle_t *const	network,
		const char8_t *const	modelFilename,
		const dwDNNPluginConfiguration *const	pluginConfiguration,
		dwProcessorType const	processorType,
		uint32_t	engineId,
		uint32_t	handleId,
		dwContextHandle_t const	context
	)

Creates and initializes a TensorRT Network from file with DLA Engine ID.

Parameters

[out]	network	A pointer to network handle that will be initialized from parameters.
[in]	modelFilename	A pointer to the name of the TensorRT model file.
[in]	pluginConfiguration	An optional pointer to plugin configuration for custom layers.
[in]	processorType	Processor that the inference should run on. Note that the model must be generated for this processor type.
[in]	engineId	Specifies the DLA engine id if the processorType is DW_PROCESSOR_TYPE_CUDLA.
[in]	handleId	Specifies the DLA device handle id if the processorType is DW_PROCESSOR_TYPE_CUDLA.
[in]	context	Specifies the handle to the context under which the DNN module is created.

Returns: DW_INVALID_ARGUMENT - if pointer to the network handle or the model filename are NULL.
DW_DNN_INVALID_MODEL - if the provided model is invalid.
DW_CUDA_ERROR - if compute capability does not met the network type's requirements.
DW_FILE_NOT_FOUND - if given model file does not exist.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: No
De-Init: No

◆ dwDNN_initializeTensorRTFromMemory()

DW_API_PUBLIC dwStatus dwDNN_initializeTensorRTFromMemory	(	dwDNNHandle_t *const	network,
		const char8_t *const	modelContent,
		uint32_t const	modelContentSize,
		const dwDNNPluginConfiguration *const	pluginConfiguration,
		dwProcessorType const	processorType,
		dwContextHandle_t const	context
	)

Creates and initializes a TensorRT Network from memory.

Parameters

[out]	network	A pointer to network handle that is initialized from parameters.
[in]	modelContent	A pointer to network content in the memory.
[in]	modelContentSize	Specifies the size of network content in memory, in bytes.
[in]	pluginConfiguration	An optional pointer to plugin configuration for custom layers.
[in]	processorType	Processor that the inference should run on. Note that the model must be generated for this processor type.
[in]	context	Specifies a handle to the context under which the DNN module is created.

Returns: DW_INVALID_ARGUMENT - if pointer to the network handle or the model content are NULL.
DW_DNN_INVALID_MODEL - if the provided model is invalid.
DW_CUDA_ERROR - if compute capability does not met the network type's requirements.
DW_SUCCESS otherwise.

Note: The network file must be created by the TensorRT_optimization tool.; DNN module will fill metadata with default values.

API Group

Init: Yes
Runtime: No
De-Init: No

◆ dwDNN_initializeTensorRTFromMemoryWithEngineId()

DW_API_PUBLIC dwStatus dwDNN_initializeTensorRTFromMemoryWithEngineId	(	dwDNNHandle_t *const	network,
		const char8_t *const	modelContent,
		uint32_t const	modelContentSize,
		const dwDNNPluginConfiguration *const	pluginConfiguration,
		dwProcessorType const	processorType,
		uint32_t	engineId,
		uint32_t	handleId,
		dwContextHandle_t const	context
	)

Creates and initializes a TensorRT Network from memory with DLA Engine ID.

Parameters

[out]	network	A pointer to network handle that is initialized from parameters.
[in]	modelContent	A pointer to network content in the memory.
[in]	modelContentSize	Specifies the size of network content in memory, in bytes.
[in]	pluginConfiguration	An optional pointer to plugin configuration for custom layers.
[in]	processorType	Processor that the inference should run on. Note that the model must be generated for this processor type.
[in]	engineId	Specifies the DLA engine id if the processorType is DW_PROCESSOR_TYPE_CUDLA.
[in]	handleId	Specifies the DLA device handle id if the processorType is DW_PROCESSOR_TYPE_CUDLA.
[in]	context	Specifies a handle to the context under which the DNN module is created.

Returns: DW_INVALID_ARGUMENT - if pointer to the network handle or the model content are NULL.
DW_DNN_INVALID_MODEL - if the provided model is invalid.
DW_CUDA_ERROR - if compute capability does not met the network type's requirements.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: No
De-Init: No

◆ dwDNN_release()

DW_API_PUBLIC dwStatus dwDNN_release ( dwDNNHandle_t const network )

Releases a given network.

Parameters

[in] network The network handle to release.

Returns: DW_INVALID_ARGUMENT - if provided network handle is NULL.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: No
De-Init: Yes

◆ dwDNN_reset()

DW_API_PUBLIC dwStatus dwDNN_reset ( dwDNNHandle_t const network )

Resets a given network.

Parameters

[in] network Network handle to reset.

Returns: DW_INVALID_ARGUMENT - if provided network handle is null.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

◆ dwDNN_setCUDAStream()

DW_API_PUBLIC dwStatus dwDNN_setCUDAStream	(	cudaStream_t const	stream,
		dwDNNHandle_t const	network
	)

Sets the CUDA stream for infer operations.

Note: The ownership of the stream remains by the callee.

Parameters

[in]	stream	The CUDA stream to be used. Default is the stream 0, resulting in synchronous operations.
[in]	network	A handle to the DNN module to set CUDA stream for.

Returns: DW_INVALID_ARGUMENT if the given network handle is NULL.
DW_SUCCESS otherwise.

API Group

Init: Yes
Runtime: Yes
De-Init: Yes

NVIDIA DriveOS Linux NSR SDK API Reference

7.0.3.0 Release

Detailed Description

Data Structures

Modules

Typedefs

Functions

Typedef Documentation

◆ dwConstDNNHandle_t

◆ dwDNNHandle_t

Function Documentation

◆ dwDNN_getCUDAStream()

◆ dwDNN_getInputBlobCount()

◆ dwDNN_getInputIndex()

◆ dwDNN_getInputSize()

◆ dwDNN_getInputTensorProperties()

◆ dwDNN_getMetaData()

◆ dwDNN_getOutputBlobCount()

◆ dwDNN_getOutputIndex()

◆ dwDNN_getOutputSize()

◆ dwDNN_getOutputTensorProperties()

◆ dwDNN_infer()

◆ dwDNN_inferRaw()

◆ dwDNN_inferSIO()

◆ dwDNN_initializeTensorRTFromFile()

◆ dwDNN_initializeTensorRTFromFileWithEngineId()

◆ dwDNN_initializeTensorRTFromMemory()

◆ dwDNN_initializeTensorRTFromMemoryWithEngineId()

◆ dwDNN_release()

◆ dwDNN_reset()

◆ dwDNN_setCUDAStream()