DriveWorks SDK Reference
5.14.77 Release
For Test and Development only

Feature 2D Detector Interface

Detailed Description

Defines 2D-based feature detection.

Data Structures

struct  dwFeature2DDetectorConfig
 Holds configuration parameters for a feature detector. More...
 

Typedefs

typedef struct dwFeature2DDetectorObject const * dwConstFeature2DDetectorHandle_t
 Handle representing a const feature detector. More...
 
typedef struct dwFeature2DDetectorObject * dwFeature2DDetectorHandle_t
 Handle representing a feature detector. More...
 

Enumerations

enum  dwFeature2DDetectorType {
  DW_FEATURE2D_DETECTOR_TYPE_STD = 0 ,
  DW_FEATURE2D_DETECTOR_TYPE_EX ,
  DW_FEATURE2D_DETECTOR_TYPE_FAST9 ,
  DW_FEATURE2D_DETECTOR_TYPE_COUNT
}
 Defines different KLT tracking algorithms. More...
 
enum  dwFeature2DSelectionMaskType {
  DW_FEATURE2D_SELECTION_MASK_TYPE_UNIFORM = 0 ,
  DW_FEATURE2D_SELECTION_MASK_TYPE_GAUSSIAN ,
  DW_FEATURE2D_SELECTION_MASK_TYPE_COUNT
}
 Feature distribution mask for extended detector. More...
 

Functions

DW_API_PUBLIC dwStatus dwFeature2DDetector_detectFromImage (dwFeatureArray *outputDetections, dwImageHandle_t image, dwFeatureArray *preTrackedFeatures, const float32_t *d_normalizedCrossCorrelation, dwFeature2DDetectorHandle_t obj)
 Detects new features and append them after old tracked features. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_detectFromPyramid (dwFeatureArray *outputDetections, const dwPyramidImage *pyramid, dwFeatureArray *preTrackedFeatures, const float32_t *d_normalizedCrossCorrelation, dwFeature2DDetectorHandle_t obj)
 Detects new features and append them after old tracked features. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_getCUDAStream (cudaStream_t *stream, dwFeature2DDetectorHandle_t obj)
 Gets the CUDA stream used by the feature Detector. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_getPVAStream (cupvaStream_t *stream, dwFeature2DDetectorHandle_t obj)
 Gets the CUPVA stream used by the feature Detector. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_getValidTrackedCount (const uint32_t **d_validTrackedCount, dwFeature2DDetectorHandle_t obj)
 dwFeature2DDetector_getValidTrackedCount More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_initDefaultParams (dwFeature2DDetectorConfig *params)
 Initializes dwFeature2DDetector parameters with default values. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_initDefaultParamsForCamera (dwFeature2DDetectorConfig *params, const dwTransformation3f *cameraToRig, dwConstCameraModelHandle_t cameraHandle)
 Initializes dwFeature2DDetector parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle) More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_initialize (dwFeature2DDetectorHandle_t *obj, const dwFeature2DDetectorConfig *config, cudaStream_t cudaStream, dwContextHandle_t context)
 Creates and initializes a feature Detector. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_release (dwFeature2DDetectorHandle_t obj)
 Releases the feature Detector. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_reset (dwFeature2DDetectorHandle_t obj)
 Resets a feature Detector. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_setCUDAStream (cudaStream_t stream, dwFeature2DDetectorHandle_t obj)
 Sets the CUDA stream for CUDA related operations. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_setMask (const uint8_t *d_mask, const uint32_t maskStrideBytes, const uint32_t maskWidth, const uint32_t maskHeight, dwFeature2DDetectorHandle_t obj)
 Sets a mask to ignore areas of the image. More...
 
DW_API_PUBLIC dwStatus dwFeature2DDetector_setPVAStream (cupvaStream_t stream, dwFeature2DDetectorHandle_t obj)
 Sets the CUPVA stream for PVA related operations. More...
 

Variables

const uint32_t DW_FEATURES2D_DETECTOR_MAX_CELL_SIZE = 128
 

Data Structure Documentation

◆ dwFeature2DDetectorConfig

struct dwFeature2DDetectorConfig
Data Fields
bool autoGenerateFeatureID Determines whether the detector generates unique feature ids for new detections automatically.

If set to true, each new detected features will be assigned with a unique feature id. If set to false, dwFeatureArray::ids will be kept. Default is true.

uint32_t blockSize for DW_PROCESSOR_TYPE_PVA_0 or DW_PROCESSOR_TYPE_PVA_1 only Block window size used to compute the Harris Corner score.

Must be 3, 5 or 7.

uint32_t cellSize Cell size in pixel to split the image into cells.

Must be an integer multiplicaiton of 4

float32_t detailThreshold for DW_FEATURE2D_DETECTOR_TYPE_STD only features in the cell that have scores higher than this value will be considered as high frequency point and will always be outputed.

If set it <= detectorScoreThreshold, all features detected are high frequency and the next numEvenDistributionPerCell will be invalid

uint32_t detectionLevel for DW_FEATURE2D_DETECTOR_TYPE_EX only STD detector always detects on level 0 pyramid image.

Level at which features are detected Default to one (half of original image resolution)

float32_t gaussianMaskCenterX for DW_FEATURE2D_DETECTOR_TYPE_EX only Ignored when not using DW_FEATURE2D_SELECTION_MASK_TYPE_GAUSSIAN

The center X of the Gaussian mask, range: [0.0f, 1.0f]

float32_t gaussianMaskCenterY for DW_FEATURE2D_DETECTOR_TYPE_EX only Ignored when not using DW_FEATURE2D_SELECTION_MASK_TYPE_GAUSSIAN

The center Y of the Gaussian mask, range: [0.0f, 1.0f]

float32_t gaussianMaskStDevX for DW_FEATURE2D_DETECTOR_TYPE_EX only Ignored when not using DW_FEATURE2D_SELECTION_MASK_TYPE_GAUSSIAN

The standard deviation in X of the Gaussian mask 1.0 means the standard deviation equals to the width of the image

float32_t gaussianMaskStDevY for DW_FEATURE2D_DETECTOR_TYPE_EX only Ignored when not using DW_FEATURE2D_SELECTION_MASK_TYPE_GAUSSIAN

The standard deviation in Y of the Gaussian mask 1.0 means the standard deviation equals to the height of the image

uint32_t gradientSize for DW_PROCESSOR_TYPE_PVA_0 or DW_PROCESSOR_TYPE_PVA_1 only Gradient window size.

Must be 3, 5 or 7.

float32_t harrisK Weigting K of the harris corner score defined as det(M) - K * trace(M) * trace(M), where M is the structural matrix 0.04 - 0.06 is used typically If set to zero the default value (5e-2) will be used.
uint32_t harrisRadius for DW_FEATURE2D_DETECTOR_TYPE_EX only STD detector use fixed harrisRadius = 1.

Radius of detector harris averaging window If set to zero the default value (1) will be used

uint32_t imageHeight Height of the images that the Detector runs on.

When PVA fast9 detector is used, the height should be in range of [65, 2448]

uint32_t imageWidth Width of the images that the Detector runs on.

When PVA fast9 detector is used, the width should be in range of [65, 3264]

bool isMaskAdjustmentEnabled for DW_FEATURE2D_DETECTOR_TYPE_EX only Switch to use mask adjusment
dwFeature2DSelectionMaskType maskType for DW_FEATURE2D_DETECTOR_TYPE_EX only STD detector always output UNIFORM distribution.

The type of feature selection mask

uint32_t maxFeatureCount Upper bound on number of features handled.

When using EX detector, maxFeatureCount must be no less than the amount of cells, i.e., maxFeatureCount >= ceil(imageWidth / cellSize) * ceil(imageHeight / cellSize).

uint32_t NMSRadius for DW_FEATURE2D_DETECTOR_TYPE_EX and DW_FEATURE2D_DETECTOR_TYPE_FAST9 STD detector use fixed NMSRadius = 3.

Radius of detector non-maximum suppression For EX detector, if it is set to zero the default value (1) will be used As for FAST9 detector, nonmax suppression will be enabled for non-zero value and will be disabled for zero

uint32_t numEvenDistributionPerCell for DW_FEATURE2D_DETECTOR_TYPE_STD only Number of features to be appended after high frequency points, they have second highest scores but are less than detectorDetailThreshold.

During detection, the input image will be splitted into cellSize x cellSize small tiles, there'll be at most numEvenDistributionPerCell second-highest-score features in each cell.

Larger numEvenDistributionPerCell will output an evener distributed feature map Smaller numEvenDistributionPerCell will have more features in high frequence points (score >= detectorDetailThreshold) and a less evener distributed feature map. Invalid if = 0, in that case, only detectorDetailThreshold takes effect.

dwProcessorType processorType for DW_FEATURE2D_DETECTOR_TYPE_STD only set to DW_PROCESSOR_TYPE_PVA_0 or DW_PROCESSOR_TYPE_PVA_1 to call PVA harris detector.
float32_t scoreThreshold Threshold to filter out low latency points.

All points whose scores are less than detectorScoreThreshold will be considered as a non-feature point. Lower value will output more features, higher value will only keep the high frequency points. If set to zero the default value will be used.

dwFeature2DDetectorType type Detecting algorithm defined by dwFeature2DDetectorType
bool useHalf for DW_FEATURE2D_DETECTOR_TYPE_EX only use half-precision floating point to calculate harris corner score
bool useNewToOldMapFromPreTracked Determines whether the detector uses dwFeatureArray::newToOldMap directly from input preTrackedFeatures in dwFeature2DDetector_detectFromImage and dwFeature2DDetector_detectFromPyramid.

If set to true, the output outputDetections::newToOldMap represents the mapping of output detections to frame(N-1) tracking If set to false, the output outputDetections::newToOldMap represents the mapping of output detections to frame(N) tracking Default is true.

Typedef Documentation

◆ dwConstFeature2DDetectorHandle_t

typedef struct dwFeature2DDetectorObject const* dwConstFeature2DDetectorHandle_t

Handle representing a const feature detector.

Definition at line 66 of file FeatureDetector.h.

◆ dwFeature2DDetectorHandle_t

typedef struct dwFeature2DDetectorObject* dwFeature2DDetectorHandle_t

Handle representing a feature detector.

Definition at line 63 of file FeatureDetector.h.

Enumeration Type Documentation

◆ dwFeature2DDetectorType

Defines different KLT tracking algorithms.

Enumerator
DW_FEATURE2D_DETECTOR_TYPE_STD 

Standard Harris Corner detector with fixed parameters, quicker.

DW_FEATURE2D_DETECTOR_TYPE_EX 

Extended Harris Corner detector with more configurable parameters, more flexible, better quality but slower.

DW_FEATURE2D_DETECTOR_TYPE_FAST9 

Fast corner detector, quicker.

This algorithm was introduced by Edward Rosten and Tom Drummond in their paper "Machine learning for high-speed corner detection" in 2006

DW_FEATURE2D_DETECTOR_TYPE_COUNT 

Definition at line 71 of file FeatureDetector.h.

◆ dwFeature2DSelectionMaskType

Feature distribution mask for extended detector.

Enumerator
DW_FEATURE2D_SELECTION_MASK_TYPE_UNIFORM 

output feature in uniform distribution

DW_FEATURE2D_SELECTION_MASK_TYPE_GAUSSIAN 

output feature in 2D gaussian distribution which has more features in center area and less ones in boundary

DW_FEATURE2D_SELECTION_MASK_TYPE_COUNT 

Definition at line 87 of file FeatureDetector.h.

Function Documentation

◆ dwFeature2DDetector_detectFromImage()

DW_API_PUBLIC dwStatus dwFeature2DDetector_detectFromImage ( dwFeatureArray outputDetections,
dwImageHandle_t  image,
dwFeatureArray preTrackedFeatures,
const float32_t d_normalizedCrossCorrelation,
dwFeature2DDetectorHandle_t  obj 
)

Detects new features and append them after old tracked features.

Features are added only in areas where the mask == 0xff means valid area, any pixel whose mask value != 0xff will be ignored during tracking.

Parameters
[out]outputDetectionsDetected features, composed of old tracked features and new detections, new detections are appended after old features. Must be created by dwFeatureArray_createNew() with DW_PROCESSOR_TYPE_GPU flag and the same maxFeatureCount when initializing detector
[in]imageSpecifies the image on which feature detection takes place, must be a CUDA image.
[in,out]preTrackedFeaturesPrevious tracked features, if detector type is DW_FEATURE2D_DETECTOR_TYPE_EX, some of old features will be marked as DW_FEATURE2D_STATUS_INVALID during detection. The dwFeatureArray can either be an empty array allocated by dwFeatureArray_createNew or a slice from feature history by dwFeatureHistoryArray_getXXX
[in]d_normalizedCrossCorrelationGPU pointer to nccScores of each feature, will be ignored if it's NULL. If the feature detector is connected to a tracker, d_normalizedCrossCorrelation can be filled by dwFeature2DTracker_trackFeatures.
[in]objSpecifies the feature detector handle.
Returns
DW_INVALID_ARGUMENT if detector handle, preTrackedFeatures or image are NULL.
DW_INVALID_ARGUMENT if preTrackedFeatures are not allocated on GPU.
DW_INVALID_HANDLE if image is not a CUDA image.
DW_SUCCESS otherwise.
Note
If there're N features in preTrackedFeatures, for DW_FEATURE2D_DETECTOR_TYPE_STD, the first N features in detectedFeatures are from preTrackedFeaturs, new detections are appended after old ones. for DW_FEATURE2D_DETECTOR_TYPE_EX, the first M(M <= N) features in detectedFeatures are from preTrackedFeaturs new detections are appended after old ones. The reason that M <= N is that some features in preTrackedFeatures will be marked as invalid to meet the restriction of uniform/gaussian distribution.

◆ dwFeature2DDetector_detectFromPyramid()

DW_API_PUBLIC dwStatus dwFeature2DDetector_detectFromPyramid ( dwFeatureArray outputDetections,
const dwPyramidImage *  pyramid,
dwFeatureArray preTrackedFeatures,
const float32_t d_normalizedCrossCorrelation,
dwFeature2DDetectorHandle_t  obj 
)

Detects new features and append them after old tracked features.

Features are added only in areas where the mask == 0xff means valid area, any pixel whose mask value != 0xff will be ignored during tracking.

Parameters
[out]outputDetectionsDetected features, composed of old tracked features and new detections, new detections are appended after old features. Must be created by dwFeatureArray_createNew() with DW_PROCESSOR_TYPE_GPU flag and the same maxFeatureCount when initializing detector
[in]pyramidSpecifies the pyramid on which feature detection takes place.
[in,out]preTrackedFeaturesPrevious tracked features, if detector type is DW_FEATURE2D_DETECTOR_TYPE_EX, some of old features will be marked as DW_FEATURE2D_STATUS_INVALID during detection. The dwFeatureArray can either be an empty array allocated by dwFeatureArray_createNew or a slice from feature history by dwFeatureHistoryArray_getXXX *
[in]d_normalizedCrossCorrelationGPU pointer to nccScores of each feature, will be ignored if it's NULL. If the feature detector is connected to a tracker, d_normalizedCrossCorrelation can be filled by dwFeature2DTracker_trackFeatures.
[in]objSpecifies the feature detector handle.
Returns
DW_INVALID_ARGUMENT if detector handle, preTrackedFeatures or image are NULL.
DW_INVALID_ARGUMENT if preTrackedFeatures are not allocated on GPU.
DW_SUCCESS otherwise.
Note
If there're N features in preTrackedFeatures, for DW_FEATURE2D_DETECTOR_TYPE_STD, the first N features in detectedFeatures are from preTrackedFeaturs, new detections are appended after old ones. for DW_FEATURE2D_DETECTOR_TYPE_EX, the first M(M <= N) features in detectedFeatures are from preTrackedFeaturs new detections are appended after old ones. The reason that M <= N is that some features in preTrackedFeatures will be marked as invalid to meet the restriction of uniform/gaussian distribution.

◆ dwFeature2DDetector_getCUDAStream()

DW_API_PUBLIC dwStatus dwFeature2DDetector_getCUDAStream ( cudaStream_t *  stream,
dwFeature2DDetectorHandle_t  obj 
)

Gets the CUDA stream used by the feature Detector.

Parameters
[out]streamThe CUDA stream currently used.
[in]objA handle to the feature Detector module.
Returns
DW_INVALID_ARGUMENT if feature Detector handle or stream are NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_getPVAStream()

DW_API_PUBLIC dwStatus dwFeature2DDetector_getPVAStream ( cupvaStream_t stream,
dwFeature2DDetectorHandle_t  obj 
)

Gets the CUPVA stream used by the feature Detector.

Parameters
[out]streamThe CUPVA stream currently used.
[in]objA handle to the feature Detector module.
Returns
DW_INVALID_ARGUMENT if feature Detector handle or stream are NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_getValidTrackedCount()

DW_API_PUBLIC dwStatus dwFeature2DDetector_getValidTrackedCount ( const uint32_t **  d_validTrackedCount,
dwFeature2DDetectorHandle_t  obj 
)

dwFeature2DDetector_getValidTrackedCount

Parameters
d_validTrackedCount
obj
Returns

◆ dwFeature2DDetector_initDefaultParams()

DW_API_PUBLIC dwStatus dwFeature2DDetector_initDefaultParams ( dwFeature2DDetectorConfig params)

Initializes dwFeature2DDetector parameters with default values.

Parameters
[in,out]paramsdwFeature2DDetector parameters; user can optionally set imageWidth/imageHeight before the API call to obtain the default parameters for the resolution
Returns
DW_INVALID_ARGUMENT if params is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_initDefaultParamsForCamera()

DW_API_PUBLIC dwStatus dwFeature2DDetector_initDefaultParamsForCamera ( dwFeature2DDetectorConfig params,
const dwTransformation3f cameraToRig,
dwConstCameraModelHandle_t  cameraHandle 
)

Initializes dwFeature2DDetector parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle)

Parameters
[out]paramsdwFeature2DDetector parameters
[in]cameraToRigtransformed camera extrinsic parameters with respect to rig
[in]cameraHandleholds camera intrinsic information
Returns
DW_INVALID_ARGUMENT if any parameter is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_initialize()

DW_API_PUBLIC dwStatus dwFeature2DDetector_initialize ( dwFeature2DDetectorHandle_t obj,
const dwFeature2DDetectorConfig config,
cudaStream_t  cudaStream,
dwContextHandle_t  context 
)

Creates and initializes a feature Detector.

Parameters
[out]objA pointer to the feature Detector handle is returned here.
[in]configSpecifies the configuration parameters.
[in]cudaStreamSpecifies the CUDA stream to use for Detector operations.
[in]contextSpecifies the handle to the context under which it is created.
Returns
DW_INVALID_ARGUMENT if feature Detector handle or context are NULL or config is invalid.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_release()

DW_API_PUBLIC dwStatus dwFeature2DDetector_release ( dwFeature2DDetectorHandle_t  obj)

Releases the feature Detector.

This method releases all resources associated with a feature Detector.

Note
This method renders the handle unusable.
Parameters
[in]objThe feature detector handle to be released.
Returns
DW_INVALID_ARGUMENT if feature Detector handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_reset()

DW_API_PUBLIC dwStatus dwFeature2DDetector_reset ( dwFeature2DDetectorHandle_t  obj)

Resets a feature Detector.

Parameters
[in]objSpecifies the feature Detector handle to be reset.
Returns
DW_INVALID_ARGUMENT if feature Detector handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_setCUDAStream()

DW_API_PUBLIC dwStatus dwFeature2DDetector_setCUDAStream ( cudaStream_t  stream,
dwFeature2DDetectorHandle_t  obj 
)

Sets the CUDA stream for CUDA related operations.

Note
The ownership of the stream remains by the callee.
Parameters
[in]streamThe CUDA stream to be used. Default is the one passed during dwFeature2DDetector_initialize.
[in]objA handle to the feature Detector module to set CUDA stream for.
Returns
DW_INVALID_ARGUMENT if feature Detector handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_setMask()

DW_API_PUBLIC dwStatus dwFeature2DDetector_setMask ( const uint8_t *  d_mask,
const uint32_t  maskStrideBytes,
const uint32_t  maskWidth,
const uint32_t  maskHeight,
dwFeature2DDetectorHandle_t  obj 
)

Sets a mask to ignore areas of the image.

Areas where mask != 0xff will not produce new features when detectNewFeatures() is called. The size of the mask must be the same as the image.

Note
If this function is called after dwFeature2DDetector_initialize(), it must be called with the same input parameters after dwFeature2DDetector_reset because reset() function will clear the existing mask.
Parameters
[in]d_maskA GPU pointer to the mask data. The data is copied.
[in]maskStrideBytesSpecifies the stride, in bytes, for each row of the mask.
[in]maskWidthSpecifies the width of the mask.
[in]maskHeightSpecifies the height of the mask.
[in]objSpecifies the feature detector handle.
Returns
DW_INVALID_ARGUMENT if feature Detector handle or d_mask are NULL or maskWidth or maskHeight are invalid.
DW_SUCCESS otherwise.

◆ dwFeature2DDetector_setPVAStream()

DW_API_PUBLIC dwStatus dwFeature2DDetector_setPVAStream ( cupvaStream_t  stream,
dwFeature2DDetectorHandle_t  obj 
)

Sets the CUPVA stream for PVA related operations.

Note
The ownership of the stream remains by the callee.
Parameters
[in]streamThe CUPVA stream to be used. Default is the one passed during dwFeature2DDetector_initialize.
[in]objA handle to the feature Detector module to set CUPVA stream for.
Returns
DW_INVALID_ARGUMENT if feature Detector handle is NULL.
DW_SUCCESS otherwise.

Variable Documentation

◆ DW_FEATURES2D_DETECTOR_MAX_CELL_SIZE

const uint32_t DW_FEATURES2D_DETECTOR_MAX_CELL_SIZE = 128

Definition at line 249 of file FeatureDetector.h.