NvMedia IOFA (nvm_iofa_stereo_sci and nvm_iofa_flow_sci)

Two sample applications, nvm_iofa_stereo_sci and nvm_iofa_flow_sci, demonstrate how to use the NvMedia Optical Flow Accelerator (OFA) APIs to perform optical flow/stereo matching. The sample uses the NVIDIA® OFA hardware engine.

nvm_iofa_stereo_sci can read Left View and Right View YUV files and perform stereo matching on input stereo pair. The sample application outputs one stereo disparity value for each 1x1/2x2/4x4/8×8 input block on the output surface. Grid size controls input block size, which is sample application’s argument.

The output stereo disparity format is fixed signed 10.5 and range of disparity is fixed [0, 127] or [0, 255]. Divide the output values by 32 to get a disparity value in terms of pixel units.

nvm_iofa_flow_sci can read input YUV file and perform optical flow estimation on input YUV file. The sample outputs one flow vector value for each 1x1/2x2/4x4/8×8 input block on the output surface. Input block size is controlled by grid size, which is sample application’s argument.

The output flow vector format is fixed signed 10.5 [Range -1024 to 1023]. For optical flow, divide each component of flow vector values by 32 to get the flow vector in terms of pixel units.

Architecture

The following figure shows the architecture of the nvm_iofa_stereo_sci and nvm_iofa_flow_sci application.

To run the sample:
  1. Follow the steps in Building and Running the NvMedia Samples.
  2. Launch the application:
$ nvm_iofa_stereo_sci -h -v 3
$ nvm_iofa_flow_sci -h -v 3

Command Line Options for nvm_iofa_stereo_sci

The sample application command syntax is as follows:

$ nvm_iofa_stereo_sci [options]

The command line options are as follows.

String parameters, such as pathnames, have a maximum length of 1024 characters. Restrictions on other types of parameters are noted individually.

Required Command Line Options
Option Parameter Description
-l Left view file name Specifies left view file name with full path.
-r Right view file name Specifies right view file name with full path.
-res <w>x<h>

Specifies the input surface dimensions, where <w> represents width and <h> represents height. Width and height are separated by a letter ‘x’ with no spaces, for example:

-res 640x480

<w> has a range of [32, 8192].

<h> has a range of [32, 8192].

Optional Command Line Options
Option Parameter Description
-h n/a Displays guidance on using this application.
-version n/a Get NvMediaIOFA Major/Minor version
-v

0: Errors (default)

1: Warnings

2: Info

3: Debug

Logging level.
-gridSize

0: Grid Size 1x1

1: Grid Size 2x2

2: Grid Size 4x4

3: Grid Size 8x8

Output Grid Size set by Application.

-o Disparity Output file name Specifies disparity output file name with full path.
-co Cost Output file name Specifies cost output file name with full path.
-stereocrcgen A pathname of a file to be created with the filetype .txt Generates CRC values of disparity map in the specified file.
-stereocrcchk A pathname of an existing file with the filetype .txt Checks CRC values generated from disparity map against values from the specified file.
-costcrcgen A pathname of a file to be created with the filetype .txt Generates CRC values of cost surface in the specified file.
-costcrcchk A pathname of an existing file with the filetype .txt Checks CRC values generated from cost surface against values from the specified file.
-if

0: YV12 (default)

1: IYUV

Input YUV file format.
-chromaFormat

0 : 400

1 : 420

2 : 422

3 : 444

Chroma Format IDC of input yuv
-ndisp

128

256

Max Disparity value in terms of pixels
-rlSearch

0: Disable reverse search (default)

1: Enable reverse search

Enable reverse search generates right view disparity map.
-frames An integer Specifies the number of input frames to be processed
-inputBuffering

1 to 8

default (1)

Input YUV pair buffering
-profile

0: Disable Profile

1: Enable Profile with Async mode

2: Enable Profile with Sync mode

Enable OFA profiling to get SW and HW overhead
-etype 0: Stereo Disparity HQ mode (default) Specifies stereo matching quality mode.
-bit_depth

8: 8 bits (default)

10: 10 bits

12: 12 bits

16: 16 bits

Number of bits per components on input surfaces.
-timeout An Integer (default – 30ms) timeout value to wait for OFA operation to finish in ms
-p1 An integer SGM penalty1 value
-p2 An integer SGM penalty2 value
-diag

0: Disable diagonal mode

1: Enable diagonal mode

SGM diagonal mode
-adaptiveP2

0: Disable adaptive P2

1: Enable adaptive P2

SGM adaptive p2 mode enabled
-alpha 0 to 3 alpha value to use along with adaptiveP2
-pass 1 to 3 Num Passes of SGM
-frameIntervalInMS An Integer Time interval between two stereo pair in milisec.
-roiMode

0: Disable ROI mode (default)

1: Enable ROI mode

ROI (Region of interest) mode

-roiPosXStartDiv32

-roiPosYStartDiv8

-roiWidthDiv32

-roiHeightDiv8

An Integer

ROI parameters

ROI top-left x position (in 32-pixel unit)

ROI top-left y position (in 8-pixel unit)

ROI width (in 32-pixel unit)

ROI height (in 8-pixel unit)

Examples

These examples demonstrate how to call nvm_iofa_stereo_sci to perform various operations.

  • Stereo Disparity estimations for YUV subsampling type 420 8-bit input
    $ nvm_iofa_stereo_sci -l left_inputfile -r rightinputfile -res wxh -o outputfile -inputBuffering 1 -frames 100 -v 3 

    The application performs stereo matching according to the parameters in the command line.

  • Stereo Disparity estimation for YUV subsampling type 400 10-bit input
    $ nvm_iofa_stereo_sci -l leftinputfile -r rightinputfile -res wxh -o outputfile -inputBuffering 3 -chromaFormat 0 -bit_depth  10 -frames 100 -v 3 

    The application performs stereo matching according to the parameters in the command line.

  • Stereo Disparity estimation for YUV subsampling type 444 16-bit input
    $ nvm_iofa_stereo_sci -l leftinputfile -r rightinputfile -res wxh -o outputfile -inputBuffering 2 -chromaFormat 3 -bit_depth  16 -frames 100 -v 3 

    The application performs stereo matching according to the parameters in the command line.

  • Stereo Disparity estimation for YUV subsampling type 420 8-bit input with CRC generation
    $ nvm_iofa_stereo_sci -l leftinputfile -r rightinputfile -res wxh -o outputfile -inputBuffering 2 -stereocrcgen crc.txt  -frames 100 -v 3 

    The application performs stereo matching according to the parameters in the command line.

  • Stereo Disparity estimation for YUV subsampling type 420 8-bit input with CRC comparison
    $ nvm_iofa_stereo_sci -l leftinputfile -r rightinputfile -res wxh -o outputfile -inputBuffering 2 -stereocrcchk crc.txt  -frames 100 -v 3 

    The application performs stereo matching according to the parameters in the command line.

Command Line Options for nvmedia_ofa_flow

The sample application command syntax is as follows:

$ nvm_iofa_flow_sci [options]

The command line options are as follows.

String parameters, such as pathnames, have a maximum length of 1024 characters. Restrictions on other types of parameters are noted individually.

Required Command Line Options
Option Parameter Description
-f Input file name Specifies input file name with full path.
-res <w>x<h>

Specifies the input surface dimensions, where <w> represents width and <h> represents height. Width and height are separated by a letter ‘x’ with no spaces, for example:

-res 640x480

<w> has a range of [32, 8192].

<h> has a range of [32, 8192].

Optional Command Line Options
Option Parameter Description
-h n/a Displays guidance on using this application.
-version n/a Get NvMediaIOFA Major/Minor version
-v

0: Errors (default)

1: Warnings

2: Info

3: Debug

Logging level.
-gridSize

0: Grid Size 1x1

1: Grid Size 2x2

2: Grid Size 4x4

3: Grid Size 8x8

Output Grid Size set by Application.

-o Flow Output file name Specifies flow output file name with full path.
-co Cost Output file name Specifies cost output file name with full path.
-flowcrcgen A pathname of a file to be created with the filetype .txt Generates CRC values of flow map in the specified file.
-flowcrcchk A pathname of an existing file with the filetype .txt Checks CRC values generated from flow map against values from the specified file.
-costcrcgen A pathname of a file to be created with the filetype .txt Generates CRC values of cost surface in the specified file.
-costcrcchk A pathname of an existing file with the filetype .txt Checks CRC values generated from cost surface against values from the specified file.
-if

0: YV12 (default)

1: IYUV

Input YUV file format.
-chromaFormat

0 : 400

1 : 420 (default)

2 : 422

3 : 444

Chroma Format IDC of input yuv (level 0 / base layer for flow estimation).
-pydChromaFormat

0 : 400

1 : 420 (default)

2 : 422

3 : 444

Chroma Format IDC of input pyramid except base layer.
-frames An integer Specifies the number of input frames to be processed
-inputBuffering

1 to 8

default (1)

Input YUV pair buffering
-profile

0: Disable Profile

1: Enable Profile with Async mode

2: Enable Profile with Sync mode

Enable OFA profiling to get SW and HW overhead
-etype 0: Optical Flow HQ mode (default) Specifies of matching quality mode.
-bit_depth

8: 8 bits (default)

10: 10 bits

12: 12 bits

16: 16 bits

Number of bits per components on input surfaces.
-timeout An Integer (default – 30ms) timeout value to wait for OFA operation to finish in ms
-p1 An integer SGM penalty1 value
-p2 An integer SGM penalty2 value
-diag

0: Disable diagonal mode

1: Enable diagonal mode

SGM diagonal mode
-adaptiveP2

0: Disable adaptive P2

1: Enable adaptive P2

SGM adaptive p2 mode enabled
-alpha 0 to 3 alpha value to use along with adaptiveP2
-pass 1 to 3 Num Passes of SGM
-frameIntervalInMS An Integer Time interval between two stereo pair in milisec.
-roiMode

0: Disable ROI mode (default)

1: Enable ROI mode

ROI (Region of interest) mode

-roiPosXStartDiv32

-roiPosYStartDiv8

-roiWidthDiv32

-roiHeightDiv8

An Integer

ROI parameters

ROI top-left x position (in 32-pixel unit)

ROI top-left y position (in 8-pixel unit)

ROI width (in 32-pixel unit)

ROI height (in 8-pixel unit)

Examples

These examples demonstrate how to call nvmedia_iofa_flow_scito perform various operations.

  • Flow estimation for YUV subsampling type 420 8-bit input
    $ nvm_iofa_flow_sci -f inputfile -res wxh -o outputfile -inputBuffering 1 -frames 100 -v 3 
  • The application performs flow estimation according to the parameters in the command line.

  • Flow estimation for YUV subsampling type 400 10-bit input
    $ nvm_iofa_flow_sci -f inputfile -res wxh -o outputfile -inputBuffering 3 -chromaFormat 0 -bit_depth 10 -frames 100 -v 3 

    The application performs flow estimation according to the parameters in the command line.

  • Flow estimation for YUV subsampling type 444 16-bit input
    $ nvm_iofa_flow_sci -f inputfile -res wxh -o outputfile -inputBuffering 2 -chromaFormat 3 -bit_depth  16 -frames 100 -v 3  

    The application performs flow estimation according to the parameters in the command line.

  • Flow estimation for YUV subsampling type 420 8-bit input with CRC generation
    $ nvm_iofa_flow_sci -f inputfile -res wxh -o outputfile -inputBuffering 2 -flowcrcgen crc.txt  -frames 100 -v 3  

    The application performs flow estimation according to the parameters in the command line.

  • Flow estimation for YCbCR subsampling type 420 8-bit input with CRC comparison
    $ nvm_iofa_flow_sci -f inputfile -res wxh -o outputfile -inputBuffering 2 -flowcrcgen crc.txt  -frames 100 -v 3  

    The application performs flow estimation according to the parameters in the command line.