SIPL Guidance on Output Image Formats
SIPL supports multiple image formats using the ICP and ISP outputs. This topic describes how the output formats are set and which parameters can be modified to override output formats under specific conditions.
ICP Output Formats
ICP output formats are dependent on the sensor used.
Based on the vendor and sensor module specifics, the output format of ICP is determined
using the PlatformCfg::DeviceBlockInfo::
CameraModuleInfo::SensorInfo::VirtualChannelInfo::cfa
and
inputFormat
fields in the PlatformCfg struct provided as input to the
INvSIPLCamera::SetPlatformCfg
API.
Refer to NvSIPLCapStructs.h for the following:
- Supported values for input format that correspond to
NvSiplCapInputFormatType
enum. - Supported values for cfa that correspond to the valid
NVSIPL_PIXEL_ORDER
flags.
SIPL however does pose restrictions such as rejecting bad combinations of input format and pixel order. Some examples of the invalid combinations are as follows.
- NVSIPL_CAP_INPUT_FORMAT_TYPE_YUV422 and NVSIPL_PIXEL_ORDER_RGBA
- NVSIPL_CAP_INPUT_FORMAT_TYPE_RAW6 and NVSIPL_PIXEL_ORDER_RGBA
- NVSIPL_CAP_INPUT_FORMAT_TYPE_RGB888 and NVSIPL_PIXEL_ORDER_RGGB
- NVSIPL_CAP_INPUT_FORMAT_TYPE_YUV422 and NVSIPL_PIXEL_ORDER_RGGB
- NVSIPL_CAP_INPUT_FORMAT_TYPE_RAW6 and NVSIPL_PIXEL_ORDER_YUV
- NVSIPL_CAP_INPUT_FORMAT_TYPE_RGB888 and NVSIPL_PIXEL_ORDER_YUV
ISP Output Formats
Refer to the table provided as part of NvSIPLCamera.hpp for documentation on INvSIPLCamera::RegisterImages API for the various ISP output formats supported by SIPL.
In 6.0 release, users are expected to call INvSIPLCamera::GetImageAttributes API before calling INvSIPLCamera::RegisterImages API in order to validate the attributes provided by the user. The sequence of API calls is as follows:
- Application calls NvSciBufAttrListCreate to create an unreconciled buffer attribute list.
- Application implements and calls OverrideImageAttributes(NvSciBufAttrs) function to set the required output format for each ISP output in case it is different from the default.
- Application calls
INvSIPLCamera::GetImageAttrs(NvSciBufAttrs)
, which checks and validates the buffer attributes in case you chose to override the default format. Otherwise, Application adds attributes for the default formats.
- Application calls
NvSciBufAttrReconcile
to reconcile all buffer attributes across other buffer attribute lists (other consumers for this buffer, which could be any downstream engine, such as VIC, IEP, and CUDA) - Application calls
NvSciBufObjAlloc
on the reconciled attribute list to create a buffer object - Application calls
INvSIPLCamera::RegisterImages(NvSciBufObj)
, which only accepts the buffer if one of the attribute lists used to reconcile and allocate was verified by SIPL usingINvSIPLCamera::GetImageAttrs
- ISP output images will be a binary match if the default NITO provided by NVIDIA is used while requesting the same output formats with same resolution.
- The ISP0 and ISP1 outputs can have different formats as long as they both are YUV type.
- The camera tuning does not have to be re-run for different formats since formats are just different ways to represent images in the memory. However, the data itself will differ as the representation of the memory varies across formats.
OverrideImageAttributes
ISP supports three types of formats:
- YUV Semi Planar Images
- YUV Packed Images–Luma is a type of YUV Packed Image
- RGBA Packed Images
Creating packed type buffers (2 and 3) are straightforward: you use
NvSciBufImageAttrKey_Plane***
type attributes to override Image formats.
For semi-planar type buffers (1), however, NvSciBuf provides attribute keys to create buffers of the above types using two methods:
- Using
NvSciBufImageAttrKey_Plane***
type attributes - Using
NvSciBufImageAttrKey_Surf***
type attributes
NvSciBuf added the Surf
type attributes to make the task of creating
multi-plane image buffers easy for the user. However, in Plane
type
attributes you must provide the height and width of each plane to determine the memory
layout of the image.
In SIPL the library oversees calculations for height and width of the buffer factoring in
multiple criteria such as sensor output resolution, input crop, downscale, and so on. This
is complicated if you supply Plane
type attributes to SIPL while trying to
create multi-plane images. Hence, SIPL restricts you to Surf
type
attributes when using multi-plane YUV images.
Examples
For YUV 444 SEMI-PLANAR UINT 16 BLOCK LINEAR image, the following attributes should be set to the corresponding values:
Attribute Name | Attribute Value |
---|---|
NvSciBufGeneralAttrKey_Types | NvSciBufType_Image |
NvSciBufImageAttrKey_SurfType | NvSciSurfType_YUV |
NvSciBufImageAttrKey_SurfBPC | NvSciSurfBPC_16 |
NvSciBufImageAttrKey_SurfMemLayout | NvSciSurfMemLayout_SemiPlanar |
NvSciBufImageAttrKey_SurfSampleType | NvSciSurfSampleType_444 |
NvSciBufImageAttrKey_SurfComponentOrder | NvSciSurfComponentOrder_YUV |
NvSciBufImageAttrKey_SurfColorStd | NvSciColorStd_REC709_ER |
NvSciBufImageAttrKey_Layout | NvSciBufImage_BlockLinearType |
For YUV 444 PACKED UINT 8 BLOCK LINEAR image, the following attributes should be set to the corresponding values
Attribute Name | Attribute Value |
---|---|
NvSciBufGeneralAttrKey_Types | NvSciBufType_Image |
NvSciBufImageAttrKey_PlaneCount | 1 |
NvSciBufImageAttrKey_Layout | NvSciBufImage_BlockLinearType |
NvSciBufImageAttrKey_PlaneColorFormat | NvSciColor_A8Y8U8V8 |
NvSciBufImageAttrKey_PlaneColorStd | NvSciColorStd_REC709_ER |