Image Authentication Status

Image authentication is performed by SIPL on RAW frame data after reception from CSI bus, unpacking (adding pad bits) and placing unpacked image data into system memory. Authentication runs on data before post-processing is applied.

The input pixel format depends on camera module configuration specified in camera platform config VirtualChannelInfo member inputFormat. Authentication can start immediately after frame is placed into memory in parallel with the image post-processing step.

Application consuming post-processed frames from SIPL must wait for image authentication status before it makes a critical decision based on image pixel data. Application can use an image, for example sensor fusion, while it waits for authentication status, but it cannot make vehicle steering decisions until authentication status is available.

Authentication status of a frame is reported to Application separately from a frame itself, to reduce overall image processing latency, because authentication can be performed in parallel to post-processing of a RAW frame.

In case image authentication is done sequentially to post-processing extra latency would be introduced to camera capture pipeline. To avoid this SIPL performs authentication in parallel to post-processing step. Therefore, ISP post-processed images may be passed UP to a camera application before authentication is finished. Exact timing depends on the size of a frame to authenticate. It is then up to the application to use frame data and then wait for authentication status before safety decision is made (e.g., before vehicle steering decision is made).

This is illustrated on a following diagram showing standard SIPL image processing pipeline with an addition of Image Authentication Status reporting.

Asynchronous notification of authentication status for each frame is done via existing SIPL notification queue INvSIPLNotificationQueue. Assumption is on camera application that it will monitor the notification queue for authentication status events and will do a matching between an event and previously received post-processed buffer.

enum NotificationType {
    ...
    NOTIF_ERROR_ICP_AUTH_FAILURE,
    NOTIF_INFO_ICP_AUTH_SUCCESS,
    ...
}
 
NvSIPLPipelineNotifier::NotificationData {
    eNotifType type; // NOTIF_ERROR_ICP_AUTH_FAILURE or NOTIF_INFO_ICP_AUTH_SUCCESS
    uint32_t pipelineId;
    uint64_t frameSequenceNumber;
}

Application can identify each individual frame based on pipeline ID (one per Sensor) and a frame sequence number (always incrementing).