Components

Core

The functions of the basic core entities are application layer construction, module lifecycle management and resource management.

CManager

Process input events and manage the lifecycle of the application.

CChannel

Provides entities to manage channels.

CBaseModule

The base class of modules that provides interfaces of processing flow and connection service.

Modules

Modules are the entities that retrieve image frames from the stream and implement specific business logic processing for the image.

SIPL Module

Call NvSIPLCamera APIs to capture images and process the images using ISP.

CUDA Module

Retrieves image frames and maps the NvSciBuf object as a CUDA array or CUDA pointer. The CUDA frames can be used in inference tasks.

Encoder Module

Retrieves image frames and feeds them into NVENC for H.264 or H.265 encoding.

Nvm2d Module

Performs 2d operations, such as color format conversion and image stitching/compositing etc.

Display Module

Retrieves image frames and displays them on the screen via OpenWFD.

FileSource Module

Reads frame data from YUV files or decodes frame data from H264/H265 files as pipeline input.

Control Channel

Control channel is a star network communication component supporting intra-process, inter-process, and inter-SoC communications. Typically use cases are as follows:

1. Consumers report heartbeat to the producer so that the producer can detect dead consumers and detach them.

2. Late consumers send a connection and disconnection request on late-attach and detach.

When using, users need to select a process as the central node for each SoC, and the central node will be connected to all other processes via NvSciIpc. In each process, users can create multiple CMsgReaders and CMsgWriters. CMsgReaders and CMsgWriters are organized by a ‘zone’. When creating a CMsgReader or CMsgWriter, users need to specify a zone id. CMsgReaders and CMsgWriters in the same zone can communicate with each other. CMsgReaders and CMsgWriters in different zones are isolated from each other. Messages sent by a non-central node will be sent to the central node first, and the central node will broadcast to others processes that have CMsgReaders in the same zone. Here is a topology example:

By design, the control channel component can be broken down into three layers: Data Link, Network and Presentation layer. The following diagram shows the hierarchical architecture.

The following diagram shows how each layer interacts with each other.

CMsgReader

Receives messages and invokes the callbacks registered by users to process messages.

CMsgWriter

Provides APIs to send messages to CMsgReaders in the same zone.

CControlChannelManager

Performs message routing and dispatching. I.e, when CIpcLink receives a message, it reports to CControlChannelManager. CControlChannelManager identifies which zone it belongs to and dispatches it to the current process’s CMsgReaders of this zone. When a user writes a message via CMsgWriter, CMsgWriter reports the message to CControlChannelManager. CControlChannelManager broadcasts this message to other processes via CIpcLink (for a non-central node, it only sends to the central node).

CIpcLink

Provides interfaces to write, read messages by NvSciIpc.