Edge AI developers are building AI applications and products for safety-critical and regulated use cases. With NVIDIA Holoscan 1.0, these applications can incorporate real-time insights and processing in milliseconds.
With the recent release of NVIDIA Holoscan 1.0, developers can more easily build production-ready applications for multimodal, real-time sensor processing. NVIDIA AI Enterprise now provides security patches and critical bug fixes with guaranteed API stability for the Holoscan software stack. This reduces the cost of product maintenance and streamlines deployment of edge AI applications at scale.
What is NVIDIA Holoscan?
Holoscan is the NVIDIA domain-agnostic multimodal real-time AI sensor processing platform that delivers the foundation for developers to build their end-to-end sensor processing pipeline. The platform was built with the architectural goals of performance, usability, and production readiness. At every step of sensor processing pipelines, Holoscan delivers optimized performance while minimizing development complexity. The pipeline includes:
- Sensor data ingress
- Accelerated computing and AI inferencing
- Real-time visualization, actuation, and data stream egress
Specifically, the Holoscan platform includes three main components. The first component is its software. Domain-agnostic, AI-enabled, real-time sensor processing pipelines are built on a foundation that includes data movement, accelerated computing, real-time visualization, and AI inferencing.
NVIDIA Holoscan is the only singular platform that brings these together with a guarantee of application performance while reducing complexity and decreasing time to market. The NVIDIA Holoscan SDK provides a low-code, high-performance environment for developers to build their pipeline in their programming language of choice through Python or C++ APIs.
The second component of the Holoscan platform is its hardware. Holoscan supports several hardware configurations for various power, size, cost, compute, and configuration needs, including:
- NVIDIA Jetson Orin family of products (Orin Nano, Orin NX, and AGX Orin)
- NVIDIA IGX Orin
- NVIDIA IGX Orin plus NVIDIA discrete GPUs, offering the combined power of integrated and discrete GPUs
- x86 systems with NVIDIA discrete GPUs (workstation or cloud)
The third component of the Holoscan platform is enterprise support, which is designed to fit the pre-release and post-release needs of medical technology products. The Holoscan software offering available through NVIDIA AI Enterprise provides full-stack, long-term (up to 10 years) software support.
Specifically, this offering releases security patches and critical bug fixes with guaranteed API stability to developers, significantly reducing the cost of product maintenance. This not only streamlines operations but also instills a higher level of confidence in deploying AI applications at scale.
Production readiness
The introduction of AI-powered computing has opened up a range of possibilities for businesses looking for new ways to deliver differentiation and value while using data more effectively. However, the AI software stack is complex, with hundreds of dependencies that heavily leverage third-party OSS libraries (Figure 3).
Increased software complexity equates to increased vulnerability. Security, stability, reliability, and production readiness are crucial but challenging for developers and organizations looking to leverage this software in their applications. This is particularly true for safety-critical and regulated industries such as medical devices or robotics.
For example, in 2023, the FDA issued new guidance for cybersecurity in medical devices, stating the potential for Refuse To Accept (RTA) premarket submissions that don’t contain information required by section 524B (Ensuring Cybersecurity of Devices) of the FD&C Act. Under the law, manufacturers must design and release updates and patches after a product goes to market, provide a software bill of materials (SBOM), and submit a plan for identifying and addressing “post-market cybersecurity vulnerabilities.”
This poses additional challenges on how to build and maintain AI-enabled platforms for such developers and companies:
- Maintaining a secure software stack is resource-intensive.
- Assessment and exploration are needed after a CVE scan for further actions and measures.
- A software package security update might require a subsequent change in a dependent API, which can break backward compatibility.
- Continuous monitoring and updating of the full stack is needed.
This means a regular cadence of security patches and bug fixes, which leaves companies with two costly options:
- Back-porting the patches and fixes to avoid breaking the stack and causing potential need of verification, validation, and re-certification.
- Rolling forward with a potential risk of breaking product features and introducing new bugs, requiring costly cycles of verification, validation, and potential re-certification.
What production-ready means for Holoscan developers
The Holoscan platform delivers an accelerated and full-stack infrastructure for real-time processing of multimodal data at the edge. Over the last releases, the Holoscan platform features and capabilities have focused on high-performance compute, low-latency, and high-throughput multimodal sensor streaming, in addition to superior usability, and decreased time to market.
Joining NVIDIA AI Enterprise, Holoscan 1.0 adds production readiness to support developers in their end-to-end product lifecycle. It also accelerates companies in their AI innovation journey while reducing the total cost of product ownership and maintenance.
Building products on NVIDIA IGX
With full-stack software support—including firmware, OS, drivers, AI frameworks, GPU acceleration, and SDKs—developers can rely on a single source of support for their entire stack while enjoying guaranteed long-term API stability and a consistent software bill of materials (SBOM). This offering is optimized to meet the heterogeneous needs of edge customers, such as:
- CVE patches and critical bug fixes with guaranteed API stability.
- Flexible software branches of different duration to choose from (nine months to 10 years), designed to fit the needs of developers of different industries and in different phases of product development. Releases occur every six months for the nine-month support branch and every 2.5 years for the 10-year branch.
- Yocto enterprise support for embedded developers preferring to deploy their products with customized Yocto-based distributions.
Yocto-based distributions enable minimal SBOM with a minimum set of required components for the device to function. They simplify FMEA, SW/SOUP hazard analysis, cybersecurity hazard analysis, and penetration testing. They lower the regulatory burden and cost of post-market maintenance. They also support efficient use of embedded hardware resources, including storage, memory, processing power, and electrical power.
For more information about Holoscan Yocto efforts providing distribution optimized for edge deployment, see Deploying AI-Accelerated Medical Devices with NVIDIA Clara Holoscan.
Developing, testing, or deploying on x86 systems
Developers can benefit from partial-stack support, exclusive of OS, on their x86 systems. For more information, see NVIDIA-Certified Systems. Maintaining a consistent software stack on x86 and IGX for the same branches minimizes the development-to-deployment barrier. This is made possible by providing consistency across x86 platforms used for development and deployment for testing, verification, and deployment.
Summary
NVIDIA Holoscan, together with the industrial-grade NVIDIA IGX Orin platform and NVIDIA AI Enterprise, relieves organizations of the burden of validating and maintaining an edge platform. It enables teams to focus on building AI, developing platforms with a growing ecosystem of first-party and third-party developers, and quickly adding business value.
In particular, medical-grade production IGX offerings are available through multiple ODM partners, including, but not limited to:
To get started, download Holoscan 1.0 and check out the release notes. Ask questions and share information in the NVIDIA Developer forums.
Join us in person or virtually at NVIDIA GTC 2024 to learn more about NVIDIA Holoscan and the latest advancements in AI sensor streaming applications.
