DEVELOPER BLOG

Data Science | Networking |

Programming the Entire Data Center Infrastructure with the NVIDIA DOCA SDK

Today, in his NVIDIA GTC Fall keynote, CEO Jensen Huang introduced a new kind of processor, the BlueField-2 data processing unit (DPU), a powerful new software development kit for the DPU, DOCA, along with a three year roadmap of DPU and AI innovation. The NVIDIA BlueField-2 DPU is the world’s first data center infrastructure on a chip architecture optimized for modern enterprise data centers. DOCA is central to enabling the DPU to offload, accelerate, and isolate data center services to propel enterprise and AI applications to new levels of performance, security, and reliability. Specifically, DOCA is designed to enable you to deliver a broad range of accelerated software-defined networking, storage, security, and management services running on current and future BlueField DPUs.

:  Block diagram shows how DOCA enables software-defined data center infrastructure applications to run on top of the NVIDIA DPU features to benefit from hardware acceleration.
Figure 1. DOCA layered under data center infrastructure applications that run on the DPU.

NVIDIA also announced BlueField-2X, the world’s first AI-Powered DPU, combining all the features of BlueField-2 with NVIDIA Ampere GPU technology. DOCA is adding support for BlueField-2X, enabling you to build AI-driven, optimized infrastructure management and cyber security applications. For more information about how DPUs improve data center security and efficiency, see What’s a DPU and NVIDIA DPU.

With DOCA and its BlueField DPU lineup, NVIDIA is re-inventing the enterprise data center stack, allowing you to build secure and accelerated infrastructure services. DOCA is to DPUs what CUDA is to GPUs. Just as CUDA enables you to program accelerated computing applications, DOCA enables you to program the acceleration of data processing, for moving data into and out of servers, VMs, and containers. DOCA sits alongside CUDA to leverage the entire range of NVIDIA AI applications in a secure, accelerated data center.

DOCA is available for select early access partners. If you are interested, you can sign up for future information on the DOCA Developer Zone page. BlueField-2 is sampling now and available with drivers, tools, accelerated libraries, and support several OS distributions.

In this post, I explore the benefits of DOCA and dive into the APIs that provide a powerful development tool for enhancing server performance, efficiency, and security, from the data center to the edge.

Solving the challenges of software-defined everything

“Software is eating the world,” said Marc Andreessen in a renowned 2011 publication. Fast forward to 2020. Now, hardware appliances that ruled the enterprise data center for ~15 years have transformed into virtualized, pay-as-you-grow, software appliances that live in every server. Modern enterprise data centers are software-defined, fully programmable, and built to serve highly distributed application workloads across cloud, core, and edge environments.

This software-defined data center provides cloud-like flexibility and agility but consumes many cores. The software-defined “performance tax” sucks CPU cores away from business applications or tenants and reduces server and data center efficiency, sometimes severely.

DOCA enables application developers and NVIDIA technology partners to deliver services running on the DPU residing inside each data center node, making the DPU an isolated and secure services domainor enclave—for networking, security, storage, and infrastructure management. DPU accelerates all the key data center infrastructure services and runs control-plane software on the DPU, such as software-defined networking (SDN) controllers, distributed storage software, or next-generation firewall agents on its programmable Arm cores. 

DOCA is the development platform to create infrastructure services applications on top of the NVIDIA DPU, separately from the business application domain.
Figure 2. DOCA enables infrastructure (networking, storage, security, and management) applications running on the DPU in an isolated and secure services domain separate from the application domain on the CPU.

Accelerating time-to-market and for infrastructure services

DOCA is an SDK that brings together APIs, drivers, libraries, sample code, documentation, and prepackaged containers that activate the acceleration, security, and virtualization features of the BlueField-2 DPU. DOCA gives you a unified set of reliable tools to develop the key data center services running on the DPU, to handle multiple types of data processing. These include directing network traffic; accelerating, virtualizing, and compressing storage; encrypting and decrypting data; scanning for security threats; integrating with remote management tools; and running control plane applications to the Arm cores. You can program these offloads and accelerators to be used separately or together and integrate them with NVIDIA AI platforms for GPUs.

DOCA empowers application developers, appliance vendors, researchers, and NVIDIA software partners to program all these DPU capabilities from one SDK. Interfaces for many functions are available both at a low-level API and through high-level programming languages. These services running on the DPU have moved beyond being software-defined and hardware-accelerated, to become AI-enabled and easy to program.

Future generations of BlueField DPU will deliver even more compute power with new and enhanced accelerators. DOCA guarantees forward DPU compatibility with extended APIs for new functionality. NVIDIA is committed to enhance the performance through architectural process technology and software innovation. We consistently advance performance and capabilities across cloud, core, and edge environments and application workloads.

Simplifying service creation on the BlueField DPU

Here’s a closer look on how DOCA handles four critical data center functions: networking, storage, security, and management.

Networking

For networking the DPU accelerates the most advanced data center SDN and network function virtualization (NFV): open virtual switching (OVS), overlay networks (such as VXLAN), network address translation (NAT), autonomous load balancing, fine-grained traffic management, and content distribution networks.

DOCA enables you to hook the service application to the DPU-accelerated engines through standard DPDK APIs such as the rte_flow library that supports different flow-based actions including overlay encapsulation, header rewrite, hairpin, and metering for a broad range of virtualized network functions.

For even greater performance and host CPU efficiency, DOCA also includes native OVS and OVS-over-DPDK applications, offering seamless acceleration through the BlueField DPU hardware-based accelerated switching and packet processing (ASAP2) technology. While the DPU hardware accelerates host networking, defined by the OVS application running on its DOCA-programmed platform, the data center SDN controller orchestrates it all and connects through the DPU out-of-band management port.

P4 is another language that can take advantage of the flexible, programmable, data path accelerators in the BlueField DPUs. P4 support is a component of DOCA enabling support for future VNFs that may be developed for P4. This support is integrated with other APIs that already have a rich ecosystem of VNF offerings. This architecture allows the simultaneous execution of programs written for Kernel, RoCE, DPDK, SPDK, P4, and P4.runtime interfaces, which can seamlessly coexist and take advantage of DPU data-path acceleration. 

Storage

For storage, the DPU supports acceleration of software-defined elastic storage, NVMe over Fabrics (NVMe-oF), RoCE, data-at-rest encryption, data deduplication, distributed error correction, and data compression. The BlueField DPU NVMe SNAP technology delivers elastic block storage functionality and presents to the host remote block storage as if it were local NVMe block storage or a VirtIO blk device with low-latency, high throughput, and high IOPS.

DOCA has full support for the SPDK open source framework that empowers you to create your own storage solutions. The emulated storage can be managed transparently to host applications by servicing NVMe PCIe accesses and implementing any custom logic to use the many valuable accelerated BlueField DPU functions. This would include encryption, ECC distributed error correction, compression, deduplication, and malware scanning. If you’re programming through DOCA, you can now easily invoke these functions for data storage without explicitly programming the individual engines.

Security

The DPU also offloads, accelerates, and isolates all key data center security services. This includes support for next-generation firewalls, micro-segmentation, data-in-motion inline encryption with transparent IPSec and TLS, and intrusion protection. The DPU has a set of dedicated security engines that includes all the building blocks of any security solution.

DOCA includes a standardized set of APIs for developing security applications. Any security service starts from packet acquisition, decryption, stateful tracking of the connection status, and deep packet inspection up to layer 7. This classifies the application traffic as trusted or malicious. It also eventually translates the defined security policy to a series of actions, such as packet allow, drop, rewrite, or redirect. Programming is done through standard DPDK APIs, such as the following:

  • rte_security for encryption and decryption
  • rte_sft library for connection awareness
  • rte_regex library for regular expression pattern matching
  • rte_dpi library for deep packet inspection on all pipeline accelerated functions

DOCA also includes native kernel cryptography for IPSec, TLS, and storage AES-XTS encryption that is inline-accelerated by the DPU and can be easily leveraged with OpenSSL or OVS-IPSec-based applications. This enables the DPU to encrypt and decrypt all the traffic transparently without the host being aware. This includes the key management performed by the DPU inside a secure enclave.

Using fast memory access to the host, based on the RDMA libraries, DOCA enables you to create tight host introspection solutions. Autonomous malware scanning extends the visibility of what’s going on to the host from the DPU. In case of an identified malicious activity, you can program the DPU hardware-based acceleration engines to perform policy enforcement at the line rate.

Infrastructure management

Traditional management runs agents on each server. However, these agents consume CPU cycles that would otherwise be available for business applications. Agents often lack visibility into the network traffic between VMs and containers on the server. If they perform functions like packet filtering or traffic telemetry, they consume even more CPU cycles. If the server suffers a serious fault, a VM or container-based management agent is unlikely to be able to report the status or reboot the server.

DOCA allows isolated, DPU-based agents to perform in-band or out-of-band management without burdening the server CPU. If the server needs a reset, or even if the tenant or business application requires a bare metal server with no agents, a DOCA-programmed DPU can still send telemetry, perform a remote reset, or allow the secure boot of the server, all without running an agent on the server CPU.

BlueField and DOCA: Better together

The BlueField DPU with DOCA provides unique opportunities to combine data center infrastructure services that normally cannot be deployed together. For example, encryption of data in motion by the CPU or a separate, look-aside crypto card cannot be combined with RoCE, compression, hashing, or overlay networks. In fact, doing such encryption disables these and many other network offloads. This is because encrypting the data first means that the network devices are blinded to the packet’s contents so it cannot perform clever packet rerouting, filtering, congestion management, and so on. However, by using BlueField DPU with DOCA, you can program a combination of RoCE, VXLAN, hash calculations, compression, and many other offloads combined with encryption and with each other.

DOCA brings access to all the functionality together into one SDK and enables you to immediately extract the DPU value by using accelerated libraries that have the same open APIs they are using today. You can also use DOCA to transparently port key control plane applications or agents from the X86 domain to the DPU Arm cores, improving server performance, efficiency, and workload and security isolation.

Programming data-processing acceleration and AI

Critically, DOCA takes this one step further to enable the most advanced, GPU-accelerated, AI workloads, as a fully integrated and tested component of the larger NVIDIA NGC-accelerated computing software platforms.

For traditional enterprise applications, DOCA accelerates data center infrastructure services in systems that include the BlueField DPU. However, for accelerated AI and data analytics workloads, there is a huge advantage for systems that include both a DPU and GPU. DOCA is integrated into the NGC certified program. It capitalizes on the enormous development, integration, and testing investments that enable our entire range of AI application frameworks (NVIDIA Jarvis, NVIDIA Merlin, NVIDIA Metropolis, NVIDIA Clara, NVIDIA Aerial, and others). DOCA integration with NGC platforms also unleashes the power of the full range of third-party software infrastructure and applications.

You can use DOCA with CUDA to accelerate compute with the GPU and data center services with the DPU. You can also use DOCA to enable GPUDirect and accelerate GPU-to-GPU communications across the network.

Simplifying data center acceleration and efficiency with the DPU

DOCA is the unified programming gateway for the NVIDIA developer community, partners, and customers to develop on the DPU and benefit from accelerated networking, storage, security, and simplified management. DOCA-developed programs running on BlueField DPUs offload, accelerate, and isolate end-to-end data center services for every server and storage node in the data center. So, enterprises and private clouds get the flexibility and scalability of software-defined services with the performance and efficiency of the hardware-accelerated DPU.

By enabling data center infrastructure to be software-defined and hardware-accelerated by the DPU, you enjoy faster performance, greater efficiency, and improved security on all servers. You achieve a new architecture in which the data center is the new unit of computing. The NVIDIA DPU delivers the features of data center infrastructure on a single chip and DOCA makes it easy to achieve greater data center scalability, performance, and threat protection.

Try DOCA today

You can experience DOCA today with the BlueField DPU software package, which includes DOCA runtime accelerated libraries for networking, storage, and security. The libraries help you program your data center infrastructure running on the DPU.

The DOCA program is available now for select early access partners. To receive news and updates about DOCA or to become an early access partner, see the DOCA page.