NVIDIA Mellanox 5T for 5G technology provides a real-time and high-performance solution for building an efficient, time-synchronized CloudRAN infrastructure.
Time synchronization and achieving high time accuracy for network traffic between O-RAN 7.2x compliant front-haul, mid-haul, and back-haul components in a cloud-native RAN (CloudRAN) environment has always been a challenge. Further, maintaining real-time and precise time-bound transport of enhanced Common Public Interface (eCPRI) packets within a specified time window over Ethernet/IP links of fronthaul radio access networks (RAN) has been a big obstacle to 5G wireless rollouts.
In this post, we discuss transforming 5G with CloudRAN flexibility, performance, and reliability using low-cost NVIDIA SmartNICs, the NVIDIA Aerial SDK, and NVIDIA GPUs with off-the-shelf servers.
The 5G revolution
Telecommunications (telco) providers are undergoing a business transformation, replacing traditional network infrastructure with off-the-shelf servers to achieve open, cost-effective, scalable, and programmable networks.
5G is the foundation for connecting billions of devices at Gbps data rates and sub-millisecond latency. Thus, 5G represents a perfect connectivity storm in terms of capacity, density, bandwidth, and low latency that will quickly supersede existing 4G LTE network architectures.
To smooth the transition to 5G networks, the telco industry has made great progress by re-architecting a 4G LTE evolved packet core into a Control and User Plane Separation (CUPS) and 5G User Plane Function (UPF)–based mobile packet core.
Virtualizing the radio access network (RAN) can bring the following benefits:
- Coordination, centralization, and virtualization in mobile networks
- Enablement of new services at the network edge
- Support for resource pooling (more cost-efficient processor sharing), network slicing, and load balancing
- Scalability (more flexible hardware capacity expansion) from high-capacity cells to low-capacity cells
- Layer interworking (tighter coupling between the application layer and the RAN)
Challenges in building time-sensitive radio access networks
While telco cloud core networks are now open, disaggregated, programmable, and efficient, similar transformations for RANs have been slow to evolve. While the O-RAN 7.2 specification provides guidance about the radio unit (RU), distributed unit (DU), and centralized unit (CU) separation of baseband unit (BBU) functionality, efficient and real-time data transmission between multiple RRUs and BBUs hasn’t been an easy feat.
Time synchronization and achieving precise time accuracy of the network traffic between RUs, CUs, and DUs in a cloud-native RAN (CloudRAN) has remained a challenge. Further, real-time and precise time-bound transport of enhanced Common Public Interface (eCPRI) packets within a specified time window over Ethernet IP links of fronthaul RAN has been a big obstacle.
These challenges coupled with the distributed, multi-cloud nature of far-edge CloudRAN deployments pose a serious challenge to realizing the dream of using time-sensitive, cloud-native 5G networks for many latency-sensitive 5G use cases.
Why are incumbent FPGA vRAN solutions inefficient?
Synchronizing clocks in distributed cloud-native edge networks has been a long-standing problem. Accurate clocks enable applications to operate on a common time axis across the different nodes and locations, which in turn precisely and accurately enables consistency, event ordering, causality, and scheduling of tasks and resources. The nature of mobile networks requires extreme precision in these areas to allow many different user equipment (UE) devices to connect simultaneously and to maintain their connections as they move between RRUs and BBUs.
Field programmable gate array-based (FPGA) network adapter cards have been a common way to solve timing reference and synchronization challenges. However, with the move to 5G, Open RAN, and eCPRI, this solution is non-optimal, inefficient, and hence unfit for the following reasons:
- Power-hungry: Typical FPGA NICs are power-hungry and consume 3x more power than a SmartNIC solution.
- Expensive: FPGAs are reconfigurable. However, they are up to 3x more expensive than purpose-built SmartNICs. FPGAs are great for implementing point functions but not suitable for massively scaled, CloudRAN deployments.
- Tightly coupled to software or applications: FPGA logic is tightly coupled to application logic implemented in the FPGAs, creating an undesired dependency between an application and the network subsystems. Further, FPGA logic is designed for predefined solution parameters, meaning that any changes to application needs would entail an FPGA logic re-write. While theoretically field-upgradable, in practice upgrading FPGAs in the field is too dangerous to be practical.
SmartNICs, on the other hand, offer a consistent, software-programmable, open interface to all applications that is decoupled from application logic. Application upgrades are easily integrated and a well-designed SmartNIC can completely offload RAN VNFs by doing all the heavy lifting of timing synchronization and real-time data transmission through hardware acceleration engines.
- Custom development: As disaggregated first-generation fronthaul systems began reaching the market, FPGAs were used to meet the real-time requirements and essentially function as a NIC within an industry-standard x86 server. However, as always FPGAs are good for prototypes, but not for mass production. FPGAs simply weren’t the optimal solution and needed custom development for each link type and speed.
As network speeds are increasing, each link speed (25/50/100/200Gbps), cabling technology (copper/optical) and transceiver (SFP, SFP+, QSFP, and so on) requires new FPGA NIC hardware design, slowing innovation and increasing cost. This also leads to interoperability issues with existing cables in the market. Further, the Ethernet link quality with FPGAs is not comparable to the market-leading NVIDIA Mellanox ConnectX SmartNIC family. Mellanox SmartNICs support all standard network speeds and link types out-of-the-box and are flexible and highly reliable.
- Closed source development: Lastly, FPGA NICs hinder open source development and often rely on proprietary tools from FPGA vendors. Common open source management tools such as secure boot, PXE boot, host management, and standard data path software libraries such as Data Plane Development Kit (DPDK) aren’t available for FPGA NICs. Standard network adapters like the Mellanox ConnectX-6 Dx are open and programmable and support all community supported software tools and libraries.
As summarized in Table 1, ASIC-based SmartNICs outperform FPGA NICs in building an efficient and O-RAN compliant CloudRAN.
|ASIC SmartNIC||FPGA NIC|
|Power-efficient||Up to 3x the power|
|Cost-effective||Up to 3x the cost|
|Open source/Community software||Tight coupling of FPGA logic and software|
|Software programmability||FPGA programmability|
|One design supports all Link speeds 10 to 200 GbE||Design per speed|
|Industry standard software for MGMT||Proprietary vendor tools|
|Improved latency||Poor latency|
|ITU-G 8273.2 compliant||Re-do compliancy for each FPGA design|
Breakthrough in time-triggered transmission technology for telco (5T for 5G)
As NVIDIA CEO Jensen Huang announced during GTC Digital 2020, breakthrough 5T for 5G technology is now available in ConnectX-6 Dx SmartNICs. ConnectX-6 Dx is the industry’s first SmartNIC to offer this super-precise time synchronization for eCPRI and provides a 5-in-1 solution:
- Real-time transmission technology
- High throughput
- Low latency
- Low power
- A broad set of in-hardware acceleration capabilities
Major 5T for 5G features
5T for 5G is Time-Triggered Transmission Technology for Telco and it delivers unique capabilities for 5G cloud-native vRAN infrastructure rollouts.
- Real-time data transmission hardware offloads
- O-RAN eCPRI encapsulation and decapsulation offloads
- Accurate packet pacing
- ASAP2 time-bound packet flow engine
Real-time data transmission hardware offloads
With 5T for 5G, multiple packet data streams are transmitted in time division multiplexed packet streams over the fronthaul network, between the distributed units and the radio units.
Multiple packet streams are scheduled by software but transmitted and accelerated by the ConnectX SmartNIC hardware. This hardware-accelerated time synchronization enables accurate and efficient management of multiple data streams across a matrix of multiple frequencies and time slices.
The benefits of this approach are increased time precision and accuracy without burdening the CPU with data transportation or synchronization tasks, and without the need for expensive FPGAs.
- Highest clock accuracy: An accurate clock reference is of paramount importance for time-triggered, real-time transmission of eCPRI packets. After all, we all know the saying “garbage in, garbage out.” 5T for 5G advanced timing hardware-offload technology exceeds the stringent ITU-T G.8273.1/G.8273.2 profile timing specifications by delivering clock accuracy of < 16 ns.
- Hardware timing synchronization offloads: 5T for 5G implements IEEE 1588v2 PTP wall clock (UTC format) in the SmartNIC. It enables any real-time software application to synchronize itself to the received timestamp information directly and accurately from the SmartNIC without any data path translations or overheads.
- O-RAN eCPRI windowing: ConnectX-6 Dx precisely and accurately transmits eCPRI packets within the one microsecond window specified in O-RAN 7.2 specification. Thus, 5T for 5G technology empowers the application to receive a dedicated eCPRI flow for a specific layer and antenna port directly from the SmartNIC into the application’s dedicated buffer. This functionality can be easily enabled using cuVNF which is part of Aerial SDK.
O-RAN eCPRI encapsulation and decapsulation offloads
Figure 4 shows the eCPRI protocol stack, which is critical for open and interoperable 5G radio access networks.
5T for 5G technology handles eCPRI packet encapsulation and decapsulation, thus freeing up CPU cycles associated with this operation. This further improves latency and reduces datapath processing overhead, further improving efficiency.
Accurate packet pacing
ConnectX-6 Dx SmartNIC enforces accurate bitrates for eCPRI flows as mentioned in the O-RAN 7.2 specification.
ASAP2 time-bound packet flow engine
Accelerated Switching and Packet Processing (ASAP2) enables software-defined, hardware-accelerated virtual network functions (VNF) and containerized network functions (CNF) to precisely steer traffic in the ingress and egress directions as desired by networks services and applications. The ASAP2 time engine embedded in ConnectX-6 Dx SmartNICs uses network accurate time as a parameter in all time-based networking operations.
For example, an application can direct the SmartNIC to send certain packets at the exact network time, specified in YY:MM:DD:HH:MM:SS.ns granularity. Having such precise and accurate timing control leads to an efficient software implementation of eCPRI fronthaul applications, which are heavily constrained in real time.
5T for 5G technology embedded in Mellanox ConnectX-6 Dx SmartNICs delivers hardware-accelerated timing synchronization and real-time transmission requirements needed for the O-RAN 7.2 specification. Breakthrough 5T for 5G technology handles disaggregated RAN deployments efficiently and economically and obviates the need for non-optimal, inefficient, and expensive FPGA NIC solutions. When combined with cuVNF and cuBBU from the NVIDIA Aerial SDK running on the NVIDIA EGX GPU platform, the combined solution offers a powerful, real-time CloudRAN solution.
The 5T for 5G technology is planned for general availability in Mellanox OFED Driver version 5.2 and later, scheduled for Q4 2020. It is also planned in the DPDK 20.11 LTS release, scheduled for November 2020.
To get started, use the following resources: