Ethernet dominates enterprise and scale-out workloads for good reasons; it’s easy to use and provides the right level of performance. When it comes to scale-up networks, however, Ethernet still faces competition from Infiniband.
Broadcom is aiming to help level up Ethernet’s scale-up capabilities with a new data center networking chip announced today called Tomahawk Ultra. The new chip follows the June 2025 debut of the Tomahawk 6 chip. Tomahawk Ultra is a subset of the Tomahawk family, optimized specifically for high performance computing (HPC) and AI scale-up applications. It shares many characteristics with Tomahawk 5 and 6, and it also offers unique features like ultra low latency, support for minimum size packets, and network efficiency.
While the Tomahawk 6 is focused on throughput with up to 102.4 Tbps of bandwidth, the Tomahawk Ultra is not. The Tomahawk Ultra provides up to 51.2 Tbps at full bandwidth. Where it excels is with ultra-low latency, coming in at 250 nanoseconds of switch latency, and with packet performance of up to 77 billion packets per second.
“Our goal here was to have something that could exceed InfiniBand with standard Ethernet, with all those characteristics, as far as all the metrics that are important for HPC,” Pete Del Vecchio, product manager for the Tomahawk switch family at Broadcom, told Network World.
Architecture optimized for minimum packet latency
The Tomahawk Ultra represents a fundamental departure from traditional Ethernet switch design priorities. Where conventional switches optimize for aggregate throughput and buffering capabilities, the Ultra prioritizes latency above all else, achieving sub-250 nanosecond switching latency while maintaining full 51.2 Tbps throughput even with minimum-sized 64-byte packets.
The minimum-sized packets is a particularly important area for HPC that hasn’t been fully addressed by Ethernet switches in the past. Del Vecchio explained that HPC applications use different API models, including MPI (Message Passing Interface) and SHEM (Symmetric Hierarchical Memory) which tend to use a lot of very small packets.
Broadcom
Support for minimum packet size allows streaming of those packets at full bandwidth. That capability is essential for efficient communication in scientific and computational workloads. It is particularly important for scale-up networks where GPU-to-switch-to-GPU communication happens in a single hop.
Lossless Ethernet gets an ‘Ultra’ boost
Another specific area of optimization for the Tomahawk Ultra is with lossless Ethernet. Broadcom has integrated support for a pair of capabilities that were first fully defined in the Ultra Ethernet Consortium’s (UEC) 1.0 specification in June.
The lossless Ethernet support is enabled via:
- Link Layer Retry (LLR): With this approach, the chip automatically detects transmission errors using Forward Error Correction (FEC) and requests retransmission. Del Vecchio explained that when errors exceed FEC capabilities, with LLR at the link layer, the switch can now request a retry of that packet and it gets retransmitted.
- Credit-Based Flow Control (CBFC): CBFC prevents packet drops due to buffer overflow. If the receiver doesn’t have any space to receive a packet, the switch will send a pause signal to the sender, Del Vecchio said. Then once there’s space available, it will send a notification that a certain number of packets can be sent.
In-network collectives (INC) reduce network operations
The Tomahawk Ultra also helps to accelerate the overall speed of HPC and AI operations through something known as in-network collectives (INC).
In-network collectives are operations where multiple compute units like GPUs need to share and combine their computational results. For example, in an “all reduce” operation, GPUs computing different parts of a problem need to average their results across the network. With Tomahawk Ultra, instead of GPUs sending data back and forth and performing computations separately, the switch itself has hardware that can reduce the number of operations. The INC capability can receive data from all GPUs, perform computational operations like averaging directly in the network and then propagate the final result back to all GPUs.
The benefits are twofold. “You’ve offloaded some computation to the network,” Del Vecchio explained. “More importantly, you’ve significantly reduced the bandwidth the data transfers in the network.”
Scale-Up Ethernet (SUE) specification competes against NV-link
Going a step further, the Tomahawk Ultra will also support Broadcom’s Scale-Up Ethernet (SUE) specification to optimize the entire application-to-application path. Broadcom contributed the full SUE specification to the Open Compute Project (OCP) in May in a bid to make it an open effort.
Scale-Up Ethernet is a framework Broadcom developed to help xPU (GPU/accelerator) manufacturers optimize their networking interface for high-performance, low-latency communication. It provides guidance for efficient Ethernet networking between processing units.
Del Vecchio noted that SUE supports optimized 10-byte headers vs. standard 46-byte headers and enables efficient data transfer with minimal overhead. The goal is to make Ethernet a viable, open alternative to proprietary interconnect technologies like Nvidia’s NVLink, with better performance and broader ecosystem support.
The SUE effort is also an alternative to the Ultra Accelerator Link (UALink) group that emerged in May.
“We don’t know what’s going to come out of the UALink. There’s no switches on the market, probably nothing in production for a couple years,” Delecchio noted. “So really, the main decision for today is to use Ethernet, or do you use NVLink.”
Source:: Network World