The rapid evolution of artificial intelligence is fundamentally changing how we architect data centers. As AI models grow more complex, the industry is shifting focus from individual server performance to the data center’s interconnected fabric. Two factors are driving this shift: expanding training clusters and inference workloads that now demand cluster-level performance.
For training, frontier models require large numbers of GPUs, and cluster sizes now exceed the capacity of a single data hall. Clusters span multiple data centers connected by wide-area networks, and the infrastructure must scale to support hundreds of thousands of GPUs across broad geographic regions.
Inference is also transforming the infrastructure. Frontier models, even at FP4 precision, now surpass the capacity of a single GPU. The push for faster token serving is increasing demand for larger inference clusters, matching the same coordinated, high-performance networking as training clusters.
Taken together, these changes make the network more than a connectivity layer. The network is becoming the system-level fabric that determines how much of the AI infrastructure can be used, how quickly jobs complete, and how predictably inference can be served.
The network is now the system
A few years ago, GPU compute power was the primary bottleneck for AI model training. As distributed training has scaled, that constraint has shifted decisively from compute to network — GPU communication now determines overall cluster efficiency. For instance, Meta’s production data shows that in large-scale Deep Neural Network training runs, network overhead accounts for up to 60% of total training iteration time — a share that increases with cluster size.
This is why we think about the next phase of AI networking as a continuum. Scale-up connects accelerators inside a server or rack, where proprietary technologies such as NVLink and emerging approaches such as UALink, have focused on extremely low latency and high bandwidth. Scale-out connects racks and pods into larger training clusters, where InfiniBand has historically been a common choice for high-performance fabrics. Scale-across connects clusters, storage, front-end networks, and data centers, where Ethernet is already the operational foundation.
At scale, for training and inference alike, the network matters as much as the compute itself. The question is no longer whether AI needs specialized networking behavior. It does. The real question is whether we deliver that behavior through a patchwork of proprietary fabrics, or through one common Ethernet foundation that can grow across the whole continuum.
Why Ethernet becomes the common foundation
Proprietary networking solutions have long dominated high-performance computing, but they introduce vendor lock-in and limit scalability across diverse hardware. InfiniBand still has a role in plenty of AI deployments, but the direction of the industry isn’t in question — Ethernet is becoming the predominant networking technology for AI infrastructure. Embracing Ethernet puts you on the right operating model from day one: open, interoperable, and built to scale across many domains.
Cisco is championing an “Ethernet-first” strategy for AI for three core reasons:
- Open Standards and Interoperability: Ethernet enables organizations to integrate components from multiple vendors. This flexibility is essential for future-proofing data centers as AI hardware evolves.
- Unmatched Scalability: InfiniBand’s proprietary fabric management struggles above ~tens of thousands of GPUs, requiring complex workarounds as clusters grow. Ethernet has no such ceiling — hyperscalers have already leveraged decades of mature switching architecture and standards-based tooling to validate Ethernet-based clusters at hundreds of thousands of GPUs across multiple data centers.
- Investment Protection — With a Learning Curve: Ethernet builds on familiar infrastructure — existing switching platforms, management tooling, and a broad engineering talent pool. That foundation matters. But AI fabric operations is not a straight extension of enterprise networking. RoCEv2 and RDMA introduce new failure modes; congestion management (PFC, ECN, buffer tuning) requires careful calibration to avoid GPU stalls; and telemetry at hundred-thousand-GPU scale demands purpose-built tooling. Skills transfer partially, not fully. The advantage over InfiniBand is a more open, composable operational model.
That operating model matters because no two AI environments look alike. Training wants ultra-low latency and predictable collective communication. Inference wants QoS that accounts for load, location, and cost. A multi-site deployment wants fault tolerance, tenant isolation, and deterministic telemetry stretched across a much bigger failure domain. Ethernet gives you one foundation that can flex to all those requirements — instead of stitching together a separate technology island for each one.
What Ethernet must deliver for AI
To earn its place as the common AI fabric, Ethernet must handle what makes AI traffic different. This traffic is synchronized, bursty, and expensive to stall. Fall behind on the network, and GPUs sit idle. Let congestion spread, and job completion times stretch out. Take too long to heal a failure, and large jobs lose efficiency.
First up: intelligent load balancing. AI fabrics must spread traffic across many paths without sacrificing single-flow performance, keeping pace with modern NIC bandwidth and putting the whole topology to work. Weighted adaptive routing, multipath transport, source-routed and path-aware forwarding — these all serve the same goal: react to hotspots fast, without introducing instability.
Second: congestion control and reliable delivery. That means fast congestion detection, precise notification, and recovery that doesn’t throw away useful work. Packet trimming, local link repair, selective retransmission, ordered and unordered retransmission, header optimization — none of these are standalone features. They’re all doing the same job: keeping AI traffic moving when the fabric is under pressure.
Third: isolation and service assurance. AI clusters increasingly run multiple tenants and multiple jobs side by side, and a fault or noisy neighbor in one must never degrade another’s performance. Delivering that guarantee without heavy per-job configuration — especially as workloads move off InfiniBand — is what separates a fabric that merely connects GPUs from one that can be trusted to run production AI at scale.
This is exactly where standards like UEC, ESUN, and Multipath Reliable Connection (MRC) earn their keep. They’re defining how Ethernet picks up the AI-specific behavior it needs — congestion control, multipath operation, reliable transport, path awareness, telemetry, interoperability — without giving up the openness that made Ethernet the right choice to begin with.
Ethernet plus P4 programmability: The multiplying factor
In AI, networking standards are evolving rapidly. New protocols such as UEC Transport and MRC are being developed to address challenges in AI and ML traffic, including congestion control, efficient use of fabric bandwidth, packet ordering, and telemetry.
New standards such as these often require capabilities in networking that can only be met in the new ASIC generation which is typically available eighteen months later at best.
Historically, this assumption made sense. ASICs are built to a fixed specification, and once set, changes are not possible. If a standard was not included in the original design, it cannot be supported by the chip.
AI is challenging this model.
AI workload requirements are evolving at an unprecedented pace. UEC and MRC are not minor updates; each introduces significant new capabilities required at the switching ASIC level. These changes are arriving faster than traditional silicon development cycles can support.
This presents a significant challenge for customers building infrastructure today. Delaying an AI buildout to wait for new hardware is not feasible. The cost of delay, including lost training runs, reduced competitiveness, and idle capital, is substantial.
Cisco’s Silicon One was designed to address this challenge.
Since Silicon One is programmable in P4: it is not limited to the initial set of applications envisioned when the ASIC was designed. P4 enables engineers and customers to define packet processing in software, separating network logic from physical hardware. When a new standard emerges, such as a revised UEC congestion response or new MRC capabilities, we can deliver these updates in software on existing hardware, often within weeks or months rather than waiting for the next product cycle.
That’s the multiplying factor. Standards set the direction for the ecosystem, but P4 programmability decides how fast customers see the benefit on real infrastructure. It also means customer-specific behavior — scheduler-aware policy, topology-specific routing, tenant isolation — doesn’t have to wait on a fixed-function silicon roadmap.
Where Cisco Silicon One fits in
Cisco Silicon One sits right at the intersection of high-performance Ethernet, emerging AI networking standards, and P4 programmability. That’s not a coincidence — AI networks need both performance and adaptability at once: performance to keep GPUs fed, adaptability to keep up with standards and customer requirements that are still very much in motion.
We have demonstrated this capability multiple times across real, production-relevant features:
- Packet Trimming: Rather than dropping packets outright during congestion events, packet trimming preserves the header while discarding the payload, allowing receivers to selectively request retransmission of only the missing data. This significantly reduces unnecessary full-flow retransmissions and improves throughput under load—delivered on existing Silicon One hardware through a P4 software update, with no silicon changes required.
- Full MRC Support: Multipath Reliable Connection introduces a comprehensive suite of load balancing and congestion control mechanisms purpose-built for AI and ML traffic patterns. Because Silicon One is P4-programmable, we were able to implement the complete MRC capability set—including its multipath load balancing and congestion response algorithms—without waiting for a new ASIC generation.
- Weighted Adaptive Routing: AI workloads generate highly bursty, asymmetric traffic that can rapidly create hotspots across a fabric. Weighted Adaptive Routing dynamically distributes flows across available paths based on real-time congestion metrics, assigning weights to steer traffic away from congested links and maximize fabric utilization. Delivering this capability on existing hardware requires only a P4 software update.
- Multi-tenant and Multi-job Isolation: Most of the AI clusters, with the exception of foundational model training, support multiple tenants and multiple jobs within each tenant. Enforcing tenant- and job-level isolation policies to prevent cross-communication is a critical service that the network operator must provide. As customers migrate from InfiniBand to Ethernet, supporting an efficient solution that minimizes configuration and network churn whenever a tenant and a job are scheduled onto the cluster becomes a key differentiator.
MRC is a good illustration of why Cisco’s SRv6 investment pays off here. Its switch-side requirements — SRv6 uSID forwarding, packet trimming, deterministic path-pinned telemetry — line up with capabilities we’ve already built through SRv6 and programmable Silicon One forwarding. And because that forwarding behavior is programmable, both these capabilities and customer-specific extensions can keep evolving hardware you’ve already deployed, as the spec matures.
This is not a theoretical advantage; it is the difference between telling a customer “we support that today” and “we’ll have silicon for that in 12 to 18 months.” In AI infrastructure, this distinction is critical.
The broader point is that programmability is essential. Given the rapid evolution of AI networking standards, it is the only viable architectural approach. Continuing to build inflexible ASICs to a fixed specification and relying on market stability is increasingly difficult to justify as new protocols are introduced.
The path forward
The future of AI depends not only on server silicon but also on the fabric connecting those servers. As we enter the era of large, multi-rack clusters, the industry needs a robust, flexible networking foundation.
That foundation comes down to a single, open building block — Ethernet — flexible enough to address three distinct scaling challenges at once:
- Scale-up, ultra-optimized: within the rack, Ethernet must match the raw, low-latency performance of dedicated scale-up fabrics between GPUs.
- Scale-out, performant and reliable: across racks and pods, it must sustain full throughput and reliable delivery as training clusters scale out to tens of thousands of GPUs.
- Scale-across, fault-tolerant and QoS-aware: across data centers and geographies, it must preserve job isolation and predictable performance as thousands of GPUs training clusters — and increasingly, inference clusters — span the wide area network.
As Ethernet evolves, it solves for all three — without giving up the open, standards-based ecosystem that makes it the right long-term choice for AI infrastructure.
Cisco is committed to delivering this foundation. By prioritizing open standards, high-performance silicon, and intelligent automation, we ensure tomorrow’s infrastructure can support today’s breakthroughs.
To be clear, this isn’t Ethernet instead of innovation. It’s Ethernet as the open foundation innovation builds on — multiplied by P4 programmability and delivered in platforms like Cisco Silicon One — so AI networks can evolve just as fast as the workloads riding on them.
Learn more about MRC and SRv6.
Additional resources:
