Cisco showcases its 100 gigabit technology at SC11, the international conference for high performance computing, networking, storage, and analysis with the National Oceanic and Atmospheric Administration and National Lambda Rail.
As usual, I’m exhausted — in a good way — at the end of an SC week. Whew!
Thanks to all who came to see my demo (showing 5.17us NetPIPE MPI latency over Ethernet via Linux VFIO and Cisco’s “Palo” NIC — no, that’s not iWARP and it’s not IBoIP a.k.a. RoCE — see my prior post for a little more info), and thanks to all who came to the Open MPI BOF. I counted about 100 people at the BOF. The BOF slides are available, if you missed the actual event.
Brock and I did a [probably incredibly embarrassing] short video spot with Rich Brueckner at the end of the show (another in the RCE-Cast <--> InsightHPC crossover series). The convention announcer guy was literally saying “The show is over; please leave” over the PA while we were recording. Whenever Rich gets to posting the video, I think you’ll see why I usually stick to writing. :-)
Linux VFIO (Virtual Function IO) is an emerging technology that allows direct access to PCI devices from userspace. Although primarily designed as a hypervisor-bypass technology for virtualization uses, it can also be used in an HPC context.
Think of it this way: hypervisor bypass is somewhat similar to operating system (OS) bypass. And OS bypass is a characteristic sought in many HPC low-latency networks these days.
Drop by the Cisco SC’11 booth (#1317) where we’ll be showing a technology preview demo of Open MPI utilizing Linux VFIO over the Cisco “Palo” family of first-generation hardware virtualized NICs (specifically, the P81E PCI form factor). VIFO + hardware virtualized NICs allow benefits such as:
Low HRT ping-pong latencies over Ethernet via direct access to L2 from userspace (4.88us)
Hardware steerage of inbound and outbound traffic to individual MPI processes
Let’s dive into these technologies a bit and explain how they benefit MPI.