Cisco Blogs

Cisco Blog > High Performance Computing Networking

Short message latency and NUMA effects

July 23, 2013 at 5:00 am PST

I’ve previously written a bunch about the effects of location, Location, LOCATION! on MPI applications.

Here’s another subtle NUMA effect that a well-tuned MPI implementation can hide from you: intelligently distributing traffic between multiple network interfaces.

Yeah, yeah, most MPI implementations have had so-called “multi-rail” support for a long time (i.e., using multiple network interfaces for MPI traffic).  But there’s more to it than that.

Read More »

Tags: , , , , ,

How many network links do you have for MPI traffic?

July 19, 2013 at 5:00 am PST

If you’re a bargain basement HPC user, you might well scoff at the idea of having more than one network interface for your MPI traffic.

“I’ve got (insert your favorite high bandwidth network name here)! That’s plenty to serve all my cores! Why would I need more than that?”

I can think of (at least) three reasons off the top of my head.

Read More »

Tags: , , , ,

Ultra low latency Ethernet (UCS “usNIC”): questions and answers

July 17, 2013 at 5:00 am PST

I have previously written a few details about our upcoming ultra low latency solution for High Performance Computing (HPC).  Since my last blog post, a few of you sent me emails asking for more technical details about it.

So let’s just put it all out there.

Read More »

Tags: , , , , ,

Open MPI and the MPI-3 MPI_T interface

July 13, 2013 at 5:00 am PST

Open MPI recently revamped its entire run-time parameter system (a.k.a., “MCA parameter system”) as part of its implementation effort for the “MPI_T” interface from MPI-3.

The MPI_T interface is a standardized interface designed for MPI tools, but can be used by regular MPI application programs, too.

Specifically, MPI_T provides programatic access to two types of MPI implementation data:

  1. Control variables: used to control the behavior of an MPI implementation
  2. Performance variables: provide access to internal MPI implementation performance metrics

Read More »

Tags: , , ,

Why MPI is Good for You (part 3)

June 24, 2013 at 1:05 pm PST

I’ve previously posted on “Why MPI is Good for You” (blog tag: why-mpi-is-good-for-you).  The short version is that it hides the typical application programmer from lots and lots of underlying network stuff; stuff that they really, really don’t want to be involved in.

Here’s another case study…

Cisco’s upcoming ultra-low latency MPI transport is implemented over an “unreliable” transport: raw Ethernet L2 frames. For latency reasons, it’s using the OpenFabrics verbs operating-system bypass API. These two facts mean that a) userspace is directly talking to the NIC hardware, and b) we don’t have a driver thread running down in the kernel that can service incoming frames regardless of what the MPI application is doing.

Read More »

Tags: , , , , ,