Cisco Blogs

Cisco Blog > High Performance Computing Networking

Latency Analogies

Multiple readers have told me that it is difficult for them to understand and/or visualize the effects of latency on their HPC applications, particularly in modern NUMA (non-uniform memory access) and NUNA (non-uniform network access) environments.

Let’s breaks down the different levels of latency in a typical modern server and network computing environments.

Read More »

Tags: , , ,

Process and memory affinity: why do you care?

I’ve written about NUMA effects and process affinity on this blog lots of times in the past.  It’s a complex topic that has a lot of real-world affects on your MPI and HPC applications.  If you’re not using processor and memory affinity, you’re likely experiencing performance degradation without even realizing it.

In short:

  1. If you’re not booting your Linux kernel in NUMA mode, you should be.
  2. If you’re not using processor affinity with your MPI/HPC applications, you should be.

Read More »

Tags: , , , ,

Process Affinity in OMPI v1.7 (part 2)

In my last post, I described the Simple mode of Open MPI v1.7’s process affinity system.

The Simple mode is actually quite flexible, and we anticipate that it will meet most users’ needs. However, some users will need more flexibility. That’s what the Expert mode is for.

Before jumping in to the Expert mode, though, let me describe two more features of the revamped v1.7 affinity system.

Read More »

Tags: , , , , ,

Process Affinity in OMPI v1.7 (part 1)

In my last post, I mentioned that we just finished a complete revamp of the Open MPI process affinity system, and provided only a few details as to what we actually did.

I did link to a SVN commit message, but I’ll wager that few readers — if anyone — actually read it.  :-)

Much of what is in the Open MPI v1.6.x series is the same as what Ralph Castain described in a prior blog post.  I’ll describe below what we changed for the v1.7 series.

Read More »

Tags: , , , ,

Taking MPI Process Affinity to the Next Level

Process affinity is a hot topic.  With commodity servers getting more and more complex internally (think: NUMA and NUNA), placing and binding individual MPI processes to specific processor, cache, and memory resources is becoming quite important in terms of delivered application performance.

MPI implementations have long offered options for laying out MPI processes across the resources allocated for the job.  Such options typically included round-robin schemes by core or by server node.  Additionally,  MPI processes can be bound to individual processor cores (and even sockets).

Today caps a long-standing effort between Josh Hursey, Terry Dontje, Ralph Castain, and myself (all developers in the Open MPI community) to revamp the processor affinity system in Open MPI.

The first implementation of the Location Aware Mapping Algorithm (LAMA) for process mapping, binding, and ordering has been committed to the Open MPI SVN trunk.  LAMA provides a whole new level of processor affinity control to the end user.

Read More »

Tags: , , , , , ,