I’ve written about NUMA effects and process affinity on this blog lots of times in the past.  It’s a complex topic that has a lot of real-world affects on your MPI and HPC applications.  If you’re not using processor and memory affinity, you’re likely experiencing performance degradation without even realizing it.

In short:

1. If you’re not booting your Linux kernel in NUMA mode, you should be.
2. If you’re not using processor affinity with your MPI/HPC applications, you should be.

Read More »

Tags: HPC, hwloc, mpi, NUMA, process affinity

In my last post, I described the Simple mode of Open MPI v1.7′s process affinity system.

The Simple mode is actually quite flexible, and we anticipate that it will meet most users’ needs. However, some users will need more flexibility. That’s what the Expert mode is for.

Before jumping in to the Expert mode, though, let me describe two more features of the revamped v1.7 affinity system.

Read More »

Tags: HPC, hwloc, mpi, NUMA, Open MPI, process affinity

Process affinity is a hot topic.  With commodity servers getting more and more complex internally (think: NUMA and NUNA), placing and binding individual MPI processes to specific processor, cache, and memory resources is becoming quite important in terms of delivered application performance.

MPI implementations have long offered options for laying out MPI processes across the resources allocated for the job.  Such options typically included round-robin schemes by core or by server node.  Additionally,  MPI processes can be bound to individual processor cores (and even sockets).

Today caps a long-standing effort between Josh Hursey, Terry Dontje, Ralph Castain, and myself (all developers in the Open MPI community) to revamp the processor affinity system in Open MPI.

The first implementation of the Location Aware Mapping Algorithm (LAMA) for process mapping, binding, and ordering has been committed to the Open MPI SVN trunk.  LAMA provides a whole new level of processor affinity control to the end user.

Read More »

Tags: HPC, hwloc, mpi, NUMA, Open MPI, process affinity

Today we feature a deep-dive guest post from Ralph Castain, Senior Architecture in the Advanced R&D group at Greenplum, an EMC company.

Jeff is lazy this week, so he asked that I provide some notes on the process binding options available in the Open MPI (OMPI) v1.5 release series.

First, though, a caveat. The binding options in the v1.5 series are pretty much the same as in the prior v1.4 series. However, future releases (beginning with the v1.7 series) will have significantly different options providing a broader array of controls. I won’t address those here, but will do so in a later post.

Read More »

Tags: HPC, hwloc, mpi, NUMA, Open MPI, process affinity, processor affinity

In the vein of awesome software releases (ahem…), Hardware Locality (hwloc) v1.2.1 has been released.  As the “.1″ implies, this is a bug fix release of a bunch of little things that crept in the 1.2 series.  A full list of the news-worthy items can be found here.

But more awesome than that is the fact that Hwloc 1.3rc1 has also been released.  The Hwloc 1.3 series brings in some major new features.  The list of new features can be found below.

Read More »

Tags: HPC, hwloc, mpi, NUMA