We released Open MPI v1.5.4 last week.  Woo hoo!

I can’t exactly predict the future, but I anticipate having one more release before transitioning it to 1.6 (i.e., transitioning it from a “feature” release series to a “stable” release series where only bug fixes will be applied).

The v1.5 series is actually progressing quite nicely towards v1.6.  It has gotten a lot of run time on real-world machines in production environments, and many bugs have shaken out.  And there are many new, shiny toys on our development trunk that are slated for v1.7 (i.e., they won’t go into the v1.5/v1.6 series).

Read More »

Tags: HPC, hwloc, mpi, Open MPI

Brice, Samuel, and I got the crazy idea to write a magazine article about hwloc to expand its reach to people outside the HPC community. We wrote something up and submitted it to Linux Pro Magazine — and they accepted it!

I just got my copy in the mail — it’s published in the July issue: “Lessons in Locality: hwloc.”

Read More »

Tags: hwloc, look-ma-i'm-published

Google today announced its Summer of Code 2011 project winners.   One of the winners was a project proposed by George Andreou based off this idea on the TCL wiki: create some kind of “native” hwloc binding for TCL.

Congratulations, George!  A (brief) abstract of George’s winning project can be found here.

There’s more details involved that what is included in that abstract, of course, but I’m excited to see hwloc continue to spread and become genuinely useful to an ever-growing community. Read More »

Tags: HPC, hwloc, TCL

Here’s some MPI quick-bites for this week:

• The MPI_MPROBE proposal was voted into MPI-3 a few weeks ago.  Yay! (see this quick slideshow for an explanation of what MPI_MPROBE is)
• The Hardware Locality project just released hwloc v1.2.  This new version now includes distance metrics between objects in the topology tree.  W00t!
• Support for large counts looks good for getting passed into MPI-3; it’s up for its first formal reading at the upcoming Forum meeting.
• The same is true for the new MPI-3 one-sided stuff; it, too, is up for its first formal reading at the upcoming Forum meeting (they haven’t sent around their new PDF yet, but they will within a week or so — stay tuned here for updates).
• Likewise, the new Fortran-08 bindings are up for their first Forum presentation next meeting.  We solved all of the outstanding Fortran issues with the F77 and F90 bindings… with the possible exception of non-blocking communication code movement.  :-(  That one is still being debated with the Fortran language standardization body — it’s a complicated issue!
• Finally — the new MPI tools interface chapter is up for a first formal reading, too.

WHEW!

That’s a lot of first formal readings in one meeting…

Tags: HPC, hwloc, mpi, MPI Forum, MPI-3

There was a great comment chain on my prior post (“Unexpected Linux Memory Migration“) which brought out a number of good points.  Let me clarify a few things from my post:

• My comments were definitely about HPC types of applications, which are admittedly a small subset of applications that run on Linux.  It is probably a fair statement to say that the OS’s treatment of memory affinity will be just fine for most (non-HPC) applications.
• Note, however, that Microsoft Windows and Solaris do retain memory affinity information when pages are swapped out.  When the pages are swapped back in, if they were bound to a specific locality before swapping, they are restored to that same locality.  This is why I was a bit surprised by Linux’s behavior.
• More specifically, Microsoft Windows and Solaris seem to treat memory locality as a binding decision — Linux treats it as a hint.
• Many (most?) HPC applications are designed not to cause paging.  However, at least some do.  A side point of this blog is that HPC is becoming commoditized — not everyone is out at the bleeding edge (meaning: some people willingly violate the “do not page” HPC mantra and are willing to give up a little performance in exchange for the other benefits that swapping provides).

To be clear, Open MPI has a few cases where it has very specific memory affinity needs that almost certainly fall outside the realm of just about all OS’s default memory placement schemes.  My point is that other applications may also have similar requirements, particularly as core counts are going up, and therefore communication between threads / processes on different cores will become more common.

Read More »

Tags: HPC, hwloc, Linux, mpi, NUMA, process affinity