Cisco Blogs


Cisco Blog > High Performance Computing Networking

The MPI C++ bindings: what happened, and why?

October 16, 2012
at 5:00 am PST

Jeff Hammond at Argonne tells me that there’s some confusion in the user community about MPI and C++.

Let me see if I can clear up some of the issues.

In this blog entry, I’ll describe what has happened to the C++ bindings over time (up to and including their removal in MPI-3), and why.  In a second blog entry, I’ll describe what this means to real-world C++ MPI applications.

Why were the C++ bindings deprecated in the first place?  For several reasons:

  1. It was felt that the Forum didn’t have enough C++ expertise to continue to maintain the C++ bindings.  For example, many errata had crept in regarding the C++ bindings since 1996.  Some of those errata were even (initially) fixed incorrectly due to subtle issues in the C++ language!
  2. Many C++ MPI developers told the Forum that the C++ bindings really didn’t offer anything useful to them over the C bindings.  Indeed, one of the most well-known MPI C++ class libraries — boost.mpi — is implemented on the MPI C bindings, not the MPI C++ bindings.  This invalidated one of the major design goals for the MPI C++ bindings: providing basic MPI C++ language building blocks to enable the building of higher-level class libraries.  In 15+ years since the MPI C++ bindings were added in MPI-2.0, this goal has failed to be realized.
  3. After multiple surveys of the user community, the Forum could only find a handful of developers who are actually using the C++ bindings in real-world applications (i.e., homework assignments don’t count).
  4. The C++ bindings were (intentionally) a more-or-less 1:1 mapping to the C bindings.  Hence, C++ MPI applications can convert to use the MPI C bindings with little or no loss of functionality (albeit at the cost of person-hours to convert their codes).

Hence, it seemed like a lot of Forum effort was being expended to keep alive a feature that very few people were using.  It therefore got deprecated in MPI-2.2.

Sidenote: Keeping the C++ bindings alive is different than keeping <insert your favorite MPI feature that no one uses> alive.  The former requires active work by the Forum (e.g., adding new C++ functions, constants, and classes in MPI-3), whereas the latter is a passive action.

Note that “deprecated” means two things:

  1. The Forum signaled its intent that it might remove the C++ bindings someday.
  2. No new work would be done to maintain or extend the existing C++ bindings.

Because of the 2nd point, many new MPI-3 functions were added without corresponding C++ bindings.  When we came close to finishing MPI-3, the Forum was then left in an awkward position: should we leave the now-incomplete C++ bindings as deprecated (but still technically in the standard), or should we just remove them altogether?

More specifically, the choice boiled down to:

  1. Add C++ bindings for all new MPI-3 functions so that there would be a full, consistent set of C++ bindings, or
  2. Remove the C++ bindings entirely

Ultimately, choice #2 was adopted: the C++ bindings were removed.

It didn’t help that no one stepped forward to design, prototype, and implement C++ bindings for all the new MPI-3 functions.  While I would not say that this was not the determining factor, it certainly was one of several nails in the C++ bindings’ coffin.

Tags: , ,

In an effort to keep conversations fresh, Cisco Blogs closes comments after 60 days. Please visit the Cisco Blogs hub page for the latest content.

8 Comments.


  1. Thanks for the concise clarification! It’s been great to discuss C++ issues with you. I do think the biggest value a C++ interface to MPI could add would be better support for generic programming. Boost.MPI has done a good job with that, though it only supports a subset of MPI 1.1 and people have complained that it doesn’t use native MPI datatypes enough. I agree that handling all possible cases requires C++ expertise well beyond the scope of the MPI Forum.

    Another place where a C++ interface could add value would be for recovering from process failure via the normal exception-handling mechanisms (rather than a nonidiomatic callback mechanism). This would only help, though, if future versions of the MPI standard better define the state of the world after a process dies.

    Anyway, great summary of the issues!

       2 likes

    • October 16, 2012 at 5:31 pm

      Mark –

      As you mentioned in a later comment, you’re working on a new C++ wrapper for your software library. Do you see yourself — or anyone else, for that matter — taking up the mantle of extending boost.mpi to support more than MPI 1.1?

      BTW, what features of MPI-2 (or MPI-3) do you want to use in boost.mpi?

         0 likes

  2. Thanks for the summary!

    I think this is (or was) the core of the problem:
    “The C++ bindings were (intentionally) a more-or-less 1:1 mapping to the C bindings.”

    As a C++ developer in HPC area, this is one of the main reasons to choose Boost.MPI instead. In short, it doesn’t feature the above-mentioned impedance mismatch: C++ is fundamentally a very different language than C and attempting to maintain things 1:1 all the way essentially amounts to actively fighting the language — rarely a good idea.

    At the same time, I do find the lack of the up-to-date features (like support limited to a subset of MPI 1.1 Mark has mentioned above) a growing annoyance.

    In the future, if any direct (as opposed to going-through-C) support for C++ were to be resurrected, I’d gladly see it done the (modern) C++ way, i.e., more along the lines of Boost.MPI (and not necessarily limited to OOP — generic programming (GP) and automatic type-inference features present in modern C++ are important enabling factors of its relatively higher expressivitiy and ease of use).

       0 likes

    • Hi Matt — I’d love to chat more, as I’ve been working on a design document for refactoring the C++ wrapper for MPI in our big software library. It would be great to learn more about your experiences working with Boost.MPI too.

         0 likes

    • October 16, 2012 at 5:56 pm

      There has been *some* talk of using more modern languages in the MPI Forum, where things like auto-type deduction can occur (and therefore the reliance on MPI datatypes could be decreased), but not much.

      One historical note that I didn’t include in the blog post is that the Forum *did* look at standardizing a C++ class library (Object Oriented MPI — or OOMPI) back in the MPI-2 timeframe. OOMPI added a bunch of C++-specific semantics on top of the existing MPI bindings. After much debate, the Forum (rightly, IMHO) decided that it should only be in the business of standardizing one set of semantics paired with language-neutral bindings. Then basing language-specific bindings on top of those.

      To be clear: the bindings that MPI provides effect *the language-neutral semantic specifications* as described in the MPI documents. This is what led the C++ bindings to be essentially a 1:1 mapping to the C (i.e., language-neutral) bindings. If bindings provide more/different semantics than what is expressed in the MPI specification, that is, by definition, a higher/different abstraction than what the MPI specification provides.

      The Forum is all in favor of higher levels of abstraction above the base bindings, especially those that take advantage of language features (like generic programming). It’s just not in the Forum’s purview to provide such higher-level abstractions.

      Make sense?

         0 likes

  3. Jeff Squyres asked: “As you mentioned in a later comment, you’re working on a new C++ wrapper for your software library. Do you see yourself — or anyone else, for that matter — taking up the mantle of extending boost.mpi to support more than MPI 1.1?”

    I’m actually refactoring an existing wrapper. I would much rather contribute to a popular standard library, such as Boost.MPI, but it would take too long to rewrite everything to use a different interface, and backwards compatibility demands that I support the existing interface for a while at least.

    My hope is that the documents I’ve been writing to guide the refactoring work might be useful to Boost.MPI developers or other groups working on a C++ interface to MPI. I would be glad to contribute directly, except that I already have a pile of work to do with the existing C++ interface and might not have time to do a good job with Boost.MPI work.

    “BTW, what features of MPI-2 (or MPI-3) do you want to use in boost.mpi?”

    One-sided communication, nonblocking collectives, and perhaps also the neighborhood collectives.

    Thanks Jeff!

       0 likes

  4. Hi,

    As far as I can recall, there was a lot more discussion around this topic, and a lot more controversy at the Forum, too.

    Extending the existing C++ bindings would be a mostly mechanical act. However, the atmosphere was such that an attempt to do this would likely be futile, and since there were much more interesting matters to ponder, there were no volunteers. In the end, the Forum got swayed by the arguments mentioned above.

    I, for one, thought then, and I still think, that the deprecation and then removal of the C++ bindings without provision of a viable alternative was a wrong move. So, I consistently argued against this proposal.

    Well, what’s been done is done. Now we are left with a situation in which both major open source implementations and their relatives, as well as, most likely, many related and independent commercial ones, are going to support the “removed” C++ bindings for the foreseeable future. The same may be true of the “removed” interfaces, by the way.

    I cannot describe the current state by another word but limbo. On one hand, we have C++ bindings that are in the products and may still be used by someone (actually, anyone). On the other hand, deprecation in MPI-2.2 stopped any further development here, including maintenance and bug fixing. I.e., many products provide MPI-2.1 support as far as C++ is concerned, while supporting MPI-2.2 and beyond elsewhere.

    How this is going to work out in the long run remains to be seen. The C++ interface may indeed die out. However, I will not wonder if someone steps in and provides complete and consistent C++ bindings for MPI-3 once, either based on the bindings that have been removed, or on boost, or on something else.

    This would be a worthy quest for those looking at how to help MPI evolve. If inclusion into the standard proper is not considered possible, the example of the MPIR debugging interface, that was officially blessed by the Forum while not becoming a part of the standard, may serve as inspiration to those concerned.

    This, by the way, is a path that may be taken by other MPI language bindings aspiring to a more formal status.

    Best regards.

    Alexander

       0 likes

    • Hi Alexander –

      My sense is that the Boost developers have thought the most about the fundamental complexities of mapping arbitrary C++ types to their corresponding MPI_Datatype, and arbitrary C++ functions to their corresponding MPI_Op, in a way that serves the needs of generic programming. We’ve thought a lot about it too. I see that as the center of a good C++ interface to MPI.

         0 likes