My sense is that the Boost developers have thought the most about the fundamental complexities of mapping arbitrary C++ types to their corresponding MPI_Datatype, and arbitrary C++ functions to their corresponding MPI_Op, in a way that serves the needs of generic programming. We’ve thought a lot about it too. I see that as the center of a good C++ interface to MPI.

   0 likes

As far as I can recall, there was a lot more discussion around this topic, and a lot more controversy at the Forum, too.

Extending the existing C++ bindings would be a mostly mechanical act. However, the atmosphere was such that an attempt to do this would likely be futile, and since there were much more interesting matters to ponder, there were no volunteers. In the end, the Forum got swayed by the arguments mentioned above.

I, for one, thought then, and I still think, that the deprecation and then removal of the C++ bindings without provision of a viable alternative was a wrong move. So, I consistently argued against this proposal.

Well, what’s been done is done. Now we are left with a situation in which both major open source implementations and their relatives, as well as, most likely, many related and independent commercial ones, are going to support the “removed” C++ bindings for the foreseeable future. The same may be true of the “removed” interfaces, by the way.

I cannot describe the current state by another word but limbo. On one hand, we have C++ bindings that are in the products and may still be used by someone (actually, anyone). On the other hand, deprecation in MPI-2.2 stopped any further development here, including maintenance and bug fixing. I.e., many products provide MPI-2.1 support as far as C++ is concerned, while supporting MPI-2.2 and beyond elsewhere.

How this is going to work out in the long run remains to be seen. The C++ interface may indeed die out. However, I will not wonder if someone steps in and provides complete and consistent C++ bindings for MPI-3 once, either based on the bindings that have been removed, or on boost, or on something else.

This would be a worthy quest for those looking at how to help MPI evolve. If inclusion into the standard proper is not considered possible, the example of the MPIR debugging interface, that was officially blessed by the Forum while not becoming a part of the standard, may serve as inspiration to those concerned.

This, by the way, is a path that may be taken by other MPI language bindings aspiring to a more formal status.

Best regards.

Alexander

   0 likes

I’m actually refactoring an existing wrapper. I would much rather contribute to a popular standard library, such as Boost.MPI, but it would take too long to rewrite everything to use a different interface, and backwards compatibility demands that I support the existing interface for a while at least.

My hope is that the documents I’ve been writing to guide the refactoring work might be useful to Boost.MPI developers or other groups working on a C++ interface to MPI. I would be glad to contribute directly, except that I already have a pile of work to do with the existing C++ interface and might not have time to do a good job with Boost.MPI work.

“BTW, what features of MPI-2 (or MPI-3) do you want to use in boost.mpi?”

One-sided communication, nonblocking collectives, and perhaps also the neighborhood collectives.

Thanks Jeff!

   0 likes

One historical note that I didn’t include in the blog post is that the Forum *did* look at standardizing a C++ class library (Object Oriented MPI — or OOMPI) back in the MPI-2 timeframe. OOMPI added a bunch of C++-specific semantics on top of the existing MPI bindings. After much debate, the Forum (rightly, IMHO) decided that it should only be in the business of standardizing one set of semantics paired with language-neutral bindings. Then basing language-specific bindings on top of those.

To be clear: the bindings that MPI provides effect *the language-neutral semantic specifications* as described in the MPI documents. This is what led the C++ bindings to be essentially a 1:1 mapping to the C (i.e., language-neutral) bindings. If bindings provide more/different semantics than what is expressed in the MPI specification, that is, by definition, a higher/different abstraction than what the MPI specification provides.

The Forum is all in favor of higher levels of abstraction above the base bindings, especially those that take advantage of language features (like generic programming). It’s just not in the Forum’s purview to provide such higher-level abstractions.

Make sense?

   0 likes

As you mentioned in a later comment, you’re working on a new C++ wrapper for your software library. Do you see yourself — or anyone else, for that matter — taking up the mantle of extending boost.mpi to support more than MPI 1.1?

BTW, what features of MPI-2 (or MPI-3) do you want to use in boost.mpi?

   0 likes

   0 likes

I think this is (or was) the core of the problem:
“The C++ bindings were (intentionally) a more-or-less 1:1 mapping to the C bindings.”

As a C++ developer in HPC area, this is one of the main reasons to choose Boost.MPI instead. In short, it doesn’t feature the above-mentioned impedance mismatch: C++ is fundamentally a very different language than C and attempting to maintain things 1:1 all the way essentially amounts to actively fighting the language — rarely a good idea.

At the same time, I do find the lack of the up-to-date features (like support limited to a subset of MPI 1.1 Mark has mentioned above) a growing annoyance.

In the future, if any direct (as opposed to going-through-C) support for C++ were to be resurrected, I’d gladly see it done the (modern) C++ way, i.e., more along the lines of Boost.MPI (and not necessarily limited to OOP — generic programming (GP) and automatic type-inference features present in modern C++ are important enabling factors of its relatively higher expressivitiy and ease of use).

   0 likes

Another place where a C++ interface could add value would be for recovering from process failure via the normal exception-handling mechanisms (rather than a nonidiomatic callback mechanism). This would only help, though, if future versions of the MPI standard better define the state of the world after a process dies.

Anyway, great summary of the issues!

   2 likes