mpicc != mpicc
In my last post, I talked about why MPI wrapper compilers are Good for you. The short version is that it is faaar easier to use a wrapper compiler than to force users to figure out what compiler and linker flags the MPI implementation needs — because sometimes they need a lot of flags.
Hence, MPI wrappers are Good for you. They can save you a lot of pain.
That being said, they can also hurt portability, as one user noted on the Open MPI user’s mailing list recently.
The user requested that Open MPI’s wrapper compiler support the same options as MPICH’s wrapper compiler. As I said in my prior blog entry, on the surface, this is a very reasonable request. After all, MPI is all about portability — so why not make the wrapper compilers be the same?
Unfortunately, there are several “like it or not, this is how the world works” kinds of reasons why wrapper compilers differ between MPI implementations. Here’s a few of them:
>> 1. Wrapper compilers are not standardized. Specifically: the MPI specification doesn’t mention wrapper compilers at all. Hence, an implementation can do whatever it wants in terms of a wrapper compiler.
Indeed, some MPI implementations have wrapper compilers, some do not. If memory serves, Cray MPI and IBM (AIX) MPI do not have wrapper compilers; you just add -lmpi (or something like it) when linking your MPI application, and all the Right magic occurs.
>> 2. If you have a wrapper compiler that understands its own command line options, you run the risk of accidentally intercepting command line options that were intended for the underlying compiler.
For example, in Open MPI, we chose a minimalist approach. Open MPI’s “mpicc” only understands a single command line option: –showme. All others are passed directly to the underlying compiler. In this way, we minimize our chances of intercepting a real compiler option.
Other MPI implementations, such as MPICH2, went a different way. They understand a handful of command line options (this is the exact portability point that the user was asking out).
>> 3. Code compiled by different C compilers can be compiled into a single executable and work just fine. Except when it doesn’t.
Specifically, most C compiler vendors work quite hard to make their object code play nicely with others. But compilers, just like any other piece of software, have bugs. And these bugs can cause incompatibilities when compiling, linking, or running with code generated by other compilers (the run-time incompatibilities can be both incredibly subtle and exceedingly agonizing to debug).
Hence, the advice offered by the Open MPI team has always been that if you want to support different compilers, you should install Open MPI multiple times — once compiled by each different compiler that you want to install.
Many HPC installations use the Environment Modules package to help users manage switching between different MPI installations.
>> 4. Even though C code (usually) works between different compiler suites, C++ and Fortran have no such guarantees. Their symbol-mangling schemes are usually quite different, making cross-linking impossible. In practical terms, this means you usually need to install separate MPI installations for each C++ and Fortran compiler that you want to support.
Of course, many C++ and Fortran compilers emulate each others’ symbol-mangling schemes when given the appropriate command line flags. But personally, I tend to distrust such things for the reasons noted in #3.
As noted above, these are annoying reasons. The user advocate side of me screams “Just make it work!!” But I’m not really sure how; there’s a lot of historical precedent and some technical issues outside the scope of MPI that would need to be solved first.