What to do about the implicit Fortran MPI interfaces (i.e., mpif.h) in MPI-3? This is something that I’ve been thinking about a lot recently.
Sidenote: Some people refer to mpif.h as “the Fortran 77 MPI interfaces.” That isn’t quite correct; there’s actually stuff in mpif.h that didn’t exist until well beyond the Fortran 77 specification, such as KIND attributes and whatnot. So if someone calls mpif.h “the Fortran 77 MPI interfaces”, you have my permission to give them a slugbug punch. Ditto if they call an “MPI process” a “rank.”
As I’ve mentioned in prior entries, we’re going to have much-updated explicit Fortran interfaces in MPI-3 (the so-called “Fortran ’03 interfaces”, but just like “the Fortran 77 interfaces”, that name isn’t quite accurate, either). As I swear I heard Snoop Dog say once, “These new Fortran explicit MPI interfaces are da fa-schizzle”. They offer a bunch of language features that MPI ignored before, and also fix some long-standing problems — most importantly with regards to asynchronous buffer control.
So the question isn’t so much about what to do for the future; it’s more a question about what to do with the past. Should we deprecate the old, ancient, decrepit, harmful, stanky, nasty Fortran implicit interfaces? The answer is not quite as obvious as I would hope.
Let’s be clear here: I’m talking about deprecating the entire mpif.h interface. Deprecating has a very, very specific meaning:
- Indicating that mpif.h may be removed from the MPI standard someday (assumedly very, very far in the future).
- Not adding any new MPI-3 functionality.
Don’t freak out: I’m NOT saying that mpif.h should be removed from the forthcoming MPI-3 standard. I’m only talking about sending a strong signal that the MPI Forum recommends that explicit interfaces should be used in the future. Given the enormous number of codes that use mpif.h, removing them from the standard in the immediate- or near-term future would be idiotic.
So let’s talk about deprecating mpif.h.
On the one hand, the old implicit style interfaces are deader than a doornail. They reflect 30+ year old programming practices that are tremendously bad for developing and debugging code bases. For example, you don’t get any compiler warnings if you call a subroutine and forget a parameter, add an extra parameter, or pass a parameter of the wrong type. C programmers: try to imagine not getting a warning if you pass an double instead of an int. Yow!
On the other hand, there is a LOT of code out there that uses these implicit MPI interfaces. There are countless thousands of scientists and engineers out there who don’t give a darn about ensuring that subroutines are prototyped before they are used — they just want to get on with the science that they are doing. Accelerate the time to solution; that’s the most important thing to them. They’ll know if they put in the right count and types of parameters if their programs don’t seg fault.
But on the other hand, using implicit interfaces is just plain bad. They can lead to easily the worst kind of progamatic error: silent data corruption. That’s when your program is generating wrong results and you don’t even know it. Your program doesn’t seg fault or otherwise give obviously wrong data — perhaps it was just a subtle, small error that gets inserted somewhere in countless thousands or millions of lines of code. This error produces a result that is still “close enough” that you didn’t notice. Now imagine that that bad result gets used in 20 other computations. The source of the error is thus compounded and potentially magnified by its repeated use. All results derived from this original bad/incorrect answer are therefore tainted; even the final conclusion of the program is possibly incorrect. Ouch.
Simply put: explicit interfaces are a valuable tool to help correctness by detecting many types of programmer error. They don’t detect all types of errors, of course — but I shrudder to imagine writing any size code base without either some type of parameter checking or a strong data representation conversion system (that Fortran does not have). Without explicit interfaces, trying to discover the source of a problem can be that much harder. I’m reminded of the Tao of Programming here: “Though a program be but three lines long, someday it will have to be maintained.”
Sidenote: If you haven’t read the Tao, go do so now. It’s a 10 minute read that is chock full of pearls of wisdom. Order it today. It was required reading in my grad school research lab. Do not pass Go; do not collect $200. Find a friend who owns it and read it now. Order your own copy so that you can re-read it once a month for the rest of your career.
But on the other hand (yes, I realize I have 4 hands now — I’m a busy guy), will we just push all the scientists and engineers away if we remove the implicit interfaces? Even though it won’t happen for a long, long time, there may come a day when mpif.h actually disappears. Fortran programmers will have to make a choice: convert to MPI’s explicit interfaces or convert to something else. Given that both options require work (you can’t just replace “include ‘mpif.h'” with “use mpi3”), which will they choose? This may seem like a minor issue, but part of our charge as the stewards of the MPI Standard is to ensure that we serve our users well, both in terms of correctness of the specification and in terms of forward portability and usability. Pushing users away from us just so that we can have a “better” spec is not a good answer.
Let’s also not forget the 2nd part to the “deprecated” definition: not adding any MPI-3 functionality to mpif.h. The issue here is that there are oodles of existing large implicit-interface-using MPI applications out there. Such applications simply cannot be entirely re-written to use the new explicit interfaces without significant time, energy, and resources (meaning: money). We specifically designed the new explicit Fortran MPI interfaces to be able to used alongside mpif.h in an application. To be clear: you can have a legacy 10-million line implicit-Fortran-MPI-interface-using application and still add a new subroutine that uses the MPI-3 explicit interfaces. So I think we’re covered from this aspect, but then again, it does require some new/additional work. Application developers will need to learn about the MPI-3 explicit interfaces, for example. That’s a good thing, but it does take time/resources/money.
All this being said — there are pros and cons to deprecating mpif.h. The Right Thing to do is not entirely obvious to me. Today, I am leaning towards deprecating mpif.h (NOT removing it!). It will be work to convert existing Fortran applications, but there’s plenty of time (several years, at a minimum) and plenty of good reasons to do so.
Do you agree? Do you have a different opinion? Leave a comment below and let me know.