Cisco Blogs

Cisco Blog > High Performance Computing Networking

MPI spice

Hello programmers.  Look at your code.  Now look at MPI.  Now back at your code.  Now back to MPI

Sadly, your code isn’t parallel.  But if it stopped using global variables, it could act like MPI. 

Look down.  Back up.  Where are you?  On a 64-node, 32-core parallel computation cluster, with the code your code could act like. 

What’s in your hand?  Back at me.  I have it.  It’s an iPhone with an app for that thing you love. 

Look again.  The app is now a fully-parallelized, highly-scalable MPI code.

Anything is possible when your code acts like MPI and not like Cobol.

I’m on a horse.

Read More »

Probabiliy of correctness

Pop quiz, hotshot: what happens if you run this program with 32 processes on your favorite parallel resource?  (copy-n-pasting this code to compile and run it yourself is CHEATING!)

  int buf, rank = MPI::COMM_WORLD.Get_rank();
  if (0 == rank) {
    for (int i = 1; i < MPI::COMM_WORLD.Get_size(); ++i) {
      MPI_Status status;
      MPI_Recv(&buf, 1, MPI_INT, MPI_ANY_SOURCE, 123,
               MPI_COMM_WORLD, &status);
      buf = i * 2;
      MPI_Send(&buf, 1, MPI_INT, status.MPI_SOURCE, 123,
  } else {
    MPI_Send(&rank, 1, MPI_INT, 0, 123, MPI_COMM_WORLD);
    MPI_Recv(&buf, 1, MPI::INT, MPI_ANY_SOURCE, 123,

The mix of C and C++ is for brevity here on the blog.  But yea; it does compile and is a valid MPI application.

Got your answer?

Read More »

Seen in the blogosphere (part deux)

In my prior blog entry, I mentioned two blog articles of interest that I had read recently, and then proceeded to comment on one of them.  I’ll finally finish up and comment on the other one: Blowing the Doors of HPC Speed-up Numbers, written by Head Monkey Douglas Eadline (disclaimer: I used to write the MPI Mechanic column and Doug would edit it before it hit the shelves in the now-defunct Cluster World magazine).

Read More »

Seen in the blogosphere…

I actively read a bunch of HPC-related news feeds and blogs out there in the interwebs (how did I live before my RSS reader?).  Two articles recently caught my attention:

Don’t get me wrong; I read lots of fine articles and blog entries.  But these two seemed to resonate fairly well with some of the messages I’ve tried to convey here in my own blog.

Read More »

Frequent flyer

A colleague recently told me that he took two round trip flights from California to Florida for the sole purpose of nudging him into the next frequent flyer status level (over 100,000 miles in a year, in his case).  “At the time, I questioned the wisdom waking up early enough to get on a plane at 6am on a Saturday morning,” he said.  “But I’ve been automatically upgraded to first class ever since.  So — good decision.”  Even more surprising is that, by random chance, he found another guy who took both of the same flights for exactly the same reason.

Parallel programming is kinda like that.  If you’re just venturing into a concurrent programming world, it seems weird, awkward, and possibly even counter-intuitive.  But once you learn how to do it right, the benefits are continual.  And you’ll find that others are doing the same thing.

Read More »