Cisco Logo


High Performance Computing Networking

A common question from new MPI developers is: which should I use to separate my messages — tags or communicators?

If you didn’t already know MPI offers two key abstractions for message passing:

The reason that people ask is because both tags and communicators are arguments to MPI’s point-to-point functions.  For example:

Let’s start by defining both terms.

A communicator is an MPI term that means an ordered group of MPI processes with a unique communication context.

For example, when an MPI job is launched, all the processes in that job are put into a default communicator named MPI_COMM_WORLD (or “MCW”, for short).  Each process gets a unique integer rank identifier in that communicator in the range [0, N) (where N is the number of processes in the job).

Communicators can be duplicated, and in the duplication process, can be subsetted and re-ordered.  Hence, an individual MPI process can belong to any number of communicators, and may be a different rank in each of them.

A tag is simply an arbitrary integer that is used to delineate the matching of point-to-point messages.

Specifically, if a sender sends a message to MCW rank X on tag Y, then the receiver must match these specifications in order to receive the message.  That is, receiver must specify MCW as its communicator, the same Y value for the tag (or the special wildcard MPI_ANY_TAG), and the sender’s rank in MCW (or the special wildcard MPI_ANY_SOURCE).

Hence, if the (rank, tag, communicator) tuple matches between the sender and receiver, the message will be delivered (…according to MPI’s ordering rules, but that’s a topic for a different blog entry).

So it looks like either (or both) of the tag and communicator arguments can be used to delineate the receipt of individual messages.  When should you use each one?  Here’s the rule of thumb I use:

For example, if I have an MPI application that uses 3 sub-libraries, then I ensure that each of my 3 sub-libraries creates and uses their own communicator. This will prevent the messages from one sub-library accidentally matching messages from a different sub-library that it knows nothing about.

Within each of those sub-libraries, for most messaging, I simply use tags. For example, I’ll use one tag for sending command messages, and a different tag for sending data messages. Or maybe I’ll use multiple different tags for commands and different types of data (depending on the application needs).

In short: use communicators when you need a whole new/safe communication scope, or you need to change your existing scope (e.g., subset and/or reorder the member processes). Use tags for everything else.

Comments Are Closed

  1. Return to Countries/Regions
  2. Return to Home
  1. All High Performance Computing Networking
  2. All Security
  3. Return to Home