Comments on: MPI_Ibarrier: Crazy?

By: Jeff Squyres

Jeff Squyres — Mon, 17 Sep 2012 16:44:55 +0000

Possibly. But that somewhat assumes a tightly-coupled MPI application.

What if the load is non-uniform across the MPI processes? Having such a “fuzzy” barrier would still allow other processes to do useful things while the slow process is still chunking away, trying to reach the milestone that everyone else has already reached.

Don’t get me wrong — I’m not saying that a fuzzy barrier is useful everywhere. Heck, barriers aren’t *needed* in many places (the class of MPI applications that *need* a blocking barrier for correctness is very small).

But fuzzy barriers are a different animal. If you think of them as milestone markers — especially in applications with lots of milestones, each of which have varying degrees of dependence on others (ranging from wholly dependent to fully independent) — then they can be a useful tool to extract more communication / computation overlap.

0 likes

By: Bibrak

Bibrak — Sun, 16 Sep 2012 09:46:04 +0000

The new feature seems a good utility but won’t a process which is slow (the 15 late guy) impact the overall parallelism of the application.

The parallelism of the entire application will then de dependant on that slow process making the entire application run slower.

0 likes

By: Squyres: Is the MPI-3 Non-Blocking Barriers a Crazy Idea? | insideHPC.com

Squyres: Is the MPI-3 Non-Blocking Barriers a Crazy Idea? | insideHPC.com — Sun, 16 Sep 2012 09:17:21 +0000

[...] the Full Story.AKPC_IDS += "31620,"; Posted in HPC, HPC Software by Rich Brueckner 0 [...]

0 likes

By: Geoffrey Irving

Geoffrey Irving — Sat, 15 Sep 2012 17:12:30 +0000

Yep, this routine comes in very handy in cases where the pattern of messages is unknown. I asked for it explicitly a while ago without knowing it existed, then had to implement one myself:

http://scicomp.stackexchange.com/questions/2876/nonblocking-version-of-mpi-barrier-in-mpi-2

0 likes

By: Jed Brown

Jed Brown — Sat, 15 Sep 2012 16:36:52 +0000

See Algorithm 2 of this paper for a practical and communication-optimal use of nonblocking barrier to set up sparse communication.
http://unixer.de/publications/img/hoefler-dsde-protocols.pdf

0 likes