Cisco Blogs


Cisco Blog > High Performance Computing Networking

Unsung heros: MPI run time environments

Most people immediately think of short message latency, or perhaps large message bandwidth when thinking about MPI.

But have you ever thought about what your MPI implementation has to do before your application even calls MPI_INIT?

Hint: it’s pretty crazy complex, from an engineering perspective.

Think of it this way: operating systems natively provide a runtime system for individual processes.  You can launch, monitor, and terminate a process with that OS’s native tools.  But now think about extending all of those operating system services to gang-support N processes exactly the same way one process is managed.  And don’t forget that those N processes will be spread across M servers / operating system instances.

Read More »

Tags: ,

Traffic in parallel

In my last entry, I gave a vehicles-driving-in-a-city analogy for network traffic.

Let’s tie that analogy back to HPC and MPI.

Read More »

Tags: , ,

Still more traffic

I periodically write about network traffic, and how general / datacenter network traffic analysis is related to MPI / HPC.

In my last entry, I mentioned how network traffic has many characteristics in common with distributed computing. Routing decisions, for example, are made independently at each network switch.

Consider if you were looking down at a city from above. Look at all the cars driving around the city streets. It’s chaos: each car/truck/bus/etc. makes its own routing decisions. Each one is a different size. Each one potentially goes in a different direction. Each one continually merges and splits from other traffic.

Yet somehow it all works.

Read More »

Tags: , ,

Traffic (redux)

I’ve written about network traffic before (see this post and this post). It’s the subject of endless blog posts, help forums, and instructional guides across the internet.

In a High Performance Computing (HPC) context, there are some fascinating aspects about network traffic that are fairly different than other types of network traffic.
Read More »

Tags: , , ,

BigMPI: You can haz moar counts!

Grumpy cat hates small MPI countsJeff Hammond has recently started developing the BigMPI library.

BigMPI is intended to handle all the drudgery of sending and receiving large messages in MPI.

In Jeff’s own words:

[BigMPI is an] Interface to MPI for large messages, i.e. those where the count argument exceeds INT_MAX but is still less than SIZE_MAX. BigMPI is designed for the common case where one has a 64b address space and is unable to do MPI communication on more than 231 elements despite having sufficient memory to allocate such buffers.

Read More »

Tags: , ,