Traffic. I find myself still thinking about my last entry today as I’m riding the blue line CTA from O’Hare airport to downtown Chicago for the MPI Forum meeting this afternoon. Here I am, being spirited downtown at a steady clip on a commuter train while I see thousands of gridlocked cars on one side of me, and easily flowing motor vehicles on the other. I will definitely reach downtown before the majority of vehicles that are only a few feet away from me on the Kennedy expressway, despite the fact that I’m quite sure that I left O’Hare long after they did.
Traffic is such a great network metaphor that is gives insight into today’s ramble: it’s well-understood that network packets may be delivered in a different order than which they were sent. What’s less understood is why.
Read More »
Tags: HPC, mpi, traffic
Traffic. It’s a funny thing. On my daily drive to work, I see (what appear to be) oddities and contradictions frequently. For example, although the lanes on my side of the highway are running fast and clear, the other side is all jammed up. But a half mile later, the other side is running fast and clear, and my lanes have been reduced to half-speed. A short distance further, I’m zipping along again at 55mph (ahem).
Sometimes the reasons behind traffic congestion are obvious. For example, when you drive through a busy interchange, it’s easy to understand how lots of vehicles entering and exiting the roadway can force you to slow down. But sometimes the traffic flow issues are quite subtle; congestion may be caused by a non-obvious confluence of second- and third-order effects.
The parallels from highway traffic to networking are quite obvious, but the analogy can go much deeper when you consider that modern computational clusters span multiple different networks — we’re entering an era of Non-Uniform Network Architectures (NUNAs).
Read More »
Tags: HPC, mpi, NUMA, NUNA, process affinity, traffic
If ever I doubted that MPI was good for the world, I think that all I would need to do is remind myself of this commit that I made into the Open MPI source code repository today. It was a single-character change — changing a 0 to a 1. But the commit log message was Tolstoyian in length:
- 87 lines of text
- 736 words
- 4225 characters
Go ahead — read the commit message. I double-dog dare you.
That tome of a commit message both represents several months of on-and-off work on a single bug, and details the hard-won knowledge that was required to understand why changing a 0 to a 1 fixed a bug.
Read More »
Tags: HPC, mpi, why-mpi-is-good-for-you
I just ran across a great blog entry about SGE debuting topology-aware scheduling. Dan Templeton does a great job of describing the need for processor topology-aware job scheduling within a server. Many MPI jobs fit exactly within his description of applications that have “serious resource needs” — they typically require lots of CPU and/or network (or other I/O). Hence, scheduling an MPI job intelligently across not only the network, but also across the network and resources inside the server, is pretty darn important. It’s all about location, location, location!
Particularly as core counts in individual server are going up.
Particularly as networks get more complicated inside individual servers.
Particularly if heterogeneous computing inside a single server becomes popular.
Particularly as resources are now pretty much guaranteed to be non-uniform within an individual server.
These are exactly the reasons that, even though I’m a network middleware developer, I spend time with server-specific projects like hwloc — you really have to take a holistic approach in order to maximize performance.
Read More »
Tags: HPC, hwloc, mpi, NUMA, NUNA
(this blog entry co-written by Brice Goglin and Samuel Thibault from the INRIA Runtime Team)
We’re pleased to announce a new open source software project: Hardware Locality (or “hwloc“, for short). The hwloc software discovers and maps the NUMA nodes, shared caches, and processor sockets, cores, and threads of Linux/Unix and Windows servers. The resulting topological information can be displayed graphically or conveyed programatically though a C language API. Applications (and middleware) that use this information can optimize their performance in a variety of ways, including tuning computational cores to fit cache sizes and utilizing data locality-aware algorithms.
hwloc actually represents the merger of two prior open source software projects:
- libtopology, a package for discovering and reporting the internal processor and cache topology in Unix and Windows servers.
- Portable Linux Processor Affinity (PLPA), a package for solving Linux topological processor binding compatibility issues
Read More »
Tags: HPC, mpi, NUMA, process affinity