I’m a fan of InsideHPC; I read it every day. I like John’s commentary; he does a great job of rounding up various newsworthy HPC-related articles. But that doesn’t always mean that I agree with every posted item. Case in point: I saw this article the other day, purportedly a primer on InfiniBand (referring to this HPCprojects article). I actually know a bit about IB; I used to work in the IB group at Cisco. Indeed, I’ve written a lot of OpenFabrics verbs-based code for MPI implementations.
There’s good information in that article, but also some fantastically unfounded and misleading marketing quotes:
- ”With large data transfers, Ethernet consumes as much as 50 per cent of the CPU cycles; the average for InfiniBand is a loss of less than 10 to 20 per cent.” He’s referring to software TCP overhead, not Ethernet overhead. There’s an enormous difference — there’s plenty of Ethernet-based technologies that are in the 10-20% overhead range.
- “There are also power savings to be had, and this is critical when HPC facilities are confronting major issues with power supplies, cooling and costs. The same study indicates that InfiniBand cuts power costs considerably to finish the same number of Fluent jobs compared to Gigabit Ethernet; as cluster size increases, more power can be saved.” Wow. Other than generating warm fuzzies for customers (“My network products are green!”), what exactly does that paragraph mean? And how exactly was it quantified?
- …I’ll stop with just those 2.
These quotes are classic marketing spin to make IB products look the better than the competition.
Read More »
(this blog entry co-written by Brice Goglin and Samuel Thibault from the INRIA Runtime Team)
We’re pleased to announce a new open source software project: Hardware Locality (or “hwloc“, for short). The hwloc software discovers and maps the NUMA nodes, shared caches, and processor sockets, cores, and threads of Linux/Unix and Windows servers. The resulting topological information can be displayed graphically or conveyed programatically though a C language API. Applications (and middleware) that use this information can optimize their performance in a variety of ways, including tuning computational cores to fit cache sizes and utilizing data locality-aware algorithms.
hwloc actually represents the merger of two prior open source software projects:
- libtopology, a package for discovering and reporting the internal processor and cache topology in Unix and Windows servers.
- Portable Linux Processor Affinity (PLPA), a package for solving Linux topological processor binding compatibility issues
Read More »
Tags: HPC, mpi, NUMA, process affinity
From the home office in Helsinki, Finland: MPI-2.2 is done! It’s done it’s done it’s done!
Finally! The MPI-2.2 document has been voted in by the MPI Forum. The official PDF document will be published on www.mpi-forum.org soon. HLRS is selling (at cost) MPI-2.2 books; contact Rolf Rabenseifner if you’re interested (I’ll be getting one!).
Read More »
Everything old is new again — NUMA is back!
With NUMA going mainstream, high performance software — MPI applications and otherwise — might need to be re-tuned to maintain their current performance levels.
A less-acknowledged aspect of HPC systems is the multiple levels of networks that are traversed to get data from MPI process A to MPI process B. The heterogeneous, multi-level network is going to become more important (again) in your applications’ overall performance, especially as per-compute-server-core-counts increase.
That is, it’s not going to only be about the bandwidth and latency of your “Ethermyriband” network. It’s also going to be about the network (or networks!) inside each compute server.
A Cisco colleague of mine (hi Ted!) previously coined a term that is quite apropos for what HPC applications now need to target: it’s no longer just about NUMA — NUMA effects are only one of the networks involved.
Think bigger: the issue is really about Non-Uniform Network Access (NUNA). Read More »
Tags: HPC, mpi, NUMA, NUNA, process affinity
In a move that will surely cause some head-scratching, Platform has acquired the intellectual property of the-MPI-previously-known-as-HP-MPI.The head scratching part is that Platform already owns Scali MPI. It’s no secret that they recently moved all Scali development to an engineering team based in China. Read More »