In my prior blog entry, I mentioned two blog articles of interest that I had read recently, and then proceeded to comment on one of them. I’ll finally finish up and comment on the other one: Blowing the Doors of HPC Speed-up Numbers, written by Head Monkey Douglas Eadline (disclaimer: I used to write the MPI Mechanic column and Doug would edit it before it hit the shelves in the now-defunct Cluster World magazine).
Here’s a great quote from Doug’s article:
I learned that an order of magnitude improvement beyond the leading edge was good, two orders of magnitude was very interesting and invited further analysis, but three orders of magnitude was often a bunch of crap.
I’ve mentioned time and again on this blog that your application performance is the only one that matters. Benchmarks are a good indicator, but they test specific metrics that may or may not be important to you. The most common one I cite here is latency: yes, there are some network interconnects out there that have absolutely fabulous (meaning: microsecond-ish) half-round-trip latency, but do your applications need that? Some do, some don’t.
I didn’t really have too much more to add to Doug’s article other than “+1 on what he said.”
Just remember: take all marketing claims with a grain of salt. Yes, the products being described probably are really cool / awesome / useful. But check them out with the apps that you care about. See how well that product awesomeness translates to your bottom line.