Cisco Blogs


Cisco Blog > High Performance Computing Networking

Hardware vs. software: user questions (part 2)

June 29, 2012 at 5:00 am PST

In my prior blog entry, I answered the first of Durga C.’s questions to me.  Here’s all three of his questions:

  1. What is the role of the hardware in an RDMA transaction?  In other words, why does one need special hardware (e.g., InfiniBand, iWARP, RoCE, etc.) hardware to do RDMA as opposed to a “normal” Ethernet NIC? (see prior blog entry)
  2. Further, can you explain why pure software solutions (e.g., Open-MX) are better than nothing when you don’t have hardware support?
  3. Also, what is the difference between “RDMA” and “RMA”?

Let’s explore the last two of those questions. Read More »

Tags: , , , , ,

Hardware vs. software: user questions

June 26, 2012 at 1:20 pm PST

Durga C., long-time listener, first-time caller, sent me a few interesting questions that I thought I’d share with everyone.  Here’s his first question:

  1. What is the role of the hardware in an RDMA transaction?  In other words, why does one need special hardware (e.g., InfiniBand, iWARP, RoCE, etc.) hardware to do RDMA as opposed to a “normal” Ethernet NIC?

This one question is surprisingly complex.  Let’s dive in…

Read More »

Tags: , , , , ,

Registered memory imbalances

June 23, 2012 at 4:36 am PST

In prior blog posts, I’ve talked about the implications of registered memory for both MPI applications and implementations.

Here’s another fun implication that was discovered within the last few months by Nathan Hjelm and Samuel Gutierrez out at Los Alamos National Labs: registered memory imbalances.

As an interesting side note: as far as we can tell, no other MPI implementation attempts to either balance registered memory between MPI processes, or handle the performance implications that occur with grossly imbalanced registered memory consumption.

Let’s review a few key points before defining what registered memory imbalances are.

Read More »

Tags: , , ,

Shared Receive Queues

October 25, 2011 at 5:00 am PST

In my last post, I talked about the so-called eager RDMA optimization, and its effects on resource consumption vs. latency optimization.

Let’s talk about another optimization: shared receive queues.

Shared receive queues are not a new idea, and certainly not exclusive to MPI implementations.  They’re a way for multiple senders to send to a single receiver while only consuming resources from a common pool.

Read More »

Tags: , , ,

MPI tradeoffs: space vs. time

October 22, 2011 at 7:42 am PST

@brockpalen asked me a question in Twitter:

@jsquyres [can you discuss] common #MPI implementation assumptions made for performance and/or resource constraints?

Good question.  MPI implementations are full of trade-offs between performance and resource consumption.  Let’s discuss a few easy ones.

Read More »

Tags: , ,