Comments on: How many ways to send? http://blogs.cisco.com/performance/how-many-ways-to-send/ Sat, 25 May 2013 02:09:43 +0000 hourly 1 By: Jeff Squyres http://blogs.cisco.com/performance/how-many-ways-to-send/#comment-548877 Jeff Squyres Mon, 13 Feb 2012 22:19:40 +0000 http://blogs.cisco.com/?p=60815#comment-548877 Ah yes, these are more good reasons why buffered sends are evil. :-)

Down with bsend!

   0 likes

]]> By: Fab Tillier http://blogs.cisco.com/performance/how-many-ways-to-send/#comment-548872 Fab Tillier Mon, 13 Feb 2012 22:04:05 +0000 http://blogs.cisco.com/?p=60815#comment-548872 There are other issues outside of performance why buffered sends are evil. A buffered send can complete to the application immediately, having been successfully copied to the internal buffer space specified via MPI_BUFFER_ATTACH. If a matching receive is never posted at the peer, there is no way for the application to reclaim that used buffer space – there’s no way of cancelling the request since it has already completed locally. There is also no way for the application to detect when it is safe to call MPI_BUFFER_DETACH. The normal options for apps to track (and cancel) their non-blocking send requests don’t work when using MPI_BSEND. Note that I single out MPI_BSEND here because there are cancelation semantics defined for the non-blocking variants.

   0 likes

]]> By: Jeff Squyres http://blogs.cisco.com/performance/how-many-ways-to-send/#comment-548469 Jeff Squyres Sun, 12 Feb 2012 17:06:58 +0000 http://blogs.cisco.com/?p=60815#comment-548469 They are evil for the following reasons:

* Buffered sends *force* an extra copy of the outgoing message (i.e., a copy from your buffer to the interner MPI buffer).
* This copy not only takes up resources (memory), it also takes *time*.
* The internal memory was also allocated by the user, which may not be optimal for the network transport that will be used for the message transfer.
* Consider if you use a buffered send for a very large message. The MPI must first copy that message before it can send it.
* Some MPI implementations may do copies (depending on the underlying network transport), in which case they’ll likely have a good scheme for pipelined copies and sending.

In short, memory copies are not necessarily evil, but if they’re employed, it’s much better to let the underlying MPI implementation *choose* to do that, and therefore use all of its available optimizations (e.g., using special memory, pipelined copies overlapping with sending, using its own limits for buffered resources, etc.).

   0 likes

]]> By: Brock Palen http://blogs.cisco.com/performance/how-many-ways-to-send/#comment-548348 Brock Palen Sun, 12 Feb 2012 01:00:45 +0000 http://blogs.cisco.com/?p=60815#comment-548348 Why are buffered sends evil?

   0 likes

]]>