Cisco Blogs

An app for that

August 27, 2010 - 16 Comments

Doug Eadline wrote a cluster rant recently entitled “A Cluster in your Pocket“, talking about the possibility of “What if your cell phone could bring you real time results from a supercomputer?”

We’ve actually idly chatted about such things in the Open MPI community for a while.  It would be tremendously fun to write an iPhone/Android app that could talk to an MPI implementation and/or application.  Perhaps a good starting point would be to have the MPI implementation talk to an iPhone/Android phone.


Indeed, a simple first feature might well be push notifications when an MPI job completes. Open MPI supports “notifier” plugins  that exactly fit this use-case scenario.  We just need someone to write a notifier that sends an iPhone/Android push notification.

Any takers?  Ping me; I’d be happy to point you in the right direction.

A next step would be conveying the actual results of the run to your phone — a text message, or a maybe a full app that conveys full, visual, meaningful results on your phone (i.e., not just the fact that the job completed).  Such an app may end up being application-specific, but perhaps a app could be constructed that could display several different kinds of results (e.g., X-Y data that can be easily graphed on the phone) such that MPI application developers could output some/all of their results in one of the supported formats that could then be pushed out to a mobile device. 

Imagine: your MPI job completes and sends a 2D graph summary of results to your phone.  You can take a quick look at it, pinch zoom to look at some of the data points closely, and think, “Hum.  That looks right.”  Or you think, “That’s bad!” and make a phone call to a colleague to re-start the job with a new data set.

Mmmm… what if you could re-submit the job from your phone? 

Ok, a) submitting a job is probably a really, really complex issue, and b) perhaps that type of feature is better suited to a parallel environment’s resource manager than the MPI implementation.  But hey — MPI is the hammer that I have, so why not start there?  🙂

Who’s got a few cycles to try out some of these ideas?  (I wish I did!)

In an effort to keep conversations fresh, Cisco Blogs closes comments after 60 days. Please visit the Cisco Blogs hub page for the latest content.


  1. Excellent point. I don't know what the status is of .gov organizations certifying the security of mobile devices in general, but having an app for data retrieval could definitely have a higher degree of security than email or SMS. The two main limiting factors at that would would be trusting SSL for data transport and the fact that the data would now be on a mobile device.But still, that's better than email or SMS, where unencrypted data could/would flow through random 3rd party servers.

  2. We have been playing with an Android app for perceus/abstractual lately. I like a separate app vs email and sms because I want strong crypto and don't trust security of anything I can easily spoof, especially since most of our users are .gov :)

  3. David --Ah, but now you have a new argument! Boss, I need an iPhone in order to run my MPI jobs..."":-)"

  4. This would be great...if only our employer allowed us to bring cellphone into work. ;-)Really, though, it is a useful idea for a lot of folks.

  5. MPI effectively killed PVM, so if you have Open MPI working, you're probably off to a good start.Not to dampen your enthusiasm, but there are a number of factors that make me dubious of whether a cluster of cell phones would ever come close to the performance of a ormal"" HPC-style cluster. Doing some back-of-the-envelope calculations on power consumption (battery and/or AC?), CPU performance, memory size, and memory performance might be useful to see what kind of target you're shooting for. For example, if the phones are *always* going to be less performance than a typical HPC-style compute cluster, then identifying the tradeoffs in terms of performance, usability, and other factors might be a good first step in seeing if there are scenarios where a cell phone cluster might make sense. Maybe 10 or 20 cell phones don't make much sense. But maybe 10 million do...? (I'm dubious, but this is certainly worth a thought experiment!) :-)Getting some of the traditional MPI benchmarks up and running on your phone cluster might be a good practical first step: ping-pong latency and bandwidth, HPL, ...etc."

  6. I am using an Openmoko neo freerunner with qtmoko operating system , qtmoko is a distribution for Debian and i can install and run any package i want , also Android OS is based on Debian too.May be after i finish this project i will start porting some packages to Android.About the use cases ,The idea started with an article in spectrum magazine about low power supercomputer for climate modeling that uses cell phones processors.Another inspiration from a conference called CS10 in New Orleans , features the Student Cluster Competition that challenges students to build, maintain, and run the most-cutting edge, commercially available high-performance computing (HPC) architectures on just 26 amps or the energy equivalent of three coffee-makers.Now i have debian on my cell phone and it is now act like a normal PC so it is much easier to try one of those ideas.And about the PVM , i am starting with the available packages from the repositories before building new packages because the Kernel is a little unstable and PVM is the only one i found .Please suggest for me an alternatives for PVM.

  7. Tarek --That sounds fun! But be sure to read Doug Eadline's article about the usefulness of such a concept (linked in my blog entry). Unless I'm missing something, you'd need some very, very specific use cases for a mobile device to be a useful part of a larger HPC job.Why PVM, BTW? It's been dead for years.

  8. Andrew --I think it would certainly be cool if something like Grisu was translated to a mobile device. I can imagine it being genuinely useful, but some thought would have to be put into viable use cases, since you don't have direct access to input files, etc. Doug Eadline said this to me in an email (slightly paraphrased):Can you imagine getting a text like this from the job scheduler:Jeff: one of your gromacs jobs appears to have hung; I have notified the admin. I'll resubmit it and let you know how it goes. Enjoy your weekend.""That would be awesome.How having a mobile version of Grisu would affect usage uptake, etc. -- I'm afraid I have no data on that. I'm just one guy voicing his opinion..."

  9. Hi Jeff ,Actually part of my graduation project is to make the cell phone a node or even the server for a cluster.Success : i already built openmpi and PVM on my phone.Now i am testing them.

  10. I would be interested in knowing if you would like to be able to interface with jobs and cancel or resubmit jobs where they get stuck or where resources on a particular cluster is oversubscribed and can be run elsewhere.Please look at Grisu Client on ARCS website, This possibly can be translated to a mobile device and ARC is looking at how it enhances uptake of services using these devices.I am particularly interested in knowing what the uptake of such a device would be.Andrew

  11. Hi BrockThat is exactly what we will do. The issue isn't how"" as the notifier already knows a variety of methods. All that is required is to have mpirun call the notifier when the job is done.Trivial addition."

  12. Why not use normal batch system email-sms gateways? this works now, there is a list of addresses on wikipedia how to 'email to sms' for each carrier,

  13. Funny thing this ... our DragonFly code does exactly this. It was designed to enable people to submit new jobs, manage jobs, resubmit, and examine output over a web interface. Specifically I was aiming to be able to use it on my BlackBerry at the time.I haven't seen much interest outside of the one customer who has been using it, but we haven't asked widely. Would be fairly simple to redo the web interface to be more friendly to smart phones. The backend was the big thing though ... has full capability to run on (effectively arbitrary) remote resources as well as local resources.If there is interest in this thing, please ping me, and we can talk about it. Its not abandoned, just in hibernation until we can figure out what to do with it.See some bits of it here and here

  14. Getting the notifier to do something at job end is trivial - I can add that option this weekend.Adding an ability to start jobs is harder because that isn't something ORTE can do - it requires an interaction with the cluster resource manager and will likely be installation-specific. Still, for larger installations (e.g., universities), you could build an app that they configured for their users. Worth pondering.No idea how to do generalized visualization - that sounds very app specific.

  15. For simple, static summary results, yes, you're probably right -- a link to a web page with an image (or whatever data is relevant) is probably sufficient. Sending an email or SMS with a URL when a job is done is pretty easy.I guess an app would be most useful for two general kinds of things:1. If there really was some kind of interface to the resource manager / scheduler (e.g., be able to submit new jobs, query the status of running jobs, etc.). But like I mentioned, that is likely to be a pretty complex issue.2. If you wanted (needed?) to do some local manipulation of the data on the phone itself, an app might be better for that because the data could be cached on the phone rather than a query to some remote sever for every action. That being said, with reasonable wifi and/or 3G access, this issue might be moot (lots of complex queries occur in the cloud"" today and the latency isn't terrible). But hey -- maybe you need to view / manipulate the data while you're on an airplane with no wifi! ;-)In the end, it probably comes down to: what visualization API are you familiar with / want to use? All things being equal, the distinction between generating the data visualization on a web page vs. on the local device might be a wash. So do it with the API that you're familiar with."

  16. I'm afraid I don't see the advantage of a dedicated app over email/SMS notification with a web link for more details and interactive features.