Evolution of clouds in HPC

I argue that clouds are v3.0 of the original idea, ASP. ASP provided a remote hardware/software environment to run your apps. It didn’t have the benefit of virtualization, standard stacks like LAMP, etc. Plus it had high costs to get started. It was destined/doomed to fail. ASP v2.0 came about with “grids”. They were the buzzword for a while, and sought to provide a “utility computing” (remember that) model? Provide a platform, make access easy, and they will come. All they overlooked with the pricing/usage model. Those were inherently broken. ASP v3.0 are clouds. They are the new buzzword. Quite a bid of fad-ish stuff in there, but quite a bit of good stuff as well. The issue with HPC is that it generally hates things that slow it down, like virtualization. I mention this as HPC has a place in the cloud, but virtualized in a different manner than say a web server. So far Amazon has stepped up to try this. Will others?

One of the larger paradigms in HPC uses MPI calls to handle interprocess communication. MPI really requires a pretty well known connection between nodes, and a fast/stable access to a communications fabric. The cloud brakes both of these issues. Maybe we need to rethink some of our programming models. Or start using systems tied together using vSMP from ScaleMP, an aggregating virtualization (build big systems from little systems) rather than cutting little systems from bigger ones. This would make the programming side easier … … and this is an issue. MPI and many other similar systems presume a relatively low latency and high speed connection to nodes, not something potentially highly non-deterministic. With OpenMP and thread based models, or even MPI on such a single machine via vSMP, this should be less of a problem. IO bandwidth in VMs is definitely an issue. Not an easy one at that. There are no good simple fast fixes, apart from using fast hardware. Virtualizing/sharing distinct storage areas will increase the load on the systems, so you will need load aware placement going on behind the scene, not just efficient job/run packing. Clouds open up new possibilities, and bring new challenges. Watching this evolve … helping it to evolve, should be lots of fun.