Many things are moving to cloud hosting … I won’t comment on being right or wrong about their moving … and HPC is one of them. This means that cluster distributions are going to follow … or could follow to some degree.
Some cluster distributions focus upon packaging, some focus upon flexibility, some focus upon GUIs.
All try to integrate some subset of needed tools. But all were effectively designed for a cluster computing model where some of the key/critical assumptions at the base of the distribution are simply not the case in the cloud, and due to the way they work, can’t easily be worked around.
These would be distros that require control over the install process. Remember, the cloud focus is upon system provisioning, so conflicting tools or redundant tools will be wasting some level of effort. A better model for this would be to integrate the API of the cloud provider into the installer, so firing up a 10 node cluster, on demand, is as easy as a 1000 node … with similar speed.
That last part is more about how the cloud resource does large allocations, and they may not be close … but its reasonable to expect that most customers don’t really want to pay for installation time and data transfer. Remember, you pay for everything in the cloud. So those distributions that focus upon loading an OS might have an issue going forward.
The sense I get talking to users and customers, is that they want their clouds as effectively “instant on” devices, for some appropriate value of “instant on”. The EC2 cluster bits are close to instant on. But we have customers asking for Ubuntu and other OSes.
And, more interesting to us, is that they are asking for “instant on” for their clusters. We work with a number of tools for these things, including Bright Cluster Manager, our own Tiburon tool, Perceus, and others. Right now, only our own Tiburon tool handles this. We’ve used it for a number of clusters, and it works quite well. I’ve been thinking about how to adapt it to the needs of the HPC cloud APIs.
икониBenchies soon. Real soon. Should be a screamer … if we designed/built it right.