Addison Snell's HPC trends: Interesting things, and a few comments we take issue with

I found the article on InsideHPC about Addison’s presentation quite useful. The presentation is available from the link above. Some points he made, I’d like to take issue with. Specifically, page 9, he notes that Windows HPC is “still coming”. I am not too sure of this. I think it has been a multi-year, almost half decade experiment, that at some point, needs to show that its revenue is greater than the cost of that revenue. The business model for it doesn’t make much sense from a simple vertical for Microsoft. It won’t really ever generate $1B type numbers for Microsoft … if it is around, it is in the noise of their business. Moreover, the cost of winning with it is huge for Microsoft. They have to fight against an entrenched competitor with effectively zero actual cost. Short version, its a Quixotic quest.

As for “Early Adoption”, this should have happened years ago. It hasn’t to any significant degree. While we and others are seeing Silverlight and C# on front end servers, we are still seeing the back end, heavy lifting systems be hard core Linux boxen. I know, as we are selling these sorts of systems, to this sort of customer. We have (non-shareable) benchmarks from customers, running the same tests on Linux boxen and Windows boxen, and the results are, to put it diplomatically, not even remotely spinnable as being anything but a rout of the windows boxen. What we are seeing, without spilling beans, is more financial services customers very focused upon extremely low latency systems, extremely high performance file systems, and extremely fast processors/accelerators. What they run on the front end of these might be Microsoft servers for various user application technologies … but the analytics, the very high performance stuff, is usually shipped to the supers on the back end. I also take issue with the statement that Linux is “continuing to fragment”. This sounds like Microsoft marketing-speak, and is actually quite untrue. Linux has coalesced nicely around several distributions, which are interchangable. You can wipe one out and put on another without pain. You have actual real choices. Many shops we deal with are “pure Redhat”, though they also like Centos (which is the Redhat rebuild) as it allows them to reduce their support license costs. A few shops we deal with are “pure SuSE”. What we do see are transitions from Windows to Linux based systems, usually where performance matters most. These transitions are made harder by the development environment that allows you to write fairly non-portable C++ (we were surprised at how hard it was to port one customers very basic code over). But when you care about performance, you want to remove as many barriers as possible, as quickly as possible. So you start with the fastest designs and work from there. So, suffice it to say, that this, coupled with a number of other things, leads us to conclude that contrary to Addison’s slide, our view is that the growth in Linux is strong, and increasing. We are seeing customers specifically looking to reduce their Windows footprint to control their software costs. And we are seeing this on deskside units as well as on other units. This doesn’t mean we don’t have windows customers. We do, and we try to deliver the fastest possible systems in this case. If Windows is a critical part of the customers environment, by all means, they should use it. If they need to focus upon performance first, and OS is an incidental aspect to be decided by application and other issues … well … this rarely works out in favor of Windows, apart from when the app is purely windows based. And even then we are seeing more customers do Windows in a window … virtualized desktop environments. Keeps installation costs down, you don’t need windows installed every where. Just access to a VM. Addison also looked at Lustre. Similar issues there. We have customers actively using it on our systems, and thus we have something of a business dependency upon it (they do, and we support them). It is drifting, sadly. Oracle is not an HPC focused company (nor should they be). Lustre is a decidedly HPC focused file system. Our concern with Lustre is similar to our concern with SGE, with VirtualBox, with OpenOffice. All of which we have direct business dependencies upon. I do agree that storage is an area to watch, and I’d suggest adding GlusterFS to the list of file systems to watch. It is maturing, and it does a number of things quite differently than Lustre and other centralized resource based file systems. Our list from a few days ago also covers a few others. This said, we see a hard growing need for more very high performance storage going forward. Additionally, we are seeing and hearing people looking to use smaller machines to do their computing. One machine with 24 to 48 processor cores could radically alter the “lower end” of HPC. And make the market much larger for its use. Its better, cheaper, faster. This is what will win in 2010 and beyond.