I finally figured out where windows makes sense in HPC

This took a while. It sorta kinda hit me while at the BoF meeting, the ISV presenter pointed out that in the HPC/cluster world, windows doesn’t seem to show up, while on the desktop it is prevalent.

The argument is based on something I have been saying and using for years.
HPC moves downstream. As the market size increases, as it has been for quite a run now, more people use it, which means more people use smaller systems to run more models.
The Microsoft argument appears to be “hey they are on windows anyway, why not call it HPC friendly and try to deal with the back end”. I don’t think this is the right argument. I think what the BoF presenter was alluding to, and what we have been saying is the right argument.
That is, focus upon the apps. Make the apps interoperate seamlessly. Batch schedulers, job queuing systems are solved problems, but they are also 1970’s and 80s technology. Todays user doesn’t want to know from these.
In 1999 or 2000, I developed a program that looked and smelled like NCBI BLAST. Took the same options, returned the same results. What it was, was a shell around BLAST, which took care of the pain of dealing with job schedulers, clusters, data motion, etc. It handled everything, so you could focus upon the calculation and analysis. It divided the job up among N processors, and re-assembled the resulting output, in order. We used to show this to customers, and our acid test was running md5sum (hash) against outputs from our version and the “standard” version. Since it returned everything in order, and did so the same way, simply faster than the original, they always matched.
Why do I mention this? Simple. This is where I have though Microsoft could add value. Make the interface to the cluster simple. I shouldn’t have to care what the cluster is running, if the interface is a bunch of C# and the web service is a well defined API (even release a server backend for it), what does it matter?
Somehow that got lost in all of what they are doing.
Don’t get me wrong, I think the Microsoft folks will eventually get it right (the interface part). I simply remain (highly) skeptical on the computing side. Focusing upon solving a solved problem, one which has been solved as efficiently, powerfully, and cost effectively as it has been, has not been IMO a great expenditure of their time or money. If I were a shareholder (probably am through an index fund) I would be questioning this direction.
I would like to see them do stuff like allow me to run a job on the cluster, seamlessly, from inside my excel sheet. The cluster doesn’t need to run windows to do this, it simply needs to know how to talk to excel. And that is a problem that Microsoft can do something about, and do pretty well.
Also I would like to build analysis programs, using Excel as the front end. Let me do “what if” scenarios. Easily. Put all sorts of user interface elements into it. Make it *EASY* to see on the web, with browsers other than IE (cough cough FOLLOW WEB STANDARDS cough cough).
As I said, there are lots of things windows can help people do better, and many things Microsoft could have done to add value and drive sales.
Eventually, accelerator cards will be sitting in PC’s which will run lots of apps. Some of these cards will be computers in their own right, not just slave co-processors. Any bets on which OS they will run?
This is why the current “install windows on clusters” approach probably won’t work. Microsoft wants you to envision a cluster as an extension of the PC. That aspect is good, its the implementation as indicated which needs a re-think. Not to mention their marketing campaign.
But of course, this presumes that the company wants to play there, and isn’t simply executing tactics to try to outflank a competitor. With their chief exec going on record the way he does with his zingers, I dunno …
In the final analysis, I like Patrick, and I think the team is trying to do good things. They have hired some of the people I used to know at Sun and elsewhere. They just might be burdened by upper management that have axes to grind and cannot envision a world or a market where windows doesn’t dominate everything.
For a salient example of this, see the Zune.