As the market (rapidly) grows

We have been pointing out for a long while that the HPC market is growing at a break-neck pace. The latest IDC numbers, continue to support these claims. Many places have the numbers and the analysis. Pointing to HPCwire’s analysis:

Yeah, that about sums it up. Now remember, that Linux is the most common and largest fraction of the HPC server market, and despite protestations (and marketing) to the contrary from interested parties, this shows no signs of letting up, or changing in any significant way. Another element of this is where the market is growing. It has not been lost on us that IDC shows the growth in the smaller priced segments as compared to the rarefied super clusters. Sure, it would be nice to win a few Columbia sized deals, but those could easily drive a small company out of business … not to mention to distraction. Not an issue, as this part of the market is shrinking rapidly in relative and absolute size. We don’t want to own a larger fraction of a declining market segment. IDC echos what we have been saying for the last decade. HPC has been going down-market (mainstream) for the duration. There is no evidence that this is changing, rather, it is accelerating as people find their systems are more capable (and once freed from the chains of typical IT requirements, they can build them as they need them, not as IT wishes to support them). To wit:

If you are not required to buy machine X from vendor Y as you have an HPC requirement, you have a fighting chance to get an HPC machine, like our many-core Pegasus deskside units (8-16 cores, up to 128 GB ram), which should be both good enough for most things, and very simple to manage and run. We have quite a few customers using these for specific apps, and they do seem to like them. They can even run windows on them, though we usually recommend running it as a VMWare session (most of their apps won’t work well with XP x64 yet, as they are 32 bit apps, and can’t make any use of the 64 bit nature of the system … and corporate purchasers are loath to purchase a 32 and 64 bit license to cover their machines, when the 32 bit is what they have pre-negotiated … it is much easier just to get the right Linux version license). Moreover, HPCwire reports

Yeah, that is about the case, though they miss a few obvious things. Like the accelerator market. But I won’t tell them. No, not at all. Also, though they may be loathe to admit it, the data in this case, is telling a difficult to swallow truth (to some publications and vendors), that this is mostly a Linux market, and it is showing no signs of becoming anything else. It is important to figure out how to fit into this market … changing it would be hard. You have to be better/cheaper/faster. See the accelerator market comment above. I thought VC’s fund this sort of stuff … oh, I know, we will talk about accelerated AI’s for social networking. Obviously an exploding market (snort). HPC wire also notes:

This is a big market. It has lots of secondary markets. Value here is not simply inverse of price but it has rapidly devolved into something like that … that is, HPC is a commodity. Treating it as anything other than a commodity is a recipe for failure in the market. HPCWire further laments

and then answers their own question

I should note that the day job provides supplemental support to organizations (big TLA’s) that don’t quite have the ability to handle the type of service/support their customers need. They are loathe to pay for service they should be able to provide to their customers. They can do it through us, or allow their customer to suffer (and find us on their own, which is what usually happens). Ease of deployment is painless. Good solutions like Tiburon, Perceus and a few others make deployments, generally, easy. Manageability is also very important to make easy … not just systems, but applications, … End user usage via DragonFly and similar minded technologies are important to have. Price is but one aspect of value. Its the whole package that is of value. It is a commodity market … 1U servers are 1U servers. The differentiation is in the “secret sauce”. Its what makes JackRabbit so bloody fast, makes our Pegasus boxen so nice. Anyone can put motherboards into chassis, slap a few processors/ram/disks in there, and claim they are a cluster vendor. We have spoken with more than a few companies recently, just like that. Don’t be fooled, they have very little depth, and quickly get in way over their head. Their “lower price” becomes a liability when they can’t support what they sell … and more importantly, can’t design a cluster worth a damn. Special sauce also comes in from years/decades in the market, dealing with all sorts of problems. Not something that can be picked up in a few weeks. You need strategic acquisitions or hires to make that happen. The value is in making it work, in making it roar. Not in having it whimper in a corner when things don’t go so well, and you engage with multiple conference calls and try to find someone with a clue at your low priced vendor. Where is the value there? There isn’t any. This market is growing, rapidly. It attracts vendors, and pretenders. Try to avoid the latter.