As the market (rapidly) grows

We have been pointing out for a long while that the HPC market is growing at a break-neck pace. The latest IDC numbers, continue to support these claims. Many places have the numbers and the analysis. Pointing to HPCwire’s analysis:

The total HPC server revenue in Q3 worked out to $3.0 billion, which represents an astounding 22.9 percent of the $13.1 billion worldwide server revenue in Q3, a 0.5 percent increase from Q2 and a 6.3 percent increase from Q3 2006.
Another way to look at it is that non-HPC server revenue declined slightly in the third quarter and only managed anemic year over year growth. Without HPC to buoy it up, enterprise server sales are flagging.

Yeah, that about sums it up.
Now remember, that Linux is the most common and largest fraction of the HPC server market, and despite protestations (and marketing) to the contrary from interested parties, this shows no signs of letting up, or changing in any significant way.
Another element of this is where the market is growing. It has not been lost on us that IDC shows the growth in the smaller priced segments as compared to the rarefied super clusters. Sure, it would be nice to win a few Columbia sized deals, but those could easily drive a small company out of business … not to mention to distraction. Not an issue, as this part of the market is shrinking rapidly in relative and absolute size. We don’t want to own a larger fraction of a declining market segment.
IDC echos what we have been saying for the last decade. HPC has been going down-market (mainstream) for the duration. There is no evidence that this is changing, rather, it is accelerating as people find their systems are more capable (and once freed from the chains of typical IT requirements, they can build them as they need them, not as IT wishes to support them).
To wit:

IDC attributes much of the HPC revenue growth to lower entry prices. The fastest growing segment is at the low end — systems priced under $50,000. With more powerful processors available, workgroups, and even individual engineers and researchers can purchase HPC systems for as little as $10,000 dollars. IDC projects the sub-$50,000 segment to have an 11.4 percent CAGR through 2011.

If you are not required to buy machine X from vendor Y as you have an HPC requirement, you have a fighting chance to get an HPC machine, like our many-core Pegasus deskside units (8-16 cores, up to 128 GB ram), which should be both good enough for most things, and very simple to manage and run. We have quite a few customers using these for specific apps, and they do seem to like them. They can even run windows on them, though we usually recommend running it as a VMWare session (most of their apps won’t work well with XP x64 yet, as they are 32 bit apps, and can’t make any use of the 64 bit nature of the system … and corporate purchasers are loath to purchase a 32 and 64 bit license to cover their machines, when the 32 bit is what they have pre-negotiated … it is much easier just to get the right Linux version license).
Moreover, HPCwire reports

Of course, all of this growth is riding on the popularity of cluster computing systems, which in the third quarter represented 68 percent of all HPC server revenue. With no competing architecture on the horizon offering comparable price/performance for the majority of applications, clusters are destined to maintain their dominance in HPC for the foreseeable future.

Yeah, that is about the case, though they miss a few obvious things. Like the accelerator market. But I won’t tell them. No, not at all.
Also, though they may be loathe to admit it, the data in this case, is telling a difficult to swallow truth (to some publications and vendors), that this is mostly a Linux market, and it is showing no signs of becoming anything else. It is important to figure out how to fit into this market … changing it would be hard. You have to be better/cheaper/faster. See the accelerator market comment above.
I thought VC’s fund this sort of stuff … oh, I know, we will talk about accelerated AI’s for social networking. Obviously an exploding market (snort).
HPC wire also notes:

Looking at the yearly HPC revenue, IDC reported server revenue topped $10 billion in 2006. According to them, adding in other elements of the ecosystem like storage and services pushes the total to $16.3 billion. Even this figure may be conservative. Tabor Research, using an end-user research approach, estimates HPC server revenue represents only 43 percent of the total spending, even excluding things like staff, facilities and power/cooling costs. Using IDC’s $10 billion server figure, the Tabor Research model would yield something closer to $23.5 billion in total spending. Now you’re talking real money.

This is a big market. It has lots of secondary markets. Value here is not simply inverse of price but it has rapidly devolved into something like that … that is, HPC is a commodity. Treating it as anything other than a commodity is a recipe for failure in the market.
HPCWire further laments

Which brings us to something of a paradox. If clusters are such a growth industry, why aren’t there more publicly traded cluster computing system vendors? The big vendors, like IBM, HP, Dell and Sun Microsystems, are public, but the HPC server business is just a slice of a much larger set of offerings. All of the HPC-only cluster system vendors, such as Appro, Linux Networx, Penguin Computing, etc., are privately held.

and then answers their own question

Theoretically, the tier two players who specialize in cluster systems should be able to compete effectively against the tier one vendors, by either adding value, undercutting prices, or both. But the commodity nature of cluster computing cuts both ways. It makes entering the market easy, but establishing long-term differentiation hard. Almost everyone is building these machines from the same commodity parts: x86 processors, standard memory chips, Linux (or Windows) software, and Ethernet or InfiniBand gear. Higher value features that address usability, ease of deployment, and manageability are only now starting to be perceived as equally important as raw performance. With thin profit margins on the hardware, vendors are using these higher value features as their “secret sauce.”

I should note that the day job provides supplemental support to organizations (big TLA’s) that don’t quite have the ability to handle the type of service/support their customers need. They are loathe to pay for service they should be able to provide to their customers. They can do it through us, or allow their customer to suffer (and find us on their own, which is what usually happens).
Ease of deployment is painless. Good solutions like Tiburon, Perceus and a few others make deployments, generally, easy. Manageability is also very important to make easy … not just systems, but applications, … End user usage via DragonFly and similar minded technologies are important to have.
Price is but one aspect of value. Its the whole package that is of value. It is a commodity market … 1U servers are 1U servers. The differentiation is in the “secret sauce”. Its what makes JackRabbit so bloody fast, makes our Pegasus boxen so nice.
Anyone can put motherboards into chassis, slap a few processors/ram/disks in there, and claim they are a cluster vendor. We have spoken with more than a few companies recently, just like that. Don’t be fooled, they have very little depth, and quickly get in way over their head. Their “lower price” becomes a liability when they can’t support what they sell … and more importantly, can’t design a cluster worth a damn.
Special sauce also comes in from years/decades in the market, dealing with all sorts of problems. Not something that can be picked up in a few weeks. You need strategic acquisitions or hires to make that happen.
The value is in making it work, in making it roar. Not in having it whimper in a corner when things don’t go so well, and you engage with multiple conference calls and try to find someone with a clue at your low priced vendor. Where is the value there?
There isn’t any.
This market is growing, rapidly. It attracts vendors, and pretenders. Try to avoid the latter.