Sad end to supercomputer in New Mexico

I’ve written about this before, about 6 months ago. Basically, the Encanto supercomputer in New Mexico, is being disassembled. The parts appear to be headed to universities in New Mexico, so its not a complete loss, but they will still have to pay for maintenance and power/cooling. What I had written before

may be summarized as “there are no silver bullets” to economic growth and prosperity. There are no magic stimuli that automatically return profits atop principal for investment purposes. And governments, which are by their very nature, extraordinarily inefficient at their tasks (which is part of the reason they should have so few tasks, and they should be limited from gaining more), should not be picking winners or subsidizing businesses. The only way to economic prosperity, which Encanto was supposed to usher in, is a concerted many year long program of making the state friendly for business, making the state lower cost for business, then getting out of the way of business to make sure they form, grow, relocate there. Or, as Churchill said in his famous speech …

Economic growth is very much like war. You need to focus upon the long term. You need to make a long term commitment, without distraction, and without siphoning off needed resources to other non-essential tasks. Building a supercomputer in New Mexico (in a “build it and they will come” mindset) is pretty much exactly the wrong approach. You need to encourage companies to form there, to create value there, to grow markets there, to build there. Towards that end, you need to fix tax policies, make the state generally business friendly, make it easier to attract and keep talent without dealing with historical baggage (becoming a right to work state is a massive step in the right direction there). If and when you get a surplus of tax receipts, you need to put some of it away for a rainy month, and reduce taxes otherwise (as this is very well known to increase economic activity). There is nothing that can be offered apart from blood, sweat, toil, and tears. But at the end of it, if you stay the course, if you don’t allow our baser political instincts to kick in and spend beyond our means for things we do not need (such as Encanto), economic growth is what you will likely get, assuming this path is done right. There are never any guarantees in life. And if people are trying to sell you silver bullets, chances are they will be willing to sell you a bridge in NY, and swampland in FL as well. Encanto was a supercomputer. But it was also endemic of a failed policy. There are others out there as well. The articles point out Amazon and other systems … which are run as P&L; type businesses … if they fail, the invisible hand of economic market forces will destroy them, reuse their reusable assets (bankruptcy) and discard the rest. When we mess with this process (ala GM, etc.) and don’t allow it to properly take its course, we delay what needs to happen for economic recovery to continue. If we hide the system from market forces, it makes the pain only worse when it is torn asunder, as Encanto was. New Mexico has many supercomputers (at Sandia, etc.). There are HPC companies there as well, though one I know and had been talking with may be no more now (also sad, but a natural part of the lifecycle of companies). Running as a P&L; type organization, with real payroll dependent upon your ability to generate profit, significantly impacts the way you chose to spend your money, what you are willing to invest in, and what you aren’t. Amazon, mentioned in the article, is trying to lower the cost of running HPC on their systems, and for some usage cases, they are doing a good job. For others, not so much. They have an incentive to try to figure out how to improve their systems and processes such that more customers will use them. Similar for Joyent, Rackspace, Sabalcore, … Systems similar to Encanto exist. I can’t tell you how many times I’ve heard of the “economic outreach” requirement for some random supercomputer center, that they find and capture revenue from local state businesses to continue to help justify their funding. Which gets to another side of this. About 14 years ago, a friend and one of my bosses/mentors at SGI made an argument that supercomputer centers were dying out, when I was seriously thinking of taking a job offer from one. I didn’t quite buy the full argument, but I did understand his point that market forces were working against these institutions, unless they could get long term grants and guaranteed customers. Not unlike securing long term revenue from good customers while doing business development to secure new ones. As the OSC and a number of other sites flourished, while some did close (AHPCRC, etc.), there wasn’t much competition for these sites … it was hard to build a supercomputer, run it, maintain it. Now, today, you have Cycle Computing (and their new CTO James Cuff @jamesdotcuff) selling virtual cluster setups on Amazon, and hopefully soon Joyent and others, where you can build almost exactly what you need, when you need it. This very rapid change over the last few years suggests far more powerful market forces are going to change the landscape of public facing high performance computing. These are real businesses, with real P&L; requirements, looking to make money in a very competitive environment. It is a mystery to me how a publicly funded organization, which isn’t under the gun to drive efficiency and #devops as hard as it can go, to keep cost per cycle down, and increase the profit per cycle and efficiency per cycle metrics, how they could even consider competing. This is not saying that non P&L; based HPC is dead. It is saying that in eras of tight budget realities, we collectively have to pick and choose our battles far more carefully (which we should have been doing from the outset). So while Encanto met a sad end, and will hopefully be resuscitated as teaching/research machines, it is incumbent to ask whether or not this model will be replicated (it will be) by clueless politicians being whispered to by lobbyists (it will be) again and again. And it is in our interests, all of our interests, to stand up and explain to them why this is a failed economic growth venture before it starts.