scalability.org

Not a math post, I promise. Really about teams. You have a team of people. You have a mission. You have a (short) timeline. You need them to focus on the problem, and find the minimum temporal path length process to achieve a resolution. You have a process, albeit informal, to address issues, which is … Read more

Years ago … ok … decades ago, when I was building my first large clusters, I worried about configuration and drift. OS installers are notoriously finicky, and one of the hard lessons is that you should spend as absolutely little time inside them as possible. Do the bare minimum work you need to in order … Read more

The link to his obituary. A wonderful person, deeply insightful, excellent communicator. Gone too soon. I will miss him. I think everyone in #HPC will.

So yesterday, an article and discussion appeared on Hacker News. In the article, the author asks reasonable questions, of how to optimize a Python code. What happened next, probably wasn’t as they intended. The article was, ostensibly, on optimizing Python code. It, after the 4th attempt at source level optimization, switched languages. It was no … Read more

Compressing tools like gzip and bzip2 have been around quite a long time. They are, well, mature. Almost boring. You depend upon them for many things. You don’t really pay attention to them until you use them for significant work. Like with NyBLE, compression and decompression are important, and time sensitive steps … well … … Read more

This is the project that I alluded to. We tuned the system, the code, the environment. We wrote tooling to massively simplify job creation and submission. Moreover, we worked around numerous issues that arise in each technological layer. Multiple simultaneous tools are being deployed to work on this, and I am hopeful that the net … Read more

I built an updated raid pair of drives with a brand new OS for the system that underlies this blog and other services. Basically, the previous system load had been updated from debian 7 through debian 9 and had accumulated lots of cruft. So I rebuilt this using my wonderful nyble system on a lab … Read more

For those who don’t know what nyble is … you can read an old post here. The short version is that it gives you an always reproducible bootable ramdisk (or stateful if you need) image (installation for stateful folk). You avoid worrying about configuration drift, as you roll a new image in ~10-20 minutes, and … Read more

In the world in which we reside, a pandemic slowly burns. This pandemic has confounded front line medical practitioners, public health organizations. It has exposed a number of troubling relationships amongst governments and organizations. It has resulted in numerous pronoucements of “X may work” from medically and scientifically illiterate political leaders. The problem is, fundamentally, … Read more

I’ve been playing with Julia for a while now. As the language is evolving quickly, I have to learn and relearn various aspects. Happily, since 1.0 dropped a while ago, its been more consistent, and the rate of change is lower than in the past. The Riemann ζ function is an interesting problem to test … Read more