scalability.org

Years ago … ok … decades ago, when I was building my first large clusters, I worried about configuration and drift. OS installers are notoriously finicky, and one of the hard lessons is that you should spend as absolutely little time inside them as possible. Do the bare minimum work you need to in order … Read more Thoughts on configuration management vs image artefact management

The link to his obituary. A wonderful person, deeply insightful, excellent communicator. Gone too soon. I will miss him. I think everyone in #HPC will.

So yesterday, an article and discussion appeared on Hacker News. In the article, the author asks reasonable questions, of how to optimize a Python code. What happened next, probably wasn’t as they intended. The article was, ostensibly, on optimizing Python code. It, after the 4th attempt at source level optimization, switched languages. It was no … Read more On optimizing scripting languages, and where they are useful and where they are not

Compressing tools like gzip and bzip2 have been around quite a long time. They are, well, mature. Almost boring. You depend upon them for many things. You don’t really pay attention to them until you use them for significant work. Like with NyBLE, compression and decompression are important, and time sensitive steps … well … … Read more Updating compressors for NyBLE

This is the project that I alluded to. We tuned the system, the code, the environment. We wrote tooling to massively simplify job creation and submission. Moreover, we worked around numerous issues that arise in each technological layer. Multiple simultaneous tools are being deployed to work on this, and I am hopeful that the net … Read more My urgent #HPC computational project, COVID-19 related

I built an updated raid pair of drives with a brand new OS for the system that underlies this blog and other services. Basically, the previous system load had been updated from debian 7 through debian 9 and had accumulated lots of cruft. So I rebuilt this using my wonderful nyble system on a lab … Read more Time to replace some hardware

For those who don’t know what nyble is … you can read an old post here. The short version is that it gives you an always reproducible bootable ramdisk (or stateful if you need) image (installation for stateful folk). You avoid worrying about configuration drift, as you roll a new image in ~10-20 minutes, and … Read more Updated nyble to support ubuntu 20.04 LTS and debian 10

In the world in which we reside, a pandemic slowly burns. This pandemic has confounded front line medical practitioners, public health organizations. It has exposed a number of troubling relationships amongst governments and organizations. It has resulted in numerous pronoucements of “X may work” from medically and scientifically illiterate political leaders. The problem is, fundamentally, … Read more There are no silver bullets

I’ve been playing with Julia for a while now. As the language is evolving quickly, I have to learn and relearn various aspects. Happily, since 1.0 dropped a while ago, its been more consistent, and the rate of change is lower than in the past. The Riemann ζ function is an interesting problem to test … Read more Performance of a julia code: Riemann ζ function implemented on CPU and GPU

(caution: I get personal at the end of this, you can see my motivation for working on this) I can’t talk in depth about it though, yet. I can talk in broad brush strokes for now. Imagine for a moment, that you have a combination of available high performance supercomputers, an urgent problem to be … Read more Topical #HPC project at the $dayjob