A new #HPC project on github, nlytiq-base

Another itch I’ve been wanting to scratch for a very long time. I had internal versions of a small version of this for a while, but I wasn’t happy with them. The makefiles were brittle. The builds, while automated, would fail, quite often, for obscure reasons. And I want a platform to build upon, to enable others to build upon. Not OpenHPC which is more about the infrastructure one needs for building/running high performance computing systems. That is a good effort, though it also needs .debs for Ubuntu/Debian, or even better, source and Makefiles. What I wanted here was a set of analytical and programming tools for working with data. Specifically, up to date tools, modern … not end-of-life packaged tools that are so badly out of date, that you can’t install modern extensions to them, or use them to bootstrap the tools you need. So the github repo is here. This is very early release of the tool chain build environment. You can configure everything from base.config, and run make. It will take a while, but it will eventually result in a fully populated analytical tree. One gotcha now will be the ATLAS build. I need to set up detection to see if there exists on machine blas/lapack/atlas, as ATLAS wants you to turn off processor throttling to build, or it fails in a strange way. I’ll add in some code to detect this. Specifically, I’ll see if I can force affinity for a specific processor and have it build on that. Not optimal, but better than failing. If this is not possible, I’ll look for the lapack/blas/atlas libs on the main unit. If they are there, great, we’ll use them. Otherwise, in the worst case, if we can’t do any of these, I’ll build the slow versions. I certainly would like to get feedback from people on what they might want in this, what additional R/Python/Go/Node/Perl packages they want embedded. And whether or not they want a mountable compressed file system image, a docker image, or whatnot else. My plan is to use this as a base for something else I’ve been wanting to build. More later, but its a start.