Scalable Informatics is now part of the HPC Advisory council

For the day job … We are happy about joining this group. Interest in our high performance storage and computing systems continues to grow across multiple sectors. As users need to store and process exponentially growing amounts of data, they need systems, fabrics and designs capable of scaling without introducing additional barriers. This group represents … Read moreScalable Informatics is now part of the HPC Advisory council

Fighting the dmraid/mdadm battles in initrd for RHEL/Centos 5.x

dmraid is a technology to turn on-board fake-RAID (fRAID) systems into usable/bootable linux machines. It works for what it does, but you do need to be careful, as many of the fRAID chipsets have interesting … er … features. Yeah. Thats it, features.
mdadm is a pure software version, requiring no assist from the bios. It can handle setting up RAID devices, and is our preferred way of creating RAID in software.
These two systems are incompatible, and if you don’t have fRAID support on your motherboard, you really can’t use dmraid.
mdadm has a mode for RAIDing full drives rather than partitions, which is roughly akin to what fRAID does.
Ok, so why is this a problem?

Read moreFighting the dmraid/mdadm battles in initrd for RHEL/Centos 5.x

Non-locality in computing

I read an article a few weeks ago that mirrors some of the things I’ve said in the past about computing on huge systems. Basically, when you have a system of sufficiently large size, the communications fabric between the nodes are such that for any ith and jth node, the latencies and transit times may not be uniform, or worse, there may be significant time cost to communicate between various nodes.
In the text, I use the concept of a light cone. This is a construct from relativity theory, with time representing the vertical axis. As a message or some data needs to move, it is constrained by something approximating the speed of information flow across the distributed computer (lets call this C{dist}). You can exchange information across two nodes when their light cones intersect.
But in the case of a very large machine with potentially millions of cores, it may take far longer than anticipated for those light cones to intersect, as not all nodes are one hop away anymore.

Read moreNon-locality in computing

Zombie modeling, a possible new HPC application?

I had a laugh when I read Professor Robert Smith?’s home page (the question mark is really there).

After my PhD, I did a postdoc at the University of Western Ontario, where I discovered infectious diseases. At first it was just the one, you know? A little HIV, you know you want to, all your friends are doing it… Before I knew it, I was studying malaria, then it was human papillomavirus. After that, it was all a blur of neglected tropical diseases that kept coming and coming and, oh god, then I was into some really hardcore stuff, man. And once you’ve tried modelling zombies, you can never go back… *sobs*

Ok … imagine having a thesis advisor like this … Man that would be fun research! Shawn of the dead as source material …

Read moreZombie modeling, a possible new HPC application?

Now thats one big pepperoni pizza!

Get ready to laugh a little … landman@metal:~$ dd if=/dev/zero of=big-huge-file-system-target.img bs=1k seek=2T count=1 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 4.4349e-05 s, 23.1 MB/s landman@metal:~$ ls -alF big-huge-file-system-target.img -rw-r–r– 1 landman landman 2251799813686272 2009-09-04 17:25 big-huge-file-system-target.img landman@metal:~$ ls -alhF big-huge-file-system-target.img -rw-r–r– 1 landman landman 2.1P 2009-09-04 17:25 big-huge-file-system-target.img 2.1PB file … Read moreNow thats one big pepperoni pizza!

HPC for Dummies book!

Doug Eadline has an e-book out. From the posting on Beowulf: It is not a real “book” (it is short), but some people may be interested in: HPC For Dummies You have to register for a copy. The author is some HPC hack. It appears to be Windows/Mac only though (not Doug’s fault, don’t … Read moreHPC for Dummies book!