Way way back in the early aughts (2000’s), we had built a set of designs for an accelerator system to speed up things like BLAST, HMMer, and other codes. We were told that no one would buy such things, as the software layer was good enough and people didn’t want black boxes. This was part … Read moreBrings a smile to my face … #BioIT #HPC accelerator
Note: I do quite a bit of development in Perl, and have my own biases, so please do take this into consideration. It is one of many languages I use, but it is by and large, my current go-to language. I’ll discuss below. According to TIOBE (yeah, I know), Perl usage is on the rise. … Read more#Perl on the rise for #DevOps
This is something we’ve been working on for a while … a completely clean, as baseline a distro as possible, version of our SIOS RAMdisk image using CentOS (and by extension, Red Hat … just need to point to those repositories). And its available to pull down and use as you wish from our download … Read moreFully RAMdisk booted CentOS 7.2 based SIOS image for #HPC , #bigdata , #storage etc.
Color me impressed … Seagate and their 60TB 3.5inch SAS drive. Yes, the 60 bay Unison units can handle this. That would be 3.6PB per 4U unit. 10x 4U per 48U rack. 36PB raw per rack. 100PB in 3 racks, 30 racks for an exabyte (EB). The issue would be the storage bandwidth wall height. … Read more@scalableinfo 60 bay Unison with these: 3.6PB raw per 4U box
While the day job builds (hyperconverged) appliances for big data analytics and storage, our partners build the tools that enable users to work easily with astounding quantities of data, and do so very rapidly, and without a great deal of code. I’ve always been amazed at the raw power in this tool. Think of a … Read moreRaw Unapologetic Firepower: kdb+ from @Kx
So we’ve been using Debian 8 as the basis of our SIOS v2 system. Debian has a number of very strong features that make it a fantastic basis for developing a platform … for one, it doesn’t have significant negative baggage/technical debt associated with poor design decisions early on in the development of the system … Read moreSystemd and non-desktop scenarios
See here, which was largely about how to architect high performance analytics platforms, and a specific shout out to our Forte NVMe flash unit, which is currently available in volume starting at $1 USD/GB. Some of the more interesting results from our testing: 24GB/s bandwidth largely insensitive to block size. 5+ Million IOPs random IO … Read moreTalk from #Kxcon2016 on #HPC #Storage for #BigData analytics is up
TL;DR version: 10GB/s write, 10GB/s read in a single 2U unit over 100Gb network to a backing file system. This is tremendous. The system and clients are using our default tuning/config. Real hyperconvergence requires hardware that can move bits to/from storage/networking very quickly. This is that. These units are available. Now. In volume. And are … Read moreNot even breaking a sweat: 10GB/s write to single node Forte unit over 100Gb net #realhyperconverged #HPC #storage
I am trying to help people conceptualize the experience. Here is a video depicting very fast, very powerful cars and their sound signatures. This is a good start. Take one of those awesome machines, and turn off half the engine. So it is literally running with 1/2 of its power turned off. Remember this. There … Read moreMassive unapologetic storage firepower part 4: On the test track with a Forte unit … vaaaaROOOOOOMMMMMMM!!!!!
We’ve got customers with very large resource machines. And software that sees all those resources and goes “gimme!!!!”. So people run. And then more people use it. And more runs. Until the resources are exhausted. And hilarity (of the bad kind) ensues. These are firedrills. I get an open ticket that “there must be something … Read moreWhen infinite resources aren't, and why software assumes they are infinite