Talk from #Kxcon2016 on #HPC #Storage for #BigData analytics is up

See here, which was largely about how to architect high performance analytics platforms, and a specific shout out to our Forte NVMe flash unit, which is currently available in volume starting at $1 USD/GB. Some of the more interesting results from our testing: 24GB/s bandwidth largely insensitive to block size. 5+ Million IOPs random IO … Read moreTalk from #Kxcon2016 on #HPC #Storage for #BigData analytics is up

Going to #KXcon2016 this weekend to talk #NVMe #HPC #Storage for #kdb #iot and #BigData

This should be fun! This is being organized and run by my friend Lara of Xand Marketing. Excellent talks scheduled, fun bits (raspberry pi based kdb+!!!). Some similarities with the talk I gave this morning, but more of a focus on specific analytics issues relevant for people with massive time series data sets and a … Read moreGoing to #KXcon2016 this weekend to talk #NVMe #HPC #Storage for #kdb #iot and #BigData

Success with rambooted Lustre v2.8.53 for #HPC #storage

[root@usn-ramboot ~]# uname -r 3.10.0-327.13.1.el7_lustre.x86_64 [root@usn-ramboot ~]# df -h / Filesystem Size Used Avail Use% Mounted on tmpfs 8.0G 4.3G 3.8G 53% / [root@usn-ramboot ~]# [root@usn-ramboot ~]# rpm -qa | grep lustre kernel-3.10.0-327.13.1.el7_lustre.x86_64 kernel-tools-3.10.0-327.13.1.el7_lustre.x86_64 kernel-devel-3.10.0-327.13.1.el7_lustre.x86_64 lustre-2.8.53_1_g34dada1-3.10.0_327.13.1.el7_lustre.x86_64.x86_64 kernel-tools-libs-devel-3.10.0-327.13.1.el7_lustre.x86_64 lustre-osd-ldiskfs-mount-2.8.53_1_g34dada1-3.10.0_327.13.1.el7_lustre.x86_64.x86_64 kernel-headers-3.10.0-327.13.1.el7_lustre.x86_64 lustre-osd-ldiskfs-2.8.53_1_g34dada1-3.10.0_327.13.1.el7_lustre.x86_64.x86_64 kernel-tools-libs-3.10.0-327.13.1.el7_lustre.x86_64 lustre-modules-2.8.53_1_g34dada1-3.10.0_327.13.1.el7_lustre.x86_64.x86_64 This means that we can run Lustre 2.8.x atop Unison. Still pre-alpha, as … Read moreSuccess with rambooted Lustre v2.8.53 for #HPC #storage

Its not perfect, but we have CentOS/RHEL 7.2 and Lustre integrated into SIOS now

Lustre is infamous for its kernel specificity, and it is, sadly, quite problematic to get running on a modern kernel (3.18+). This has implications for quite a large number of things, including whole subsystems with a partial back-porting to earlier kernels … which quite often misses very critical bits for stability/performance. I am not a … Read moreIts not perfect, but we have CentOS/RHEL 7.2 and Lustre integrated into SIOS now