JR5: Marathon run

For those benchmark pr0n viewers … here is Scalable Informatics Inc. JackRabbit JR5 unit with 48 drives. Simple benchmark. How long does it take you to write 1TB. How about 524 seconds?

[root@jr5 ~]# !498
mpirun -np 8 ./io-bm.exe -n 1024 -f /data/file -w -d -s -v
--------------------------------------------------------------------------
[[8861,1],3]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:
Module: OpenFabrics (openib)
  Host: jr5.scalableinformatics.com
Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
[tid=2] offset = 274877906944
[tid=3] offset = 412316860416
[tid=4] offset = 549755813888
[tid=6] offset = 824633720832
[tid=4] file name = /data/file
[tid=3] file name = /data/file
N=1024 gigabytes will be written in total
each thread will output 128.000 gigabytes
page size                     ... 4096 bytes
number of elements per buffer ... 2097152
number of buffers per file    ... 8192
[tid=0] offset = 0
[tid=5] offset = 687194767360
[tid=7] offset = 962072674304
[tid=2] file name = /data/file
[tid=1] offset = 137438953472
[tid=6] file name = /data/file
[tid=0] file name = /data/file
[tid=5] file name = /data/file
[tid=7] file name = /data/file
[tid=1] file name = /data/file
[jr5.scalableinformatics.com:18852] 7 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics
[jr5.scalableinformatics.com:18852] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
Thread=1: time = 524.425s IO bandwidth = 249.935 MB/s
Thread=2: time = 524.523s IO bandwidth = 249.888 MB/s
Thread=3: time = 524.539s IO bandwidth = 249.881 MB/s
Thread=4: time = 524.543s IO bandwidth = 249.878 MB/s
Thread=5: time = 524.496s IO bandwidth = 249.901 MB/s
Thread=6: time = 524.514s IO bandwidth = 249.892 MB/s
Thread=7: time = 524.506s IO bandwidth = 249.896 MB/s
Thread=0: time = 524.507s IO bandwidth = 249.896 MB/s
Naive linear bandwidth summation = 1999.167 MB/s
More precise calculation of Bandwidth = 1999.027 MB/s
[root@jr5 ~]# ls -alF /data
total 1073742792
drwxr-xr-x  2 root root            17 Jul 22 12:01 ./
drwxr-xr-x 25 root root          4096 Jul 21 06:31 ../
-rwx------  1 root root 1099511627776 Jul 22 12:10 file*

This is RAID6, not RAID0 like some other people report their benchmark numbers with. We are almost done with testing.