#sc09 [T-0] NFS over 10GbE at 1 GB/s

Just thought we’d run some nice little performance tests using io-bm (yeah, I know, I have to release it already). Remember, this is booth 635 if you want us to do this live … Here is a write, from the Pegasus, to the JR4. Over a single 10GbE link.

scalable@pegasus:~$ /opt/openmpi133/bin/mpirun -v -np 4 `pwd`/io-bm.exe -n 32 -w -f /data/jr4/nfs/io-bm-test
...
Thread=3: time = 32.682s IO bandwidth = 250.655 MB/s
Thread=0: time = 32.669s IO bandwidth = 250.761 MB/s
Thread=2: time = 32.643s IO bandwidth = 250.955 MB/s
Thread=1: time = 32.656s IO bandwidth = 250.856 MB/s
Naive linear bandwidth summation = 1003.226 MB/s
More precise calculation of Bandwidth = 1002.619 MB/s

and, for laughs … the read

scalable@pegasus:~$ /opt/openmpi133/bin/mpirun -v -np 4 `pwd`/io-bm.exe -n 32 -r -f /data/jr4/nfs/io-bm-test
libibverbs: Fatal: couldn't read uverbs ABI version.
--------------------------------------------------------------------------
[[34320,1],1]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:
Module: OpenFabrics (openib)
  Host: pegasus
Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
libibverbs: Fatal: couldn't read uverbs ABI version.
libibverbs: Fatal: couldn't read uverbs ABI version.
libibverbs: Fatal: couldn't read uverbs ABI version.
[pegasus:23119] 3 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics
[pegasus:23119] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
Thread=2: time = 29.533s IO bandwidth = 277.382 MB/s
Thread=3: time = 29.808s IO bandwidth = 274.829 MB/s
Thread=0: time = 29.845s IO bandwidth = 274.487 MB/s
Naive linear bandwidth summation = 1101.185 MB/s
More precise calculation of Bandwidth = 1097.944 MB/s
Thread=1: time = 29.845s IO bandwidth = 274.486 MB/s

This is a 32 GB read and write. I’ll do a larger read and write for completeness.