The TB sprint updated …

Previous results here. 12.4TB/hour.
A new JackRabbit unit with some updates.
New results: 1TB written in 228.2 seconds. 15.8TB/hour writes

Run status group 0 (all jobs):
  WRITE: io=1024.6GB, aggrb=4597.1MB/s, minb=4597.1MB/s, maxb=4597.1MB/s, mint=228167msec, maxt=228167msec

and for the reads …

Run status group 0 (all jobs):
   READ: io=1024.6GB, aggrb=5341.9MB/s, minb=5341.9MB/s, maxb=5341.9MB/s, mint=196392msec, maxt=196392msec

This is 18.3TB/hour reads.
Writing 1PB on this machine would take almost 65 hours. So if we could break the writes across 65 machines (9 racks), we could write 1PB in 1 hour.
Somewhat of an improvement over the last measurement. Reading that 1PB would take 56 hours, or, from 7 racks of 8 machines each, 1 hour.
1PB/hour is 1000TB/hour or 1TB every 3.6 seconds. Thats ~278GB/s
This is quite reachable. 1TB/s, or 3.6PB/hour is also quite doable, without an infinite amount of rack space, power, cooling, and capital.
And no, there are no PCIe Flash cards here, and this is spinning rust on the server. 108TB usable of spinning rust. 65 machines would provide for 7PB, so at 1PB/hour, we could read/write that 7PB rather quickly. If we bumped the machine count up by 3.6, the total storage would be 25.3PB and we’d have 3.6PB/hour read rate.
Nice … just need someone to send in a PO for this now …

3 thoughts on “The TB sprint updated …”

  1. Did you happen to capture the CPU utilization during this? hmmm. I wonder… Are there any typical read+transform+write benchmarks? I can think of a few different levels of transform that could be interesting.

  2. @Jason
    Utilization was low … below 20%. We are running our burn in load (2/3 of it anyway) where we have an 8 proc GAMESS run going while doing disk IO and checking for CRC errors. Very system intensive. Does a good job of finding issues in weak system components.
    Missing the IB or 10GbE card for some reason. Will remedy that when it arrives.

Comments are closed.