10,000 drives for 80 GB/s?

Just read this today at the Reg. Argonne has lots of GPUs, and disks. Separated with a big old IB fabric.

Hmmm…. 10,000 disks to get 80,000,000,000 B/s.


Delivering that to GPUs.


I think we can do better.

Just 80 of our JR4 units can certainly read and write at that speed, and we can get them 2 GPUs in the same box as the disks. 160 GPUs (Tesla’s at that). 1920 disks.

If I believe these numbers from the article, 80 GB/s for 10,000 drives means, roughly, 8 MB/s per disk.

In our case, this 80 GB/s would be using 40 MB/s per disk.

We can drive them more efficiently, and have demonstrated a sustained 1.6 GB/s. Ok, so we’d only need 50 of these servers, the extra 30 are engineering margin. We come closer to 64 MB/s, which is within what we normally observe.


Viewed 6420 times by 1225 viewers