Interesting …

At Storagemojo, Robin Harris has an interesting take on the evolution of storage systems.

SNW is the past. NAB is the future.
Storage is in the midst of a massive transition from an IOPS focus to a bandwidth focus.

This is interesting to us, given how much bandwidth we can provide from our JackRabbit storage systems. The issue for us is finding the right protocol to pull 750 MB/s per small unit, and distribute this to consumers of the data.

Viewed 14055 times by 3314 viewers

2 thoughts on “Interesting …

  1. Oh hell yes… I got the hardware but sure is a pain to pull that data across the wire.. (even 10gige)

  2. We did spend quite a bit of time on this. Looks like quite a bit of performance is left on the floor.

    We will be working on the NFSoverRDMA for IB, and trying to figure out the 10GbE stack issues. It looks like most of the 10 GbE are really good with threaded codes … they can overlap lots of packet processing. Just not single threads.

    IB on the other hand has a really poor TCPIP implementation … it leaves 70+% of its performance on the floor.

    It looks like the design points for the stacks (and possibly the silicon) was on really small messages, low latency, and lots of them. Not large simple TCP type things.

    AoE or FCoE might be good for getting the performance out of the nets. There are problems with those for other reasons, but not as bad as this.

    GlusterFS also looks good over IB. It used the SDP. I wish there was a way to make it go faster on 10GbE.

    Well, we can build single boxen up to 32 cores, and 256 GB ram. Forces people to keep computing co-located with data storage. I don’t like that. We need a better way. Worth some thought.

Comments are closed.