Ruminations on performance … the possible, the impossible, and things in between

We are often asked what differentiates us from our competition. One of the more important aspects is our raw uncompromising performance. Our systems are fast. Not in a marketing number sense (I’ll get to this in a minute). But in real application fast. We take a no holds barred approach to performance design. And our customers do see this.
Design, implementation, … these are critical elements. Software stack, tuning …. these are critical. Performance density … how much performance you can squeeze in per rack unit, is getting to be just as important as capacity density … how much storage you can squeeze in per dollar.
And this gets to a definition of what I call a “marketing number” or “marketing benchmarks”. A marketing benchmark is a number whose derivation is in terms of optimal theoretical conditions, with assumptions that are rarely if ever true in the real world, often using unrealistic configurations for end users.
That is, marketing uses their marketing numbers to make (often specious) claims on performance density and raw performance. Its a shame that they are not meaningful. Every time you pick up a component of some system, you see these numbers. For example, did you know all SATA run at 3Gb/s? My gosh … they are fast !!! 375 MB/s that is …

… except that they don’t. 3Gb/s is the interface speed. Not the disk speed. If your data goes cache to cache (disk to machine) is it possible to hit 3Gb/s? Possibly, assuming the system isn’t doing other things.
And herein lies the problem. We have to fight against these (obviously unrealistic) marketing numbers. It would help us tremendously if, next time you get one, you say “prove it” to the vendor. I’d really like to see their 8 port SATA card get 2.4 GB/s. No … really. I’d like to see this.
Yet this is what passes for “measurement”.
What we do is … curiously … measure. And then report. Yeah, I know, our benchmark report is hideously out of date. We are working on it. Will have a new one up in a month or so.
But our measurements are … indeed … measurements. So when we hear some of the more specious claims … one of my favorite being a 3GB/s to disk from one of our (likely former as they were recently purchased) competitors, and I ask the folks who fell for this marketing to tell me what they actually measure, its between 1/10th to 1/5th of that number. Publicly reported performance bears this out.
Sure its possible to get 3 GB/s using their setup. If you don’t mind a RAID0, with tiny reads coming from disk cache to main memory, assuming nothing else is running.
Please, raise your hand if this is your use case. I’d like to count how many people use this config. Really. Please respond here if you do. Obviously I am missing something quite important.
Now compare those 48 drives in a RAID0 to 24 drives in two RAID6’s with some hot spares. How can half the number of drives more than double the actual measured performance of the other solution? Its impossible … right?
Not really, because it does happen.
Other tests we have run recently on our hardware have included Microsoft’s SQLIO benchmark. We have some good results from this. Very good results. I’ll be reporting on them soon in our new benchmark report.
Of course we are asked for even more. A SATA drive will max out close to 100 IOPs (100+/-10) for a single read/write scenario. So 24 drives are physically incapable of more than about 2400 IOPs. Its impossible to get higher.
Well, again, when we measure something more that 5900 IOPs (8k random reads for a 40GB file on a 24GB server) for a 24 bay unit (under Windows 2008 R2 Server x64), we can see the benefit to NCQ and elevators in the disks/RAIDs. 100+/-10 physical IOPs, and at least another 100+ IOPs in flight, that the queuing can re-order and graduate.
Marketing numbers are frequently impossible to obtain in practice. Measurement …. real meaningful measurement … tells you whats possible. Design focus on uncompromising performance is what makes our systems perform.
In this day and age, where performance density counts as much as storage density, and reasonable prices for all of these … yeah … you can say we have significant differentiation. Its not the only thing that makes us different, and better. But it helps not having to purchase many large expensive units to do what we can do in fewer, far less expensive units.

2 thoughts on “Ruminations on performance … the possible, the impossible, and things in between”

  1. “A SATA drive will max out close to 100 IOPs…”
    You mean the average consumer-grade SATA spindle hard disk?

  2. @tb
    The enterprise class do. The consumer-grade units are physically the same, so modulo firmware bits, this will be quite similar in number for them.
    We can measure it using some rigs here if needed.

Comments are closed.