On the difference between marketing numbers, and measured numbers ...
By joe
- 1 minutes read - 185 wordsI should define what I mean by marketing numbers. These are best effort benchmarking numbers assuming the best of all possible test cases, with equipment functioning solely for the benchmark test purposes. These are not benchmark results you will normally achieve in practice. They represent an extrema in performance. Measured benchmark numbers are sensitive to many factors. You need to perform several tests, make sure you can construct an “average” and make an assumption about the shape of the distribution around that average. You can guess to first approximation that you have a Gaussian distribution around your mean, in which case you can use measures such as standard deviation to define an approximate error bar. The size of your error bar will be very much a function of the load on the system, and many other factors. I mention this, as we are again comparing real measured performance against these marketing numbers at various opportunities. Sure, a RAID0 across 48 drives can do lots of IOPs. Especially with an SSD cache. But how is the performance with real applications which aren’t all that cache friendly?