# Final sprint before shipping

Now that I (think) understand most of the major issues here, and I can be reasonably sure that I have a good grasp of the tuning, I think I want to take it out on the test track and give it one final once over.
Lets open the throttle. Wide.

I can tune the IO scheduler, number of outstanding IO requests (for sorting), various buffer cache, and the works. I now have the clock left alone (need to set it that way by default), so it is running full speed.
Sanity check: How does buffer cache look?
 root@jackrabbit1:~# hdparm -tT /dev/md0 /dev/md0: Timing cached reads: 4908 MB in 2.00 seconds = 2455.69 MB/sec Timing buffered disk reads: 2060 MB in 3.00 seconds = 685.95 MB/sec
Good-n-fast. None of this 1.1 GB/s stuff we see when powernow is on.
Quick-n-dirty bonnie++
 Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP jackrabbit1 32096M 568841 85 279055 54 834536 82 491.6 0 jackrabbit1,32096M,,,568841,85,279055,54,,,834536,82,491.6,0,,,,,,,,,,,,,
Not bad. Saw as high as 950MB/s sequential input in testing. Saw 600+ MB/s in testing. Some additional IO tuning is possible (deadline will favor writes over reads).
Since we have 2GB RAID cache, 1 GB per card, we need to let this get past the RAID cache boundary. This was one of my objections with some others testing we had seen in the past. Their test cases were entirely cache bound, and therefore effectively meaningless as an indicator of performance (other than cache performance). Lets get out of the RAID cache regime, into the region where it needs to spill to disk. This hits the power curve hard. If your performance falls off a ledge at the size of your RAID cache, you have … problems.
For laughs, I grabbed a snapshot of a few seconds of dstat, running in a window above iozone.

—-total-cpu-usage—- -dsk/total—-dsk/sda—–dsk/sdc– -net/total- —paging– —system–
0 27 71 0 0 2| 0 400M: 0 200M: 0 200M| 529B 1176B| 0 0 |2946 772
0 11 75 12 0 2| 0 964M: 0 483M: 0 482M| 192B 484B| 0 0 |6421 967
0 19 68 11 0 1| 0 684M: 0 341M: 0 342M| 192B 484B| 0 0 |4722 379
0 25 75 0 0 0| 0 80M: 0 40M: 0 40M| 126B 370B| 0 0 | 777 400
0 22 74 4 0 0| 0 642M: 0 322M: 0 320M| 340B 386B| 0 0 |4498 930
0 2 75 21 0 2| 0 1326M: 0 662M: 0 664M| 126B 386B| 0 0 |8902 564

Each adapter can provide about 780 MB/s during writes. We are mostly filling 2 of these up. We can add more.
The corresponding line from IOzone is
2097152 1024 699383 938361 1333639 1344250 1342067 1147908 1343132 1154050 1342384 555308 708427 1328317 1340149
I would argue that we are still in cache. This is bursty, and still only 2GB of IO.
Looking at a few lines of dstat while we are at the 4 GB size (64k record size), I see

—-total-cpu-usage—- -dsk/total—-dsk/sda—–dsk/sdc– -net/total- —paging– —system–