A baseline before tuning

Building a large JackRabbit. 2 raid controllers, quite a few other goodies (CF boot!). Doing some testing for burn in, including our “simple-write” benchmark from a few posts ago. I haven’t done any tuning yet.

Honest.

The throttle is not cracked wide open, the JackRabbit is not running at full potential. It doesn’t even have its full complement of RAM, or cache. The folks shipping those to us shipped us the wrong RAM, so these are 2x 2GB sticks that we had for our testing unit.

I just want to make sure these comments are in place, before I present the numbers. I started it up, constructed the RAIDs, built a file system, and now I am running the simple-write test.

root@jr1:~# ./simple-write.bash
+ sync
+ echo -n 'start at '
start at + date
Thu Feb 14 08:17:35 EST 2008
+ dd if=/dev/zero of=/big/local.file.9635 bs=8388608 count=100000 oflag=direct

Right now, dstat is reporting:

<code>
root@jr1:~# dstat
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw 
  0   7  74  17   0   2|   0  1188M|   0     0 |   0     0 |9015  2705 
  0   7  74  17   0   1|   0  1194M| 724B 1728B|   0     0 |9059  2691 
  0   7  74  18   0   1|   0  1157M|   0     0 |   0     0 |8797  2683 
  0   7  72  19   0   2|   0  1192M| 256B  808B|   0     0 |9056  2709 
</code>

vmstat 1 is reporting

<code>
root@jr1:~# vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  1      0 3212840   3056 604156    0    0    25  4596  281  173  0  0 99  0
 0  1      0 3212652   3056 604156    0    0     0 1220928 9053 2692  0  9 74 17
 0  1      0 3212728   3056 604156    0    0     0 1138976 8529 2658  0  8 74 18
 1  0      0 3212692   3064 604148    0    0     0 1115677 8348 2662  0  8 74 18
 2  0      0 3212736   3064 604156    0    0     0 1195712 8895 2663  0  8 74 17
 1  0      0 3212836   3064 604156    0    0     0 1220320 9052 2694  0  8 74 17
 1  0      0 3212688   3064 604156    0    0     0 1209704 8945 2685  0  9 74 17
</code>

atop is reporting 45% utilization of each RAID card.

Yeah, ok, you want the numbers.

Fine. Here they are.


root@jr1:~# ./simple-write.bash
+ sync
+ echo -n 'start at '
start at + date
Thu Feb 14 08:17:35 EST 2008
+ dd if=/dev/zero of=/big/local.file.9635 bs=8388608 count=100000 oflag=direct
100000+0 records in
100000+0 records out
838860800000 bytes (839 GB) copied, 711.93 seconds, 1.2 GB/s
+ sync
+ echo -n 'stop at '
stop at + date
Thu Feb 14 08:29:27 EST 2008

This is running off of our CF load

root@jr1:~# parted /dev/sdc print
Model: ATA SanDisk SDCFX3-4 (scsi)
Disk /dev/sdc: 4110MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 3997MB 3997MB primary ext3 boot
2 3997MB 4104MB 107MB primary xfs

and the unit doesn’t have much ram or cache.

top - 08:32:58 up  8:38,  6 users,  load average: 0.02, 0.44, 0.44
Tasks: 161 total,   1 running, 160 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
<strong>Mem:   4043716k total,</strong>   831552k used,  3212164k free,     3152k buffers
Swap:        0k total,        0k used,        0k free,   604216k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
 5591 root      19  -1 90244  14m 5876 S    1  0.4   2:42.51 Xorg               
 9662 root      20   0 19092 1372  976 R    1  0.0   0:00.11 top                
    1 root      20   0  3980  844  576 S    0  0.0   0:03.32 init               
    2 root      15  -5     0    0    0 S    0  0.0   0:00.00 kthreadd           

It has 24 of 48 possible disks. Yes, this performance was across 24 disks.

2x RAID6 striped.

pre-tuning bonnie data


root@jr1:~# bonnie++ -u root -d /big -f
Using uid:0, gid:0.
Writing intelligently...done
Rewriting...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
jr1 8G 629915 96 354727 82 787563 92 619.0 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 18682 97 +++++ +++ 16978 87 18539 97 +++++ +++ 17635 97
jr1,8G,,,629915,96,354727,82,,,787563,92,619.0,1,16,18682,97,+++++,+++,16978,87,18539,97,+++++,+++,17635,97

This machine comes in well under $20k.

I have some tuning I want to do today. Take it out to the test track. Open the throttle.

Wide.

Viewed 6427 times by 1174 viewers

Facebooktwittergoogle_plusredditpinterestlinkedinmail