post #1 of 1
Thread Starter 
I'm trying to troubleshoot the horrid (current) performance of my Perc 5/i
Previously I was able to saturate a PCI-e 2.0 x4 lane, and currently my performance is a dismal 97mb/s read and 40mb/s write (direct r/w not cached)

Here are the "benchmarks"
Code:
hdparm -Tt /dev/sda
/dev/sda:
 Timing cached reads:   2596 MB in  2.01 seconds = 1292.49 MB/sec
 Timing buffered disk reads: 290 MB in  3.10 seconds =  93.54 MB/sec

 sync;time bash -c "(dd if=/dev/zero of=bf bs=8k count=500000; sync)"
500000+0 records in
500000+0 records out
4096000000 bytes (4.1 GB) copied, 82.178 s, 49.8 MB/s

real    1m24.891s
user    0m4.327s
sys     0m16.117s

Details on the controller:
Perc 5/i with 1GB Dimm installed (only 512mb usable due to SoC limitation)
80mm fan directly attached

Details on the array:

7xWD Caviar Black 2tb in raid 5
1xWD Caviar Black 2tb dedicated Hot Spare

Details on the system:
AMD A8-3870k APU
16GB DDR3-1600
64gb Samsung 830 SSD OS
128gb OCZ Vertex 4 SSD Game Servers

Details on the network:
4000 MTU throughout except router SoC and Modem (1500 MTU)
OpenWRT at the heart of the router

Useful output from the MegaCLI utility:
Current VD Status: (Click to show)
Code:
Adapter 0 -- Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :Massives
RAID Level          : Primary-5, Secondary-0, RAID Level Qualifier-3
Size                : 10.913 TB
Parity Size         : 1.818 TB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives    : 7
Span Depth          : 1
Default Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU
Current Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Is VD Cached: No



Exit Code: 0x00
Full adapter info: (Click to show)
Code:
No
Direct PD Mapping                : No
BIOS Enumerate VDs               : No
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : No
Maintain PD Fail History         : No
Disable Puncturing               : No
Zero Based Enclosure Enumeration : No
PreBoot CLI Enabled              : Yes
LED Show Drive Activity          : No
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : No
Enable Led Header                : No
Delay during POST                : 0 
EnableCrashDump                  : No
Disable Online Controller Reset  : No
EnableLDBBM                      : No
Un-Certified Hard Disk Drives    : Block
Treat Single span R1E as R10     : No
Max LD per array                 : 16
Power Saving option              : All power saving options are enabled
Default spin down time in minutes: 0 
Enable JBOD                      : No
Time taken to detect CME         : 60s

Exit Code: 0x00
iostat output (showing heavy usage) (Click to show)
Code:
Linux 3.10.1-2-ck (array)    07/31/2013      _x86_64_        (4 CPU)

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sda               0.39   769.91  102.80   14.57  1604.61  3138.41    80.82     5.09   43.41    3.80  322.80   2.79  32.75
sdb               0.19     9.76   46.66   13.99  6625.48  3030.81   318.44     2.08   34.30    0.79  146.12   0.62   3.77
sdc               0.09     0.16    0.61    0.30    16.68    19.49    79.96     0.00    1.01    0.71    1.60   0.49   0.04

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sda               0.40 13782.80  156.60  168.60  2120.00 47956.00   307.97    15.53   43.83    7.69   77.40   2.55  82.88
sdb               0.00     4.60  225.40    0.40 57651.20    20.00   510.82     0.19    0.83    0.82    5.00   0.72  16.26
sdc               0.00     0.00    0.60    0.00     7.20     0.00    24.00     0.00    0.00    0.00    0.00   0.00   0.00

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sda               0.40  3758.20  136.80   50.60  1833.60 14360.00   172.82     9.47   29.48   10.55   80.64   4.87  91.26
sdb               0.00    29.00  155.80    0.80 39680.00   119.20   508.29     0.13    0.84    0.83    2.50   0.74  11.54
sdc               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00


This has gotten ridiculous... I just did a file copy from the array to a different folder on the array from the command line utilizing an application that shows realtime file transfer speed. Top write speed was momentary 150mb/s (1/4th the normal write speed of the array) bottom write speed was a terrifying 200kb/s. Moving a 1gb file on this array *was* instantaneous before.

Edit: flashed the latest LSi firmware image (had latest dell) and things improved but they still aren't good. 156mb/s sustained transfer speeds.
Edited by Xaero252 - 7/31/13 at 1:33pm
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
Core i7 920 D0 4.2ghz HT (1.3625v) Asus R3E 2xGTX 460 (non SLi, no overclock) 6x2gb G.skill @ 6-8-6-24-1T 
Hard DriveOptical DriveOSMonitor
WD-VR 300GBx1, 2xWD 1tb,2x60gb Agility Some crappy combo burner... Arch x64 3xDell U2410f rev A02 
KeyboardPowerCaseMouse
X-Armor U9BL TT Toughpower 1200w (NTB more efficient) Mountain Mods Pinnacle 24 CYO Roccat Kone (R.I.P. A4Tech x7) 
Mouse Pad
Steelpad Experience I-1 
  hide details  
Reply