disabling Perc 5/i write cache/fixing IO scheduling?
Howdy. I just inherited a Dell 2950 with a Perc 5/i.
I really don't want to use it, but from all the reading I did, there is no IT/unraid mode for it, so I was stuck setting the 6 2TB drives as 6 independent raid0 drives.
I wish I could just bypass the damn card and connect the drives directly to a sata card, but the case and backplane do not seem to make this possible.
So, the 6 drives show up in linux, I partitioned them and created various software raid slices on top (raid1, raid5 and raid6).
They work fine, but there is something very wrong with a block layer somewhere. If I send a bunch of writes, the IO scheduler seems to introduce terrible latency where my whole system hangs for a few seconds trying to read simple binaries while from what I can tell, the I/O platters spend all their time writing the backlog of what's being sent.
I tried to disable the write cache to let linux and its 32GB of RAM, do it better, but I didn't see a real improvement:
newmagic:~# megacli -LDSetProp -DisDskCache -L0 -a0 (0,1,2,3,4,5)
newmagic:~# megacli -LDGetProp -DskCache -Lall -a0
Adapter 0-VD 0(target id: 0): Disk Write Cache : Disabled
Adapter 0-VD 1(target id: 1): Disk Write Cache : Disabled
Adapter 0-VD 2(target id: 2): Disk Write Cache : Disabled
Adapter 0-VD 3(target id: 3): Disk Write Cache : Disabled
Adapter 0-VD 4(target id: 4): Disk Write Cache : Disabled
Adapter 0-VD 5(target id: 5): Disk Write Cache : Disabled
I'm also only getting about 5MB/s sustained write speed, which is pathetic. I have lots of servers with normal sata cards, software raid, and I get 50 to 100MB/s normally.
I'm hoping the Perc 5/i card is not _that_ bad?
If I access drives plugged directly into the motherboard's sata port, I get perfect speed. I've also added an SSD with bcache to frontload one of the raid arrays that is so slow, and sure enough, it becomes usuable.
Still, I'd like to fix the underlying problem.
megacli -LdPdInfo -a0 output for the first drive below. Any idea of what I could have missed that causes this?
Number of Virtual Disks: 6
Virtual Drive: 0 (Target Id: 0)
RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0
Size : 1.818 TB
Sector Size : 512
Parity Size : 0
State : Optimal
Strip Size : 64 KB
Number Of Drives : 1
Span Depth : 1
Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy : Disabled
Encryption Type : None
Is VD Cached: No
Number of Spans: 1
Span: 0 - Number of PDs: 1
PD: 0 Information
Enclosure Device ID: 8
Slot Number: 0
Drive's position: DiskGroup: 0, Span: 0, Arm: 0
Enclosure position: N/A
Device Id: 0
Sequence Number: 2
Media Error Count: 0
Other Error Count: 1
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA
Raw Size: 1.819 TB [0xe8e088b0 Sectors]
Non Coerced Size: 1.818 TB [0xe8d088b0 Sectors]
Coerced Size: 1.818 TB [0xe8d00000 Sectors]
Sector Size: 0
Firmware state: Online, Spun Up
Device Firmware Level: AB50
Shield Counter: 0
Successful diagnostics completion on : N/A
Connected Port Number: 0
Inquiry Data: WD-WMAZA0374092WDC WD20EARS-00MVWB0 50.0AB50
FDE Capable: Not Capable
FDE Enable: Disable
Needs EKM Attention: No
Foreign State: None
Device Speed: Unknown
Link Speed: Unknown
Media Type: Hard Disk Device
Drive Temperature : N/A
PI Eligibility: No
Drive is formatted for PI information: No
PI: No PI
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No