So, I always thought this was a load of crap... until this week.
I recently upgraded from 1TB drives to 2TB drives in my pool. After the migration was complete I ran a simple dd benchmark to make sure things were good and saw this:
Code:
I replaced the cable, same thing. The power connector, same thing. So then I replaced the drive and it got better. But once I put the new drive in the same slot as the old drive things got bad again. Very puzzling. Then I remembered the youtube video of (I think it was Brandon Gregg) literally shouting at hard drives and making service times spike during I/O ops. So I pulled the drive cage out, and sure enough one of the screws holding the drive had not been tightened enough. I pulled all four drives out their cage and put them back in a different order and make sure they were nice and tight. That alone literally solved the problem:
Before:
Code:
After:
Code:
I recently upgraded from 1TB drives to 2TB drives in my pool. After the migration was complete I ran a simple dd benchmark to make sure things were good and saw this:
Code:
Code:
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 125.6 0.0 13686.5 1.4 0.2 11.1 1.3 18 16 c6t0d0
0.0 126.2 0.0 13835.3 1.4 0.2 11.3 1.3 18 16 c6t2d0
0.0 125.0 0.0 13688.1 1.4 0.2 11.1 1.3 18 16 c6t3d0
0.0 108.6 0.0 11309.7 0.0 7.5 0.0 69.4 0 82 c6t7d0 <------- Yikes!!!
0.0 126.0 0.0 13834.5 0.0 1.6 0.0 12.4 1 18 c7t0d0
0.0 128.8 0.0 13840.9 0.0 1.5 0.1 12.0 1 18 c7t2d0
0.0 124.0 0.0 13684.1 0.0 2.3 0.0 18.2 0 24 c7t3d0
0.0 127.0 0.0 13835.3 1.3 0.1 10.0 1.1 17 14 c7t4d0
0.0 125.6 0.0 13685.7 1.3 0.2 10.6 1.3 17 16 c6t1d0
0.0 123.2 0.0 13684.9 0.0 2.1 0.0 16.8 0 24 c7t1d0
Before:
Code:
Code:
# ./dd_bench mp2
+ dd if=/dev/zero of=//mp2/dd.tst bs=1024000 count=10000
10000+0 records in
10000+0 records out
10240000000 bytes (10 GB) copied, 68.8307 s, 149 MB/s
real 1m8.84s
user 0m0.04s
sys 0m14.70s
+ dd if=//mp2/dd.tst of=/dev/null bs=1024000
10000+0 records in
10000+0 records out
10240000000 bytes (10 GB) copied, 20.8435 s, 491 MB/s
real 0m20.86s
user 0m0.04s
sys 0m10.88s
Code:
Code:
# ./dd_bench mp2
+ dd if=/dev/zero of=//mp2/dd.tst bs=1024000 count=10000
10000+0 records in
10000+0 records out
10240000000 bytes (10 GB) copied, 22.0691 s, 464 MB/s
real 0m22.13s
user 0m0.04s
sys 0m15.53s
+ dd if=//mp2/dd.tst of=/dev/null bs=1024000
10000+0 records in
10000+0 records out
10240000000 bytes (10 GB) copied, 18.0732 s, 567 MB/s
real 0m18.08s
user 0m0.04s
sys 0m10.61s