Overclock.net › Forums › Specialty Builds › Servers › ZFS storage advice
New Posts  All Forums:Forum Nav:

ZFS storage advice - Page 2

post #11 of 13
@tycoonbob

Cheers for the info. smile.gif Concerning controllers, as much as a SATA II controller can push (for example) a sequential 234MB/s per channel, there's not necessarily any guarantee that it can push 234MB/s of random I/O though is there, since the workload is different? Also, wouldn't that same principle apply to even proper SAS RAID controllers such as LSI, Areca etc?
Mythica
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel i3 530 Gigabyte GA-H55M-D2H Palit nVidia GT430 Corsair Dominator 4GB TW3X4G1333C9A 
Hard DriveHard DriveOSMonitor
Western Digital Scorpio Black Samsung HD204UI Linux Mint 17 HP L1800 
KeyboardPowerCaseMouse
Logitech Basic Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Mythica
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel i3 530 Gigabyte GA-H55M-D2H Palit nVidia GT430 Corsair Dominator 4GB TW3X4G1333C9A 
Hard DriveHard DriveOSMonitor
Western Digital Scorpio Black Samsung HD204UI Linux Mint 17 HP L1800 
KeyboardPowerCaseMouse
Logitech Basic Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
post #12 of 13
Thread Starter 
Hey guys. An update, post-deployment. I got the HDDs, seller actually sent me 95 drives and I think 5 or 6 of them were actually 73GB models that got mixed in. Yay for me.

Regarding the MSA70... This thing is awesome. So awesome that I bought a second one. The unit has SAS expanders in it, but they appear to be arranged properly to give good performance with mechanical HDDs. Although I'm sure if I filled up a MSA70 with SSDs it would bottleneck at the 4x SAS link.

I tried several different schemes for arranging drives and eventually settled on 12 x 2-drive mirrors (RAID10) using ZFS, with a hot-spare. This gives me fantastic performance (so far). I haven't really done any reliable benchmarking to get *true facts* about the performance but I have windows server 2012 VMs running on Xen Cloud Platform with NFS-based storage, and the VMs will cold boot to login screen in about 14-18 seconds. This is easily as fast as i've seen with high-end SATA III SSDs, and keep in mind i'm still stuck with gigabit ethernet, no 10GbE or infiniband yet.

My second MSA70 enclosure just shipped out today, due to be delivered by thursday or friday. When I get it, i'm planning on reorganizing my drives so that I have 24 x 2-drive mirrors w/ 1 hot-spare per enclosure, but with the mirrors split over each enclosure. Theoretically, if the array was healthy, i could unplug or otherwise lose an entire enclosure and the array would stay online. Raid-1 of JBOD enclosures makes me happy.

Again, no hard numbers yet but I did a zpool scrub just a bit ago that completed in less than 4 minutes, with a bit over 80GiB of actual data on the pool.

Regarding the actual MSA70 enclosure... it's very well laid out, it's quiet enough that I would be comfortable with having it in a working environment, and it seems to be easily the cheapest way of adding lots of bays to a system right now. There's also the MSA60 which takes 12 x 3.5" drives in a 2U form, and they're available on ebay for around $150 including shipping.
Workstation
(14 items)
 
  
CPUMotherboardGraphicsRAM
Core i7-920 D0 EVGA X58 SLI Micro BFG GeForce GTS 250 OC 1GB 24GB (6x4GB) G.Skill Sniper SR2 
Hard DriveOptical DriveOSMonitor
2x WD Raptor 74GB 10K 8MB (RAID0) LG BD/HD-DVD/DVD-RW Combo Windows 7 Ultimate x64 (RTM) Gateway FHD2400 1920x1200 2ms 
MonitorKeyboardPowerCase
Dell UltraSharp 3008WFP 2560x1600 8ms Deck Legend Frost XFX 850W Modular 80PLUS Silver Cooler Master HAF 932 
MouseMouse Pad
Logitech G9 12"x12"x3/8" Polyethelyne 
  hide details  
Reply
Workstation
(14 items)
 
  
CPUMotherboardGraphicsRAM
Core i7-920 D0 EVGA X58 SLI Micro BFG GeForce GTS 250 OC 1GB 24GB (6x4GB) G.Skill Sniper SR2 
Hard DriveOptical DriveOSMonitor
2x WD Raptor 74GB 10K 8MB (RAID0) LG BD/HD-DVD/DVD-RW Combo Windows 7 Ultimate x64 (RTM) Gateway FHD2400 1920x1200 2ms 
MonitorKeyboardPowerCase
Dell UltraSharp 3008WFP 2560x1600 8ms Deck Legend Frost XFX 850W Modular 80PLUS Silver Cooler Master HAF 932 
MouseMouse Pad
Logitech G9 12"x12"x3/8" Polyethelyne 
  hide details  
Reply
post #13 of 13
Sorry I didn't see this sooner, but I would recommend Striped/Mirrored vDev's. I've used all iterations of ZFS, and regardless of implementation, amount of RAM in the system, amount of SSD, etc, I've always been disappointed with anything other than mirrored/striped. Z2 seems like overkill with 32GB drives at 10kRPM - rebuild times on those should be measured in minutes with that setup.

Also, for a little bit of self-promoting - if you ever want to do some additional reading on ZFS, check out my blog at zfsbuild.com Kinda out of date as of right now, lots of higher-priority projects filling the queue.
My System
(13 items)
 
   
CPUMotherboardGraphicsRAM
Intel Core2 Quad Q6600 Asus Maximus Formula Zotac Geforce 8800GT 8GB Kingston DDR2800 
Hard DriveOptical DriveOSMonitor
80GB Intel X25M G2 Ben-Q DVD+-RW Windows 7 Ultimate Dual Dell 19" Trinitrons + 1 Hanns-G 19" LCD 
KeyboardPowerCaseMouse
MS Natural Multimedia Rosewill RX750-D-B Antec 900 Razer Diamondback 
Mouse Pad
Razer eXactMat 
CPUCPUMotherboardRAM
Intel Xeon Intel Xeon 440BX Desktop Reference Platform Kingston ValueRam - 72GB 
Hard DriveHard DriveHard DriveOS
Seagate Barracuda ES.2 Hitachi HUS724040ale640 Micron M500DC VMware ESXi 6.0 
Other
Synology ds2015xs 
  hide details  
Reply
My System
(13 items)
 
   
CPUMotherboardGraphicsRAM
Intel Core2 Quad Q6600 Asus Maximus Formula Zotac Geforce 8800GT 8GB Kingston DDR2800 
Hard DriveOptical DriveOSMonitor
80GB Intel X25M G2 Ben-Q DVD+-RW Windows 7 Ultimate Dual Dell 19" Trinitrons + 1 Hanns-G 19" LCD 
KeyboardPowerCaseMouse
MS Natural Multimedia Rosewill RX750-D-B Antec 900 Razer Diamondback 
Mouse Pad
Razer eXactMat 
CPUCPUMotherboardRAM
Intel Xeon Intel Xeon 440BX Desktop Reference Platform Kingston ValueRam - 72GB 
Hard DriveHard DriveHard DriveOS
Seagate Barracuda ES.2 Hitachi HUS724040ale640 Micron M500DC VMware ESXi 6.0 
Other
Synology ds2015xs 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › ZFS storage advice